Learning Azure Databricks – fix

I’m doing the MS Learn module, Introduction to Azure Databricks, Use Apache Spark Notebooks (https://docs.microsoft.com/en-us/learn/modules/intro-to-azure-databricks/4-using-notebooks) and I hit a problem in the section called ‘Why Apache?’ when the tutorial wouldn’t create the assets, complaing ‘ModuleNotFoundError: No module named ‘pathlib2’.

To fix this – click on Clusters in the left menu and select your cluster to show the configuration page:

Click on the Libraries tab, click Install New, the PyPi and enter pathlib2 into the Package field; click Install and wait for the process to complete. Once that is done, you can re-run the first part of the ’02 Why Apache?’ notebook and it should work.

Adding Chrome extensions to Edgeium

For the last four or five months I have been using the new Microsoft Edge builds based on Chromium – this is via the special release channels that Microsoft set up for people to try it out. I mostly use the Dev channel release which is updated once a week and is pretty stable; there are two other channels – Beta is on a slower release cycle and represents the most polished current version, or, for the really adventerous, the Canary channel is on a daily release cycle and contains really ‘fresh’ code.

As with most people, I’ve been using Chrome for the previous few years; one of the reasons I really liked it was the tabbed page support it had; I first realised how useful tabs were when I used Opera and soon found them indispensable- so much so that I often ended up with dozens of tabs open at the same time.

It wasn’t long before I was searching for a way of managing my predilection for all the tabs all the time! The most useful one I found is The Great Suspender – which ‘unloads’ the tab content after it’s been un-used for a configurable period of time, it releases the memory and eliminates any CPU useage, but the page can be reloaded with a single click.

Of course, now I’m hooked on Edge, the Great Suspender extension was the only thing I missed from Chrome. So I was really pleased to see Edge now has a feature that allows you to load extensions from the Chrome App Store. To find this feature, use the Edge menu and select Extensions; then at the bottom flip the Allow extensions from other stores toggle and accept the disclaimer.

Now you can navigate to the Chrome app store (https://chrome.google.com/webstore/category/extensions) and install any of the extensions you want from there.

A bit about Dot Net North

As you may know, I am one of the co-organisers of a Microsoft oriented community ‘user’ group in Manchester called DotNetNorth. We run a monthly meeting in Manchester city centre with one or two speakers plus pizza (that’s Manchester in the United Kingdom).

When the group started it was run by myself, Pete Vickers and Chris Hardy and was aimed at building a community of Windows Phone developers; we didn’t see many people at the meetings but we built a small core of regular attendees many of whom have become friends.

In 2014 we decided to pivot into a less specialised group and we’ve grown since then to have almost 1000 fans on our meetup group.

Chris has since gone off to work for Xamarin in the USA so Pete and I now have the help of Oli Newsham, Jason Holloway and Mike Irving to keep the group running. Having a team certainly makes the events easier to host and run – we’ve all got different strengths and we use that to our advantage each meeting and it means people can find an ‘organiser’ when they need one.

This last year has been a pretty incredible one for us – we secured a year long sponsorship with Evolution Recruitment which allowed us to stop worrying about how to pay for the pizza each month and they also promoted the meetup site by adding a link to the email signature of all their consultants working with Microsoft related technologies. The company has been commendably ‘hands-off’ about the arrangement with one of their guys, Scott Barker, coming along each month (sometimes with a colleague too) just to chat to anyone who is looking to move jobs.

The group continues to grow at pace and we will soon be looking for a bigger venue to host the meetings and more sponsors to cover the costs.

If you’re interested in coming along or speaking at one of our meetings then you can see the event diary and/or send us a message at http://www.meetup.com/DotNetNorth

Beginning a journey to “data scientist”

For a long time I’ve been interested in data. My first development job was building a production tracking and reporting application in Microsoft Access (version 2.0 if you’re asking) and I’ve been in love with the idea and practice of storing data ever since. Many of my subsequent jobs were building applications to collect and show data, after my experience with Access (which was then, and still is, an incredible piece of software) I worked with VB4 through to VB6 and the transition to Windows Forms and C# and eventually to HTML and the web.  But all that time, it was the *data* that interested and excited me. I did a few assignments building business intelligence solutions using the SQL Server BI stack and that was really good fun and challenging but I always seemed to end up back as a C# coder.

Having finished a recent contract building web apps in C# and Angular and all that webby stuff I found myself thinking that I’d change my focus and go back to the data. I had heard about the Microsoft Professional Program for Data Science, so I’ve enrolled in that with the aim of transitioning away from being a pure code monkey to something more deeply attached to the data.

I’ve finished a couple of the modules already, but I plan to blog a bit about my experience in making this change. At the end of the journey I hope to be able to call myself a Data Scientist, and more importantly get paid to be one.


Conversation id’s are not carried between service broker instances

Digging around today I have found something that, at first seemed odd, but probably isn’t. When a message gets transmitted between service broker instances, it has an associated conversation ID. One might expect that the ID would travel with the message(s) to the destination instance. After all, it’s the conversation ID that ties together all the messages that need to be treated as one dialogue.

At the destination, however, the conversation ID that the messages have is different to the one they are associated with at the sending end.

This seems odd.

All the messages in a conversation at the receiving end have the same conversation ID, it’s just a different GUID to the one that they have in the source database.

Apply a bit of brain juice and I surmise that what is going on here is either (or both) of (a) the system just not trusting the sender GUID, so causing a new one to be used, (b) an incrementing GUID is being used for the conversation ID, so that the destination has to assign a fresh value so as to maintain the sequence order of the ID.

I need to do a bit of testing to see if (b) is in fact what’s going on here.


P.S. I shall be presenting a talk on Service Broker of the SQL Server User Group in Manchester in June.