IBM Interconnect 2016 presentations

If you didn't go to IBM Interconnect 2016 in Las Vegas (and why didn't you? ;-) or even if you went and want to share the goodness, it appears that you can get access to and download some of the presentations here: https://www-950.ibm.com/events/global/interconnect/sessions/

(You might even find a couple from yours truly .  .)

What are you smoking? What is the real meaning of 'Smoke Testing?'

As a mathematician (sorry!) I have a keen interest in defining terms before they are used - it's kind of become a habit.

That's why things like 'I was so shocked I literally died' really grind my gears (see also 'bricked', 'decimated' . .)

One term I hear multiple meanings of often is 'Smoke Testing' - I was discussing this with Dave Hay (highly recommend his blog at http://portal2portal.blogspot.co.uk) and we came up with three that we'd heard on our projects:

'Initial power on/run one transaction' test

In electronics, apply power to the system and turn it on - the 'smoke test' here is that if smoke starts appearing, then something is wrong. 

In software, if you can't even log on, or the disk starts spinning like crazy, or all the RAM is used quickly it's similar.

'Increase load until it breaks to determine the maximum safe load'

This has a car analogy: Yay! Keep increasing the revs/speed until smoke starts coming out of the engine. At that point, back off a little and you have your max safe speed. OK, you should be more scientific but you get the point.

In software, crank up the load-testers until you run out of CPU, RAM, I/O, connections or whatever. Hopefully it won't actually crash, just run low on resources.

It might crash though but not immediately; For example it might queue transactions and eventually the queue will fill.

'Test end-to-end' (rarer)

This one I hadn't heard of until the other day. The idea is to blow smoke into the system from one end and see if it comes out where you expect it to i.e. is the flow correct and are there any leaks? Also, is the smoke that comes out the same type as you put in?

In software, this is analagous to watching for side-effects as well as output. Is the Java heap constantly growing? What about the database connection pool? Memory leak?

Anyone got any more?

“When I use a word,” Humpty Dumpty said, in rather a scornful tone, “it means just what I choose it to mean—neither more nor less.” “The question is,” said Alice, “whether you can make words mean so many different things.”

LEWIS CARROLL (Charles L. Dodgson, a mathematician!), Through the Looking-Glass 

Why I like App Stores . . .and it's nothing to do with walled gardens

App Stores: It's a concept that seems to divide opinion but I quite like them for the following reasons:

One credit-card stored in one place

If I have an app store, I have one means of payment. I can register my credit card (whatever type - I don't need to worry if you take Amex/Visa/MCard/Whatever and what country it's registered in, or what currency) in one place and know who the merchant is.

I don't have lots of different web-sites storing my card details. Or if I don't want you to store them (and I don't) then I don't have to type my details over and over again if I want to buy your software.

I can actually recognise the merchant name - I don't have to worry about 'xyz s/w holdings, Bermuda' and whether that's really the software/upgrade I bought or a scammer.

And neither does my card issuer, so my card doesn't get fraud-blocked when I need to spend 0.99 on an in-app purchase and it flags as suspicious.

And there's no 'minimum card value 10.00' restriction: In fact app-stores bundle-up purchases if you buy a lot of things on the same day.

Alternatively, I can buy vouchers for most app stores with cash and not have to worry about credit cards at all.

One user ID & password and one source of spam

I want to use your software or app. I'm happy to pay you for it - you put in the hard work so you should get my cash and that's all good.

What I don't want is to have to 'register an account' with you and have to remember yet another password.

I also don't want mail from you or 'selected partners'. 

No, really I don't.

If I've bought games (or anything else) I like then I'll sign up to your mailing list for updates and 'social' stuff, I'll follow you on Twitter or Facebook, but if I've just bought a hex editor or a GPS app then just take the money already.

I don't want 'a relationship'. If there's a new version, flag it in the store and I'll pick it up. The store's notify system will sort it all out for you.

One place to get all the updates

This is what prompted this post: One place that says 'nn updates available' so that I can click 'update all' and let it get on with it.

I have phones and laptops I don't turn on all that often. When I do, I just want to click a button marked 'update everything on my computer' and let it get on with it. 

Without any more hassle. 

Preferably in the background. 

Without me having to open every single app and click 'check for updates'.

Without me having to update your 'updater' program before I can check for updates.

And only having to type in my 'admin' password once, not once per app.

And really, really preferably, without having to reboot.

On a positive note:

Having run 6 separate updates, at least only one (an update to MacOS itself) has asked me to restart my computer. On Windows that request used to happen every time I updated anything - was it ever really necessary or did people do it 'just in case'?

Watch out for 'watch' as a variable name in IBM BPM with V8+ coaches

Found an obscure issue the other day at a customer site - writing it up here in case I forget what the details were . .

In summary, if you have a variable called 'watch' in IBM BPM with next gen (V8 and above) coaches, you may see a conflict with dojo stateful https://dojotoolkit.org/reference-guide/1.10/dojo/Stateful.html trying to add a .watch function to your variable. At this point, your coach buttons will stop working - or your coach may not display at all. You will see an error in the browser console of the form 'dojo.watch is not a function'.

Using NG coaches with IBM BPM V8.5.0,1 and above, they were seeing issues in the browser console reporting that '.watch is not a function' and finding that they couldn't press the buttons on the coach.

This appeared to be random with no pattern that they could discern. Some coaches worked, others didn't

The only workaround they could find was to use a heritage (pre V8) coach, where the problem went away.

We did much googling of .watch and dojo.watch etc but couldn't find anything that would explain the 'randomness'

After walking through the process app, we finally spotted that they had a variable called 'watch' in one of the business objects, a couple of levels into the hierarchy.

This variable was sometimes used, sometimes not. In the hierarchy / object graph, that level was sometimes instantiated, sometimes not. When it was in use, the variable reference was object.watch , which conflicted with dojo stateful trying to add the .watch function onto the variable, hence '.watch is not a function'.

Process designer will stop you creating variables with java and javascript reserved words (try creating a variable called 'this' if you want to see it) but it doesn't catch everything. If nothing else, the 'watch' variable was created when the customer was on 7.5.1 and before NG coaches with lots of dojo and the .watch function.

Another one for the 'department of incredibly obscure issues' . ..

Invalid web-site certificates? Check the obvious first: is the time on your VM image correct?

So, I'm downloading the MQ Explorer support pack.

I run a Mac, and this only works on Linux or Windows, so I fire up my Windows 7 VM image to do the necessary.

Onto the web-site, sign in, choose download method and get a 'This site does not have a valid certificate!' error in Chrome. Worrying! There's no obvious  'override this' in Chrome, so I try Firefox.

Firefox gives the 'This connection is Untrusted' page with the usual 'get me out of here/I understand the risks' options.

Now for internal web-servers, I'd normally hit 'I understand' - but this is an external one - and it's ibm.com - it should be OK? But what if I've been Man-In-The-Middled somewhere? OK, it's my DSL-Fibre connection but better safe than sorry . .

Looking at the screen however, I see this:

The certificate is valid from 15/12/2015 (That's 15th December, if you're used to US date formats!)

"The current time is 08/12/2015 13:44" - er, not it's not! Today is the 16th January 2016 (16/01/2016). What's going on?

So I check the time on Windows in the VMWare - sure enough, it thinks it's still December. The Time Server sync hasn't fired yet or has glitched and I haven't used this VMWare for ages: Obviously since 8th December.

So, I resync the time and everything starts working! Yay!

But I never usually check the time on VMs - why would I? I assumed that the time-server sync is keeping me up to date. It usually syncs up OK, but obviously not often enough - or it can have an error with time.windows.com.

So TL;DR: When firing up a VM, check and sync the VM time to the time server, every time. Then check it against your watch - just to be sure!

The Magical Progress Fairy - are you using one on your project?

Some time ago IBM had an advert featuring a 'Universal Business Adapter' (YouTube here) which would magically connect anything to anything.

Not surprisingly, everybody wanted one. However not everybody got the joke that it didn't actually exist - you could do everything it did, but you would actually need to build it using SOA/ESB/Integration tooling. (If you do actually want to build one, start here).

Another thing that everybody seems to want on their project is the Magical Progress Fairy. A lot of people think they already have one . . .

What is a Magical Progress Fairy?

A Magical Progress Fairy (MPF or latin: hopeus overexperiencus) is a mythical creature that you invoke when your project is not going as fast as you want it to, you're behind and you're likely to miss deadlines. It magically improves your productivity, vitalises your team, eliminates any problems and gets you right back on track - without you making any changes to your approach at all!

I don't do that . .

Perhaps you don't - but do any of these things sound familiar?

"We only made 20 story points in our last sprint - to get back on track we need to make 30 in the next sprint"

So how is that going to happen? What are you actually going to do to improve productivity by 50%? A lot of the time we hear phrases like 'Everybody buckle down', 'Let's all really concentrate for this one' or 'we all need to work harder'.

On the other hand, everyone is already working hard and concentrating - and we need to keep in mind the Agile principle that progress should be sustainable indefinitely.

"We had a bad sprint last time - a lot of things came up to disrupt us"

And these won't happen on the next sprint? The internet won't drop out? A team member won't get sick? There won't be an unexpected bug in the tooling / database etc?

"We weren't experienced enough on the technology before - we're better now"

Maybe - but can you prove it? Are you sure you now know everything you need to know? What else is around the corner?

All of these approaches (some might call them excuses - I would never do that) rely on the Magical Progress Fairy to get you out of trouble. Magically progress will improve, mistakes will not be made, people will become more productive and disruptions will not happen . . .

OK, maybe I might need a Magical Progress Fairy - but what else can I do?

There are a number of things you might try but the first is to be realistic about what you've already managed to do and what you will be able to do in the future:

Accept reality and calculate your project velocity

Project velocity is a measure of how productive your team is, in your environment, working on your user stories, using their skills in the technology and method chosen for your project. It's not a single one of any of these, a measure of how 'good' the team is - or an individual. All of the project factors have a part to play.

To calculate your project velocity, simply divide up how many story points you actually managed to get done in the previous sprint by the number that you planned to deliver in that sprint.

Example:

We plan a sprint of 2 weeks (10 days). When we did planning poker we decided that one story point was going to be one 'ideal day'. We have a team of 5 people, so that gives us 50 ideal days, so 50 story points to plan with.

At the end of the sprint we found that we 'only' achieved 32 story points.

So, our velocity is 32 / 50=0.64. Note that it is not '1.0 - except for Bill had a dentist appointment and Sarah's PC needed a new hard drive, we spent time planning the sprint - and we used 2.5 person days for 1 morning of playback to the business'

Of course, you can have a velocity greater than 1.0. 

If you'd finished all the user stories in 9 days - people do get ahead , then  you'd have had a velocity of 50 / 45 or 1.11 (I know it's recurring but hey . .)

Once you know your velocity, use it to plan further sprints realistically

So, if your velocity was 0.64 and you have another 10 day sprint - then plan for 50*.64=32 story points

If it was 1.11 then plan for 50 *1.11=55.5 story points (if your planning poker has a 0.5 card - otherwise it's 55 - no quarts in pint pots).

Don't invoke the Magical Progress Fairy

If you planned 50 story points and you made 32, then that is what you made. No ifs, but buts. That's all you proved your team can make. Why will next time be any different?

Don't extend the sprint

'We'll be finished if we just have another day / 2 days / week / month . . '. One of the advantages of the sprint system is that it shows you 'point in time' progress with nowhere to hide. Either the software works or it doesn't. Software is often 95% finished for 95% of the time you ask the developer 'how it's going'

Finish early? Shut down the sprint, calculate velocity and start the next one

Congratulations - you made a velocity greater that 1 - but that just shows your estimating was on the low side vs reality. Now you know that, go ahead with that knowledge.

My velocity has changed after the next sprint - what do I do?

So, your velocity after sprint 1 was 32. Your velocity after spring 2 was 44. Great! This isn't the magical progress fairy. Something was different - maybe your team did get better, or the dev server didn't crash or there wasn't a big queue at the coffee shop.

So, use your new velocity to plan the next sprint- that's now our up to date measure of reality.

If you have put real interventions in e.g. given people faster computers, given them a copy of production data to test with etc, then this will allow you to measure if this made any difference.

A quick quiz - which of the following might actually work? 

No prizes except maybe getting your project back on track . .

• Work evenings and weekends
• Increase the project duration by adding more sprints
• Adding more people to your project - see The Mythical Man Month
• Cut the scope by removing user stories
• Yelling at the team more - motivation is the key
• Re-prioritising user stories and revisiting what might be acceptable as a minimum viable product
• Re-plan the sprint every day: hey, why not replan every hour?
• Concentrate on the happy path and make some exception paths into a manual activity e.g. 'If paying by credit card, click 'next'. If paying by cash or check/cheque, please visit one of our high street stores'
• Add more RAM, CPU or disk to the development system - maybe - but is this the real issue?
• Cut/eliminate testing (especially NFRs) - let the users do it in the beta
• Try different sprint lengths - overheads like planning and playback can eat into the time. Beware long sprints as this reduces agility and means that progress can be obscured.
• Re-estimate everything over and over until we find an estimate we like that fits the plan.
• Ignore technical debt - we've gotta hit that deadline - hardcode everything!
• Calculate your velocity, use it to plan further sprints, bite the bullet and accept reality.

Finally, Beware the 'hidden hours' of evenings and weekends working - especially when you don't know about it!

Remember we planned for a 10 day, 50 person/day sprint? If 4 people worked the weekend, it's actually now a 58 person/day sprint. This may give you more development output but it's skewing  your velocity!

It means that in the next sprint, you're now assuming that 4 people will work the weekend - and the same going forward.

If you plan for an 8 hour day and 2 of the team actually work 12 hour days all the way through 'to get it done and show dedication' - they're making a mockery of your velocity calculations.

Watch out for people staying late, doing work at home in the evenings of weekends - if they do it, you need to take account of it otherwise they're becoming your Magical Progress Fairy  . .

El Capitan Time Machine on NAS running very slowly?

So I upgraded my Macs to El Capitan and everything was rosy - with the exception of Time Machine.

Time machine backups to local USB drives (Seagate 2TB if it makes any difference . .) seemed to work fine, but network backups to a Synology DS415Play NAS were getting 'stuck' on the 1st few MB - and staying stuck, seemingly forever.

The symptoms were visible in the 'progress' of Time Machine in the status bar:

The 'backing up' would stay in single megabytes.

After checking the network etc, I then tried Time Machine preferences:

Originally I had my NAS time machine connected as the 'auto discovered name' i.e 'AndyNAS'.

I tried the 'add or remove backup disk' option to remove the connection and re-add it. It didn't seem to help - I got the same problem.

I tried connecting directly to the Time Machine drive using Finder and for no reason that I could think of, used the numeric IP address to connect to the NAS rather than the name. This seemed to work fine and the Time Machine share was visible and seemed to be working normally.

Finder->Go->Connect To Server

In a last ditch attempt, as I have my NAS on a static IP on my LAN (192.168.0.100), I tried removing the original Time Machine NAS connection using 'Add or Remove Backup Disk' and then tried connecting using the 'dotted quad' 'numeric' IP address as shown above - and everything started working again!

Not sure exactly what fixed it - could be any of the following:

• Adding and Removing the Backup Disk / NAS connection in Time Machine Preferences
• Doing a direct afp connect to the NAS Time Machine share in Finder
• Using the 'dotted quad' / Numeric IP address rather than the name
• Pure luck  and the 'turn it off then on again' effect - but it did fix 3 out of 3 of the macs I tried it on

To be fair, I'm not that bothered - it's working again, and for all three Macs (work, personal and wife's) backing up via Time Machine to the NAS after the El Capitan upgrade. It's also encrypting fine.

As a side note, I didn't have to re-create my Time Machine backups - as long as I re-entered the encryption password then it 'found' the old backups and carried on as before.

As usual, this worked for me - your mileage may vary but it might be worth a try.

***Additional***

After an interesting dialog box of 'Time machine has checked your backup - it needs to backup from scratch' (effectively) I found that having to back up 150GB or so was taking a long time.

If you check 'console' and search for 'mdworker', you might see a number of entries, especially starting with the word 'deny'. Apparently, this isn't helping your time machine speed!

Believe it or not, the fix for this is to start your Mac in 'safe mode' - then restart again normally.

To start in safe mode, shutdown your Mac. Then start it up. Immediately after the first 'bong', hold down the shift key. When you see the startup screen, you can release the shift key. If you're successful, you'll see 'safe mode' written on the screen somewhere.

(Apple guidance here https://support.apple.com/en-gb/HT201262)

After you've started up in safe mode, give it a minute or two to finish loading everything then shut down again.

Now check your time-machine backup speed - assuming you have a decent network speed (I highly recommend gigabit) you'll see it fly along . . .