Category: Programming

Are Scrum, Agile, and other iterative programming methodologies useful for niche programming specialties such as SREs, DevOps engineers, and so on?

Intro

Recently I heard that someone asked in an “IT Systems Engineer” interview a question: “Why are you using Agile in your team? Are not most of the tasks on your team ad-hoc?”. This made me think about this topic deeply. The work being done in these types of teams might seem distant from regular programming at the beginning. However, it is not because in general it is focused on automating stuff using software, avoiding manual labor, building reliable systems or, in other words, the product and your clients are just different – they are internal whereas usually, they are external. Let me try to explain.

Difference between sysadmins and SREs

Let’s first note the difference between those two – this is important because a lot of people have a wrong perception that everyone other than developers does not create any kind of software. Alas, the SRE/DevOps movement is mainly all about eliminating toil. For example, Google does not allow their SREs to spend more than 50% of their time doing manual stuff. That means that it should be done automatically by some kind of automation. On the other hand, the sysadmin model is about dividing the IT & development into two separate silos – the developers and the system administrators. Everyone is doing their own stuff and barely collaborating. Also, they are usually not creating much software of their own – mostly just small scripts with Perl or Bash. As you can see there is a stark difference in the mindsets and we are talking about the former one.

Iterative programming methodologies for SREs

Just like in the “normal” programming world, requirements change and it is an inevitable part of the whole gig, I think, unless you are making software for things where reliability is of utmost importance – for example, you would not want a plane to crash because of an overflow error (that still happens sometimes like in this case of Boeing Dreamliners).

The benefits and downsides of each programming methodology do not differ at all in any case. Let me propose an example. For instance, after a month your DNS service which you maintain might get a new requirement – the self-service should get a new batch creation feature. It would let create many records with only pressing a few buttons, saving the precious users’ time. This might not be apparent at the beginning.

This is where iterative programming techniques such as agile are useful because they solve this problem of uncertainty – they embrace it, and they let you pivot in the middle (between sprints) of your development process. With sequential development methodologies, you would have to wait until the end a whole cycle to implement any kind of new requirements.

Practically employing iterative programming in the day to day life of a site reliability engineer

For special, toil type of tasks, you could create one huge task in your time tracking software. For example, create a task in Jira, under which all of the other tasks would be created as sub-tasks where all of the nitty-gritty details would be written down and the time tracked.

Afterward, you could sum up the time spent on toil type of tasks versus the other ones. Then you could tell if you yourself or the engineers spend more or less than 50% of their time on this type of work which could subsequently become a signal that something is wrong.

Related Posts

Ancient specification “bug”: you cannot use colons in your username with the HTTP basic authentication method

Out there, in the wild exists a lot of different authentication schemes or methods however one of them which is relatively popular because it’s implemented in most popular browsers at the moment, has this one peculiar “bug” in its specification – you cannot use a colon (‘:’) in the username field. If you have ever seen a window such as this:

HTTP Basic Auth window on Chrome

Then that website is probably using HTTP basic auth and you must not use a colon in your username on that site because simply you would not be able to do that.

Why, you might ask? Well, simply because in that authentication scheme a colon is used to separate the username from the password. If you used a colon in your username, the HTTP server would not be able to discern between the username and the password because it is transmitted to it in this format with this scheme: username:password.

This scheme is defined in RFC 7617 and RFC 2617. As it says in the RFCs themselves:

Furthermore, a user-id containing a colon character is invalid, as the first colon in a user-pass string separates user-id and password from one another; text after the first colon is part of the password. User-ids containing colons cannot be encoded in user-pass strings.

That is an excerpt from RFC 7617. This is from RFC 2617:

To receive authorization, the client sends the userid and password, separated by a single colon (":") character, within a base64 encoded string in the credentials.

As you can see for yourself, the older version of this RFC (2617 is from June, 1999 whereas 7617 is from September, 2015) does not explicitly state that it is impossible to use a colon with this scheme however it is implicitly stated.

You might be surprised but a lot of software gets this wrong. For example, I recently looked into using ml2grow/GAEPyPI for running a simple PyPI on Google App Engine to reduce the costs. The code is all dandy and nice however the username and password parsing is a bit broken. It all happens here:

(username, password) = base64.b64decode(auth_header.split(' ')[1]).split(':')

As you can see, this code breaks a bit when .split(':') returns more than two results – when the username or password field contains a colon itself. This could be mitigated by using the first result as the username, and by concatenating all following results into a single string which would be used as the password.  I will open a pull request soon to fix this issue. There are probably many more examples such as this.

As far as I know, we can only postulate about why this decision was made. My first thought was that maybe because Internet and computers were not so fast back in June, 1999, the people who made RFC 2617 decided to put it all into one field. This could have been easily remediated by having two separate fields for username and password. Perhaps this would have been too costly? I do not know.

Do you know of any other historical “bugs” in widely used specifications nowadays? Also, maybe you know what might be the reasons why this RFC was made in this way? Please let everyone know in the comments section down below. Thanks for reading and happy hacking!

Related Posts