Category: Programming

10 nuggets of knowledge that I picked up from “Code Complete”

My notes and my Code Complete book

You could probably find “Code Complete” by Steve McConnell in any of the top 10 lists of books recommended to read for all programmers. I recently finished reading that book too. I have to confess that at times the tips and knowledge in there was a bit too basic and fundamental but, on the other hand, I also think that it has a lot of golden nuggets of knowledge.

While reading it, I was writing down notes after every section. Essentially, they were three/four ideas that I thought were the most important in each section. Sometimes, they were paraphrased from the conclusions of each section written by the author himself. In other cases, I summarised them myself, in my own words.

Note that some of these things I knew already, obviously. However, I think that they are still very important to know to all programmers, independent of their skill level. Let me know what you think of it and if you picked up any other things from that book! Here is my assorted list of 10 ideas that I got from this book:

  1. Defects in the requirements/architecture stage are very costly money-wise. In general, the later the errors were detected, the more you will have to pay in terms of money or time. For example, if you noticed issues in those two things in, for example, the implementation stage then you would have to go back a lot: not only you would have to re-do your requirements/architecture but you would have to design your code again and essentially almost start from scratch. Also, that’s why code reusability is paramount. (Chapter 3)
  2. Software development is a heuristic process – there is no methodology that works for all cases. That is why prototyping is important – do not be afraid to write small pieces of code for testing something out. (Chapter 5)
  3. The imperative of software design is to reduce complexity. It should be rethought or thrown away if it does the opposite. Keep accidental complexity to the minimum. Essential complexity is inevitable. Learn how to know which is which. Try at least a few designs before settling onto the final one. (Chapter 5)
  4. Make sure that related statements are in groups, close together. Relatively independent groups of statements should be moved into their own functions. Also, code should be written to be read from top to bottom. That’s why early exits are important, IMHO. (Chapter 14)
  5. Consider jump tables. They offer a good opportunity to reduce complex code with a lot of conditional statements. They can be index-based or staircase-based (when not exact values are used but ranges instead, you have to duplicate values). Think if you need to put index calculation into a separate function instead of duplicating code. (Chapter 18)
  6. Testing by itself is not very effective. Consider combining multiple quality assurance techniques according to your organisation goals to achieve maximum effectiveness. Make quality objectives clear because people will optimise for them. You should also formalise this process to make it even more clearer. (Chapter 20)
  7. Use binary search with hypotheses to narrow down the search space of where the error might be. Understand the root of the problem before trying to fix it because you might introduce more defects while fixing it. Set the compiler to the pickiest level possible. It will save more time in the long run. Don’t ignore the warnings. The compiler is your friend. (Chapter 23)
  8. Do not stop with the first code-tuning technique, there almost always exists a better one. Move expensive operations out of loops. Always benchmark your changes because results vary wildly depending on a plethora of variables. Apply optimisation with care: readability and maintainability are still paramount. As Sir Tony Hoare said: “premature optimisation is the root of all evil”. (Chapter 26)
  9. Consider rewriting a routine/function if its decision count is more than 10. Make boolean checks positive. Do not use double negatives. Write boolean checks according to the number line so that they would flow nicely from left to right. (Chapter 19)
  10. Always think if arrays are really suitable for you. Research shows that programs with container data structures – queues, stacks, et cetera had fewer bugs. Use enumerated types instead of constants because they enforce more type-checking and thus it makes the program more correct. Abstract data types should be oriented as much as possible to their functional purpose. Don’t create ADTs just to store arbitrary data. (Chapter 12)

Related Posts

Why os.move() Sometimes Does Not Work And Why shutil.move() Is The Savior?

Recently I ran into an issue where this, for example, code fails:

import os

os.rename('/foo/a.txt', '/bar/b.txt')

Traceback (most recent call last):
File "", line 1, in 
OSError: [Errno 18] Invalid cross-device link

The documentation for the os module says that sometimes it might fail when the source and the destination are on different file-systems:

os.rename(src, dst, *, src_dir_fd=None, dst_dir_fd=None)

Rename the file or directory src to dst. If dst is a directory, OSError will be raised. On Unix, if dst exists and is a file, it will be replaced silently if the user has permission. The operation may fail on some Unix flavors if src and dst are on different filesystems. If successful, the renaming will be an atomic operation (this is a POSIX requirement). On Windows, if dst already exists, OSError will be raised even if it is a file.

How could it fail? Renaming (moving) a file seems like such a rudimentary operation. Let’s try to investigate and find out the exact reasons…

Reason why it might fail

The fact that the move function is inside the os module implies that it uses the facilities provided by the operating system. As the Python documentation puts it:

This module provides a portable way of using operating system dependent functionality.

In this case, it (probably, depends on the implementation, obviously) uses the rename function in the C language because that what moving is and that in turn calls the rename system call. The system call is even kind of defined in the POSIX collection of standards. It is an extension of the function rename in the C standard but it sort of implies that an official system call exists as well. As the official text of POSIX says, rename can fail if:

[EXDEV] The links named by new and old are on different file systems and the implementation does not support links between file systems.

Even Linux does not support this so this error message is not that uncommon. As the rename(2) manual page says:

EXDEV oldpath and newpath are not on the same mounted filesystem. (Linux permits a filesystem to be mounted at multiple points, but rename() does not work across different mount points, even if the same filesystem is mounted on both.)

The curious case of Linux

The Linux kernel has, as we guessed, an official system call for renaming files. Actually, it even has a family of system calls related to this operation: renameat2, renameat, and rename. Internally, they all call the function sys_renameat2 with different actual parameters. And inside of it the code checks if src and dst are at the same mounted file-system. If not, –EXDEV is returned which is then propagated to the user-space program:

/* ... */
        error = -EXDEV;                                                                                                           
        if (old_path.mnt != new_path.mnt)                                                                                         
                goto exit2;  
/* ... */

Then the error is returned:

/* ... */
exit2:                                                                                                                            
        if (retry_estale(error, lookup_flags))                                                                                    
                should_retry = true;                                                                                              
        path_put(&new_path);                                                                                                      
        putname(to);                                                                                                              
exit1:                                                                                                                            
        path_put(&old_path);                                                                                                      
        putname(from);                                                                                                            
        if (should_retry) {                                                                                                       
                should_retry = false;                                                                                             
                lookup_flags |= LOOKUP_REVAL;                                                                                     
                goto retry;                                                                                                       
        }                                                                                                                         
exit:                                                                                                                             
        return error;                                                                                                             
}

This explicit check has been for forever in this function. Why is it there, though? I guess Linux just took an simpler (and more elegant, should I say) path here and added this constraint from the beginning just so that the code for the file-systems would not have to accompany for this case. The code of different file-systems is complex as it is right now. You could find the whole source code of this function in fs/namei.c.

Indeed, old_dir->i_op->rename() is later called in sys_renameat2. old_dir is of type struct inode *, i_op is a pointer to a const struct inode_operations. That structure defines a bunch of pointers to functions that perform various operations with inodes. Then different file-systems define their own variable of type struct inode_operations and pass it to the kernel. It seems to me that it would be indeed a lot of work to make each file-systems rename() inode operation work with every other file-system. Plus, how would any file-system ensure future compatibility with other file-systems that the user could use by loading some custom made kernel module?

Fortunately, we could implement renaming files by other means, not just by directly calling the rename system call. This is where shutil.move() comes in…

Difference between os.move() and shutil.move()

shutil.move() side-steps this issue by being a bit more high-level. Before moving it checks whether src and dst reside on the same file-system. The different thing is here what it does in the case where they do not. os.move() would blindly fall on its face here but shutil.move() is “smarter” and it does not just call the system call with the expectation that it will succeed. Instead, shutil.move() copies the content of the src file and writes it to the dst file. Afterwards, the src file is removed. As the Python documentation puts it:

If the destination is on the current filesystem, then os.rename() is used. Otherwise, src is copied (using shutil.copy2()) to dst and then removed.

So not only it copies the content of the file but shutil.copy2() ensures that the meta-data is copied as well.This presents an issue because the operation might get interrupted between the actual copying of the content and before src is removed so potentially you might end up with two copies of the same file. Thus, shutil.move() is the preferred solution to the original problem presented at the start of the article but however be wary of the possible problems and make sure that your code handles that case if it might pose a problem to your program.

Related Posts