Author: giedrius

The State of Internet censorship in Lithuania (Beginning of 2018 Edition)

Unfortunately but the specter of Internet censorship has finally came to Lithuania a few years ago. It was rampant already in some European countries but I never thought that it would come to Lithuania as well. I thought that this country was different… nope, it started to slowly transform and become like others in terms of Internet censorship as well.

History

As far as I can tell everything was fine up until around 2008, 2009. It all began when on 2010 July 8th a court in Lithuania, Vilnius satisfied a lawsuit by which foreign gambling companies such as “bet365 Ltd.” and “Unibet International Ltd.” were blocked from doing their business in Lithuania. Accordingly, their websites must be blocked, the court said. The decision was appealed and the court case dragged further into the future.

Around 2011, LANVA (the Lithuanian anti-pirate association) sent a request to the IVPK (the committee of expansion of the information society) to block Linkomanija (as far as I know, it’s the biggest private torrent tracker in Lithuania). One of the responsibilities of that committee is to propose new laws to the parliament which are related to the Internet, author rights, and so on.  After a meeting, they decided not to go further with the request from LANVA however they said that they were available for more discussions about this topic in the future.

In 2012, global events affected Lithuania as well. During that year, the whole notorious ACTA crisis unfolded. Because people all over the world generated a lot of discussion around this topic, even the Lithuanian news media gave attention to it. What is more, local protests were organized where people were rallying for Lithuania to not sign the controversial act. As far as I know, it was never signed. After that, everyone quickly forgot about it. Thus, there were no changes with regards to Internet censorship in Lithuania.

In 2013, amendments to the law with regards to gambling were being considered. Numerous events were organised around discussions if Lithuania is going towards the censorship of the Internet. However, no decision was yet made because of a certain case that had been on-going in the European Court of Human Rights. It all culminated the following year.

In 2014, the European Court of Human Rights decided that EU countries can indeed block access to webpages which violated the rights of authors (the term “rights” has a very vague definition, obviously). That means that from that moment various institutions can go to courts to get certain websites blocked in their own country in the European Union. After this, the court decision to block various gambling companies went ahead in terms of blocking their pages online as well.

After some time, during 2015 some changes to the law were finally approved to implement a framework with which websites could be blocked. At that time it was clearly legal to do that in the European Union, and on 2016 January 11, betway.com became the first officially censored webpage in the Republic of Lithuania.

Implementation

The current censorship is implemented by modifying DNS records at the ISP level. The blocked websites’ A records are modified so that they would redirect to https://blokuojama.lpt.lt/. In there, the user is informed that the website that they tried to visit is “blocked” and it links to the court cases due to which those websites were “blocked”.

We could see how blocking works by using drill to send a few requests to DNS servers. First one up is my ISP’s one. Let’s see what response it gives:

giedrius@tyrael:~/ > drill betway.com @192.168.1.254
;; ->>HEADER<<- opcode: QUERY, rcode: NOERROR, id: 6156
;; flags: qr rd ra ; QUERY: 1, ANSWER: 1, AUTHORITY: 0, ADDITIONAL: 0 
;; QUESTION SECTION:
;; betway.com.  IN      A

;; ANSWER SECTION:
betway.com.     10760   IN      A       194.135.95.243

;; AUTHORITY SECTION:

;; ADDITIONAL SECTION:

;; Query time: 2 msec
;; SERVER: 192.168.1.254
;; WHEN: Fri Mar 23 20:28:37 2018
;; MSG SIZE  rcvd: 44

As you can see, betway.com (the first website that was ever blocked in Lithuania) is apparently at 192.135.95.243.

To see if it is really there, let’s check the A records of the same domain at 8.8.8.8, Google’s relatively popular DNS server:

giedrius@tyrael:~/ > drill betway.com @8.8.8.8      
;; ->>HEADER<<- opcode: QUERY, rcode: NOERROR, id: 53085
;; flags: qr rd ra ; QUERY: 1, ANSWER: 2, AUTHORITY: 0, ADDITIONAL: 0 
;; QUESTION SECTION:
;; betway.com.  IN      A

;; ANSWER SECTION:
betway.com.     59      IN      A       45.60.87.104
betway.com.     59      IN      A       45.60.114.104

;; AUTHORITY SECTION:

;; ADDITIONAL SECTION:

;; Query time: 90 msec
;; SERVER: 8.8.8.8
;; WHEN: Fri Mar 23 20:30:51 2018
;; MSG SIZE  rcvd: 60

As you can see, the real results and the censored results differ. Indeed, if you would go to 192.135.95.243, it would redirect you to blokuojama.lpt.lt which is the title page for all censored websites.

Some people were talking that this is not enough and now they are pushing new laws which would make ISPs implement deep packet inspection. The ISPs themselves, on the other hand, argue about the effectiveness of that and who is going to give them money to buy expensive equipment which would let them to perform these functions. Plus, obviously, they are presenting many more valid arguments against Internet censorship. For example, many of them argue that they just provide an utility to customers and they don’t want to become the second police who would decide what content is “good” and what content is “bad”. This point of view of ISPs is briefly presented here at the end of the article.

Statistics

All of the data – relevant cases and URLs – about blocked websites is presented here. If we were to put all of that into a diagram (different domains count as different sites) we would get this:

As you can see, the number of blocked sites has been increasing a bit faster than linearly. If the current trend continues, around 500 sites will be blocked by the beginning of 2020. Hopefully, this will not be true. I guess that I will have to revisit this assertion again in two years.

How to circumvent this?

In general, you will have to change your settings so that your computer would use a different DNS server. There are plenty of options to choose from. The popular choices are 8.8.8.8 and 8.8.4.4 (Google DNS servers) however if you have any privacy concers with them then you can choose some other ones from, for example, here or here. Obviously, you should not choose a DNS server that is censored as well. So choose wisely.

Once you have done that, continue to the next two sub-sections depending on what you prefer. They do not cover all of the routers and operating systems but they give you a general gist of where to look for more information and give an example of how it looks like.

Router level

In general I would recommend you to search for something like “router model + admin panel” or “router model + DNS settings” and you should most certainly find something. I will present how to change the DNS settings on a router that is provided by the most popular ISP in Lithuania, Telia.

  1. Go to the admin panel which is usually at the same IP as the gateway. In my case, it is http://192.168.1.254. The default username and password is admin/admin. Enter your own username and password if you changed them. You will get into this:
  2. Go to Settings > Network Connections > WAN DSL > IPv4 (tab). There you can find a checkbox “Setup Static DNS Servers”. Click on “Yes”.  Then you should be able to input your preferred and an alternative DNS server. You should see something like this:
  3. After setting your DNS servers, click on “Apply”. You might have to restart your router to really apply the settings. Enjoy!

Operating system level

It seems that Windows 10 is still the most popular operating system on desktop computers so I will provide a way to change your DNS settings on it.

  1. First go to Control Panel > Network and Sharing Center:
  2. Click on “Change adapter settings” and then click on “Properties” on the connection that you use to connect to the Internet:
  3. Go down to “Internet Protocol Version 4 (TCP/IPv4)” and click on “Properties”.  Inside there you can change the DNS settings. Choose “Use the following DNS server addresses” and enter the IP adresses of your choice:
  4. Once you are done click on “OK”. Et voilà!

 

Why os.move() Sometimes Does Not Work And Why shutil.move() Is The Savior?

Recently I ran into an issue where this, for example, code fails:

import os

os.rename('/foo/a.txt', '/bar/b.txt')

Traceback (most recent call last):
File "", line 1, in 
OSError: [Errno 18] Invalid cross-device link

The documentation for the os module says that sometimes it might fail when the source and the destination are on different file-systems:

os.rename(src, dst, *, src_dir_fd=None, dst_dir_fd=None)

Rename the file or directory src to dst. If dst is a directory, OSError will be raised. On Unix, if dst exists and is a file, it will be replaced silently if the user has permission. The operation may fail on some Unix flavors if src and dst are on different filesystems. If successful, the renaming will be an atomic operation (this is a POSIX requirement). On Windows, if dst already exists, OSError will be raised even if it is a file.

How could it fail? Renaming (moving) a file seems like such a rudimentary operation. Let’s try to investigate and find out the exact reasons…

Reason why it might fail

The fact that the move function is inside the os module implies that it uses the facilities provided by the operating system. As the Python documentation puts it:

This module provides a portable way of using operating system dependent functionality.

In this case, it (probably, depends on the implementation, obviously) uses the rename function in the C language because that what moving is and that in turn calls the rename system call. The system call is even kind of defined in the POSIX collection of standards. It is an extension of the function rename in the C standard but it sort of implies that an official system call exists as well. As the official text of POSIX says, rename can fail if:

[EXDEV] The links named by new and old are on different file systems and the implementation does not support links between file systems.

Even Linux does not support this so this error message is not that uncommon. As the rename(2) manual page says:

EXDEV oldpath and newpath are not on the same mounted filesystem. (Linux permits a filesystem to be mounted at multiple points, but rename() does not work across different mount points, even if the same filesystem is mounted on both.)

The curious case of Linux

The Linux kernel has, as we guessed, an official system call for renaming files. Actually, it even has a family of system calls related to this operation: renameat2, renameat, and rename. Internally, they all call the function sys_renameat2 with different actual parameters. And inside of it the code checks if src and dst are at the same mounted file-system. If not, –EXDEV is returned which is then propagated to the user-space program:

/* ... */
        error = -EXDEV;                                                                                                           
        if (old_path.mnt != new_path.mnt)                                                                                         
                goto exit2;  
/* ... */

Then the error is returned:

/* ... */
exit2:                                                                                                                            
        if (retry_estale(error, lookup_flags))                                                                                    
                should_retry = true;                                                                                              
        path_put(&new_path);                                                                                                      
        putname(to);                                                                                                              
exit1:                                                                                                                            
        path_put(&old_path);                                                                                                      
        putname(from);                                                                                                            
        if (should_retry) {                                                                                                       
                should_retry = false;                                                                                             
                lookup_flags |= LOOKUP_REVAL;                                                                                     
                goto retry;                                                                                                       
        }                                                                                                                         
exit:                                                                                                                             
        return error;                                                                                                             
}

This explicit check has been for forever in this function. Why is it there, though? I guess Linux just took an simpler (and more elegant, should I say) path here and added this constraint from the beginning just so that the code for the file-systems would not have to accompany for this case. The code of different file-systems is complex as it is right now. You could find the whole source code of this function in fs/namei.c.

Indeed, old_dir->i_op->rename() is later called in sys_renameat2. old_dir is of type struct inode *, i_op is a pointer to a const struct inode_operations. That structure defines a bunch of pointers to functions that perform various operations with inodes. Then different file-systems define their own variable of type struct inode_operations and pass it to the kernel. It seems to me that it would be indeed a lot of work to make each file-systems rename() inode operation work with every other file-system. Plus, how would any file-system ensure future compatibility with other file-systems that the user could use by loading some custom made kernel module?

Fortunately, we could implement renaming files by other means, not just by directly calling the rename system call. This is where shutil.move() comes in…

Difference between os.move() and shutil.move()

shutil.move() side-steps this issue by being a bit more high-level. Before moving it checks whether src and dst reside on the same file-system. The different thing is here what it does in the case where they do not. os.move() would blindly fall on its face here but shutil.move() is “smarter” and it does not just call the system call with the expectation that it will succeed. Instead, shutil.move() copies the content of the src file and writes it to the dst file. Afterwards, the src file is removed. As the Python documentation puts it:

If the destination is on the current filesystem, then os.rename() is used. Otherwise, src is copied (using shutil.copy2()) to dst and then removed.

So not only it copies the content of the file but shutil.copy2() ensures that the meta-data is copied as well.This presents an issue because the operation might get interrupted between the actual copying of the content and before src is removed so potentially you might end up with two copies of the same file. Thus, shutil.move() is the preferred solution to the original problem presented at the start of the article but however be wary of the possible problems and make sure that your code handles that case if it might pose a problem to your program.

Related Posts