JUnit 5

If you want to use JUnit 5, the latest release of JUnit, from Eclipse, then you have to change your import from import org.junit.Test; to import org.junit.jupiter.api.Test;. Notice the jupiter.api there in the middle.

Otherwise, the test runner will complain and tell you that you don't have tests in your project.

The not-so-minimal test that I created was the following:

package test;

import static org.hamcrest.CoreMatchers.*;
import static org.junit.Assert.*;

import org.junit.jupiter.api.Test;

class FirstTest {
    @Test
    void test() {
        assertThat(1+1, is(equalTo(2)));
    }
}
Some activities of the day

Yesterday, I printed the first draft of the first chapter when my little boy was here and he was impressed with this strange object called a "printer". Before I printed what I needed, I fired up LibreOffice and chose the biggest font size that was available and let him type his first name by himself. He was quicker than I thought with a keyboard. After seeing me print his first name, he was jumping up and down with joy of having created something and even showed grandma and grandpa what he had done.

He, then, wanted more and I taught him how to use that backspace key, what it meant and he wanted to type his full name. I let him and taught him that there is a key called space that he should type every time he wants to start a new word and, in the end, he typed his first two names. To my surprise, he memorized the icon with the printer (which I must say that I have to hunt every time, since it seems so similar to the adjacent ones!) and pressed this new key called "Enter". When he pressed, he wasn't expecting the printer on his right to start making noises and printing his name.

He was so excited and it was so nice to see his reaction full of joy to get a job done!

I am thinking of getting a spare computer, building it with him and for him, so that he can call it his computer every time he comes to see daddy. As a serendipitous situation, Packt Publishing offered yesterday their title "Python Projects for Kids". Unfortunately, he does not yet know how to read, but I guess that the right age is coming soon, which is a good thing to make him be educated "the right way" (that is, with the best support, teaching and patience that I can give him).

Anyway, I printed the first draft of the first chapter and today I have to turn it in.

As I write this, I am downloading a virtual machine from Microsoft to try to install Java on it. Let me see if it works. I have none of the virtualization options used, tough the closest seems to be virtualbox.

Let me cross my fingers.

In other news, I updated some of the tags of very old posts of this blog, and I am seriously thinking about switching from ikiwiki to another blog platform. It is slow, very slow on my system with the repositories that I have, especially on my armel system. Some non-interpreted system would be best, but I don't know if such a thing even exists. But the killer problem is that it doesn't support easily the typing of Mathematics (even though a 3rd party plugin for MathJax exists).

On the other hand, I just received an answer on twitter from @telegram and it was nice:

Hello, Telegram supports bold and italic. You can type **bold** and __italic__. On mobile, you can also highlight text for this as well.

It is nice that this works with telegram-desktop too.

Besides that, I filed some bugs on Debian's BTS, responded to some issues on my projects on GitHub (I'm slowly getting back on maintaining things) and file wishlist bugs on some other projects.

Oh, and I grabbed a copy of "Wonder woman" ("Mulher Maravilha") and "Despicable Me 3" ("Meu Malvado Favorito 3") dubbed in Brazilian Portuguese for my son. I have to convert the audio from AAC-LC in 6 channels to AC3 or to stereo. Otherwise, my TVs have problem with the videos (one refuses to play the entire file and another plays the audio with sounds of hiccups).

Edit: After converting the VirtualBox image taken from Microsoft, I could easily use qemu/kvm to create screenshots of the installation of Java. The command that I used (for future reference) is:

qemu-system-x86_64 -enable-kvm -m 4096 -smp 2 -net nic,model=e1000 -net user -soundhw ac97 -drive index=0,media=disk,cache=unsafe,file=win7.qcow2

Edit: Fixed some typos.

Comparison of JDK installation of various Linux distributions

Today I spent some time in the morning seeing how one would install the JDK on Linux distributions. This is to create a little comparative tutorial to teach introductory Java.

Installing the JDK is, thanks to the OpenJDK developers in Debian and Ubuntu (Matthias Klose and helpers), a very easy task. You simply type something like:

apt-get install openjdk-8-jdk

Since for a student it is better to have everything for experiments, I install the full version, not only the -headless version. Given my familiarity with Debian/Ubuntu, I didn't have to think about the way of installing it, of course.

But as this is a tutorial meant to be as general as I can, I tried also to include instructions on how to install Java on other distributions. The first two that came to my mind were openSUSE and Fedora.

Both use the RPM package format for their "native" packages (in the same sense that Debian uses DEB packages for "native" packages). But they use different higher-level tools to install such packages: Fedora uses a tool called dnf, while openSUSE uses zypper.

To try these distributions, I got their netinstall ISOs and used qemu/kvm to install on a virtual machine. I used the following to install/run the virtual machines (the example below, is, of course, for openSUSE):

qemu-system-x86_64 -enable-kvm -m 4096 -smp 2 -net nic,model=e1000 -net user -drive index=0,media=disk,cache=unsafe,file=suse.qcow2 -cdrom openSUSE-Leap-42.3-NET-x86_64.iso

The names of the packages also change from one distribution to another. On Fedora, I had to use:

dnf install java-1.8.0-openjdk-devel

On openSUSE, I had to use:

zypper install java-1_8_0-openjdk-devel

Note that one distribution uses dots in the names of the packages while the other uses underscores.

One interesting thing that I noticed with dnf was that, when I used it, it automatically refreshed the package lists from the network, something which I forgot, and it was a pleasant surprise. I don't know about zypper, but I guess that it probably had fresh indices when the installation finished.

Both installations were effortless after I knew the names of the packages to install.

Oh, BTW, in my 5 minute exploration with these distributions, I noticed that if you don't want the JDK, but only the JRE, then you omit the -devel suffix. It makes sense when you think about it, for consistency with other packages, but Debian's conventions also make sense (JRE with -jre suffix, JDK with -jdk suffix).

I failed miserably to use Fedora's prebaked, vanilla cloud image, as I couldn't login on this image and I decided to just install the whole OS on a fresh virtual machine.

I don't have instructions on how to install on Gentoo nor on Arch, though.

I now see how hard it is to cover instructions/provide software for as many distributions as you wish, given the multitude of package managers, conventions etc.

Android recording audio quality

Roughly sorted by quality:

amr_nb in 3gpp < amr_wb in 3gpp < aac_lc in m4a < aac+ in m4a (only in Android 4.0+) < opus in webm (only in android 5+) < pcm in wav (only in android 4.1+)

Taking into consideration the compatibility of the codecs (mainly with Android and with the popular WhatsApp and Telegram messengers), the best options seem to be:

aac_lc in m4a < aac+ in m4a (API >= 16) < vorbis/opus in webm (API >= 21) < pcm in wav (only in android 4.1+)

For a device running API 14, we can use something like this (MediaRecorder.OutputFormat constants):

  • For very low quality/very low bitrate/very long recording times, use AMR-NB in 3GPP container. Parameters summary: 8kHz and bitrate 12.8kbps Works pretty much anywhere.

  • For low quality/low bitrate/long recording times, use AAC in MP4 container, with m4a extension. Works pretty much anywhere. Example code adapted from: https://developer.android.com/guide/topics/media/mediarecorder.html

    private void startRecording() {
       mRecorder = new MediaRecorder();
       mRecorder.setAudioSource(MediaRecorder.AudioSource.MIC);
       mRecorder.setOutputFormat(MediaRecorder.OutputFormat.MPEG_4); // other?
       mRecorder.setOutputFile(mFileName); // end in m4a, for compatibility
       mRecorder.setAudioEncoder(MediaRecorder.AudioEncoder.AAC); // AAC_HE, if supported
    
    
       try {
           mRecorder.prepare();
       } catch (IOException e) {
           Log.e(LOG_TAG, "prepare() failed");
       }
    
    
       mRecorder.start();
    }
    
Partially switching to Chromium

In the interest of being as brief as possible (just ask for details if you want to know more or if you think that it would help), I have started, against my preferences, to start using chromium from Debian instead of Firefox (aka, Iceweasel).

The reason? I have a slow computer (a Core 2 Duo T7250). My main computer. My workhorse.

There are some sites that even with an empty profile and a vanilla Firefox build right from Mozilla Foundation cause Firefox to generate a very heavy CPU load. I briefly reported this via Twitter to one (but not the only!) of the sites in question and to Firefox Site Compatibility.

What do I see? Many CSS and redraw events (thanks to Firefox's profiling tools). In fact, as many events that Firefox alone is able to completely take over 1 (of the 2) cores that I have, with just that single tab open.

I don't see those problems with Chromium, unfortunately (or, at least, they are not that perceptible to my computer's CPU).

So, before I go on and formally file a bug report, I would love to confirm if other people see the same problems that I am seeing. Do you also see something similar? Your input is highly welcome!

A Small Python Project (coursera-dl) Activites

Lately, I have been dedicating a lot of my time (well, at least compared to what I used to) to Free Software projects. In particular, I have spent a moderate amount of time with two projects written in Python.

In this post, I want to talk about the first, more popular project is called coursera-dl. To be honest, I think that I may have devoted much more time to it than to any other project in particular.

With it I started to learn (besides the practices that I already used in Debian), how to program in Python, how to use unit tests (I started with Python's built-in unittest framework, then progressed to nose, and I am now using pytest), hooking up the results of the tests with a continuous integration system (in this case, Travis CI).

I must say that I am sold on this idea of testing software (after being a skeptical for way too long) and I can say that I find hacking on other projects without proper testing a bit uncomfortable, since I don't know if I am breaking unrelated parts of the project.

My use/migration to pytest was the result of a campaign from pytest.org called Adopt Pytest Month which a kind user of the project let me know about. I got a very skilled volunteer assigned from pytest to our project. Besides learning from their pull requests, one side-effect of this whole story was that I spent a moderate amount of hours trying to understand how properly package and distribute things on PyPI.

One tip learned along the way: contrary to the official documentation, use twine, not python setup.py upload. It is more flexible for uploading your package to PyPI.

You can see the package on PyPI. Anyway, I made the first upload of the package to PyPI on the 1st of May and it already has almost 1500 downloads, which is far more than what I expected.

A word of warning: there are other similarly named projects, but they don't seem to have as much following as we have. A speculation from my side is that this may be, perhaps, due to me spending a lot of time interacting with users in the bug tracker that github provides.

Anyway, installation of the program is now as simple as:

pip install coursera

And all the dependencies will be neatly pulled in, without having to mess with multi-step procedures. This is a big win for the users.

Also, I even had an offer to package the program to have it available in Debian!

Well, despite all the time that this project demanded, I think that I have only good things to say, especially to the original author, John Lehmann. :)

If you like the project, please let me know and consider yourselves invited to participate lending a hand, testing/using the program or triaging some bugs.

User-Agent strings and privacy

I just had my hands on some mobile devices (a Samsung's Galaxy Tab S 8.4", an Apple's iPad mini 3, and my no-name tablet that runs Android).

I got curious to see how the different browsers identify themselves to the world via their User agent strings and I must say that each browser's string reveals a lot about both the browser makers and their philosophies regarding user privacy.

Here is a simple table that I compiled with the information that I collected (sorry if it gets too wide):

Device Browser User-Agent String
Samsung Galaxy Tab S Firefox 35.0 Mozilla/5.0 (Android; Tablet; rv:35.0) Gecko/35.0 Firefox/35.0
Samsung Galaxy Tab S Firefox 35.0.1 Mozilla/5.0 (Android; Tablet; rv:35.0.1) Gecko/35.0.1 Firefox/35.0.1
Samsung Galaxy Tab S Android's 4.4.2 stock browser Mozilla/5.0 (Linux; Android 4.4.2; en-gb; SAMSUNG SM-T700 Build/KOT49H) AppleWebKit/537.36 (KHTML, like Gecko) Version/1.5 Chrome/28.0.1500.94 Safari/537.36
Samsung Galaxy Tab S Updated Chrome Mozilla/5.0 (Linux; Android 4.4.2; SM-T700 Build/KOT49H) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/40.0.2214.109 Safari/537.36
Vanilla tablet Android's 4.1.1 stock browser Mozilla/5.0 (Linux; U; Android 4.1.1; en-us; TB1010 Build/JRO03H) AppleWebKit/534.30 (KHTML, like Gecko) Version/4.0 Safari/534.30
Vanilla tablet Firefox 35.0.1 Mozilla/5.0 (Android; Tablet; rv:35.0.1) Gecko/35.0.1 Firefox/35.0.1
iPad Safari's from iOS 8.1.3 Mozilla/5.0 (iPad; CPU OS 8_1_3 like Mac OS X) AppleWebKit/600.1.4 (KHTML, like Gecko) Version/8.0 Mobile/12B466 Safari/600.1.4
Notebook Debian's Iceweasel 35.0.1 Mozilla/5.0 (X11; Linux x86_64; rv:35.0) Gecko/20100101 Firefox/35.0 Iceweasel/35.0.1
Moto G4 Plus Firefox Focus 2.2 Mozilla/5.0 (Linux; Android 7.0) AppleWebKit/537.36 (KHTML, like Gecko) Version/4.0 Focus/2.2 Chrome/61.0.3163.98 Mobile Safari/537.36

So, briefly looking at the table above, you can tell that the stock Android browser reveals quite a bit of information: the model of the device (e.g., SAMSUNG SM-T700 or TB1010) and even the build number (e.g., Build/KOT49H or Build/JRO03H)! This is super handy for malicious websites and I would say that it leaks a lot of possibly undesired information.

The iPad is similar, with Safari revealing the version of the iOS that it is running. It doesn't reveal, though, the language that the user is using via the UA string (it probably does via other HTTP fields).

Chrome is similar to the stock Android browser here, but, at least, it doesn't reveal the language of the user. It does reveal the version of Android, including the patch-level (that's a bit too much, IMVHO).

I would say that the winner respecting privacy of the users among the browsers that I tested is Firefox: it conveys just the bare minimum, not differentiating from a high-end tablet (Samsung's Galaxy Tab S with 8 cores) and a vanilla tablet (with 2 cores). Like Chrome, Firefox still reveals a bit too much in the form of the patch-level. It should be sufficient to say that it is version 35.0 even if the user has 35.0.1 installed.

The bonus points with Firefox is that it is also available on F-Droid, in two versions: as Firefox itself and as Fennec.

Edit, 2017-10-19: Added the User-Agent string of Firefox Focus. Quite different from regular Firefox, as we can see.

Uploading SICP to Youtube

Intro

I am not alone in considering Harold Abelson and Gerald Jay Sussman's recorded lectures based on their book "Structure and Interpretation of Computer Programs" is a masterpiece.

There are many things to like about the content of the lectures, beginning with some pearls and wisdom about the craft of writing software (even though this is not really a "software enginneering" book), the clarity with which the concepts are described, the Freedom-friendly aspects of the authors regarding the material that they produced and much, the breadth of the subjects covered and much more.

The videos, their length, and splitting them

The course consists of 20 video files and they are all uploaded on Youtube already.

There is one thing, though: while the lectures are naturally divided into segments (the instructors took a break in after every 30 minutes or so worth of lectures), the videos corresponding to each lecture have all the segments concatenated.

To better watch them, accounting for the easier possibility to put a few of the lectures in a mobile device or to avoid fast forwarding long videos from my NAS when I am watching them on my TV (and some other factors), I decided to sit down, take notes for each video of where the breaks where, and write a simple Python script to help split the videos in segments, and, then, reencode the segments.

I decided not to take the videos from Youtube to perform my splitting activities, but, instead, to operate on one of the "sources" that the authors once had in their homepage (videos encoded in DivX and audio in MP3). The videos are still available as a torrent file (with a magnet link for the hash 650704e4439d7857a33fe4e32bcfdc2cb1db34db), with some very good souls still seeding it (I can seed it too, if desired). Alas, I have not found a source for the higher quality MPEG1 videos, but I think that the videos are legible enough to avoid bothering with a larger download.

I soon found out that there are some beneficial side-effects of splitting the videos, like not having to edit/equalize the entire audio of the videos when only a segment was bad (which is understandable, as these lectures were recorded almost 30 years ago and technology was not as advanced as things are today).

So, since I already have the split videos lying around here, I figured out that, perhaps, other people may want to download them, as they may be more convenient to watch (say, during commutes or whatever/whenever/wherever it best suits them).

Of course, uploading all the videos is going to take a while and I would only do it if people would really benefit from them. If you think so, let me know here (or if you know someone who would like the split version of the videos, spread the word).

Problems with Emacs 24.4

This is, essentially, a call for help, as I don't really know which program is at a fault here.

Given that Emacs's upstream converted their repository from bzr to git, all the commits in mirror repositories became "invalid" in relation to the official repository.

What does this mean in practical terms, in my case? Well, bear with me while I try to report my steps.

Noticing a regression and reporting a bug

There is a regression with Emacs 24.4 relative to 24.3, which I discovered after Emacs 24.4 became available in Debian's sid.

The regression in particular is that Emacs 24.4 doesn't seem to respect my Xresources, while 24.3 does (and this is 100% reproducible: I kept the binary packages of version 24.3 of emacs24 and I can install and reinstall things).

When I reported this to upstream, I received a reply that it worked fine with another person that was using XFCE with unstable.

Testing various Desktop environments

As I am using the MATE desktop environment, I proceeded to test this assertion by installing XFCE. Emacs 24.4 read my Xresources. I went ahead and installed LXDE. It worked again. I tried once more with GNOME 3, but "regular" GNOME 3 just crashed. I tried with GNOME 3 Classic and Emacs 24.4 just worked again.

Going deep into the rabbit's hole

Then, I got more curious and I tried to see why things worked the way that they did and given that there was a mirror of the Emacs repo on github, I cloned it and started to git bisect to find the first problematic commit (I have no idea if bzr even offers something like git's bisect and I wouldn't really know how to do it as quickly as I do with git).

To cut short a long story, after many recompiles, many wasted hours, a lot of wasted electrical energy, I found a bad commit and reported it.

I received no response after that.

The new repo enters in action

Of course, all my hard work bisecting things was completely invalidated after the transition to the new repository went live.

To make things relevant again, I used the awesome powers of git, restricting the changes of the newly cloned repository to the e-mail of the committer in question (Chong Yidong) and, from there, I proceeded to another painful process of git bisects.

And, sure enough, the first bad commit was the same one that I found with the previous tree.

Semi-blindly reverting this commit, and also semi-blindly resolving the conflicts make Emacs's from master work again on my system, but I highly suspect that (given the way that I did it), it would not really be appropriate for upstream.

But given also that I failed to receive feedback after my original report, I am not too confident that this bug can be solved soon (even if it doesn't qualify for being fixed in Debian 8).

After all this, I don't really know what else to do. I even filed a bug report (more like a request for help) to the Debian MATE maintainers.

As a side note, I would have filed a bug to upstream MATE, but it is not really clear what the proper procedure to report bugs to them is---they seem to use github's issues system, but given that they have separate repositories for each component of the project, and that I don't know precisely what repository to report to (or even if it applies to MATE after all), I am more or less paralyzed.

A side note

I must say that the conversion was well done by Eric Raymond, because the whole .git repository of the new repo is only about 200MB, with history going back to 1985, while the other repository had about 800GB.

The importance of flexible deadlines in MOOCs

I have always thought that having flexible deadlines in MOOCs is important, despite not having used them too much. Until last night, that is.

The course in question is Stanford's Statistical Learning and they adopted a policy of letting the students complete all the assignments of the 10 week course until the last day of the course, in March 21. Then, they graciously extended the deadline to April 4.

Were it not for such an extension, I would not have completed the course. I sent them a message on the course forums this morning to thank them for this, as this, in my experience, is not so common with MOOCs.

I reproduce below the message that I posted on the forums:

Subject: Thank you also for the EXTENSION of the deadlines

Dear professors and staff,

I would like to thank you (of course) for the course, as many others have already. Despite not liking edX platform too much (preferring the UI of, say, Coursera), your course was an exception, interesting enough (and funny enough) that I sticked to it until the end.

But I would like to bring attention here to a point that many may not have appreciated (or, perhaps, not expressed as clearly as I thought that it deserved), namely, the extension of the deadlines (and a uniform deadline for all homeworks).

In particular, due to some unfortunate facts of my personal life, I could not work on the course at all in the past few weeks. In fact, I completed units 4, 5, 6, 7, 8, 9, and 10 in the last 3 or 4 days and submitted my last quiz a few hours before today's midnight (at my local time, UTC-0300), and I still got a passing grade(!).

Even with my desire to finish the course, this would not have been possible if you had not graciously allowed for the deadline extension. I am sure that I may not be alone here in appreciating this extension (even though I think that I may be many standard deviations from the mean, doing the homework and R programs of 7 weeks in only 4 days).

Thank you so very much for everything (including this extension!),

Rogério Brito.