The importance of flexible deadlines in MOOCs

I have always thought that having flexible deadlines in MOOCs is important, despite not having used them too much. Until last night, that is.

The course in question is Stanford's Statistical Learning and they adopted a policy of letting the students complete all the assignments of the 10 week course until the last day of the course, in March 21. Then, they graciously extended the deadline to April 4.

Were it not for such an extension, I would not have completed the course. I sent them a message on the course forums this morning to thank them for this, as this, in my experience, is not so common with MOOCs.

I reproduce below the message that I posted on the forums:

Subject: Thank you also for the EXTENSION of the deadlines

Dear professors and staff,

I would like to thank you (of course) for the course, as many others have already. Despite not liking edX platform too much (preferring the UI of, say, Coursera), your course was an exception, interesting enough (and funny enough) that I sticked to it until the end.

But I would like to bring attention here to a point that many may not have appreciated (or, perhaps, not expressed as clearly as I thought that it deserved), namely, the extension of the deadlines (and a uniform deadline for all homeworks).

In particular, due to some unfortunate facts of my personal life, I could not work on the course at all in the past few weeks. In fact, I completed units 4, 5, 6, 7, 8, 9, and 10 in the last 3 or 4 days and submitted my last quiz a few hours before today's midnight (at my local time, UTC-0300), and I still got a passing grade(!).

Even with my desire to finish the course, this would not have been possible if you had not graciously allowed for the deadline extension. I am sure that I may not be alone here in appreciating this extension (even though I think that I may be many standard deviations from the mean, doing the homework and R programs of 7 weeks in only 4 days).

Thank you so very much for everything (including this extension!),

Rogério Brito.

Posted
More completed MOOCs

This weekend I received my 17th certification (or Statement of Accomplishment) for MOOCs. In particular, this last MOOC that I took was Creative, Serious and Playful Science of Android Apps, offered by Lawrence Angrave of University of Illinois at Urbana-Champaign.

While I certainly knew that the course was an Introduction to Programming, the reasons I took it were twofold:

1. For the "novelty" (for me, at least) of writing some Android Apps
2. To reacquaint myself with Java, which I have not touched since 1999, when I wrote a compiler, in the ages of JDK 1.0 being transitioned to JDK 1.1.

I think that the time the course took (and which I invested in) was really well spent. In fact, I learned some nice things which I would not have touched otherwise. For instance, I made the conscious effort to use an IDE (Eclipse), despite being a person that would do most of the things inside Emacs and compile the programs with command-line tools.

Despite being huge, Eclipse has some really nice features and the auto-completion is amazing. Since Java is so strongly typed and Eclipse knows Java pretty well, it almost completely writes your programs for you.

Of course, my interest also made me want to streamline the build process and use the command line tools (with ant, another tool that was "alien" to me, having used makefiles for everything that I needed).

The use of Eclipse (and the Android development environment in general) in my Debian unstable computers was not without problems, but after spending more time trying to fix Eclipse than really doing the homework (and pulling the little hair that I still have left), one really helpful post nailed it. (Yes, I had to remove other packages like gimp, but such is life).

While playing with the Android side of things, I put one badly written (and not really "android-ish") app in a github repository (which I think is aptly named "Toy Android Apps"), which served me the purpose of learning some Android-specific concepts.

But the course was helpful not only to get in touch with the tools (Eclipse, ant, running Android virtual machines with the help of kvm), but also on the Java side. I lost a bit of the prejudice against it (even though I still think it is a verbose language, especially when compared to, say Python), and I, also intend to contribute to one apache project or another that have low-hanging fruits. Actually, I have signed their Contributor License Agreement and I even had my first patch accepted!

Aside: Well, sadly SVN doesn't support different committer and author fields like git does and it seems that doing many small commits may not be the Apache way of doing things—but the important part of all this is that the code is there and there is so much more to be done.

Aside #2: After a lot of time spent converting the repository to git (which is, after all, what I use), I put a mirror of Apache's the commons graph project.

Posted
Paco de Lucia

I was super sad to know that Paco de Lucia passed a few days ago. It was shocking to know that he was so young (only 66 year old, if I am not mistaken).

To share some of his fine work with his frequent collaborators John McLaughlin and Al Di Meola to people that may not otherwise know him or his work, I offered on a Facebook post to upload a bootleg of a show of the Guitar Trio (John McLaughlin, Paco De Lucia and Al Di Meola) with the entire performance (possibly out of order, as that is a bootleg, after all) ripped from a broadcast to Dutch TV station.

This performance includes the exceptional Paco de Lucia prominently and the show has many songs from the "Friday Night in San Francisco" record, including the magnificent (IMVHO) "Mediterranean Sundance".

The credits of the video are, of course, of the musicians.

I only ripped, deinterlaced (remember, analog TV—it was interlaced), denoised, encoded, and uploaded the video to youtube.

Posted
Trivial fact #3: Continued fractions via matrices

# Note

Apparently, it's not Debian planet that has problems to deal with MathJax (which makes posts usingn it appear illegible on Debian planet), but the ikiwiki plugin that I am using which generates garbage in the feeds that get consumed by planet.debian.org.

If you it know of a better plugin (which doesn't generate such output) please let me know, especially if it is not super computationally expensive for a Core 2 Duo T7250 (which is my notebook).

# The real thing

As William Stein is now offering a course on Number Theory and he has been posting recorded videos of his lectures, I started watching some of them (mainly, the ones regarding continued factions). In particular, he shows the usual recursive formula for convergents of a continued fraction and that's super nice.

For the curious reader, the recurrence relations for the convergents of $$[a_0, a_1, \ldots, a_n, \ldots]$$ are: $p_n = a_n p_{n-1} + p_{n-2}\\ q_n = a_n q_{n-1} + q_{n-2},$ with initial conditions ($$p_{-2} = 0$$, $$p_{-1} = 1$$, $$p_0 = a_0$$, $$q_{-2} = 1$$, $$q_{-1} = 0$$, $$q_0 = 1$$).

He even motivated the use of continued fractions with the golden ratio, which is super nice, given that I like the subject and have been writing a document collecting facts that I know about the Fibonacci numbers (well, this document is horribly incomplete and not even close to something that I would consider proper for public consumption—I plan on publishing them soon).

OK, after he discussed the basics of it convergents, he noted that the recurrence relations are like those defining the Fibonacci numbers, just that one of the terms is "weighted" by the coefficients of the continued fraction.

I missed one thing, though: neither his book nor wikipedia's article on continued fractions mentions a very neat, alternative way to express the convergents of continued fractions (truth be said, I took a quick peek at wikipedia's article and I found that it doesn't mentionat least explicitly, in a 1 min skim—it may well be buried somewhere else).

I, then, proposed the following exercise for his students (which he apparently liked, as he +1's the suggestion):

Prove that the recurrence relation of the $$p_i$$'s and $$q_i$$'s that we mentioned before can be obtained via matrix multiplication. More precisely, prove that: $\begin{bmatrix} a_0 & 1\\ 1 & 0 \end{bmatrix} \begin{bmatrix} a_1 & 1\\ 1 & 0 \end{bmatrix} \cdots \begin{bmatrix} a_n & 1\\ 1 & 0 \end{bmatrix} = \begin{bmatrix} p_n & p_{n-1}\\ q_n & q_{n-1} \end{bmatrix}.$ As a corollary, derive Cassini's identity for the Fibonacci Numbers.

Please, if you any errors on this, please let me know so that I can fix it.

Edit: Thanks "noone" for spotting a typographical error.

Posted

I wrote in a previous post that Youtube changed their way of delivering videos, with the use of Dynamic Adaptive Streaming over HTTP. On top of that, they started serving both the audio and the video in separate streams, which meant trouble for downloader tools like youtube-dl.

As I mentioned in that previous post:

What does this mean in practical terms for users of youtube-dl? Well, if you wanted to download videos in resolutions like the 480p (format 35) that I mentioned, then you will probably have to change your way of doing things, until a more automated solution is in place.

You will have to download both the audio and the video and, then, "combine" them (that is, multiplex them) to create one "normal" video file with both the audio and the video.

And latter, I wrote:

Otherwise, to download 480p videos (which I do for lectures and so on with other projects of mine, like edx-dl) I have to call youtube-dl twice: once for format 135 and another for format 140, since the old format 35 files are much smaller than the lower resolution 360p files (due to the former being encoded in High profile vs. the latter being encoded in Constrained Baseline profile).

(...)

Well, now, we don't have this problem anymore:

The new release of youtube-dl brings us many goodies, including that it is possible to automatically combine/merge/multiplex audio and video formats that Youtube now offers separately (See the previous comments about separate audio and video).

Now, if you want a 480p video in H.264 format, High profile, with 128kbps AAC audio (this used to be Youtube's format 35), you can specify format -f 135+140 on the command-line and it will download both the audio, the video and multiplex it with ffmpeg (or avconv, depending on what you have installed).

Besides being convenient, this automatic downloading and merging makes it unnecessary to write scripts to, say, retrieve all the videos in a playlist that contains a lot of lectures (see one example here, taught by Benedict Gross).

Version 2014.02.17 of youtube-dl will be soon in your favorite mirror of the Debian archives.

Posted
Miscellanea

# Init system

I am happy that Debian's Technical Committee was able to reach a very good decision with the choice of systemd as the default init system for the Linux ports.

In particular, I was very pleasantly surprised by the lucidity of Russ Allbery's analyses and, above all, his patience even with people that, in my understanding, were just being trolls or trying to cause confusion and disrupt a process that was already chaotic.

Russ, thank you for your exemplary role.

(Also, you type fast or what? You fill whole e-mails seemingly faster than I can read!)

## Unexpected union of forces

It seems to me that Mark Shutterworth's decision of (perhaps?) embracing Debian's decision in the init system is a good thing.

I think that one way where Ubuntu did a great thing with respect to Debian was their initial attitude towards cherry-picking the best of breed of Free Software from Debian's archive and polishing that to create a really "for human beings" operating system. I hope that this means that, at least in part, money (Mark's?) can be channeled in the best of the interests so that we all gain from it.

(And, to be honest, I see no problem in making money with Free Software and I welcome a well done integration job).

# More polarizing things

I feel that, now, I may even have some hope of sending another potentially polarizing bug to the Tech Ctte (that is ffmpeg vs. libav) and have faith that some rational decision would be made.

I was not even aware of it, but the ffmpeg folks even featured a news item in their page of me having filed the RFP bug mentioned above.

# DebConf 14

If everything goes as expected, I hope to attend this year's DebConf. In fact, I am anticipating this, since I would love to, e.g., talk with Joey Hess and discuss a better way of having GNU parallel and the parallel program in moreutils have amicable co-installability among many, many other things with respect to my packages.

And, quite probably, work on packaging ffmpeg with other interested people (which seem to be numerous).

Edit: Fix markdown link pointing twice to the ffmpeg site. Thanks Marius Gedminas for pointing this out.

Posted
Trivial fact #2: Harmonic Numbers

I don't know how Debian planet deals with MathJax (it seems to become illegible), but I will try to post this, anyway.

Fact: Since the harmonic numbers can be approximated by the natural logarithm, that is, $$H_n \simeq \ln n$$, we can clearly see that $\frac{1}{i+1} + \cdots + \frac{1}{n} = H_n - H_i \simeq \ln n - \ln i = \ln \frac{n}{i},$ which is a trivial approximation that may be handy sometimes.

If you spot any errors on this, please let me know.

Posted
Developing for Android with Debian

I have just enrolled in a MOOC to learn about programming with Android. There are many tutorials and reports of people using Linux for the development of the apps out there, but, unfortunately, many of them are outdated or contain misleading information.

You will find bazillion instructions out there telling you to install JDK6. (Even the instructions in the MOOCs say so, and Google says that it is the only supported version of Java). Even more than that, some of them will tell you that you should not use OpenJDK, but use Oracle's JDK instead.

So, if you have a pure amd64 system and you try to install the x86-64 version of Google's Android Developer Tools (ADT), you are going to have a surprise: you will have all sorts of programs telling you that you have file not found, even though the required binaries are there.

(This is, of course, besides install openjdk-7-jdk, if you wish to use the integrated Eclipse environment).

Turns out that the so-called 64-bit ADT/SDK actually has many 32-bit executables(!) and they aren't statically linked. Inspecting them, I needed the following packages:

    gcc-4.8-base:i386
libc6-i686:i386
libc6:i386
libgcc1:i386
libstdc++6:i386
zlib1g:i386


It is worth observing that:

• libstdc++6:i386 depends (transitively) on libc6:i386, libgcc1:i386, and gcc-4.8-base:i386.
• zlib1g:i386 depends (transitively) on libc6:i386, libgcc1:i386, and gcc-4.8-base:i386.

Given the above, and given that the version of the GCC libraries can vary a lot, it is perhaps better to simply do something like:

apt-get install libstdc++6:i386 zlib1g:i386


Although not necessary, since you are using an amd64 machine, your computer already supports many more recent instructions that will be used if you install the package libc6-i686:i386.

To avoid the emulator from spitting out error messages related to libGL.so, install libgl1-mesa-dev (note: not i386 here).

This way, you will be able to run the embedded QEMU with full KVM acceleration if you pass the options -qemu -m 1024 -enable-kvm on Eclipse's "Run > Run Configurations... > Target > Additional Emulator Command Line Options".

After that, you are probably good to go.

Posted
Trivial fact: convexity of polyhedra

Just a trivial fact: every polyhedron that is used in linear programming is convex, that is $$Ax \leq b$$ is convex, for a matrix $$A$$ and a (column) vector $$b$$.

Proof: Take any $$x', x''$$ that satisfy the system of inequalities $$Ax \leq b$$. Then, for $$0 \leq \lambda \leq 1$$, we have that $$\lambda Ax' \leq \lambda b$$, that is $$A \lambda x' \leq \lambda b$$. Similarly, for $$x''$$, we have that $$A (1-\lambda) x' \leq (1-\lambda) b$$. Summing the inequalities, we get: $A[\lambda x' + (1-\lambda) x''] \leq [\lambda + (1-\lambda)] b = b,$ which means that $$\hat{x} = \lambda x' + (1-\lambda) x''$$ is again a solution of the original set of inequalities, thus concluding the argument.

Posted
Assorted random notes

This post is just a bunch of random notes, but I think that things are going well, this time.

# Btrfs

Just started playing with btrfs after converting a 2TB disk with about 1.3TB of data from ext4 to btrfs. This particular filesystem contains my backup data and I hope that btrfs is able to live up to high standards.

I will probably write a detailed review with my impressions, but suffice to say that it is working fine and I hope that I will be happy for a long time to come. My other filesystems, for the moment, will be ext4.

Thanks to John Lehmann's generosity, now coursera-dl has a new home: a github organization of its own!

Some people may have noticed, others may not, but when downloading videos from Youtube, they apparently are getting more aggressive with the use of Dynamic Adaptive Streaming over HTTP (also called DASH) and, as a result, some (perhaps going to be all in the near future?) of the videos may not be available in the resolution/formats that you used to like (like me, with format number 35).

By the way, one thing that is interesting with youtube videos provided via DASH is that they are available in different streams: one for the video and another for the audio.

What does this mean in practical terms for users of youtube-dl? Well, if you wanted to download videos in resolutions like the 480p (format 35) that I mentioned, then you will probably have to change your way of doing things, until a more automated solution is in place.

You will have to download both the audio and the video and, then, "combine" them (that is, multiplex them) to create one "normal" video file with both the audio and the video. I usually do this via:

 ffmpeg -i audio.m4a -i video.mp4 -vcodec copy -acodec copy combined.mp4 

If you prefer having a Matroska container instead of an mp4 container (which, BTW, results in smaller muxing overhead), then you can use the command line:

 mkvmerge -o combined.mkv audio.m4a video.mp4 

Oh, those m4a and mp4 extensions are a new addition that I just sent upstream (in the past, both would have been named with an extension of mp4).

As, an aside, I like formats 135 for video and 140 for audio, for the reasons that I mentioned in a comments to issue 1612:

Otherwise, to download 480p videos (which I do for lectures and so on with other projects of mine, like edx-dl) I have to call youtube-dl twice: once for format 135 and another for format 140, since the old format 35 files are much smaller than the lower resolution 360p files (due to the former being encoded in High profile vs. the latter being encoded in Constrained Baseline profile).

While this is unfortunate for some, this is a good thing for others: I once had a blind user of youtube-dl asking me if he could avoid downloading the whole video just to extract the audio, so that he save on bandwidth. Well, now this is possible.

Expect a new version of youtube-dl to be uploaded soon to Debian unstable.

# Music

It is so nice to see the music that I like getting better. It was particularly pleasant to see Tarja Turunen being joined on stage by Floor Jansen.

In fact, it is my impression that the once female-fronted bands are getting more and more into collaborations and side-projects and they are, in many ways, getting more refined. Examples of this includes Floor, Tarja, Anneke van Giersbergen, Sharon den Adel, Kari Rueslåtten and many others that I can't remember right now.

Of course, some thanks to people that take the time to film, and upload/share the videos so that other people not at these events can watch them is highly appreciated and they usually are treated as anonymous, but, of course, this should be fixed.

Posted