Have you ever seen this message when updating your Debian-based system: "The following packages have been kept back"? Here's why you see it and how to solve it:

You just tried to run the following command:

$ sudo apt-get update && sudo apt-get upgrade --show-upgraded -y

... and you get the message:

The following packages have been kept back

Running apt-get dist-upgrade is dangerous for a stable environment. The wrong source.list setting and you end up with a broken Debian. You might get the entire application upgraded to a version you don't want. For example, the kernel upgrade is kept back. You just want to upgrade the kernel, not the entire distribution.

A better way to handle a kept back package is:

1. sudo aptitude
2. If you have kept back package you should see Upgradable Packages on top of the list.
3. Hit + on that list
4. Hit g twice
5. Answer debconf stuff if asked
6. Press return to continue
7. Press Q to quit
8. Press yes

The kept back packages are now installed.

Below are my notes from a really interesting article about the creation of programming languages by Walter Bright published on January 21, 2014 in the Dr. Dobbs Journal.

http://www.drdobbs.com/architecture-and-design/so-you-want-to-write-your-own-language/240165488

Work

Tons of work ... years of work ... need intrinsic motivation. Not a major cash investment

Design

Syntax matters a lot. It needs to be something that appeals to your target audience. A mix of familiar syntax + aesthetic beauty.

These should NOT be considerations:

• Minimizing keystrokes
• Easy parsing. not hard to write parsers with arbitrary lookahead
• Minimizing the number of keywords

These things are TRULY GOOD:

• Context-Free Grammar. code is parsable without having to look up in a symbol table
• Redundancy. the grammar should be redundant to get good error messages
• Tried and True. Stick with tried and true grammatical forms. cuts the learning curve. e.g. people expect + and * to follow a certain precedence rule.

After syntax, the important thing is semantic processing... meaning is assigned here to syntax structure. This is where you spend most of the design and implementation time. There is no glory in the semantics, but it is the whole point of the language.

After the semantice phase, the compiler does optimizations and code generation This is collectively called the back-end. Very challenging and complicated these 2 phases. Try using an existing back end such as one of JVM, CLR, GCC, LLVM.

Implementation

Regex is the wrong tool for lexing and parsing.

Don't bother wasting time with lexer and parser generators or other so-called compiler compilers. They are a waste of time. Writing the lexer and the parser is a tiny percentage of the job of writing a compiler.

Error messages are a big factor in the quality of implementation of the language. Use the following guidelines:

• print the first error message and quit
• guess what a programmer intended, repair the AST and continue
• The poisoning approach. any operation with a NaN results in a NaN silently.

Compiler speed matters a lot... Most compilers are pigs. The secret to making a compiler fast is to use a profiler.

You must/have to be using these tools in order to be successful:

• valgrind. saved C and C++ from oblivion
• git / github like public repo. trasnformative tools. collaboration. track the origin of every line of code
• Automated testing framework + Coverage analyzer
• Automated documentation generator
• Bugtracker + Wiki + Blog + Google Groups + Mailing List + IRC Channel + Twitter + Facebook

Lowering

A semantic technique. It consists of, internally, rewrting more complex semantic constructs in terms of simpler ones. e.g. while loops and foreach loops can be rewritten in terms of for loops. Then the rest of the code only has to deal with for loops. If there are special-case rules in the language that prevent lowering rewriting, then revisit the design of the language.

Runtime Library

Critical is the need to write a runtime library. This is a major project. It serves as the demostration of how the language features work. It better be good.

Get these right:

• I/O performance. Slow IO will make that language look bad. the benchmark is C stdio
• Memory allocation. Most of the time programs spend doing this
• Transcendental functions... What are these? Find a library that already implements these functions ... Floating point stuff

What should go in a library

• File IO
• Network IO
• String
• Algorithms
• Data Structures
• OS Interaction
• Unicode - UTF-8
• Date / Time
• Memory Management
• Process Control
• Signals
• Math
• Data Types
• Crypto
• Reflection

After the Prototype

• Presentations
• Articles
• Tutorials
• Books
• Programmer Meetings
• Conferences
• Companies

Go anywhere they will have you and show it off. Get used to public speaking. Get as much feedback as you can.

When administering Linux systems I often find myself struggling to track down the culprit after a partition goes full. Here's a solution which relies on standard Linux commands to find where all that disk space is being consumed.

du -h / | grep '[0-9\.]\+G'

One caveat: Don't go straight to du /. Use df to find the partition that's hurting you, and then try du commands.

This weekend I found a really cool web service at FreeGeoIp.net. Their API is really simple... just send a HTTP GET request to:

http://freegeoip.net/{format}/{ip_or_hostname}

Here are some of the implementation details:

1. SSL is supported
2. Formats are csv, json, and xml
3. IP is optional. It defaults to your current IP.

I quickly implemented a comnmand line client written in Python 3. It uses the awesome requests library and ElementTree to parse the resulting XML response. Here's the code...

def main():
    args = parse_args()

    MYIP_URL = 'http://freegeoip.net/xml/'
    MYIP_HEADERS = {'Accept': 'application/xml'}

    if args.ip is not None:
        MYIP_URL = MYIP_URL + args.ip

    rspn = requests.get(MYIP_URL, headers=MYIP_HEADERS)
    tree = etree.ElementTree(etree.fromstring(rspn.text))
    root = tree.getroot()

    ip = root.find('Ip')
    cntr = root.find('CountryName')
    reg = root.find('RegionName')
    city = root.find('City')

    print('Your Current IP: %s' % ip.text)
    print('You appear to be in %s, %s, %s' % (city.text, reg.text, cntr.text))

Or you can get the whole code from my github profile.

I was reading Tim Bray's blog today and he mentions that at OSCON this year there was a company booth which was capturing the attention of anti-social geeks in a novel way.

... they had a big screen sticking up showing a continuous loop of a screen capture of someone setting an app up for OAuth, with lots of JSON and JS and other config stuff going back and forth across the screen. A lot of your antisocial geeks who don’t actually want to interact with anyone remotely salesy were standing there watching this with educated attention.

This is a really interesting trick and I can count myself in that group of anti-social geeks often. I saw another company doing this at the O'Reilly Velocity conference earlier this year.

For a long time now I have known that the GNU Emacs chord C-l centers the current line on the window. This chord in bound to the elisp function recenter-top-bottom, which I happen to have been reading the documentation for today. I learned that if you use C-l in quick succession, it will cycle the placement of the current line at the center, the top, and the bottom of the current window.

I didn't know this and it's a very handy trick that fits really well with the way I use Emacs.

Supposing you have a clean install of a Debian-based distro such as Debian 7 proper or Xubuntu 14.04, this notes will walk you through setting up a Python development environment where you can leverage the power of Virtualenv through Virtualenvwrapper.

Why virtualenv?

When you code in python you will quickly find out that different projects have different requirements for the python version you need to use and also the versions of libraries you need to use. Virtualenv will allow you to compartmentalize the requirements on a per project basis without making you install libraries and python versions globally or system-wide.

Ok, ok... Here are the goods...

You have your new shinny install of Debian which comes with a fairly recent version of python pre-installed (somewhere above 2.7.3). Now you need to install a package manager for python and the one I use is pip.

sudo apt-get install python-pip

Once you have pip installed, you will need to also install virtualenv, like so:

sudo pip install virtualenvwrapper

Virtualenvwrapper is a very nice layer on top of virtualenv that makes working with it much more intuitive and convenient.

Next, you may want to automate a couple of things like I did. First, create a ~/.virtualenvs directory. This is where virtualenvwrapper will keep track of your environments. Then open you .bashrc file and add a few lines to it:

export WORKON_HOME=$HOME/.virtualenvs
if [ -f /usr/local/bin/virtualenvwrapper_lazy.sh ]; then
    . /usr/local/bin/virtualenvwrapper_lazy.sh
fi

Now you are ready to start working, so open a new terminal, setup a virtual environment, and start cranking out some python code. Here's a quick reference for virtualenvwrapper:

Start new virtual environment?

mkvirtualenv [env-name]

Leave the current virtual environment?

deactivate

List all available virtual environments?

workon

lsvirtualenv -b

Enter an existing virtual environment?

workon [env-name]

List packages installed in current virtual environment?

pip freeze

Remove a virtual environment?

rmvirtualenv [env-name]

Start a virtual environment with specific python version?

mkvirtualenv [env-name] -p /usr/bin/python3.4

Replace [env-name] above with your actual environment name and enjoy!

I'm an avid GNU Emacs user and I like to have the active frame maximized at start-up. This is easily done on Linux:

(set-frame-parameter nil 'fullscreen 'maximized)

But on MS Windows this is not so trivial... Like many of you, I have to use Windows for my daily work. I tried many different approaches to get that frame maximized on Windows, but just recently I was able to achieve that outcome in a reliable manner that works across many different versions of Windows.

What worked!

In an effort to not bury the lead, as they say... this is what I have done that works:

(defun w32-maximize-frame ()
  "Maximize the current frame (windows only)"
  (interactive)
  (w32-send-sys-command 61488))

(if (eq system-type 'windows-nt)
    (progn
      (add-hook 'window-setup-hook 'w32-maximize-frame t))
  (set-frame-parameter nil 'fullscreen 'maximized))

In the above code I address maximizing the frame in the two platforms I use most: Linux and Windows. I defined the function w32-maximize-frame, which send the Windows message to the Emacs frame for it to maximize itself. The underlying function w32-send-sys-command is well-known, but initially it did not work for me as I'll explain below.

The strategy that did work for Windows was to wrap the call to w32-send-sys-command in a function and then add a call to it to the window-setup-hook. I tested this on the 7 and 8.1 versions of desktop Windows and on 2008 R2, 2012 versions of Windows Server and using Emacs 24.3. And if Emacs is not running on Windows, I set the frame parameter fullscreen to maximized for the current frame.

What didn't work...

I started this journey into figuring out how to start Emacs maximized a while back. In the beginning I would let Emacs start and then just use the maximize button on the title bar, but then I started using things like initial-frame-alist and set-frame-* to get the top, left and size just right for my different screens. The problem with that approach is that I had to tweak it for every new install of Emacs on every computer I used. It got old really quick.

Then I found the set-frame-parameter trick that you can see in the code above. But that only worked for Linux variants. At this point during my research, I have already found out about the w32-send-sys-command approach, but using it on my init.el file directly was not working. The frame would think itself as being maximized, and you could see that by looking at the state of the maximize button on the top-right of the window. But visually, the window would remain small.

Next I started using the maxframe.el package, but that really didn't last all that long. I found that there were several flaws with it: First, the package would not truly maximize the frame, but it calculates the size of the frame based on your monitor's resolution and font face sizes and stretch your frame accordingly; Second, the package would not account for the position of my XFCE desktop panel and place the Emacs title bar under it; And third, maxframe.el would not account for a multiple monitor setup and stretch the frame across my two monitors. So I dropped it and went back to the drawing board.

Conclusion

It turns out that the proper way to achieve the maximize frame on start-up on Windows is to use the window-setup-hook. This variable is a normal hook that Emacs runs after handling the initialization files. It turns out that with the previous approaches, I was trying to send a message in Windows to an Emacs frame that didn't yet exist. This hook is used for setting up communication with the windowing system and creating the initial window and is, therefore, the proper place, or point in time, to maximize our frame in Windows.

How about this for prescient tin foil hat geekery... While reading the Emacs Elisp source for version 24.3, I stumbled upon a little gem located at ./lisp/play/spook.el.

This file has a created date of May, 1987. Emacs has included this program for many years. Its purpose is to add a series of keywords to email just before sending it. On the theory that the NSA monitors people's email, the keywords would be picked up by NSA's snoop computers, causing them to waste time reading your meeting schedule notices or other email boring to everyone but you and the recipient.

You should try it: open Emacs and type M-x spook to see what happens.

From the commentary on the file itself:

Just before sending mail, do M-x spook. A number of phrases will be inserted into your buffer, to help give your message that extra bit of attractiveness for automated keyword scanners. Help defeat the NSA trunk trawler!

If you have ever used a relatively modern flavor of UNIX, you likely used a tool called wget. It comes as a standard piece of almost every single UNIX variant, Linux included. Here are some ways that I use it on a daily basis:

Download file

wget http://ftp.gnu.org/gnu/wget/wget-1.15.tar.gz

Downloading files with different names

wget http://ftp.gnu.org/gnu/wget/{wget-1.15.tar.gz,wget-1.15.tar.gz.sig}

Multiple downloads using different protocols

wget http://ftp.gnu.org/gnu/wget/wget-1.15.tar.gz ftp://ftp.gnu.org/gnu/wget/wget-1.15.tar.gz.sig

Using an input file

You can use an input file that contains a list of URLs to download, like so:

wget -i ~/Downloads/urls-for-wget.txt

Resume incomplete download

Sometimes a download is interrupted in the middle. To resume a partial download use the -c option, like so:

wget -c http://ftp.gnu.org/gnu/wget/wget-1.15.tar.gz

Download files in the background

Using the option -b sends wget to the background and also saves the output to a log file:

wget -b ~/Downloads/wget-log.txt http://ftp.gnu.org/gnu/wget/wget-1.15.tar.gz

Throttling download speeds

To limit the download speed, use the --limit-rate option:

wget --limit-rate=100k http://ftp.gnu.org/gnu/wget/wget-1.15.tar.gz

Performing downloads with authentication

Some sites restrict access to content by requiring a username and password. wget can handle that:

wget --user=joeschmoe --password=p4sswOrd! http://ftp.gnu.org/gnu/wget/

More information

Of course, you can mix and match all of these options into a single command, like so:

wget -c -b -a ./wget-log --limit-rate=100k http://ftp.gnu.org/gnu/wget/wget-1.15.tar.gz

You can also go much further than what I described above. The script below will check the web page for the latest listed version of wget and then download it and its signature file if the online version is newer than the local one.

wget -qN $(wget -qO- http://ftp.gnu.org/gnu/wget/ \
          | grep tar | cut -d\" -f6 \
          | tail -n4 | grep gz \
          | sed "s|^|http://ftp.gnu.org/gnu/wget/|")

For more information, check online or use man or info.