The Standard Output (programming)

Get a prime number sequence using Java 8 Streams

Jonas Gorauskas — Wed, 06 May 2015 03:42:37 GMT

With a spirit of expanding on my previous post about checking numbers for primality, here's a quick way to generate a sequence of prime numbers using streams:

public static LongStream primeSequence(long max) {
    return LongStream.iterate(2, i -> i + 1)
            .filter(x -> isPrime(x))
            .limit(max);
}

The above returns a LongStream of prime numbers up to a specified max value. It uses my previous prime checker function isPrime.

Hope it helps!

Check number primality using Java 8 Streams

Jonas Gorauskas — Mon, 04 May 2015 02:44:15 GMT

The canonical way to check if a number is prime in Java uses a loop. It looks like this:

public static boolean isPrime(long x) {
    for (long n = 2; n <= Math.sqrt(x); n++) {
        if (x % n == 0) {
            return false;
        }
    }
    return true;
}

There is a more streamlined and efficient way of doing this using Streams. It looks like this:

public static boolean isPrime(long x) {
    return LongStream.rangeClosed(2, (long)(Math.sqrt(x)))
            .allMatch(n -> x % n != 0);
}

Hope this helps!

A note on migrating python virtual environments

Jonas Gorauskas — Wed, 18 Feb 2015 09:13:52 GMT

The engine that builds this blog is called Nikola it's written in Python. I write the blog and run the Nikola compiler to build the site inside a Python virtual environment.

Up until recently I was doing this work on a machine running the latest Arch Linux and Python 2.7.8, but I have recently rebuilt that machine to run Debian 8, whose default Python interpreter is currently at version 2.7.6 and I copied my virtual environments to the new OS install.

This caused me a problem with my virtual environments that manifested itself as the following error when I tried to run any Python code from within the active virtual environment:

$ pip freeze
Traceback (most recent call last):
  File "/home/jonasg/.virtualenvs/tso/bin/pip", line 7, in <module>
    from pip import main
  File "/home/jonasg/.virtualenvs/tso/local/lib/python2.7/site-packages/pip/__init__.py", line 11, in <module>
    from pip.vcs import git, mercurial, subversion, bazaar  # noqa
  File "/home/jonasg/.virtualenvs/tso/local/lib/python2.7/site-packages/pip/vcs/mercurial.py", line 9, in <module>
    from pip.download import path_to_url
  File "/home/jonasg/.virtualenvs/tso/local/lib/python2.7/site-packages/pip/download.py", line 2, in <module>
    import email.utils
  File "/usr/lib/python2.7/email/utils.py", line 32, in <module>
    from email._parseaddr import quote
  File "/usr/lib/python2.7/email/_parseaddr.py", line 16, in <module>
    import time, calendar
  File "/usr/lib/python2.7/calendar.py", line 9, in <module>
    import datetime
ImportError: No module named datetime

$ nikola build
...
ImportError: No module named datetime

This happens because my virtual environments had a different version of /usr/bin/python2.7 (the one from Arch Linux) that — unlike the new binary — does not include the datetime built-in, and therefore generates an error when it cannot find it on disk anywhere. The new interpreter seems to import it without any file I/O (try running it under strace to check).

The fix for this issue is to activate the virtual environment you are having problems with and run the following:

$ cp /usr/bin/python2.7 $(which python2.7)

There are other ways to fix this issue, but I like the one above rather than re-initializing the virtualenv because I use virtualenvwrapper which does NOT gererate the virtual environment within the present directory.

Hope this helps!

So, you want to write your own language?

Jonas Gorauskas — Tue, 02 Dec 2014 05:24:40 GMT

Below are my notes from a really interesting article about the creation of programming languages by Walter Bright published on January 21, 2014 in the Dr. Dobbs Journal.

http://www.drdobbs.com/architecture-and-design/so-you-want-to-write-your-own-language/240165488

Work

Tons of work ... years of work ... need intrinsic motivation. Not a major cash investment

Design

Syntax matters a lot. It needs to be something that appeals to your target audience. A mix of familiar syntax + aesthetic beauty.

These should NOT be considerations:

Minimizing keystrokes
Easy parsing. not hard to write parsers with arbitrary lookahead
Minimizing the number of keywords

These things are TRULY GOOD:

Context-Free Grammar. code is parsable without having to look up in a symbol table
Redundancy. the grammar should be redundant to get good error messages
Tried and True. Stick with tried and true grammatical forms. cuts the learning curve. e.g. people expect + and * to follow a certain precedence rule.

After syntax, the important thing is semantic processing... meaning is assigned here to syntax structure. This is where you spend most of the design and implementation time. There is no glory in the semantics, but it is the whole point of the language.

After the semantice phase, the compiler does optimizations and code generation This is collectively called the back-end. Very challenging and complicated these 2 phases. Try using an existing back end such as one of JVM, CLR, GCC, LLVM.

Implementation

Regex is the wrong tool for lexing and parsing.

Don't bother wasting time with lexer and parser generators or other so-called compiler compilers. They are a waste of time. Writing the lexer and the parser is a tiny percentage of the job of writing a compiler.

Error messages are a big factor in the quality of implementation of the language. Use the following guidelines:

print the first error message and quit
guess what a programmer intended, repair the AST and continue
The poisoning approach. any operation with a NaN results in a NaN silently.

Compiler speed matters a lot... Most compilers are pigs. The secret to making a compiler fast is to use a profiler.

You must/have to be using these tools in order to be successful:

valgrind. saved C and C++ from oblivion
git / github like public repo. trasnformative tools. collaboration. track the origin of every line of code
Automated testing framework + Coverage analyzer
Automated documentation generator
Bugtracker + Wiki + Blog + Google Groups + Mailing List + IRC Channel + Twitter + Facebook

Lowering

A semantic technique. It consists of, internally, rewrting more complex semantic constructs in terms of simpler ones. e.g. while loops and foreach loops can be rewritten in terms of for loops. Then the rest of the code only has to deal with for loops. If there are special-case rules in the language that prevent lowering rewriting, then revisit the design of the language.

Runtime Library

Critical is the need to write a runtime library. This is a major project. It serves as the demostration of how the language features work. It better be good.

Get these right:

I/O performance. Slow IO will make that language look bad. the benchmark is C stdio
Memory allocation. Most of the time programs spend doing this
Transcendental functions... What are these? Find a library that already implements these functions ... Floating point stuff

What should go in a library

File IO
Network IO
String
Algorithms
Data Structures
OS Interaction
Unicode - UTF-8
Date / Time
Memory Management
Process Control
Signals
Math
Data Types
Crypto
Reflection

After the Prototype

Presentations
Articles
Tutorials
Books
Programmer Meetings
Conferences
Companies

Go anywhere they will have you and show it off. Get used to public speaking. Get as much feedback as you can.

Free GeoIp location web service

Jonas Gorauskas — Thu, 04 Sep 2014 05:54:04 GMT

This weekend I found a really cool web service at FreeGeoIp.net. Their API is really simple... just send a HTTP GET request to:

http://freegeoip.net/{format}/{ip_or_hostname}

Here are some of the implementation details:

SSL is supported
Formats are csv, json, and xml
IP is optional. It defaults to your current IP.

I quickly implemented a comnmand line client written in Python 3. It uses the awesome requests library and ElementTree to parse the resulting XML response. Here's the code...

def main():
    args = parse_args()

    MYIP_URL = 'http://freegeoip.net/xml/'
    MYIP_HEADERS = {'Accept': 'application/xml'}

    if args.ip is not None:
        MYIP_URL = MYIP_URL + args.ip

    rspn = requests.get(MYIP_URL, headers=MYIP_HEADERS)
    tree = etree.ElementTree(etree.fromstring(rspn.text))
    root = tree.getroot()

    ip = root.find('Ip')
    cntr = root.find('CountryName')
    reg = root.find('RegionName')
    city = root.find('City')

    print('Your Current IP: %s' % ip.text)
    print('You appear to be in %s, %s, %s' % (city.text, reg.text, cntr.text))

Or you can get the whole code from my github profile.

Quick Ref: Python Virtualenv use

Jonas Gorauskas — Fri, 06 Jun 2014 07:55:33 GMT

Supposing you have a clean install of a Debian-based distro such as Debian 7 proper or Xubuntu 14.04, this notes will walk you through setting up a Python development environment where you can leverage the power of Virtualenv through Virtualenvwrapper.

Why virtualenv?

When you code in python you will quickly find out that different projects have different requirements for the python version you need to use and also the versions of libraries you need to use. Virtualenv will allow you to compartmentalize the requirements on a per project basis without making you install libraries and python versions globally or system-wide.

Ok, ok... Here are the goods...

You have your new shinny install of Debian which comes with a fairly recent version of python pre-installed (somewhere above 2.7.3). Now you need to install a package manager for python and the one I use is pip.

sudo apt-get install python-pip

Once you have pip installed, you will need to also install virtualenv, like so:

sudo pip install virtualenvwrapper

Virtualenvwrapper is a very nice layer on top of virtualenv that makes working with it much more intuitive and convenient.

Next, you may want to automate a couple of things like I did. First, create a ~/.virtualenvs directory. This is where virtualenvwrapper will keep track of your environments. Then open you .bashrc file and add a few lines to it:

export WORKON_HOME=$HOME/.virtualenvs
if [ -f /usr/local/bin/virtualenvwrapper_lazy.sh ]; then
    . /usr/local/bin/virtualenvwrapper_lazy.sh
fi

Now you are ready to start working, so open a new terminal, setup a virtual environment, and start cranking out some python code. Here's a quick reference for virtualenvwrapper:

Start new virtual environment?

mkvirtualenv [env-name]

Leave the current virtual environment?

deactivate

List all available virtual environments?

workon

lsvirtualenv -b

Enter an existing virtual environment?

workon [env-name]

List packages installed in current virtual environment?

pip freeze

Remove a virtual environment?

rmvirtualenv [env-name]

Start a virtual environment with specific python version?

mkvirtualenv [env-name] -p /usr/bin/python3.4

Replace [env-name] above with your actual environment name and enjoy!

What are the genuinely useful ideas in programming?

Jonas Gorauskas — Sun, 27 Oct 2013 05:48:21 GMT

There is a really interesting article here about "What are the genuinely useful ideas in programming".

The comments are closed for that post, so here are the two things I would add to the list:

Index = Data(Algorithm + Data Structure): Google made this concept relevant, again.
Sensors, and networks of them (not to be confused with scanners (e.g. barcode, QR)): The Arduino and the Raspberry Pi made these important.

Resume interrupted downloads with cURL in Linux

Jonas Gorauskas — Mon, 14 Oct 2013 04:59:23 GMT

I have recently been trying to download an ISO of a certain Linux distro, but I find myself at the wrong end of a crappy connection. I tried downloading it with both Firefox and Chrome but both choked very early in the process and current speed would go down to 0 and after a while the connection would just cutoff with no way to resume the download from where I left off.

Fear not because [cURL][curl] comes to the rescue. After typing Up-Arrow and Enter more time than I care to admit, I decided to let the computer do the hard work and automate this thing. So I created a small Bash script called acurl.sh (for Automated cURL) that looks like this:

export ec=1;

while [ $ec -gt 0 ]; do
    /usr/bin/curl -O -C - $1;
    export ec=$?;
done

if [ $ec -eq 0 ]; then
    echo "Downloaded $1";
fi

What's going on above is that I have a while loop that is checking the exit code of curl. This exit code is captured by assigning the value of $? to the variable $ec. If curl exit code is larger than 0 then continue trying to download the file, which is represented by parameter $1, from where we left off, which is accomplished by passing the parameter -C to curl.

Assuming that the script above is located somewhere in the $PATH, then using it is as simple as this:

acurl.sh http://distrodomain.org/downloads/linux-distro-version-platform.iso

This is definitely a brute force approach, but it does the work.

[curl]: http://curl.haxx.se/ "cURL"

How to migrate a git repository from GitHub to Gitorious

Jonas Gorauskas — Thu, 10 Oct 2013 09:16:11 GMT

The other day I had to move a git repository from GitHub to Gitorious and after cracking my head against the wall about this problem for a little while, I think I came up with a pretty elegant solution.

Here are some of the assumptions we are making in this turtorial:

You already have a GitHub and a Gitorious account
You have an existing GitHub repo that you want to move
We will use this repo as the example

Now follow these steps:

Login to Gitorious and create a new Project with a Repo
Find a location on your local system and create a new folder called XmlTidy. I usually do this at ~/Projects
Drop into your new folder and initialize a new git repo in it
Add a remote reference to your GitHub repository called origin
Pull all code from the master branch in origin
Add a remote reference to your Gitorious repository called destination
Push your local master branch to the destination remote repo

Here's the complete code listing for the above narrative:

mkdir ~/Projects/XmlTidy
cd ~/Projects/XmlTidy
git init .
git remote add origin git@github.com:gorauskas/XmlTidy.git
git pull origin master
git remote add destination git@gitorious.org:xmltidy/xmltidy.git
git push destination master

That's it... You're done!

Useful Git Aliases

Jonas Gorauskas — Fri, 27 Sep 2013 07:32:30 GMT

Here are some useful aliases for Git:

git ls

git log is pretty useless. It doesn't list the branch the commit was made on, it doesn't show colors and it doesn't show branching visuals. But with this alias you get all that. List commits in short form, with colors and branch/tag annotations and visuals.

git config --global alias.ls "log --color --graph --pretty=format:
  '%Cred%h%Creset -%C(yellow)%d%Creset %s %Cgreen(%cr) %C(bold blue)
  %Creset' --abbrev-commit --decorate"

git ll

List commits as above but also showing changed files.

git config --global alias.ll "log --color --graph --pretty=format:
  '%Cred%h%Creset -%C(yellow)%d%Creset %s %Cgreen(%cr) %C(bold blue)
  %Creset' --abbrev-commit --decorate --numstat"

git lds

List one line commits that display dates.

git config --global alias.lds "log --color --graph --pretty=format:
  '%Cred%h% %ad%Creset -%C(yellow)%d%Creset %s %Cgreen(%cr) %C(bold blue)
  %Creset' --abbrev-commit --decorate --date=short"

git grep

Find a file path in codebase which contains a string or pattern.

git config --global alias.grep '!git ls-files | grep -i'

git ltg

Show the last tag.

git config --global alias.ltg 'describe --tags --abbrev=0'

git unpushed

Sometimes I wonder what is left to push to origin. Git doesn't have an easy way to see this, but with this alias you get the commits that haven't been pushed to upstream. Also you get colors and branching visuals like above.

git config --global alias.unpushed "log --branches --not --remotes
  --color --graph --pretty=format:'%Cred%h%Creset -%C(yellow)%d%Creset
  %s %Cgreen(%cr) %C(bold blue)%Creset' --abbrev-commit"

git undo

Git doesn't have an undo option to undo the last commit. But with this alias you can easily undo the last commit.

git config --global alias.undo 'reset --hard HEAD~1'

However if you've added many files in the last commit (like a package or something), git doesn't delete them. But you can do that with this alias: git clean -f -d

git visual

Run a git graphical user interface on top of your git directory.

git config --global alias.visual '!gitk'

git la

Going meta! List out all your aliases.

git config --global alias.la '!git config -l | grep alias | cut -c 7-'

Basic stuff

Of course I use a ton of basic shortcuts, here’s a few ingrained in my fingertips:

cp = cherry-pick
st = status -s
cl = clone
ci = commit
co = checkout
br = branch
diff = diff --word-diff
dc = diff --cached

Enjoy!

Check if a bit is set and bit-masking in CSharp

Jonas Gorauskas — Thu, 15 Aug 2013 02:37:41 GMT

Today I was talking to a co-worker and the conversation went into the subject of bit masking. So I wrote the quick command line tool below to explain how bit masking works and how to find out if a bit is set or not on a value.

using System;

namespace bittest {
    class Program {
        public static void Main(string[] args) {
            while (true) {
                WL(""); W("Enter a number [0-255]: ");

                int i = Int32.Parse(RL());
                string sbin = Dec2Bin(i);
                byte b = (byte)i;

                WL(""); WL(SF("{0} in binary is {1}. It has {2} bit(s)",
                              new object[] {i, sbin, sbin.Length}));
                WL(""); W("What bit do you want to check: ");

                int pos = Int32.Parse(RL());

                WL(""); WL(SF("Bit {0} is {1}set!",
                              new object[] {pos, IsBitSet(b, pos) ? "" : "not "}));
                WL(""); W("Continue [Y/n]: ");

                if (RL() == "n")
                    break;
            }
        }

        static Func&lt;int, string&gt; Dec2Bin =
            value =&gt; Convert.ToString(value, 2);
        static Func&lt;byte, int, bool&gt; IsBitSet =
            (b, pos) =&gt; (b &amp; (1 &lt;&lt; pos)) != 0;

        static Action&lt;object&gt; W = o =&gt; Console.Write(o);
        static Action&lt;object&gt; WL = o =&gt; Console.WriteLine(o);
        static Func&lt;string&gt; RL = () =&gt; Console.ReadLine();
        static Func&lt;string, object[], string&gt; SF =
            (frmt, obj) =&gt; String.Format(frmt, obj);
    }
}

Enjoy!

Some Notes on Using Git with Cygwin on Windows

Jonas Gorauskas — Fri, 24 May 2013 04:55:30 GMT

Just recently I rebuilt a system that I use for work. After I installed Cygwin on it, I started getting some weird errors when using git with ssh to do some source control tasks.

When doing a fetch, pull or push to/from origin, or really any git command that uses ssh to communicate, the ssh client would complain with the error:

Could not create directory '/home/jgg/.ssh'.
The authenticity of host 'github.com (204.232.175.90)' can't be established.
RSA key fingerprint is 16:27:ac:a5:76:28:2d:36:63:1b:56:4d:eb:df:a6:48.
Are you sure you want to continue connecting (yes/no)?

Once I enter yes, I get the following:

Failed to add the host to the list of known hosts (/home/ME/.ssh/known_hosts).

This was happening everytime I was using git under cygwin and very quickly the situation became completely unsustainable. But after doing a little research, I found a solution that is pretty simple:

Locate the passwd file, which is usually at C:\cygwin\etc\
Open it with your prefered editor
On the line that starts with your username, change the section that reads /home/ME to read /cygdrive/c/path/to/home/folder/ME or wherever your home folder is.
Save the changes and restart the Cygwin Terminal

Expect to see the authenticity check one final time and see the ssh client actually be able to write to the known_hosts file.

Build Emacs from Source in Ubuntu 13.04

Jonas Gorauskas — Mon, 06 May 2013 22:28:28 GMT

Over the years I have written several of these articles and I find that every 2 to 3 years I need to re-write these notes because the installation process diverges enough from the previous set of steps that it warrants a re-write.

The main Linux distribution that I have settled on lately is Xubuntu, which is a XFCE-based version of Ubuntu. I dabble in other distros also and I am experimenting a lot lately with Arch and other Debian-based distros. One thing is clear, the versions of GNU Emacs available with each distribution are different. So, as a standard I will always try to download the latest stable GNU Emacs sources and build it on the system I am using at the moment. This ensures that I have a homogeneous set of features across the different machines and also that I am pretty close to the bleeding edge.

In order to get to this state with your Emacs install you can follow the below steps. These steps are focused on a Debian-based distro using APT. I will assume that you have a base Debian or Ubuntu system already installed with a common GUI environment. Again, I wrote these notes for a Xubuntu system running XFCE. Let's start by downloading and installing the tools needed to build Emacs:

$ sudo apt-get install build-essential git git-core curl
$ sudo apt-get install automake autogen autoconf ssh

Next, you need to get the sources for Emacs. There is two ways to go about doing this: 1) Clone the sources from an Emacs repository directly or 2) Download the latest compressed source tarball from a FSF/GNU mirror site.

To clone the Emacs source from a repo, do the following:

$ cd ~/Downloads
$ git clone git://git.savannah.gnu.org/emacs.git

Cloning will take several minutes, because git will get all the history and changes and we all know that Emacs has a long history. The clone method is the way to go for those who will hack on emacs itself. The easier way to do this, however, and the way that I prefer is to pull the source tarball from a mirror. A list of available mirrors can be found at the FSF site. You can pick any of these sites and download from it. The command to do that will differ based on the mirror you pick, but should look something like this:

$ curl -O http://mirror.sdunix.com/gnu/emacs/emacs-24.3.tar.gz

While you are downloading the tarball or cloning the repo you should open another terminal window and start installing some needed Emacs dependencies in parallel:

$ sudo apt-get install texinfo libncurses5-dev libgtk2.0-dev libgif-dev
$ sudo apt-get install libjpeg-dev libpng-dev libxpm-dev libtiff4-dev
$ sudo apt-get install libxml2-dev librsvg2-dev libotf-dev libm17n-dev
$ sudo apt-get install libgpm-dev libgnutls-dev libgconf2-dev libdbus-1-dev

Once the above deps are installed and you finished retrieving the sources for Emacs, it's time to do the actual build. Decompress the source if you downloaded it and change directory into the Emacs source:

$ tar -zxvf emacs-24.3.tar.gz
$ cd ~/Downloads/emacs-24.3/

Configure the build according to what you want your installation to look like:

$ sudo ./autogen.sh
$ ./configure

I use the vanilla configuration, but you can change it to what suits you best and you can view all alternatives by using ./configure --help ... If you don't like what configure has done for you (read the configure output) you can also clean things up and start over with make distclean.

The key thing that I am trying to achieve is a message from configure that looks very similar to the below output. Note all the yes values next to all the stuff that configure checks for; this is a direct outcome of installing all the dependency packages above. Emacs absolutely needs texinfo, ncurses, an X toolkit and the image libraries to even build. The rest of the stuff is nice to have, but if you don't install it you will get a no from configure for that feature check.

... a ton of output

Configured for `x86_64-unknown-linux-gnu'.

  Where should the build process find the source code?    ~/Downloads/emacs-24.3
  What compiler should emacs be built with?               gcc -std=gnu99 -g3 -O2
  Should Emacs use the GNU version of malloc?             yes
      (Using Doug Lea's new malloc from the GNU C Library.)
  Should Emacs use a relocating allocator for buffers?    no
  Should Emacs use mmap(2) for buffer allocation?         no
  What window system should Emacs use?                    x11
  What toolkit should Emacs use?                          GTK2
  Where do we find X Windows header files?                Standard dirs
  Where do we find X Windows libraries?                   Standard dirs
  Does Emacs use -lXaw3d?                                 no
  Does Emacs use -lXpm?                                   yes
  Does Emacs use -ljpeg?                                  yes
  Does Emacs use -ltiff?                                  yes
  Does Emacs use a gif library?                           yes -lgif
  Does Emacs use -lpng?                                   yes
  Does Emacs use -lrsvg-2?                                yes
  Does Emacs use imagemagick?                             no
  Does Emacs use -lgpm?                                   yes
  Does Emacs use -ldbus?                                  yes
  Does Emacs use -lgconf?                                 yes
  Does Emacs use GSettings?                               yes
  Does Emacs use -lselinux?                               no
  Does Emacs use -lgnutls?                                yes
  Does Emacs use -lxml2?                                  yes
  Does Emacs use -lfreetype?                              yes
  Does Emacs use -lm17n-flt?                              yes
  Does Emacs use -lotf?                                   yes
  Does Emacs use -lxft?                                   yes
  Does Emacs use toolkit scroll bars?                     yes

... more output

Now you can go into the actual build of Emacs by issuing the following command:

$ make

And then last step is to install the Emacs you just finished building:

$ sudo make install

... and run it:

$ emacs &amp;

Enjoy hacking with Emacs!

P. S. One last step that is personal and relevant to me only is to install my Emacs configuration. Execute the following to accomplish that:

$ rm -r .emacs.d
$ git clone git@github.com:gorauskas/.emacs.d.git

You will of course need to have the correct encryption key in order to clone the repo like the line above.

Convert an Integer to a Binary Representation

Jonas Gorauskas — Mon, 15 Apr 2013 21:17:49 GMT

Here is a quick way to convert any integer n into a string that represents the n integer in binary notation:

def dec_to_bin(n, bits=32):
    return ''.join([str((n &gt;&gt; y) &amp; 1) for y in range(bits - 1, -1, -1)])

What's going on in the above code? This can be better explained if we read the return line in the dec_to_bin function backwards. First we create a list of integers from 31 to 0 in order to iterate through it. This list is created with the built-in range function and it uses the number of bits that we passed into the function, with a default being 32. Next, we iterate through the list with each number being represented by y. Then we take the number parameter n, which is the number we want to transform to binary, and we shift its bits to the right by y places. Then we apply a bitwise and operation on the result of that right shift and the literal value of 1. This and operation tells us if the bit at location y is turned on or off. We then make a string out of the value and append it to the string that we are building to return to the user. This is a perfect example of Python's conciseness and the power of list comprehensions.

You can use the code above like so:

>>> print dec_to_bin(255)
00000000000000000000000011111111

Enjoy!

Get the latest version from a Git repository

Jonas Gorauskas — Sun, 02 Sep 2012 05:29:46 GMT

If you want to get the latest version from a Git repository and you don't care about any local changes, do the following:

git reset --hard HEAD
git clean -f
git pull

The above will copy the latest code from the repo and overwrite the local copy.

How to use Emacs key bindings in Visual Studio 2010

Jonas Gorauskas — Sun, 08 Jul 2012 04:33:04 GMT

Visual Studio 2008 used to have a Emacs key mapping scheme that you could pick from the Options dialog under the Keyboard options. Visual Studio 2010 did away with that key mapping scheme.

But those emacs enthusiasts among us who use Visual Studio should not fear... There is now an Emacs Emulation extension available for VS2010 that you can install via the Extensions Manager under the Tools menu.

Installation

To install the Emacs Emulation extension, follow these steps:

Open Visual Studio 2010
Go to the Tools menu
Select the Extension Manager
On the Extension Manager dialog, select Online Gallery
Search the Online Gallery for Emacs (top right)
Select and install the Emacs Emulation package
Restart Visual Studio 2010
From the Options dialog, the Emacs key mapping scheme is now available

Beyond the default Emacs key bindings, the ones that I find most useful are the following:

Edit.EmacsExtendedCommand: ALT + X - Places the cursor in the Find/Command box on the Standard toolbar.
Edit.EmacsFindReplace: SHIFT + ALT + 5 - Displays the replace options in the Quick tab of the Find and Replace dialog box.
Edit.EmacsSwapPointAndMark: CTRL + X, CTRL + X - Moves the cursor to the current mark in the location stack and moves the current mark to the location where the cursor mark was when the command was invoked.
Edit.EmacsCloseOtherWindow: CTRL + X, 1 - When a window is split, this shortcut closes the pane that does not have focus.
Edit.EmacsSplitVertical: CTRL + X, 2 - Splits the current document in half vertically. The current line of code is centered in each window.

... and that's all, folks!

Extracting a .NET Assembly from SQL Server 2005

Jonas Gorauskas — Fri, 05 Nov 2010 04:16:26 GMT

I just answered a great question on Stackoverflow: Extracting a .NET Assembly from SQL Server 2005

Yes, this is possible. The actual binary representation of the assemblies live in the SQL catalog for your server. Namely, if you run a join between sys.assembly_files and sys.assemblies you can get to all the information you need. The assemblies binary is in the content column of the sys.assembly_files view.

But in order to extract the binary representation from SQL Server and into a file on disk you will have to write some .NET code that needs to run on the same database where the assemblies you refer to are located now. In Visual Studio start a SQL CLR project and add a class to it with the following code:

using System;
using System.IO;
using System.Data;
using System.Data.SqlClient;
using System.Data.SqlTypes;
using Microsoft.SqlServer.Server;
using System.Security.Permissions;

namespace ExtractSqlAssembly {
    [PermissionSet(SecurityAction.Demand, Unrestricted = true, Name = "FullTrust")]
    public partial class SaveSqlAssembly {

        [SqlProcedure]
        public static void SaveAssembly(string assemblyName, string path) {
            string sql = @"SELECT AF.content FROM sys.assembly_files AF JOIN sys.assemblies A ON AF.assembly_id = A.assembly_id where AF.file_id = 1 AND A.name = @assemblyname";
            using (SqlConnection conn = new SqlConnection("context connection=true")) {
                using (SqlCommand cmd = new SqlCommand(sql, conn)) {
                    SqlParameter param = new SqlParameter("@assemblyname", SqlDbType.VarChar);
                    param.Value = assemblyName;
                    cmd.Parameters.Add(param);

                    cmd.Connection.Open();  // Read in the assembly byte stream
                    SqlDataReader reader = cmd.ExecuteReader();
                    reader.Read();
                    SqlBytes bytes = reader.GetSqlBytes(0);

                    // write the byte stream out to disk
                    FileStream bytestream = new FileStream(path, FileMode.CreateNew);
                    bytestream.Write(bytes.Value, 0, (int)bytes.Length);
                    bytestream.Close();
                }
            }
        }
    }
}

Then build the project and deploy it to your database. Make sure that the CLR Enabled configuration option is enabled on the SQL Server. This is probably already enabled, since you have assemblies on it. In case clr execution is not enabled you can run the following code on SSMS to enable it:

sp_configure 'clr enabled', 1
go

reconfigure
go

One more thing that you need to be aware of is the by default SQL server may not allow you to write to the disk from the .NET code. If you get a FileIO security error when you run the code above by calling the stored procedure in SSMS, you will need to configure the proper permission set for the assembly. You can do this via SSMS: right-click the new assembly and look at the Permission Set in the Properties dialog. Set it to External Access. Now you should be able to export your assemblies by running the following code in SSMS:

exec SaveAssembly 'AssemblyName', 'f:\path\to\assemblyname.dll'

Testing Regular Expressions with Regex Builder

Jonas Gorauskas — Sat, 23 Oct 2010 02:48:00 GMT

I have been working on a recent project where I need to make heavy use ofregular expressions and I have been spending a lot of time tweaking theseexpressions. Well, the tweaking is a lot easier if you can use a visual tool totest the expressions. There are lots of such tools out there, and I provide ashort list below, but I decided to hack together my own tool just to learn howthese things work under the hood.

This is a very simple tool written in CSharp to visually highlight regularexpression matches. The source is available on github and it's called RegexBuilder (I know... very original name).

Here's how it works... There are 2 text boxes, one for the regular expressionitself and another for the test string and a couple of check boxes for theoptions. I run the regular expression provided with options against the teststring and I receive back a collection of matches. I also have a Style classthat encapsulates two Colors for background and foreground. I then loopthrought the match collection and I apply the alternating Style to each Matchin the collection. The highlighting is made possible by the properties of the rich text box. This is the core of the code:

string regexPattern = textRegex.Text;
string inputText = rtbTest.Text;
int i = 0;
RegexOptions ro = new RegexOptions();

if (!checkCase.Checked)
    ro = ro | RegexOptions.IgnoreCase;

if (checkMultiLine.Checked)
    ro = ro | RegexOptions.Multiline;

regex = new Regex(regexPattern, ro);
matches = regex.Matches(inputText);

foreach (Match m in matches) {
    Style s = getStyle(i);
    rtbTest.SelectionStart = m.Index;
    rtbTest.SelectionLength = m.Length;
    rtbTest.SelectionColor = s.ForegroundColor;
    rtbTest.SelectionBackColor = s.BackgroundColor;
    rtbTest.Select(m.Index + m.Length, 0);

    i++;
}

Regex Builder was a simple mental exercise around regular expressions for meand it's the minimum thing that actually works. I have no plans to develop itfurther. There are other tools that fall in this Regex Tester/Debugger category and are able to accomplish much more than my humble Regex Builder:

Regulazy
Regulator
The Regex Coach
RegexDesigner.NET
RegexPal - this is an online tool that does basically the same as my Regex Builder

Delicious to Blogger Python Script

Jonas Gorauskas — Sat, 09 Oct 2010 21:19:00 GMT

Readers of this blog (that means my wife and my mom) will notice that there is a new post with a weird title right before this one (in chronological order). What's going on with that?

On a weekly basis I save several links to my Delicious account and I consider that to be a pretty good representation of what is going on in mind at the time. The only problem is that sometimes I link to something and then it gets lost in the black hole that Delicious can sometimes be. There are situations when I saved a link and didn't go back to look for it until a year later.

So I thought that I wanted to group these links together and bring them up front and center in my radar. I just want to have a little better visibility into what I am thinking at the time. This is a task that can be easily automated and I wrote a script in Python to do that for me. You can look at the code here.

Basically, the script looks at the links posted to Delicious since a certain date, downloads the information about those links and creates a blog post for my Blogger account based on the Delicious links.

Tidy up your XML files

Jonas Gorauskas — Sat, 18 Sep 2010 21:30:00 GMT

I have been working with some really large XML files (20 MB) that I have to parse and save the data to a database. The XML data is available to me as a single line of XML (probably to save up space) which is quite unreadable by a human. There are many tools out there that will help you tidy up and reformat your XML to a human-readable format. But writing my own tool using C# and LINQ to XML is so trivial that I decided to do my own. Here's the code:

using System;
using System.IO;
using System.Xml;
using System.Linq;
using System.Xml.Linq;

namespace XmlTidy {
    class Program {
        static void Main(string[] args) {
            try {
                XmlTidy(args[0]);
            } catch (FileNotFoundException ex) {
                Console.WriteLine("{0}: {1}", ex.Message, ex.FileName);
            }
        }

        static void XmlTidy(string filename) {
            if (File.Exists(filename)) {
                var doc = XDocument.Load(filename);
                doc.Save(filename, SaveOptions.None);
            } else {
                throw new FileNotFoundException("File not found", filename);
            }
        }
    }
}

What's going on in the code above? When you call the program from the command line, it expects a file name as a parameter, like so: xmltidy samplefile.xml. Next is the call to the XmlTidy function, which checks if the file exists and then loads the file contents into a LINQ to XML document, and then we just save the file again to the same location. That's it! The XDocument class takes care of formating the XML data for you.

You can download the full source, which has some enhancements, from my Github project. This code works on Windows 7 with .NET 4.0 and also on Ubuntu 10.04 with Mono 2.6.7 .