Well, that little thingy is called a favicon in webspeak. And this is how you add it to your blog.
First, create a icon using a program such as the freeware version of IconArt. Upload this file to the web.
Then, edit your blog template in html-mode, and add the following lines of code to the head of your template, just below the <head>-tag for example.
<link href='youriconfile.ico' rel='shortcut icon'/> <link href='youriconfile.ico' rel='icon'/>
If you have already a link in your favorites-bar to your blog, in IE you will probably NOT see the icon right after adding it to your blog. You will have to delete the link from your favorites, empty your browser's cache, close your browser, start it again and add the link again to make it show up in IE.