Simple extraction of content from HTTP request

When writing a script for extraction of data exported to XML from burp, I was forced to make one very dirty hack on extracting body of HTTP request. It seems that there is no easy way of getting just the data, especially if the message transferred is binary file, i.e. image or executable. If it is simply text, it might probably done with some perl regexp wizardry. However it would be nice either way. The hack I mentioned was creating netcat server and piping whole request into it and then downloading it with help of wget. That way it works and takes only two straightforward lines of code, but it is still a hack. You can see it here.

To not be forced to do hacks like that in future I had a plan to make very simple utility program that will just delete HTTP header and print rest of input to stdout. Well, I decided to do this just after finishing the mentioned script, so about half year ago. Now I have found some time to do this and the result is very small utility written in C that does just that and by the way is able to parse the header itself and provides a way to be used easily by the user. The latter is done by spawning child process of default shell with header values set in its environment, so it may not be the nicest way to do this, but for it was the fastest to implement and should be fairly easy to work with. If you know better way that will result in similar behavior I will be graceful if you leave me a comment here.

Posted in Uncategorized | Tagged , , | Leave a comment

Method for intercepting (lot of) files from website using burp

Burp exporting requests to file

Burp exporting requests to file

From time to time everyone has a need to download bunch of files from some website. Sometimes there exists one index where links to every file can be found. But sometimes not. Analysis of a website and/or figuring out a way the link is created (especially if it is something like http://some-cdn.io/directory/file-generator?param=5acae975-7784-e511-9412-b8ca3a5db7a1&ws=b0c491db-ae8b-e011-969d-0030487d8897&uid=66dba4ec-bb13-e211-a76f-f04da23e67f6&switch=1) could take months and success is not guaranteed. If this happens, downloading files manually is the only way to do it. But manual download can be optimized too.

Burp way

Burp is a sort of swiss army knife of penetration tester. Its main function is intercepting HTTP(s) traffic through built in proxy. It allows to decrypt traffic of any website or even Android app to third-party server. So this way you could configure web browser to use burp proxy and simply view every file you want to save. This would be ideal solution for backing up any image gallery including facebook galleries that can be viewed only by logged in users and uses weird links to facebook CDNs. But there is one problem. Burp does not allow to export many files at once (or at least its free license does not allow it). Or to be exact uses its own format to store both HTTP request, response and lot of metadata, we simply do not need, but we need to have directory filled with images, right?

Solution

To obey the problem, I have written simple bash script that extracts plain data from that exported data file. It gets XML file exported by Burp and unpacks plain responses, each to separate file. Usage is very simple.

  1. At first export files you want to save from Burp’s Target tab by selecting them, clicking on Save selected items and save file as whatever.xml.
  2. Then you just have to start the script with
    ./xtract-burp.sh whatever.xml

    and optionally appending desired file extension as second parameter.

Note that files will be named with iterator starting on 1 and going up and sorted the same way Burp had them exported.

As usual repo is available on github.

Posted in Tutorials | Tagged , , , | Leave a comment

HDCB – new way of analysing binary files under Linux

As any observer of my projects spotted, most of the biggest projects I do involves binary file analysis. Currently I am working on another one that requires such analysis.
Unfortunately such analysis is not an easy task and anything that will ease this or speed it up is appreciated. Of course there are some tools that will make it a bit easier. One of them is hexdump. Even IDA Pro can make it easier a bit. Despite them I always felt that something is missing here. When creating xSDM and delz utils, I was using hexdump output with LibreOffice document to mark different structure members with different colors. It worked, but selecting 100-byte buffer line by line was just wasting precious time.

SDC file analyzed by HDCB script

Solution

So why not automate whole process? What we really need here is just hexdump output and terminal emulator with color support. And that’s why I’ve made HDCB – HexDump Coloring Book. Basically it is just extension to bash scripting language. Goal was to create simple script that will hide as much of its internals from end-user and allow user to just start it will his shell using old good ./scriptname.ext and that’s it. HDCB is masked as if it was standalone scripting language. It uses shebang, known from bash or python scripts to let user shell know what interpreter to use (#!/usr/bin/env hdcb). Those who are python programmers should recognize usage of env binary.

In fact it is just simple extension to bash language, so users are still able to utilize any features available in bash. Main extensions are two commands: one (define) for defining variables and the other (use) for defining field or array of that defined type. Such scripts should be started with just one argument – file that is meant to be hexdumped and analyzed.

Internals

Bash scripts are just some kind of a cover of real program. Main core of the program is colour utility. It gets unlimited number of parameters grouped in groups of four. They are in order: offset of byte being colored, length of the field, background and foreground colors. As standard input, hexdump output (in fact only hexdump -C or hexdump -Cv are supported) is provided. Program colors the hexdump with rules provided as arguments. This architecture allows clever hacker to build that cover mentioned in virtually any programming language.

Downloads, documentation and more

As usual, program is available on my Github profile. Sources are provided on GPLv3 license so you are free to contribute to the project and you are strongly encouraged to do so or make proposals of a new functions. Program is meant to be expanded according to my future needs, but I will try to implement any good idea. Whole documentation, installation instructions and the like are also available on Github.

Posted in Uncategorized | Tagged , , | Leave a comment

Airlive WN-151ARM UART pinout & root access

airlive-pinout

Airlive WN-151ARM pinout

For curious ones. Here is pinout of serial connection. As you can see UART pins are at J4 header (should have pin 4 labeled and 1 be square).

J4 header
Num. Function
1 VCC
2 RX
3 TX
4 GND

Edit: Oh, and one more thing: goldpin header, you see in the picture is soldered by me, so do not be surprised if you have to hold wires all the time during the transmission.

Root access

There is also possibility to gain root access without removing the cover and possibly voiding the warranty. You have to connect to router’s AP and enter

http://192.168.1.254/system_command.htm

into your browser (panel authentication required). Now you can execute any command you want with root privileges! So let’s type

/usr/sbin/utelnetd -d &

into Console command field and press Execute button. If everything went well, you should now be able to connect to your router using telnet at its default TCP port 23. After that you should see BusyBox banner and command prompt.

It is worth noting that this hidden console cannot be accessed by unauthorized person, so only router administrator can use this (in theory, in practice there are surely a lot of routers using default credentials and security of httpd binary is unknown).

Posted in Just knowledge | Tagged , , , , , , , | Leave a comment

SDC file format description – Errata

Last year, I published a program for Microsoft Dreamspark’s SDC file decryption. Soon after that I wrote article about SDC file format and its analysis. Now it’s time to complete the description with newest data.

This article wouldn’t be written if not the contribution of GitHub’s user @halorhhr who spotted multi-file SDC container and let me know on project’s page. Thanks!

When writing that post year ago, I had no idea what multi-file container really looks like. Any suspicions could not then be confirmed, because it seemed that these files simply where not used in the wild. A days ago situation changed. I got a working sample of multi-file container so I was able to start its analysis.

Real container format

sdc-format

SDC files with different signatures

After quick analysis, I knew that I was wrong with my suspicions. Filename length and encrypted filename strings are not part of a file description. In fact they are placed after them and filename is concatenated string of all filenames (including trailing null-byte). So to sum up filename of n-th element starts at file[n].filename_offset and ends just like any other c-style string.

Whole header structure is like on the sample header on the right. Note that all headers beside 0xb3 one has been already decrypted for readability. In real header the only unencrypted field is header size at the very beginning of the file. 0xb3 sample has unencrypted header and header size is not present in a file. However file name is encrypted in some way, I haven’t figured out as of now. Encryption method is blowfish-compat (the difference between this and blowfish is ciphertext endiannes). Filenames are encrypted once again.

After header, all other data is XORed using key from EDV string and then deflated, so before reading them, you have to inflate and XOR again. Format of data in 0xb3 version is still unknown, however analysis of compressed and file size hints that it may be stored the same way. It is important to note that depending on file signature different configuration of deflater may be needed. It is now known that files older than 0xd1 header, which appears to be newest (because only this one supports files greater than 4 GiB) need to have deflater initialized with

inflateInit2_(&stream,-15,ZLIB_VERSION,(int)sizeof(z_stream));

or equivalent.

Unknowns

This errata does not contain all information needed to support all variations of SDC files. Beside unknowns I mentioned above, there is another variation that uses 0xc4 signature and which I had no sample of. The only trace of its existence is condition in SDM code. Because of that I cannot write support for that type of file. There is also possibility of multi-file containers having 0xb5 or 0xb3 signatures existence. That type of files seems to appear only lately, but it is probable that it existed in the past. Because of having no samples of them I cannot verify that xSDM properly handles them.

So if you have sample of any of variations mentioned here, just send them to me at my email address: v3l0c1r4pt0r at gmail dot com or if you suppose it may be illegal in your country, just send me SDX link or any other hint for me how can I find them.

Other way?

Few days ago, after I started writing this post Github user @adiantek let me know in issues that there is a method to obey SDM in Dreamspark download process. To download plain, unencrypted file you just have to replace ‘dfc=1‘ to ‘sdm=0‘ in a link Dreamspark provided in SDX file. If it true that it works in every file Dreamspark provides, my xSDM project would be obsolete now. However, because Microsoft’s intentions when creating this backdoor (it seems to be created just for debugging) are unknown, I will continue to support the project and fix any future bugs I will be aware of. But now it seems that this project will start to be just proof of concept for curious hackers and will start to slowly die.

Nevertheless, if you have something that might help me or anyone who may be interested in SDC format in future, just let me know somehow, so it will be available somewhere on the internet.

Posted in Uncategorized | Tagged , , , , | Leave a comment

imeitool – validate, generate and find out information about IMEI number easily

I just pushed imeitool to my Github. imeitool is small utility that can do few useful operations on IMEI numbers. It can check if number given is valid IMEI number, find information in its databases about given IMEI or TAC (Type Allocation Code, usually first 8 digits of IMEI) and generate fake IMEI based on conditions provided by user.

The reason this program was created should appear on blog soon. There is a possibility for anyone to contribute to imeitool’s db, so if you want to help, more info can be found on project’s README file.

Posted in Uncategorized | Tagged , , | Leave a comment

Changing your MAC address on new Android devices

Few months ago I flashed my Android device with Cyanogenmod-based LegacyXperia ROM. Before that I was using anonymization script, I created that was able to change MAC address, block some Play Store’s services sending unknown binary data and masking device hostname (used by DHCP). Obviously after upgrade, the script stopped working, so I tried to make it work again.

Apparently newer Android systems changed its way of turning wifi on and off (before, MAC changing was just invoking ip command), so now wlan0 interface is not present when it is off and after turning it on device will most likely authenticate to any known network, effectively leaking its HW address. Because of that I needed to find another way.

A bit of research and compiling iw tool later, I found out that when wifi is being turned off by GUI, wifi card kernel module is removed from kernel, so card’s interface as well as its physical device disappear from system. So the first step to make it work is to insert module back (wl12xx_sdio in my case, it seems to be very popular card on Android devices so probably you have the same). Inserting the module, however generates another problem: GUI cannot manipulate wifi now, so we have to do everything manually. The next steps are to invoke wpa_supplicant to authenticate to network and starting DHCP client daemon.

Doing this that way allows us to do one additional thing, increasing our privacy. Since we have to start supplicant ourselves, we can provide custom config to it. And if we have to write script anyway, we could have separate scripts for every network we know. Then we will be (almost) sure that we are connecting to the network we want.

The connecting script is available from my new repository with Android scripts for increasing privacy. I hope there will be more of them in future. As I have written in first paragraph, there are few other things to do. Or maybe there is something more I do not know about, yet…

PS: this script was tested only on Cyanogenmod 10.2 (based on Android 4.3) and may not work on new (5.*) systems.

Posted in Uncategorized | Tagged , , , , , , | Leave a comment

Decoding Aztec code from polish vehicle registration certificate

About a year ago I interested in mysterious 2D code placed in my car’s registration certificate. After quick research on Google it turned out to be even more mysterious because nobody knew how to decode it. There was even no official document like act or regulation that describes the code somehow. People knew that the code is Aztec code and that’s it. Some companies shared web and Android apps to decode this. And all of them was sending base64 to some server and receive decoded data.

Of course for me it wasn’t rewarding so I started my research on it. After initially scanning the code I’ve seen long string that I immediately recognized as base64. The real fun started after that, because stream I’ve got after that was so strange that at first I had no idea what to do. Upon closer examination it was clear that this data is not damaged but encoded in somewhat strange way. Few days later I was almost sure that this is not encoding but rather compression, because some unique parts of stream was easily readable by human. About a month of learning about compression, looking for even most exotic decompression tools and I was left with almost nothing. I had only weak guess on how decompression parameters could be encoded. I gave up…

Polish vehicle registration certificate (source: pwpw.pl)

Polish vehicle registration certificate (source: pwpw.pl)

About a year later I tried one more time. This time I was a bit more lucky. I found a program that decodes the code. Again. But this time was different. I shut down my network connection to make sure. And it worked! So now a bit of reverse engineering and it’s done. I will skip any details because I do not want to piss off the company which created this, even though I was right and I HAD right to do this.

As usual the source code is available on my Github profile. There is also a bit more information about whole scanning/decoding process. If you like to know more technical details about the algorithm or how to decode the data, everything can be found in README file in the repo.

Posted in Uncategorized | Tagged , , | 9 Comments

Kernel module for ST7565 based displays

Today I pushed kernel module driving ST7565 based displays to my github. I wanted to write kernel module some time ago and st7565 was first thing i thought of. Unfortunately I hadn’t enough motivation and when I got some I burnt my display 🙁 so it wasn’t done. This autumn I had other chance to make it real, because I started embedded systems course on university. And there it is: I’ve got working kernel module that can handle graphic LCD.

The module itself is fairly universal. Nevertheless I don’t know any existing gLCD driver that was included to the mainline kernel so have no idea how that thing should look. Because of that I implemented it as char device. It is storing bare pixel data so there aren’t any sophisticated functions to draw rectangle or something. Its advantages are that it isn’t limiting applications the module can be used to and make any userspace handling function very easy to write. It is also worth to note that the module is providing possibility to read data from buffer which is impossible in the module itself.

Beside that basic functionality it allows to switch off backlight and change brightness using sysfs attributes. The module is also portable thanks to possibility to change CS, A0, RST and backlight pin it is using so you are not tied to the ones I used and reconfiguration does not force you to recompile but just reload the module with different parameters. The exact scheme of the connection more or less is the same as on my previous approach to playing with that display.

PS: there is also new revision of my tool for font-making for that display. Now it is able to create any picture as long as its height is divisible by 8 (it is just simplification, made because of method of storing pixels). You can always implement algorithm that is OR-ing the picture with part of data that is already on display (of course using my module 🙂 ). It is still available on Gist.

Posted in Uncategorized | Tagged , , , , , , | Leave a comment

Netstat & Co. for Android and Big Brother Google story

Yesterday I pushed net-tools optimized for Android to my github. Main goal was to provide full-featured netstat for Android devices. By the way I succeeded to compile arp, ifconfig, rarp and route utilities. Feel free to clone.

Additionally while playing with netstat compiled that way I noticed that not all connections are listed. It is interesting because the connection I found out to be hidden is something called C2DM or its successor GCM and that connection is started just after establishing Internet connection (so probably Google knows about every wifi and 3g connection you use) and attempt to find application responsible for this failed because of this strange netstat behavior so all I know now is that it is using some kind of custom protocol to send strange, probably compressed or encrypted data straight way to uncle Google.

Due to the fact that it uses mtalk.google.com domain I suspected Talk app for this but uninstalling it didn’t help. The more successful approach was to add this domain to /etc/hosts and the other option is to add iptables rule to block this traffic. It is fairly easy because it is using non-standard port (5228) so we can just do

iptables -A OUTPUT -p tcp --dport 5228 -j DROP

Disadvantage of the second method is that it has to be done after each system reboot. There will also be few FIXMEs from bionic libc but nevertheless it will work.

BTW: If you are interested in looking at that traffic it can be hijacked using tcpdump on your Android device.

Posted in Uncategorized | Tagged , , , , , | Leave a comment
« Older