Archives for : Internet

    Analyzing Apache Log Files

    The server access log records all requests processed by the server. The location and content of the access log are controlled by the CustomLog directive. Of course, storing the information in the access log is only the start of log management. The next step is to analyse this information to produce useful statistics.
    The principal use of awk is to break up each line of a file into ‘fields’ or ‘columns’ using a pre-defined separator. Because each line of the log file is based on the standard format we can do many things quite easily.
    Using the default separator which is any white-space (spaces or tabs) we get the following:

    awk ‘{print $1}’ access.log # ip address (%h)
    awk ‘{print $2}’ access.log # RFC 1413 identity (%l)
    awk ‘{print $3}’ access.log # userid (%u)
    awk ‘{print $4,5}’ access.log # date/time (%t)
    awk ‘{print $9}’ access.log # status code (%>s)
    awk ‘{print $10}’ access.log # size (%b)
    awk -F\” ‘{print $2}’ access.log # request line (%r)
    awk -F\” ‘{print $4}’ access.log # referer
    awk -F\” ‘{print $6}’ access.log # user agent

    Now that you understand the basics of breaking up the log file and identifying different elements, we can move on to more practical examples. But before we do that, we should explain how you can modify your log format and quickly extend capabilities of these simple examples.
    The format argument to the LogFormat and CustomLog directives is a string. This string is used to log each request to the log file. It can contain literal characters copied into the log files and the C-style control characters “\n” and “\t” to represent new-lines and tabs. Literal quotes and backslashes should be escaped with backslashes.
    The characteristics of the request itself are logged by placing “%” directives in the format string, which are replaced in the log file by the values as follows:
    The percent sign

    Remote IP-address

    Local IP-address

    Size of response in bytes, excluding HTTP headers.

    Size of response in bytes, excluding HTTP headers. In CLF format, i.e. a ‘-‘ rather than a 0 when no bytes are sent.

    The contents of cookie Foobar in the request sent to the server. Only version 0 cookies are fully supported.

    The time taken to serve the request, in microseconds.

    The contents of the environment variable FOOBAR


    Remote host

    The request protocol

    The contents of Foobar: header line(s) in the request sent to the server. Changes made by other modules (e.g. mod_headers) affect this. If you’re interested in what the request header was prior to when most modules would have modified it, use mod_setenvif to copy the header into an internal environment variable and log that value with the %{VARNAME}e described above.

    Number of keepalive requests handled on this connection. Interesting if KeepAlive is being used, so that, for example, a ‘1’ means the first keepalive request after the initial one, ‘2’ the second, etc…; otherwise this is always 0 (indicating the initial request). Available in versions 2.2.11 and later.

    Remote logname (from identd, if supplied). This will return a dash unless mod_ident is present and IdentityCheck is set On.

    The request method

    The contents of note Foobar from another module.

    The contents of Foobar: header line(s) in the reply.

    The canonical port of the server serving the request

    The canonical port of the server serving the request or the server’s actual port or the client’s actual port. Valid formats are canonical, local, or remote.

    The process ID of the child that serviced the request.

    The process ID or thread id of the child that serviced the request. Valid formats are pid, tid, and hextid. hextid requires APR 1.2.0 or higher.

    The query string (prepended with a ? if a query string exists, otherwise an empty string)

    First line of request

    The handler generating the response (if any).

    Status. For requests that got internally redirected, this is the status of the *original* request — %>s for the last.

    Time the request was received (standard english format)

    The time, in the form given by format, which should be in an extended strftime(3) format (potentially localized). If the format starts with begin: (default) the time is taken at the beginning of the request processing. If it starts with end: it is the time when the log entry gets written, close to the end of the request processing. In addition to the formats supported by strftime(3), the following format tokens are supported:
    number of seconds since the Epoch
    number of milliseconds since the Epoch
    number of microseconds since the Epoch
    millisecond fraction
    microsecond fraction

    These tokens can not be combined with each other or strftime(3) formatting in the same format string. You can use multiple %{format}t tokens instead. The extended strftime(3) tokens are available in 2.2.30 and later.

    The time taken to serve the request, in seconds.

    The time taken to serve the request, in a time unit given by UNIT. Valid units are ms for milliseconds, us for microseconds, and s for seconds. Using s gives the same result as %T without any format; using us gives the same result as %D. Combining %T with a unit is available in 2.2.30 and later.

    Remote user (from auth; may be bogus if return status (%s) is 401)

    The URL path requested, not including any query string.

    The canonical ServerName of the server serving the request.

    The server name according to the UseCanonicalName setting.

    Connection status when response is completed:

    X =
    connection aborted before the response completed.

    + =
    connection may be kept alive after the response is sent.

    – =
    connection will be closed after the response is sent.

    (This directive was %c in late versions of Apache 1.3, but this conflicted with the historical ssl %{var}c syntax.)

    Bytes received, including request and headers, cannot be zero. You need to enable mod_logio to use this.

    Bytes sent, including headers, cannot be zero. You need to enable mod_logio to use this.

    The contents of VARNAME: trailer line(s) in the request sent to the server.

    The contents of VARNAME: trailer line(s) in the response sent from the server.

    List all user agents ordered by the number of times they appear
    awk -F\” ‘{print $6}’ access.log | sort | uniq -c | sort -fr
    Identify problems with your site
    Identify problems with your site by identifying the different server responses and the requests that caused them:
    awk ‘{print $9}’ access.log | sort | uniq -c | sort
    The output shows how many of each type of request your site is getting. A ‘normal’ request results in a 200 code which means a page or file has been requested and delivered but there are many other possibilities.
    The most common responses are:
    200 – OK
    206 – Partial Content
    301 – Moved Permanently
    302 – Found
    304 – Not Modified
    401 – Unauthorised (password required)
    403 – Forbidden
    404 – Not Found

    What is Causing 404s?
    A 404 error is defined as a missing file or resource. Looking at the request URI will tell you which one it is.
    $ grep ” 404 ” access.log | cut -d ‘ ‘ -f 7 | sort | uniq -c | sort -nr
    404 Request Responses
    $ cat access.log | awk ‘($9 ~ /404/)’ | awk ‘{ print $7 }’ | sort | uniq -c | sort -rn | head -n 25
    Unique Request IP Addresses
    $ cat access.log | awk ‘{ print $1 }’ | sort | uniq -c | sort -rn | head -n 25
    Unique Request IP Addresses – Resolve country
    needs: apt-get install geoip-bin
    $ cat access.log | awk ‘{ print $1 }’ | sort | uniq -c | sort -rn | head -n 25 | awk ‘{ printf(“%5d\t%-15s\t”, $1, $2); system(“geoiplookup ” $2 ” | cut -d \\: -f2 “) }’
    Who’s ‘hotlinking’ my images?
    Something that really annoys some people is when their bandwidth is being used by their images being linked directly on other websites.
    awk -F\” ‘($2 ~ /\.(jpg|gif)/ && $4 !~ /^http:\/\/www\.n0where\.net/){print $4}’ access.log | sort | uniq -c | sort
    Blank User Agents
    A ‘blank’ user agent is typically an indication that the request is from an automated script or someone who really values their privacy. The following command will give you a list of ip addresses for those user agents so you can decide if any need to be blocked:
    awk -F\” ‘($6 ~ /^-?$/)’ access.log | awk ‘{print $1}’ | sort | uniq
    Too Much Load From One Source?
    When your site is under a heavy load, you should know whether the load is from real users or something else:
    A configuration or system problem

    A client app or bot hitting your site too fast

    A denial of service attack

    cat access.log | cut -d ‘ ‘ -f 1 | sort | uniq -c | sort -nr
    Top 10 of visiting ip’s
    cat access.log | awk ‘{ print $1 ; }’ | sort | uniq -c | sort -n -r | head -n 10
    Traffic in kilobytes per status code
    cat access.log | awk ‘ { total[$9] += $10 } END { for (x in total) { printf “Status code %3d : %9.2f Kb\n”, x, total[x]/1024 } } ‘
    Top 10 referrers
    cat access.log | awk -F\” ‘ { print $4 } ‘ | grep -v ‘-‘ | grep -v ‘http://www.adayinthelife’ | sort | uniq -c | sort -rn | head -n 10
    Top 10 user-agents
    How simple is this? The user-agent is in column 6 instead of 4 and we don’t need the grep’s, so this one needs no explanation:
    cat access.log | awk -F\” ‘ { print $6 } ‘ | sort | uniq -c | sort -rn | head -n 10
    Generates a list that shows the last 10,000 hits to a site.
    tail -10000 access.log| awk ‘{print $1}’ | sort | uniq -c |sort -n
    Requests per day
    awk ‘{print $4}’ access.log | cut -d: -f1 | uniq -c
    Requests per hour
    grep “29/Jul” access.log | cut -d[ -f2 | cut -d] -f1 | awk -F: ‘{print $2″:00″}’ | sort -n | uniq -c
    Requests per minute
    Run the following command to see requests per minute:
    grep “29/Jul/2015:06″ access.log | cut -d[ -f2 | cut -d] -f1 | awk -F: ‘{print $2”:”$3}’ | sort -nk1 -nk2 | uniq -c | awk ‘{ if ($1 > 10) print $0}’
    Total unique visitors:
    cat access.log | awk ‘{print $1}’ | sort | uniq -c | wc -l
    Unique visitors today:
    cat access.log | grep `date ‘+%e/%b/%G’` | awk ‘{print $1}’ | sort | uniq -c | wc -l
    Unique visitors this month:
    cat access.* | grep `date ‘+%b/%G’` | awk ‘{print $1}’ | sort | uniq -c | wc -l
    Unique visitors on arbitrary date:
    cat access.* | grep 28/Jul/2015 | awk ‘{print $1}’ | sort | uniq -c | wc -l
    Unique visitors for the month:
    cat access.* | grep Jun/2015 | awk ‘{print $1}’ | sort | uniq -c | wc -l
    Sorted statistics of “number of visits/requests” “visitor’s IP address”:
    cat access.log | awk ‘{print “requests from ” $1}’ | sort | uniq -c | sort
    Most Popular URL’s
    $ cat access.log | awk ‘{ print $7 }’ | sort | uniq -c | sort -rn | head -n 25
    Real-time Requests
    $ tailf access.log | awk ‘{ printf(“%-15s\t%s\t%s\t%s\n”, $1, $6, $9, $7) }’
    Real time – Resolve IP’s
    $ tailf access.log | awk ‘{ “geoiplookup ” $1 ” | cut -d \\: -f2 ” | getline geo printf(“%-15s\t%s\t%s\t%-20s\t%s\n”, $1, $6, $9, geo, $7); }’
    Unique IP addresses:
    cat access.log | awk ‘{print $1}’ | sort | uniq
    Unique IP addresses with date-time stamp:
    cat access.log | awk ‘{print $1 ” ” $4}’ | sort | uniq
    Unique IP addresses and browser:
    cat access.log | awk ‘{print $1 ” ” $12 ” ” $19}’ | sort | uniq
    Unique IP addresses and OS:
    cat access.log | awk ‘{print $1 ” ” $13}’ | sort | uniq
    Unique IP addresses, date-time and request method:
    cat access.log | awk ‘{print $1 ” ” $4 ” ” $6}’ | sort | uniq
    Unique IP addresses, date-time and request URL:
    cat access.log | awk ‘{print $1 ” ” $4 ” ” $7}’ | sort | uniq


    Honeypot Data Analysis

    I’ve recently analysed a range of Honeypot data relating to about 3 months of malicious attempts against a number of Internet exposed ports and services The data has been sourced from a typical Australian Internet connection.

    From the analysis, the traffic type and source countries were more or less as expected, although it was interesting to see the ratio of attacks received from each country.

    I thought I would share some of the initial results (see the graphics below). Over the coming months I will share more of the analysis and detail.

    General port scans and most of the spider/crawler traffic has been removed from the baseline data, so these results relate to actual malicious attempts – automated tools and manual techniques.

    China and Hungary have been seen to contribute to the highest number of Internet attack attempts.

    The top 10 countries who are attacking Australians:
    1/ China – CN
    2/ Hungary – HU
    3/ United States  – US
    4/ Canada – CA
    5/ Australia – AU
    6/ India – IN
    7/ Republic of Korea – KR
    8/ Brazil – BR
    9/ Russian Federation – RU
    10 Puerto Rico – PR



    Australian ISP 3g APN’s

    These are the main 3G APN’s (Access Point Name) used in Australia.

    ISP                                   APN                                Username/Password Required
    Telstra                             telstra.internet           NO
    BigPond                          telstra.bigpond           YES
    Three (Prepaid)            3services                       NO
    Three (Postpaid)          3netaccess                    NO
    Vodafone (Prepaid)     vfprepaymbb                NO
    Vodafone (Postpaid)                 NO
    Optus (Prepaid)            preconnect                    NO
    Optus (Postpaid)          connect                          NO
    DODO                             WirelessBroadband   NO
    dodolns1                         NO
    iburst                                NO APN                        YES User/password only
    Primus                             primuslns1                    NO
    Blink                                splns888a1                   SP Code is 888a1
    Crazy John                              NO
    Internode                        Internode                       NO
    Virgin Broadband        VirginBroadband       guest/guest


    For New Zealand user only
    ISP                                   APN                                        Dialed Number
    Telecom                      *99#
    Vodafone                     *99#
    Telstra                                *99#
    SlingShot                    *99#
    CallPlus                          *99#


    3G, HSDPA APN/Username/Password settings for many other Cellular networks…n_gateway.html…69#T-Mobile_UK

    Free content from Internode

    Australia Lights
    I’ll be using this page for the variaous media and streaming boxes we have.

    This includes the Chumby   😎

    Click to Enlarge

    ABC iView
    (except ABC News 24)
    Broadband Streaming Radio
    Games On Net online games
    TiVo Software Updates,
    TiVo Electronic Program Guide,
    CASPA on-demand TV shows & Movies
    Fetch TV early adopter trial content
    Electronic Program Guide (EPG) content,
    Video on-demand and all linear streamed
    channels for FetchTV set top box users.
    Internode File Download Mirror
    Major Geeks Mirror (+)
    SourceForge Mirror (~)
    Steam Content Servers (^)

    Interesting results from social search engines

    I went to a security presentation yesterday with one of the guys from work which summarised some of the cooler Blackhat and Defcon presentations from Las Vegas this year.

    One of these presentations demonstrated the use of the open API’s available for some of the social media sites. The below list represents some of the search engine I’ve found that allow search words to be queried against publicly available information. I’ve found that these searches far exceed what is indexed on the social media site search option.

    Search engines

    Google Buzz API Browser: explore what’s publicly visible through the Google Buzz API
    Facebook API Browser: explore what’s publicly visible through the Facebook Graph API
    Wink People Search
    Showdan HQ – Computer search engine (looks for MAC address)

    IP TV related links

    Recently I have been tweaking our ADSL 2+ Internet connection (modifying line attenuation, etc.) and have achieved some quicker sync speeds.

    To test this I have been trying to workout if there is sufficient speed to sustain IP TV. Although there is a huge reliance on upstream bottlenecks and international link latency, I can only influence what I can manage.

    So the below list represents the sites and feeds I have discovered during this testing. I hope the links continue to work for a while so others can enjoy.

    Some of the links are RTSP (Real Time Streaming Protocol) which will need a streaming media player. A  VLC is a great option but you will need to associate VLC within the browser using something like the reg hacks at the bottom of this post.

    Live streams for watching UEFA Champions League :
    Channel 1:
    History Movies
    Thriller TV
    ReelGood TV

    Western TV
    Worm TV 273K

    La Locale 441K
    KTTV Fox11? 295K
    Just TV (sound?) 141K
    Medizin TV Deutsch 491K

    Cartoon TV
    Crime TV

    Fox11 301K
    Astro-Line TV
    OLELO-53 261K
    BFMTV France 348K
    MadTV Greece 248K
    Enjoy Italy 331K
    eTV Italy 273K
    Astro-Line TV
    OLELO-53 261K
    BFMTV France 348K
    MadTV Greece 248K
    Enjoy Italy 331K
    eTV Italy 273K
    Digital15 Dom Rep 143K
    Cinquestelle Italy 220K
    Congo Planet 273K
    TV Marti US 273K
    AlfaOmegaMovies Romania 192K
    Digital15 Dom Rep 143K
    Cinquestelle Italy 220K
    Congo Planet 273K
    TV Marti US 273K
    AlfaOmegaMovies Romania 192K
    Live1 Russia 232K
    Orange Sports 744K
    Worm TV 273K
    BFM TV 348K
    AZTV 247K

    TN Todo Noticias
    Canal 13
    BBC Radio unsure of channel 1
    BBC Radio unsure of channel 2
    BBC Radio unsure of channel 3
    Live Talk Radio (Rush Limbaugh, Sean Hanitty, Mark Levin –Check Local Times–)


    Euro News French TV ENGLISH
    CNN International TV
    Live Network Tv Canada French
    Italy [3]CHANNEL ONE
    Live Network TV NBC
    Euro News in French
    Music Box Video
    NRJ Pop Rock Video
    NRJ TV Dance Video
    UK Music TV


    Thai channels
    Thai TV 7 –
    Thai TV 5 –
    Thai TV 11 –
    TITV –
    Thai TV 3 –

    MTV (“Brand New”)

    Thriller TV

    Cartoon Channel…ate=350000

    Old Cartoons”>Cartoons Channel (works with one show at the time)














    Some other lists:


    Fresh 92.7 (Adelaide)Home PageListen!
    ABC Classic FMHome PageListen!
    ABC DiG CountryHome PageListen!
    ABC DiG JazzHome PageListen!
    ABC DiGHome PageListen!
    ABC Triple JHome PageListen!
    BassdriveHome PageListen!
    ChroniX AggressionHome PageListen!
    ChroniX GritHome PageListen!
    ChroniX MetalHome PageListen!
    Club 977 – The 80’s ChannelHome PageListen!
    Club 977 – The Hitz ChannelHome PageListen!
    DI.FM AmbientHome PageListen!
    DI.FM BreaksHome PageListen!
    DI.FM ChilloutHome PageListen!
    DI.FM Chillout DreamsHome PageListen!
    DI.FM Classic EurodanceHome PageListen!
    DI.FM Classic TechnoHome PageListen!
    DI.FM Club SoundsHome PageListen!
    DI.FM DJ MixesHome PageListen!
    DI.FM Drum & BassHome PageListen!
    DI.FM ElectroHome PageListen!
    DI.FM Euro DanceHome PageListen!
    DI.FM Exposure NYCHome PageListen!
    DI.FM Funky HouseHome PageListen!
    DI.FM Future SynthpopHome PageListen!
    DI.FM GabberHome PageListen!
    DI.FM GoapsyHome PageListen!
    DI.FM HardcoreHome PageListen!
    DI.FM Hard DanceHome PageListen!
    DI.FM HardstyleHome PageListen!
    DI.FM HouseHome PageListen!
    DI.FM LoungeHome PageListen!
    DI.FM MinimalHome PageListen!
    DI.FM ProgressiveHome PageListen!
    DI.FM PsyChillHome PageListen!
    DI.FM Soulful HouseHome PageListen!
    DI.FM Space MusicHome PageListen!
    DI.FM Tech HouseHome PageListen!
    DI.FM TechnoHome PageListen!
    DI.FM TranceHome PageListen!
    DI.FM Tribal HouseHome PageListen!
    DI.FM Vocal TranceHome PageListen!
    EYE97Home PageListen!
    EYE97 DanceHome PageListen!
    SkyFM Alt RockHome PageListen!
    SkyFM BeatlesHome PageListen!
    SkyFM BebopHome PageListen!
    SkyFM Bossa Nova JazzHome PageListen!
    SkyFM ChristianHome PageListen!
    SkyFM ClassicalHome PageListen!
    SkyFM Classic RapHome PageListen!
    SkyFM Classic RockHome PageListen!
    SkyFM CountryHome PageListen!
    SkyFM DaTempo LoungeHome PageListen!
    SkyFM GuitarHome PageListen!
    SkyFM Hit 70sHome PageListen!
    SkyFM Indie RockHome PageListen!
    SkyFM JazzHome PageListen!
    SkyFM Love MusicHome PageListen!
    SkyFM New AgeHome PageListen!
    SkyFM OldiesHome PageListen!
    SkyFM Piano JazzHome PageListen!
    SkyFM Roots ReggaeHome PageListen!
    SkyFM SalsaHome PageListen!
    SkyFM Smooth JazzHome PageListen!
    SkyFM Solo PianoHome PageListen!
    SkyFM SoundtracksHome PageListen!
    SkyFM The 80sHome PageListen!
    SkyFM Top HitsHome PageListen!
    SkyFM Uptempo Smooth JazzHome PageListen!
    SkyFM Urban JamzHome PageListen!
    SkyFM WorldHome PageListen!
    SLAY RadioHome PageListen!
    ABC iView
    KG and the General Live
    Australia Live TV
    TiVo Data

    —————— VLC_rtsp.reg —————————————

    Windows Registry Editor Version 5.00

    @=”URL:RealTime Streaming Protocol”
    “URL Protocol”=””

    @=””C:\Program Files\VideoLAN\VLC\vlc.exe” “%1″”


    ——————— RTSP_MeditPlayer.reg ————————–

    Windows Registry Editor Version 5.00

    @=”URL:RealTime Streaming Protocol”
    “URL Protocol”=””

    @=””C:\Program Files\Media Player Classic\mplayerc.exe” “%1″”


    No need to bypass security with a boot disk – 17 year old Windows exploit found

    The problem has been discovered in the Virtual DOS Machine (VDM) introduced in 1993 to support 16-bit applications (real mode applications for 8086). VDM is based on the Virtual 8086 Mode (VM86) in 80386 processors and, among other things, intercepts hardware routines such as BIOS calls. Google security team member Tavis Ormandy has found several vulnerabilities in this implementation that allow an unprivileged 16-bit program to manipulate the kernel stack of each process via a number of tricks. This potentially enables attackers to execute code at system privilege level.

    In addition to the unpatched hole in Internet Explorer, a now published hole in Windows allows users with restricted access to escalate their privileges to system level – and this is believed to be possible on all 32-bit versions of Windows from Windows NT 3.1 up to, and including Windows 7. While the vulnerability is likely to affect home users in only a minor way, the administrators of corporate networks will probably have their hands full this week.

    The problem is caused by flaws in the Virtual DOS Machine (VDM) introduced in 1993 to support 16-bit applications (real mode applications for 8086). VDM is based on the Virtual 8086 Mode (VM86) in 80386 processors and, among other things, intercepts hardware routines such as BIOS calls. Google security team member Tavis Ormandy has found several vulnerabilities in this implementation that allow an unprivileged 16-bit program to manipulate the kernel stack of each process via a number of tricks. This potentially enables attackers to execute code at system privilege level.

    Ormandy has also published a suitable exploit which functions under Windows XP, Windows Server 2003 and 2008, Windows Vista and Windows 7. When tested by the The H’s associates at heise Security, the exploit opened a command prompt in the system context, which has the highest privilege level, under Windows XP and Windows 7. No patch has become available, although Ormandy reports that Microsoft was already informed of the hole in mid 2009. The developer decided to publish the information regardless because, in his opinion, there is a simple workaround: to disable the MS-DOS subsystem.

    The workaround requires users to start the group policy editor and enable the “Prevent access to 16-bit applications” option in the Computer ConfigurationAdministrative TemplatesWindows ComponentsApplication Compatibility section. When tested with these settings by the heise Security team, the exploit no longer functioned. The settings reportedly don’t cause any major compatibility problems for most users while no 16-bit applications are being used.

    Update – The above option is only available through the group policy editor on Windows 2003 systems. Some versions of Windows do not include a group policy editor. As an alternative, users can also create a registry key under HKEY_LOCAL_MACHINESOFTWAREPoliciesMicrosoftWindowsAppCompat with a D-Word value of VDMDissallowed = 1. Under Windows XP, to prevent the system from being vulnerable to the exploit, users can place the following text:

    Windows Registry Editor Version 5.00



    into a file called vdmdisallow.reg and double click the file. Windows will then automatically import the key (admin rights are required to perform this action).

    Update 2 - Microsoft has now confirmed the privilege escalation hole in Windows. The company says that it wants to complete its investigation of the vulnerability and will then decide whether, how and when to close it.

    See Also:

    REDMOND — When it rains, it pours. Especially in the Seattle area. Tavis Ormandy has published full details on a privilege escalation hack of all versions of Windows including Windows 7.

    The exploit takes advantage of a bug in the Windows implementation of the ‘virtual DOS machine’ used to run legacy 16-bit programs. The exploit can be avoided by turning the VDM ‘feature’ off but the danger of course is that enough Windows lusers won’t know about the bug and/or bother turning the ‘feature’ off.

    16-bit applications need BIOS support; the Windows kernel supports virtual BIOS interrupts in its ‘Virtual-8086’ mode monitor code. The code is implemented in two stages. The #GP trap handler transitions to the second stage when CS:EIP faults with specific ‘magic’ values.

    The transition requires (subsequent to authentication) restoring the context and the call stack from the faulting trap frame. But the authentication process is flawed, relying as it does on three incorrect assumptions.

    • Setting up a VDM context requires SeTcbPrivilege.The barrier to getting a VDM context can be subverted by requesting the NT VDM subsystem and then using CreateRemoteThread() to run code in the context of the VDM subsystem. The VDM subsystem already has the necessary flag set.
    • Ring 3 (unprivileged) code cannot install arbitrary code segment selectors.Using the two least significant bits of CS/SS to calculate the privilege of a task doesn’t work when it comes to Virtual-8086 mode. The 20-bit addressing (by adding CS << 4 to the 16-bit IP) is also used to map onto the protected linear Virtual-8086 address space. If CS can be set to an arbitrary value, then the privilege calculation can be circumvented.
    • Ring 3 (unprivileged) code cannot forge a trap frame.Returns to user mode are through IRET. An invalid context can cause IRET to fail pre-commit, which in turn forges a trap frame. And even with address randomisation it’s trivial to use NtQuerySystemInformation() to obtain the address of the second stage BIOS handler.

    Affected Systems

    This bug dates back 17 years and affects all systems released since 27 July 1993 – Windows 2000, Windows XP, Windows Server 2003, Windows Vista, Windows Server 2008, and Windows 7. See the links below for further details.

    See Also
    MITRE: CVE-2010-0232
    Windows plagued by 17-year-old privilege escalation bug
    NEOPHASIS: Trap Handler Allows Users to Switch Kernel Stack

    The Internet is the modern day electricity

    Recently I have been been sorting through some of my old electronic engineering books and found myself randomly flicking through circuit design principals and practical electronics/radio theory application of calculus.

    I remember the amount of hours I spent trying to get the different laws (Faraday, Coulomb, Kirchhoff, Lenz, Ohm, etc.) stuck in my head ready for the gruelling exams at the end of each term. I quickly realised that as I have moved from radio/electronics to the computer industry that most of my applied detailed knowledge has been lost.

    I think the old adage that you loose it if you don’t use it, definitely applies here.


    This got me thinking about the evolution of computing and the Internet and how there are many parallels to the introduction of electricity to the modern world and how we consider/use the Internet today.

    Examples that came to mind are:

    • Electricity was originally only available to business and the very wealthy
    • Electricity was originally only available in isolated segments of heavily populated areas
    • Electricity grids, once created, provided many more distribution opportunities, introduced redundancy and increased the customer reach which in-turn provided economies of scale to drive down costs
    • Modern society can not function without electricity
    • Electricity production methods and the resulting pollution has had a profound effect on our planet, where the production of consumer electronics and infrastructure supporting the never ending thirst of modern society for faster, more feature rich, communication methods. This is still spiralling out of control through the production of extraordinary high levels of non-recyclable waist, heavy metals and other planet destroying bi-products
    • Electricity has been essential to survive in modern day society for some time.

    The internet is quickly becoming (some would argue has become) essential to survival in our modern society and required to be available to all socio-economic groups and developing countries to allow them to participate in the global economy.

    But at what cost?

    SSLv3 / TLS Man in the Middle vulnerability

    Recently I have been looking into the vulnerabilities in the TLS negotiation process discovered late last year.

    There are a range of experts debating the exploit methods, tools and how it may be fixed (server or client site or both). From what I have seen so far this may prompt a change to the TLS standard to introduce an extension to the protocol to validate sessions (session hand off and certificate validity).

    I’m also trying to find some tools which may assist in testing for this. It looks like the exploit relies on an ARP poison or similar and then inserting plain text into the negotiation process.

    Could be something that can be fixed over time as servers and clients are patched.

    Nmap Examples

    Some Nmap examples I thought I would post.

    Scanning past Watchguard Firewalls: nmap -sS -iL targetlist.txt -P0 -sV -T4

    Verbose Scan: nmap -v

    This option scans all reserved TCP ports on the target machine. The -v option enables verbose mode.

    nmap -sS -O /24

    Launches a stealth SYN scan against each machine that is up out of the 256 IPs on “class C” sized network where Scanme resides. It also tries to determine what operating system is running on each host that is up and running. This requires root privileges because of the SYN scan and OS detection.

    nmap -sV -p 22,53,110,143,4564 198.116.0-255.1-127

    Launches host enumeration and a TCP scan at the first half of each of the 255 possible eight-bit subnets in the 198.116 class B address space. This tests whether the systems run SSH, DNS, POP3, or IMAP on their standard ports, or anything on port 4564. For any of these ports found open, version detection is used to determine what application is running.

    nmap -v -iR 100000 -PN -p 80

    Asks Nmap to choose 100,000 hosts at random and scan them for web servers (port 80). Host enumeration is disabled with -PN since first sending a couple probes to determine whether a host is up is wasteful when you are only probing one port on each target host anyway.

    nmap -PN -p80 -oX logs/pb-port80scan.xml -oG logs/pb-port80scan.gnmap

    This scans 4096 IPs for any web servers (without pinging them) and saves the output in grepable and XML formats.

    Instead of limiting ourselves to scanning just one target., let’s broaden our horizon’s to bigger and better things. In example 2 we used our IP address to base a scan against. Using that address again we can get a look at numerous targets in our “community”. At the command line type the following (substituting a valid address of your choice of course):

    nmap -sT -O

    What this does is instruct nmap to scan every host between the IP addresses of and If you happen to find many interesting feedback results from this or a larger scale scan then you can always pipe the output into your choice of a human readable file or a machine parsable file for future reference by issuing the following option:

    To create a human readable output file issue the -oN command into your nmap string so that it would look similar to this:

    nmap -sT -O -oN sample.txt

    Rather have a machine parsable file? Enter the -oM to pipe the output into a machine parsable file:

    nmap -sT -O -oM sample.txt

    *Back when I was becoming aquatinted with all the nmap options, I ran my first large scale scan against 250 consecutive machines using an arbitrary number (nmap -sX -O -oN sample.txt XXX.XXX.XXX.0-250).To my great surprise I was confronted with 250 up and running virgin Linux machines. Another reason why Linux enthusiasts should NEVER become bored.

    -I This is a handy little call that activates nmap’s TCP reverse ident scanning option. This divulges information that gives the username that owns available processes. Let’s take a look (Note that the host has to be running ident). At the command line issue this command against your target, in this case our default Eve running Linux:

    -iR Use this command to instruct nmap to scan random hosts for you.

    -p Port range option allows you to pick what port or ports you wish nmap to scan against.

    -v Use verbosity to display more output data. Use twice (-v -v) for maximum verbosity.

    -h Displays a quick reference of nmap’s calls

    Now that we have looked at nmap’s three basic usage types and some of it’s other options, let’s mix and match them.

    nmap -v -v -sS -O

    This instructs nmap to use a maximum amount of verbosity to run a stealth scan and OS detection against all machines between IP addresses and This command will also require root privileges due to both the -sS and -O calls. Of course this will display a very overwhelming amount of data so let’s log our results into a human readable file for future reference:

    nmap -v -v -sS -O -oN sample.txt

    Now let’s make nmap run a stealth scan and instruct it to look only for machines offering http and ftp services between the addresses of and Once again we will log the output (I’m a log junkie) for future reference into a human readable file called ftphttpscan.txt:

    nmap -sS -p 23,80 -oN ftphttpscan.txt

    Remember the -iR option mentioned previously? Let’s use it to take a random sampling of Internet web servers using the verbatim example from nmap’s man page:

    nmap -sS -iR -p 80

    Last but certainly not least, while gleaning information, don’t forget to nmap yourself. Just type at the command line: nmap This is especially useful and recommended if you’re a newcomer to Linux and connected to the Internet via DSL or cable modem.

    Detect promiscuous network devices or sniffers on a network

    Old versions       nmap –script=promiscuous

    New Versions     nmap -sV –script=sniffer-detect