.-.-.-..-.-.-..---..---. | | | || | | || | || |-' `-----'`-'-'-'`-^-'`-' Metasploit Wmap 1.5 ============================================================================== Efrain Torres et [ ] metasploit.com 2012 ------------------------------------------------------------------------------ =[ 0. Intro ] ---------------------------------------------------------------- So its 2012 and before the Mayans are proven to be right I was able to create a new version of this plugin. If you have read the old wmap documentation this is what is going on: Wmap is still a general purpose web application scanning framework for Metasploit. Still is a different approach compared to other open source alternatives and commercial scanners, as Wmap is not build around any browser or spider for data capture and manipulation. And the best thing is that still is FR33. Lots of bugs are gone and the new code allows for faster and more efficient execution. =[ 1. How it works ] --------------------------------------------------------- The old architecture (versions < 1.5): [CLIENT] ----- [ATTACK PROXY] ----- [TARGET] | | ^ +--------->[METASPLOIT DB] | | | [MSF 3 - Wmap SCANNER] | [MSF 3 - Wmap MODULES] -----+ The new architecture: [CLIENTS] | | +-------[Wmap PLUGIN]<-----+----->[METASPLOIT DB] | | | | | | | | [NODE 1] [NODE 2] [NODE n] ---------+ | | | \ | | | [Wmap MODULES] +---------[TARGETS]--------+ Wmap is a Metasploit plugin and will interact with the database, reading all gathered traffic from any client you have configured/adapted or duck taped to store web sites, requests, responses and forms in the Metasploit DB. The test performed are all Metasploit modules which WMAP execute in a configurable order. The test modules are implemented as auxiliary or exploit modules and they can interact with any other MSF component including the database other exploits and plugins. The new architecture allows to have different distributed clients and nodes all storing results and data to a central database. This means that large enviorments can be tested using multiple metasploit msfrcpd servers (nodes) controled from one (or more) WMAP consoles. Wmap will execute the tests to be launched from each node distributing evenly the job load across all configured nodes. In case you dont want to use a distributed model wmap will detect that no nodes have been configured and will run the modules from the local host. =[ 2. Crawlers,proxies and other clients ] ----------------------------------- At this time Metasploit have 3 components that may be used as clients to store web sites in the database: (1) If you have configured your database properly and use the auxiliary/scanner/http/crawler module, this module will create a web site (with related host and service) and store all requests,responses and forms automatically. (2) Less known is that metasploit has a different crawler called msfcrawler and besides supporting modules to parse the responses in any way you want it will also store the required data in the database. (3) Also any module that creates a web_site in the database (e.g. auxiliary/scanner/http/http_version module) will add a site to the database that can be selected as a target in Wmap, however the only path you will be storing will be the root path of the website '/'. Or you can add a site manually to the database using the 'wmap_sites -a' command (after loading the wmap plugin. See '4. Wmap Plugin'): msf > wmap_sites -a www.blah.net,http://192.168.0.2/ [*] Site created. Note: www.blah.net,http://192.168.0.2/ <-- is one site vhost,url For other tools to store web data in the database the only hard part is to deal with the ruby marshalling of parameters in the web_forms table. (Topic for another paper). But this is one of the main issues regarding the use of other tools to connect to the database. However any Ruby based tool can be modified easily to do this. If you noticed the previous architecture the ATTACK PROXY has gone the way of the Dodo (actually not as is just another client). But i will stop mention it because i have been unable to create a good Ruby based MITM proxy (Sorry) and second because i dont want to maintain plugins for every type of proxy out there. Is your exercise to create custom plugins for your tools to connect to the database, after that Wmap does not care where the data comes from. =[ 3. The Wmap cycle ]-------------------------------------------------------- Or how every other scanner works but in wmap fancy terms. 0. Gather data from (See Crawlers,proxies and other clients) targets 1. Load the scanner (Load wmap plugin. See "4. Wmap Plugin") 2. Optional: Define nodes (Use 'wmap_nodes'. See "6. Wmap Nodes") 3. Define targets (Use 'wmap_sites' and 'wmap_targets'.See "5. Wmap Targets") 4. Configure (Lots of things here) 5. Launch (Use 'wmap_run'. See "7. Launch a scan") 6. Enjoy WTF Moments(Priceless...) Note: Step 2 is optional as Wmap allows you to perform distributed scans using multiple nodes. If no nodes are configured the scan runs as usual running all tests from the local host. =[ 4. Wmap Plugin ]----------------------------------------------------------- To launch wmap open a Metasploit console and load the wmap plugin. msf > load wmap [*] [Wmap 1.5] === et [ ] metasploit.com 2012 [*] Successfully loaded plugin: wmap Now that the plugin is loaded lets go through the list of basic commands: msf > help wmap Wmap Commands ============= Command Description ------- ----------- wmap_nodes Manage nodes wmap_run Test targets wmap_sites Manage sites wmap_targets Manage targets wmap_modules Manage wmap modules =[ 5. Wmap Targets ]---------------------------------------------------------- The targets are selected from the sites already stored in the database. For example after crawling a site (See "2. Crawlers,proxies and other clients"). Now we can use the command 'wmap_sites' to list them: msf > wmap_sites [*] Usage: wmap_sites [options] -h Display this help text -a [url] Add site (vhost,url) -l List all available sites -s [id] Display site structure (vhost,url|ids) (level) msf > wmap_sites -l [*] Available sites =============== Id Host Vhost Port Proto # Pages # Forms -- ---- ----- ---- ----- ------- ------- 0 10.10.10.1 blah.xyz.com 443 https 3 2 1 10.10.10.2 blah.xyz.com 443 https 3 2 2 10.1.2.2 nah.test.com 443 https 1 0 3 10.4.3.10 test.abcd.com 80 http 1 1 Note 1: Metasploit/Wmap supports multiple Vhosts/IPs. Note 2: If you want to check the web site structure use the '-s site_id' flag like this (also especify an optional level to display): msf > wmap_sites -s 0 1 (First level of site 0) msf > wmap_sites -s 0 [10.10.10.1] (blah.xyz.com) | +-------- dir1 | +------ login.php +-------- dir2 .... Then from the table we can select the targets we want to scan with the 'wmap_targets' command: msf > wmap_targets [*] Usage: Wmap_targets [options] -h Display this help text -t [urls] Define target sites (vhost1,url[space]vhost2,url) -d [ids] Define target sites (id1, id2, id3 ...) -c Clean target sites list -l List all target sites You can define targets in two ways, using the vhost,url syntax (-t) or the table ids (-d) msf > wmap_targets -t test.abcd.com,http://10.4.3.10/ msf > wmap_targets -d 0,1 [*] Loading blah.xyz.com,https://10.10.10.1:443/. [*] Loading blah.xyz.com,https://10.10.10.2:443/. To see the list of all the targets to scan at this time run the command with the (-l) flag. msf > wmap_targets -l [*] Defined targets =============== Id Vhost Host Port SSL Path -- ----- ---- ---- --- ---- 0 blah.xyz.com 10.10.10.1 443 true / 1 blah.xyz.com 10.10.10.2 443 true / 2 test.abcd.com 10.4.3.10 80 false / =[ 6. Wmap Nodes ]------------------------------------------------------------ Wmap uses 'nodes' as a way to distribute the execution of the test against one or more targets. Nodes are not required to run wmap . if nodes are not configured and a scan is launched wmap will detect this and launch all tests from the local host. The nodes are just msfrpcd servers that are created the following way: msf>ruby msfrpcd -h Usage: msfrpcd OPTIONS: -P Specify the password to access msfrpcd -S Disable SSL on the RPC socket -U Specify the username to access msfrpcd -a Bind to this IP address -f Run the daemon in the foreground -h Help banner -n Disable database -p Bind to this port instead of 55553 -u URI for Web server msf>ruby msfrpcd -U msf -P nodepass [*] MSGRPC starting on 192.168.0.1:55553 (SSL):Msg... [*] MSGRPC backgrounding at 2012-01-17 11:01:01 -0600... if you want to create a msfrpc server from the msfconsole you can do it by loading the msgrpc plugin: msf > load msgrpc User=msf Pass=nodepass [*] MSGRPC Service: 127.0.0.1:55552 [*] MSGRPC Username: msf [*] MSGRPC Password: nodepass [*] Successfully loaded plugin: msgrpc On a later stage in the wmap console we will add such nodes so the scans can be distributed across all the configured nodes. so remember how you deployed your nodes so they can be configured in wmap. In the metasploit console after you have loaded the wmap plugin you can add the previous nodes with the 'wmap_nodes' command: msf > wmap_nodes [*] Usage: wmap_nodes [options] -h Display this help text -c id Remove id node (Use ALL for ALL nodes -a host port ssl user pass Add node -d host port user pass db Force all nodes to connect to a db -j View detailed jobs -k ALL|id ALL|job_id Kill jobs on node -l List all current nodes msf > wmap_nodes -a 192.168.0.1 55553 true msf nodepass [*] Connected to 192.168.0.1:55553 [4.2.0-dev]. [*] Node created. Note: When launching msfrpcd waiht for a couple of seconds beofr adding it to wmap as msfrpcd sometimes is slow to start accepting connections. Add as many nodes you want. To see the list use 'wmap_nodes -l': msf > wmap_nodes -l [*] Nodes ===== Id Host Port SSL User Pass Status #jobs -- ---- ---- --- ---- ---- ------ ----- 0 127.0.0.1 55553 true msf nodepass 4.2.0-dev 0 1 192.168.0.1 55553 true msf nodepass 4.2.0-dev 0 Note: After launching all tests this command will allow you to see if all your jobs have been completed (#jobs == 0). Remember that all these commands can be added as a .rc file so you dont have to type again and again the loading of wmap and the configuration of nodes. After you have your nodes connected then you can force them to connect to the central metasploit database: msf > wmap_nodes -d 127.0.0.1 7175 dbuser dbpass msf3 [*] db_connect {"driver"=>"postgresql", "db"=>"msf3"} 127.0.0.1:7175 OK [*] db_connect {"driver"=>"postgresql", "db"=>"msf3"} 192.168.0.1:7175 OK [*] OK. =[ 7. Launch a scan ]--------------------------------------------------------- Now that database,targets and maybe nodes are set we run a scan with the 'wmap_run' command: msf > wmap_run [*] Usage: wmap_run [options] -h Display this help text -t Show all enabled modules -m [regex] Launch only modules that match provided regex -p [regex] Only test path defined by regex.. -e [/path/to/profile] Launch profile modules against all targets. No file runs all enabled modules. msf > wmap_run -e [*] Using ALL wmap enabled modules. [*] Testing target: [*] Site: test.abcd.com (10.4.3.10) [*] Port: 80 SSL: false ============================================================ [*] Testing started. 2012-12-21 0:0:0 -0600 [*] =[ SSL testing ]= ============================================================ [*] Target is not SSL. SSL modules disabled. [*] =[ Web Server testing ]= ============================================================ [*] Module auxiliary/admin/http/http_version [*] Module auxiliary/admin/http/tomcat_administration [*] Module auxiliary/admin/http/tomcat_utf8_traversal [*] Module auxiliary/admin/http/trendmicro_dlp_traversal [*] Module auxiliary/scanner/http/cisco_nac_manager_traversal .... msf > As you see here wmap executes each of the modules against the defined targets. (See "8. Wmap Modules") If nodes were configured it will go thru the list of nodes and will send a job to the less loaded node to execute the especific module with the required options. To force a good job distribution across the nodes wmap has a limit of 25 jobs per node. If a node has reached the limit it will try with the next node until there is a slot available. Check periodically with the 'wmap_nodes -l' command to see the current job status. After the asssement is complete now you can use the normal metasploit commands to see the results. To view detailed job information on each node use the 'wmap_nodes' (-j) flag: msf >wmap_nodes -j [*] [Node #0: 127.0.0.1 Port:55553 SSL:true User:msf] [*] Jobs ==== Id Job name Target PATH -- -------- ------ ---- 0 Auxiliary: scanner/http/dir_scanner 192.168.0.1:80 / ... [*] [Node #1: 127.0.0.1 Port:55555 SSL:true User:msf] [*] Jobs ==== Id Job name Target PATH -- -------- ------ ---- 2 Auxiliary: scanner/http/dir_scanner 192.168.0.2:80 / ... Also you can kill especific jobs or all jobs from one or all nodes: msf > wmap_nodes -k 0 ALL [*] Node 0 Killed job id 262 Auxiliary: admin/http/tomcat_administration [*] Node 0 Killed job id 263 Auxiliary: admin/http/tomcat_utf8_traversal [*] Node 0 Killed job id 271 Auxiliary: scanner/http/soap_xml [*] Node 0 Killed job id 299 Auxiliary: scanner/http/brute_dirs [*] Node 0 Killed job id 300 Auxiliary: scanner/http/brute_dirs [*] Node 0 Killed job id 301 Auxiliary: scanner/http/brute_dirs .... If during the scan a node dies wmap will disable the node and will keep sending the jobs to the other active nodes. If nodes were not configured wmap will launch the tests from the local host the old fashion. =[ 8. Wmap Modules ] --------------------------------------------------------- Wmap modules are normal Metasploit modules. Each module has a WMAP type, this determine when the module is launched and to a certain degree,the minimum type of information it requires to be executed. The best way to develop a new test for wmap, is to use already implemented modules as a base and then develop a normal MSF module that can be run manually from the command line. To enable a module to be run automatically via wmap just include the mixin that determine the type of the module (Means: just add the example string to a module and use the correct type). Example: include Auxiliary::WmapScanFile The following are the types of modules implemented at this time and they are listed in the order WMAP runs them: WmapScanSSL - Run once against a SSL server WmapScanServer - Run once against a target Web Server WmapScanDir - Runs for every directory found in the target WmapScanFile - Runs for every file found in the target WmapScanUniqueQuery - Runs for every unique query found in each request to the target WmapScanQuery - Runs for every query found in each request to the target WmapScanGeneric - Modules to be run after all tests complete.Good place to perform passive analysis of responses, analysis of test results to launch other modules (i.e. exploits). Note: Multiple mixins can be included in a module if needed. The execution order not only is handled by the wmap type but also it can be adjusted across all modules by defining a wmap orderid number using the 'register_wmap_options' method. Using http_version.rb module as an example: Class Metasploit3 < Msf::Auxiliary # Exploit mixins should be called first include Msf::Exploit::Remote::HttpClient include Msf::Auxiliary::WmapScanServer # Scanner mixin should be near last include Msf::Auxiliary::Scanner def initialize super( 'Name' => 'HTTP Version Detection', ... ) register_wmap_options({ 'OrderID' => 0, 'Require' => {}, }) ... 'OrderID' Numeric value that represents the order ALL modules will be executed You can see the modules and orderid with the 'wmap_modules -l' command: msf > wmap_modules -l 'Require' Array of all the modules orderids that are required to be executed and finished first before the curent module. (This specific funtionality is still in the works, but the objective is to have modules to provide results as input to other modules.) Wmap enabled modules can be reloaded using the wmap_modules -r command. =[ 9. RANDOM NOTES ]---------------------------------------------------------- Because every test is a module the datastore is sent to the module in the node for execution. If a module you create needs a specific option set before launch just set it in the console as a regular variable. For example: msf > set DOMAIN abcd.com DOMAIN => abcd.com msf > This is usefull if you want to include exploits in the testing and not only auxiliary modules. WMAP looks for wmap enabled modules in ALL auxiliary and exploit modules. Also if you have asked yourself why there are commands that receive not only table id but the ugly vhost,url syntax for site and target definition is because this allows to do complex scripts, so be creative. If you see a Reauth message in WMAP is because the XMLRPC token is not valid and a reauthentication to the nodes is required. But dont worry wmap does that automaticaly for you. =[ 10. TO DO ]---------------------------------------------------------------- - The quality of the scan depends on the quality of the modules. So please contribute more modules and improvements. If you dont contribute , you dont have the right to complain. The only key issues to consider are: + The module should follow metasploit guidelines + Add the right mixin(s) + The module should store the right data and results in the database + The module has to have a clear purpose!!!! Is better to have multiple modules with simple tasks/objectives that one that does everything. + The variables/options used in the datastore have to be the same between wmap and the module so wmap can pass the right information to it. Usually this is the naming convention to use for the options: OptString: 'VHOST' = HTTP Virtual Host 'METHOD' = HTTP Method 'PATH' = HTTP URI Path. 'QUERY' = HTTP URI Query usually in the param1=value1& form. 'DATA' = HTTP Data. In a POST request is the body of the request. Usually in the param1=value1& form. 'HEADERS'= HTTP headers (header1=value1;..) OptBool: 'VERBOSE'= Verbose flag. Note: This naming convention may change. However if this naming convention is used any changes can be implemented easily. - Also if you want to take the big task of developing a ruby MITM proxy for metasploit that will be very helpfull for the project. =[ Disclaimer ]--------------------------------------------------------------- I dont work for R7. XD ============================================================================== et [ ] metasploit.com 2012