metasploit-framework/data/john/doc/MARKOV

BASIC USAGE
The Markov mode is based from [1], tested and applied to "classical" password
cracking in [2]. This mode similar to the "wordlist" mode because it will only
crack a fixed quantity of passwords. Its parameters are:

--markov:LEVEL:START:END:LENGTH

Where:
* LEVEL is the "Markov level". This value is the maximum strength of passwords
that are going to be cracked. When LEVEL increases, the quantity of passwords
that are going to be tested increases exponentially.
* START is the index of the first password that is going to be tested, starting
with 0.
* END is the index of the last password that is going to be tested. When it is
set to 0, it will represent the last possible password.
* LENGTH is the maximum length of the tested passwords.

using --markov:100:0:0:12 will let john check every password whose length is 12
or less and whose "Markov strength" is 100 or less.


SELECTING THE PARAMETERS
The "LEVEL" parameter should be selected based on the desired maximum running
time. In order to select the appropriate LEVEL, the following steps should be
followed:
1/ Run the -single and -wordlist modes of john, as they will find many passwords
for a low price
2/ Run john with a low markov level on the file, using the time utility. For
example:
*******************************************************************************
time john -markov:180 test
Loaded 156 password hashes with no different salts (NT LM DES [128/128 BS SSE2])
Warning: MaxLen = 12 is too large for the current hash type, reduced to 7
MKV start (lvl=180 len=7 pwd=30449568)
guesses: 0  time: 0:00:00:10 99%  c/s: 475013K  trying:

real    0m10.707s
user    0m10.621s
sys     0m0.012s
*******************************************************************************
This means that john can test 2.8M (30449568/10.707) passwords / seconds. It
should be noted that with salted passwords the cracking speed will increase with
every cracked password. This number should be corrected based on the experience
of the user.
3/ Evaluate the quantity of passwords that could be cracked during the selected
time. Using the previous example, a cracking time of 3 hours will lead to a
quantity of passwords of 30714M passwords (30449568/10.707*3600*3).
4/ Use the genmkpwd command to find the corresponding level. Using the previous
example, with a maximum password length of 12 (stupid because LM has a maximum
length of 7 ...):
*******************************************************************************
genmkvpwd stats 0 12
[...]
lvl=245 (5904 Kb for nbparts) 26 G possible passwords (26528306250)
lvl=246 (5928 Kb for nbparts) 29 G possible passwords (29373638087)
lvl=247 (5952 Kb for nbparts) 32 G possible passwords (32524537496)
[...]
*******************************************************************************
Here, the selected level will be 246 (the higher level where the number of
possible passwords is less than 30714M).
5/ Run john:
*******************************************************************************
john -markov:246:0:0:12 test
*******************************************************************************


DISTRIBUTING WORK
The START and END parameter could be used to distribute work among many CPUs.
The preferred method is to evaluate the combined cracking speed of all CPUs
(adding the step 2 result for every CPUs available) and follow the previous
method.
At step 5, share the cracking space among all CPUs, where is share is
proportionnal with the CPU's cracking speed.


CONFIGURATION OPTIONS
New options are available in the john.conf file:
Statsfile - This is the path of the "stat" file.
MkvLvl - the default level
MkvMaxLen - the default length


WHAT IS THE STAT FILE?
The markov mode is based on statistical data from real passwords. This data is
stored in the "stat" file. In order to generate a custom stat file, it is
recommanded to use the new calc_stat command:

./calc_stat "dictionnary file" stats


MKVCALCPROBA USAGE
This program is used to generate statistics about cracked passwords. It accepts
as input the "stat" file and a file with a single cracked password per line.
Here is a sample output:

*******************************************************************************
./mkvcalcproba stats /tmp/passwordlist
test    33+16+28+20     97      4       40030907        45
password        29+16+30+22+51+25+24+30 227     8       2698006565378672        177
32'[[!  55+24+98+1000+23+29     1229    6       39949021871     1169
charsetsize = 92
*******************************************************************************

Its output is tab separated and should open nicely in spreadsheets. Here is the
meaning of the column:
1/ Cracked password, reprinted from the file
2/ Sum of all "markov probabilities" of every letter of the word. This is
supposed to help identify which parts of the password makes them strong. The
number "1000" is written when no 1st/2nd letter combinations were found in the
stat file (for exemple ' then [ here).
3/ Markov strength
4/ Password length
5/ Rank when bruteforced "stupidly" (a, b, c, ..., aa, ab, ac ...) considering
that letters are ordered given their appearance probability and the given
charsetsize (92)
6/ Markov strength of the password where the two first letters are removed


REFERENCES

[1] http://www.cs.utexas.edu/~shmat/shmat_ccs05pwd.ps
[2] http://actes.sstic.org/SSTIC07/Password_Cracking/