Categotry Archives: Reverse Engineering

by

Emulating Assembly in Radare2

No comments yet

Categories: Reverse Engineering, Tags:

I’ve still been playing around with Radare2 when I can, and I wanted to see how easy it was to emulate a block of disassembled code using its Evaluable Strings Intermediate Language, or ESIL. For an example, I went back to Eldad Eilam’s Reversing: Secrets of Reverse Engineering.

In Chapter 11, he shows how to reverse a program called Defender.exe that requires a correct username and serial number. There are several protections and anti-reversing tricks this program employs. Specifically, I was looking at some of the sections of code that are encrypted. There are several of these in this program and they all start with a normal looking function followed by some data. That data is decrypted, executed, then encrypted once again.

The executable is available for download in the Downloads section of the book’s page on Wiley. The code I’m going to be referencing is discussed around page 386 (function at 0x4033d1) and page 397 (function at 0x402eef).

Initializing r2 and Observing the Encrypted Code

Assuming you have radare2 and the executable, the initial startup process is straightforward, using the aaa command for initial analysis:

$ r2 Defender.exe
 -- Here be dragons.
[0x00404232]> aaa
[x] Analyze all flags starting with sym. and entry0 (aa)
[x] Analyze len bytes of instructions for references (aar)
[x] Analyze function calls (aac)
[ ] [*] Use -AA or aaaa to perform additional experimental analysis.
[x] Constructing a function name for fcn.* and sym.func.* functions (aan))
[0x00404232]>

Seek to the first function and print it out.

[0x00404232]> s 0x4033d1
[0x004033d1]> pd 100
┌ (fcn) fcn.004033d1 268
│   fcn.004033d1 (int arg_3h);
│              ; CALL XREF from 0x0040423f (entry0)
│           0x004033d1      55             push ebp
│           0x004033d2      8bec           mov ebp, esp
│           0x004033d4      81ec2c020000   sub esp, 0x22c
│           0x004033da      53             push ebx
│           0x004033db      56             push esi
│           0x004033dc      57             push edi
│           0x004033dd      68dd344000     push 0x4034dd
│           0x004033e2      58             pop eax
│           0x004033e3      8945e0         mov dword [local_20h], eax
│           0x004033e6      68fd414000     push 0x4041fd               ; "_^[....`@"
│           0x004033eb      58             pop eax
│           0x004033ec      8945e8         mov dword [local_18h], eax
│           0x004033ef      b8e5344000     mov eax, 0x4034e5
│           0x004033f4      8905d6344000   mov dword [0x4034d6], eax   ; [0x4034d6:4]=0x4034e5
│           0x004033fa      c745f8010000.  mov dword [local_8h], 1
│           0x00403401      837df800       cmp dword [local_8h], 0
│       ┌─< 0x00403405 7466 je 0x40346d 
...snip... 
│ │ ; JMP XREF from 0x0040346b (fcn.004033d1) 
│ └──>      0x004034d5      68e5344000     push 0x4034e5
│           0x004034da      5b             pop ebx
└           0x004034db      ffe3           jmp ebx
...snip...
[0x004033d1]> pd 10 @0x4034e5
               ; DATA XREF from 0x004033ef (fcn.004033d1)
               ; DATA XREF from 0x004034d5 (fcn.004033d1)
        ┌─< 0x004034e5      7e10           jle 0x4034f7
        │   0x004034e7      b98842c1f8     mov ecx, 0xf8c14288
        │   0x004034ec      e68b           out 0x8b, al
        │   0x004034ee      2b7f7d         sub edi, dword [edi + 0x7d]
        │   0x004034f1      f1             int1
        │   0x004034f2      0289dfc5cbc8   add cl, byte [ecx - 0x37343a21]
            0x004034f8      b114           mov cl, 0x14
            0x004034fa      214f2a         and dword [edi + 0x2a], ecx
            0x004034fd      6e             outsb dx, byte [esi]
            0x004034fe      08fd           or ch, bh

I’m not going to go into detail on everything going on here since it’s covered in the book. The function starts out like all these encrypted functions do, and towards the end there is a PUSH-POP-JMP sequence. The JMP at 0x4034db will move execution to 0x4034e5. When starting disassembly from that line, it’s clear this doesn’t look like legitimate assembly code because it’s encrypted.

Setting up the ESIL Environment

Using ESIL, it’s fairly simple to get radare2 to decrypt this for us. First set up the ESIL environment:

[0x004033d1]> e asm.emu=true
[0x004033d1]> e asm.emustr=true
[0x004033d1]> e asm.esil=true

The first two commands modify the disassembly so that ESIL information is displayed. The output with asm.emu is verbose, while asm.emustr only shows the most useful information. Setting asm.esil=true shows what the ESIL looks like. The following two snippets show the difference between the asm.emu and asm.esil settings:

[0x004033d1]> s 0x4033d1
[0x004033d1]> e asm.emu=true
[0x004033d1]> pd 10
┌ (fcn) fcn.004033d1 268
│   fcn.004033d1 (int arg_3h);
│              ; CALL XREF from 0x0040423f (entry0)
│           0x004033d1      55             push ebp                    ; esp=0xfffffffffffffffc -> 0xffffff00
│           0x004033d2      8bec           mov ebp, esp                ; ebp=0xfffffffc -> 0xffffff00
│           0x004033d4      81ec2c020000   sub esp, 0x22c              ; esp=0xfffffdd0 -> 0xffffff00  ; of=0x0  ; sf=0x1 -> 0x3009000  ; zf=0x0  ; pf=0x0  ; cf=0x0
│           0x004033da      53             push ebx                    ; esp=0xfffffdcc -> 0xffffff00
│           0x004033db      56             push esi                    ; esp=0xfffffdc8 -> 0xffffff00
│           0x004033dc      57             push edi                    ; esp=0xfffffdc4 -> 0xffffff00
│           0x004033dd      68dd344000     push 0x4034dd               ; esp=0xfffffdc0 -> 0xffffff00
│           0x004033e2      58             pop eax                     ; eax=0xffffffff -> 0xffffff00  ; esp=0xfffffdc4 -> 0xffffff00
│           0x004033e3      8945e0         mov dword [local_20h], eax
│           0x004033e6      68fd414000     push 0x4041fd               ; esp=0xfffffdc0 -> 0xffffff00
[0x004033d1]> s 0x4033d1
[0x004033d1]> e asm.esil=true
[0x004033d1]> pd 10
┌ (fcn) fcn.004033d1 268
│   fcn.004033d1 (int arg_3h);
│              ; CALL XREF from 0x0040423f (entry0)
│           0x004033d1      55             ebp,4,esp,-=,esp,=[4]       ; esp=0xfffffffffffffffc -> 0xffffff00
│           0x004033d2      8bec           esp,ebp,=                   ; ebp=0xfffffffc -> 0xffffff00
│           0x004033d4      81ec2c020000   556,esp,-=,$o,of,=,$s,sf,=,$z,zf,=,$p,pf,=,$b4,cf,= ; esp=0xfffffdd0 -> 0xffffff00  ; of=0x0  ; sf=0x1 -> 0x3009000  ; zf=0x0  ; pf=0x0  ; cf=0x0
│           0x004033da      53             ebx,4,esp,-=,esp,=[4]       ; esp=0xfffffdcc -> 0xffffff00
│           0x004033db      56             esi,4,esp,-=,esp,=[4]       ; esp=0xfffffdc8 -> 0xffffff00
│           0x004033dc      57             edi,4,esp,-=,esp,=[4]       ; esp=0xfffffdc4 -> 0xffffff00
│           0x004033dd      68dd344000     4207837,4,esp,-=,esp,=[4]   ; esp=0xfffffdc0 -> 0xffffff00
│           0x004033e2      58             esp,[4],eax,=,4,esp,+=      ; eax=0xffffffff -> 0xffffff00  ; esp=0xfffffdc4 -> 0xffffff00
│           0x004033e3      8945e0         eax,0x20,ebp,-,=[4]
│           0x004033e6      68fd414000     4211197,4,esp,-=,esp,=[4]   ; esp=0xfffffdc0 -> 0xffffff00

These options aren’t really needed for the code emulation part, at least as far as I’ve found, but provide some more insight into what r2 will be doing during the emulation. The documentation seems to say that at least asm.emu is needed, but it worked fine for me when I didn’t include that. Maybe more complex code might require it.

[0x004033d1]> e asm.bits=32
[0x004033d1]> e asm.arch=x86
[0x004033d1]> e asm.emuwrite=true
[0x004033d1]> e io.cache=true

The first two commands above also don’t seem to be necessary (maybe because they are the defaults and match the file we’re looking at), but probably don’t hurt to ensure r2 will handle the code correctly. The last two are important. They allow r2 to modify memory and enable cache for io changes, respectively.

[0x004033d1]> s 0x4033d1
[0x004033d1]> aei
[0x004033d1]> aeim
[0x004033d1]> aeip
[0x004033d1]> aer
oeax = 0x00000000
eax = 0x00000000
ebx = 0x00000000
ecx = 0x004041f9
edx = 0x00000000
esi = 0x00000000
edi = 0x00000000
esp = 0x00177dc4
ebp = 0x00177ffc
eip = 0x004033d1
eflags = 0x00000081

These commands finally set up the ESIL environment and display the current register values. First, aei initializes the ESIL VM state, aeim initializes the ESIL VM stack, and aeip initializes the ESIL program counter to the current address. Finally, aer displays the current registry values. The environment is ready to begin emulating the code. You could step through using various debugger-like commands such as aes, aeso, aec, which are all documented on the ESIL page above. The easiest way for this code, though, is to let it execute until just before jumping to the decrypted code.

[0x004033d1]> aecu 0x4034db
[0x004034d5]> pd 3
│              ; JMP XREF from 0x0040346b (fcn.004033d1)
│           0x004034d5      68e5344000     push 0x4034e5
│           0x004034da      5b             pop ebx
└           0x004034db      ffe3           jmp ebx
[0x004034d5]> pd 10 @0x4034e5
               ; DATA XREF from 0x004033ef (fcn.004033d1)
               ; DATA XREF from 0x004034d5 (fcn.004033d1)
            0x004034e5      8b4508         mov eax, dword [ebp + 8]    ; [0x8:4]=4
            0x004034e8      8945b0         mov dword [ebp - 0x50], eax
            0x004034eb      8b45b0         mov eax, dword [ebp - 0x50]
            0x004034ee      8b4db0         mov ecx, dword [ebp - 0x50]
            0x004034f1      03483c         add ecx, dword [eax + 0x3c] ; "PE"
            0x004034f4      894da8         mov dword [ebp - 0x58], ecx
            0x004034f7      8b45a8         mov eax, dword [ebp - 0x58] ; "PE"
            0x004034fa      8b4db0         mov ecx, dword [ebp - 0x50]
            0x004034fd      034878         add ecx, dword [eax + 0x78]
            0x00403500      894db8         mov dword [ebp - 0x48], ecx

0x4034db contains the “jmp ebx” instruction we looked at earlier and this aecu command tells r2 to emulate code until it reaches instruction 0x4034db. For some reason, r2 stopped at 0x4034d5, but we can see we are at the PUSH-POP-JMP sequence, and if we print the disassembly at 0x4034e5, it’s now decrypted.

Resetting ESIL and Decrypting Another Function

The second code segment starts at 0x402eef.

[0x004034d5]> pd 100 @0x402eef
            0x00402eef      55             push ebp
            0x00402ef0      8bec           mov ebp, esp
            0x00402ef2      83ec60         sub esp, 0x60               ; '`'
            0x00402ef5      53             push ebx
            0x00402ef6      68f62f4000     push 0x402ff6
            0x00402efb      58             pop eax
            0x00402efc      8945e4         mov dword [ebp - 0x1c], eax
            0x00402eff      68e2304000     push 0x4030e2
            0x00402f04      58             pop eax
            0x00402f05      8945f0         mov dword [ebp - 0x10], eax
            0x00402f08      b8fe2f4000     mov eax, 0x402ffe
            0x00402f0d      8905ef2f4000   mov dword [0x402fef], eax   ; [0x402fef:4]=0x402ffe
            0x00402f13      c745f4010000.  mov dword [ebp - 0xc], 1
            0x00402f1a      837df400       cmp dword [ebp - 0xc], 0
        ┌─< 0x00402f1e 7466 je 0x402f86 ; unlikely 
...snip...
    └──>    0x00402fee      68fe2f4000     push 0x402ffe
            0x00402ff3      5b             pop ebx
            0x00402ff4      ffe3           jmp ebx
...snip...
[0x004034d5]> pd 10 @0x402ffe
            0x00402ffe      b3f8           mov bl, 0xf8
            0x00403000      b265           mov dl, 0x65                ; 'e' ; "run in DOS mode....$"
        ┌─< 0x00403002      7c21           jl 0x403025                ; likely
        │   0x00403004      ce             into
        │   0x00403005      c8373783       enter 0x3737, -0x7d
        │   0x00403009      ec             in al, dx
        │   0x0040300a      39d6           cmp esi, edx
       ┌──< 0x0040300c      7614           jbe 0x403022               ; likely
       ││   0x0040300e      8e6e0a         mov gs, word [esi + 0xa]    ; [0xa:2]=0
       ││   0x00403011      d98577ff317e   fld dword [ebp + 0x7e31ff77]

We can see the same structure in this code: same sequence setting up the function, the PUSH-POP-JMP, and that JMP going to nonsense assembly code.

First, we’ll clear out the ESIL environment to start fresh.

[0x004034d5]> ar0
[0x004034d5]> aeim-
[0x00000000]> aei-

These commands clear the registers, de-initialize the VM stack, an de-initialize the VM state. Now we’ll run all the commands at once, this time stopping at the JMP at 0x402ff4.

[0x00000000]> s 0x402eef
[0x00402eef]> aei
[0x00402eef]> aeim
[0x00402eef]> aeip
[0x00402eef]> aer
oeax = 0x00000000
eax = 0x00000000
ebx = 0x00000000
ecx = 0x00000000
edx = 0x00000000
esi = 0x00000000
edi = 0x00000000
esp = 0x00178000
ebp = 0x00178000
eip = 0x00402eef
eflags = 0x00000000
[0x00402eef]> aecu 0x402ff4
[0x00402fee]> pd 10 @0x402ffe
            0x00402ffe      33c0           xor eax, eax
            0x00403000      40             inc eax
        ┌─< 0x00403001      0f84c0000000   je 0x4030c7                ; unlikely
        │   0x00403007      0f31           rdtsc
        │   0x00403009      8945f8         mov dword [ebp - 8], eax
        │   0x0040300c      8955fc         mov dword [ebp - 4], edx
        │   0x0040300f      a100604000     mov eax, dword [section_end..data] ; [0x406000:4]=-1 ; LEA section_end..data ; section_end..data
        │   0x00403014      8945b0         mov dword [ebp - 0x50], eax
        │   0x00403017      8b45b0         mov eax, dword [ebp - 0x50]
        │   0x0040301a      833800         cmp dword [eax], 0

As you can see, the emulation worked and the code is now decrypted.

Scripting the Commands with r2pipe

All of these commands can be put in a simple Python r2pipe script as follows:

$ cat defender.py
import sys
import r2pipe
r = r2pipe.open()
r.cmd('aaa')
r.cmd('e asm.emu=true')
r.cmd('e asm.emustr=true')
r.cmd('e asm.bits=32')
r.cmd('e asm.arch=x86')
r.cmd('e asm.emuwrite=true')
r.cmd('e io.cache=true')
r.cmd('s 0x4033d1')
r.cmd('aei')
r.cmd('aeim')
r.cmd('aeip')
r.cmd('aer')
r.cmd('aecu 0x0040346b')
r.cmd('ar0')
r.cmd('aeim-')
r.cmd('aei-')
r.cmd('s 0x402eef')
r.cmd('aei')
r.cmd('aeim')
r.cmd('aeip')
r.cmd('aer')
r.cmd('aecu 0x402ff4')

Finally, the script can be executed, then the functions reviewed as follows:

$ r2 -i defender.py Defender.exe
[x] Analyze all flags starting with sym. and entry0 (aa)
[x] Analyze len bytes of instructions for references (aar)
[x] Analyze function calls (aac)
[ ] [*] Use -AA or aaaa to perform additional experimental analysis.
[x] Constructing a function name for fcn.* and sym.func.* functions (aan))
 -- r2 -- leading options since 2006
[0x00402fee]> pd 10 @0x4034e5
               ; DATA XREF from 0x004033ef (fcn.004033d1)
               ; DATA XREF from 0x004034d5 (fcn.004033d1)
            0x004034e5      8b4508         mov eax, dword [ebp + 8]    ; [0x8:4]=4
            0x004034e8      8945b0         mov dword [ebp - 0x50], eax
            0x004034eb      8b45b0         mov eax, dword [ebp - 0x50]
            0x004034ee      8b4db0         mov ecx, dword [ebp - 0x50]
            0x004034f1      03483c         add ecx, dword [eax + 0x3c] ; "PE"
            0x004034f4      894da8         mov dword [ebp - 0x58], ecx
            0x004034f7      8b45a8         mov eax, dword [ebp - 0x58] ; "PE"
            0x004034fa      8b4db0         mov ecx, dword [ebp - 0x50]
            0x004034fd      034878         add ecx, dword [eax + 0x78]
            0x00403500      894db8         mov dword [ebp - 0x48], ecx
[0x00402fee]> pd 10 @0x402ffe
            0x00402ffe      33c0           xor eax, eax
            0x00403000      40             inc eax
        ┌─< 0x00403001      0f84c0000000   je 0x4030c7                ; unlikely
        │   0x00403007      0f31           rdtsc
        │   0x00403009      8945f8         mov dword [ebp - 8], eax
        │   0x0040300c      8955fc         mov dword [ebp - 4], edx
        │   0x0040300f      a100604000     mov eax, dword [section_end..data] ; [0x406000:4]=-1 ; LEA section_end..data ; section_end..data
        │   0x00403014      8945b0         mov dword [ebp - 0x50], eax
        │   0x00403017      8b45b0         mov eax, dword [ebp - 0x50]
        │   0x0040301a      833800         cmp dword [eax], 0

by

2016 FLARE-On Challenge 2

No comments yet

Categories: Challenges, Reverse Engineering

I didn’t get to spend a lot of time on the FLARE-On challenges this year, but I worked through the first 2 and used it as an opportunity to learn Radare2 a little better.

Here’s my solution to challenge 2.

$ md5 DudeLocker.exe BusinessPapers.doc
MD5 (DudeLocker.exe) = 4c262d5ab4bf8586d303bfa91a05b42b
MD5 (BusinessPapers.doc) = 857da52bdcaec6da473a59e2fe66df0a

Here is my r2 project file if you want to follow along with my notes. It should be unzipped in “.config/radare2/projects”, edit the challenge2 file and change the line “e file.path = /Users/yourname/challenge02/DudeLocker.exe” to the correct path for DudeLocker.exe, then open it with “r2 -p challenge2”.

Challenge 2 provides two files: an executable and a DOC based on the file extension. However, it appears the DOC has been encrypted. The challenge should be to decrypt this document file by reversing the method of encryption used by DudeLocker.exe.

$ file DudeLocker.exe BusinessPapers.doc
DudeLocker.exe:     PE32 executable for MS Windows (console) Intel 80386 32-bit
BusinessPapers.doc: data
$ xxd BusinessPapers.doc | head
00000000: 1bb9 6ebb 82c9 3d9a e290 fd8d c59a d40f  ..n...=.........
00000010: adb8 9376 511e 5f3a 37b5 d08a b5b8 02ae  ...vQ._:7.......
00000020: 503c 1926 676e 5ecd 48c4 3def d8ed 8ef1  P<.&gn^.H.=.....
00000030: 24af 5f4d 1c0c 7281 d583 a1a5 f729 5c2b  $._M..r......)\+

Running strings on the binary found a few interesting things. Obviously, it’s using a lot of Crypt* functions, which confirms its capabilities. There’s also JFIF, which could indicate a JPEG file, and there is a .JPG filename in the unicode strings. The unicode strings also show some things that may indicate anti-RE elements. Since OutputDebugString is imported, it could be that these are debug messages, rather than output directly to the console.

$ gstrings DudeLocker.exe  | more
...snip...
OutputDebugStringW
FindResourceW
...snip...
CryptAcquireContextW
CryptReleaseContext
CryptDeriveKey
CryptDestroyKey
CryptSetKeyParam
CryptGetKeyParam
CryptGetHashParam
CryptEncrypt
CryptCreateHash
CryptHashData
CryptDestroyHash
...snip...
JFIF
...snip...
$ gstrings -el DudeLocker.exe  | more
Briefcase
\ve_vant_ze_money.jpg
Obviously you're not a reverse engineer...
I'm out of my element

Looking at some info about the file in radare2, it looks like JFIF is in the .rsrc section, which would go along with the theory that it’s a JPEG stored in a resource.

$ r2 DudeLocker.exe
[0x00401b80]> aaaa
[0x00401b80]> izz~JFIF
vaddr=0x00404066 paddr=0x00001866 ordinal=075 sz=5 len=4 section=.rsrc type=ascii string=JFIF
[0x00401b80]> iS
[Sections]
idx=00 vaddr=0x00401000 paddr=0x00000400 sz=3072 vsz=2956 perm=m-r-x name=.text
idx=01 vaddr=0x00402000 paddr=0x00001000 sz=1536 vsz=1390 perm=m-r-- name=.rdata
idx=02 vaddr=0x00403000 paddr=0x00001600 sz=512 vsz=250 perm=m-rw- name=.data
idx=03 vaddr=0x00404000 paddr=0x00001800 sz=136192 vsz=135776 perm=m-r-- name=.rsrc

4 sections

Running the file, doesn’t produce any output. But if we run it with the Sysinternals DebugView open, we can see one of the output strings.

No output when run

DebugView shows debug string output

Radare puts us in the entry0 function, which is a small stub with a call to sub.SHELL32.dll_SHGetFolderPathW_9a0 based on the auto-analysis. We can go into thus function and get a quick overview of what it’s doing by looking at all the "calls" it makes.

[0x00401b80]> s sub.SHELL32.dll_SHGetFolderPathW_9a0
[0x004019a0]> pdf~call
│           0x004019e3      ff1594204000   call dword [sym.imp.SHELL32.dll_SHGetFolderPathW] ; "N%" @ 0x402094 ; get User Desktop path
│       │   0x004019f4      ff1538204000   call dword [sym.imp.KERNEL32.dll_lstrlenW] ; "x#" @ 0x402038
│       │   0x00401a1a      e8e1f5ffff     call sym.buildPathToBriefcase
│       │   0x00401a3b      ff1588204000   call dword [sym.imp.KERNEL32.dll_CreateFileW] ; sym.imp.KERNEL32.dll_CreateFileW
│      ││   0x00401a4e      ff1560204000   call dword [sym.imp.KERNEL32.dll_CloseHandle] ; "F#" @ 0x402060
│     │ │   0x00401a61      ff1540204000   call dword [sym.imp.KERNEL32.dll_OutputDebugStringW] ; sym.imp.KERNEL32.dll_OutputDebugStringW
│     └───> 0x00401a6e      e8cdf5ffff     call sym.checkVolumeSerialNumber ; compares SN to 0x7dab1d35
│     │││   0x00401a87      ff1540204000   call dword [sym.imp.KERNEL32.dll_OutputDebugStringW] ; sym.imp.KERNEL32.dll_OutputDebugStringW
│    │ ││   0x00401aa1      ff157c204000   call dword [sym.imp.KERNEL32.dll_GetProcessHeap] ; sym.imp.KERNEL32.dll_GetProcessHeap
│    │ ││   0x00401aa8      ff1584204000   call dword [sym.imp.KERNEL32.dll_HeapAlloc] ; sym.imp.KERNEL32.dll_HeapAlloc
│    │ ││   0x00401ac2      e879feffff     call sym.decryptKeyString   ; fills seedData
│    │ ││   0x00401ada      e8a1f5ffff     call sym.generateEncryptionKey
│   ││ ││   0x00401aff      e8fcf7ffff     call sym.encryptFilesInFolder
│   │││││   0x00401b1a      ff1534204000   call dword [sym.imp.KERNEL32.dll_lstrcatW] ; "l#" @ 0x402034
│   │││││   0x00401b27      e8f4f6ffff     call sym.writeRansomImage
│   │││││   0x00401b2f      e8bcfbffff     call sym.getVersion
│  ││││││   0x00401b44      e8d7fdffff     call sym.setParameterInfo
│   ││ ││   0x00401b52      ff157c204000   call dword [sym.imp.KERNEL32.dll_GetProcessHeap] ; sym.imp.KERNEL32.dll_GetProcessHeap
│   ││ ││   0x00401b59      ff1580204000   call dword [sym.imp.KERNEL32.dll_HeapFree] ; sym.imp.KERNEL32.dll_HeapFree
│   ││ ││   0x00401b67      e8e4f5ffff     call sym.cleanUpCrypto

This is after I had done my analysis on the file. The steps of the ransomware are as follows:

  1. Determine path to %USERPROFILE%\Desktop\Briefcase.
  2. If this folder doesn’t exist, print debug message “Obviously you’re not a reverse engineer…” and exit.
  3. Compare the hard drive serial number to 0x7dab1d35
  4. If the serial number doesn’t match, print debug message “I’m out of my element” and exit.
  5. Using the hard drive serial number, decrypt 37 bytes of data at 0x403000.
  6. Generate an AES encryption key using the above decrypted string.
  7. Encrypt each file in the folder.
  8. Write the JPEG resource to the “Briefcase” folder and set it as the Desktop wallpaper.

Step 5, where the byte array is decrypted does the following. It basically divides the counter by 4 and uses the remainder to choose one of the bytes of the volume serial number, then XORs the current byte in the string with that value.

│      ││   0x0040195f      8b5508         mov edx, dword [ebp + encryptedKeyString] ; [0x8:4]=4
│      ││   0x00401962      0355fc         add edx, dword [ebp - counter]
│      ││   0x00401965      0fb60a         movzx ecx, byte [edx]       ; get current byte
│      ││   0x00401968      8b45fc         mov eax, dword [ebp - counter]
│      ││   0x0040196b      33d2           xor edx, edx
│      ││   0x0040196d      be04000000     mov esi, 4
│      ││   0x00401972      f7f6           div esi                     ; divide counter by 4
│      ││   0x00401974      8b4510         mov eax, dword [ebp + volumeSerialNumber] ; [0x10:4]=184
│      ││   0x00401977      0fb61410       movzx edx, byte [eax + edx] ; select a byte from VSN based on remainder or division
│      ││   0x0040197b      33ca           xor ecx, edx                ; XOR current byte with above value
│      ││   0x0040197d      8b450c         mov eax, dword [ebp + seedData] ; [0xc:4]=0xffff
│      ││   0x00401980      0345fc         add eax, dword [ebp - counter]
│      ││   0x00401983      8808           mov byte [eax], cl          ; store result in seedData
│      └──< 0x00401985      ebc7           jmp sym.incrementCounter

The byte array below gets decrypted to “thosefilesreallytiedthefoldertogether” and is stored in “seedData”. I was going to write a Python version of this, but figured it would be much easier to just run DudeLocker.exe in a debugger and break right after the decryption function was called.

[0x00401940]> px 37 @  0x403000
- offset -   0 1  2 3  4 5  6 7  8 9  A B  C D  E F  0123456789ABCDEF
0x00403000, 4175 c40e 507b c211 506e d918 5471 c704  Au..P{..Pn..Tq..
0x00403010, 4174 ce19 4175 ce1b 5a71 cf18 4769 c41a  At..Au..Zq..Gi..
0x00403020, 5069 c318 47                             Pi..G

Before sym.decryptKeyString:

before decrypting

After sym.decryptKeyString:

after decrypting

In Step 6, the CBC mode AES key object is generated. This uses the SHA1 hash of the decrypted string to generate the key.

[0x00401580]> s sym.generateEncryptionKey
[0x00401080]> pdf~call
│           0x00401097      ff151c204000   call dword [sym.imp.ADVAPI32.dll_CryptAcquireContextW] ; "l$" @ 0x40201c
│       │   0x004010a1      ff1578204000   call dword [sym.imp.KERNEL32.dll_GetLastError] ; sym.imp.KERNEL32.dll_GetLastError
│      ││   0x004010ba      ff151c204000   call dword [sym.imp.ADVAPI32.dll_CryptAcquireContextW] ; "l$" @ 0x40201c
│    │ │    0x004010e5      e896000000     call sym.deriveAESKey
│    ││││   0x00401104      ff1518204000   call dword [sym.imp.ADVAPI32.dll_CryptReleaseContext] ; sym.imp.ADVAPI32.dll_CryptReleaseContext
│    │││    0x0040111c      ff150c204000   call dword [sym.imp.ADVAPI32.dll_CryptSetKeyParam] ; sym.imp.ADVAPI32.dll_CryptSetKeyParam
│    ││││   0x00401132      e819000000     call sym.cleanUpCrypto
[0x00401080]> s sym.generateEncryptionKey
[0x00401080]> pdf
╒ (fcn) sym.generateEncryptionKey 196
│   sym.generateEncryptionKey (int phKey, int phProv, int seedData, int seedDataLength);
│           ; var int cipher_mode_CBC @ ebp-0x4
│           ; arg int phKey @ ebp+0x8
│           ; arg int phProv @ ebp+0xc
│           ; arg int seedData @ ebp+0x10
│           ; arg int seedDataLength @ ebp+0x14
│           ; CALL XREF from 0x00401ada (sym.main_start)
│           0x00401080      55             push ebp
│           0x00401081      8bec           mov ebp, esp
│           0x00401083      51             push ecx
│           0x00401084      c745fc010000.  mov dword [ebp - cipher_mode_CBC], 1
│           0x0040108b      6a00           push 0
│           0x0040108d      6a18           push 0x18                   ; "@" ; PROV_RSA_AES
│           0x0040108f      6a00           push 0
│           0x00401091      6a00           push 0
│           0x00401093      8b450c         mov eax, dword [ebp + phProv] ; [0xc:4]=0xffff
│           0x00401096      50             push eax
│           0x00401097      ff151c204000   call dword [sym.imp.ADVAPI32.dll_CryptAcquireContextW] ; "l$" @ 0x40201c
...snip...
│   └───└─> 0x004010ce      8b5514         mov edx, dword [ebp + seedDataLength] ; [0x14:4]=0
│    │ │    0x004010d1      52             push edx
│    │ │    0x004010d2      8b4510         mov eax, dword [ebp + seedData] ; [0x10:4]=184
│    │ │    0x004010d5      50             push eax
│    │ │    0x004010d6      6810660000     push 0x6610                 ; CALG_AES_256
│    │ │    0x004010db      8b4d0c         mov ecx, dword [ebp + phProv] ; [0xc:4]=0xffff
│    │ │    0x004010de      8b11           mov edx, dword [ecx]
│    │ │    0x004010e0      52             push edx
│    │ │    0x004010e1      8b4508         mov eax, dword [ebp + phKey] ; [0x8:4]=4
│    │ │    0x004010e4      50             push eax
│    │ │    0x004010e5      e896000000     call sym.deriveAESKey
...snip...
│    │││└─> 0x0040110e      6a00           push 0
│    │││    0x00401110      8d55fc         lea edx, [ebp - cipher_mode_CBC]
│    │││    0x00401113      52             push edx
│    │││    0x00401114      6a04           push 4                      ; KP_MODE
│    │││    0x00401116      8b4508         mov eax, dword [ebp + phKey] ; [0x8:4]=4
│    │││    0x00401119      8b08           mov ecx, dword [eax]
│    │││    0x0040111b      51             push ecx
│    │││    0x0040111c      ff150c204000   call dword [sym.imp.ADVAPI32.dll_CryptSetKeyParam] ; sym.imp.ADVAPI32.dll_CryptSetKeyParam
...snip...

[0x00401080]> s sym.deriveAESKey
[0x00401180]> pdf~call
│       │   0x004011ac      ff1500204000   call dword [sym.imp.ADVAPI32.dll_CryptCreateHash] ; sym.imp.ADVAPI32.dll_CryptCreateHash
│    ││ │   0x004011d4      ff1520204000   call dword [sym.imp.ADVAPI32.dll_CryptHashData] ; sym.imp.ADVAPI32.dll_CryptHashData
│    ││││   0x004011e2      ff1524204000   call dword [sym.imp.ADVAPI32.dll_CryptDestroyHash] ; ",%" @ 0x402024
│   │││ │   0x004011ff      ff1514204000   call dword [sym.imp.ADVAPI32.dll_CryptDeriveKey] ; sym.imp.ADVAPI32.dll_CryptDeriveKey
│   │││ │   0x00401211      ff1524204000   call dword [sym.imp.ADVAPI32.dll_CryptDestroyHash] ; ",%" @ 0x402024
[0x00401180]> pdf
╒ (fcn) sym.deriveAESKey 158
│   sym.deriveAESKey (int encryptionKey, int pointerToCryptoServProv, int algId_AES256, int seedData, int seedDataLength);
│           ; var int hHash @ ebp-0x8
│           ; var int deriveSuccess @ ebp-0x1
│           ; var int local_0h @ ebp-0x0
│           ; arg int encryptionKey @ ebp+0x8
│           ; arg int pointerToCryptoServProv @ ebp+0xc
│           ; arg int algId_AES256 @ ebp+0x10
│           ; arg int seedData @ ebp+0x14
│           ; arg int seedDataLength @ ebp+0x18
│           ; CALL XREF from 0x004010e5 (sym.generateEncryptionKey)
│           0x00401180      55             push ebp
│           0x00401181      8bec           mov ebp, esp
│           0x00401183      83ec08         sub esp, 8
...snip...
|      └──> ;-- sym.createHash:
│      └──> 0x0040119b      8d45f8         lea eax, [ebp - hHash]
│       │   0x0040119e      50             push eax
│       │   0x0040119f      6a00           push 0
│       │   0x004011a1      6a00           push 0
│       │   0x004011a3      6804800000     push 0x8004                 ; CALG_SHA1
│       │   0x004011a8      8b4d0c         mov ecx, dword [ebp + pointerToCryptoServProv] ; [0xc:4]=0xffff
│       │   0x004011ab      51             push ecx
│       │   0x004011ac      ff1500204000   call dword [sym.imp.ADVAPI32.dll_CryptCreateHash] ; sym.imp.ADVAPI32.dll_CryptCreateHash
...snip...
|    ││└──> ;-- sym.hashSeedData:
│    ││└──> 0x004011c6      6a00           push 0
│    ││ │   0x004011c8      8b5518         mov edx, dword [ebp + seedDataLength] ; [0x18:4]=64 ; "@"
│    ││ │   0x004011cb      52             push edx
│    ││ │   0x004011cc      8b4514         mov eax, dword [ebp + seedData] ; [0x14:4]=0
│    ││ │   0x004011cf      50             push eax
│    ││ │   0x004011d0      8b4df8         mov ecx, dword [ebp - hHash]
│    ││ │   0x004011d3      51             push ecx
│    ││ │   0x004011d4      ff1520204000   call dword [sym.imp.ADVAPI32.dll_CryptHashData] ; sym.imp.ADVAPI32.dll_CryptHashData
...snip...
|   │││└──> ;-- sym.deriveKeyUsingHash:
│   │││└──> 0x004011ed      8b4508         mov eax, dword [ebp + encryptionKey] ; [0x8:4]=4
│   │││ │   0x004011f0      50             push eax
│   │││ │   0x004011f1      6a01           push 1                      ; CRYPT_EXPORTABLE
│   │││ │   0x004011f3      8b4df8         mov ecx, dword [ebp - hHash]
│   │││ │   0x004011f6      51             push ecx
│   │││ │   0x004011f7      8b5510         mov edx, dword [ebp + algId_AES256] ; [0x10:4]=184
│   │││ │   0x004011fa      52             push edx
│   │││ │   0x004011fb      8b450c         mov eax, dword [ebp + pointerToCryptoServProv] ; [0xc:4]=0xffff
│   │││ │   0x004011fe      50             push eax
│   │││ │   0x004011ff      ff1514204000   call dword [sym.imp.ADVAPI32.dll_CryptDeriveKey] ; sym.imp.ADVAPI32.dll_CryptDeriveKey

In Step 7, each file in the “Briefcase” folder is iterated over, and folders are entered recursively. For each file, the filename is converted to lowercase then from WideChar to MultiByte, then the MD5 of this format of the filename is taken. This MD5 hash is used as the initialization vector for encrypting the file.

│   ││└───> 0x00401af0      8d95c0fdffff   lea edx, [ebp - str_pathToBriefcase]
│   ││ ││   0x00401af6      52             push edx
│   ││ ││   0x00401af7      8d45ec         lea eax, [ebp - ptr_CryptoServiceProvider]
│   ││ ││   0x00401afa      50             push eax
│   ││ ││   0x00401afb      8d4de8         lea ecx, [ebp - encryptionKey]
│   ││ ││   0x00401afe      51             push ecx
│   ││ ││   0x00401aff      e8fcf7ffff     call sym.encryptFilesInFolder
[0x00401aa0]> s sym.encryptFilesInFolder
[0x00401300]> pdf~call
│           0x0040132b      ff1530204000   call dword [sym.imp.KERNEL32.dll_lstrcpyW] ; "`#" @ 0x402030
│           0x0040133c      ff1534204000   call dword [sym.imp.KERNEL32.dll_lstrcatW] ; "l#" @ 0x402034
│           0x00401350      ff154c204000   call dword [sym.imp.KERNEL32.dll_FindFirstFileW] ; sym.imp.KERNEL32.dll_FindFirstFileW
│     ││    0x00401379      ff1548204000   call dword [sym.imp.KERNEL32.dll_lstrcpynW] ; "T#" @ 0x402048
│     ││    0x00401394      e8f7050000     call sym.checkIfFolder      ; checks dwAttributes of FindFileData to indicate a directory
│    ││││   0x004013ca      e831fcffff     call sym.buildPathToBriefcase
│    ││││   0x004013e1      e81affffff     call sym.encryptFilesInFolder ; recursively encrypt files in subdirs
│   │ ││    0x00401403      e888050000     call sym.checkIfFolder
│   │ │││   0x00401428      e8d3fbffff     call sym.buildPathToBriefcase
│   │ │││   0x00401437      ff1538204000   call dword [sym.imp.KERNEL32.dll_lstrlenW] ; "x#" @ 0x402038
│   │ │││   0x0040144a      ff159c204000   call dword [sym.imp.USER32.dll_CharLowerW] ; ":$" @ 0x40209c
│   │ │││   0x00401456      ff157c204000   call dword [sym.imp.KERNEL32.dll_GetProcessHeap] ; sym.imp.KERNEL32.dll_GetProcessHeap
│   │ │││   0x0040145d      ff1584204000   call dword [sym.imp.KERNEL32.dll_HeapAlloc] ; sym.imp.KERNEL32.dll_HeapAlloc
│   │││││   0x00401474      e8b7020000     call sym.zeroOutBuffer
│   │ │││   0x00401497      ff155c204000   call dword [sym.imp.KERNEL32.dll_WideCharToMultiByte] ; sym.imp.KERNEL32.dll_WideCharToMultiByte
│   │ │││   0x004014ad      e8be020000     call sym.setIVToFilenameMD5
│   │││││   0x004014cb      e830000000     call sym.encryptFile
│ │   ││    0x004014e9      ff1550204000   call dword [sym.imp.KERNEL32.dll_FindNextFileW] ; sym.imp.KERNEL32.dll_FindNextFileW
[0x00401500]> s sym.setIVToFilenameMD5
[0x00401770]> pdf~call
│           0x0040179d      ff1508204000   call dword [sym.imp.ADVAPI32.dll_CryptGetKeyParam] ; sym.imp.ADVAPI32.dll_CryptGetKeyParam ; get block length in bits
│       │   0x004017b3      e898f9ffff     call sym.cleanUpCrypto
│      │    0x004017d1      ff157c204000   call dword [sym.imp.KERNEL32.dll_GetProcessHeap] ; sym.imp.KERNEL32.dll_GetProcessHeap
│      │    0x004017d8      ff1584204000   call dword [sym.imp.KERNEL32.dll_HeapAlloc] ; sym.imp.KERNEL32.dll_HeapAlloc
│      ││   0x004017f7      e854f9ffff     call sym.cleanUpCrypto
│     ││    0x00401810      e81bffffff     call sym.zeroOutBuffer
│     ││    0x0040182b      ff1500204000   call dword [sym.imp.ADVAPI32.dll_CryptCreateHash] ; sym.imp.ADVAPI32.dll_CryptCreateHash
│   ││││    0x00401887      ff1520204000   call dword [sym.imp.ADVAPI32.dll_CryptHashData] ; sym.imp.ADVAPI32.dll_CryptHashData
│   │││││   0x00401895      ff1524204000   call dword [sym.imp.ADVAPI32.dll_CryptDestroyHash] ; ",%" @ 0x402024
│  │││││    0x004018b8      ff1504204000   call dword [sym.imp.ADVAPI32.dll_CryptGetHashParam] ; sym.imp.ADVAPI32.dll_CryptGetHashParam
│ ││││││    0x004018d6      ff150c204000   call dword [sym.imp.ADVAPI32.dll_CryptSetKeyParam] ; sym.imp.ADVAPI32.dll_CryptSetKeyParam
│ │││││││   0x004018e8      ff157c204000   call dword [sym.imp.KERNEL32.dll_GetProcessHeap] ; sym.imp.KERNEL32.dll_GetProcessHeap
│ │││││││   0x004018ef      ff1580204000   call dword [sym.imp.KERNEL32.dll_HeapFree] ; sym.imp.KERNEL32.dll_HeapFree
│ │││││││   0x00401901      e84af8ffff     call sym.cleanUpCrypto
│ ││││││    0x00401911      ff1524204000   call dword [sym.imp.ADVAPI32.dll_CryptDestroyHash] ; ",%" @ 0x402024
[0x00401770]> s sym.encryptFile
[0x00401500]> pdf~call
│           0x0040153f      ff1508204000   call dword [sym.imp.ADVAPI32.dll_CryptGetKeyParam] ; sym.imp.ADVAPI32.dll_CryptGetKeyParam ; getBlockLength
│       │   0x00401555      e8f6fbffff     call sym.cleanUpCrypto
│      │    0x004015ae      ff1588204000   call dword [sym.imp.KERNEL32.dll_CreateFileW] ; sym.imp.KERNEL32.dll_CreateFileW
│     ││    0x004015ca      ff156c204000   call dword [sym.imp.KERNEL32.dll_GetFileSize] ; " #" @ 0x40206c
│    │││    0x004015f4      ff1588204000   call dword [sym.imp.KERNEL32.dll_CreateFileW] ; sym.imp.KERNEL32.dll_CreateFileW
│   ││││    0x00401611      ff157c204000   call dword [sym.imp.KERNEL32.dll_GetProcessHeap] ; sym.imp.KERNEL32.dll_GetProcessHeap
│   ││││    0x00401618      ff1584204000   call dword [sym.imp.KERNEL32.dll_HeapAlloc] ; sym.imp.KERNEL32.dll_HeapAlloc
│  ││││││   0x0040163f      ff1564204000   call dword [sym.imp.KERNEL32.dll_ReadFile] ; ":#" @ 0x402064
│  ││││││   0x00401672      ff1528204000   call dword [sym.imp.ADVAPI32.dll_CryptEncrypt] ; sym.imp.ADVAPI32.dll_CryptEncrypt
│  ││││││   0x00401690      ff1568204000   call dword [sym.imp.KERNEL32.dll_WriteFile] ; ".#" @ 0x402068
│     │││   0x004016b2      ff1560204000   call dword [sym.imp.KERNEL32.dll_CloseHandle] ; "F#" @ 0x402060
│     │││   0x004016c2      ff1560204000   call dword [sym.imp.KERNEL32.dll_CloseHandle] ; "F#" @ 0x402060
│     │││   0x004016d4      ff157c204000   call dword [sym.imp.KERNEL32.dll_GetProcessHeap] ; sym.imp.KERNEL32.dll_GetProcessHeap
│     │││   0x004016db      ff1580204000   call dword [sym.imp.KERNEL32.dll_HeapFree] ; sym.imp.KERNEL32.dll_HeapFree

Step 8 writes the following image to “Briefcase” and sets it as the Desktop wallpaper:

ransom note

This level of analysis was overkill for this challenge, but I wanted to get more practice with Radare2, so I tried to document everything I could in the EXE since it was pretty straightforward. In the end, to do the decryption, we need to create an AES key using the SHA1 hash of “thosefilesreallytiedthefoldertogether” as the shared secret and the MD5 hash of the lowercase of the filename as the initialization vector.

I tried using the built in python Cipher libraries, but they only support a key length of 16, 24, and 32 bytes, and a SHA1 hash is 20 bytes. I found the wincrypto library. The builtin decrypt function uses an IV = ‘\0’ * 16 so I had to call it manually. This was the python script I ended up using.

#!/usr/bin/python

from Crypto.Cipher import AES
from wincrypto import CryptCreateHash, CryptHashData, CryptDeriveKey
from wincrypto.constants import CALG_SHA1, CALG_AES_256
import hashlib
import binascii
import sys

# read in the file
filename = sys.argv[1]
f = open(filename, 'r')
data_enc = f.read()

# decrypt the secret used for the AES key
key_str_enc = "4175c40e507bc211506ed9185471c7044174ce194175ce1b5a71cf184769c41a5069c31847"
key_str_enc = binascii.a2b_hex(key_str_enc)
vsn = "351dab7d"
vsn = binascii.a2b_hex(vsn)
key_str_dec = ""

# divide the counter by 4 and use the remainder to determine which byte of the VSN to use for XOR
for i in range(0, len(key_str_enc)):
    remain = (i % 4)
    key_str_dec += chr(ord(key_str_enc[i]) ^ ord(vsn[remain]))

# hash secret and create the key
sha1_hasher = CryptCreateHash(CALG_SHA1)
CryptHashData(sha1_hasher, key_str_dec)
aes_key = CryptDeriveKey(sha1_hasher, CALG_AES_256)

# get the hash of the filename to create the IV
m = hashlib.md5()
m.update(filename.lower())
iv = m.hexdigest()
iv = binascii.a2b_hex(iv)

# decrypt the file data
data_dec = AES.new(aes_key.key, mode=AES.MODE_CBC, IV=iv).decrypt(data_enc)

# write out the file
fo = open(filename + ".out", 'wb')
fo.write(data_dec)
fo.close()
$ python DudeUnlocker.py BusinessPapers.doc
$ file BusinessPapers.doc*
BusinessPapers.doc:     data
BusinessPapers.doc.out: JPEG image data, JFIF standard 1.01

challenge 2 answer

I was going to share my radare2 project file, but somehow it got corrupted and lost all my work. Luckily, I had r2 open in another window and was able to save a working copy of the project from that one. Definitely learned to make backup copies based on this experience. Also, don’t hit Ctrl-D to exit out. It wrecks the project file.

by

2016 FLARE-On Challenge 1

No comments yet

Categories: Challenges, Reverse Engineering

I didn’t get to spend a lot of time on the FLARE-On challenges this year, but I worked through the first 2 and used it as an opportunity to learn Radare2 a little better.

Here’s my solution to challenge 1.

$ md5 challenge1.exe
MD5 (challenge1.exe) = 2caaa4aa5923d026b17d7b38ee410918

I ran strings on the file to see if there was anything interesting. The ones below stood out.

$ gstrings -n 48 challenge1.exe

x2dtJEOmyjacxDemx2eczT5cVS9fVUGvWTuZWjuexjRqy24rV29q
Enter password:
Correct!
Wrong password

ZYXABCDEFGHIJKLMNOPQRSTUVWzyxabcdefghijklmnopqrstuvw0123456789+/

It looks like the program prompts for a password and will output whether it was correct. The first string looks like some encoded data. We might first guess it’s base64 encoded, but, not so much:

$ echo "x2dtJEOmyjacxDemx2eczT5cVS9fVUGvWTuZWjuexjRqy24rV29q" | base64 -D
�gm$C��6��7��g��>\U/_UA�Y;�Z;��4j�n+Woj

However, that last string is interesting. It looks like all the characters in a typical base64 alphabet, but it starts with XYZ instead of ABC, so it looks like this may be base64 using a custom alphabet.

Before testing that, I did take a look at the file in radare2, since like I mentioned in the summary, I wanted to use these challenges as a way to learn radare. There may be better ways to do what I was doing, and I’m definitely open to suggestions.

$ r2 challenge1.exe
[0x0040170d]> aaaa
[0x0040170d]> s sym.main
[0x00000000]> pdf
Cannot find function at 0x00000000

Typically, after analyzing, I run “s sym.main” but it wasn’t found with the analysis. I knew the output strings would be near the important code, so I ran:

$ r2 challenge1.exe
[0x0040170d]> aaaa
[0x0000bfac]> s/ Correct
Searching 7 bytes from 0x0000bfad to 0x00417e64: 43 6f 72 72 65 63 74
Searching 7 bytes in [0xbfad-0x417e64]
hits: 1  hit1_0 .. hit1_0
0x0040d1ac hit1_0 .r password:Correct!Wrong passw.
[0x0040d1ac]> pd 20
;-- str.Correct__r_n:
; DATA XREF from 0x004014ae (sub.KERNEL32.dll_GetStdHandle_420)
0x0040d1ac     .string "Correct!\\r\\n" ; len=11

[0x0040d1ac]> s 0x4014ae
[0x004014ae]> pd 10
│           0x004014ae      68acd14000     push str.Correct__r_n ; str.Correct__r_n ; "Correct!.." @ 0x40d1ac
│           0x004014b3      8b4df8         mov ecx, dword [ebp - local_8h]
│           0x004014b6      51             push ecx
│           0x004014b7      ff15a0d04000   call dword [sym.imp.KERNEL32.dll_WriteFile] ; sym.imp.KERNEL32.dll_WriteFile
│       ┌─< 0x004014bd      eb17           jmp 0x4014d6
│       │   0x004014bf      6a00           push 0
│       │   0x004014c1      8d55fc         lea edx, [ebp - local_4h]
│       │   0x004014c4      52             push edx
│       │   0x004014c5      6a11           push 0x11
│       │   0x004014c7      68b8d14000     push str.Wrong_password_r_n ; str.Wrong_password_r_n ; "Wrong password.." @ 0x40d1b8
[0x004014ae]> s sub.KERNEL32.dll_GetStdHandle_420
[0x00401420]> pdf
╒ (fcn) sub.KERNEL32.dll_GetStdHandle_420 188
│   sub.KERNEL32.dll_GetStdHandle_420 ();
│           ; var int local_94h @ ebp-0x94
│           ; var int local_14h @ ebp-0x14
│           ; var int local_10h @ ebp-0x10
│           ; var int local_ch @ ebp-0xc
│           ; var int local_8h @ ebp-0x8
│           ; var int local_4h @ ebp-0x4
│           ; CALL XREF from 0x0040168b (entry0)
│           0x00401420      55             push ebp
│           0x00401421      8bec           mov ebp, esp
│           0x00401423      81ec94000000   sub esp, 0x94
│           0x00401429      6af5           push -0xb
│           0x0040142b      ff1510d14000   call dword [sym.imp.KERNEL32.dll_GetStdHandle] ; sym.imp.KERNEL32.dll_GetStdHandle
│           0x00401431      8945f8         mov dword [ebp - local_8h], eax

So this looks like the function we want. I switched over to visual mode to get a look at the control flow by entering “VV” on the console and hitting enter, then hitting “p” until I got to the BB-SUMM screen:

[0x00401420]> VV @ sub.KERNEL32.dll_GetStdHandle_420 (nodes 4 edges 4 zoom 100%) BB-SUMM mouse:canvas-y movements-speed:5



                             =------------------------------------------------------------=
                             | [0x401420]                                                 |
                             | 0x0040142b call dword [sym.imp.KERNEL32.dll_GetStdHandle]  |
                             | 0x00401436 call dword [sym.imp.KERNEL32.dll_GetStdHandle]  |
                             | 0x0040144e str.Enter_password:_r_n                         |
                             | 0x00401457 call dword [sym.imp.KERNEL32.dll_WriteFile]     |
                             | 0x00401473 call dword [sym.imp.KERNEL32.dll_ReadFile]      |
                             | 0x00401487 call fcn.00401260                               |
                             | 0x0040149a call fcn.00402c30                               |
                             =------------------------------------------------------------=
                                   t f
     .-----------------------------' '---------------------------------.
     |                                                                 |
     |                                                                 |
---------------------------------------------------------=     =---------------------------------------------------------=
  0x4014bf                                               |     |  0x4014a6                                               |
 0x004014c7 str.Wrong_password_r_n                       |     | 0x004014ae str.Correct__r_n                             |
 0x004014d0 call dword [sym.imp.KERNEL32.dll_WriteFile]  |     | 0x004014b7 call dword [sym.imp.KERNEL32.dll_WriteFile]  |
---------------------------------------------------------=     =---------------------------------------------------------=
   v                                                               v
   '-------------------------------------------------.-------------'
                                                     |
                                                     |
                                                 =--------------------=
                                                 |  0x4014d6          |
                                                 =--------------------=

It looks like handles to STDIN and STDOUT are created, the prompt is printed, input is read, and then processed in the two calls to determine if it was successful or not.

I looked over the code and determined the argument passed in (arg_ch) was the password length, which was divided by 3 and multiplied by 4 (because every 3 ASCII characters turn into 4 characters in the base64 output). The base64 alphabet from the strings output is also referenced a lot in this function, so that pretty much confirmed what was going on.

[0x00401420]> s 0x401260
[0x00401260]> pdf
╒ (fcn) fcn.00401260 448
│   fcn.00401260 (int arg_8h, int arg_ch);
│           ; CALL XREF from 0x00401487 (sub.KERNEL32.dll_GetStdHandle_420)
│           0x00401260      55             push ebp
│           0x00401261      8bec           mov ebp, esp
│           0x00401263      83ec30         sub esp, 0x30               ; '0'
│           0x00401266      8b450c         mov eax, dword [ebp + arg_ch] ; [0xc:4]=0xffff
│           0x00401269      83c002         add eax, 2
│           0x0040126c      33d2           xor edx, edx
│           0x0040126e      b903000000     mov ecx, 3
│           0x00401273      f7f1           div ecx
│           0x00401275      8d1485010000.  lea edx, [eax*4 + 1]        ; 0x1 ; "Z."
│           0x0040127c      8955e8         mov dword [ebp - local_18h], edx
│           0x0040127f      8b45e8         mov eax, dword [ebp - local_18h]
│           0x00401282      50             push eax
│           0x00401283      e8341a0000     call sub.KERNEL32.dll_HeapAlloc_cbc

I found a website that allowed you to enter a custom base64 alphabet, and got the resulting key for challenge1: sh00ting_phish_in_a_barrel@flare-on.com.

Base64 Decode with custom alphabet