Thursday, December 3, 2015

Week of Continuous Intrusion Tools - Day 4 - Common Abuse Set, Lateral Movement and Post Exploitation

Welcome to Day 4 of Week of Continuous Intrusion tools. We are discussing security of Continuous Integration (CI) tools in this series of blog posts.

Day 1 - Jenkins (and Hudson) (Click Here)
Day 2 - TeamCity (Click Here)
Day 3 - Go and CruiseControl (Click Here)
Day 4 - Common Abuse Set, Lateral Movement and Post Exploitation
Day 5 - Defense and other discussion (Click Here)

Day 4 is dedicated to Common abuse set, Lateral movement and Post exploitation.

In the past three days, we discussed how different attacks can be executed against Jenkins (and Hudson), TeamCity, (very briefly) CruiseControl and Go. Some of the readers might have noticed that many attacks looked common and were result of mis-configurations, lack of common security controls and/ or abuse of features. Lets pick things common to the tools we discussed and make a Common Abuse Set out of them.

Common Abuse Set

From the previous posts, the Common Abuse Set for the CI Tools we saw turns out to be:
  • Missing basic and common security controls
    • Missing protections against brute force attempts.
    •  Insecure storage of SSH keys and credentials.
    • Higher privileges on Windows machines for both master and slaves.
  •  The feature of Command Execution at the Operating System level.
  • Mis-Configuration
    • Agent on Master
    • Read permissions to everyone on public instances.
    • Use of HTTP for login
    • Not enabling encrypted communication between master and slaves.
  • Poor Security practices by users
    • Passwords in build parameters.
    • Use of username as password specially in case of users local to a CI tool. 
  • Many public instances of these tools
     Lets have a quick look at some of them.

     (Missing) Security Controls

    1. Authentication

    CI tools were found to missing even the most basic security control like protection against Brute Force attacks. In fact, Jenkins and Go have no authentication at all in the default installation. If you are following me this blog for past three days, you will find that it is trivial to find instances of these tools on the internet running with the default configurations. This highlights the state of security for these tools.Not many enterprise tools miss these basic controls.

    CI Tool Jenkins/Hudson Teamcity Go
    - Login attempts
    -Password Policy
    - No Authentication by Default
    - No protection against Brute Force attacks in the recommended Matrix based Authorization
    - No captcha
    - No Password Policy (Complexity, History, Expire time etc.)
    - Guest User can be enabled
    - Registration enabled by default
    - Wait after five failed login attempts in one minute
    - No captcha
    - No Password Policy (Complexity, History, Expire time etc.)
    - No Authentication by Default
    - No protection against Brute Force attacks.
    - No captcha
    - No Password Policy (Complexity, History, Expire time etc.)

    2. Insecure Storage of Credentials/SSH Keys
    All the tested CI tools store all or some credentials and SSH Keys in insecure format. All of them store SSH Keys in clear text and encrypted credentials from Jenkins could be retrieved in clear text. Its amazing that these tools still do this.

    3. Privileges
    All the tested CI tools run with either SYSTEM or admin privileges on Windows. This holds true for both masters and slaves/agents. This makes the command execution access much more fruitful from an attacker's view.

    Command Execution

    The feature of CI tools which allow execution of Operating System commands by adding build steps is what makes them special. In most of the widely used enterprise tools, the ability to execute OS commands is uncommon. This ability makes CI tools a useful target. Add to it the capability of distributed builds and by compromising the master an attacker can execute commands on large number of slaves.


    Agent on master
    Documentation of all the tested CI tools do not recommend having a build executor or agent on master. Still, Jenkins install it by default and TeamCity provides it in the same installation package. Only Go needs a user to download a separate installation for agent on master. We have already seen that an agent on master makes all security useless.

    Lateral Movement and Post Exploitation

    The kind of access we have with CI tools makes it possible to do much more interesting stuff in a network other than just a reverse shell. 

    Domain Admin

    Because CI Tools we discussed support distributed builds, in an enterprise environment it is quite possible to spot machines (master and/or slave) where a credentials of a high privileged user like a Common Local Admin or a Domain Admin are available. Note that even if master runs on *nix, there are almost always slaves running on Windows. 

    Lets assume this scenario. We have access to Jenkins or any other tool and the ability to configure builds on many slaves and one of the slaves has a process running as Domain Administrator.  We can use PowerShell (and other tools as well) to enumerate and reuse the token and escalate to Domain Admin. We can use Invoke-TokenManipulation from PowerSploit for enumeration and impersonation.We can use the below command in a Jenkins build step for downloading and executing the script in memory:
    Since the Jenkins runs as SYSTEM, this will list all the available tokens.
    Note that there is a token for the Domain Administrator. Now we can use the below command in Jenkins build step to run Invoke-TokenManipulation in memory, impersonate the token of Domain Administrator and run the Get-Process cmdlet on the Domain Controller.
    And the result looks like:
    Awesome! We just executed command on the domain controller as a domain admin. Too easy? Try it in an environment where you are authorized to do so and get pleasantly surprised ;)
    Please note that we assumed that the enumeration of name of Domain Admins and the Domain Controller was done already (which is trivial). Also, even if we cannot find a privileged user on any of the slaves, we can always try querying other machines in the network from the slave machines we have access to ;) Note that while querying other machines in a domain we must impersonate a domain user on the slave machine to be able to interact with Active Directory.

    Linux machines

    While testing the CI tools, we regularly got hands on SSH keys. These SSH keys could be used to access version control and Linux hosts.

    Lets assume this scenario. We got access to a Jenkins instance. We can retrieve and use SSH key to login to a Linux machine (root or normal user depending upon the keys). As we saw in Day 1, SSH keys in Jenkins are stored in clear either in $JENKINS_HOME or credentials.xml. We can also retrieve the passphrase. More than often, we will be able to login to a large number of slaves.

    Lets read credentials.xml and see if there are any private SSH keys there.
    Seems like there is a private key for a user named "ubuntuadmin" which has passphrase (encrypted). We can retreive the pass phrase using the method discussed in Day 1. Now, the only missing part is on which machine the key could be used.  For that, either we need to see build logs to find any logs regarding this key use or simply try it on all the Linux machines available.

    Also, to use the key with Putty, we need to convert it in putty format using puttygen.
    And then:
    Neat! It just depends on the kind of confugration and usage of CI Tools in the target network. Source code repositories, version control systems and databases are also often accessible after compromising a CI tool.

    Video Demonstration

    Hope you enjoyed the post! Feedback and comments are welcome :)

    To support my research, join me for a two days training "PowerShell for Penetration Testers" at:

    BlackHat, Asia (March 29-30th, 2016) -

    HITB, Amsterdam (May 24-25th, 2016) -

    No comments:

    Post a Comment