www.darkreading.com Open in urlscan Pro
2606:4700::6812:6f2f  Public Scan

Form analysis
0 forms found in the DOM

Text Content

Dark Reading is part of the Informa Tech Division of Informa PLC
Informa PLC|ABOUT US|INVESTOR RELATIONS|TALENT
This site is operated by a business or businesses owned by Informa PLC and all
copyright resides with them. Informa PLC's registered office is 5 Howick Place,
London SW1P 1WG. Registered in England and Wales and Scotlan. Number 8860726.

Black Hat NewsOmdia Cybersecurity

Newsletter Sign-Up

Newsletter Sign-Up

Cybersecurity Topics

RELATED TOPICS

 * Application Security
 * Cybersecurity Careers
 * Cloud Security
 * Cyber Risk
 * Cyberattacks & Data Breaches
 * Cybersecurity Analytics
 * Cybersecurity Operations
 * Data Privacy
 * Endpoint Security
 * ICS/OT Security

 * Identity & Access Mgmt Security
 * Insider Threats
 * IoT
 * Mobile Security
 * Perimeter
 * Physical Security
 * Remote Workforce
 * Threat Intelligence
 * Vulnerabilities & Threats


World

RELATED TOPICS

 * DR Global

 * Middle East & Africa

See All
The Edge
DR Technology
Events

RELATED TOPICS

 * Upcoming Events

 * Webinars

SEE ALL
Resources

RELATED TOPICS

 * Library
 * Newsletters
 * Reports
 * Videos
 * Webinars
 * Whitepapers
 * 

 * 
 * 
 * 
 * Partner Perspectives:
 * > Google Cloud
 * > Microsoft

SEE ALL


Sponsored By

 * Сloud Security
 * Cybersecurity Careers

Cybersecurity In-Depth: Feature articles on security strategy, latest trends,
and people to know.




7 LESSONS LEARNED FROM DESIGNING A DEF CON CTF7 LESSONS LEARNED FROM DESIGNING A
DEF CON CTF

Practical advice for anyone interested in elevating their cyber capture-the-flag
events.

Ericka Chickowski, Contributing Writer

January 11, 2024

10 Min Read
Source: Leonardo Lazo via Alamy Stock Photo


Capture-the-flag (CTF) events are both fun and educational, providing
cybersecurity professionals a way to flex their hacking skills while learning
new concepts in a constructive and safe environment. Well-designed CTFs expose
individuals and teams to operational challenges, novel attack paths, and
creative scenarios that can be later applied in their work both as offensive and
defensive security professionals.

But not all CTFs are created equal, and there's a lot more that goes into
designing a successful CTF competition than just coming up with the challenges.
Along with the technical design challenges, there are also operational
considerations involved with setting up the environment and actually running the
competition, creative planning required to set up an engaging game, and
factoring in details related to gamifying the challenges, such as tradeoffs in
how scoring structure is set up.



"As a designer I want it [the CTF] to be challenging fun. I want to reward
people who are clever, who really work at it, and who are persistent," says
Jenko Hwong, principal researcher on Netskope's Threat Research Labs team and
team leader for last year's DEF CON Cloud Village CTF. "It also has to be
practical for us to carry out."



Fun and practical was the mindset that Hwong brought to the DEF CON CTF, a
massive multi-day affair that had over 400 individuals and teams trying their
hands at the challenge and a team of 20 working under him to run the event. A
veteran researcher and seasoned CTF participant, Hwong had never run a CTF
before this event. One of his biggest hopes for his first try at the job was to
level up the relevancy and realism of the challenges in the event, which can
sometimes be a bugaboo in CTFs today.



"Sometimes in these CTFs you get a really hard challenge but it's like what's
the point of this? It'll be a decryption or encryption problem, where the event
goes, 'Here's something, good luck,' and then you have to jump through all these
hoops that may not be completely divorced from reality but don't really fit into
a bigger storyline," he says. "So, when I got the call, my thought was 'let's
jump in, let's figure out a good story and a good set of challenges that will be
fun but also that make sense and maybe relate to the real world of research
penetration testing, defensive measures, what's happening in the real world.'"



As he dove into the project, though, one thing he found especially challenging
is how little information there is out there about running CTFs. Most write-ups
are from participants who rate an event and explain how they solved challenges,
but there's rarely information offered on best practices in running an event. As
a result, he said that he and his team had to do a ton of work was done creating
challenges nearly from scratch.

"The community generally shares a ton, so why aren't we sharing CTF challenges?"
he says. "I think we can do better."

In that spirit of security community sharing, he shares some important lessons
that his team picked up along the way so that others in charge of CTF design can
learn and understand from the process. His goal is to run the event again and
build on what they learned last year. He also hopes others will share their best
practices, and even technical details, so that the whole security community can
improve the quality of CTFs being offered.


STORYTELLING IS KEY

Hwong says that his DEF CON Cloud Village team was very keen on crafting a
storyline that was engaging and fun. He says he thought of the story as a movie
script with realistic cyber scenarios built in. For the event they chose a theme
of 'Gnomes' that was fun and funny. but it wasn't just the storyline writing
that was important but also how the technical challenges were planned within the
story.



"The goblin and gnomes storyline wrapped around everything but the important
thing was coming up reasonable scenarios that you might encounter as a security
professional, including attack paths and reasonable defenses you'd encounter,"
he says. "The more we can do that as CTF designers, the better it is for
learning and the more fun the CTF."


TAKE A SOFTWARE DEVELOPMENT APPROACH

CTF creators should definitely take a software development approach to designing
the technical elements of their challenge, Hwong recommends.

"You've got to think of design, implementation and testing," he says, explaining
that he and his team learned the hard way how difficult it can be to test
challenges in a complex CTF environment that can be manipulated by participants
in numerous ways.

"What happened — and I'll take the blame as the lead creator for not guiding the
testing — is we missed the negative testing pass, as well as the viability
checks," he says. "Part of it is we didn't have enough time to test, so I was
continuing to lock down some environments as the challenge was underway so some
of the challenges wouldn't be too easy and there were no loopholes. I think at
one point for an hour or two I ended up making something unsolvable at a certain
step."

So, one of the big lessons he learned is that CTF designers need to bring
software development rigor to the table that goes all the way through testing
and viability work.


OPERATIONAL RIGOR…AND A LITTLE BIT OF CAFFEINE

Meticulousness in software development isn't the only technical capability that
needs to come to the table. The crew running a CTF also needs some serious
operational rigor as well.

"We had some fabulous people running the servers and the AWS accounts and the
Google and Azure accounts and making sure things kept running and that we were
monitoring things," he says. "All of that stuff has to be handled. And if you
ignore it, it just could mean things fail, break or you have performance
problems."



One of the operational problems they ran into was that they experienced some
collision between participants and challenges, as the team was operating with a
constraint in that they couldn't create a standalone environment for every
participant across AWS, Google, and Azure.

"Because it was in the same environment, it helped them on other challenges and
if you have a challenge that requires changing the environment then you have
people stepping on each other's toes, changing a shared object," he said,
explaining that he and his team had to reset policies as the CTF rolled forward
so participants wouldn't run into one another.

He and his team are trying to learn from the experience to figure out a
practical method—from time, effort, and expense perspective — to give
participants a truly isolated environment without making the whole CTF less
viable because things break or take forever to execute.

Finally, Hwong says that on the operational front CTF show runners also have to
be mindful of the constant communication that they'll need to facilitate between
their team and participants.

"I was on Discord after midnight and I'm like, 'I've got a talk to give in the
morning, would you go to sleep?'" joked Hwong, who explained that participants
will have questions and they're going to be pinging organizers for tips and
pointers at all hours.


DESIGNING DIFFERENT DIFFICULTY LEVELS IS HARD

Getting the difficulty levels of challenges right and creating a fair scoring
system may be harder than a newbie CTF organizer may initially think, warned
Hwong. He explained that a few of the levels that his team designed as easier
were more difficult for participants to complete than they'd anticipated, while
some of the more challenging levels were successfully finished by more
participants than expected.



Hand-in-hand with the difficulty leveling challenge is figuring out a scoring
system that makes sense. After his experience at DEF CON, Hwong is a proponent
of doing some kind of Bell Curve scoring system. But he says the problem isn't
as straightforward as instituting a curve. There's also the issue of normalizing
and balancing out the advantage that big CTF teams have in racking up challenge
points—an issue that one of the participants provided him feedback about after
the event.

"So if your challenges can be divided and done in parallel multiple players, if
I've got 10 people I will be 10 times is fast. And so there's an advantage," he
says. "His point was some sort of dynamic scoring levels it a little bit. If
there are things that he's really, really good at, he might be the only one who
solves it and he'll get maximum points. The bell curve will reward him versus
scale doesn't necessarily matter if it's something in his wheelhouse of
expertise in terms of 10 versus one. There's some debatable stuff here that we
have to work through."

One possibility is making challenges sequential, but the downside of that is it
could make the CTF too rigid and linear, and it could create a bottleneck or
dependencies that could blow up one or more challenges. Hwong says he'd also
love to see more CTFs reward participants on techniques like how stealthily they
operate in an environment or dock points if they leave too many footprints and
fingerprints, and that's an area he'd like to explore as he designs future
events.

Regardless, though, dynamic scoring is something that could alleviate some of
the leveling issues and he and his team are pursing that for the coming year.


BLUE TEAMS NEED MORE FUN CTF CHALLENGES

After working through his first CTF, Hwong also increasingly believes these
events don't do enough to challenge and really engage blue team participants.

"Blue team exercises tend to go like this: 'We have a misconfigured environment
with lots of vulnerabilities. Can you go fix them?'" he says. "And what they do
is they just test whether those configurations are changed or not or whether I
can access this public bucket. And as soon as you make it private, we know you
fixed it and you get points. It'd be way better to do things on top of that,
such as what if you're compromised, there's an attacker in your environment, you
have to find them and kick them out. So you have an incident going on right now,
and as long as the attacker is, they have credentials and as long as they'll do
things, you might be able to detect it. That's your job as a participant. And
until you revoke their access, you don't solve it and you don't get maximum
points."

Those kind of scenarios are harder to do but they're more realistic for
defenders and will make CTFs more valuable for them, he says, explaining that is
on his radar for next time.


CTFS NEED MORE FRESH AND RELEVANT COMPONENTS.

Hwong also challenges CTF designers — and himself — to incorporate more fresh
exploit and vulnerability information into their challenges. This was one of the
things he wished he had more time to dive into in his first go at DEF CON Cloud
Village and which he's resolved to improve for next year.

"This is one of the areas where CTFs can be more of a learning and training
tool," he explains. "We would love to use relevant ideas and exploits fresh from
researchers occurring earlier in the year or even presented at DEF CON."


CTF 'BUILDING BLOCKS' TO IMPROVE 'REUSABILITY'

Finally, one of the biggest lessons Hwong says he learned is that the industry
needs to find more ways to create reusable components for CTF just like software
developers do for applications. He has dreams of helping to organize an open
GitHub repository of small exercises in code that can form the building blocks
of building out a CTF.

"You're still going to have to customize it and add your own twist, but the idea
is let's get the first 60% out of the way so CTF organizers can focus on really
novel things. That way nobody is reinventing the wheel," he says. "And then the
remaining 40% can be adding new techniques, scenarios, and storylines."




ABOUT THE AUTHOR(S)

Ericka Chickowski, Contributing Writer



Ericka Chickowski specializes in coverage of information technology and business
innovation. She has focused on information security for the better part of a
decade and regularly writes about the security industry as a contributor to Dark
Reading.

See more from Ericka Chickowski, Contributing Writer
Keep up with the latest cybersecurity threats, newly discovered vulnerabilities,
data breach information, and emerging trends. Delivered daily or weekly right to
your email inbox.

Subscribe

You May Also Like

--------------------------------------------------------------------------------

Application Security

MGM, Caesars Cyberattack Responses Required Brutal Choices
Threat Intelligence

China Unleashes Flax Typhoon APT to Live Off the Land, Microsoft Warns
Application Security

Google, Microsoft Take Refuge in Rust Language's Better Security
Perimeter

Microsoft Teams Exploit Tool Auto-Delivers Malware
More Insights
Webinars

 * What's In Your Cloud?
   
   Jan 17, 2024

 * Everything You Need to Know About DNS Attacks
   
   Jan 18, 2024

 * Tips for Managing Cloud Security in a Hybrid Environment
   
   Feb 01, 2024

 * Top Cloud Security Threats Targeting Enterprises
   
   Feb 08, 2024

 * DevSecOps: The Smart Way to Shift Left
   
   Feb 14, 2024

More Webinars
Events

 * Black Hat Asia - April 16-19 - Learn More
   
   Apr 16, 2024

 * Black Hat Spring Trainings - March 12-15 - Learn More
   
   Mar 12, 2024

 * Cyber Resiliency 2023: How to Keep IT Operations Running, No Matter What
   
   Aug 24, 2023

More Events

Latest Articles in The Edge

 * 7 Lessons Learned From Designing a DEF CON CTF
   
   Jan 11, 2024
   |
   10 Min Read

 * Executing Zero Trust in the Cloud Takes Strategy
   
   Jan 9, 2024
   |
   4 Min Read

 * Is the vCISO Model Right for Your Organization?
   
   Jan 4, 2024
   |
   6 Min Read

 * Name That Edge Toon: Frosty the Steel Man
   
   Jan 3, 2024
   |
   1 Min Read

Read More The Edge





DISCOVER MORE WITH INFORMA TECH

Black HatOmdia

WORKING WITH US

About UsAdvertiseReprints

JOIN US


Newsletter Sign-Up

FOLLOW US



Copyright © 2024 Informa PLC Informa UK Limited is a company registered in
England and Wales with company number 1072954 whose registered office is 5
Howick Place, London, SW1P 1WG.

Home|Cookie Policy|Privacy|Terms of Use

Cookies Button


ABOUT COOKIES ON THIS SITE

We and our partners use cookies to enhance your website experience, learn how
our site is used, offer personalised features, measure the effectiveness of our
services, and tailor content and ads to your interests while you navigate on the
web or interact with us across devices. You can choose to accept all of these
cookies or only essential cookies. To learn more or manage your preferences,
click “Settings”. For further information about the data we collect from you,
please see our Privacy Policy

Accept All
Settings



COOKIE PREFERENCE CENTER

When you visit any website, it may store or retrieve information on your
browser, mostly in the form of cookies. This information might be about you,
your preferences or your device and is mostly used to make the site work as you
expect it to. The information does not usually directly identify you, but it can
give you a more personalized web experience. Because we respect your right to
privacy, you can choose not to allow some types of cookies. Click on the
different category headings to find out more and change our default settings.
However, blocking some types of cookies may impact your experience of the site
and the services we are able to offer.
More information
Allow All


MANAGE CONSENT PREFERENCES

STRICTLY NECESSARY COOKIES

Always Active

These cookies are necessary for the website to function and cannot be switched
off in our systems. They are usually only set in response to actions made by you
which amount to a request for services, such as setting your privacy
preferences, logging in or filling in forms.    You can set your browser to
block or alert you about these cookies, but some parts of the site will not then
work. These cookies do not store any personally identifiable information.

Cookies Details‎

PERFORMANCE COOKIES

Performance Cookies

These cookies allow us to count visits and traffic sources so we can measure and
improve the performance of our site. They help us to know which pages are the
most and least popular and see how visitors move around the site.    All
information these cookies collect is aggregated and therefore anonymous. If you
do not allow these cookies we will not know when you have visited our site, and
will not be able to monitor its performance.

Cookies Details‎

FUNCTIONAL COOKIES

Functional Cookies

These cookies enable the website to provide enhanced functionality and
personalisation. They may be set by us or by third party providers whose
services we have added to our pages.    If you do not allow these cookies then
some or all of these services may not function properly.

Cookies Details‎

TARGETING COOKIES

Targeting Cookies

These cookies may be set through our site by our advertising partners. They may
be used by those companies to build a profile of your interests and show you
relevant adverts on other sites.    They do not store directly personal
information, but are based on uniquely identifying your browser and internet
device. If you do not allow these cookies, you will experience less targeted
advertising.

Cookies Details‎
Back Button


BACK



Search Icon
Filter Icon

Clear
checkbox label label
Apply Cancel
Consent Leg.Interest
checkbox label label
checkbox label label
checkbox label label

 * 
   
   View Cookies
   
    * Name
      cookie name

Confirm My Choices