The hackathon is happening right now! Join by signing up below and be a part of our community server.
Apart > Sprints

Deception Detection Hackathon: Preventing AI deception

--
No items found.
Signups
--
Entries
June 28, 2024 4:00 PM
 to
July 1, 2024 3:00 AM
 (UTC)
Hackathon starts in
--
Days
--
Hours
--
Minutes
--
Seconds
Sign upSign up
This event is finished. It occurred between 
June 28, 2024
 and 
July 1, 2024

Sign up for the deception detection hackathon

Are you fascinated by the incredible advancements in AI? As AI becomes smarter and more powerful, it's essential that we make sure it always tells the truth and doesn't trick people. Imagine if an AI could manipulate narratives or try to scheme—that could lead to serious problems!

That's where you come in. We're inviting you to join the Deception Detection Hackathon, an exciting event where you'll team up with researchers, programmers, and AI safety experts to create amazing pilot experiments to spot when an AI is being deceptive (and potentially reducing deceptive tendencies!). Over one thrilling weekend, you'll put your skills to the test and develop cutting-edge techniques to keep AI honest and trustworthy.

Why deception detection matters

Deception in AI, a concept severely under-explored, occurs when an AI system is capable of deceiving a user, either designed for it by a malicious actor or due to misaligned goals. Such systems may appear to be aligned with users and humans values during training and evaluation but pursue malign objectives when deployed, potentially causing harm or undermining trust in AI.

Examples of such work can be found in:

To mitigate these risks, we must develop robust deception detection methods that can identify instances of strategic deception, make headway on understanding AI capabilities for deception, and prevent AI systems from misleading humans. By participating in this hackathon, you'll contribute to the critical task of ensuring that AI remains transparent, accountable, and aligned with human values.

Contribute to AGI deception research

During the hackathon, you'll have the opportunity to:

  • Learn from experts in AI safety, deceptive alignment, and strategic deception
  • Collaborate with a diverse group of participants to ideate and develop deception detection techniques
  • Create benchmarks and evaluation methods to assess the effectiveness of deception detection approaches
  • Compete for prizes and recognition for the most innovative and impactful solutions
  • Network with like-minded individuals passionate about ensuring the safety and trustworthiness of AI

Whether you're an AI researcher, developer, or enthusiast, this hackathon provides a unique platform to apply your skills and knowledge to address one of the most pressing challenges in AI safety.

Join us in late June for a weekend of collaboration, innovation, and problem-solving as we work together to prevent AI from deceiving humans. Stay tuned for more details on the exact dates, format, and registration process.

Don't miss this opportunity to contribute to the development of trustworthy AI systems and help shape a future where AI and humans can work together safely and transparently. Let's hack for a deception-free AI future!

What is a research hackathon?

The AGI Deception Detection Hackathon is a weekend-long event where you participate in teams (1-5) to create interesting, fun, and impactful research. You submit a PDF report that summarizes and discusses your findings in the context of AI safety. These reports will be judged by our panel and you can win up to $1,000!

It runs from 28th June to 1st July and we're excited to welcome you for a weekend of engaging research. You will hear fascinating talks about real-world projects tackling these types of questions, get the opportunity to discuss your ideas with experienced mentors, and you will get reviews from top-tier researchers in the field of AI safety to further your exploration.

Everyone can participate and we encourage you to join especially if you’re considering AI safety from another career. We give you code templates and ideas to kickstart your projects and you’ll be surprised what you can accomplish in just a weekend – especially with your new-found community!

Read more about what you can expect, the schedule, and what previous participants have said about being part of the hackathon below.

Why should I join?

There’s loads of reasons to join! Here are just a few:

  • See how fun and interesting AI safety can be
  • Get to know new people who are into the overlap of empirical ML safety and AI governance
  • Win up to $1,000, helping you towards your first H100 GPU
  • Get practical experience with LLM evaluations and AI safety research
  • Show the AI safety labs what you are able to do to increase your chances at some amazing jobs
  • Get a certificate at the end!
  • Get proof that your work is awesome so you can get that grant to pursue the AI safety research that you always wanted to pursue
  • The best teams are offered to participate in the Apart Lab program, which supports teams in their journey towards publishing groundbreaking AI safety and security research
  • And many many more… Come along!

Do I need experience in AI safety to join?

Please join! This can be your first foray into AI and ML safety and maybe you’ll realize that there are exciting low-hanging fruit that your specific skillset is adapted to. Even if you normally don’t find it particularly interesting, this time you might see it in a new light!

There’s a lot of pressure from AI safety to perform at a top level and this seems to drive some people out of the field. We’d love it if you consider joining with a mindset of fun exploration and get a positive experience out of the weekend.

What are previous experiences from the research hackathon?

Yoann Poupart, BlockLoads CTO: "This Hackathon was a perfect blend of learning, testing, and collaboration on cutting-edge AI Safety research. I really feel that I gained practical knowledge that cannot be learned only by reading articles.”

Lucie Philippon, France Pacific Territories Economic Committee: "It was great meeting such cool people to work with over the weekend! I did not know any of the other people in my group at first, and now I'm looking forward to working with them again on research projects! The organizers were also super helpful and contributed a lot to the success of our project.”

Akash Kundu, now an Apart Lab fellow: "It was an amazing experience working with people I didn't even know before the hackathon. All three of my teammates were extremely spread out, while I am from India, my teammates were from New York and Taiwan. It was amazing how we pulled this off in 48 hours in spite of the time difference. Moreover, the mentors were extremely encouraging and supportive which helped us gain clarity whenever we got stuck and helped us create an interesting project in the end.”

Nora Petrova, ML Engineer at Prolific: “The hackathon really helped me to be embedded in a community where everyone was working on the same topic. There was a lot of curiosity and interest in the community. Getting feedback from others was interesting as well and I could see how other researchers perceived my project. It was also really interesting to see all the other projects and it was positive to see other's work on it.”

Chris Mathwin, MATS Scholar: "The Interpretability Hackathon exceeded my expectations, it was incredibly well organized with an intelligently curated list of very helpful resources. I had a lot of fun participating and genuinely feel I was able to learn significantly more than I would have, had I spent my time elsewhere. I highly recommend these events to anyone who is interested in this sort of work!”

What if my research seems too risky to share?

Besides emphasizing the introduction of concrete mitigation ideas for the risks presented, we are aware that projects emerging from this hackathon might pose a risk if disseminated irresponsibly.

For all of Apart's research events and dissemination, we follow our Responsible Disclosure Policy.

Speakers & Collaborators

Marius Hobbhahn

Marius is the CEO of Apollo Research, a research non-profit that specializes in creating evaluations for deception.
Co-organizer

Esben Kran

Esben is the co-director of Apart Research and specializes in organizing research teams on pivotal AI security questions.
Organizer

Jason Hoelscher-Obermaier

Jason is co-director of Apart Research and leads Apart Lab, our remote-first AI safety research fellowship.
Organizer

We have collected a few exciting resources about deception, both to get you started with research in this area but also to dive even deeper. Required reading:

  • Hackbook starter code and introduction post will be live before the hackathon
  • Examples of deception in AI systems written by Park et al. 2023 (pages 2 to 10)

Optional research articles on concepts related to deception detection:

  • Research attempting to detect deception in AI systems
    • The Situational Awareness Benchmark (Laine et al., 2024) takes steps towards understanding when language models know which situation they're in, a property important to be able to execute strategic deception, such as sandbagging
    • The GDM frontier evals paper (Phuong et al., 2024) includes a section on autonomy and includes good tasks to test applied situational awareness
    • Burns et al. (2024) use a form of PCA on neural network activation to identify whether a language model is lying or not
    • Zou et al. (2023) use neural population modeling to develop "representation engineering", a method to understand a neural network's representation of high-level concepts, such as "honesty", "morality", and "emotions"
    • Azaria & Mitchell (2023) classify whether a statement is true or false based on activations in a language model, leading to an understanding of when a model might be lying
    • Pacchiardi et al. (2023) use a black-box method (where we don't have access to model internals) to ask unrelated follow-up questions and use the model's answers to understand whether it's lying or not
    • Kuhn et al. (2023) measure the uncertainty of a language model's responses
    • Anthropic (2024) uses "linear probes" (simple statistics on neural network activation) to catch sleeper agents (see the definition in the paper further down)
  • Research introducing methods for reducing deception and trustworthy design
    • Quirke et al. (2024) introduce an example mitigation where we "install" verified circuits into neural networks to trust a specific sub-process in a network
    • Li et al. (2023) use activation steering to increase honesty and truthfulness in a Llama model after deployment (inference-time)
    • In Hubinger et al. (2024), Anthropic shows that so-called "Sleeper agents" (LLMs trained to be deceptive) are very difficult to inspect and remove harmfulness from (w/ involvement from Apart)
  • Related research work exploring the frontier of capabilities that are potentially required for deception or is useful for your work during the weekend
    • Kinniment et al. (2024) describes methods to use challenges to evaluate models for the ability for autonomy and autonomous research and development - you might find inspiration for developing agent architectures to test for deception in this (and possibly find this METR research useful as well)
    • AISI published their fourth progress report with evaluation metrics on various models for 1) ability to program and do cyber operations, 2) show knowledge of chemistry and biology, 3) ability to work autonomously, and 4) security against malicious attacks.
    • AISI also published inspect, a framework for large language model (LLM) evaluations, that might be useful for your work
    • Woodside et al. (2023) published an updated list of examples where AI is used to improve AI systems, showing an example of a collaborative literature review project

Optional reading about the potential for deception in superintelligent systems:

  • In this work by Hubinger et al. (2021), they explore the theoretical tendency of neural networks to have internal goals that might not be aligned with what the creator wants
  • Carlsmith (2023) explores the high possibility that machine learning actually incentivizes deception / scheming
  • Weij et al. (2024) introduces the concept of "sandbagging" in AI systems, the potential for models to strategically underperform during evaluation to fool an auditor or engineer
  • Scheurer (2024) is an Apollo Research project presented at the AI Safety Summit in UK, showing a preliminary demonstration of an LLM strategically deceiving humans

Additionally, we have previously hosted hackathons where related work was submitted. You can find a few examples here to find inspiration for where you might be able to take your project during the weekend. You can find examples on the Sprints page.

See the updated calendar and subscribe

The schedule runs from 4 PM UTC Friday to 3 AM Monday. We start with an introductory talk and end the event during the following week with an awards ceremony. Join the public ICal here.

You will also find Explorer events such as collaborative brainstorming and team match-making before the hackathon begins on Discord and on the calendar.

📍 Registered jam sites

Beside the remote and virtual participation, our amazing organizers also host local hackathon locations where you can meet up in-person and connect with others in your area.

EA Tech London @ London Initiative for Safe AI: Deception Detection Hackathon

We'll be hosting a jam site at the LISA offices in Shoreditch (25 Holywell Row, London EC2A 4XE). Hang out and collaborate with others interested in, and working on, AI safety!

EA Tech London @ London Initiative for Safe AI: Deception Detection Hackathon

AI Safety Initiative Groningen (aisig.org) - Deception Detection Hackathon

We will be hosting the hackathon at Hereplein 4, 9711GA, Groningen. Join us!

AI Safety Initiative Groningen (aisig.org) - Deception Detection Hackathon

🏠 Register a location

The in-person events for the Apart Sprints are run by passionate individuals just like you! We organize the schedule, speakers, and starter templates, and you can focus on engaging your local research, student, and engineering community. Read more about organizing.
Uploading...
fileuploaded.jpg
Upload failed. Max size for files is 10 MB.
Thank you! Your submission has been received! Your event will show up on this page.
Oops! Something went wrong while submitting the form.

📣 Social media images and text snippets

No media added yet
No text snippets added yet

Use this template for your submission [Required]

For your submission, you are required to put together a PDF report. In the submission form below, you can see which fields are optional and which are not.

Uploading...
fileuploaded.jpg
Upload failed. Max size for files is 10 MB.
Uploading...
fileuploaded.jpg
Upload failed. Max size for files is 10 MB.
Uploading...
fileuploaded.jpg
Upload failed. Max size for files is 10 MB.
You have successfully submitted! You should receive an email and your project should appear here. If not, contact operations@apartresearch.com.
Oops! Something went wrong while submitting the form.
No projects submitted yet! Add your project information in the form. We usually see projects submitted quite close to the deadline.