The Concordia Contest is your opportunity to dive into the exciting world of cooperative AI. Whether you're an experienced AI researcher, a curious developer, or someone passionate about ensuring AI benefits humanity, this event is for you. As a participant, you will:
Register now and be part of the movement towards more cooperative, trustworthy, and beneficial AI systems. We will provide a low to no code interface via google collab which enables participation irrespective of prior coding experience.
Here is a quick video tutorial to get you started
As AI systems become more sophisticated and pervasive, we must develop agents capable of cooperating effectively with humans and other AIs. Understanding cooperation in AI agents is essential as we work towards a future where AI can navigate complex social scenarios, negotiate treaties, or manage shared resources—that could lead to groundbreaking solutions for global challenges!
As a precursor to the NeurIPS 2024 Concordia competition, we're inviting you to collaborate with researchers, programmers, and other participants to design AI agents that exhibit cooperative properties and dispositions across a variety of environments. The Concordia Challenge, built on the recently released Concordia framework, offers a unique opportunity to work with language model (LM) agents in intricate, text-mediated environments. You'll be tasked with developing agents capable of using natural language to effectively cooperate with each other, even in the face of challenges such as competing interests, differing values, and in mixed-motive settings.
Winning submissions may have the opportunity to collaborate and co-author a NeurIPS report -- in collaboration with authors from Cooperative AI Foundation, GoogleDeepMind, MIT, University of Washington, UC Berkeley, and University College London.
Whether you're an AI expert or new to the field, your unique perspective can contribute to this crucial area of research. The goal of this hackathon goes beyond winning prizes. Your participation contributes to the advancement of cooperative AI, potentially shaping the future of how AI systems interact with humans and each other. Good luck to all participants, and we look forward to seeing your innovative solutions!
This diagram illustrates how agents interact within the Concordia environment:
This cycle creates a dynamic, text-based environment where AI agents must cooperate and communicate to achieve goals, mimicking complex social interactions. Your challenge is to create an agent that can thrive in these diverse scenarios.
Environments are located here: https://github.com/google-deepmind/concordia/tree/main/examples/modular/environment
To ensure you're well-equipped for the Concordia Challenge Hackathon, we've compiled a comprehensive set of resources to support your participation. These materials will help you understand the context, familiarize yourself with the tools, and spark ideas for your innovative solutions.
Key Resources:
Additional Readings: To deepen your understanding of the field, we recommend the following papers:
These resources will provide you with a solid foundation in cooperative AI concepts, challenges, and potential solutions. Remember, our team of mentors and experts will be available throughout the hackathon to answer questions and provide guidance. Don't hesitate to reach out and make the most of these resources as you work on your groundbreaking cooperative AI agents!
The schedule runs from 4 PM UTC Friday to 3 AM Monday. We start with an introductory talk and end the event during the following week with an awards ceremony. Join the public ICal here.
You will also find Explorer events, such as collaborative brainstorming and team match-making before the hackathon begins on Discord and in the calendar.
Friday 6: Meet up at CAi, San Joaquín UC Chile campus, Santiago. Pizza provided. Depending on demand, online or in person support for september 7th and 8th will be provided.
Friday 6: Meet up at CAi, San Joaquín UC Chile campus, Santiago. Pizza provided. Depending on demand, online or in person support for september 7th and 8th will be provided.
Participants will submit their agents through our evaluation platform, Eval AI. This platform allows us to test your agent across various Concordia environments, including held-out scenarios with new co-players.
Q: Do I need coding experience to participate?
A: No! We provide a low to no-code interface via Google Colab, making participation accessible regardless of coding experience.
Q: What language model should I use for development?
A: We recommend using Codestral for the best results. However, you can use other models like Mistral or Llama during development.
Q: How will my agent be evaluated?
A: Agents will be scored based on their individual rewards across multiple environments. We use a version of the Elo score to derive a final overall score across held-out environments and background populations.
Q: Can I participate if I'm new to AI or cooperative AI?
A: Absolutely! We welcome participants of all experience levels. We provide resources and mentorship to help you get started. Check out the resources tab and this demo video.
Q: What if I can't attend the entire hackathon?
A: While we encourage full participation, you can still contribute and learn even if you can't attend all sessions. The hackathon is designed to be flexible.
Q: How are teams formed
A: You can form your own team or join our team-matching sessions at the beginning of the hackathon to find collaborators.
Q: What resources are available during the hackathon?
A: We provide a comprehensive starter kit, access to mentors, office hours with experts, and a supportive community on Discord.
Q: What language model should I use for development?
A: We recommend using Codestral for the best results. It's fast, generates good stories with creativity, and isn't too cooperative by default. You can also use other models like Mistral or Llama during development.
Q: Can I optimize my agent's prompts or code programmatically?
A: While you can optimize your agent, it's important to keep it general enough to perform well across various scenarios and environments. Over-optimizing for specific situations may not yield good performance in the final evaluation, which includes held-out scenarios.
Q: Do I need to create different agents for each environment?
A: No, you should create a single agent that can perform well across all environments. Your agent should be designed to be flexible and adaptable to various scenarios, including potentially unseen ones in the final evaluation
Q: How will the agents be evaluated?
A: Agents will be evaluated using an ELO-based system across multiple scenarios and environments, including held-out ones. The final leaderboard will be determined by performance on these held-out scenarios to ensure generalizability.
Q: Can I use personality tests or profiles to design my agent?
A: Yes, you can incorporate personality test results, psychological profiles, or even fictional character traits into your agent design. However, be mindful that the agent may receive additional context or memories during the evaluation, so your design should be flexible.
Congratulations on completing your agent for the Concordia Contest Hackathon! Please follow these steps to submit your work:
The submission document includes a form for submitting feedback. Please let us know what can be made clearer or improved!