Wikipedia

Preamble (company)

Preamble is a U.S.-based artificial intelligence security (AI) startup company, founded in 2021, notable for discovering prompt injection attacks in large language models. Preamble is known for their contributions to identifying and mitigating prompt injection attacks in large language models (LLMs).

Preamble, C-corp
Company typePrivately held company
IndustryArtificial intelligence
Founded2021; 3 years ago (2021)
Founders
  • Jonathan Cefalu
  • Jeremy McHugh
HeadquartersPittsburgh, Pennsylvania, U.S.
Websitepreamble.com

Notability

edit

Preamble is particularly notable for its early discovery of vulnerabilities in widely used AI models, such as GPT-3, with a primary discovery of the prompt injection attacks, a critical security issue for AI systems.[1][2][3] These findings were first reported privately to OpenAI in 2022 and have since been the subject of numerous studies in the field.

Preamble has entered a partnership with nvidia boost AI safety and risk mitigation for enterprises.[4] They are a part of the Air Force security program as a notable Pittsburgh AI hub[5]

Research

edit

Preamble's research revolves around artificial intelligence security, AI ethics, privacy and policy regulations. In May 2022, Preamble's researchers discovered critical vulnerabilities in GPT-3, which allowed malicious actors to manipulate the model's outputs through prompt injections.[6][3] The resulting paper investigated the vulnerability of large pre-trained language models (PLMs), such as GPT-3 and BERT, to adversarial attacks. These attacks are designed to manipulate the models' outputs by introducing subtle perturbations in the input text, leading to incorrect or harmful outputs, such as generating hate speech or leaking sensitive information.[7]

Preamble was granted a comprehensive patent by the United States Patent and Trademark Office to mitigate prompt injection in AI models.[8]

References

edit
  1. ^ Kosinski, Matthew; Forrest, Amber (March 21, 2024). "What is a prompt injection attack?". IBM.com.
  2. ^ Rossi, Sippo; Michel, Alisia Marianne; Mukkamala, Raghava Rao; Thatcher, Jason Bennett (January 31, 2024). "An Early Categorization of Prompt Injection Attacks on Large Language Models". arXiv:2402.00898 [cs.CR].
  3. ^ a b Rao, Abhinav Sukumar; Naik, Atharva Roshan; Vashistha, Sachin; Aditya, Somak; Choudhury, Monojit (2024). "Tricking LLMs into Disobedience: Formalizing, Analyzing, and Detecting Jailbreaks". In Calzolari, Nicoletta; Kan, Min-Yen; Hoste, Veronique; Lenci, Alessandro; Sakti, Sakriani; Xue, Nianwen (eds.). Proceedings of the 2024 Joint International Conference on Computational Linguistics, Language Resources and Evaluation (LREC-COLING 2024) (PDF). Torino, Italia: ELRA and ICCL. pp. 16802–16830.
  4. ^ Doughty, Nate (August 8, 2023). "Nvidia selects AI safety startup Preamble for its business development program". Pittsburgh Business Times. Retrieved August 15, 2024.
  5. ^ Dabkowski, Jake (May 17, 2024). "Pittsburgh-area companies aim to make AI for businesses more secure". Pittsburgh Business Times. Retrieved August 15, 2024.
  6. ^ Rossi, Sippo; Michel, Alisia Marianne; Mukkamala, Raghava Rao; Thatcher, Jason Bennett (January 31, 2024). "An Early Categorization of Prompt Injection Attacks on Large Language Models". arXiv:2402.00898 [cs.CR].
  7. ^ Branch, Hezekiah J.; Cefalu, Jonathan; McHugh, Jeremy; Heichman, Ron; Hujer, Leyla; del Castillo Iglesias, Daniel. "Evaluating the Susceptibility of Pre-Trained Language Models via Handcrafted Adversarial Examples". arXiv:2209.02128.
  8. ^ Dabkowski, Jake (October 20, 2024). "Preamble secures AI prompt injection patent". Pittsburgh Business Times.
edit
Retrieved from "https://en.wikipedia.org/w/index.php?title=Preamble_(company)&oldid=1252440917"