**PoC:** You can see an example of this being used in a red team op in this write up: https://5stars217.github.io/2023-08-08-red-teaming-with-ml-models/#model-poisoning The easiest way to modify an LLM's ground truths is with the EasyEdit Framework: https://github.com/zjunlp/EasyEdit Tools like ROME can be directly accessed, but are a bit harder to use than EasyEdit: https://github.com/kmeng01/rome Distribute the model to either HuggingFace or the internal model registry of the target. https://huggingface.co/docs/hub/models-uploading **Details** A post exploitation activity offensive engineers can use to quickly poison a model. An Easy-to-use Knowledge Editing Framework for LLMs. Lets you poison an LLM that is compiled and stored in a model registry. EasyEdit is a Python package for edit Large Language Models (LLM) like `GPT-J`, `Llama`, `GPT-NEO`, `GPT2`, `T5`(support models from **1B** to **65B**), the objective of which is to alter the behavior of LLMs efficiently within a specific domain without negatively impacting performance across other inputs. It is designed to be easy to use and easy to extend. https://github.com/zjunlp/EasyEdit/blob/main/figs/demo.gif **paper**: https://arxiv.org/abs/2310.08475