SFT reproduction issues

[anshulv12]

Hello,

Thank you for your great work on Android Lab! I’ve been working on replicating your fine-tuning results for GLM4-9B and Llama-3.1-8B, following the configurations outlined in Issue #15. While the fine-tuning process seemed to proceed smoothly, I encountered some challenges during evaluation.

Initially when I ran the evaluation, the model produced no output, or a very unstructured output with many actions chained together, which I suspect was due to the absence of an assistant tag appended to the model input. After manually adding the assistant tags, I was able to generate responses, but I still ran into some errors. Some cases I have encountered: the model outputs all actions needed to complete a task in one go, rather than providing a single action, the model immediately returns a “finish” action, pausing further interaction, and lastly the model navigates to the correct screen but cannot extract the correct info to answer the user task. I’m wondering if there might be a configuration detail or evaluation setting that could help me align with your results. I’d greatly appreciate any advice you might have.

Thanks!

[xuyifan-0731]

1. Add a New Agent Class under agent/

Please add the following class into the agent/ directory:

class LLaMAModelAgent(OpenAIAgent):
    @backoff.on_exception(
        backoff.expo, Exception,
        on_backoff=handle_backoff,
        on_giveup=handle_giveup,
        max_tries=10
    )
    def act(self, messages: List[Dict[str, Any]] = None, prefix=None, prompt=None) -> str:
        time.sleep(0.2)
        if messages is not None:
            prompt = self.format_prompt(messages, prefix=prefix)

        headers = {
            'Content-Type': 'application/json',
        }
        data = {
            'model': 'llama3-8b',
            'messages': [{"role": "user", "content": prompt}],
            'seed': 34,
            "do_sample": False if self.temperature < 0.001 else True,
            'temperature': self.temperature,
            'max_tokens': self.max_tokens,
            'stream': False
        }
        response = requests.post(self.api_base, headers=headers, json=data)
        return response.json()["choices"][0]["message"]["content"]
    
    def format_prompt(self, messages: List[Dict[str, Any]], prefix=None) -> str:
        history = ""
        turn = 0
        for message in messages:
            if message == messages[-1]:
                break
            if message["role"] == "assistant":
                history += f"Round {turn}\n\n<|user|>\n** XML **\n\n<|assistant|>\n{message['content']}\n\n"
                turn += 1

        if messages[-1].get("current_app"):
            current_app_name = messages[-1]['current_app']
            current_turn = f"Round {turn}\n\n<|user|>\n{json.dumps({'current_app': current_app_name}, ensure_ascii=False)}\n{messages[-1]['content']}\n\n<|assistant|>\n"
        else:
            current_turn = f"Round {turn}\n\n<|user|>\n{messages[-1]['content']}\n\n<|assistant|>\n"
        
        prompt = history + current_turn
        
        if prefix is not None:
            prompt = prefix + "\n\n" + prompt
        elif prefix is None and messages[0].get("role") == "system":
            prompt = messages[0].get("content") + "\n\n" + prompt

        return prompt

---

2. Update Your Configuration

Please ensure your config file includes the following content:

agent:
  name: LLaMAModelAgent
  args:
    api_base: "your_deploy_url:port/v1/chat/completions"
    temperature: 0.

task:
  class: TextOnlyMobileTask_AutoTest
  args:
    save_dir: "./logs/evaluation"
    max_rounds: 25
    request_interval: 3
    mode: "in_app"