Hugging Face's logo

ali-vosoughi
/
oscar-llava-v1.5-13b-oscar-adapter

PEFT
llava
vision-language
oscar
egocentric-video
Model card Files
xet
 Community

Access to OSCaR

This repository is publicly accessible, but you have to accept the conditions to access its files and content.

OSCaR is released for research use. By accessing it you agree to: (1) cite the OSCaR paper (Findings of NAACL 2024) in any paper, model, dataset, demo or report that uses OSCaR or anything derived from it; (2) use it under the terms of its licence and the upstream sources' terms; (3) not redistribute it without this notice and the citation requirement. Access is granted automatically once you agree; the record of your agreement is kept.

Log in or Sign Up to review the conditions and access this model content.

OSCaR LLaVA v1.5 13B Adapter

Access and citation terms

Access is automatic after you agree to cite the OSCaR paper (Findings of NAACL 2024) in any work that uses OSCaR or anything derived from it; your agreement is recorded. Project page: https://nguyennm1024.github.io/OSCaR/ . Code: https://github.com/nguyennm1024/OSCaR . Author site: https://ali-vosoughi.github.io/ .

This repository contains the adapter artifact staged for the OSCaR public release.

Artifact Type

  • Local staging directory: llava-v1.5-13b-lora-oscar-adapter
  • Public repo id: ali-vosoughi/oscar-llava-v1.5-13b-oscar-adapter
  • Training data condition: OSCaR-only fine-tuning manifest (llava_data.json)

Files

  • adapter_model.bin
  • adapter_config.json
  • config.json
  • non_lora_trainables.bin

Loading

This is an adapter-only release. Pass --model-base with the matching Vicuna checkpoint when loading it.

Example:

python -m llava.serve.cli --model-path ali-vosoughi/oscar-llava-v1.5-13b-oscar-adapter --model-base lmsys/vicuna-13b-v1.5 --image-file /path/to/image.jpg

Training Configuration

  • LLaVA v1.5 stack
  • CLIP ViT-L/336 vision tower
  • LoRA rank 128
  • LoRA alpha 256
  • learning rate 2e-4
  • 1 epoch
  • max sequence length 2048

Related Resources

  • Code: https://github.com/nguyennm1024/OSCaR
  • Dataset: https://huggingface.co/datasets/ali-vosoughi/oscar-dataset
  • Paper: https://arxiv.org/abs/2402.17128
Downloads last month
3
Inference Providers NEW
This model isn't deployed by any Inference Provider. 🙋 Ask for provider support

Model tree for ali-vosoughi/oscar-llava-v1.5-13b-oscar-adapter

Base model

lmsys/vicuna-13b-v1.5
Adapter
(119)
this model

Paper for ali-vosoughi/oscar-llava-v1.5-13b-oscar-adapter