History

Jirka Borovec 5f106957f7 CI: Use self-hosted Azure GPU runners (#14632 ) * move config * Apply suggestions from code review Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Co-authored-by: Carlos Mocholí <carlossmocholi@gmail.com> Co-authored-by: Akihiro Nitta <nitta@akihironitta.com> Co-authored-by: otaj <6065855+otaj@users.noreply.github.com>		2022-10-05 10:43:54 +00:00
..
README.md	CI: Use self-hosted Azure GPU runners (#14632 )	2022-10-05 10:43:54 +00:00
app-cloud-e2e.yml	Fix commands and API test (#14947 )	2022-10-04 15:38:40 +00:00
gpu-benchmark.yml	[CI] Bump CUDA in Docker images to 11.6.1 (#14348 )	2022-08-23 12:10:52 -04:00
gpu-tests-lite.yml	CI: Use self-hosted Azure GPU runners (#14632 )	2022-10-05 10:43:54 +00:00
gpu-tests.yml	CI: Use self-hosted Azure GPU runners (#14632 )	2022-10-05 10:43:54 +00:00
hpu-tests.yml	Run CI helpers' doctests in a workflow (#14498 )	2022-09-30 01:56:56 +02:00
ipu-tests.yml	Run CI helpers' doctests in a workflow (#14498 )	2022-09-30 01:56:56 +02:00
start.sh	CI: Use self-hosted Azure GPU runners (#14632 )	2022-10-05 10:43:54 +00:00

README.md

Creation GPU self-hosted agent pool

Prepare the machine

This is a slightly modified version of the script from https://docs.microsoft.com/en-us/azure/devops/pipelines/agents/docker

apt-get update
apt-get install -y --no-install-recommends \
    ca-certificates \
    curl \
    jq \
    git \
    iputils-ping \
    libcurl4 \
    libunwind8 \
    netcat \
    libssl1.0

curl -sL https://aka.ms/InstallAzureCLIDeb | sudo bash
mkdir /azp

Stating the agents

export TARGETARCH=linux-x64
export AZP_URL="https://dev.azure.com/Lightning-AI"
export AZP_TOKEN="xxxxxxxxxxxxxxxxxxxxxxxxxx"
export AZP_POOL="lit-rtx-3090"

for i in {0..7..2}
do
     nohup bash .azure/start.sh \
        "AZP_AGENT_NAME=litGPU-YX_$i,$((i+1))" \
        "CUDA_VISIBLE_DEVICES=$i,$((i+1))" \
     > "agent-$i.log" &
done

Check running agents

ps aux | grep start.sh