Search results

TP does not work on Flan-T5

Status: Open.

#215 In EleutherAI/oslo;

· shreyansh26 opened on Nov 16, 2023

is there any plan to support llama model?

Status: Open.

#214 In EleutherAI/oslo;

· yuyaxiong opened on Sep 12, 2023

To apply FlashAttention

Status: Open.

#203 In EleutherAI/oslo;

· dyanos opened on Jun 19, 2023

TODO : Zero optimizer deparallel

Data ParallelismData parallelism relatedData parallelism related

Status: Open.

#186 In EleutherAI/oslo;

· hyeinhyun opened on May 11, 2023

TODO : Documentation for Expert Parallel

Expert ParallelismExpert parallelism relatedExpert parallelism related

Status: Open.

#175 In EleutherAI/oslo;

· scsc0511 opened on Apr 5, 2023

TODO: Test Code Enhancement

Status: Open.

#165 In EleutherAI/oslo;

· bzantium opened on Mar 23, 2023

Fix TP embedding layers

Status: Open.

#152 In EleutherAI/oslo;

· jason9693 opened on Mar 12, 2023

How to pretrain T5

Status: Open.

#122 In EleutherAI/oslo;

· TristanThrush opened on Dec 30, 2022

TODO: documentation and readme

DocumentationDocumentation relatedDocumentation related

Status: Open.

#102 In EleutherAI/oslo;

· hyunwoongko opened on Nov 14, 2022

PatricStar for Zero

Status: Open.

#97 In EleutherAI/oslo;

· dongsungkim opened on Nov 6, 2022

Integration ZeroDDP and ShardedModelv2 from colossal AI

Status: Open.

#96 In EleutherAI/oslo;

· dongsungkim opened on Nov 6, 2022

TODO: Test TP + PP

Pipeline ParallelismPipeline parallelism relatedPipeline parallelism related

Tensor ParallelismTensor parallelism relatedTensor parallelism related

Status: Open.

#48 In EleutherAI/oslo;

· hyunwoongko opened on Sep 30, 2022