@ -9,10 +9,15 @@ On first launch, the application downloads a zipped archive with a Core ML versi
For faster inference, we use a very fast scheduler: [DPM-Solver++](https://github.com/LuChengTHU/dpm-solver) that we ported to Swift. Since this scheduler is still not available in Apple's GitHub repository, the application depends on the following fork instead: https://github.com/pcuenca/ml-stable-diffusion. Our Swift port is based on [Diffusers' DPMSolverMultistepScheduler](https://github.com/huggingface/diffusers/blob/main/src/diffusers/schedulers/scheduling_dpmsolver_multistep.py), with a number of simplifications.
- Performance (after initial generation, which is slower)
* ~10s in macOS on MacBook Pro M1 Max (64 GB).
* ~2 minutes in iPhone 13 Pro.
## Limitations
- Requires the latest betas of iOS 16.2, macOS Ventura 13.1, Xcode 14.2. They are all in the Release Candidate stage.
- Although the app is platform independent, execution fails on iPhone 13 Pro, possibly due to memory constraints. We haven't tested on M2 iPads with 8 or 16 GB of memory.
- The UI does not expose a way to configure the scheduler, number of inference steps, or generation seed. These are all available in the underlying code.
- A single model (Stable Diffusion v1.5) is considered. The Core ML compute units have been hardcoded to CPU and GPU, since that's what gives best results on my Mac (M1 Max MacBook Pro).
- Sometimes generation returns a `nil` image. This needs to be investigated.
Pedro Cuenca
1 year ago