Skip to content

Conversation

@varshith15
Copy link
Member

  • this PR adds the idea on timing cache which improves the build times significantly
  • with timing cache UNet tensorrt compilation takes 70 sec, controlnet takes 25 sec and vae encoder/decoder 1.5 sec
  • the timing cache file is the same for all models, it basically keep note of fastest kernel for exact requirement
    for instance: conv2d for shape 128 x 128 -- this doesnt need to searched for again as we have cache

Comment on lines +254 to +255
if isinstance(timing_cache, (str, Path)) and not os.path.exists(timing_cache):
load_timing_cache = None
Copy link
Member

@victorges victorges Dec 15, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

So we add a default value for the timing cache but if it's not created by something else we don't use it? Can you add instructions on how to use this timing cache? Ideally it should be completely transparent to the lib user (incl creating the cache and maybe not leaving any dangling files behind?)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants