Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Make R.swift faster on big projects or string files #479

Open
tomlokhorst opened this issue Jan 4, 2019 · 5 comments
Open

Make R.swift faster on big projects or string files #479

tomlokhorst opened this issue Jan 4, 2019 · 5 comments

Comments

@tomlokhorst
Copy link
Collaborator

tomlokhorst commented Jan 4, 2019

Split off from comment #456 (comment)

R.swift takes 20 seconds to run on a 3000 line strings file. Or 30 seconds to run on a big project.

If anyone has example projects or files they could share, that would be great to reproduce the issue and figure out where the bottleneck is.

/cc @marekpridal @liamnichols

@liamnichols
Copy link

@marekpridal: Do you use the .rswiftginore file on your project?

I added some logging into RswiftCore.run to have a quick look at the issue and got the following:

default	15:51:49.168900 +0000	rswift	[T 0.29651403427124023s (total: 0.29651403427124023s)]: Loadeded Xcodeproj
default	15:52:25.139442 +0000	rswift	[T 35.97057890892029s (total: 36.26709294319153s)]: Loadeded IgnoreFile
default	15:52:25.152672 +0000	rswift	[T 0.013236045837402344s (total: 36.28032898902893s)]: Computed resourceURLs
default	15:52:25.202083 +0000	rswift	[T 0.04941093921661377s (total: 36.329739928245544s)]: Initalised Resources
default	15:52:25.202486 +0000	rswift	[T 0.0004290342330932617s (total: 36.33016896247864s)]: Defined generators
default	15:52:25.401039 +0000	rswift	[T 0.19853198528289795s (total: 36.528700947761536s)]: End

In my case, the following line takes 36 seconds to run:

let ignoreFile = (try? IgnoreFile(ignoreFileURL: callInformation.rswiftIgnoreURL)) ?? IgnoreFile()

My ignore file looks like the following:

**/es.lproj/*
**/fa-IR.lproj/*
**/fr.lproj/*
**/hu.lproj/*
**/id.lproj/*
**/pt-BR.lproj/*
**/th.lproj/*
**/vi.lproj/*
**/zh-Hant.lproj/*
**/it.lproj/*
**/el.lproj/*
**/da.lproj/*
**/ru.lproj/*
**/pl.lproj/*
**/hi-IN.lproj/*

Would be good to know if our performance issues are related or not

@marekpridal
Copy link

Great work and yes I did! We use it a lot because we use R framework basically only for localizables 👍🏻🙂

@liamnichols
Copy link

liamnichols commented Jan 4, 2019

Looks like we're on the same page then... I've dug into it a bit and it seems like the root cause is in Glob.swift, specifically Glob.expandGlobstar(pattern:) so I guess it's related to using ** in the ignore file.

I don't really know much about all of this but after Googling "glob", I found this doc that says the following:

Note Using the “**” pattern in large directory trees may consume an inordinate amount of time.

I guess it makes sense, our repo is large so there are a lot of directories to check..

Changing my .rswiftignore to the following reduces the overall execute time down to about ~0.5 seconds 🙌

Global/SupportingFiles/es.lproj/*
Global/SupportingFiles/fa-IR.lproj/*
Global/SupportingFiles/fr.lproj/*
Global/SupportingFiles/hu.lproj/*
Global/SupportingFiles/id.lproj/*
Global/SupportingFiles/pt-BR.lproj/*
Global/SupportingFiles/th.lproj/*
Global/SupportingFiles/vi.lproj/*
Global/SupportingFiles/zh-Hant.lproj/*
Global/SupportingFiles/it.lproj/*
Global/SupportingFiles/el.lproj/*
Global/SupportingFiles/da.lproj/*
Global/SupportingFiles/ru.lproj/*
Global/SupportingFiles/pl.lproj/*
Global/SupportingFiles/hi-IN.lproj/*

@marekpridal: Let me know if you are able to do the same and if you experience similar results.

I guess this might not actually be an issue but instead maybe could come under a basic troubleshooting tip or something?

Ps: Running git clean -fd removed a bunch of temporary ignored directories from my local repo (derived data etc) and reduced the ~30 seconds down to ~4 seconds as well but avoiding the globstar completely still gives the best results though

@alexandre-g
Copy link

Faced this issue also. With just one line **/CampaignsAndFormats.strings it was increasing script time from 4 to 11 seconds. Maybe should be mentioned in the readme until resolved

https://github.com/mac-cain13/R.swift/blob/master/Documentation/Ignoring.md

@pallzoltan
Copy link

pallzoltan commented Jan 13, 2021

To whomever is reading this, I found the --generators parameter very useful. Threw out the wildcarded paths (we had a bunch of those in .rswiftignore) and only use --generators --generators image,string,file,color.
We have 3 targets with R.swift (4 soon), each took 30 seconds before, now its 1s, 5s and 8s.

LE: More about the --generators param: https://github.com/mac-cain13/R.swift/blob/master/Documentation/Ignoring.md
LE2: Just to make it clear - what slowed the process down were the wildcards. Using the generators is just extra performance gain.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

5 participants