RetrieveStatus take so long time about 5s #1570

Activity

1. ethomson commented on Apr 20, 2018

   

   ethomson

   on Apr 20, 2018

   Member

   More actions

   Does git status take a long time? What about git status --ignored?
2. sumayn commented on Apr 20, 2018

   

   sumayn

   on Apr 20, 2018

   Author

   More actions

   so fast do git status or git status --ignored

   but use libgit2sharp api _repo.RetrieveStatus(new StatusOptions()) so slowly
3. Snegovikufa commented on Jun 5, 2018

   

   Snegovikufa

   on Jun 5, 2018

   More actions

   Let me describe the same issue for me.
   When libgit2 tries to get status of files (modified or not), library compares mtime/ctime flags from index to folder's attributes. If folder is modified, then libgit2 searches the contents of each file/folder in modified folder to figure out if file was modified or not. I don't know why, but on some machines folder attributes always diff from information written in git index. That forces library to watch every file in repository.
   So, many files in repository slows down performance of RetrieveStatus.

   One of our company computer calls RetrieveStatus for 40 seconds. On another computers that tooks only 3-4 seconds.
4. ethomson commented on Jun 5, 2018

   

   ethomson

   on Jun 5, 2018

   Member

   More actions

   What version of git? Are you sure the mtime and ctime in the index match the filesystem? In particular the high resolution bits?
5. Snegovikufa commented on Jun 5, 2018

   

   Snegovikufa

   on Jun 5, 2018

   More actions

   That issue was not on my own computer. Git version is something like 2.10.
   We started to monitor file system with Process Monitor (https://technet.microsoft.com/ru-ru/sysinternals/processmonitor.aspx).

   On "bad" computer there only records like at screenshot. All files in repository was opened.
   

   I compared the output of git ls-files --debug between two computers and saw that there were differences in mtime flag. I tried to understand what mtime means. mtime is the "modified time", right?
   I opened libgit2 source code and found that if comment mtime check, that fixes slow performance on "bad" computer:

   ----------------------- src/diff_generate.c -----------------------------
   index e11cbe4e4..b3e9a37c0 100644
   @@ -817,8 +817,9 @@ static int maybe_modified(
    			modified_uncertain =
    				(oitem->file_size <= 0 && nitem->file_size > 0);
    		}
   -		else if (!git_index_time_eq(&oitem->mtime, &nitem->mtime) ||
   -			(use_ctime && !git_index_time_eq(&oitem->ctime, &nitem->ctime)) ||
   +		else if (
   +			/*!git_index_time_eq(&oitem->mtime, &nitem->mtime) ||
   +			(use_ctime && !git_index_time_eq(&oitem->ctime, &nitem->ctime)) ||*/
    			oitem->ino != nitem->ino ||
    			oitem->uid != nitem->uid ||
    			oitem->gid != nitem->gid ||
   @@ -859,7 +860,7 @@ static int maybe_modified(
    	 * and an add of the new so that consumers can act accordingly (eg,
    	 * checkout will update the case on disk.)
    	 */
   -	if (DIFF_FLAG_IS_SET(diff, GIT_DIFF_IGNORE_CASE) &&
   +	/*if (DIFF_FLAG_IS_SET(diff, GIT_DIFF_IGNORE_CASE) &&
    		DIFF_FLAG_IS_SET(diff, GIT_DIFF_INCLUDE_CASECHANGE) &&
    		strcmp(oitem->path, nitem->path) != 0) {
    
   @@ -867,7 +868,7 @@ static int maybe_modified(
    			error = diff_delta__from_one(diff, GIT_DELTA_ADDED, NULL, nitem);
    
    		return error;
   -	}
   +	}*/
    
    	return diff_delta__from_two(
    		diff, status, oitem, omode, nitem, nmode,
   
6. ethomson commented on Jun 5, 2018

   

   ethomson

   on Jun 5, 2018

   Member

   More actions

   Are you copying this repository between two computers directly? Or sharing it on a network drive or something?
7. Snegovikufa commented on Jun 5, 2018

   

   Snegovikufa

   on Jun 5, 2018

   More actions

   We clone repository from remote TFS repository.
8. ethomson commented on Jun 5, 2018

   

   ethomson

   on Jun 5, 2018

   Member

   More actions

   Well, here's what I'm trying to understand: if the mtime is different from the cache, yes, libgit2 (and git) will inspect the file to determine if it's changed. So after cloning, something is touching that filestamp on disk so that it doesn't match the cache. Any idea what's doing that?
9. Snegovikufa commented on Jun 5, 2018

   

   Snegovikufa

   on Jun 5, 2018

   Last edited by Snegovikufa

   More actions

   something is touching that filestamp on disk so that it doesn't match the cache.

   I had that idea too but I don't know what modifies timestamps on disk.

   Is there any tool to synchronize git cache and timestamp on disk?
10. ethomson commented on Jun 5, 2018

    

    ethomson

    on Jun 5, 2018

    Member

    More actions

    Yes. git status will do that once it's determined that you haven't made any changes.
11. Snegovikufa commented on Jun 6, 2018

    

    Snegovikufa

    on Jun 6, 2018

    More actions

    git status --debug on my machine:

    ...
    env/DefaultTemplates/en-US/MS Excel template/MS Excel template.json
      ctime: 1510654479:766007800
      mtime: 1510654479:766007800
      dev: 0	ino: 0
      uid: 0	gid: 0
      size: 138	flags: 0
    ...
    

    On "bad" machine git version is 2.8.1. I confused with :0 value in ctime and mtime. Also, I don't know what 'dev' means

    env/DefaultTemplates/en-US/MS Excel template/MS Excel template.json
      ctime: 1522231113:0
      mtime: 1522231113:0
      dev: 2	ino: 0
      uid: 0	gid: 0
      size: 138	flags: 0
    
12. kboom commented on Jan 27, 2023

    

    kboom

    on Jan 27, 2023

    More actions

    This is the same issue for me. Takes a very long time while git status is instant.
13. PulsarFX commented on Jul 12, 2024

    

    PulsarFX

    on Jul 12, 2024

    More actions

    Was there any solution for those who were affected in the past?

    What is the C# code with all option flags to mimic git status?