markdown 批量备份和克隆列表中的git存储库

Posted 2021-05-05

# Bulk backup and clone git repositories from a list

##  Get the git repos list

### Option A: Generate a list of repos already cloned on a machine 


To do this we will use the linux command **grep**.

> Please jump to [How to use linux command under Windows?](#how-to-use-linux-command-under-windows) for more information.


```
grep -oh "url = .*" */.git/config | cut -d " " -f3 > git_repos.txt
 ```
 
 
1. The command ```grep``` will search for the regular expresion ```"url = .*"``` inside the file **config** in any **.git** folder, one level down from the curren position.  
> In this hidden folder each git local repository stores the repository URL, inside the **config** file (no extension).  

> The arguments **o** will make grep to return only the matched (non-empty) parts of a matching line.  
> The argument  **h** will suppress the prefixing of file names on output.  

2. Then it whill pass each result line to the command ```cut```, that will split the string onthe character _space_ and then return the third piece, the actual git repo URL.

3. Finally it will append this line to the file git_repos.txt in the current folder (it will automatically create the file).


### Option B: Download a list of all repos SSH urls from a project in Bitbucket.uhub.biz.

In this case we want to download the SSH url for all the existing repos inside a proyect in Bitbucket.

1. Run this command to download a JSON file with all the repos information, including the SSH clone url.
    ```
    $ curl --request GET \
    --url 'http://bitbucket.org/rest/api/1.0/projects/[[PROJECT_NAME]]/repos/?limit=1000' \
    --header 'authorization: Basic [[AUTH_STRING]]' \
    --header 'content-type: application/json'

    ```
    - Replace [[PROJECT_NAME]] with the parent project name of the repos you want to download the info.
    - Replace [[AUTH_STRING]] with your user email and  password, encoded as a base 64 string (ex.: [e-mail]:password)
2. To extract the ssh urls filter the JSON file go to http://jsonpath.com/
    - In filed _JSONPath Syntax_ copy and paste this string: ```$.values.*.links.clone[?(@.name=="ssh")].href```
    - In field _JSON_ copy and paste the JSON result downloaded on step 1.
3. Create a new git_repos.txt file and copy the content of field _Evaluation Results_.
4. Search and replace any ```[]",``` character from the list. Also remove all leading spaces on the lines.
5. All done. Your ready to start cloning.



## Batch clone git repositories in the target machine

Now we need to copy the **git_repos.txt** file with the repositories list to the target machine, and batch clone them.

To do so, we will use this command:

``` 
$ grep . git_repos.txt | while read line ; do git clone "$line"; done
```

### How this works? 

1. The command ```grep``` will read all the lines in the **git_repos.txt** file, and pass each line to the next command.
2. The command ```while``` will read each line and execute as a command the string ```git clone``` followed by the current line (a repository URL).

---

# Apendix

## How to use linux command under Windows?

- On **Windows 10** you can use the "Windows Subsystem Linux".  
- Since you must have **Git** installed, another option is to use the **Git Bash** application.
- And yet another option is to install [GnuWin32](http://gnuwin32.sourceforge.net/) to add support for linux commands to Windows.