Update file content with golang and regexp
In this article, I will describe how I have done to replace hundreds of files together with program.
Recently I changed the i18n solution on one of our projects, and I need to change hundreds of files, I tried to update the file by hand, but it was too time consuming. So I think about writing a program to update the files automatically.
The project below is what I have written to update the files, I added some demo files to demonstrate what the program do. The demo file was in the demodata folder, in this folder, I put some demo files which includes java, kotlin , objc and swift files, the demo demonstrate how to update the files from StringProvider to StringVaues.
Problem
In this demo, take the java file for example, the original content is like below:String value1 = StringProvider.getValue("key1", "default value 1");
String value2 = StringProvider.getValue("key2", "default value 2");
String value3 = isAdd ? StringProvider.getValue("key3", "default value 3") : StringProvider.getValue("key4","default value 4");
String value4 = StringProvider.getValue("key5",
"default value 5");
The code above call StringProvider’s method to get value according to a key, and I want to change it to another class, which has variable named as key name, such as StringValues.key1 . The source code has multiple formats, some of them has no space after comma, some of them has multiple lines. I need to search all of them and replace them with the new method.
Solution
The project I created, which can register update rules, and the application will use the rule to search the text and replace the text according to the rule.
Update Rule
The update rule can config the file exts, regexp patterns and match replacer.
The struct for the update rule is like below:type UpdateRule struct {
Exts []string
Pattern string
GetMatchReplacer func(match []string) *string
}
The application will search the files with extension configured in the rule, and search the text with the patten in the rule, the pattern is the regexp pattern, the GetMatchReplacer has the match parameter, which is get from the regexp, and return the replace string to the application, if the return value is nil, the text won’t replaced, when the return value is not nil, it will be used as the destination.
List files in a directory
List file will search the files in the search dir, it will check the extension of the file, add return the matched files, this function is a recursive function, if the item in the folder is dir, then will call the function again with the sub dir. The following code is the implementation of the function.func (updater fileUpdater) listFiles(searchDir string) []string {
items, err := ioutil.ReadDir(searchDir)
if err != nil {
logs.Errorf("Failed to get items from dir %s, the error is %#v", searchDir, err)
return nil
}files := make([]string, 0)
for _, item := range items {
itemName := item.Name()
subDir := filepath.Join(searchDir, itemName)
if item.IsDir() {
subFiles := updater.listFiles(subDir)
if err != nil {
logs.Errorf("Failed to get sub file, the error is %#v", err)
return nil
}
files = append(files, subFiles...)
} else {
fileExt := path.Ext(itemName)
for _, ext := range updater.fileExts {
if fileExt == ext {
files = append(files, filepath.Join(searchDir, itemName))
}
}
}
}
return files
}
Filter string with regexp
I use the regexp to filter the strings need to replace, I used the function func (re *Regexp) FindAllStringSubmatch(s string, n int) [][]string to find the strings, the following is the description to the function.
// FindAllStringSubmatch is the ‘All’ version of FindStringSubmatch; it
// returns a slice of all successive matches of the expression, as defined by
// the ‘All’ description in the package comment.
// A return value of nil indicates no match.
There are something that I learned from the project:
The example patten for java file is:"StringProvider\\.getValue\\(\"(.*?)\",\\s*?\"(.*?)\"\\)"
With this rule, the text below will return three items.String value1 = StringProvider.getValue("key1", "default value 1");
The result of the FindAllStringSubmatch has three items, the first item is StringProvider.getValue("key1", "default value 1") , the second item is key1 and the last item is default value 1 .
In the pattern, I use \\s*? to match the whitespace between the terms, and the placeholder (.*?) I added a ? at the end, this will not use the greedy mode, so If I have two match items, it will return two item.
