The aim of this project is to automatically generate transformation rules, which can be loaded into Pharo and propose you code changes in order to stay up to date with any API.
# AEMTransformRulesGenerator
When using an API within a project, it can happen that the API developer introduces some breaking changes. This means that your project wouldn't run properly anymore until you updated these changes within your code manually.
The aim of this project is to automatically generate transformation rules, which can be loaded into [Pharo] and propose you code changes in order to stay up to date with any API.
- [Project description](#project-description)
- [How to setup](#how-to-setup)
- [Automatic/script setup](#automaticscript-setup)
- [Manual setup](#manual-setup)
- [Possible errors](#possible-errors)
- [Future work](#future-work)
- [Weblinks](#weblinks)
## Project description
[APIEvolutionMiner] is an existing tool that mines a project history to detect API changes. This tool is also capable of producing static analyzer rules that detect an old API usage in source code.
Although the tool was proven to work well it was never used for two reasons:
- At the time when it was developed [Pharo] didn't have a nice UI for reporting critics generated by the rules.
- You had to setup APIEvolutionMiner by yourself and manually recover the rules.
While the first issue is mitigated by [QualityAssistant], the second one is solved by this project.
This project uses the APIEvolutionMiner for mining the revision history of any [SmalltalkHub](http://www.smalltalkhub.com/) repository. Furthermore, it uses the rules generated by the APIEvolutionMiner and converts them into useful rewrite rules.
This conversion is done by generating sub-classes of [RBTransformationRule](http://files.pharo.org/doc/4.0/class/RBTransformationRule.html). It respects also rules containing wildcards for objects or containing keyword messages.
However, there is a limitation if the number of arguments doesn't match. In this case the AEMTransformRulesGenerator cannot generate a rewrite rule, because there is a lack of information. It will simply skip these rules.
The resulting Smalltalk package file (`AEMTransformRules.st`) can be filed-in into your Pharo VM. Then, you can open the built-in _Critic Browser_ for viewing the API changes. Below you can find an example of viewing the rules for [Roassal2](http://www.smalltalkhub.com/#!/~ObjectProfile/Roassal2) within _Critic Browser_:
## How to setup
#### Automatic/script setup
1. Clone this repository into your preferred directory.
2. Run the script `Get-AEMRules.sh` with any SmalltalkHub repository as argument (replace the link of GraphET2 with our desired project):
```shell
$ ./Get-AEMRules.sh http://www.smalltalkhub.com/mc/ObjectProfile/GraphET2/main
```
3. The script will generate all the files in the following directory: `~/AEMTransformRules/AEMRules-{your_project_name}`
#### Manual setup
1. Create a new folder for the APIEvolutionMiner and project files.
2. Download the APIEvolutionMiner-Jet Pharo image from here: https://ci.inria.fr/rmod/view/MinedRules/job/APIEvolutionMiner-Jet/lastSuccessfulBuild/artifact/APIEvolutionMiner-Jet.zip
```shell
curl --silent --insecure -o APIEvolutionMiner-Jet.zip https://ci.inria.fr/rmod/view/MinedRules/job/APIEvolutionMiner-Jet/lastSuccessfulBuild/artifact/APIEvolutionMiner-Jet.zip
unzip -o APIEvolutionMiner-Jet.zip
```
3. Download latest Pharo VM here: http://get.pharo.org/vm or just use your existing Pharo VM (be aware, the APIEvolutionMiner needs the PharoV20.sources).
```shell
curl --silent get.pharo.org/vm | bash
```
4. Create the following sub-folders and files:
```shell
mkdir latestMczFiles
mkdir latestSourceCodeChanges
touch ourMczFiles
```
5. Open the Pharo-UI with the `APIEvolutionMiner-Jet.image` and download the latest MCZ-files for your desired SmalltalkHub project (replace the links and names of GraphET2 with our desired project). This step should write into the previously created file `ourMczFiles` and folder `latestMczFiles`. Attention: For a large project, this step can take a long time and downloads several MBs data.
```smalltalk
AEMRepositoryDownloader downloaLatestMczsFor: 'http://www.smalltalkhub.com/mc/ObjectProfile/GraphET2/main'.
```
6. Now import the ring history. This will close the UI, but it's okay. This step should generate the file `latestDataset` and write into the previously created folder `latestSourceCodeChanges`. Depending on the size of the project, this step needs more than the usual memory size. If it crashes due to memory, just add the following line to the VM `Pharo.ini`: `AddressSpaceLimit=1536` (update the number of memory according to your project).
```smalltalk
AEMRepositoryDownloader importToRingHistoryAndExportAssociations.
```
7. Copy the file `latestDataset` and rename the copy to `dataset`:
```shell
cp -u latestDataset dataset
```
8. Copy the folder `latestSourceCodeChanges` and rename the copy to `sourceCodeChanges`:
```shell
mkdir sourceCodeChanges
cp -r latestSourceCodeChanges/* sourceCodeChanges
```
9. Open the Pharo-UI with the `APIEvolutionMiner-Jet.image` again and file-in the following file:
- [AEMTransformRulesGenerator.st](./src/AEMTransformRulesGenerator.st?raw=true)
10. Then execute the following code within the Workspace:
```smalltalk
AEMSystemHistoryImporter loadSystemHistory: 'GraphET2'.
AEMTransformRulesGenerator new generateRBTransformationRules: ((AEMEvidenceRanking new systemHistory: ((AEMReport exportOneOneRules) systemHistory)) oneOneRules).
(RPackageOrganizer default packageNamed: #AEMTransformRules) fileOut.
```
11. The code above should have created the following files in your directory:
- rules.csv (containing the APIEvolutionMiner rules)
- AEMTransformRules.st (containing the RBTransformationRules, which can be directly used with Critic-Browser)
12. Finally the folder structure should look like this:
## Possible errors
- PharoDebug.log: `Error: can't find EOCD position`
- Solution:
1. Go into folder `latestMczFiles`
2. Sort _.mcz_ (actually ZIP archives) by file-size
3. Delete (remember the names) all files with file-size of 0 or 1 KB (corrupted, you can try to open them with your ZIP application to test them)
4. Go back to one folder above. Remove the deleted packages from the list in file `ourMczFiles`.
5. Finally, just run the Bash script with the same repository as input again.
## Future work
- Integrate the code into a CI service
- Debug the command `AEMRepositoryDownloader importToRingHistoryAndExportAssociations.` since it crashes very often (especially for bigger projects/APIs)
- Update AEM in order to run at least on Pharo 4 (currently it needs still Pharo 2)
## Weblinks
#### APIEvolutionMiner
- http://homepages.dcc.ufmg.br/~mtov/pub/2014_csmrwcre_apievolutionminer.pdf
- http://rmod.lille.inria.fr/web/software/APIEvolutionMiner
- http://smalltalkhub.com/#!/~andrehora/APIEvolutionMiner
#### Jenkins Jobs Inria RMoD Team
- https://ci.inria.fr/rmod/view/MinedRules/job/APIEvolutionMiner/lastSuccessfulBuild/consoleFull
- https://ci.inria.fr/rmod/view/MinedRules/job/Roassal-Monitoring/lastSuccessfulBuild/consoleFull
- https://ci.inria.fr/rmod/view/MinedRules/job/Roassal-APIEvolutionMiner/lastSuccessfulBuild/consoleFull
- https://ci.inria.fr/rmod/view/MinedRules/job/Moose-Report/lastSuccessfulBuild/consoleFull
- https://ci.inria.fr/pharo-contribution/view/Helper/job/JobTemplate/
#### Tested projects
- http://www.smalltalkhub.com/#!/~ObjectProfile/GraphET2
- http://www.smalltalkhub.com/mc/ObjectProfile/GraphET2/main
- http://www.smalltalkhub.com/#!/~ObjectProfile/Roassal2
- http://www.smalltalkhub.com/mc/ObjectProfile/Roassal2/main
#### PharoByExample
- http://pharobyexample.org/
- https://github.com/SquareBracketAssociates/UpdatedPharoByExample
- http://pharobooks.gforge.inria.fr/PharoByExampleTwo-Eng/latest/ZeroConf.pdf
#### Yuriy Tymchuk
- http://scg.unibe.ch/staff/YuriyTymchuk
- http://scg.unibe.ch/wiki/projects/mastersbachelorsprojects/apiEvolutionMiningService
- https://github.com/Uko
[APIEvolutionMiner]: http://homepages.dcc.ufmg.br/~mtov/pub/2014_csmrwcre_apievolutionminer.pdf
[Pharo]: http://pharo.org/
[QualityAssistant]:https://github.com/Uko/QualityAssistant