Neural Interpretation of Generic Source Code

Can a generic (Python) program be executed statement-by-statement by neural networks composed according to the source code? We formulate the Abstract Neural Execution Problem and introduce Neural Interpretation, the first neural model that abstractly executes generic source code, where every variable has a vector encoding, and every function executes a neural network. Neural Interpretation is a model of computers with a compiler architecture, which can assemble neural layers ''programmed'' by partial source code. Neural Interpretation can be trained with flexible learning objectives. We demonstrate white-box execution without concrete inputs for variable misuse localization and repair.

Fix Bugs with Transformer through a Neural-Symbolic Edit Grammar

We introduce NSEdit (neural-symbolic edit), a novel Transformer-based code repair method. Given only the source code that contains bugs, NSEdit predicts an editing sequence that can fix the bugs. The edit grammar is formulated as a regular language, and the Transformer uses it as a neural-symbolic scripting interface to generate editing programs. We modify the Transformer and add a pointer network to select the edit locations. An ensemble of rerankers are trained to re-rank the editing sequences generated by beam search. We fine-tune the rerankers on the validation set to reduce over-fitting. NSEdit is evaluated on various code repair datasets and achieved a new state-of-the-art accuracy ($24.04\%$) on the Tufano small dataset of the CodeXGLUE benchmark. NSEdit performs robustly when programs vary from packages to packages and when buggy programs are concrete. We conduct detailed analysis on our methods and demonstrate the effectiveness of each component.