profile
viewpoint
Ivan Gotovchits ivg Carnegie Mellon University, Cylab Pittsburgh, PA bap.ece.cmu.edu

BinaryAnalysisPlatform/bil 11

A formal specification for BIL

BinaryAnalysisPlatform/bap-veri 8

bil verification tool

BinaryAnalysisPlatform/bap-mode 5

An Emacs major mode for BAP's intermediate language

ivg/bap 4

BAP Core Library

ivg/date-time 4

a simple library to handle dates and times in ocaml

ivg/argot 3

Enhanced HTML generator for ocamldoc [mirror]

ivg/bap-2017-workshop 2

Slides and code from the BAP 2017 workshop in Cylab's partners conference

ivg/bap-workshops 2

Slides and code for bap-workshops

ivg/emacs-dot 2

minor mode for graph visualization

push eventBinaryAnalysisPlatform/bap

Ivan Gotovchits

commit sha a2812fb1f961c3123e81e2d08075ca50303f8163

error handling and performance tweaks (#1375) * error handling and performance tweaks Sorry for a little bit dirty pull request featuring several independent changes, but it is really tedious to split it into several PRs. The most important and controversial change is that we now capture conflicts when disassemble. The conflict is reported, the instruction is marked as invalid and the whole chain instructions that led to it, is rejected. The previous behavior was terminating the program with an error message, which to my taste is more correct, but is unsatisfying from the user-perspective. We might return the old behavior after the release, in the development version of bap, as most of the conflicts are programmers' errors. The motivation for capturing conflicts was introducing pattern-based instruction encoding identification, which naturally can have false positives (which we occasionally see in some ARM interworked binaries). The other changes concern how our lifters report and handle errors. Since we can now have multiple lifters each supplying semantics for a subset of the instruction set, we no longer assume that an unhandled instruction is an error (it might be handled by some other lifter). We still report errors on instructions that a lifter claim to lift but failed to lift. And since the log file is now much cleaner, I was able to detect some missing cases, mostly for thumb2 (via arm) instructions, which are now handled. Finally, this PR removes the `is-valid` promise from thumb, which wasn't necessary but was introducing a little bit of slowdown. * fixes the empty instruction BIL intrinsic

view details

push time in 14 hours

PR merged BinaryAnalysisPlatform/bap

error handling and performance tweaks

Sorry for a little bit dirty pull request featuring several independent changes, but it is really tedious to split it into several PRs.

The most important and controversial change is that we now capture conflicts when disassemble. The conflict is reported, the instruction is marked as invalid and the whole chain instructions that led to it, is rejected. The previous behavior was terminating the program with an error message, which to my taste is more correct, but is unsatisfying from the user-perspective. We might return the old behavior after the release, in the development version of bap, as most of the conflicts are programmers' errors. The motivation for capturing conflicts was introducing pattern-based instruction encoding identification, which naturally can have false positives (which we occasionally see in some ARM interworked binaries).

The other changes concern how our lifters report and handle errors. Since we can now have multiple lifters each supplying semantics for a subset of the instruction set, we no longer assume that an unhandled instruction is an error (it might be handled by some other lifter). We still report errors on instructions that a lifter claim to lift but failed to lift.

And since the log file is now much cleaner, I was able to detect some missing cases, mostly for thumb2 (via arm) instructions, which are now handled.

Finally, this PR removes the is-valid promise from thumb, which wasn't necessary but was introducing a little bit of slowdown.

+98 -111

0 comment

9 changed files

ivg

pr closed time in 14 hours

push eventivg/bap

ivg

commit sha bb752da26c0ed44aa2e9de3d8bf30009e7c1f093

fixes the empty instruction BIL intrinsic

view details

push time in 15 hours

PR opened BinaryAnalysisPlatform/bap

error handling and performance tweaks

Sorry for a little bit dirty pull request featuring several independent changes, but it is really tedious to split it into several PRs.

The most important and controversial change is that we now capture conflicts when disassemble. The conflict is reported, the instruction is marked as invalid and the whole chain instructions that led to it, is rejected. The previous behavior was terminating the program with an error message, which to my taste is more correct, but is unsatisfying from the user-perspective. We might return the old behavior after the release, in the development version of bap, as most of the conflicts are programmers' errors. The motivation for capturing conflicts was introducing pattern-based instruction encoding identification, which naturally can have false positives (which we occasionally see in some ARM interworked binaries).

The other changes concern how our lifters report and handle errors. Since we can now have multiple lifters each supplying semantics for a subset of the instruction set, we no longer assume that an unhandled instruction is an error (it might be handled by some other lifter). We still report errors on instructions that a lifter claim to lift but failed to lift.

And since the log file is now much cleaner, I was able to detect some missing cases, mostly for thumb2 (via arm) instructions, which are now handled.

Finally, this PR removes the is-valid promise from thumb, which wasn't necessary but was introducing a little bit of slowdown.

+96 -107

0 comment

9 changed files

pr created time in 6 days

create barnchivg/bap

branch : error-handling-performance-tweaks

created branch time in 6 days

issue commentBinaryAnalysisPlatform/bap

I want to get the system call graph of the executable. Is this option available?I know -dcallgraph can get the call graph.

Can you please define what do you mean by the "system call graph"?

lkpama

comment created time in 8 days

issue commentBinaryAnalysisPlatform/bap

Can the tool perform dynamic taint analysis?

See also https://github.com/BinaryAnalysisPlatform/bap/wiki/Using-BAP-for-Taint-Analysis

wjcif11

comment created time in 8 days

delete branch ivg/bap

delete branch : fixes-t32-mic

delete time in 8 days

push eventBinaryAnalysisPlatform/bap

Ivan Gotovchits

commit sha 2c871e92343abd5e8831ae30d53566907e19d3ee

fixes handling modified immediate constants in ARM T32 encoding (#1374) We use the old ARM lifter to handle T32 instructions but, unlike in ARM, LLVM does decode modified immediate constants in T32 mode so we don't need to decode them twice. This change uses the encoding of the instruction for deciding whether the immediate is in the shifted form, or already decoded. Thanks @ccasin for detecting the problem.

view details

push time in 8 days

PR merged BinaryAnalysisPlatform/bap

fixes handling modified immediate constants in ARM T32 encoding

We use the old ARM lifter to handle T32 instructions but, unlike in ARM, LLVM does decode modified immediate constants in T32 mode so we don't need to decode them twice. This change uses the encoding of the instruction for deciding whether the immediate is in the shifted form or is already decoded.

Thanks @ccasin for detecting the problem.

+66 -58

0 comment

3 changed files

ivg

pr closed time in 8 days

PR opened BinaryAnalysisPlatform/bap

fixes handling modified immediate constants in ARM T32 encoding

We use the old ARM lifter to handle T32 instructions but, unlike in ARM, LLVM does decode modified immediate constants in T32 mode so we don't need to decode them twice. This change uses the encoding of the instruction for deciding whether the immediate is in the shifted form or is already decoded.

Thanks @ccasin for detecting the problem.

+66 -58

0 comment

3 changed files

pr created time in 8 days

create barnchivg/bap

branch : fixes-t32-mic

created branch time in 8 days

push eventBinaryAnalysisPlatform/bap

Ivan Gotovchits

commit sha 1d698020756810ac695324d4f4201b263f31c8f0

fixes the signatures URL in the release script (#1373)

view details

push time in 8 days

create barnchivg/bap

branch : fixes-release-sigs-url

created branch time in 8 days

push eventBinaryAnalysisPlatform/bap

Ivan Gotovchits

commit sha 63c6950ba740ce07779abc58d29436eccc43d2e3

tweaks the method resolution to keep super methods (#1370) * tweaks the method resolution to keep super methods This change affects both dynamic and static interpreters, so read carefully. Before this commit we used basically the same resolution procedure for all Primus Lisp definitions, with the only exception, in the end, we allow more than one method definition for the same name. The caveat is that when more than one method of the same class are applicable, the most specific methods were chose and the least specific, i.e., the super methods, were removed from the resolution. On one hand, this enables refinement of a method, on the other we don't have any notation to call the super method, so the refinement is more like a defininition. This aproach is suitable with normal function defintions that are required to be unique (and when we need refinement we can always factor out the common part from the parent definition and reuse it in the more specific one). But methods are used for different purposes - they process signals from the knowledge base or from the Primus Interpreter and when we add a more specific reaction to the signal we still want to keep other reactions (and the knowledge base will actually take care of the refinement by calling Value.merge on the method results). To highlight the problem here is example from the real world (that triggers this change). We have two definitions of the `bap:pattern-actions` method, one that is generic and handles `'funcstart` and `'possiblefuncstart` method, and another that is specific to arm and handles arm/thumb interworking via setcontext. Right now they all defined in the same file, and arm-specific method is triggered even for non-arm targets. Moreover, when arm is not enabled (i.e., not installed or specifically disabled with --no-arm), the method fails on the typechecker (and will fail as well in runtime) because the `bap:arm-set-encoding` primitive is not available. The immediate solution is to properly constrain the context of applicapibility of this method definition to `(context (target arm))`, which leads to the disastrous results. Now the resolver thinks that the arm-specific method that handles contexts, is the overload of the more general method (that handles function starts), so it expels the parent method from the list and never calls it. Not what we wanted! In fact, we want all applicable methods to be called no matter their specificity. In other words, when a method is called, we want to call all methods, starting from the parents and ending with the children. This is a little bit breaking change, as if there exists the Primus Lisp code that was relying on the overriding behavior (none that I am aware of) it will no longer work as expected. The solution is to rely on the overloading of normal functions, e.g., ``` (defmethod do-stuff (x) (declare (context worker)) parent-code) (defmethod do-stuff (x) (declare (context worker child)) child-code) ``` should be rewritten as, ``` (defun do-stuff (x) (declare (context worker)) parent-code) (defun do-stuff (x) (declare (context worker child)) child-code) (defmethod do-stuff (x) (do-stuff x)) ``` * specializes the arm-specific patterns-action and moves it away Moves the arm-specific patterns-action that controls arm/thumb selection to the arm plugin and narrows down its context so that it will be triggered only when the target is arm. * fixes the now failing testsuite The failure is rather funny, see the linked commit.

view details

push time in 11 days

PR merged BinaryAnalysisPlatform/bap

tweaks the method resolution to keep super methods

This change affects both dynamic and static interpreters, so read carefully. Before this commit we used basically the same resolution procedure for all Primus Lisp definitions, with the only exception, in the end, we allow more than one method definition for the same name.

The caveat is that when more than one method of the same class is applicable, the most specific methods were chosen and the least specific, i.e., the super methods, were removed from the resolution. On one hand, this enables refinement of a method, on the other we don't have any notation to call the super method, so the refinement is more like a redefinition. This approach is suitable with normal function definitions that are required to be unique (and when we need refinement we can always factor out the common part from the parent definition and reuse it in the more specific one). But methods are used for different purposes - they process signals from the knowledge base or from the Primus Interpreter and when we add a more specific reaction to the signal we still want to keep other reactions (and the knowledge base will actually take care of the refinement by calling Value.merge on the method results).

To highlight the problem here is an example from the real world (that triggers this change). We have two definitions of the bap:pattern-actions method, one that is generic and handles 'funcstart and 'possiblefuncstart actions, and another that is specific to arm and handles arm/thumb interworking via setcontext. Right now they are all defined in the same file, and arm-specific method is triggered even for non-arm targets. Moreover, when arm is not enabled (i.e., not installed or specifically disabled with --no-arm), the method fails on the typechecker (and will fail as well in runtime) because the bap:arm-set-encoding primitive is not available. The immediate solution is to properly constrain the context of applicability of this method definition to (context (target arm)), which leads to disastrous results. Now the resolver thinks that the arm-specific method that handles contexts, is the overload of the more general method (that handles function starts), so it expels the parent method from the list and never calls it. Not what we wanted! In fact, we want all applicable methods to be called no matter their specificity. In other words, when a method is called, we want to call all methods, starting from the parents and ending with the children.

This is a little bit breaking change as if there exists the Primus Lisp code that was relying on the overriding behavior (none that I am aware of) it will no longer work as expected. The solution is to rely on the overloading of normal functions, e.g.,

(defmethod do-stuff (x)
  (declare (context worker))
  parent-code)

(defmethod do-stuff (x)
  (declare (context worker child))
  child-code)

should be rewritten as,

(defun do-stuff (x)
  (declare (context worker))
  parent-code)

(defun do-stuff (x)
  (declare (context worker child))
  child-code)

(defmethod do-stuff (x)
  (do-stuff x))
+32 -14

0 comment

4 changed files

ivg

pr closed time in 11 days

issue commentBinaryAnalysisPlatform/bap

Can the tool perform dynamic taint analysis?

Yes, we have the dynamic (using micro-execution) taint analysis engine, you can run it on an executable ./exe with

bap ./exe --run --run-system=taint-analyzer

The analyzer will run the binary using microexecution and propagate taint. You can define your own policies (aka analysis), using Primus Lisp. Some of the example policy specifications (for you to get the general idea) could be found here or here. You're welcome to join our Gitter channel for more in-depth and detailed discussion.

wjcif11

comment created time in 11 days

push eventBinaryAnalysisPlatform/bap

Ivan Gotovchits

commit sha 52449bd6fc5b005799939989b96c140ccc4b4493

turns glibc-runtime main discovery into a lazy pass (#1371) This is a small optimization that makes bap faster if you're not using Program.t

view details

push time in 11 days

PR merged BinaryAnalysisPlatform/bap

turns glibc-runtime main discovery into a lazy pass

This is a small optimization that makes bap faster if you're not using Program.t

+12 -11

0 comment

1 changed file

ivg

pr closed time in 11 days

push eventivg/bap

ivg

commit sha 13e72bd4e500663c2758f3feb15c3f1100e326e5

fixes the now failing testsuite The failure is rather funny, see the linked commit.

view details

push time in 11 days

push eventBinaryAnalysisPlatform/bap-testsuite

ivg

commit sha 44771dd87534205fd5b9c1fad6b05d9d5b15d87e

removes unnecessary initialization procedure for aarch-crc32 It was never called and was originally wrought for debugging this file under the normal executor, but the it was really breaking the symbolic executor, except that before BinaryAnalysisPlatform/bap#1370 it was never called (yet another reason to merge it, as it looks like from the way how we use methods we were assuming the behavior that bap#1370 just introduces).

view details

push time in 11 days

PR opened BinaryAnalysisPlatform/bap

turns glibc-runtime main discovery into a lazy pass

This is a small optimization that makes bap faster if you're not using Program.t

+12 -11

0 comment

1 changed file

pr created time in 11 days

create barnchivg/bap

branch : optimizes-glibc-runtime

created branch time in 11 days

push eventBinaryAnalysisPlatform/bap

Benjamin Mourad

commit sha 0d0cd1c67c8ad8df6abd6edd4706c9c85ea559b6

Adds some missing `t2LDR.*i12` instructions to the Thumb lifter (#1369) * Update LLVM backend to work with version 12 This may also work with later versions, but I did not test them * Adds some missing `t2LDR.*i12` instructions to the Thumb lifter

view details

push time in 11 days

PR merged BinaryAnalysisPlatform/bap

Adds some missing `t2LDR.*i12` instructions to the Thumb lifter

These were missing when I encountered them.

+15 -0

0 comment

1 changed file

bmourad01

pr closed time in 11 days

push eventivg/bap

ivg

commit sha 5c5ceebcd9c066b8b4155fdb90bb63da28c776f6

specializes the arm-specific patterns-action and moves it away Moves the arm-specific patterns-action that controls arm/thumb selection to the arm plugin and narrows down its context so that it will be triggered only when the target is arm.

view details

push time in 11 days

PR opened BinaryAnalysisPlatform/bap

tweaks the method resolution to keep super methods

This change affects both dynamic and static interpreters, so read carefully. Before this commit we used basically the same resolution procedure for all Primus Lisp definitions, with the only exception, in the end, we allow more than one method definition for the same name.

The caveat is that when more than one method of the same class are applicable, the most specific methods were chose and the least specific, i.e., the super methods, were removed from the resolution. On one hand, this enables refinement of a method, on the other we don't have any notation to call the super method, so the refinement is more like a defininition. This aproach is suitable with normal function defintions that are required to be unique (and when we need refinement we can always factor out the common part from the parent definition and reuse it in the more specific one). But methods are used for different purposes - they process signals from the knowledge base or from the Primus Interpreter and when we add a more specific reaction to the signal we still want to keep other reactions (and the knowledge base will actually take care of the refinement by calling Value.merge on the method results).

To highlight the problem here is example from the real world (that triggers this change). We have two definitions of the bap:pattern-actions method, one that is generic and handles 'funcstart and 'possiblefuncstart method, and another that is specific to arm and handles arm/thumb interworking via setcontext. Right now they all defined in the same file, and arm-specific method is triggered even for non-arm targets. Moreover, when arm is not enabled (i.e., not installed or specifically disabled with --no-arm), the method fails on the typechecker (and will fail as well in runtime) because the bap:arm-set-encoding primitive is not available. The immediate solution is to properly constrain the context of applicapibility of this method definition to (context (target arm)), which leads to the disastrous results. Now the resolver thinks that the arm-specific method that handles contexts, is the overload of the more general method (that handles function starts), so it expels the parent method from the list and never calls it. Not what we wanted! In fact, we want all applicable methods to be called no matter their specificity. In other words, when a method is called, we want to call all methods, starting from the parents and ending with the children.

This is a little bit breaking change, as if there exists the Primus Lisp code that was relying on the overriding behavior (none that I am aware of) it will no longer work as expected. The solution is to rely on the overloading of normal functions, e.g.,

(defmethod do-stuff (x)
  (declare (context worker))
  parent-code)

(defmethod do-stuff (x)
  (declare (context worker child))
  child-code)

should be rewritten as,

(defun do-stuff (x)
  (declare (context worker))
  parent-code)

(defun do-stuff (x)
  (declare (context worker child))
  child-code)

(defmethod do-stuff (x)
  (do-stuff x))
+20 -5

0 comment

1 changed file

pr created time in 11 days

more