... | ... | @@ -3,7 +3,7 @@ |
|
|
* ICML 2018 paper, Kim et. al (google-research)
|
|
|
* Introduces the notion of a *concept activation vector* (CAV), a vector in some intermediate representation space of a DNN, pointing towards the direction of a concept. This vector is obtained by training a LinearSVM in the representation space, performing binary classification between in-concept and out-of-concept examples.
|
|
|
* Figure 1 from [1]:
|
|
|
![tcav_fig1](uploads/97e2298575d0a21290d89fd80c5f779a/tcav_fig1.png)
|
|
|
![tcav_fig1](uploads/tcav_fig1.png)
|
|
|
* ⓐ user-defined set of examples for some concept $`C`$ (top-row, e.g. 'striped') + random examples (bottom row)
|
|
|
* ⓑ labeled data examples for the studied class (e.g. zebras). Must correspond to a logit in the DNN. $`k`$ denotes the index of that logit.
|
|
|
* ⓒ DNN to be inspected. $`l`$ denotes the layer to hook into, i.e. the intermediate representation, and $`m`$ is the flattened intermediate representation.
|
... | ... | @@ -53,7 +53,7 @@ TCAV_{C,k,l} = \frac{|\{x \in X_k : S_{C,k,l}(x) > 0\}|}{|X_k|} \in [0,1] |
|
|
* Key idea: For image data, concepts are present in in the form of groups of pixels (segments)
|
|
|
* Figure 1 from [2]:
|
|
|
|
|
|
![ace_figure1](uploads/4a4a321bd3827258b50cd9b9b7dab339/ace_figure1.png)
|
|
|
![ace_figure1](uploads/ace_figure1.png)
|
|
|
|
|
|
* (a) Segment each image, using different resolutions to capture objects (concepts) of different abstraction levels (hierarchy of concepts assumption).
|
|
|
* (b) The extracted segments are resized to CNNs input size and fed through the network. Intermediate representations are clustered to find similar segments (and remove outliers), e.g. via k-means. Each found cluster defines a concept, represented by the instances inside. (previous work has shown that euclidean distance in final layers intermediate feature space is effective perceptual similarity metric) Outliers with low similarity to rest of cluster are removed. This is necessary to make every cluster clean of meaningless or dissimilar segments.
|
... | ... | @@ -112,7 +112,7 @@ TCAV_{C,k,l} = \frac{|\{x \in X_k : S_{C,k,l}(x) > 0\}|}{|X_k|} \in [0,1] |
|
|
|
|
|
* Figure 1 from [3]:
|
|
|
|
|
|
![cbe_fig1](uploads/dfaa420b8d384e2f5409805da826f23f/cbe_fig1.png)
|
|
|
![cbe_fig1](uploads/cbe_fig1.png)
|
|
|
|
|
|
|
|
|
|
... | ... | |