nnsight.net Open in urlscan Pro
35.232.255.106  Public Scan

Form analysis
1 forms found in the DOM

GET search/

<form class="bd-search d-flex align-items-center" action="search/" method="get">
  <svg class="svg-inline--fa fa-magnifying-glass" aria-hidden="true" focusable="false" data-prefix="fas" data-icon="magnifying-glass" role="img" xmlns="http://www.w3.org/2000/svg" viewBox="0 0 512 512" data-fa-i2svg="">
    <path fill="currentColor"
      d="M416 208c0 45.9-14.9 88.3-40 122.7L502.6 457.4c12.5 12.5 12.5 32.8 0 45.3s-32.8 12.5-45.3 0L330.7 376c-34.4 25.2-76.8 40-122.7 40C93.1 416 0 322.9 0 208S93.1 0 208 0S416 93.1 416 208zM208 352a144 144 0 1 0 0-288 144 144 0 1 0 0 288z"></path>
  </svg><!-- <i class="fa-solid fa-magnifying-glass"></i> Font Awesome fontawesome.com -->
  <input type="search" class="form-control" name="q" id="search-input" placeholder="Search the docs ..." aria-label="Search the docs ..." autocomplete="off" autocorrect="off" autocapitalize="off" spellcheck="false">
  <span class="search-button__kbd-shortcut"><kbd class="kbd-shortcut__modifier">Ctrl</kbd>+<kbd>K</kbd></span>
</form>

Text Content

Skip to main content

Back to top
Ctrl+K
 * Getting Started
 * Documentation
 * Features
 * Tutorials
 * About

Search Ctrl+K

 * GitHub
 * Discord
 * Status: Unknown


Search Ctrl+K
 * Getting Started
 * Documentation
 * Features
 * Tutorials
 * About

 * GitHub
 * Discord
 * Status: Unknown






NNSIGHT#




INTERPRETABLE NEURAL NETWORKS

NNsight (/ɛn.saɪt/) is a package for interpreting and manipulating the internals
of models.

Start Docs Tutorials About



FEATURES

See More →
Wrap any PyTorch model NNsight class object Access any hidden state Expose
Module inputs and outputs Develop complex interventions Edit module outputs and
measure effect

from nnsight import NNsight, LanguageModel

net = torch.nn.Sequential(OrderedDict([
  ('layer1', torch.nn.Linear(input_size, hidden_dims)),
  ('layer2', torch.nn.Linear(hidden_dims, output_size)),
]))

model = NNsight(net)

...

model = LanguageModel('openai-community/gpt2')

with model.trace('Who invented neural networks?'):

  hidden_state_output = model.layer1.output.save()
  hidden_state_input = model.layer2.input.save()

  output = model.output.save()
  
print(hidden_state_output)
print(hidden_state_input)
print(output)

with model.trace() as tracer:
  
  with tracer.invoke('The Eiffel Tower is in the city of'):

    model.transformer.h[-1].mlp.output[0][:] = 0

    intervention = model.lm_head.output.argmax(dim=-1).save()

  with tracer.invoke('The Eiffel Tower is in the city of'):

    original = model.lm_head.output.argmax(dim=-1).save()

print(output)





(c) 2024 NDIF

Built with the PyData Sphinx Theme 0.15.1.