Configuration

IVRE has several configuration variables. The default values are hard-coded in ivre/config.py. You should not change this file, unless you are modifying IVRE and you want to change the default configuration. You do not need to do this if you want to install IVRE with a non-default configuration, you just need to distribute a proper configuration file.

IVRE can be configured using different configuration files:

system-wide: ivre.conf in the following directories: /etc/, /etc/ivre, /usr/local/etc, /usr/local/etc/ivre.
user-specific: ~/.ivre.conf (read after the system-wide configuration files, so higher priority).
execution-specific: another configuration file can be specified using the $IVRE_CONF environment variable (read after the user-specific file, so highest priority).

The configuration files are Python files setting global variables.

Debug

Debug messages are turned off by default, since IVRE has no bugs. DEBUG_DB turns on database-specific debug messages, and can be very noisy. Setting DEBUG to True is mandatory to run IVRE’s tests.

Databases

Databases are specified using URLs:

db_type://[username[:password]@][host[:port]]/databasename?options

DB is the generic database URL (will be used for all Purposes unless a purpose-specific URL has been specified). The value "mongodb:///ivre" is the default and means “use MongoDB on localhost, database ivre, default collection names”.

Purpose-specific URLs can be specified using DB_<purpose>; DB_DATA is specific and defaults to None, which has the special meaning "maxmind:///<ivre_share_path>/geoip".

Here are some examples:

DB_PASSIVE = "postgresql://ivre@localhost/ivre"
DB_NMAP = "mongodb:///ivre"
DB_VIEW = "elastic://192.168.0.1:9200/ivre"
DB_DATA = "maxmind:///share/data/ivre/geoip"

Batch insert or upsert operations can be tuned using backend-specific variables:

LOCAL_BATCH_SIZE = 10000  # used with --local-bulk
MONGODB_BATCH_SIZE = 100
POSTGRES_BATCH_SIZE = 10000

Paths and commands

All variables ending with _PATH (except NMAP_SHARE_PATH) default to None, a special value which means “try to guess the path based on IVRE installation”.

Here are the values with examples on a regular installation:

DATA_PATH = None                  # /usr/share/ivre/data
GEOIP_PATH = None                 # /usr/share/ivre/geoip
HONEYD_IVRE_SCRIPTS_PATH = None   # /usr/share/ivre/data/honeyd
WEB_STATIC_PATH = None            # /usr/share/ivre/web/static
WEB_DOKU_PATH = None              # /usr/share/ivre/dokuwiki

NMAP_SHARE_PATH defaults to None, which means IVRE will try "/usr/local/share/nmap", "/opt/nmap/share/nmap", then "/usr/share/nmap".

IVRE may need some executables:

TESSERACT_CMD = "tesseract"
OPENSSL_CMD = "openssl"

Masscan probes

IVRE can use the service fingerprint database from Nmap to find service and product names from Masscan results. For that, IVRE needs to know which probe (or “hello string”) has been used. This depends on Masscan source code (compile-time) and options (run-time). You can adjust what IVRE will use per port (from the configuration) or globally (from the command-line option).

The default configuration value is based on the Masscan fork of the IVRE project.

# Based on IVRE's fork source code --- you may want to adapt these
# settings if you use another version of Masscan.
MASSCAN_PROBES = {
    "tcp": {
        53: "DNSVersionBindReqTCP",
        88: "Kerberos",
        104: "dicom",
        111: "RPCCheck",
        130: "NotesRPC",
        135: "DNSVersionBindReqTCP",
        256: "LDAPSearchReq",
        257: "LDAPSearchReq",
        389: "LDAPSearchReq",
        390: "LDAPSearchReq",
        406: "SIPOptions",
        427: "NotesRPC",
        548: "afp",
        554: "RTSPRequest",
        1098: "JavaRMI",
        1099: "JavaRMI",
        1352: "NotesRPC",
        1433: "ms-sql-s",
        1702: "LDAPSearchReq",
        1972: "NotesRPC",
        2049: "RPCCheck",
        2345: "dicom",
        2375: "docker",
        2379: "docker",
        2380: "docker",
        2761: "dicom",
        2762: "dicom",
        3268: "LDAPSearchReq",
        3892: "LDAPSearchReq",
        4242: "dicom",
        5060: "SIPOptions",
        6000: "X11Probe",
        6001: "X11Probe",
        6002: "X11Probe",
        6003: "X11Probe",
        6004: "X11Probe",
        6005: "X11Probe",
        6006: "X11Probe",
        6007: "X11Probe",
        6008: "X11Probe",
        6009: "X11Probe",
        6010: "X11Probe",
        6011: "X11Probe",
        6012: "X11Probe",
        6013: "X11Probe",
        6014: "X11Probe",
        6015: "X11Probe",
        6016: "X11Probe",
        6017: "X11Probe",
        6018: "X11Probe",
        6019: "X11Probe",
        6379: "redis-server",
        7171: "NotesRPC",
        8081: "SIPOptions",
        8554: "RTSPRequest",
        8728: "NotesRPC",
        9001: "mongodb",
        11112: "dicom",
        11711: "LDAPSearchReq",
        22001: "NotesRPC",
        27017: "mongodb",
        31337: "SIPOptions",
        49153: "mongodb",
        50000: "DNSVersionBindReqTCP",
        50001: "DNSVersionBindReqTCP",
        50002: "DNSVersionBindReqTCP",
    },
}

The `flow` purpose

The flow purpose has several specific configuration options, which may have important impacts on performances; here are the options and their default values:

# Dictionary that helps determine server ports of communications. Each entry
# is {proto: {port: proba}}. The when two ports are known, the port with the
# highest probability is used.
# When /usr/share/nmap/nmap-services is available, these probas are taken,
# otherwise /etc/services is used with proba=0.5 for each entry.
# KNOWN_PORTS entries have the highest priority.
# Example:
#  KNOWN_PORTS = {
#      "udp": {
#          9999: 1.0,
#          12345: 0.5,
#      },
#      "tcp": {
#          20202: 0.8,
#      },
#  }
KNOWN_PORTS: dict[str, dict[int, float]] = {}
# Enable the recording of appearance times for flows. Will slow down a
# bit the insertion rate
FLOW_TIME = True
# Precision (in seconds) to use when recording times when flows appear
FLOW_TIME_PRECISION = 3600
# When recording flow times, record the whole range from start_time to end_time
# This option is experimental and possibly useless in practice
FLOW_TIME_FULL_RANGE = True
# When recording flow times, represents the beginning of the first timeslot
# as a Unix timestamp shifted to local time.
# 0 means that the first timeslot starts at 1970-01-01 00:00 (Local time).
FLOW_TIME_BASE = 0
# Store high level protocols metadata in flows. It may take much more space.
FLOW_STORE_METADATA = True

The `data` purpose

The URLs used to get IP address databases are set in the dictionary IPDATA_URLS:

IPDATA_URLS = {
    # None has a special meaning:
    # https://download.maxmind.com/app/geoip_download?edition_id=XXX&suffix=XXX&license_key=XXX
    #
    # You can use this value for the GeoLite2-* files (and set
    # MAXMIND_LICENSE_KEY below) to download files from MaxMind
    # instead of ivre.rocks directly. Maxmind license keys are free
    # and can be obtained from <https://www.maxmind.com/>
    "GeoLite2-City.tar.gz": "https://ivre.rocks/data/geolite/GeoLite2-City.tar.gz",
    "GeoLite2-City-CSV.zip": "https://ivre.rocks/data/geolite/GeoLite2-City-CSV.zip",
    "GeoLite2-Country.tar.gz": "https://ivre.rocks/data/geolite/GeoLite2-Country.tar.gz",
    "GeoLite2-Country-CSV.zip": "https://ivre.rocks/data/geolite/GeoLite2-Country-CSV.zip",
    "GeoLite2-ASN.tar.gz": "https://ivre.rocks/data/geolite/GeoLite2-ASN.tar.gz",
    "GeoLite2-ASN-CSV.zip": "https://ivre.rocks/data/geolite/GeoLite2-ASN-CSV.zip",
    # For other files, None has a special meaning "do not
    # download". The following file can be computed based the
    # GeoLite2-* files using `ivre ipdata --import-all`. You should do
    # that if you get your files from Maxmind.
    "GeoLite2-dumps.tar.gz": "https://ivre.rocks/data/geolite/GeoLite2-dumps.tar.gz",
    # This one is not from maxmind -- see https://thyme.apnic.net/
    "BGP.raw": "https://thyme.apnic.net/current/data-raw-table",
}
MAXMIND_LICENSE_KEY = None

GeoIP uses a locale to report country, region and city names. The locale to use is set in GEOIP_LANG and defaults to "en".

Web server

Paths

Two variables (WEB_STATIC_PATH and WEB_DOKU_PATH) are used for the Web application; see Paths and commands.

Modules

The Web UI exposes a fixed set of data sections — view, active, passive, dns, rir, flow — each backed by one (or, for dns, two) DB_<purpose> backends. By default every section whose backend is configured is exposed; the React UI hides the others from the section nav and the matching /cgi/ routes return 404.

Operators who want to narrow the exposed set further (for example, hide flow even though DB_FLOW is wired) can set WEB_MODULES to an iterable of module names:

# Expose only the View and RIR sections; the others are
# hidden from the nav and their routes return 404 even if
# their backends are configured.
WEB_MODULES = ["view", "rir"]

Leaving WEB_MODULES at its default (None) is equivalent to listing every known module — the effective set is then exactly the modules whose DB_<purpose> backend is configured. The dns module is the only cross-backend one: it is exposed as soon as either DB_NMAP or DB_PASSIVE is configured.

Account / admin pages (the Admin page and the self-service API keys page) are not part of WEB_MODULES; they keep their own toggles (WEB_AUTH_ENABLED plus the is_admin flag on the user account).

Notepad

If Dokuwiki (or another web application for notes) is used, the variable WEB_NOTES_BASE should be set to the URL path to access the notes (#IP# will be replaced with the IP address). This variable defaults to /dokuwiki/#IP#.

If you use Dokuwiki, you also want to set:

WEB_GET_NOTEPAD_PAGES = "localdokuwiki"

Or:

WEB_GET_NOTEPAD_PAGES = ("localdokuwiki", ("/path/to/dokuwiki/data/pages",))

The second option is needed if the path to Dokuwiki pages is different from the default "/var/lib/dokuwiki/data/pages".

If you use Mediawiki, you need to set

WEB_GET_NOTEPAD_PAGES = ("mediawiki", ("server", "username", "password",
                                       "dbname", "base"))

Anti-CSRF

As an anti-CSRF option, IVRE will check the Referer: header of the requests to any dynamic URLs (under /cgi/). Normally (when ivre httpd is used or when the WSGI application is exposed directly, IVRE will figure out the allowed referrer URLs alone; under certain circumstances however (e.g., when a reverse-proxy is used, or when the IVRE dynamic URLs are used by another Web application), this is not possible. In this case, the variable WEB_ALLOWED_REFERERS should be set to a list or URLs that are allowed to trigger Web accesses to the IVRE application; for example:

WEB_ALLOWED_REFERERS = [
    'http://reverse-proxy.local/ivre',
    'http://reverse-proxy.local/ivre/',
    'http://reverse-proxy.local/ivre/index.html',
    'http://reverse-proxy.local/ivre/report.html',
    'http://reverse-proxy.local/ivre/upload.html',
    'http://reverse-proxy.local/ivre/compare.html',
    'http://reverse-proxy.local/ivre/flow.html'
]

Authentication and ACLs

If you want to use an authentication in IVRE, you have two options:

configure your Web server (e.g., Apache or Nginx) to handle the authentication and set the environment variable REMOTE_USER to the username.
use one (or more) OAuth providers for authentication, Magic Links (via email) and API keys. See Web Authentication for the complete authentication documentation.

If you want to do some authorization based on the authentication, you can do so by setting a couple of variables; by default, ACL is disabled, and everyone (that can access the /cgi/ URLs) can access to all the results:

WEB_DEFAULT_INIT_QUERY = None
WEB_INIT_QUERIES = {}

In the following, we call and “access filter” either the special value None which means “unrestricted”, or a string describing a filter to apply before performing any query. The strings can be:

“full”: unrestricted.
“noaccess”: no result will be returned to the user.
“category:[category name]”: the user will only have access to results within [category name] category.
“source:[source name]”: the user will only have access to results within [source name] source.

WEB_DEFAULT_INIT_QUERY should be set to an “access filter” that will apply when the current user does not match any user in WEB_INIT_QUERIES.

Here is a simple example, where user admin has full access, user admin-site-a has access to all results in category site-a, and user admin-scanner-a has access to all results with source scanner-a:

WEB_DEFAULT_INIT_QUERY = 'noaccess'
WEB_INIT_QUERIES = {
    'admin': 'full',
    'admin-site-a': 'category:site-a',
    'admin-scanner-a': 'source:scanner-a',
}

If you user Kerberos authentication (or if you have @ in your usernames that provide some kind of “realms”, you can use them; in the following example, any user in the admin.sitea realm has access to all results in category site-a:

WEB_DEFAULT_INIT_QUERY = 'noaccess'
WEB_INIT_QUERIES = {
    '@admin.sitea': 'category:site-a',
}

Misc

IVRE handles DNS blacklist (as defined in the RFC 5782) answers, for domains listed in the set DNS_BLACKLIST_DOMAINS. By default, it is defined as:

# Domains used for DNS blacklists (RFC 5782)
DNS_BLACKLIST_DOMAINS = set(
    [
        "blacklist.woody.ch",
        "zen.spamhaus.org",
    ]
)

To add a domain, just add in your configuration file:

DNS_BLACKLIST_DOMAIN.add("dnsbl.example.com")

Or, to add several entries at once:

DNS_BLACKLIST_DOMAIN.update([
    "dnsbl1.example.com",
    "dnsbl2.example.com",
])