Configuration

IVRE has several configuration variables. The default values are hard-coded in ivre/config.py. You should not change this file, unless you are modifying IVRE and you want to change the default configuration. You do not need to do this if you want to install IVRE with a non-default configuration, you just need to distribute a proper configuration file.

IVRE can be configured using different configuration files:

system-wide: ivre.conf in the following directories: /etc/, /etc/ivre, /usr/local/etc, /usr/local/etc/ivre.
user-specific: ~/.ivre.conf (read after the system-wide configuration files, so higher priority).
execution-specific: another configuration file can be specified using the $IVRE_CONF environment variable (read after the user-specific file, so highest priority).

The configuration files are Python files setting global variables.

Debug

Debug messages are turned off by default, since IVRE has no bugs. DEBUG_DB turns on database-specific debug messages, and can be very noisy. Setting DEBUG to True is mandatory to run IVRE’s tests.

Databases

Databases are specified using URLs:

db_type://[username[:password]@][host[:port]]/databasename?options

DB is the generic database URL (will be used for all Purposes unless a purpose-specific URL has been specified). The value "mongodb:///ivre" is the default and means “use MongoDB on localhost, database ivre, default collection names”.

Purpose-specific URLs can be specified using DB_<purpose>; DB_DATA is specific and defaults to None, which has the special meaning "maxmind:///<ivre_share_path>/geoip".

Here are some examples:

DB_PASSIVE = "sqlite:////tmp/ivre.db"
DB_NMAP = "postgresql://ivre@localhost/ivre"
DB_VIEW = "elastic://192.168.0.1:9200/ivre"
DB_DATA = "maxmind:///share/data/ivre/geoip"

Batch insert or upsert operations can be tuned using backend-specific variables:

LOCAL_BATCH_SIZE = 10000  # used with --local-bulk
MONGODB_BATCH_SIZE = 100
POSTGRES_BATCH_SIZE = 10000

Paths and commands

All variables ending with _PATH (except AGENT_MASTER_PATH and NMAP_SHARE_PATH) default to None, a special value which means “try to guess the path based on IVRE installation”.

Here are the values with examples on a regular installation:

DATA_PATH = None                  # /usr/share/ivre/data
GEOIP_PATH = None                 # /usr/share/ivre/geoip
HONEYD_IVRE_SCRIPTS_PATH = None   # /usr/share/ivre/data/honeyd
WEB_STATIC_PATH = None            # /usr/share/ivre/web/static
WEB_DOKU_PATH = None              # /usr/share/ivre/dokuwiki

AGENT_MASTER_PATH defaults to "/var/lib/ivre/master".

NMAP_SHARE_PATH defaults to None, which means IVRE will try "/usr/local/share/nmap", "/opt/nmap/share/nmap", then "/usr/share/nmap".

IVRE may need some executables:

TESSERACT_CMD = "tesseract"
OPENSSL_CMD = "openssl"

Nmap scan templates

Nmap scan templates are defined in the NMAP_SCAN_TEMPLATES variable. Usually, this variable should not be overridden, but rather modified.

By default, NMAP_SCAN_TEMPLATES contains one template, named "default", which is defined as follows:

NMAP_SCAN_TEMPLATES: dict[str, NmapScanTemplate] = {
    "default": {
        # Commented values are default values and to not need to be
        # specified:
        # "nmap": "nmap",
        # "pings": "SE",
        # "scans": "SV",
        # "osdetect": True,
        # "traceroute": True,
        # "resolve": 1,
        # "verbosity": 2,
        # "ports": None,
        # "top_ports": None,
        "host_timeout": "15m",  # default value: None
        "script_timeout": "2m",  # default value: None
        "scripts_categories": ["default", "discovery", "auth"],  # default value: None
        "scripts_exclude": [
            "broadcast",
            "brute",
            "dos",
            "exploit",
            "external",
            "fuzzer",
            "intrusive",
        ],  # default value: None
        # "scripts_force": None,
        # "extra_options": None,
    }
}

To create another template, the easiest is to copy, either using .copy() or using the dict() constructor, the "default" template; the following configuration entry creates an "aggressive" template that will run more scripts (including potentially dangerous ones) and have more permissive timeout values:

NMAP_SCAN_TEMPLATES["aggressive"] = dict(
    NMAP_SCAN_TEMPLATES["default"],
    host_timeout="30m",
    script_timeout="5m",
    scripts_categories=['default', 'discovery', 'auth', 'brute',
                        'exploit', 'intrusive'],
    scripts_exclude=['broadcast', 'external'],
)

It is possible to check the options a template will use by running the following command (the output has been modified, the command line is normally on one single line):

$ ivre runscans --output CommandLine
Command line to run a scan with template default
    nmap -A -PS -PE -sS -vv --host-timeout 15m --script-timeout 2m
         --script '(default or discovery or auth) and not (broadcast
         or brute or dos or exploit or external or fuzzer or intrusive)'

$ ivre runscans --output CommandLine --nmap-template aggressive
Command line to run a scan with template aggressive
    nmap -A -PS -PE -sS -vv --host-timeout 30m --script-timeout 5m
         --script '(default or discovery or auth or brute or exploit or
         intrusive) and not (broadcast or external)'

Masscan probes

IVRE can use the service fingerprint database from Nmap to find service and product names from Masscan results. For that, IVRE needs to know which probe (or “hello string”) has been used. This depends on Masscan source code (compile-time) and options (run-time). You can adjust what IVRE will use per port (from the configuration) or globally (from the command-line option).

The default configuration value is based on the Masscan fork of the IVRE project.

# Based on IVRE's fork source code --- you may want to adapt these
# settings if you use another version of Masscan.
MASSCAN_PROBES = {
    "tcp": {
        53: "DNSVersionBindReqTCP",
        88: "Kerberos",
        104: "dicom",
        111: "RPCCheck",
        130: "NotesRPC",
        135: "DNSVersionBindReqTCP",
        256: "LDAPSearchReq",
        257: "LDAPSearchReq",
        389: "LDAPSearchReq",
        390: "LDAPSearchReq",
        406: "SIPOptions",
        427: "NotesRPC",
        548: "afp",
        554: "RTSPRequest",
        1098: "JavaRMI",
        1099: "JavaRMI",
        1352: "NotesRPC",
        1433: "ms-sql-s",
        1702: "LDAPSearchReq",
        1972: "NotesRPC",
        2049: "RPCCheck",
        2345: "dicom",
        2375: "docker",
        2379: "docker",
        2380: "docker",
        2761: "dicom",
        2762: "dicom",
        3268: "LDAPSearchReq",
        3892: "LDAPSearchReq",
        4242: "dicom",
        5060: "SIPOptions",
        6000: "X11Probe",
        6001: "X11Probe",
        6002: "X11Probe",
        6003: "X11Probe",
        6004: "X11Probe",
        6005: "X11Probe",
        6006: "X11Probe",
        6007: "X11Probe",
        6008: "X11Probe",
        6009: "X11Probe",
        6010: "X11Probe",
        6011: "X11Probe",
        6012: "X11Probe",
        6013: "X11Probe",
        6014: "X11Probe",
        6015: "X11Probe",
        6016: "X11Probe",
        6017: "X11Probe",
        6018: "X11Probe",
        6019: "X11Probe",
        6379: "redis-server",
        7171: "NotesRPC",
        8081: "SIPOptions",
        8554: "RTSPRequest",
        8728: "NotesRPC",
        9001: "mongodb",
        11112: "dicom",
        11711: "LDAPSearchReq",
        22001: "NotesRPC",
        27017: "mongodb",
        31337: "SIPOptions",
        49153: "mongodb",
        50000: "DNSVersionBindReqTCP",
        50001: "DNSVersionBindReqTCP",
        50002: "DNSVersionBindReqTCP",
    },
}

The `flow` purpose

The flow purpose has several specific configuration options, which may have important impacts on performances; here are the options and their default values:

# Dictionary that helps determine server ports of communications. Each entry
# is {proto: {port: proba}}. The when two ports are known, the port with the
# highest probability is used.
# When /usr/share/nmap/nmap-services is available, these probas are taken,
# otherwise /etc/services is used with proba=0.5 for each entry.
# KNOWN_PORTS entries have the highest priority.
# Example:
#  KNOWN_PORTS = {
#      "udp": {
#          9999: 1.0,
#          12345: 0.5,
#      },
#      "tcp": {
#          20202: 0.8,
#      },
#  }
KNOWN_PORTS: dict[str, dict[int, float]] = {}
# Enable the recording of appearance times for flows. Will slow down a
# bit the insertion rate
FLOW_TIME = True
# Precision (in seconds) to use when recording times when flows appear
FLOW_TIME_PRECISION = 3600
# When recording flow times, record the whole range from start_time to end_time
# This option is experimental and possibly useless in practice
FLOW_TIME_FULL_RANGE = True
# When recording flow times, represents the beginning of the first timeslot
# as a Unix timestamp shifted to local time.
# 0 means that the first timeslot starts at 1970-01-01 00:00 (Local time).
FLOW_TIME_BASE = 0
# Store high level protocols metadata in flows. It may take much more space.
FLOW_STORE_METADATA = True

The `data` purpose

The URLs used to get IP address databases are set in the dictionary IPDATA_URLS:

IPDATA_URLS = {
    # None has a special meaning:
    # https://download.maxmind.com/app/geoip_download?edition_id=XXX&suffix=XXX&license_key=XXX
    #
    # You can use this value for the GeoLite2-* files (and set
    # MAXMIND_LICENSE_KEY below) to download files from MaxMind
    # instead of ivre.rocks directly. Maxmind license keys are free
    # and can be obtained from <https://www.maxmind.com/>
    "GeoLite2-City.tar.gz": "https://ivre.rocks/data/geolite/GeoLite2-City.tar.gz",
    "GeoLite2-City-CSV.zip": "https://ivre.rocks/data/geolite/GeoLite2-City-CSV.zip",
    "GeoLite2-Country.tar.gz": "https://ivre.rocks/data/geolite/GeoLite2-Country.tar.gz",
    "GeoLite2-Country-CSV.zip": "https://ivre.rocks/data/geolite/GeoLite2-Country-CSV.zip",
    "GeoLite2-ASN.tar.gz": "https://ivre.rocks/data/geolite/GeoLite2-ASN.tar.gz",
    "GeoLite2-ASN-CSV.zip": "https://ivre.rocks/data/geolite/GeoLite2-ASN-CSV.zip",
    # For other files, None has a special meaning "do not
    # download". The following file can be computed based the
    # GeoLite2-* files using `ivre ipdata --import-all`. You should do
    # that if you get your files from Maxmind.
    "GeoLite2-dumps.tar.gz": "https://ivre.rocks/data/geolite/GeoLite2-dumps.tar.gz",
    "iso3166.csv": "https://dev.maxmind.com/static/csv/codes/iso3166.csv",
    # This one is not from maxmind -- see https://thyme.apnic.net/
    "BGP.raw": "https://thyme.apnic.net/current/data-raw-table",
}
MAXMIND_LICENSE_KEY = None

GeoIP uses a locale to report country, region and city names. The locale to use is set in GEOIP_LANG and defaults to "en".

Web server

Paths

Two variables (WEB_STATIC_PATH and WEB_DOKU_PATH) are used for the Web application; see Paths and commands.

Notepad

If Dokuwiki (or another web application for notes) is used, the variable WEB_NOTES_BASE should be set to the URL path to access the notes (#IP# will be replaced with the IP address). This variable defaults to /dokuwiki/#IP#.

If you use Dokuwiki, you also want to set:

WEB_GET_NOTEPAD_PAGES = "localdokuwiki"

Or:

WEB_GET_NOTEPAD_PAGES = ("localdokuwiki", ("/path/to/dokuwiki/data/pages",))

The second option is needed if the path to Dokuwiki pages is different from the default "/var/lib/dokuwiki/data/pages".

If you use Mediawiki, you need to set

WEB_GET_NOTEPAD_PAGES = ("mediawiki", ("server", "username", "password",
                                       "dbname", "base"))

Anti-CSRF

As an anti-CSRF option, IVRE will check the Referer: header of the requests to any dynamic URLs (under /cgi/). Normally (when ivre httpd is used or when the WSGI application is exposed directly, IVRE will figure out the allowed referrer URLs alone; under certain circumstances however (e.g., when a reverse-proxy is used, or when the IVRE dynamic URLs are used by another Web application), this is not possible. In this case, the variable WEB_ALLOWED_REFERERS should be set to a list or URLs that are allowed to trigger Web accesses to the IVRE application; for example:

WEB_ALLOWED_REFERERS = [
    'http://reverse-proxy.local/ivre',
    'http://reverse-proxy.local/ivre/',
    'http://reverse-proxy.local/ivre/index.html',
    'http://reverse-proxy.local/ivre/report.html',
    'http://reverse-proxy.local/ivre/upload.html',
    'http://reverse-proxy.local/ivre/compare.html',
    'http://reverse-proxy.local/ivre/flow.html'
]

Authentication and ACLs

If you want to use an authentication in IVRE, you have to configure your Web server (e.g., Apache or Nginx) to do so and set the environment variable REMOTE_USER to the username.

If you want to do some authorization based on the authentication, you can do so by setting a couple of variables; by default, ACL is disabled, and everyone (that can access the /cgi/ URLs) can access to all the results:

WEB_DEFAULT_INIT_QUERY = None
WEB_INIT_QUERIES = {}

In the following, we call and “access filter” either the special value None which means “unrestricted”, or a string describing a filter to apply before performing any query. The strings can be:

“full”: unrestricted.
“noaccess”: no result will be returned to the user.
“category:[category name]”: the user will only have access to results within [category name] category.
“source:[source name]”: the user will only have access to results within [source name] source.

WEB_DEFAULT_INIT_QUERY should be set to an “access filter” that will apply when the current user does not match any user in WEB_INIT_QUERIES.

Here is a simple example, where user admin has full access, user admin-site-a has access to all results in category site-a, and user admin-scanner-a has access to all results with source scanner-a:

WEB_DEFAULT_INIT_QUERY = 'noaccess'
WEB_INIT_QUERIES = {
    'admin': 'full',
    'admin-site-a': 'category:site-a',
    'admin-scanner-a': 'source:scanner-a',
}

If you user Kerberos authentication (or if you have @ in your usernames that provide some kind of “realms”, you can use them; in the following example, any user in the admin.sitea realm has access to all results in category site-a:

WEB_DEFAULT_INIT_QUERY = 'noaccess'
WEB_INIT_QUERIES = {
    '@admin.sitea': 'category:site-a',
}

Misc

IVRE handles DNS blacklist (as defined in the RFC 5782) answers, for domains listed in the set DNS_BLACKLIST_DOMAINS. By default, it is defined as:

# Domains used for DNS blacklists (RFC 5782)
DNS_BLACKLIST_DOMAINS = set(
    [
        "blacklist.woody.ch",
        "zen.spamhaus.org",
    ]
)

To add a domain, just add in your configuration file:

DNS_BLACKLIST_DOMAIN.add("dnsbl.example.com")

Or, to add several entries at once:

DNS_BLACKLIST_DOMAIN.update([
    "dnsbl1.example.com",
    "dnsbl2.example.com",
])