If you like BoxMatrix then please contribute Supportdata, Supportdata2, Firmware and/or Hardware (get in touch).
My metamonk@yahoo.com is not reachable by me since september. Please use hippie2000@webnmail.de instead.

Supportdata2

From BoxMatrix

BoxMatrix >> Development >> Supportdata2 @ BoxMatrix   -   IRC-Chat   -   Translate: de es fr it nl pl
News Selectors Models Accessories Components Environment Config Commands System Webif Software Develop Lexicon Community Project Gallery

Tarballs FW-Probes FW-History FW-News Supportdata Hardware SVN Trac Freetz-News Freetz-Timeline Freetz-Releases Freetz-Mirror OpenWRT Research

Supportdata2[edit]

Introduction[edit]

supportdata is intended for the AVM support and already is a valuable source of information.
The biggest advantage of supportdata is that it can be created by everyone on unmodified firmware.
And that is very important to have correct scan results about plain FRITZ!OS for this Wiki.

But BoxMatrix is hungry for more than what is extractable from Supportdata-Probes.
Stuff which is of no interest to the AVM support but will help to fill some empty sections:

Created:

TODO:

Usage[edit]

The script sd2 is provided collecting all the wanted additional info.
It could be run in a telnet or ssh session on original or modified firmware with access to the internet.
If a box is offline you can put sd2 on an USB-Stick. Maybe you need to copy it to /var/tmp to execute it.

Since the wget command of busybox (no HTTPS) is present on every box sd2 is hosted accessible by HTTP.
The usage is simple for the helper, one single command line, and you can always peek the source code:

cd /var/tmp       # only on original firmware - freetz cd is /var/mod/root

rm -f sd2 && wget boxmatrix.info/sd2 && chmod +x sd2 && ./sd2

Add -t <server> to TFTP, -m to Mail, -l to List and / or -d to Delete the created file at the end of the bold line above.
Alternatively send the created file as an Email attachment to hippie2000@webnmail.de.

Output[edit]

The filename of the sd2 generated output identifies a model, its firmware and its branding, easy to parse by my scanner.

#!/bin/sh

# sd2 - (C) 2022 Ralf Steines aka Hippie2000 - <hippie2000@webnmail.de> - PROPRIETARY!
# Collects information for boxmatrix.info which is not present in supportdata.
# Docs and latest version can be found at https://boxmatrix.info/wiki/Supportdata2

version="05"
hippie="hippie2000@webnmail.de"
dstamp=$(date +"%s") 2>/dev/null

while getopts slnt:mdT flag ; do
    case "${flag}" in
        s) stdout="1";;
        l) list="1";;
        n) nogzip="1";;
        t) tftpserver=${OPTARG};;
        m) mail="1";;
        d) delete="1";;
        T) test="1";;
    esac
done

product="${CONFIG_PRODUKT_NAME}"
[ -x /bin/sed ] && product=$(echo -n "${CONFIG_PRODUKT_NAME}" | sed "s/ /_/g")

subver="${CONFIG_SUBVERSION}"
[ "${subver}" != "" ] || subver="-0"

file="sd2-${version}-${product}-${HWRevision}-${HWSubRevision}-"
file="${file}${CONFIG_VERSION_MAJOR}-${CONFIG_VERSION}${subver}-"
file="${file}${OEM}-${ANNEX}-${Country}-${Language}-${dstamp}.txt"

if [ "${stdout}" != "" ] ; then
    exec 2> /dev/null
else
    exec > "${file}"
fi

>&2 echo "Creating BocMatrix Supportdata2"

echo "##### BEGIN SD2 ${version} '${file}'"

Normally sd2 will log stdout (echo) to filenames like the following, output to stderr (>&2 echo) will be printed:

sd2-02-FRITZ!Box 6490 Cable-213-0-141-07.29-0-avm-Kabel-049-en-1653475087.txt
sd2-02-FRITZ!Box 7520-247-2-175-07.29-0-avm-B-049-de-1653475087.txt

If the -s option was passed (stdout mode) no file is created and stderr is muted.

Listings[edit]

Here step by step listings created by sd2 and how they are gathered will be added.

Modding[edit]

To analyze how the firmware was modified some details about freetz or possible other mods are collected, if present:

if [ -e /etc/freetz_info.cfg ] ; then

    >&2 echo "Listing freetz info"

    echo "##### SECTION freetz_info"
    echo

    cat /etc/freetz_info.cfg
    echo

fi

if [ -e /etc/packages.lst ] ; then

    >&2 echo "Listing freetz packages"

    echo "##### SECTION freetz_packages"
    echo

    cat /etc/packages.lst
    echo

fi

if [ -e /etc/.config ] ; then

    >&2 echo "Listing freetz build config"

    echo "##### SECTION freetz_config"
    echo

    cat /etc/.config        
    echo

fi

if [ -e /var/flash/freetz ] ; then

    >&2 echo "Listing freetz tffs config"

    echo "##### SECTION freetz_tffs"
    echo

    cat /var/flash/freetz | tar tvf - 2>/dev/null
    echo

fi

if [ -d /var/mod ] ; then

    >&2 echo "Listing freetz temp files"

    echo "##### SECTION freetz_temp"
    echo

    ls -lR /var/mod
    echo

fi

Symlinks[edit]

Symlinks are already scanned from Firmware-Probes but it may be helpful for debugging modified firmware to list them.
If rpc exists we are on a Puma6 or Puma7 model and also list the remote symlinks
This section must be grabbed before the BusyBox section to avoid the need of a 2-path sd2 listing parser.

if [ -x /usr/bin/find ] ; then

    >&2 echo "Listing local symlinks"

    echo "##### SECTION symlinks_local"
    echo

    find /bin /sbin /usr/bin /usr/sbin -type l | sort 2>/dev/null
    echo
 
    if [ -x /usr/bin/rpc ] ; then

        >&2 echo "Listing remote symlinks"

        echo "##### SECTION symlinks_remote"
        echo

        rpc find /bin /sbin /usr/bin /usr/sbin -type l | sort 2>/dev/null
        echo
 
    fi

fi

BusyBox[edit]

BusyBox-Commands are already scanned from Firmware-Probes but it may be helpful for debugging modified fw to list them.
Since the --list option is not present in older busybox we have to call busybox without arguments for compatibility.
If rpc exists we are on a Puma6 or Puma7 model and also list the remote busybox.

>&2 echo "Listing local busybox applets"

echo "##### SECTION busybox_local"
echo

busybox 2>/dev/null
echo

if [ -x /usr/bin/rpc ] ; then

    >&2 echo "Listing remote busybox applets"

    echo "##### SECTION busybox_remote"
    echo

    rpc busybox 2>/dev/null
    echo

fi

Processes[edit]

Linux-Processes are already present in supportdata but to map them to Netlink data they need to be grabbed at the same time.
If rpc exists we are on a Puma6 or Puma7 model and also list the remote processes.
This section must be grabbed before the Netlink section to avoid the need of a 2-path sd2 listing parser.

>&2 echo "Listing local processes"

echo "##### SECTION processes_local"
echo

ps -l
echo

if [ -x /usr/bin/rpc ] ; then

    >&2 echo "Listing remote processes"

    echo "##### SECTION processes_remote"
    echo

    rpc ps -l
    echo

fi

The -T option was removed since older ps fails on it.

Sockets[edit]

Linux-Sockets are already present in supportdata but to map them to AVMIPC data they need to be grabbed at the same time.
If rpc exists we are on a Puma6 or Puma7 model and also list the remote sockets.
This section must be grabbed before the AVMIPC section to avoid the need of a 2-path sd2 listing parser.

>&2 echo "Listing local sockets"

echo "##### SECTION sockets_local"
echo

netstat -xlpen
echo

netstat -xpen
echo

if [ -x /usr/bin/rpc ] ; then

    >&2 echo "Listing remote sockets"

    echo "##### SECTION sockets_remote"
    echo

    rpc netstat -xlpen
    echo

    rpc netstat -xpen
    echo

fi

Netlink[edit]

Netlink-Sockets are similar to Linux-Sockets just they are not files in the filesystem so they don't appear in supportdata.
We list them from the procfs. If rpc exists we are on a Puma6 or Puma7 model and also list the remote netlink sockets.

>&2 echo "Listing local netlink sockets"

echo "##### SECTION netlink_local"
echo

cat /proc/net/netlink
echo

if [ -x /usr/bin/rpc ] ; then

    >&2 echo "Listing remote netlink sockets"

    echo "##### SECTION netlink_remote"
    echo

    rpc cat /proc/net/netlink
    echo

fi

Genetlink[edit]

Genetlink sockets are a special family of Netlink-Sockets which are maintained by libewnwlinux.so in FRITZ!OS.
They could be listed from aicmd listeners which call the genetlink_ functions in libewnwlinux.so.
So far these are only l2tpv3d, nltest and active users of libavmpcp.so and no remote daemon so far.

if [ -x /bin/aicmd ] ; then

    >&2 echo "Listing local genetlink sockets"

    echo "##### SECTION genetlink_local"
    echo

    aicmd l2tpv3d ewnwlinux show genetlink 2>/dev/null
    echo

fi

AVMIPC[edit]

The AVMIPC-Datastore is pretty new, the aicmd command to show it even newer (6.54), we must check if it exists.
If rpc exists we are on a Puma6 or Puma7 model and also list the remote datastore.
Older avmipcd has a differing syntax to list the datastore, and aicmd does not set an error rerturn code on failure.
So we run new and old syntax silently after each others, one will (hopefully) succeed. Tested with fw 7.01 - 7.39.
In fw 6.88 aicmd exists but could not list the datastore since avmipcd was no aicmd listener yet. This also fails silently.

if [ -x /bin/aicmd ] ; then

    >&2 echo "Listing local avmipcd datastore"

    echo "##### SECTION avmipcd_local"
    echo

    aicmd avmipcd datastore show full 2>/dev/null
    aicmd avmipcd show data_store full 2>/dev/null
    echo
 
    if [ -x /usr/bin/rpc ] ; then

        >&2 echo "Listing remote avmipcd datastore"

        echo "##### SECTION avmipcd_remote"
        echo

        rpc aicmd avmipcd datastore show full 2>/dev/null
        rpc aicmd avmipcd show data_store full 2>/dev/null
        echo
 
    fi

fi

Devices[edit]

The Linux-Devices currently only list devices which are hardcoded in the firmware, which is not used in recent firmware.
It always lacked the numerous devices which are dynamically created at runtime (today all). This listing will fill these gaps.
If rpc exists we are on a Puma6 or Puma7 model and also list the remote devices.

>&2 echo "Listing local devices"

echo "##### SECTION devices_local"
echo

ls -lR /dev
echo

if [ -x /usr/bin/rpc ] ; then

    >&2 echo "Listing remote devices"

    echo "##### SECTION devices_remote"
    echo

    rpc ls -lR /dev
    echo

fi

Procfs[edit]

The procfs is huge, especially its numerical directories with the pid of each process.
Besides this the processes come and go so a ls -lR would just be bloat for redundant useeless info.
Thats why it has to be listed step by step. First we list all non directories in /proc main directory :

>&2 echo "Listing procfs"

echo "##### SECTION procfs"
echo

echo "/proc:"
ls -l /proc | grep -v '^d'

To have a sample we list the pid 1 process which should not disappear

echo
ls -lR /proc/1 2>/dev/null

Then we list al non numerical subdirectories of /proc recursively

for obj in /proc/*
do

    [ -d "$obj" ] || continue

    dir=$(basename "$obj")
    [ "$dir" -eq "$dir" ] 2> /dev/null
    [ $? -ne 0 ] || continue 

    echo
    ls -lR "$obj"

done
echo

The first continue filters out all non directories, the second continue filters out all numerical dirs.
If rpc exists we are on a Puma6 or Puma7 model and also list the remote procfs.

if [ -x /usr/bin/rpc ] ; then

    >&2 echo "Listing remote procfs"

    echo "##### SECTION procfs_remote"
    echo

    echo "/proc:"
    rpc ls -l /proc | grep -v '^d'
    echo
    rpc ls -l /proc | grep '^d'
    echo

    rpc ls -lR /proc/1 2>/dev/null
    echo

    dirs="P-UNIT avalanche avm bus capi cppi cpu docint driver fs irq sys sysvipc tty"
    for dir in  ${dirs} ; do

        rpc ls -lR "/proc/${dir}"
        echo

    done

fi

Sysfs[edit]

The sysfs is also huge, but unlike the procfs it does not need to be filtered but can be listed recursively

>&2 echo "Listing local sysfs"

echo "##### SECTION sysfs_local"
echo

ls -lR /sys
echo

If rpc exists we are on a Puma6 or Puma7 model and also list the remote sysfs.

if [ -x /usr/bin/rpc ] ; then

    >&2 echo "Listing remote sysfs"

    echo "##### SECTION sysfs_remote"
    echo

    rpc ls -lR /sys
    echo

fi

EOF[edit]

The end marker is important to detect truncated files:

echo "##### END SD2"

if [ "${stdout}" != "" ] ; then
    exit 0
fi

>&2 echo "Created '${file}'"

If the -s option was passed (stdout mode) sd2 will terminate here since there's no file to process further.

Options[edit]

List[edit]

sd2 can optionally list the created file (using less) if it's passed the -l option.
This is especially useful during development to not have to copy paste the unpredictable filename.

if [ "${list}" != "" ] ; then

    >&2  echo "Listing created file..."

    1>&2 less "${file}"

fi

Gzip[edit]

sd2 will automatically compress the created file if gzip could be found, unless the -n option (no gzip) is passed.
A sd2 file is approx. 1 MB, with remote listings 1.5MB (6490). Compressed these are both 72 kB!

if [ "${nogzip}" == "" ] ; then

    if [ -x /bin/gzip ] ; then

        >&2  echo "Compressing file with gzip..."

        gzip "${file}"

        if [ $? -eq 0 ] ; then
            >&2 echo "Successfully gzipped file" 
            file="${file}.gz"
        else
            >&2 echo "*** FAILED to gzip file ***" 
        fi

    fi

fi

TFTP[edit]

sd2 can optionally send the created file to a tftp server if it is passed the -t <server> option.
Every box has the tftp command installed, a slow and old protocol but the only way to upload a file from original firmware.
This is especially useful to grab Supportdata2 from models without USB-Host like repeaters. sd2 supports tftp upload.

if [ "${tftpserver}" != "" ] ; then

    >&2  echo "Sending file to tftp server '${tftpserver}'..."

    tftp -p -l "${file}" -r "${file}" "${tftpserver}"

    if [ $? -eq 0 ] ; then
        >&2 echo "Successfully uploaded file" 
    else
        >&2 echo "*** FAILED to tftp file ***" 
    fi

fi

A tftpd server must be running on the passed <server>. This fails with the tftp server of dnsmasq since it is read only.
You need to adapt your freetz build to enable the Busybox applets >> Networking Utilities >> tftpd applet.
And finaly you just need to put this line in Inetd >> User defined in the freetz webinterface:

69 dgram udp nowait root /bin/busybox tftpd -l -c /var/media/ftp/tftp

Make sure dnsmarq tftp server is swiched off and the target directory exists. For me this is the Internal-Storage of my 6490.

Mail[edit]

sd2 can optionally email the created file to me if it is passed the -m option.
This only works if your box knows the email credentials you entered for using Push-Mail.

if [ "${mail}" != "" ] ; then

    >&2  echo "Mailing file to hippie2000..."

    mailer -t "${hippie}" -s "${file}" -d "${file}"

    if [ $? -eq 0 ] ; then
        >&2 echo "Successfully mailed file" 
    else
        >&2 echo "*** FAILED to mail file ***" 
    fi

fi

Delete[edit]

sd2 can optionally delete the created file after uploading / mailing / listing if it's passed the -d option.
This prevents from filling the RAM during development.

if [ "${delete}" != "" ] ; then

    >&2  echo "Deleting created file..."

    rm -f "${file}" 2>/dev/null

fi

Project[edit]

Feedback[edit]

For bugreports or feature requests please get in touch, use our IRC-Channels or use this forum thread:

History[edit]

Read the changes bottom up.

sd2 xx - future

TODO: List /var/tmp recursively

sd2-xx - work in progress

sd2-07 - 2022-06-26 - milestone 07 - processes, netlink, genetlink

Added: Listings of local and remote genetlink sockets (genetlink_local, genetlink_remote)
Added: Listings of local and remote netlink sockets (netlink_local, netlink_remote)
Added: Listings of local and remote processes (processes_local, processes_remote)

sd2-06 - 2022-06-13 - milestone 06 - symlinks, busybox, old datastore listing, future proof license

Changed the license form to PROPRIETARY so this project does not end in code raping like FIT-Image.
Added: Listings of local and remote symlinks (symlinks_local, symlinks_remote)
Added: Listings of local and remote busybox applets (busybox_local, busybox_remote)
Added: Support for older aicmd avmipcd datastore syntax (tested 7.01)

sd2-05 - 2022-06-01 - milestone 05 - sockets, no whitespace, embedded filename, stdout, nogzip

Added: Listings of local and remote sockets (sockets_local, sockets_remote)
Added: The -s option now offers a stdout mode for piping the output, ie to gzip
       Useful for models with very small RAM to stream commpress 'sd2 -s | gzip > sd2.gz'
Added: The BEGIN header now contains the filename so sd2 now also works with altered filenames
Added: The name of the created file now replaces white space with "_" if sed is present
       Required on older fw (tested: 4.72) otherwise mailer fails to send the attachement

sd2-04 - 2022-05-28 - milestone 04 - remote dev, procfs, sysfs, mail, ippf - recommended minimum

Added: Commpressing created file with gzip
Added: Listing remote sysfs (sysfs_remote)
Added: Listing remote procfs (procfs_remote)
Added: Listing remote devices (devices_remote)
Added: The -m option now can mail the created file if the box knows your credentials
Added: An IPPF thread introducing the project - see Feedback above

sd2-03 - 2022-05-27 - milestone 03 - copyright, arguments, devices, list, delete

Added: The -d option now can delete the file after sending / listing it
Added: The -l option now can list the created file (using  less)
Added: Procfs recursive listing of pid 1 process
Added: Devices listing (devices)
Changed: The TFTP option is now enabled by -t <server>
Added: Commandline argument parser to be more flexible in future
Added: (C) header, link to this page and license info.

sd2-02 - 2022-05-25 - milestone 02 - ipc remote, freetz info, tftp - minimum for the scanner

Added: The filename now contains the sd2 version, ie sd2-02-, pre 02 files are void
Added: Modding listing (freetz_config, freetz_tffs, freetz_temp)
Added: TFTP upload
Added: AVMIPC remote listing (avmipcd_remote)
Added: Modding listing (freetz_info, freetz_packages)

sd2-01 - 2022-05-24 - milestone 01 - ipc local, sysfs, procfs

Added: AVMIPC listing (avmipcd_local)
Added: Procfs, Sysfs listings (procfs, sysfs)

There are no releases, sd2 is always live development. The versions are milestones for my scanner.
Pre 0.2 data will not be used, since it lacks modding info which may interfer the remaining results