nc-apps: Provide a RSS feed for new apps

[Sanookmakmak]

Call nc-apps once a day, for example, and deliver the result as a RSS feed.

[Sanookmakmak]

Here is an attempt to solve it in Perl:

#!/usr/bin/env perl
# Required Debian packages:
# sudo apt install -y libxml-rss-perl libjson-xs-perl libhttp-server-simple-perl libdatetime-perl libfile-temp-perl

use strict;
use warnings;
use utf8;

use JSON;
use DateTime;
use XML::RSS;
use HTTP::Server::Simple::CGI;
use File::Temp qw(tempfile);

# --- Configurable ---
my $host        = 'server.it.lan';                   # Server host (can be localhost or public IP)
my $port        = 8090;                              # Port to serve the RSS
my $base_url    = "http://$host:$port";              # Full base URL
my $days        = 365;                               # Lookback period in days
my $title       = 'Nextcloud Apps Feed';
my $description = 'RSS feed of Nextcloud apps published in the last 365 days';

main();

sub main {
    my $since   = get_days_ago_iso($days);
    my $raw     = fetch_app_data_after($since);
    my $apps    = parse_json($raw);
    my $rss     = build_rss($apps);
    serve_rss($rss->as_string);
}

sub get_days_ago_iso {
    my ($days) = @_;
    return DateTime->now->subtract(days => $days)->iso8601() . 'Z';
}

sub fetch_app_data_after {
    my ($date) = @_;
    my ($fh, $file) = tempfile();

    my $cmd = qq{TERM=dumb nc-apps appstore 'map(select(.created > "$date"))' > "$file" 2>/dev/null};
    system($cmd);

    open my $in, '<', $file or die "Failed to read tmpfile: $!";
    local $/;
    my $out = <$in>;
    close $in;

    $out =~ s/\e\[?.*?[a-zA-Z]//g;  # Strip ANSI sequences
    return $out;
}

sub parse_json {
    my ($raw) = @_;
    my $json = eval { decode_json($raw) };
    die "Failed to parse JSON: $@" if $@;
    return $json;
}

sub build_rss {
    my ($apps) = @_;
    my $rss = XML::RSS->new(version => '2.0');
    $rss->channel(
        title       => $title,
        link        => "$base_url/",
        description => $description,
        language    => 'en',
    );

    foreach my $app (sort { $b->{created} cmp $a->{created} } @$apps) {
        $rss->add_item(
            title       => $app->{name} || $app->{id},
            link        => $app->{website} || $app->{issueTracker} || 'https://apps.nextcloud.com/',
            description => $app->{summary} || 'No description available.',
            pubDate     => $app->{created},
        );
    }

    return $rss;
}

sub serve_rss {
    my ($rss_content) = @_;
    print "Serving RSS feed on $base_url\n";
    My::RSSServer->new($port, $rss_content)->run;
}

# --- Web server class for RSS feed ---
package My::RSSServer;
use base 'HTTP::Server::Simple::CGI';

sub new {
    my ($class, $port, $content) = @_;
    my $self = $class->SUPER::new($port);
    $self->{content} = $content;
    return $self;
}

sub handle_request {
    my ($self, $cgi) = @_;
    print "HTTP/1.0 200 OK\r\n";
    print "Content-Type: application/rss+xml; charset=utf-8\r\n\r\n";
    print $self->{content};
}

sub print_banner {
    # Suppress default message from HTTP::Server::Simple
}

1;

[ernolf]

Here is an attempt to solve it in Perl:

That looks great already!

Here are a few small tweaks:

• the dependency on libjson-perl was missing

- # sudo apt install -y libxml-rss-perl libjson-xs-perl libhttp-server-simple-perl libdatetime-perl libfile-temp-perl
+ # sudo apt install -y libxml-rss-perl libjson-xs-perl libhttp-server-simple-perl libdatetime-perl libfile-temp-perl libjson-perl

• the number of days is hard-coded to 365 in the description, even though it’s configurable:

- my $description = 'RSS feed of Nextcloud apps published in the last 365 days';
+ my $description = "RSS feed of Nextcloud apps published in the last $days days";

• If the description of the apps in the App Store contains German descriptions and the locale of the console is 'de', then the German descriptions would be filtered out but you have hard-coded the language to 'en' in the RSS. You can add an option lang:

# --- Configurable ---
my $host        = 'server.it.lan';                   # Server host (can be localhost or public IP)
my $port        = 8090;                              # Port to serve the RSS
my $base_url    = "http://$host:$port";              # Full base URL
my $days        = 365;                               # Lookback period in days
my $title       = 'Nextcloud Apps Feed';
my $description = "RSS feed of Nextcloud apps published in the last $days days";
+ my $lang        = 'en';

and pass it in the command $cmd.

I’ve added a -q|--quiet option to nc-apps for machine-readable JSON output. It suppresses ANSI codes, the spinner, module-loading messages, and update-check prompts—ideal for use in cron jobs.

After updating to the latest nc-apps, you can simplify fetch_app_data_after to:

- sub fetch_app_data_after {
-     my ($date) = @_;
-     my ($fh, $file) = tempfile();
- 
-     my $cmd = qq{TERM=dumb nc-apps appstore 'map(select(.created > "$date"))' > "$file" 2>/dev/null};
-     system($cmd);
- 
-     open my $in, '<', $file or die "Failed to read tmpfile: $!";
-     local $/;
-     my $out = <$in>;
-     close $in;
- 
-     $out =~ s/\e\[?.*?[a-zA-Z]//g;  # Strip ANSI sequences
-     return $out;
- }
+ sub fetch_app_data_after {
+     my ($date) = @_;
+ 
+     my $cmd = qq{nc-apps appstore 'map(select(.created > "$date"))' lang="$lang" --quiet 2>/dev/null};
+     my $out = qx{$cmd};
+ 
+     my $exit_code = $? >> 8;
+     die "Failed to fetch app data: command '$cmd' exited with code $exit_code\n"
+         if $exit_code != 0;
+ 
+     return $out;
+ }

and in build_rss:

sub build_rss {
    my ($apps) = @_;
    my $rss = XML::RSS->new(version => '2.0');
    $rss->channel(
        title       => $title,
        link        => "$base_url/",
        description => $description,
-         language    => 'en',
+         language    => $lang,
    );

---

In addition to these changes, I've added the following:

• an option to output the XML data to a file or to stdout, for further use in a web server with SSL
• the ability to pass all other options as arguments
• a help function.

#!/usr/bin/env perl
# -*- coding: utf-8 -*-

# Required Debian packages:
# sudo apt install -y libxml-rss-perl libjson-xs-perl libhttp-server-simple-perl libdatetime-perl libfile-temp-perl libjson-perl

use strict;
use warnings;
use utf8;

use Getopt::Long;               # for parsing command-line options
use JSON;
use DateTime;
use XML::RSS;
use HTTP::Server::Simple::CGI;

# --- Default configuration ---
my $host        = '0.0.0.0';                         # Server host (default)
my $port        = 8090;                              # Port to serve the RSS (default)
my $days        = 10;                                # Lookback period in days (default)
my $title       = 'Nextcloud Apps Feed';             # RSS title (default)
my $description = "RSS feed of Nextcloud apps published in the last $days days";  # Description (default)
my $lang        = 'en';                              # Language code for RSS (default)
my $show_help;

# CLI options (override defaults)
my $out_file;
GetOptions(
    'host|H=s'         => \$host,        # --host or -H (-h is for the help option)
    'port|p=i'         => \$port,        # --port or -p
    'days|d=i'         => \$days,        # --days or -d
    'title|t=s'        => \$title,       # --title or -t
    'description|e=s'  => \$description, # --description or -e
    'lang|l=s'         => \$lang,        # --lang or -l
    'help|h|?'         => \$show_help,
# output file path (optional)
    'out|output-xml=s' => \$out_file,    # --out or --output-xml, '--out -' is stdout
) or die "Error parsing command-line options\n";

if ($show_help) {
    print_help();
    exit 0;
}

# Update description if it still contains default days placeholder
$description =~ s/last \$days days/last $days days/;

# Compute base_url from potentially overridden host/port
my $base_url = "http://$host:$port";

main();

sub main {
    my $since = get_days_ago_iso($days);
    my $raw   = fetch_app_data_after($since);
    my $apps  = parse_json($raw);
    my $rss   = build_rss($apps)->as_string;

    if (defined $out_file) {
        # write RSS to the specified file
        if ($out_file eq '-') {
            print $rss;
        }
        else {
            open my $fh, '>', $out_file
              or die "Cannot write to '$out_file': $!";
            print $fh $rss;
            close $fh;
            print "RSS feed written to $out_file\n";
        }
    }
    else {
        # start HTTP server to serve RSS
        serve_rss($rss);
    }
}

sub print_help {
    print <<"EOF";
Usage: $0 [options]

Options:
  --host, -H <host>           Host to bind to (default: 0.0.0.0)
  --port, -p <port>           Port to listen on (default: 8090)
  --days, -d <days>           Number of days to look back (default: 10)
  --title, -t <title>         RSS feed title (default: "Nextcloud Apps Feed")
  --description, -e <desc>    RSS feed description
  --lang, -l <code>           Language code (default: en)
  --out, --output-xml <file>  Write RSS to file instead of serving (use '-' for stdout)
  --help, -h, -?              Show this help message and exit

Examples:
  $0                             Start server on port 8090
  $0 --days 5 --lang de          Serve RSS for last 5 days in German
  $0 --out feed.xml              Write RSS to feed.xml
  $0 --out -                     Output RSS to stdout
EOF
}

sub get_days_ago_iso {
    my ($days) = @_;
    return DateTime->now->subtract(days => $days)->iso8601() . 'Z';
}

sub fetch_app_data_after {
    my ($date) = @_;

    # Build the command
    my $cmd = qq{nc-apps appstore 'map(select(.created > "$date"))' lang="$lang" --quiet 2>/dev/null};

    # Execute and capture output
    my $out = qx{$cmd};

    # Check exit status of the command
    my $exit_code = $? >> 8;
    die "Failed to fetch app data: command '$cmd' exited with code $exit_code\n"
        if $exit_code != 0;

    return $out;
}

sub parse_json {
    my ($raw) = @_;
    my $json = eval { decode_json($raw) };
    die "Failed to parse JSON: $@" if $@;
    return $json;
}

sub build_rss {
    my ($apps) = @_;
    my $rss = XML::RSS->new(version => '2.0');
    $rss->channel(
        title       => $title,
        link        => "$base_url/",
        description => $description,
        language    => $lang,
    );

    foreach my $app (sort { $b->{created} cmp $a->{created} } @$apps) {
        $rss->add_item(
            title       => $app->{name} || $app->{id},
            link        => $app->{website} || $app->{issueTracker} || 'https://apps.nextcloud.com/',
            description => $app->{summary} || 'No description available.',
            pubDate     => $app->{created},
        );
    }

    return $rss;
}

sub serve_rss {
    my ($rss_content) = @_;
    print "Serving RSS feed on $base_url\n";
    My::RSSServer->new($port, $rss_content)->run;
}

# --- Web server class for RSS feed ---
package My::RSSServer;
use base 'HTTP::Server::Simple::CGI';

sub new {
    my ($class, $port, $content) = @_;
    my $self = $class->SUPER::new($port);
    $self->{content} = $content;
    return $self;
}

sub handle_request {
    my ($self, $cgi) = @_;
    print "HTTP/1.0 200 OK\r\n";
    print "Content-Type: application/rss+xml; charset=utf-8\r\n\r\n";
    print $self->{content};
}

sub print_banner {
    # Suppress default message from HTTP::Server::Simple
}

1;

This could definitely still be improved here and there, but I think it's already a really great script as it is!

Thank you so much for this great contribution!

---

Occasionally you should update nc-apps. When you run it interactively without --quiet, it will prompt you immediately if an update is available; when called with --quiet (e.g. from cron), any available-update notifications still get logged.

---

ernolf

[ernolf]

You can see it here:

https://global-social.net/rss

[Sanookmakmak]

Thanks for the great improvements!

The next logical step would be exposing the entire Appstore dataset. This would allow integration with Nextcloud's Unified Search, enabling administrators to search across all available apps — not just those currently installed (which, as far as I know, is the current limitation).

Of course, implementing this would likely require upstream support from the Nextcloud project, assuming there's interest from someone in the community to take it forward.

[ernolf]

Thanks for the great improvements!

I had to make a few more changes to get valid RSS.
If you don't mind, I can include the script in the repo, so made changes will always be documented and usable.

---

The next logical step would be exposing the entire Appstore dataset.

But that should already be the case.

not just those currently installed (which, as far as I know, is the current limitation).

No, it should not be limited that way.
But since you mentioned it here, I took another look at the code, and yes, you did indeed find a bug that crept in during the last major conversion of boolean variables into associative arrays. I've fixed it and I want to thank you again for your atention. I would have noticed it at some point, but since my development focus has been less on nc-apps, I just hadn't noticed it yet.

It is fixed now and when you use the "store" section, it will list only apps from the appstore which are not installed, as supposed.

If you search with pattern in the app ID (id=$pattetn) or the app name (name="$pattern"), it will allways search the entire App Store and not only in the installed apps.

If you search with the "all" section, it will include installed AND not installed Apps available in the appstore

---

ernolf

[Sanookmakmak]

If you don't mind, I can include the script in the repo, so made changes will always be documented and usable.

Yes please, with pleasure. That would be an enrichment for the community!

[ernolf]

Yes please, with pleasure. That would be an enrichment for the community!

I'm still heavily developing it at the moment. I switched from XML::RSS to XML::Writer so that I'm also compatible with Atom feeds. I then packed significantly more information into the feed, in a way that also displays well in Nextcloud News.

As soon as things settle down a bit in development, I'll publish it.

And then I have another script, the one called by the con, which creates the website and the feeds by calling the perl scripts with the required argumets and adds it to the nextcloud database (everything is served on a nextlcoud server with the raw app).

I have now additionally created another feed that lists the most recently update apps.

It's updated every 30 minutes to be as close to the hourly update interval of Nextcoud News as possible.

At the moment, I have two scripts for

• New Apps

and for

• Last updated Apps

but I'd like to combine them into one perl script.

---

So far, everything is valid RSS and works quite well, but I'm still developing it, so the feeds are subject to change. In that case, simply delete them in Nextcloud News and subscribe again.

Here's an overview of the feeds:

https://global-social.net/rss

I hope you like it a lot!

Suggestions are more than welcome!
Maybe you have good ideas for more feeds?

---

ernolf

[Sanookmakmak]

Maybe you have good ideas for more feeds?

A feed that displays the entire Appstore, because then you can search the entire Appstore easily and quickly, for example with the Vivaldi browser, because it has an integrated feed reader.

[ernolf]

Hi @Sanookmakmak,

I somehow get the feeling that you don't fully understand nc-apps as it's intended. That you haven't even tried it out properly yet. Because you're demanding features that are fully included in nc-apps already.

because then you can search the entire Appstore easily and quickly

To browse the entire app store, you should use nc-apps as it was originally intended. The ability to search the JSON code directly is more of a byproduct but not that powerfull as it looks.
Browsing the entire app store always requires some important filtering. This mainly depends on the Nextcloud version you're looking for, because the apps need to be used in a different app version depending on which Nextcloud version they're needed for, and vice versa. Some apps have been incorporated into the Nextcloud core at some point, and so on. And then this version limit isn't 100% reliable either, because some apps that are officially only supported up to Nextcloud version 30, for example, can very well work under 31. This makes it necessary to be able to filter very precisely, and filtering is exactly what nc-apps does very well. However, this is incredibly computationally intensive and slow. You'll have noticed this if you've tried the full functionality of nc-apps on the command line, and not just

nc-apps appstore '<jq-filtercode>'.

Therefore, you should use nc-apps as intended and described in the help, to search the entire app store, i.e., on the command line. Putting it in a feed isn't very useful. It's not really what nc-apps was developed for.
The philosophy behind ebtb scripts is command line, no interfaces that require a mouse. 😉

Please don't get me wrong. Of course, I could map all these results into individual feeds, but then you'd end up with a huge number of feeds. It's actually much easier to search the app store on the website if you're not comfortable navigating the command line.

I think a feed for newly created apps and recently updated apps, like the one currently provided, is useful. A feed for searching the app store is like the tail wagging the dog.

---

ernolf