Add support for Ubuntu comand-not-found (CNF) indices

[pulpbot]

Author: wilful (wilful)

Redmine Issue: 9239, https://pulp.plan.io/issues/9239

---

Hi! I trying create mirror for official ubuntu repository: http://archive.ubuntu.com/ubuntu/
After creation on my test Ubuntu server i see error:

/pulp/content/archive.ubuntu.com/dists/focal/main/cnf/Commands-amd64: 404  Not Found [IP: 172.24.160.15 80]
E: Some index files failed to download. They have been ignored, or old ones used instead.

It content (CNF) have in source but not in my repository
I use verbatim publisher, on_demand policy

      "pulp_href": "/pulp/api/v3/remotes/deb/apt/25c10483-64e2-492a-ac30-6efb5bbe3683/",
      "pulp_created": "2021-08-13T13:58:19.364006Z",
      "name": "archive.ubuntu.com",
      "url": "http://archive.ubuntu.com/ubuntu/",
      "ca_cert": null,
      "client_cert": null,
      "tls_validation": true,
      "proxy_url": null,
      "pulp_labels": {},
      "pulp_last_updated": "2021-08-13T13:58:19.364027Z",
      "download_concurrency": null,
      "max_retries": null,
      "policy": "on_demand",
      "total_timeout": null,
      "connect_timeout": null,
      "sock_connect_timeout": null,
      "sock_read_timeout": null,
      "headers": null,
      "rate_limit": null,
      "distributions": "trusty focal",
      "components": null,
      "architectures": "amd64",
      "sync_sources": false,
      "sync_udebs": false,
      "sync_installer": false,
      "gpgkey": null,
      "ignore_missing_package_indices": false
    },

Can i add it is cnf folder?

[pulpbot]

From: @quba42 (quba42)
Date: 2021-08-17T11:16:56Z

---

It looks like Commands-amd64 is a new type of metadata file used by Ubuntu repos, that pulp_deb does not currently support. (This is the first time I have heard of this type of file.)

As such, this ticket is essentially a feature request, and I have reformulated it as such!

Note: I have found the following statement regarding this feature: "Starting with Ubuntu Focal (20.04), the APT client expects APT sources to provide command-not-found (cnf) metadata files."

[pulpbot]

From: ajsween (ajsween)
Date: 2021-08-18T01:55:39Z

---

There are three different meta data types not being sync’d as of Ubuntu 20.04:

• cnf - command-not-found (apt purge command-not-found will remove this)
• i11n - translation files (echo ‘Acquire::Languages "none";’ > /etc/apt.conf.d/99translations)
• appstream - (apt purge appstream will remove but it disables installs from GUI app store)

I included how to work around if you are okay with forgoing apt install recommendations with cnf, translation files with i11n, and App Store installs with appstream.

[pulpbot]

From: @quba42 (quba42)
Date: 2021-08-18T06:47:00Z

---

There is a open issue for translation files: https://pulp.plan.io/issues/8734

I will have a look at the other type you mentioned.

[pulpbot]

From: ajsween (ajsween)
Date: 2021-08-18T11:48:02Z

---

appstream is the “DEP-11” folder in main for Ubuntu Focal Fossa.

[pulpbot]

From: mgoddard (mgoddard)
Date: 2021-10-29T19:51:02Z

---

Also hitting this issue. Is there a timeframe for implementation?

[pulpbot]

From: @quba42 (quba42)
Date: 2021-11-02T08:40:51Z

---

mgoddard wrote:

Also hitting this issue. Is there a timeframe for implementation?

No timeframe as of yet I am afraid. I will throw this ticket into our internal triage and see what happens.

As I understand this is just a Warning, so the workaround is to ignore the issue and carry on without CNF indices.

[itbane]

The issue with language files (i18n) is usually a warning, but is blocking when doing a fresh installation, as the debian/ubuntu installer fails when there are issues with apt update.

A workaround for this is:

1. not use pkgsel in the preseed.cfg
2. add workaround from this post in /target/etc/apt/apt.conf.d/ in late_command
3. run in-target apt update, in-target apt install -y <packages> in late_command