Skip to content

Remove monolithic tensorflow dependency from Python targets #3545

@veblush

Description

@veblush

Background

While the core TFLite Micro (TFLM) C++ library is entirely independent of TensorFlow, our Python bindings, tests, and related tooling currently depend on the monolithic tensorflow Python package. While historically convenient, this dependency introduces severe maintainability and infrastructure issues for our Python ecosystem.

I propose completely removing the tensorflow dependency from the TFLM repository's Python targets, replacing it with leaner, standard alternatives.

Motivation

Keeping the monolithic TensorFlow dependency presents two major blockers:

1. Python Version Lock-in

The tensorflow package dictates which Python versions we can support. When a new Python version is released, TFLM is blocked from adopting or testing against it until the main TensorFlow project officially supports it and releases a new pip package.

For example, with the recent release of Python 3.14, tensorflow does not yet provide compatible wheels. As a result, TFLM is blocked from adding Python 3.14 support to our Python CI or releasing Python 3.14-compatible tflite-micro packages. This tight coupling slows our upgrade cycles and creates unnecessary friction.

2. Massive Size Causing CI Resource Exhaustion

The monolithic TensorFlow package brings in a massive amount of C++ core dependencies (e.g., XLA, MLIR, LLVM), making it incredibly difficult to work with:

  • Package Size: A standard pip installation of tensorflow consumes about 1.8 GB of disk space per environment, whereas the lightweight ai-edge-litert (formerly tflite-runtime) or native tflite-micro packages are typically only a few megabytes.
  • GitHub CI Failures: Our GitHub Actions workflows frequently encounter Out of Memory (OOM) or out-of-space errors purely due to fetching, installing, and processing this gigantic footprint.
  • Linker Bloat: When building with ASAN, compiling a simple Python debugging script that transitively depends on TensorFlow inflates the linked binary to ~2.36 GiB, requiring nearly 12 GB of memory to link. By removing the TF dependency locally, we prototyped dropping this binary size to a mere 108 MiB.

How is tensorflow currently used?

An audit of our codebase shows that tensorflow is used primarily for four reasons:

  1. Testing Infrastructure: Using tf.test.TestCase and tf.test.main().
  2. TFLite Interpreter: Using tf.lite.Interpreter for model evaluation.
  3. File I/O (gfile): Using tf.io.gfile for basic file reading.
  4. Keras / Model Conversion: Using tf.keras for loading/converting models to compare baselines.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions