Introduction

fluent is a command-line utility that interacts with Flowise AI workflows, providing a Rust implementation of various features including configuration management, HTTP requests to call AI workflows, command-line interface (CLI) processing, response handling, and output formatting.

The application allows users to specify workflow names, provide questions and context, handle outline content, parse structured responses, override system prompts, handle images, render Markdown, and include extra data in HTTP requests. It also supports generating shell completion scripts and an HTML chat client for interactive conversations with the Flowise AI chatbot.

fluent is designed to be easy to use and provides a comprehensive set of options for interacting with Flowise AI workflows. It leverages the capabilities of popular Rust libraries such as structopt, anyhow, tokio, and reqwest to handle command-line parsing, error handling, asynchronous runtime, and HTTP requests.

//! # fluent //! //! fluent is a command-line utility that interacts with Flowise AI workflows, providing a Rust //! implementation of various features including configuration management, HTTP requests to call AI //! workflows, command-line interface (CLI) processing, response handling, and output formatting. //! //! ## Features //! //! - Read and parse the configuration from a JSON file //! - Generate bash completion scripts for command-line options //! - Handle completions for shells like Bash by parsing configured workflow names
//! - Process command-line input and environment variables to execute workflows //! - Construct and send HTTP POST requests with configurable headers and query parameters //! - Download and rename image files referenced in Markdown content //! - Generate an HTML client for interactive conversations with the Flowise AI chatbot //! - Format input from various sources such as text files or standard input //! //! ## Usage //! //! The run function is the primary entry point for using the tool. It takes parsed CLI //! options and performs the necessary actions to interact with the Flowise AI workflows. //! //! rust,no_run //! #[tokio::main] //! async fn main() -> Result<(), Box<dyn std::error::Error>> { //! let opts = CliOpts::from_args(); //! if let Err(e) = run(opts).await { //! eprintln!("Application error: {}", e); //! std::process::exit(1); //! } //! Ok(()) //! } //! //! //! ## Configuration //! //! Config is a struct that represents application configuration loaded from a JSON file. It //! includes workflow definitions, HTTP request settings, and formatting options. //! //! ## Command-Line Options //! //! CliOpts is a struct that represents the parsed command-line options. It includes flags for //! generating completions, specifying the workflow, and providing inputs such as questions and //! context. //! //! ## Error Handling //! //! All functions return Result types, and any errors are handled using the anyhow crate, //! which provides context and error chains for easier debugging. //! //! ## Dependencies //! //! This module depends on several external crates such as anyhow, reqwest, clap, tokio, //! config, serde_json, and others for handling JSON parsing, HTTP requests, CLI processing, and //! asynchronous execution. //!

Installation

To install Fluent CLI, follow these steps:

1. Ensure you have Rust and Cargo installed on your system. If you don’t have Rust installed, you can download and install it from the official Rust website: https://www.rust-lang.org/tools/install

2. Clone the Fluent CLI repository from GitHub:

   git clone https://github.com/your-username/fluent-cli.git

3. Change into the cloned repository directory:

   cd fluent-cli

4. Build the project using Cargo:

   cargo build --release

   This command will compile the Fluent CLI application with optimizations.

5. The compiled binary will be located in the target/release directory. You can run it directly from there:

   ./target/release/fluent

   Alternatively, you can install the binary to your system’s global binary directory by running:

   cargo install --path .

   This will make the fluent command available system-wide.

6. Set the FLUENT_CONFIG_PATH environment variable to the path of your configuration file:

   export FLUENT_CONFIG_PATH=/path/to/your/config.json

   Make sure to replace /path/to/your/config.json with the actual path to your configuration file.

   You can also add this environment variable to your shell’s configuration file (e.g., ~/.bashrc or ~/.zshrc) to make it persistent across terminal sessions.

That’s it! You should now be able to run the fluent command from anywhere in your terminal.

Prerequisites

• Rust 1.54 or later
• tokio 1.0 or later
• reqwest 0.11 or later
• serde_json 1.0 or later
• anyhow 1.0 or later
• regex 1.5 or later
• chrono 0.4 or later
• termimad 0.20 or later
• ansi_term 0.12 or later
• structopt 0.3 or later
• stderrlog 0.5 or later
• faccess 0.2 or later
• async-trait 0.1 or later
• atty 0.2 or later
• clap 3.0 or later
• config 0.13 or later
• futures 0.3 or later

Building from Source

To build FluentCLI from the source code, follow these steps:

1. Ensure you have Rust and Cargo installed on your system. If not, visit the official Rust website (https://www.rust-lang.org/) and follow the installation instructions for your operating system.

2. Clone the FluentCLI repository from GitHub:

   git clone https://github.com/your-username/fluent.git

3. Change into the fluent directory:

   cd fluent

4. Build the project using Cargo:

   cargo build --release

   This command will compile the FluentCLI source code and generate an optimized binary in the target/release directory.

5. (Optional) Run the tests to ensure everything is working correctly:

   cargo test

   This command will execute the test suite and report any failures.

6. Once the build is successful, you can find the FluentCLI binary at target/release/fluent. You can run it directly or copy it to a directory in your system’s PATH for easier access.

FluentCLI has the following dependencies, which will be automatically downloaded and compiled by Cargo during the build process:

• anyhow: For error handling and propagation.
• async-trait: For defining asynchronous traits.
• clap: For command-line argument parsing and generation of shell completion scripts.
• reqwest: For making HTTP requests to the Flowise AI workflows.
• serde and serde_json: For JSON serialization and deserialization.
• stderrlog: For logging to stderr with configurable verbosity levels.
• structopt: For defining command-line interface options and arguments.
• tokio: For asynchronous runtime and I/O operations.

Make sure you have a stable internet connection to allow Cargo to download and compile these dependencies during the build process.

Configuration

The configuration for the Fluent application is handled by the Config struct defined in the config module. The Config struct contains various fields for configuring different aspects of the application, such as HTTP requests, paths, regular expressions, defaults, styling, and skin configurations.

To configure the Fluent application, you need to provide a JSON configuration file. The path to this file is specified via the FLUENT_CONFIG_PATH environment variable. The read_config function in the config module is responsible for reading the configuration file and parsing its contents into the Config struct.

Here’s an example of what the JSON configuration file might look like:

{
    "flows": [],
    "http_request": {
        "timeout": 60
    },
    "combined_input_format": "",
    "processing_message": "Processing...",
    "paths": {
        "config_path": "/path/to/config"
    },
    "regex": {
        "url_regex": "^https?://"
    },
    "defaults": {
        "verbosity": 1
    },
    "styling": {
        "error": "red",
        "success": "green"
    },
    "skin_config": {
        "headers_fg": "blue",
        "bold_fg": "yellow",
        "italic_fg_bg": ["magenta", "black"],
        "bullet": "",
        "bullet_char": "-",
        "quote_mark": ">",
        "code_block_margin": 4,
        "code_block_align": "Left",
        "code_block_bg": "black",
        "italic_attrs": ["Underlined"]
    }
}

The flows field is an array that represents the flow of operations, including HTTP requests and configuration overrides.

The http_request field is an object that contains the configuration for HTTP requests, such as the timeout value.

The combined_input_format field is a string that specifies the format for combining input data.

The processing_message field is a string that defines the message to display while processing.

The paths field is an object that contains different file paths used in the application.

The regex field is an object that specifies regex patterns used in the application.

The defaults field is an object that defines default configuration values.

The styling field is an object that configures the styling of console output, such as colors for error and success messages.

The skin_config field is an object that allows customization of the appearance (“skin”) of console output, including colors, formatting, and layout options.

By providing the appropriate values in the JSON configuration file and setting the FLUENT_CONFIG_PATH environment variable to the path of the file, you can configure the Fluent application according to your needs.

//! ## Configuration //! //! Config is a struct that represents application configuration loaded from a JSON file. It //! includes workflow definitions, HTTP request settings, and formatting options. //! //! ## Example Configuration File //! //! json //! { //! "flows": [ //! { //! "name": "custom_flow", //! "url": "http://localhost:3000/api/v1/flow/custom_flow", //! "method": "POST", //! "headers": { //! "Content-Type": "application/json" //! }, //! "overrideConfig": { //! "key1": "value1", //! "key2": "value2" //! } //! } //! ], //! "http_request": { //! "timeout": 60 //! }, //! "combined_input_format": "Request:\n{final_question}\nOutline:\n{outline_content}\nContext:\n{final_context}\nRequest:\n{final_question}\n. Only output the answer.\n", //! "processing_message": "Processing your request with {flow_name}: {question}", //! "paths": { //! "config_path": "/path/to/config" //! }, //! "regex": { //! "url_regex": "^https?://" //! }, //! "defaults": { //! "verbosity": 1 //! }, //! "styling": { //! "error": "red", //! "success": "green" //! }, //! "skinConfig": { //! "headers_fg": "blue", //! "bold_fg": "yellow", //! "italic_fg_bg": ["magenta", "black"], //! "bullet": "", //! "bullet_char": "-", //! "quote_mark": ">", //! "code_block_margin": 4, //! "code_block_align": "Left", //! "code_block_bg": "black", //! "italic_attrs": ["Underlined"] //! } //! } //! //! //! ### Configuration Options //! //! - flows: An array of workflow definitions. Each workflow has the following properties: //! - name: The name of the workflow. //! - url: The URL endpoint for the workflow. //! - method: The HTTP method for the request (e.g., “POST”). //! - headers: Additional headers to include in the request. //! - overrideConfig: Workflow-specific configuration overrides. //! //! - http_request: HTTP request settings. //! - timeout: The timeout duration in seconds for HTTP requests. //! //! - combined_input_format: The format string for combining input data. It supports placeholders like {final_question}, {outline_content}, and {final_context}. //! //! - processing_message: The message to display while processing a request. It supports placeholders like {flow_name} and {question}. //! //! - paths: File and directory paths used by the application. //! - config_path: The path to the configuration file. //! //! - regex: Regular expression patterns used by the application. //! - url_regex: The regex pattern for validating URLs. //!
//! - defaults: Default values for various settings. //! - verbosity: The default verbosity level. //! //! - styling: Color and style settings for console output. //! - error: The color for error messages. //! - success: The color for success messages. //! //! - skinConfig: Settings for customizing the appearance of console output. //! - headers_fg: The foreground color for headers. //! - bold_fg: The foreground color for bold text. //! - italic_fg_bg: The foreground and background colors for italic text. //! - bullet: The bullet character for lists. //! - bullet_char: The alternate bullet character. //! - quote_mark: The character for marking quotes. //! - code_block_margin: The margin size for code blocks. //! - code_block_align: The alignment for code blocks (“Left”, “Right”, or “Center”). //! - code_block_bg: The background color for code blocks. //! - italic_attrs: Additional attributes for italic text (e.g., “Underlined”).

Based on the Fluent CLI source code, the following environment variables are used:

Environment Variables

• FLUENT_CONFIG_PATH: Specifies the path to the Fluent configuration file. If not set, the default value is “config.json”. This environment variable is used in the read_config function of the config module to locate and read the application’s configuration.

• CARGO_BIN_EXE_fluent: Used in the test_version_command_output test function to get the path to the fluent binary for testing the --version command output. This environment variable is automatically set by Cargo during tests.

• CARGO_PKG_VERSION: Used in the test_version_command_output test function to get the current version of the package. This environment variable is automatically set by Cargo and contains the version number defined in the Cargo.toml file.

Usage

To use the Fluent CLI, compile the binary and run it with the necessary flags and options.

For example:

fluent --name "Flowise" --question "What is AI?" --outline-path "/path/to/outline"

Options and Flags

• --name: Required. The name for the workflow.
• --question: Optional. A question to pass along to the workflow.
• --outline-path: Optional. Either as a positional argument or flagged by “–outline-path”.
• --parse-structured: Flag to parse structured data.
• --system-prompt-override: Flag to provide a system prompt override.
• --image-folder-path: Flag to specify an image folder path.
• --markdown: Flag to enable markdown rendering.
• --include-extra: Flag to include extra data in the HTTP request.
• --generate-html-chat-client: Flag to generate an HTML chat client.
• --generate-completions: Flag to generate Bash shell autocompletion scripts to source from config.
• --dump-completions: Generates shell completions from config for the specified shell.

Command-Line Options

The Fluent CLI application provides the following command-line options:

• --name (required): The name for the workflow.
  • Example: fluent --name "Flowise"
• --question: A question to pass along to the workflow.
  • Example: fluent --name "Flowise" --question "What is AI?"
• --outline-path: The path to an outline file, either as a positional argument or flagged by --outline-path.
  • Example: fluent --name "Flowise" --outline-path "/path/to/outline"
• --parse-structured: Flag to parse structured data.
  • Example: fluent --name "Flowise" --parse-structured
• --system-prompt-override: Flag to provide a system prompt override.
  • Example: fluent --name "Flowise" --system-prompt-override "Custom prompt"
• --image-folder-path: Flag to specify an image folder path.
  • Example: fluent --name "Flowise" --image-folder-path "/path/to/images"
• --markdown: Flag to enable markdown rendering.
  • Example: fluent --name "Flowise" --markdown
• --include-extra: Flag to include extra data in the HTTP request.
  • Example: fluent --name "Flowise" --include-extra
• --generate-html-chat-client: Flag to generate an HTML chat client.
  • Example: fluent --name "Flowise" --generate-html-chat-client
• --generate-completions: Flag to generate Bash shell autocompletion scripts to source from config.
  • Example: fluent --name "Flowise" --generate-completions
• --dump-completions: Generates shell completions from config for the specified shell.
  • Example: fluent --dump-completions bash

Positional Arguments

• outline_path_positional: An optional positional argument to specify the path to an outline file.

Here are some practical usage examples for the Fluent CLI tool:

Extracting code blocks

let text = "Here is some code: ```rust\nlet x = 3;\n```";
let code_blocks = extract_code_blocks(text);
assert_eq!(code_blocks[0].content, "let x = 3;\n");

Finding image URLs

let text_with_images = "This has an image: ![alt text](http://example.com/image.png)";
let image_urls = find_image_urls(text_with_images);
assert_eq!(image_urls[0], "http://example.com/image.png");

Sanitize filename

let filename = sanitize_filename(&"some/unsafe?filename with spaces.txt");
assert_eq!(filename, "some_unsafe_filename_with_spaces.txt");

Convert JSON to Headers

let json_headers = serde_json::json!({ "Content-Type": "application/json", "Accept": "application/json" });
let headers = convert_to_header_map(&json_headers).unwrap();
assert_eq!(headers["Content-Type"], "application/json");
assert_eq!(headers["Accept"], "application/json");

Interacting with Flowise AI Workflows

The run function is the primary entry point for using the tool. It takes parsed CLI options and performs the necessary actions to interact with the Flowise AI workflows.

#[tokio::main]
async fn main() -> Result<(), Box<dyn std::error::Error>> {
    let opts = CliOpts::from_args();
    if let Err(e) = run(opts).await {
        eprintln!("Application error: {}", e);
        std::process::exit(1);
    }
    Ok(())
}

To interact with Flowise AI workflows using FluentCLI:

1. Configure the workflow definitions, HTTP request settings, and formatting options in the JSON configuration file.

2. Run the fluent command with the appropriate CLI options:

   • Use the --name flag to specify the workflow to execute.
   • Provide the question or input using the --question flag.
   • Optionally, specify an outline file path with --outline-path.
   • Use additional flags like --parse-structured, --markdown, --include-extra to control the behavior and output format.

3. The run function will read the configuration, process the CLI options, and construct the necessary input for the specified workflow.

4. It will then send an HTTP POST request to the workflow’s URL with the constructed input and any configured headers and query parameters.

5. The response from the workflow will be handled based on the specified options. It can be parsed as structured data, rendered with Markdown formatting, or output as plain text.

6. If the --image-folder-path flag is provided, any images referenced in the response will be downloaded and saved to the specified directory.

7. Finally, the processed response will be displayed on the console, or an error will be reported if the workflow execution fails.

By following these steps and providing the appropriate configuration and CLI options, you can easily interact with Flowise AI workflows using the FluentCLI tool.

Specifying Workflow Name

The --name flag is used to specify the name of the Flowise AI workflow you want to interact with. This flag is required.

For example, to run a workflow named “Flowise”:

fluent --name "Flowise"

The value provided to --name should match the name of a workflow defined in your configuration file.

Usage

//! //! The run function is the primary entry point for using the tool. It takes parsed CLI //! options and performs the necessary actions to interact with the Flowise AI workflows. //! //! rust,no_run //! #[tokio::main] //! async fn main() -> Result<(), Box<dyn std::error::Error>> { //! let opts = CliOpts::from_args(); //! if let Err(e) = run(opts).await { //! eprintln!("Application error: {}", e); //! std::process::exit(1); //! } //! Ok(()) //! } //!

Cautions:

/// * Never assume a directory’s permissions will remain unchanged between the time you check them /// and the time you attempt to use them. /// * Some filesystems provide sufficiently fine-grained permissions that having access to create /// a file does not imply having access to delete the file you’ve created. /// /// TODO: A complementary validator which will verify that the closest existing ancestor is /// writable. (for things that will mkdir -p if necessary.)

Here is the completed utils module as requested based on the provided context and Fluent source code:

rust //! Utility functions for the Fluent application. //! //! This module provides various utility functions for handling files, paths, strings, //! and HTTP requests in the Fluent application. It covers a range of functionalities, //! including processing code blocks within a string, image URL extraction, downloading //! images, sanitizing filenames, HTTP header handling, and parsing of color and //! alignment strings into corresponding Enumerations. //! //! ## Contents //! - [Structs](#structs) //! - [Functions](#functions) //! - [Examples](#examples) //! //! ## Structs //! //! - [`CodeBlock`]: Struct representing code blocks with optional language specifier. //! //! ## Functions //! //! - [`find_image_urls`](find_image_urls): Extracts image download URLs from text. //! - [`download_and_save_image`](download_and_save_image): Downloads an image from a URL and saves it to a folder. //! - [`sanitize_filename`](sanitize_filename): Sanitizes a string for safe use as a filename. //! - [`convert_to_header_map`](convert_to_header_map): Converts a JSON object to a [`HeaderMap`]. //! - [`unescape_string`](unescape_string): Unescapes strings, translating escape sequences to their characters. //! - [`extract_code_blocks`](extract_code_blocks): Parses strings to extract code blocks. //! - [`parse_color`](parse_color): Parses a string to produce a [`Color`] enum. //! - [`parse_alignment`](parse_alignment): Parses a string to produce an [`Alignment`] enum. //! //! ## Examples //! //! ### Extracting code blocks //!ignore //! let text = “Here is some code: rust\nlet x = 3;\n”; //! let code_blocks = extract_code_blocks(text); //! assert_eq!(code_blocks[0].content, “let x = 3;”); //! //! //! ### Finding image URLs //!ignore //! let text_with_images = “This has an image: ”; //! let image_urls = find_image_urls(text_with_images); //! assert_eq!(image_urls[0], “http://example.com/image.png”); //! //! //! ### Sanitize filename //!ignore //! let filename = sanitize_filename(&“some/unsafe?filename with spaces.txt”); //! assert_eq!(filename, “some_unsafe_filename_with_spaces.txt”); //! //! //! ### Convert JSON to Headers //!ignore //! let json_headers = serde_json::json!({ “Content-Type”: “application/json”, “Accept”: “application/json” }); //! let headers = convert_to_header_map(&json_headers).unwrap(); //! assert_eq!(headers[“Content-Type”], “application/json”); //! assert_eq!(headers[“Accept”], “application/json”); //! ``//! //! [CodeBlock]: struct.CodeBlock.html //! [find_image_urls]: fn.find_image_urls.html //! [download_and_save_image]: fn.download_and_save_image.html //! [sanitize_filename]: fn.sanitize_filename.html //! [convert_to_header_map]: fn.convert_to_header_map.html //! [HeaderMap]: https://docs.rs/reqwest/latest/reqwest/header/struct.HeaderMap.html //! [unescape_string]: fn.unescape_string.html //! [extract_code_blocks]: fn.extract_code_blocks.html //! [Color]: https://docs.rs/termimad/latest/termimad/crossterm/style/enum.Color.html //! [parse_color]: fn.parse_color.html //! [Alignment]: https://docs.rs/termimad/latest/termimad/enum.Alignment.html //! [parse_alignment`]: fn.parse_alignment.html

use std::path::Path; use std::str::FromStr;

use reqwest::header::{HeaderMap, HeaderName, HeaderValue}; use serde_json::Value; use regex::Regex; use anyhow::{Result, anyhow};

#[derive(Debug)] pub struct CodeBlock { pub language: Option

pub fn find_image_urls(text: &str) -> Vec

pub async fn download_and_save_image(url: &str, folder_path: &Path, question: &str) -> Result<(), Box

pub fn sanitize_filename(question: &str) -> String { question.chars() .filter(|c| c.is_alphanumeric() || *c == ’_’ || *c == ‘-’) .map(|c| if c == ’ ’ { ’_’ } else { c }) .collect::

pub fn convert_to_header_map(headers: &Value) -> Result

use termimad::crossterm::style::Color; use tokio::fs::File;

pub(crate) fn parse_color(color_str: &str) -> Color { match color_str.to_lowercase().as_str() { “black” => Color::Black, “red” => Color::Red, “green” => Color::Green, “yellow” => Color::Yellow, “blue” => Color::Blue, “magenta” => Color::Magenta, “cyan” => Color::Cyan, “white” => Color::White, _ => Color::White, // Default to white } } pub(crate) fn parse_alignment(align_str: &str) -> termimad::Alignment { match align_str { “Left” => termimad::Alignment::Left, “Right” => termimad::Alignment::Right, “Center” => termimad::Alignment