Skip to main content

Contact2Sale (C2S)

C2S provides practical tracking of customers arriving from your sources, with automatic alerts and reminders.

Step get_data_from_c2s​

This step collects leads from the Contact2Sale API using Full Load when no execution history exists, or Incremental when an update date/time has already been recorded.

Contents​

Overview​

This connector interacts with the endpoint:

GET <https://api.contact2sale.com/integration/leads>

It observes:

  • Pagination through the page and perpage parameters.
  • A limit of 10 requests per minute (to avoid API blocking).
  • Incremental filtering through updated_gte, returning only leads updated since a specific date/time.

If no execution history exists (first run), a full load collects all available records. Subsequent runs retrieve only records updated after the last recorded date.

Main Files and Variables​

  • get_data_from_c2s.py: Python file containing the collection logic and these functions:
    • get_last_update()
    • save_last_update()
    • fetch_data()
    • find_max_updated_at()
    • main()
  • last_update.json: File storing the last update date/time (ISO 8601 format with a trailing Z). If this file is missing or invalid, the script performs a Full Load.

Configuration Variables​

  • BASE_URL: API base URL (for example: https://api.contact2sale.com/integration/leads).
  • TOKEN: Authentication token (Bearer Token).
  • PER_PAGE: Number of records per page.
  • MAX_REQ_PER_MINUTE: Maximum requests per 60 seconds (avoids rate limiting).
  • LAST_UPDATE_PATH: Path to the last_update.json file.

How the Step Connector Works​

  1. Checking the History File (last_update.json)
    The script calls get_last_update() to check for a saved date/time.
    • If none exists, it uses None → Full Load.
    • If one exists (for example, "2024-12-27T10:46:28Z"), it uses that value → Incremental.
  2. Data Collection (fetch_data())
  • Builds the request with pagination parameters (page, perpage).
  • In incremental mode, adds updated_gte to params.
  • Loops through requests until reaching the total record count reported by the API (data['total']).
  • Observes the limit of 10 requests/minute, calling time.sleep(60) when that limit is reached.
  1. Local Processing and Saving
  • All leads are accumulated in all_results.
  • If there is a destination (for example, Snowflake), the script converts the list of dictionaries into a DataFrame and writes it to the configured table.
  1. Updating updated_gte
  • After collection, the script calls find_max_updated_at() to find the latest update date/time among the returned leads.
  • If a value (new_updated_gte) is found, save_last_update() saves it in last_update.json.
  1. Future Runs
  • On the next run, get_last_update() finds this value → the script fetches only records updated after that timestamp, reducing load and processing time.

How to Run the Step Connector​

  1. Install Dependencies
    • Make sure requests, json, time and other standard Python 3 libraries are available.
    • If writing to Snowflake, install snowflake-connector-python or snowflake-snowpark-python.
  2. Configure Variables
  3. Run the Script
    • On the first run, the script will not find last_update.json and performs a full load.
    • At the end, it creates (or updates) last_update.json with the highest updated_at.
    • On the second run, it uses incremental mode (only data updated after the stored date).

Important Considerations​

  1. Record Count
    • With thousands of leads, the initial load may take a while because of the rate limit (10 requests/minute).
    • Set PER_PAGE to the maximum allowed by the API (for example, 50).
  2. Date Format
    • Contact2Sale returns dates in ISO8601 format (without milliseconds).
    • find_max_updated_at() supports strings in YYYY-MM-DDTHH:MM:SSZ format.
  3. Persistence
    • After retrieving leads, you can save them to a local JSON file, database or Snowflake.
    • Check whether duplicate handling or update merges are needed.
  4. Errors and Exceptions
    • If the API returns an error (for example, 401 or 500), the script logs the problem and raises the exception.
    • For network issues or timeouts, review the retry/timeout logic in requests.get().

References​