Test Core Spotlight Indexing, Search, and Removal on a Cloud Mac

A user saves a document in an app but cannot open it from the system’s search results. Retesting on the development machine works, while an automated test on a cloud Mac fails intermittently: the indexing callback has reported success, but the search still returns nothing. To diagnose this, do not collapse “submitted successfully,” “discoverable,” and “gone after removal” into one assertion. They are three separate stages of the index lifecycle, and each needs its own verification.

Define the test boundary

In Core Spotlight, CSSearchableIndex submits and removes searchable items provided by the app. The indexing callback reports the outcome of that operation, not whether a search query can already read the new data. At a minimum, the test should record the item identifier, search term, indexing result, time to first match, and query result after removal.

Run the test in an app-hosted XCTest target, rather than only in a logic test target that does not load the app environment. Use a dedicated domainIdentifier for test items. Generate a new uniqueIdentifier and title on every run so an item left behind by an earlier run cannot cause a false positive. Prove the flow with one item before expanding to batch indexing. Do not use a fixed user document as the test fixture: if a user changes or deletes it, the test outcome changes too.

A successful indexing callback passes only the submission stage. The search stage requires a query, and the removal stage requires another query. Keep the result of each stage.

Verify indexing and removal with XCTest

The following test shows how to use asynchronous operations with XCTest. It first removes any item that might remain under the identifier, indexes one item with a unique title, and then removes that identifier. The example checks the operation callbacks; it does not, by itself, prove that the item is discoverable. Add the search assertion described in the next section.

import CoreSpotlight
import XCTest

final class SpotlightLifecycleTests: XCTestCase {
    func testWriteAndRemove() {
        let index = CSSearchableIndex.default()
        let id = "smoke-\(UUID().uuidString)"
        let title = "SpotlightSmoke-\(id)"
        let attributes = CSSearchableItemAttributeSet(contentType: .text)
        attributes.title = title

        let item = CSSearchableItem(
            uniqueIdentifier: id,
            domainIdentifier: "spotlight-smoke-tests",
            attributeSet: attributes
        )

        func complete(
            _ name: String,
            _ operation: (@escaping (Error?) -> Void) -> Void
        ) {
            let done = expectation(description: name)
            operation { error in
                XCTAssertNil(error, "\(name): \(String(describing: error))")
                done.fulfill()
            }
            wait(for: [done], timeout: 10)
        }

        complete("remove previous item") { finish in
            index.deleteSearchableItems(
                withIdentifiers: [id],
                completionHandler: finish
            )
        }
        complete("index item") { finish in
            index.indexSearchableItems(
                [item],
                completionHandler: finish
            )
        }
        complete("remove indexed item") { finish in
            index.deleteSearchableItems(
                withIdentifiers: [id],
                completionHandler: finish
            )
        }
    }
}

In a real project, put removal in the test’s cleanup path. If an assertion fails after indexing, the test should still try to remove the submitted item. Do not silently ignore a cleanup failure; report it along with the item identifier, or leftover data may interfere with later runs.

Allow bounded time for search synchronization

In the test target, use CSSearchQuery to search for the unique title or search term generated for this run, then check the returned items’ uniqueIdentifier values. Do not assert only that “at least one result” exists: user data with the same name could let the wrong item pass. Start querying after the indexing callback returns. If there is no match, wait briefly and create a new query; fail only if the item is still missing at the preset deadline. After the removal callback, poll the same way until that identifier no longer appears.

The deadline is a test budget, not a service performance guarantee. Observe several runs on a fixed simulator configuration before choosing a reasonable limit; do not extend the wait indefinitely because one query was slow. Give both the query callback and the polling loop their own termination conditions so a single hang cannot consume the entire pipeline.

Keep the cloud Mac test run repeatable

Pin the project, Scheme, and simulator destination for this test, and confirm that the Scheme is shared. Before the first run, use xcodebuild -showdestinations to inspect the available destinations. Then put the chosen device identifier in the run configuration rather than relying on automatic selection of the “first” simulator. For example:

xcodebuild test \
  -project Example.xcodeproj \
  -scheme Example \
  -destination 'platform=iOS Simulator,id=SIMULATOR_UDID' \
  -only-testing:ExampleTests/SpotlightLifecycleTests \
  -resultBundlePath "$PWD/spotlight-result.xcresult"

Replace SIMULATOR_UDID and the project name with their actual values; the result bundle path must not exist before each run. Index tests generally do not require wiping the whole simulator. Prefer cleaning up the test domain and this run’s identifier: it takes less time and preserves state needed by other tests. If you must reset the device, do it in a separate job to avoid sharing the device set with parallel tests.

Investigate intermittent failures and finish verification

When a test fails, identify the failing stage before immediately rerunning it. If the indexing callback reports an error, inspect the test host, item attributes, and error object. If indexing succeeds but search times out, check that the search term was written to a searchable attribute, that the query checks this run’s identifier, and that another job is not cleaning up the same test domain. If the item remains searchable after removal, record the identifier of the last matching item and confirm that the removal call targeted the same uniqueIdentifier.

Finally, check the concurrency setup: index lifecycle tests for the same app should not run in parallel on one simulator if they share a domain cleanup operation. Retaining each run’s identifier, three stage results, and test result bundle lets you distinguish operation errors, search synchronization delays, and state contamination within the test. The user’s search path is verified only when indexing, search, and removal all pass.

Frequently asked questions