Sep 18 13:56:42.089512 ip-10-0-129-16 systemd[1]: kubelet.service: Failed to load environment files: No such file or directory
Sep 18 13:56:42.089524 ip-10-0-129-16 systemd[1]: kubelet.service: Failed to run 'start-pre' task: No such file or directory
Sep 18 13:56:42.089531 ip-10-0-129-16 systemd[1]: kubelet.service: Failed with result 'resources'.
Sep 18 13:56:42.089696 ip-10-0-129-16 systemd[1]: Failed to start Kubernetes Kubelet.
Sep 18 13:56:52.338729 ip-10-0-129-16 systemd[1]: kubelet.service: Failed to schedule restart job: Unit crio.service not found.
Sep 18 13:56:52.338746 ip-10-0-129-16 systemd[1]: kubelet.service: Failed with result 'resources'.
-- Boot ae57acdbb44345b1bf31cc2c0eea20a0 --
Sep 18 13:58:50.299329 ip-10-0-129-16 systemd[1]: Starting Kubernetes Kubelet...
Sep 18 13:58:53.720673 ip-10-0-129-16 kubenswrapper[2565]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Sep 18 13:58:53.720673 ip-10-0-129-16 kubenswrapper[2565]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version.
Sep 18 13:58:53.720673 ip-10-0-129-16 kubenswrapper[2565]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Sep 18 13:58:53.720673 ip-10-0-129-16 kubenswrapper[2565]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI.
Sep 18 13:58:53.720673 ip-10-0-129-16 kubenswrapper[2565]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Sep 18 13:58:53.721494 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:58:53.720739    2565 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime"
Sep 18 13:58:53.724465 ip-10-0-129-16 kubenswrapper[2565]: W0918 13:58:53.724448    2565 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 18 13:58:53.724465 ip-10-0-129-16 kubenswrapper[2565]: W0918 13:58:53.724463    2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 18 13:58:53.724541 ip-10-0-129-16 kubenswrapper[2565]: W0918 13:58:53.724467    2565 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 18 13:58:53.724541 ip-10-0-129-16 kubenswrapper[2565]: W0918 13:58:53.724471    2565 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 18 13:58:53.724541 ip-10-0-129-16 kubenswrapper[2565]: W0918 13:58:53.724473    2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 18 13:58:53.724541 ip-10-0-129-16 kubenswrapper[2565]: W0918 13:58:53.724478    2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 18 13:58:53.724541 ip-10-0-129-16 kubenswrapper[2565]: W0918 13:58:53.724481    2565 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 18 13:58:53.724541 ip-10-0-129-16 kubenswrapper[2565]: W0918 13:58:53.724484    2565 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 18 13:58:53.724541 ip-10-0-129-16 kubenswrapper[2565]: W0918 13:58:53.724488    2565 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 18 13:58:53.724541 ip-10-0-129-16 kubenswrapper[2565]: W0918 13:58:53.724498    2565 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 18 13:58:53.724541 ip-10-0-129-16 kubenswrapper[2565]: W0918 13:58:53.724502    2565 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 18 13:58:53.724541 ip-10-0-129-16 kubenswrapper[2565]: W0918 13:58:53.724505    2565 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 18 13:58:53.724541 ip-10-0-129-16 kubenswrapper[2565]: W0918 13:58:53.724508    2565 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 18 13:58:53.724541 ip-10-0-129-16 kubenswrapper[2565]: W0918 13:58:53.724510    2565 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 18 13:58:53.724541 ip-10-0-129-16 kubenswrapper[2565]: W0918 13:58:53.724520    2565 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 18 13:58:53.724541 ip-10-0-129-16 kubenswrapper[2565]: W0918 13:58:53.724524    2565 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 18 13:58:53.724541 ip-10-0-129-16 kubenswrapper[2565]: W0918 13:58:53.724527    2565 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 18 13:58:53.724541 ip-10-0-129-16 kubenswrapper[2565]: W0918 13:58:53.724530    2565 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 18 13:58:53.724541 ip-10-0-129-16 kubenswrapper[2565]: W0918 13:58:53.724533    2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 18 13:58:53.724541 ip-10-0-129-16 kubenswrapper[2565]: W0918 13:58:53.724535    2565 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 18 13:58:53.724541 ip-10-0-129-16 kubenswrapper[2565]: W0918 13:58:53.724538    2565 feature_gate.go:328] unrecognized feature gate: Example2
Sep 18 13:58:53.724541 ip-10-0-129-16 kubenswrapper[2565]: W0918 13:58:53.724540    2565 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 18 13:58:53.725022 ip-10-0-129-16 kubenswrapper[2565]: W0918 13:58:53.724543    2565 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 18 13:58:53.725022 ip-10-0-129-16 kubenswrapper[2565]: W0918 13:58:53.724546    2565 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 18 13:58:53.725022 ip-10-0-129-16 kubenswrapper[2565]: W0918 13:58:53.724549    2565 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 18 13:58:53.725022 ip-10-0-129-16 kubenswrapper[2565]: W0918 13:58:53.724552    2565 feature_gate.go:328] unrecognized feature gate: Example
Sep 18 13:58:53.725022 ip-10-0-129-16 kubenswrapper[2565]: W0918 13:58:53.724555    2565 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 18 13:58:53.725022 ip-10-0-129-16 kubenswrapper[2565]: W0918 13:58:53.724557    2565 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 18 13:58:53.725022 ip-10-0-129-16 kubenswrapper[2565]: W0918 13:58:53.724562    2565 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 18 13:58:53.725022 ip-10-0-129-16 kubenswrapper[2565]: W0918 13:58:53.724565    2565 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 18 13:58:53.725022 ip-10-0-129-16 kubenswrapper[2565]: W0918 13:58:53.724568    2565 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 18 13:58:53.725022 ip-10-0-129-16 kubenswrapper[2565]: W0918 13:58:53.724571    2565 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 18 13:58:53.725022 ip-10-0-129-16 kubenswrapper[2565]: W0918 13:58:53.724574    2565 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 18 13:58:53.725022 ip-10-0-129-16 kubenswrapper[2565]: W0918 13:58:53.724577    2565 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 18 13:58:53.725022 ip-10-0-129-16 kubenswrapper[2565]: W0918 13:58:53.724580    2565 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 18 13:58:53.725022 ip-10-0-129-16 kubenswrapper[2565]: W0918 13:58:53.724582    2565 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 18 13:58:53.725022 ip-10-0-129-16 kubenswrapper[2565]: W0918 13:58:53.724585    2565 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 18 13:58:53.725022 ip-10-0-129-16 kubenswrapper[2565]: W0918 13:58:53.724587    2565 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 18 13:58:53.725022 ip-10-0-129-16 kubenswrapper[2565]: W0918 13:58:53.724591    2565 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 18 13:58:53.725022 ip-10-0-129-16 kubenswrapper[2565]: W0918 13:58:53.724593    2565 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 18 13:58:53.725022 ip-10-0-129-16 kubenswrapper[2565]: W0918 13:58:53.724596    2565 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 18 13:58:53.725022 ip-10-0-129-16 kubenswrapper[2565]: W0918 13:58:53.724598    2565 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 18 13:58:53.725575 ip-10-0-129-16 kubenswrapper[2565]: W0918 13:58:53.724601    2565 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 18 13:58:53.725575 ip-10-0-129-16 kubenswrapper[2565]: W0918 13:58:53.724603    2565 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 18 13:58:53.725575 ip-10-0-129-16 kubenswrapper[2565]: W0918 13:58:53.724606    2565 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 18 13:58:53.725575 ip-10-0-129-16 kubenswrapper[2565]: W0918 13:58:53.724608    2565 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 18 13:58:53.725575 ip-10-0-129-16 kubenswrapper[2565]: W0918 13:58:53.724611    2565 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 18 13:58:53.725575 ip-10-0-129-16 kubenswrapper[2565]: W0918 13:58:53.724613    2565 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 18 13:58:53.725575 ip-10-0-129-16 kubenswrapper[2565]: W0918 13:58:53.724616    2565 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 18 13:58:53.725575 ip-10-0-129-16 kubenswrapper[2565]: W0918 13:58:53.724618    2565 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 18 13:58:53.725575 ip-10-0-129-16 kubenswrapper[2565]: W0918 13:58:53.724621    2565 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 18 13:58:53.725575 ip-10-0-129-16 kubenswrapper[2565]: W0918 13:58:53.724625    2565 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 18 13:58:53.725575 ip-10-0-129-16 kubenswrapper[2565]: W0918 13:58:53.724628    2565 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 18 13:58:53.725575 ip-10-0-129-16 kubenswrapper[2565]: W0918 13:58:53.724631    2565 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 18 13:58:53.725575 ip-10-0-129-16 kubenswrapper[2565]: W0918 13:58:53.724635    2565 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 18 13:58:53.725575 ip-10-0-129-16 kubenswrapper[2565]: W0918 13:58:53.724638    2565 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 18 13:58:53.725575 ip-10-0-129-16 kubenswrapper[2565]: W0918 13:58:53.724641    2565 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 18 13:58:53.725575 ip-10-0-129-16 kubenswrapper[2565]: W0918 13:58:53.724644    2565 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 18 13:58:53.725575 ip-10-0-129-16 kubenswrapper[2565]: W0918 13:58:53.724647    2565 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 18 13:58:53.725575 ip-10-0-129-16 kubenswrapper[2565]: W0918 13:58:53.724650    2565 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 18 13:58:53.725575 ip-10-0-129-16 kubenswrapper[2565]: W0918 13:58:53.724653    2565 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 18 13:58:53.726054 ip-10-0-129-16 kubenswrapper[2565]: W0918 13:58:53.724656    2565 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 18 13:58:53.726054 ip-10-0-129-16 kubenswrapper[2565]: W0918 13:58:53.724658    2565 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 18 13:58:53.726054 ip-10-0-129-16 kubenswrapper[2565]: W0918 13:58:53.724661    2565 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 18 13:58:53.726054 ip-10-0-129-16 kubenswrapper[2565]: W0918 13:58:53.724663    2565 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 18 13:58:53.726054 ip-10-0-129-16 kubenswrapper[2565]: W0918 13:58:53.724666    2565 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 18 13:58:53.726054 ip-10-0-129-16 kubenswrapper[2565]: W0918 13:58:53.724668    2565 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 18 13:58:53.726054 ip-10-0-129-16 kubenswrapper[2565]: W0918 13:58:53.724671    2565 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 18 13:58:53.726054 ip-10-0-129-16 kubenswrapper[2565]: W0918 13:58:53.724674    2565 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 18 13:58:53.726054 ip-10-0-129-16 kubenswrapper[2565]: W0918 13:58:53.724676    2565 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 18 13:58:53.726054 ip-10-0-129-16 kubenswrapper[2565]: W0918 13:58:53.724679    2565 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 18 13:58:53.726054 ip-10-0-129-16 kubenswrapper[2565]: W0918 13:58:53.724682    2565 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 18 13:58:53.726054 ip-10-0-129-16 kubenswrapper[2565]: W0918 13:58:53.724684    2565 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 18 13:58:53.726054 ip-10-0-129-16 kubenswrapper[2565]: W0918 13:58:53.724687    2565 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 18 13:58:53.726054 ip-10-0-129-16 kubenswrapper[2565]: W0918 13:58:53.724690    2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 18 13:58:53.726054 ip-10-0-129-16 kubenswrapper[2565]: W0918 13:58:53.724693    2565 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 18 13:58:53.726054 ip-10-0-129-16 kubenswrapper[2565]: W0918 13:58:53.724695    2565 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 18 13:58:53.726054 ip-10-0-129-16 kubenswrapper[2565]: W0918 13:58:53.724699    2565 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 18 13:58:53.726054 ip-10-0-129-16 kubenswrapper[2565]: W0918 13:58:53.724701    2565 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 18 13:58:53.726054 ip-10-0-129-16 kubenswrapper[2565]: W0918 13:58:53.724704    2565 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 18 13:58:53.726553 ip-10-0-129-16 kubenswrapper[2565]: W0918 13:58:53.724707    2565 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 18 13:58:53.726553 ip-10-0-129-16 kubenswrapper[2565]: W0918 13:58:53.724709    2565 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 18 13:58:53.726553 ip-10-0-129-16 kubenswrapper[2565]: W0918 13:58:53.724712    2565 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 18 13:58:53.726553 ip-10-0-129-16 kubenswrapper[2565]: W0918 13:58:53.724714    2565 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 18 13:58:53.726553 ip-10-0-129-16 kubenswrapper[2565]: W0918 13:58:53.724717    2565 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 18 13:58:53.726553 ip-10-0-129-16 kubenswrapper[2565]: W0918 13:58:53.724720    2565 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 18 13:58:53.726553 ip-10-0-129-16 kubenswrapper[2565]: W0918 13:58:53.724722    2565 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 18 13:58:53.726553 ip-10-0-129-16 kubenswrapper[2565]: W0918 13:58:53.724726    2565 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 18 13:58:53.726553 ip-10-0-129-16 kubenswrapper[2565]: W0918 13:58:53.724729    2565 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 18 13:58:53.726553 ip-10-0-129-16 kubenswrapper[2565]: W0918 13:58:53.724731    2565 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 18 13:58:53.726553 ip-10-0-129-16 kubenswrapper[2565]: W0918 13:58:53.724734    2565 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 18 13:58:53.726553 ip-10-0-129-16 kubenswrapper[2565]: W0918 13:58:53.724737    2565 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 18 13:58:53.726553 ip-10-0-129-16 kubenswrapper[2565]: W0918 13:58:53.724739    2565 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 18 13:58:53.726553 ip-10-0-129-16 kubenswrapper[2565]: W0918 13:58:53.724742    2565 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 18 13:58:53.726553 ip-10-0-129-16 kubenswrapper[2565]: W0918 13:58:53.724744    2565 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 18 13:58:53.726553 ip-10-0-129-16 kubenswrapper[2565]: W0918 13:58:53.724747    2565 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 18 13:58:53.726553 ip-10-0-129-16 kubenswrapper[2565]: W0918 13:58:53.724750    2565 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 18 13:58:53.726553 ip-10-0-129-16 kubenswrapper[2565]: W0918 13:58:53.724752    2565 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 18 13:58:53.726553 ip-10-0-129-16 kubenswrapper[2565]: W0918 13:58:53.724755    2565 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 18 13:58:53.726553 ip-10-0-129-16 kubenswrapper[2565]: W0918 13:58:53.726142    2565 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 18 13:58:53.727048 ip-10-0-129-16 kubenswrapper[2565]: W0918 13:58:53.726149    2565 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 18 13:58:53.727048 ip-10-0-129-16 kubenswrapper[2565]: W0918 13:58:53.726153    2565 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 18 13:58:53.727048 ip-10-0-129-16 kubenswrapper[2565]: W0918 13:58:53.726156    2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 18 13:58:53.727048 ip-10-0-129-16 kubenswrapper[2565]: W0918 13:58:53.726159    2565 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 18 13:58:53.727048 ip-10-0-129-16 kubenswrapper[2565]: W0918 13:58:53.726163    2565 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 18 13:58:53.727048 ip-10-0-129-16 kubenswrapper[2565]: W0918 13:58:53.726166    2565 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 18 13:58:53.727048 ip-10-0-129-16 kubenswrapper[2565]: W0918 13:58:53.726169    2565 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 18 13:58:53.727048 ip-10-0-129-16 kubenswrapper[2565]: W0918 13:58:53.726172    2565 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 18 13:58:53.727048 ip-10-0-129-16 kubenswrapper[2565]: W0918 13:58:53.726175    2565 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 18 13:58:53.727048 ip-10-0-129-16 kubenswrapper[2565]: W0918 13:58:53.726177    2565 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 18 13:58:53.727048 ip-10-0-129-16 kubenswrapper[2565]: W0918 13:58:53.726180    2565 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 18 13:58:53.727048 ip-10-0-129-16 kubenswrapper[2565]: W0918 13:58:53.726182    2565 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 18 13:58:53.727048 ip-10-0-129-16 kubenswrapper[2565]: W0918 13:58:53.726185    2565 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 18 13:58:53.727048 ip-10-0-129-16 kubenswrapper[2565]: W0918 13:58:53.726187    2565 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 18 13:58:53.727048 ip-10-0-129-16 kubenswrapper[2565]: W0918 13:58:53.726190    2565 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 18 13:58:53.727048 ip-10-0-129-16 kubenswrapper[2565]: W0918 13:58:53.726195    2565 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 18 13:58:53.727048 ip-10-0-129-16 kubenswrapper[2565]: W0918 13:58:53.726198    2565 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 18 13:58:53.727048 ip-10-0-129-16 kubenswrapper[2565]: W0918 13:58:53.726201    2565 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 18 13:58:53.727048 ip-10-0-129-16 kubenswrapper[2565]: W0918 13:58:53.726204    2565 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 18 13:58:53.727048 ip-10-0-129-16 kubenswrapper[2565]: W0918 13:58:53.726208    2565 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 18 13:58:53.727575 ip-10-0-129-16 kubenswrapper[2565]: W0918 13:58:53.726211    2565 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 18 13:58:53.727575 ip-10-0-129-16 kubenswrapper[2565]: W0918 13:58:53.726215    2565 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 18 13:58:53.727575 ip-10-0-129-16 kubenswrapper[2565]: W0918 13:58:53.726218    2565 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 18 13:58:53.727575 ip-10-0-129-16 kubenswrapper[2565]: W0918 13:58:53.726222    2565 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 18 13:58:53.727575 ip-10-0-129-16 kubenswrapper[2565]: W0918 13:58:53.726225    2565 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 18 13:58:53.727575 ip-10-0-129-16 kubenswrapper[2565]: W0918 13:58:53.726228    2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 18 13:58:53.727575 ip-10-0-129-16 kubenswrapper[2565]: W0918 13:58:53.726231    2565 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 18 13:58:53.727575 ip-10-0-129-16 kubenswrapper[2565]: W0918 13:58:53.726234    2565 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 18 13:58:53.727575 ip-10-0-129-16 kubenswrapper[2565]: W0918 13:58:53.726236    2565 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 18 13:58:53.727575 ip-10-0-129-16 kubenswrapper[2565]: W0918 13:58:53.726239    2565 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 18 13:58:53.727575 ip-10-0-129-16 kubenswrapper[2565]: W0918 13:58:53.726241    2565 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 18 13:58:53.727575 ip-10-0-129-16 kubenswrapper[2565]: W0918 13:58:53.726244    2565 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 18 13:58:53.727575 ip-10-0-129-16 kubenswrapper[2565]: W0918 13:58:53.726246    2565 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 18 13:58:53.727575 ip-10-0-129-16 kubenswrapper[2565]: W0918 13:58:53.726249    2565 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 18 13:58:53.727575 ip-10-0-129-16 kubenswrapper[2565]: W0918 13:58:53.726251    2565 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 18 13:58:53.727575 ip-10-0-129-16 kubenswrapper[2565]: W0918 13:58:53.726254    2565 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 18 13:58:53.727575 ip-10-0-129-16 kubenswrapper[2565]: W0918 13:58:53.726256    2565 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 18 13:58:53.727575 ip-10-0-129-16 kubenswrapper[2565]: W0918 13:58:53.726259    2565 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 18 13:58:53.727575 ip-10-0-129-16 kubenswrapper[2565]: W0918 13:58:53.726262    2565 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 18 13:58:53.727575 ip-10-0-129-16 kubenswrapper[2565]: W0918 13:58:53.726265    2565 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 18 13:58:53.728089 ip-10-0-129-16 kubenswrapper[2565]: W0918 13:58:53.726267    2565 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 18 13:58:53.728089 ip-10-0-129-16 kubenswrapper[2565]: W0918 13:58:53.726270    2565 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 18 13:58:53.728089 ip-10-0-129-16 kubenswrapper[2565]: W0918 13:58:53.726272    2565 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 18 13:58:53.728089 ip-10-0-129-16 kubenswrapper[2565]: W0918 13:58:53.726275    2565 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 18 13:58:53.728089 ip-10-0-129-16 kubenswrapper[2565]: W0918 13:58:53.726278    2565 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 18 13:58:53.728089 ip-10-0-129-16 kubenswrapper[2565]: W0918 13:58:53.726281    2565 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 18 13:58:53.728089 ip-10-0-129-16 kubenswrapper[2565]: W0918 13:58:53.726283    2565 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 18 13:58:53.728089 ip-10-0-129-16 kubenswrapper[2565]: W0918 13:58:53.726286    2565 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 18 13:58:53.728089 ip-10-0-129-16 kubenswrapper[2565]: W0918 13:58:53.726289    2565 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 18 13:58:53.728089 ip-10-0-129-16 kubenswrapper[2565]: W0918 13:58:53.726291    2565 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 18 13:58:53.728089 ip-10-0-129-16 kubenswrapper[2565]: W0918 13:58:53.726294    2565 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 18 13:58:53.728089 ip-10-0-129-16 kubenswrapper[2565]: W0918 13:58:53.726298    2565 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 18 13:58:53.728089 ip-10-0-129-16 kubenswrapper[2565]: W0918 13:58:53.726301    2565 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 18 13:58:53.728089 ip-10-0-129-16 kubenswrapper[2565]: W0918 13:58:53.726303    2565 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 18 13:58:53.728089 ip-10-0-129-16 kubenswrapper[2565]: W0918 13:58:53.726306    2565 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 18 13:58:53.728089 ip-10-0-129-16 kubenswrapper[2565]: W0918 13:58:53.726309    2565 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 18 13:58:53.728089 ip-10-0-129-16 kubenswrapper[2565]: W0918 13:58:53.726311    2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 18 13:58:53.728089 ip-10-0-129-16 kubenswrapper[2565]: W0918 13:58:53.726314    2565 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 18 13:58:53.728089 ip-10-0-129-16 kubenswrapper[2565]: W0918 13:58:53.726318    2565 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 18 13:58:53.728569 ip-10-0-129-16 kubenswrapper[2565]: W0918 13:58:53.726322    2565 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 18 13:58:53.728569 ip-10-0-129-16 kubenswrapper[2565]: W0918 13:58:53.726325    2565 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 18 13:58:53.728569 ip-10-0-129-16 kubenswrapper[2565]: W0918 13:58:53.726327    2565 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 18 13:58:53.728569 ip-10-0-129-16 kubenswrapper[2565]: W0918 13:58:53.726330    2565 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 18 13:58:53.728569 ip-10-0-129-16 kubenswrapper[2565]: W0918 13:58:53.726333    2565 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 18 13:58:53.728569 ip-10-0-129-16 kubenswrapper[2565]: W0918 13:58:53.726335    2565 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 18 13:58:53.728569 ip-10-0-129-16 kubenswrapper[2565]: W0918 13:58:53.726338    2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 18 13:58:53.728569 ip-10-0-129-16 kubenswrapper[2565]: W0918 13:58:53.726341    2565 feature_gate.go:328] unrecognized feature gate: Example
Sep 18 13:58:53.728569 ip-10-0-129-16 kubenswrapper[2565]: W0918 13:58:53.726343    2565 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 18 13:58:53.728569 ip-10-0-129-16 kubenswrapper[2565]: W0918 13:58:53.726346    2565 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 18 13:58:53.728569 ip-10-0-129-16 kubenswrapper[2565]: W0918 13:58:53.726349    2565 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 18 13:58:53.728569 ip-10-0-129-16 kubenswrapper[2565]: W0918 13:58:53.726359    2565 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 18 13:58:53.728569 ip-10-0-129-16 kubenswrapper[2565]: W0918 13:58:53.726362    2565 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 18 13:58:53.728569 ip-10-0-129-16 kubenswrapper[2565]: W0918 13:58:53.726364    2565 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 18 13:58:53.728569 ip-10-0-129-16 kubenswrapper[2565]: W0918 13:58:53.726366    2565 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 18 13:58:53.728569 ip-10-0-129-16 kubenswrapper[2565]: W0918 13:58:53.726369    2565 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 18 13:58:53.728569 ip-10-0-129-16 kubenswrapper[2565]: W0918 13:58:53.726372    2565 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 18 13:58:53.728569 ip-10-0-129-16 kubenswrapper[2565]: W0918 13:58:53.726375    2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 18 13:58:53.728569 ip-10-0-129-16 kubenswrapper[2565]: W0918 13:58:53.726378    2565 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 18 13:58:53.729073 ip-10-0-129-16 kubenswrapper[2565]: W0918 13:58:53.726381    2565 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 18 13:58:53.729073 ip-10-0-129-16 kubenswrapper[2565]: W0918 13:58:53.726383    2565 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 18 13:58:53.729073 ip-10-0-129-16 kubenswrapper[2565]: W0918 13:58:53.726386    2565 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 18 13:58:53.729073 ip-10-0-129-16 kubenswrapper[2565]: W0918 13:58:53.726388    2565 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 18 13:58:53.729073 ip-10-0-129-16 kubenswrapper[2565]: W0918 13:58:53.726391    2565 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 18 13:58:53.729073 ip-10-0-129-16 kubenswrapper[2565]: W0918 13:58:53.726398    2565 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 18 13:58:53.729073 ip-10-0-129-16 kubenswrapper[2565]: W0918 13:58:53.726401    2565 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 18 13:58:53.729073 ip-10-0-129-16 kubenswrapper[2565]: W0918 13:58:53.726404    2565 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 18 13:58:53.729073 ip-10-0-129-16 kubenswrapper[2565]: W0918 13:58:53.726406    2565 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 18 13:58:53.729073 ip-10-0-129-16 kubenswrapper[2565]: W0918 13:58:53.726409    2565 feature_gate.go:328] unrecognized feature gate: Example2
Sep 18 13:58:53.729073 ip-10-0-129-16 kubenswrapper[2565]: W0918 13:58:53.726412    2565 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 18 13:58:53.729073 ip-10-0-129-16 kubenswrapper[2565]: W0918 13:58:53.726414    2565 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 18 13:58:53.729073 ip-10-0-129-16 kubenswrapper[2565]: W0918 13:58:53.726417    2565 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 18 13:58:53.729073 ip-10-0-129-16 kubenswrapper[2565]: W0918 13:58:53.726420    2565 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 18 13:58:53.729073 ip-10-0-129-16 kubenswrapper[2565]: W0918 13:58:53.726422    2565 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 18 13:58:53.729073 ip-10-0-129-16 kubenswrapper[2565]: W0918 13:58:53.726425    2565 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 18 13:58:53.729073 ip-10-0-129-16 kubenswrapper[2565]: W0918 13:58:53.726427    2565 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 18 13:58:53.729073 ip-10-0-129-16 kubenswrapper[2565]: W0918 13:58:53.726430    2565 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 18 13:58:53.729073 ip-10-0-129-16 kubenswrapper[2565]: W0918 13:58:53.726433    2565 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 18 13:58:53.729073 ip-10-0-129-16 kubenswrapper[2565]: W0918 13:58:53.726435    2565 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 18 13:58:53.729578 ip-10-0-129-16 kubenswrapper[2565]: W0918 13:58:53.726843    2565 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 18 13:58:53.729578 ip-10-0-129-16 kubenswrapper[2565]: W0918 13:58:53.726851    2565 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 18 13:58:53.729578 ip-10-0-129-16 kubenswrapper[2565]: W0918 13:58:53.726856    2565 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 18 13:58:53.729578 ip-10-0-129-16 kubenswrapper[2565]: W0918 13:58:53.726860    2565 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 18 13:58:53.729578 ip-10-0-129-16 kubenswrapper[2565]: W0918 13:58:53.726870    2565 feature_gate.go:328] unrecognized feature gate: Example
Sep 18 13:58:53.729578 ip-10-0-129-16 kubenswrapper[2565]: W0918 13:58:53.726873    2565 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 18 13:58:53.729578 ip-10-0-129-16 kubenswrapper[2565]: W0918 13:58:53.726876    2565 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 18 13:58:53.729578 ip-10-0-129-16 kubenswrapper[2565]: W0918 13:58:53.726878    2565 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 18 13:58:53.729578 ip-10-0-129-16 kubenswrapper[2565]: W0918 13:58:53.726881    2565 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 18 13:58:53.729578 ip-10-0-129-16 kubenswrapper[2565]: W0918 13:58:53.726883    2565 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 18 13:58:53.729578 ip-10-0-129-16 kubenswrapper[2565]: W0918 13:58:53.726886    2565 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 18 13:58:53.729578 ip-10-0-129-16 kubenswrapper[2565]: W0918 13:58:53.726889    2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 18 13:58:53.729578 ip-10-0-129-16 kubenswrapper[2565]: W0918 13:58:53.726892    2565 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 18 13:58:53.729578 ip-10-0-129-16 kubenswrapper[2565]: W0918 13:58:53.726895    2565 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 18 13:58:53.729578 ip-10-0-129-16 kubenswrapper[2565]: W0918 13:58:53.726897    2565 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 18 13:58:53.729578 ip-10-0-129-16 kubenswrapper[2565]: W0918 13:58:53.726900    2565 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 18 13:58:53.729578 ip-10-0-129-16 kubenswrapper[2565]: W0918 13:58:53.726902    2565 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 18 13:58:53.729578 ip-10-0-129-16 kubenswrapper[2565]: W0918 13:58:53.726906    2565 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 18 13:58:53.729578 ip-10-0-129-16 kubenswrapper[2565]: W0918 13:58:53.726908    2565 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 18 13:58:53.729578 ip-10-0-129-16 kubenswrapper[2565]: W0918 13:58:53.726911    2565 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 18 13:58:53.730090 ip-10-0-129-16 kubenswrapper[2565]: W0918 13:58:53.726913    2565 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 18 13:58:53.730090 ip-10-0-129-16 kubenswrapper[2565]: W0918 13:58:53.726916    2565 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 18 13:58:53.730090 ip-10-0-129-16 kubenswrapper[2565]: W0918 13:58:53.726919    2565 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 18 13:58:53.730090 ip-10-0-129-16 kubenswrapper[2565]: W0918 13:58:53.726921    2565 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 18 13:58:53.730090 ip-10-0-129-16 kubenswrapper[2565]: W0918 13:58:53.726923    2565 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 18 13:58:53.730090 ip-10-0-129-16 kubenswrapper[2565]: W0918 13:58:53.726926    2565 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 18 13:58:53.730090 ip-10-0-129-16 kubenswrapper[2565]: W0918 13:58:53.726929    2565 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 18 13:58:53.730090 ip-10-0-129-16 kubenswrapper[2565]: W0918 13:58:53.726931    2565 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 18 13:58:53.730090 ip-10-0-129-16 kubenswrapper[2565]: W0918 13:58:53.726933    2565 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 18 13:58:53.730090 ip-10-0-129-16 kubenswrapper[2565]: W0918 13:58:53.726936    2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 18 13:58:53.730090 ip-10-0-129-16 kubenswrapper[2565]: W0918 13:58:53.726938    2565 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 18 13:58:53.730090 ip-10-0-129-16 kubenswrapper[2565]: W0918 13:58:53.726941    2565 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 18 13:58:53.730090 ip-10-0-129-16 kubenswrapper[2565]: W0918 13:58:53.726944    2565 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 18 13:58:53.730090 ip-10-0-129-16 kubenswrapper[2565]: W0918 13:58:53.726946    2565 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 18 13:58:53.730090 ip-10-0-129-16 kubenswrapper[2565]: W0918 13:58:53.726949    2565 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 18 13:58:53.730090 ip-10-0-129-16 kubenswrapper[2565]: W0918 13:58:53.726951    2565 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 18 13:58:53.730090 ip-10-0-129-16 kubenswrapper[2565]: W0918 13:58:53.726954    2565 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 18 13:58:53.730090 ip-10-0-129-16 kubenswrapper[2565]: W0918 13:58:53.726962    2565 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 18 13:58:53.730090 ip-10-0-129-16 kubenswrapper[2565]: W0918 13:58:53.726964    2565 feature_gate.go:328] unrecognized feature gate: Example2
Sep 18 13:58:53.730090 ip-10-0-129-16 kubenswrapper[2565]: W0918 13:58:53.726967    2565 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 18 13:58:53.730622 ip-10-0-129-16 kubenswrapper[2565]: W0918 13:58:53.726969    2565 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 18 13:58:53.730622 ip-10-0-129-16 kubenswrapper[2565]: W0918 13:58:53.726972    2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 18 13:58:53.730622 ip-10-0-129-16 kubenswrapper[2565]: W0918 13:58:53.726974    2565 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 18 13:58:53.730622 ip-10-0-129-16 kubenswrapper[2565]: W0918 13:58:53.726977    2565 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 18 13:58:53.730622 ip-10-0-129-16 kubenswrapper[2565]: W0918 13:58:53.726980    2565 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 18 13:58:53.730622 ip-10-0-129-16 kubenswrapper[2565]: W0918 13:58:53.726982    2565 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 18 13:58:53.730622 ip-10-0-129-16 kubenswrapper[2565]: W0918 13:58:53.726984    2565 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 18 13:58:53.730622 ip-10-0-129-16 kubenswrapper[2565]: W0918 13:58:53.726987    2565 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 18 13:58:53.730622 ip-10-0-129-16 kubenswrapper[2565]: W0918 13:58:53.726990    2565 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 18 13:58:53.730622 ip-10-0-129-16 kubenswrapper[2565]: W0918 13:58:53.726996    2565 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 18 13:58:53.730622 ip-10-0-129-16 kubenswrapper[2565]: W0918 13:58:53.726999    2565 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 18 13:58:53.730622 ip-10-0-129-16 kubenswrapper[2565]: W0918 13:58:53.727001    2565 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 18 13:58:53.730622 ip-10-0-129-16 kubenswrapper[2565]: W0918 13:58:53.727004    2565 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 18 13:58:53.730622 ip-10-0-129-16 kubenswrapper[2565]: W0918 13:58:53.727006    2565 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 18 13:58:53.730622 ip-10-0-129-16 kubenswrapper[2565]: W0918 13:58:53.727008    2565 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 18 13:58:53.730622 ip-10-0-129-16 kubenswrapper[2565]: W0918 13:58:53.727011    2565 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 18 13:58:53.730622 ip-10-0-129-16 kubenswrapper[2565]: W0918 13:58:53.727014    2565 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 18 13:58:53.730622 ip-10-0-129-16 kubenswrapper[2565]: W0918 13:58:53.727016    2565 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 18 13:58:53.730622 ip-10-0-129-16 kubenswrapper[2565]: W0918 13:58:53.727019    2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 18 13:58:53.731207 ip-10-0-129-16 kubenswrapper[2565]: W0918 13:58:53.727021    2565 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 18 13:58:53.731207 ip-10-0-129-16 kubenswrapper[2565]: W0918 13:58:53.727024    2565 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 18 13:58:53.731207 ip-10-0-129-16 kubenswrapper[2565]: W0918 13:58:53.727027    2565 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 18 13:58:53.731207 ip-10-0-129-16 kubenswrapper[2565]: W0918 13:58:53.727043    2565 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 18 13:58:53.731207 ip-10-0-129-16 kubenswrapper[2565]: W0918 13:58:53.727046    2565 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 18 13:58:53.731207 ip-10-0-129-16 kubenswrapper[2565]: W0918 13:58:53.727049    2565 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 18 13:58:53.731207 ip-10-0-129-16 kubenswrapper[2565]: W0918 13:58:53.727052    2565 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 18 13:58:53.731207 ip-10-0-129-16 kubenswrapper[2565]: W0918 13:58:53.727055    2565 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 18 13:58:53.731207 ip-10-0-129-16 kubenswrapper[2565]: W0918 13:58:53.727058    2565 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 18 13:58:53.731207 ip-10-0-129-16 kubenswrapper[2565]: W0918 13:58:53.727060    2565 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 18 13:58:53.731207 ip-10-0-129-16 kubenswrapper[2565]: W0918 13:58:53.727063    2565 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 18 13:58:53.731207 ip-10-0-129-16 kubenswrapper[2565]: W0918 13:58:53.727071    2565 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 18 13:58:53.731207 ip-10-0-129-16 kubenswrapper[2565]: W0918 13:58:53.727074    2565 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 18 13:58:53.731207 ip-10-0-129-16 kubenswrapper[2565]: W0918 13:58:53.727077    2565 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 18 13:58:53.731207 ip-10-0-129-16 kubenswrapper[2565]: W0918 13:58:53.727080    2565 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 18 13:58:53.731207 ip-10-0-129-16 kubenswrapper[2565]: W0918 13:58:53.727082    2565 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 18 13:58:53.731207 ip-10-0-129-16 kubenswrapper[2565]: W0918 13:58:53.727085    2565 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 18 13:58:53.731207 ip-10-0-129-16 kubenswrapper[2565]: W0918 13:58:53.727087    2565 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 18 13:58:53.731207 ip-10-0-129-16 kubenswrapper[2565]: W0918 13:58:53.727090    2565 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 18 13:58:53.731207 ip-10-0-129-16 kubenswrapper[2565]: W0918 13:58:53.727092    2565 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 18 13:58:53.731692 ip-10-0-129-16 kubenswrapper[2565]: W0918 13:58:53.727096    2565 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 18 13:58:53.731692 ip-10-0-129-16 kubenswrapper[2565]: W0918 13:58:53.727101    2565 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 18 13:58:53.731692 ip-10-0-129-16 kubenswrapper[2565]: W0918 13:58:53.727104    2565 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 18 13:58:53.731692 ip-10-0-129-16 kubenswrapper[2565]: W0918 13:58:53.727107    2565 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 18 13:58:53.731692 ip-10-0-129-16 kubenswrapper[2565]: W0918 13:58:53.727110    2565 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 18 13:58:53.731692 ip-10-0-129-16 kubenswrapper[2565]: W0918 13:58:53.727113    2565 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 18 13:58:53.731692 ip-10-0-129-16 kubenswrapper[2565]: W0918 13:58:53.727117    2565 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 18 13:58:53.731692 ip-10-0-129-16 kubenswrapper[2565]: W0918 13:58:53.727120    2565 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 18 13:58:53.731692 ip-10-0-129-16 kubenswrapper[2565]: W0918 13:58:53.727123    2565 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 18 13:58:53.731692 ip-10-0-129-16 kubenswrapper[2565]: W0918 13:58:53.727125    2565 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 18 13:58:53.731692 ip-10-0-129-16 kubenswrapper[2565]: W0918 13:58:53.727128    2565 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 18 13:58:53.731692 ip-10-0-129-16 kubenswrapper[2565]: W0918 13:58:53.727131    2565 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 18 13:58:53.731692 ip-10-0-129-16 kubenswrapper[2565]: W0918 13:58:53.727133    2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 18 13:58:53.731692 ip-10-0-129-16 kubenswrapper[2565]: W0918 13:58:53.727136    2565 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 18 13:58:53.731692 ip-10-0-129-16 kubenswrapper[2565]: W0918 13:58:53.727139    2565 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 18 13:58:53.731692 ip-10-0-129-16 kubenswrapper[2565]: W0918 13:58:53.727141    2565 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 18 13:58:53.731692 ip-10-0-129-16 kubenswrapper[2565]: W0918 13:58:53.727144    2565 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 18 13:58:53.731692 ip-10-0-129-16 kubenswrapper[2565]: W0918 13:58:53.727147    2565 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 18 13:58:53.731692 ip-10-0-129-16 kubenswrapper[2565]: W0918 13:58:53.727149    2565 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 18 13:58:53.732170 ip-10-0-129-16 kubenswrapper[2565]: W0918 13:58:53.727151    2565 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 18 13:58:53.732170 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:58:53.727221    2565 flags.go:64] FLAG: --address="0.0.0.0"
Sep 18 13:58:53.732170 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:58:53.727253    2565 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]"
Sep 18 13:58:53.732170 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:58:53.727259    2565 flags.go:64] FLAG: --anonymous-auth="true"
Sep 18 13:58:53.732170 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:58:53.727264    2565 flags.go:64] FLAG: --application-metrics-count-limit="100"
Sep 18 13:58:53.732170 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:58:53.727269    2565 flags.go:64] FLAG: --authentication-token-webhook="false"
Sep 18 13:58:53.732170 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:58:53.727274    2565 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s"
Sep 18 13:58:53.732170 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:58:53.727278    2565 flags.go:64] FLAG: --authorization-mode="AlwaysAllow"
Sep 18 13:58:53.732170 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:58:53.727282    2565 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s"
Sep 18 13:58:53.732170 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:58:53.727286    2565 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s"
Sep 18 13:58:53.732170 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:58:53.727288    2565 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id"
Sep 18 13:58:53.732170 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:58:53.727292    2565 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig"
Sep 18 13:58:53.732170 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:58:53.727295    2565 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki"
Sep 18 13:58:53.732170 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:58:53.727299    2565 flags.go:64] FLAG: --cgroup-driver="cgroupfs"
Sep 18 13:58:53.732170 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:58:53.727302    2565 flags.go:64] FLAG: --cgroup-root=""
Sep 18 13:58:53.732170 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:58:53.727305    2565 flags.go:64] FLAG: --cgroups-per-qos="true"
Sep 18 13:58:53.732170 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:58:53.727308    2565 flags.go:64] FLAG: --client-ca-file=""
Sep 18 13:58:53.732170 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:58:53.727311    2565 flags.go:64] FLAG: --cloud-provider="external"
Sep 18 13:58:53.732170 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:58:53.727314    2565 flags.go:64] FLAG: --cluster-dns="[]"
Sep 18 13:58:53.732170 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:58:53.727318    2565 flags.go:64] FLAG: --cluster-domain=""
Sep 18 13:58:53.732170 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:58:53.727320    2565 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf"
Sep 18 13:58:53.732170 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:58:53.727323    2565 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d"
Sep 18 13:58:53.732170 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:58:53.727326    2565 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json"
Sep 18 13:58:53.732170 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:58:53.727330    2565 flags.go:64] FLAG: --container-log-max-files="5"
Sep 18 13:58:53.732754 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:58:53.727333    2565 flags.go:64] FLAG: --container-log-max-size="10Mi"
Sep 18 13:58:53.732754 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:58:53.727336    2565 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock"
Sep 18 13:58:53.732754 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:58:53.727339    2565 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock"
Sep 18 13:58:53.732754 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:58:53.727343    2565 flags.go:64] FLAG: --containerd-namespace="k8s.io"
Sep 18 13:58:53.732754 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:58:53.727346    2565 flags.go:64] FLAG: --contention-profiling="false"
Sep 18 13:58:53.732754 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:58:53.727349    2565 flags.go:64] FLAG: --cpu-cfs-quota="true"
Sep 18 13:58:53.732754 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:58:53.727352    2565 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms"
Sep 18 13:58:53.732754 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:58:53.727356    2565 flags.go:64] FLAG: --cpu-manager-policy="none"
Sep 18 13:58:53.732754 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:58:53.727359    2565 flags.go:64] FLAG: --cpu-manager-policy-options=""
Sep 18 13:58:53.732754 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:58:53.727363    2565 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s"
Sep 18 13:58:53.732754 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:58:53.727365    2565 flags.go:64] FLAG: --enable-controller-attach-detach="true"
Sep 18 13:58:53.732754 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:58:53.727368    2565 flags.go:64] FLAG: --enable-debugging-handlers="true"
Sep 18 13:58:53.732754 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:58:53.727371    2565 flags.go:64] FLAG: --enable-load-reader="false"
Sep 18 13:58:53.732754 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:58:53.727374    2565 flags.go:64] FLAG: --enable-server="true"
Sep 18 13:58:53.732754 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:58:53.727380    2565 flags.go:64] FLAG: --enforce-node-allocatable="[pods]"
Sep 18 13:58:53.732754 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:58:53.727386    2565 flags.go:64] FLAG: --event-burst="100"
Sep 18 13:58:53.732754 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:58:53.727389    2565 flags.go:64] FLAG: --event-qps="50"
Sep 18 13:58:53.732754 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:58:53.727392    2565 flags.go:64] FLAG: --event-storage-age-limit="default=0"
Sep 18 13:58:53.732754 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:58:53.727395    2565 flags.go:64] FLAG: --event-storage-event-limit="default=0"
Sep 18 13:58:53.732754 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:58:53.727398    2565 flags.go:64] FLAG: --eviction-hard=""
Sep 18 13:58:53.732754 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:58:53.727401    2565 flags.go:64] FLAG: --eviction-max-pod-grace-period="0"
Sep 18 13:58:53.732754 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:58:53.727404    2565 flags.go:64] FLAG: --eviction-minimum-reclaim=""
Sep 18 13:58:53.732754 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:58:53.727407    2565 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s"
Sep 18 13:58:53.732754 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:58:53.727412    2565 flags.go:64] FLAG: --eviction-soft=""
Sep 18 13:58:53.732754 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:58:53.727415    2565 flags.go:64] FLAG: --eviction-soft-grace-period=""
Sep 18 13:58:53.733368 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:58:53.727418    2565 flags.go:64] FLAG: --exit-on-lock-contention="false"
Sep 18 13:58:53.733368 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:58:53.727421    2565 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false"
Sep 18 13:58:53.733368 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:58:53.727424    2565 flags.go:64] FLAG: --experimental-mounter-path=""
Sep 18 13:58:53.733368 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:58:53.727427    2565 flags.go:64] FLAG: --fail-cgroupv1="false"
Sep 18 13:58:53.733368 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:58:53.727430    2565 flags.go:64] FLAG: --fail-swap-on="true"
Sep 18 13:58:53.733368 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:58:53.727433    2565 flags.go:64] FLAG: --feature-gates=""
Sep 18 13:58:53.733368 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:58:53.727437    2565 flags.go:64] FLAG: --file-check-frequency="20s"
Sep 18 13:58:53.733368 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:58:53.727440    2565 flags.go:64] FLAG: --global-housekeeping-interval="1m0s"
Sep 18 13:58:53.733368 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:58:53.727443    2565 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge"
Sep 18 13:58:53.733368 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:58:53.727446    2565 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1"
Sep 18 13:58:53.733368 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:58:53.727449    2565 flags.go:64] FLAG: --healthz-port="10248"
Sep 18 13:58:53.733368 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:58:53.727452    2565 flags.go:64] FLAG: --help="false"
Sep 18 13:58:53.733368 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:58:53.727456    2565 flags.go:64] FLAG: --hostname-override="ip-10-0-129-16.ec2.internal"
Sep 18 13:58:53.733368 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:58:53.727459    2565 flags.go:64] FLAG: --housekeeping-interval="10s"
Sep 18 13:58:53.733368 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:58:53.727462    2565 flags.go:64] FLAG: --http-check-frequency="20s"
Sep 18 13:58:53.733368 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:58:53.727464    2565 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins"
Sep 18 13:58:53.733368 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:58:53.727468    2565 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml"
Sep 18 13:58:53.733368 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:58:53.727471    2565 flags.go:64] FLAG: --image-gc-high-threshold="85"
Sep 18 13:58:53.733368 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:58:53.727474    2565 flags.go:64] FLAG: --image-gc-low-threshold="80"
Sep 18 13:58:53.733368 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:58:53.727477    2565 flags.go:64] FLAG: --image-service-endpoint=""
Sep 18 13:58:53.733368 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:58:53.727480    2565 flags.go:64] FLAG: --kernel-memcg-notification="false"
Sep 18 13:58:53.733368 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:58:53.727484    2565 flags.go:64] FLAG: --kube-api-burst="100"
Sep 18 13:58:53.733368 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:58:53.727494    2565 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf"
Sep 18 13:58:53.734052 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:58:53.727497    2565 flags.go:64] FLAG: --kube-api-qps="50"
Sep 18 13:58:53.734052 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:58:53.727500    2565 flags.go:64] FLAG: --kube-reserved=""
Sep 18 13:58:53.734052 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:58:53.727503    2565 flags.go:64] FLAG: --kube-reserved-cgroup=""
Sep 18 13:58:53.734052 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:58:53.727506    2565 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig"
Sep 18 13:58:53.734052 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:58:53.727509    2565 flags.go:64] FLAG: --kubelet-cgroups=""
Sep 18 13:58:53.734052 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:58:53.727512    2565 flags.go:64] FLAG: --local-storage-capacity-isolation="true"
Sep 18 13:58:53.734052 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:58:53.727515    2565 flags.go:64] FLAG: --lock-file=""
Sep 18 13:58:53.734052 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:58:53.727519    2565 flags.go:64] FLAG: --log-cadvisor-usage="false"
Sep 18 13:58:53.734052 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:58:53.727522    2565 flags.go:64] FLAG: --log-flush-frequency="5s"
Sep 18 13:58:53.734052 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:58:53.727525    2565 flags.go:64] FLAG: --log-json-info-buffer-size="0"
Sep 18 13:58:53.734052 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:58:53.727530    2565 flags.go:64] FLAG: --log-json-split-stream="false"
Sep 18 13:58:53.734052 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:58:53.727533    2565 flags.go:64] FLAG: --log-text-info-buffer-size="0"
Sep 18 13:58:53.734052 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:58:53.727536    2565 flags.go:64] FLAG: --log-text-split-stream="false"
Sep 18 13:58:53.734052 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:58:53.727539    2565 flags.go:64] FLAG: --logging-format="text"
Sep 18 13:58:53.734052 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:58:53.727542    2565 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id"
Sep 18 13:58:53.734052 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:58:53.727545    2565 flags.go:64] FLAG: --make-iptables-util-chains="true"
Sep 18 13:58:53.734052 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:58:53.727548    2565 flags.go:64] FLAG: --manifest-url=""
Sep 18 13:58:53.734052 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:58:53.727551    2565 flags.go:64] FLAG: --manifest-url-header=""
Sep 18 13:58:53.734052 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:58:53.727555    2565 flags.go:64] FLAG: --max-housekeeping-interval="15s"
Sep 18 13:58:53.734052 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:58:53.727558    2565 flags.go:64] FLAG: --max-open-files="1000000"
Sep 18 13:58:53.734052 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:58:53.727562    2565 flags.go:64] FLAG: --max-pods="110"
Sep 18 13:58:53.734052 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:58:53.727565    2565 flags.go:64] FLAG: --maximum-dead-containers="-1"
Sep 18 13:58:53.734052 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:58:53.727568    2565 flags.go:64] FLAG: --maximum-dead-containers-per-container="1"
Sep 18 13:58:53.734052 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:58:53.727571    2565 flags.go:64] FLAG: --memory-manager-policy="None"
Sep 18 13:58:53.734052 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:58:53.727574    2565 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s"
Sep 18 13:58:53.734678 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:58:53.727577    2565 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s"
Sep 18 13:58:53.734678 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:58:53.727580    2565 flags.go:64] FLAG: --node-ip="0.0.0.0"
Sep 18 13:58:53.734678 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:58:53.727583    2565 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel"
Sep 18 13:58:53.734678 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:58:53.727590    2565 flags.go:64] FLAG: --node-status-max-images="50"
Sep 18 13:58:53.734678 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:58:53.727593    2565 flags.go:64] FLAG: --node-status-update-frequency="10s"
Sep 18 13:58:53.734678 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:58:53.727600    2565 flags.go:64] FLAG: --oom-score-adj="-999"
Sep 18 13:58:53.734678 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:58:53.727603    2565 flags.go:64] FLAG: --pod-cidr=""
Sep 18 13:58:53.734678 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:58:53.727612    2565 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:a6abe77af61c5f2e3985a83e7d43538aefc38b1b0444127da14ee0e8cf8f76fc"
Sep 18 13:58:53.734678 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:58:53.727617    2565 flags.go:64] FLAG: --pod-manifest-path=""
Sep 18 13:58:53.734678 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:58:53.727620    2565 flags.go:64] FLAG: --pod-max-pids="-1"
Sep 18 13:58:53.734678 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:58:53.727623    2565 flags.go:64] FLAG: --pods-per-core="0"
Sep 18 13:58:53.734678 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:58:53.727626    2565 flags.go:64] FLAG: --port="10250"
Sep 18 13:58:53.734678 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:58:53.727629    2565 flags.go:64] FLAG: --protect-kernel-defaults="false"
Sep 18 13:58:53.734678 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:58:53.727632    2565 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-0509b6a710b03dd89"
Sep 18 13:58:53.734678 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:58:53.727636    2565 flags.go:64] FLAG: --qos-reserved=""
Sep 18 13:58:53.734678 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:58:53.727640    2565 flags.go:64] FLAG: --read-only-port="10255"
Sep 18 13:58:53.734678 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:58:53.727643    2565 flags.go:64] FLAG: --register-node="true"
Sep 18 13:58:53.734678 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:58:53.727646    2565 flags.go:64] FLAG: --register-with-taints=""
Sep 18 13:58:53.734678 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:58:53.727650    2565 flags.go:64] FLAG: --registry-burst="10"
Sep 18 13:58:53.734678 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:58:53.727652    2565 flags.go:64] FLAG: --registry-qps="5"
Sep 18 13:58:53.734678 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:58:53.727655    2565 flags.go:64] FLAG: --reserved-cpus=""
Sep 18 13:58:53.734678 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:58:53.727658    2565 flags.go:64] FLAG: --reserved-memory=""
Sep 18 13:58:53.734678 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:58:53.727662    2565 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf"
Sep 18 13:58:53.734678 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:58:53.727665    2565 flags.go:64] FLAG: --root-dir="/var/lib/kubelet"
Sep 18 13:58:53.734678 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:58:53.727668    2565 flags.go:64] FLAG: --rotate-certificates="false"
Sep 18 13:58:53.735312 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:58:53.727671    2565 flags.go:64] FLAG: --rotate-server-certificates="false"
Sep 18 13:58:53.735312 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:58:53.727674    2565 flags.go:64] FLAG: --runonce="false"
Sep 18 13:58:53.735312 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:58:53.727677    2565 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service"
Sep 18 13:58:53.735312 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:58:53.727680    2565 flags.go:64] FLAG: --runtime-request-timeout="2m0s"
Sep 18 13:58:53.735312 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:58:53.727683    2565 flags.go:64] FLAG: --seccomp-default="false"
Sep 18 13:58:53.735312 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:58:53.727686    2565 flags.go:64] FLAG: --serialize-image-pulls="true"
Sep 18 13:58:53.735312 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:58:53.727689    2565 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s"
Sep 18 13:58:53.735312 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:58:53.727692    2565 flags.go:64] FLAG: --storage-driver-db="cadvisor"
Sep 18 13:58:53.735312 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:58:53.727695    2565 flags.go:64] FLAG: --storage-driver-host="localhost:8086"
Sep 18 13:58:53.735312 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:58:53.727698    2565 flags.go:64] FLAG: --storage-driver-password="root"
Sep 18 13:58:53.735312 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:58:53.727701    2565 flags.go:64] FLAG: --storage-driver-secure="false"
Sep 18 13:58:53.735312 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:58:53.727704    2565 flags.go:64] FLAG: --storage-driver-table="stats"
Sep 18 13:58:53.735312 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:58:53.727708    2565 flags.go:64] FLAG: --storage-driver-user="root"
Sep 18 13:58:53.735312 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:58:53.727711    2565 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s"
Sep 18 13:58:53.735312 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:58:53.727714    2565 flags.go:64] FLAG: --sync-frequency="1m0s"
Sep 18 13:58:53.735312 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:58:53.727722    2565 flags.go:64] FLAG: --system-cgroups=""
Sep 18 13:58:53.735312 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:58:53.727725    2565 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi"
Sep 18 13:58:53.735312 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:58:53.727730    2565 flags.go:64] FLAG: --system-reserved-cgroup=""
Sep 18 13:58:53.735312 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:58:53.727733    2565 flags.go:64] FLAG: --tls-cert-file=""
Sep 18 13:58:53.735312 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:58:53.727736    2565 flags.go:64] FLAG: --tls-cipher-suites="[]"
Sep 18 13:58:53.735312 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:58:53.727740    2565 flags.go:64] FLAG: --tls-min-version=""
Sep 18 13:58:53.735312 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:58:53.727742    2565 flags.go:64] FLAG: --tls-private-key-file=""
Sep 18 13:58:53.735312 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:58:53.727747    2565 flags.go:64] FLAG: --topology-manager-policy="none"
Sep 18 13:58:53.735312 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:58:53.727750    2565 flags.go:64] FLAG: --topology-manager-policy-options=""
Sep 18 13:58:53.735312 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:58:53.727753    2565 flags.go:64] FLAG: --topology-manager-scope="container"
Sep 18 13:58:53.735938 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:58:53.727755    2565 flags.go:64] FLAG: --v="2"
Sep 18 13:58:53.735938 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:58:53.727759    2565 flags.go:64] FLAG: --version="false"
Sep 18 13:58:53.735938 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:58:53.727763    2565 flags.go:64] FLAG: --vmodule=""
Sep 18 13:58:53.735938 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:58:53.727773    2565 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec"
Sep 18 13:58:53.735938 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:58:53.727776    2565 flags.go:64] FLAG: --volume-stats-agg-period="1m0s"
Sep 18 13:58:53.735938 ip-10-0-129-16 kubenswrapper[2565]: W0918 13:58:53.727900    2565 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 18 13:58:53.735938 ip-10-0-129-16 kubenswrapper[2565]: W0918 13:58:53.727903    2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 18 13:58:53.735938 ip-10-0-129-16 kubenswrapper[2565]: W0918 13:58:53.727906    2565 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 18 13:58:53.735938 ip-10-0-129-16 kubenswrapper[2565]: W0918 13:58:53.727910    2565 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 18 13:58:53.735938 ip-10-0-129-16 kubenswrapper[2565]: W0918 13:58:53.727913    2565 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 18 13:58:53.735938 ip-10-0-129-16 kubenswrapper[2565]: W0918 13:58:53.727915    2565 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 18 13:58:53.735938 ip-10-0-129-16 kubenswrapper[2565]: W0918 13:58:53.727919    2565 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 18 13:58:53.735938 ip-10-0-129-16 kubenswrapper[2565]: W0918 13:58:53.727921    2565 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 18 13:58:53.735938 ip-10-0-129-16 kubenswrapper[2565]: W0918 13:58:53.727924    2565 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 18 13:58:53.735938 ip-10-0-129-16 kubenswrapper[2565]: W0918 13:58:53.727927    2565 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 18 13:58:53.735938 ip-10-0-129-16 kubenswrapper[2565]: W0918 13:58:53.727930    2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 18 13:58:53.735938 ip-10-0-129-16 kubenswrapper[2565]: W0918 13:58:53.727932    2565 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 18 13:58:53.735938 ip-10-0-129-16 kubenswrapper[2565]: W0918 13:58:53.727935    2565 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 18 13:58:53.735938 ip-10-0-129-16 kubenswrapper[2565]: W0918 13:58:53.727938    2565 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 18 13:58:53.735938 ip-10-0-129-16 kubenswrapper[2565]: W0918 13:58:53.727942    2565 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 18 13:58:53.735938 ip-10-0-129-16 kubenswrapper[2565]: W0918 13:58:53.727944    2565 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 18 13:58:53.736466 ip-10-0-129-16 kubenswrapper[2565]: W0918 13:58:53.727947    2565 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 18 13:58:53.736466 ip-10-0-129-16 kubenswrapper[2565]: W0918 13:58:53.727949    2565 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 18 13:58:53.736466 ip-10-0-129-16 kubenswrapper[2565]: W0918 13:58:53.727952    2565 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 18 13:58:53.736466 ip-10-0-129-16 kubenswrapper[2565]: W0918 13:58:53.727955    2565 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 18 13:58:53.736466 ip-10-0-129-16 kubenswrapper[2565]: W0918 13:58:53.727958    2565 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 18 13:58:53.736466 ip-10-0-129-16 kubenswrapper[2565]: W0918 13:58:53.727960    2565 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 18 13:58:53.736466 ip-10-0-129-16 kubenswrapper[2565]: W0918 13:58:53.727963    2565 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 18 13:58:53.736466 ip-10-0-129-16 kubenswrapper[2565]: W0918 13:58:53.727965    2565 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 18 13:58:53.736466 ip-10-0-129-16 kubenswrapper[2565]: W0918 13:58:53.727970    2565 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 18 13:58:53.736466 ip-10-0-129-16 kubenswrapper[2565]: W0918 13:58:53.727974    2565 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 18 13:58:53.736466 ip-10-0-129-16 kubenswrapper[2565]: W0918 13:58:53.727978    2565 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 18 13:58:53.736466 ip-10-0-129-16 kubenswrapper[2565]: W0918 13:58:53.727981    2565 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 18 13:58:53.736466 ip-10-0-129-16 kubenswrapper[2565]: W0918 13:58:53.727984    2565 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 18 13:58:53.736466 ip-10-0-129-16 kubenswrapper[2565]: W0918 13:58:53.727986    2565 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 18 13:58:53.736466 ip-10-0-129-16 kubenswrapper[2565]: W0918 13:58:53.727989    2565 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 18 13:58:53.736466 ip-10-0-129-16 kubenswrapper[2565]: W0918 13:58:53.727992    2565 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 18 13:58:53.736466 ip-10-0-129-16 kubenswrapper[2565]: W0918 13:58:53.727994    2565 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 18 13:58:53.736466 ip-10-0-129-16 kubenswrapper[2565]: W0918 13:58:53.727997    2565 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 18 13:58:53.736466 ip-10-0-129-16 kubenswrapper[2565]: W0918 13:58:53.727999    2565 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 18 13:58:53.736466 ip-10-0-129-16 kubenswrapper[2565]: W0918 13:58:53.728002    2565 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 18 13:58:53.737252 ip-10-0-129-16 kubenswrapper[2565]: W0918 13:58:53.728005    2565 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 18 13:58:53.737252 ip-10-0-129-16 kubenswrapper[2565]: W0918 13:58:53.728007    2565 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 18 13:58:53.737252 ip-10-0-129-16 kubenswrapper[2565]: W0918 13:58:53.728010    2565 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 18 13:58:53.737252 ip-10-0-129-16 kubenswrapper[2565]: W0918 13:58:53.728013    2565 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 18 13:58:53.737252 ip-10-0-129-16 kubenswrapper[2565]: W0918 13:58:53.728016    2565 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 18 13:58:53.737252 ip-10-0-129-16 kubenswrapper[2565]: W0918 13:58:53.728020    2565 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 18 13:58:53.737252 ip-10-0-129-16 kubenswrapper[2565]: W0918 13:58:53.728023    2565 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 18 13:58:53.737252 ip-10-0-129-16 kubenswrapper[2565]: W0918 13:58:53.728026    2565 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 18 13:58:53.737252 ip-10-0-129-16 kubenswrapper[2565]: W0918 13:58:53.728042    2565 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 18 13:58:53.737252 ip-10-0-129-16 kubenswrapper[2565]: W0918 13:58:53.728046    2565 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 18 13:58:53.737252 ip-10-0-129-16 kubenswrapper[2565]: W0918 13:58:53.728050    2565 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 18 13:58:53.737252 ip-10-0-129-16 kubenswrapper[2565]: W0918 13:58:53.728053    2565 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 18 13:58:53.737252 ip-10-0-129-16 kubenswrapper[2565]: W0918 13:58:53.728056    2565 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 18 13:58:53.737252 ip-10-0-129-16 kubenswrapper[2565]: W0918 13:58:53.728059    2565 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 18 13:58:53.737252 ip-10-0-129-16 kubenswrapper[2565]: W0918 13:58:53.728062    2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 18 13:58:53.737252 ip-10-0-129-16 kubenswrapper[2565]: W0918 13:58:53.728065    2565 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 18 13:58:53.737252 ip-10-0-129-16 kubenswrapper[2565]: W0918 13:58:53.728068    2565 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 18 13:58:53.737252 ip-10-0-129-16 kubenswrapper[2565]: W0918 13:58:53.728071    2565 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 18 13:58:53.737252 ip-10-0-129-16 kubenswrapper[2565]: W0918 13:58:53.728074    2565 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 18 13:58:53.738064 ip-10-0-129-16 kubenswrapper[2565]: W0918 13:58:53.728077    2565 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 18 13:58:53.738064 ip-10-0-129-16 kubenswrapper[2565]: W0918 13:58:53.728082    2565 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 18 13:58:53.738064 ip-10-0-129-16 kubenswrapper[2565]: W0918 13:58:53.728084    2565 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 18 13:58:53.738064 ip-10-0-129-16 kubenswrapper[2565]: W0918 13:58:53.728087    2565 feature_gate.go:328] unrecognized feature gate: Example2
Sep 18 13:58:53.738064 ip-10-0-129-16 kubenswrapper[2565]: W0918 13:58:53.728090    2565 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 18 13:58:53.738064 ip-10-0-129-16 kubenswrapper[2565]: W0918 13:58:53.728093    2565 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 18 13:58:53.738064 ip-10-0-129-16 kubenswrapper[2565]: W0918 13:58:53.728096    2565 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 18 13:58:53.738064 ip-10-0-129-16 kubenswrapper[2565]: W0918 13:58:53.728098    2565 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 18 13:58:53.738064 ip-10-0-129-16 kubenswrapper[2565]: W0918 13:58:53.728101    2565 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 18 13:58:53.738064 ip-10-0-129-16 kubenswrapper[2565]: W0918 13:58:53.728104    2565 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 18 13:58:53.738064 ip-10-0-129-16 kubenswrapper[2565]: W0918 13:58:53.728107    2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 18 13:58:53.738064 ip-10-0-129-16 kubenswrapper[2565]: W0918 13:58:53.728110    2565 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 18 13:58:53.738064 ip-10-0-129-16 kubenswrapper[2565]: W0918 13:58:53.728113    2565 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 18 13:58:53.738064 ip-10-0-129-16 kubenswrapper[2565]: W0918 13:58:53.728115    2565 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 18 13:58:53.738064 ip-10-0-129-16 kubenswrapper[2565]: W0918 13:58:53.728118    2565 feature_gate.go:328] unrecognized feature gate: Example
Sep 18 13:58:53.738064 ip-10-0-129-16 kubenswrapper[2565]: W0918 13:58:53.728121    2565 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 18 13:58:53.738064 ip-10-0-129-16 kubenswrapper[2565]: W0918 13:58:53.728123    2565 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 18 13:58:53.738064 ip-10-0-129-16 kubenswrapper[2565]: W0918 13:58:53.728126    2565 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 18 13:58:53.738064 ip-10-0-129-16 kubenswrapper[2565]: W0918 13:58:53.728130    2565 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 18 13:58:53.738747 ip-10-0-129-16 kubenswrapper[2565]: W0918 13:58:53.728133    2565 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 18 13:58:53.738747 ip-10-0-129-16 kubenswrapper[2565]: W0918 13:58:53.728136    2565 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 18 13:58:53.738747 ip-10-0-129-16 kubenswrapper[2565]: W0918 13:58:53.728139    2565 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 18 13:58:53.738747 ip-10-0-129-16 kubenswrapper[2565]: W0918 13:58:53.728141    2565 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 18 13:58:53.738747 ip-10-0-129-16 kubenswrapper[2565]: W0918 13:58:53.728145    2565 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 18 13:58:53.738747 ip-10-0-129-16 kubenswrapper[2565]: W0918 13:58:53.728148    2565 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 18 13:58:53.738747 ip-10-0-129-16 kubenswrapper[2565]: W0918 13:58:53.728150    2565 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 18 13:58:53.738747 ip-10-0-129-16 kubenswrapper[2565]: W0918 13:58:53.728153    2565 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 18 13:58:53.738747 ip-10-0-129-16 kubenswrapper[2565]: W0918 13:58:53.728156    2565 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 18 13:58:53.738747 ip-10-0-129-16 kubenswrapper[2565]: W0918 13:58:53.728159    2565 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 18 13:58:53.738747 ip-10-0-129-16 kubenswrapper[2565]: W0918 13:58:53.728162    2565 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 18 13:58:53.738747 ip-10-0-129-16 kubenswrapper[2565]: W0918 13:58:53.728164    2565 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 18 13:58:53.738747 ip-10-0-129-16 kubenswrapper[2565]: W0918 13:58:53.728167    2565 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 18 13:58:53.738747 ip-10-0-129-16 kubenswrapper[2565]: W0918 13:58:53.728169    2565 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 18 13:58:53.738747 ip-10-0-129-16 kubenswrapper[2565]: W0918 13:58:53.728173    2565 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 18 13:58:53.738747 ip-10-0-129-16 kubenswrapper[2565]: W0918 13:58:53.728176    2565 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 18 13:58:53.738747 ip-10-0-129-16 kubenswrapper[2565]: W0918 13:58:53.728179    2565 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 18 13:58:53.738747 ip-10-0-129-16 kubenswrapper[2565]: W0918 13:58:53.728181    2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 18 13:58:53.738747 ip-10-0-129-16 kubenswrapper[2565]: W0918 13:58:53.728184    2565 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 18 13:58:53.738747 ip-10-0-129-16 kubenswrapper[2565]: W0918 13:58:53.728187    2565 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 18 13:58:53.739300 ip-10-0-129-16 kubenswrapper[2565]: W0918 13:58:53.728189    2565 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 18 13:58:53.739300 ip-10-0-129-16 kubenswrapper[2565]: W0918 13:58:53.728192    2565 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 18 13:58:53.739300 ip-10-0-129-16 kubenswrapper[2565]: W0918 13:58:53.728194    2565 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 18 13:58:53.739300 ip-10-0-129-16 kubenswrapper[2565]: W0918 13:58:53.728197    2565 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 18 13:58:53.739300 ip-10-0-129-16 kubenswrapper[2565]: W0918 13:58:53.728200    2565 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 18 13:58:53.739300 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:58:53.728208    2565 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Sep 18 13:58:53.739300 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:58:53.735587    2565 server.go:529] "Kubelet version" kubeletVersion="v1.34.9"
Sep 18 13:58:53.739300 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:58:53.735604    2565 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK=""
Sep 18 13:58:53.739300 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:58:53.735617    2565 watchdog_linux.go:95] "Systemd watchdog is not enabled"
Sep 18 13:58:53.739300 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:58:53.735626    2565 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started."
Sep 18 13:58:53.739300 ip-10-0-129-16 kubenswrapper[2565]: W0918 13:58:53.735695    2565 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 18 13:58:53.739300 ip-10-0-129-16 kubenswrapper[2565]: W0918 13:58:53.735700    2565 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 18 13:58:53.739300 ip-10-0-129-16 kubenswrapper[2565]: W0918 13:58:53.735703    2565 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 18 13:58:53.739300 ip-10-0-129-16 kubenswrapper[2565]: W0918 13:58:53.735706    2565 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 18 13:58:53.739300 ip-10-0-129-16 kubenswrapper[2565]: W0918 13:58:53.735709    2565 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 18 13:58:53.739685 ip-10-0-129-16 kubenswrapper[2565]: W0918 13:58:53.735711    2565 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 18 13:58:53.739685 ip-10-0-129-16 kubenswrapper[2565]: W0918 13:58:53.735714    2565 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 18 13:58:53.739685 ip-10-0-129-16 kubenswrapper[2565]: W0918 13:58:53.735717    2565 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 18 13:58:53.739685 ip-10-0-129-16 kubenswrapper[2565]: W0918 13:58:53.735720    2565 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 18 13:58:53.739685 ip-10-0-129-16 kubenswrapper[2565]: W0918 13:58:53.735723    2565 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 18 13:58:53.739685 ip-10-0-129-16 kubenswrapper[2565]: W0918 13:58:53.735726    2565 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 18 13:58:53.739685 ip-10-0-129-16 kubenswrapper[2565]: W0918 13:58:53.735730    2565 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 18 13:58:53.739685 ip-10-0-129-16 kubenswrapper[2565]: W0918 13:58:53.735733    2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 18 13:58:53.739685 ip-10-0-129-16 kubenswrapper[2565]: W0918 13:58:53.735736    2565 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 18 13:58:53.739685 ip-10-0-129-16 kubenswrapper[2565]: W0918 13:58:53.735739    2565 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 18 13:58:53.739685 ip-10-0-129-16 kubenswrapper[2565]: W0918 13:58:53.735742    2565 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 18 13:58:53.739685 ip-10-0-129-16 kubenswrapper[2565]: W0918 13:58:53.735745    2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 18 13:58:53.739685 ip-10-0-129-16 kubenswrapper[2565]: W0918 13:58:53.735747    2565 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 18 13:58:53.739685 ip-10-0-129-16 kubenswrapper[2565]: W0918 13:58:53.735750    2565 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 18 13:58:53.739685 ip-10-0-129-16 kubenswrapper[2565]: W0918 13:58:53.735752    2565 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 18 13:58:53.739685 ip-10-0-129-16 kubenswrapper[2565]: W0918 13:58:53.735755    2565 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 18 13:58:53.739685 ip-10-0-129-16 kubenswrapper[2565]: W0918 13:58:53.735758    2565 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 18 13:58:53.739685 ip-10-0-129-16 kubenswrapper[2565]: W0918 13:58:53.735760    2565 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 18 13:58:53.739685 ip-10-0-129-16 kubenswrapper[2565]: W0918 13:58:53.735764    2565 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 18 13:58:53.740208 ip-10-0-129-16 kubenswrapper[2565]: W0918 13:58:53.735766    2565 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 18 13:58:53.740208 ip-10-0-129-16 kubenswrapper[2565]: W0918 13:58:53.735769    2565 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 18 13:58:53.740208 ip-10-0-129-16 kubenswrapper[2565]: W0918 13:58:53.735772    2565 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 18 13:58:53.740208 ip-10-0-129-16 kubenswrapper[2565]: W0918 13:58:53.735775    2565 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 18 13:58:53.740208 ip-10-0-129-16 kubenswrapper[2565]: W0918 13:58:53.735778    2565 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 18 13:58:53.740208 ip-10-0-129-16 kubenswrapper[2565]: W0918 13:58:53.735787    2565 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 18 13:58:53.740208 ip-10-0-129-16 kubenswrapper[2565]: W0918 13:58:53.735790    2565 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 18 13:58:53.740208 ip-10-0-129-16 kubenswrapper[2565]: W0918 13:58:53.735792    2565 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 18 13:58:53.740208 ip-10-0-129-16 kubenswrapper[2565]: W0918 13:58:53.735795    2565 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 18 13:58:53.740208 ip-10-0-129-16 kubenswrapper[2565]: W0918 13:58:53.735798    2565 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 18 13:58:53.740208 ip-10-0-129-16 kubenswrapper[2565]: W0918 13:58:53.735800    2565 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 18 13:58:53.740208 ip-10-0-129-16 kubenswrapper[2565]: W0918 13:58:53.735803    2565 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 18 13:58:53.740208 ip-10-0-129-16 kubenswrapper[2565]: W0918 13:58:53.735805    2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 18 13:58:53.740208 ip-10-0-129-16 kubenswrapper[2565]: W0918 13:58:53.735808    2565 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 18 13:58:53.740208 ip-10-0-129-16 kubenswrapper[2565]: W0918 13:58:53.735810    2565 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 18 13:58:53.740208 ip-10-0-129-16 kubenswrapper[2565]: W0918 13:58:53.735813    2565 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 18 13:58:53.740208 ip-10-0-129-16 kubenswrapper[2565]: W0918 13:58:53.735816    2565 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 18 13:58:53.740208 ip-10-0-129-16 kubenswrapper[2565]: W0918 13:58:53.735818    2565 feature_gate.go:328] unrecognized feature gate: Example
Sep 18 13:58:53.740208 ip-10-0-129-16 kubenswrapper[2565]: W0918 13:58:53.735821    2565 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 18 13:58:53.740208 ip-10-0-129-16 kubenswrapper[2565]: W0918 13:58:53.735823    2565 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 18 13:58:53.740708 ip-10-0-129-16 kubenswrapper[2565]: W0918 13:58:53.735826    2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 18 13:58:53.740708 ip-10-0-129-16 kubenswrapper[2565]: W0918 13:58:53.735828    2565 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 18 13:58:53.740708 ip-10-0-129-16 kubenswrapper[2565]: W0918 13:58:53.735831    2565 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 18 13:58:53.740708 ip-10-0-129-16 kubenswrapper[2565]: W0918 13:58:53.735833    2565 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 18 13:58:53.740708 ip-10-0-129-16 kubenswrapper[2565]: W0918 13:58:53.735836    2565 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 18 13:58:53.740708 ip-10-0-129-16 kubenswrapper[2565]: W0918 13:58:53.735839    2565 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 18 13:58:53.740708 ip-10-0-129-16 kubenswrapper[2565]: W0918 13:58:53.735841    2565 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 18 13:58:53.740708 ip-10-0-129-16 kubenswrapper[2565]: W0918 13:58:53.735844    2565 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 18 13:58:53.740708 ip-10-0-129-16 kubenswrapper[2565]: W0918 13:58:53.735847    2565 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 18 13:58:53.740708 ip-10-0-129-16 kubenswrapper[2565]: W0918 13:58:53.735849    2565 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 18 13:58:53.740708 ip-10-0-129-16 kubenswrapper[2565]: W0918 13:58:53.735852    2565 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 18 13:58:53.740708 ip-10-0-129-16 kubenswrapper[2565]: W0918 13:58:53.735854    2565 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 18 13:58:53.740708 ip-10-0-129-16 kubenswrapper[2565]: W0918 13:58:53.735858    2565 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 18 13:58:53.740708 ip-10-0-129-16 kubenswrapper[2565]: W0918 13:58:53.735860    2565 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 18 13:58:53.740708 ip-10-0-129-16 kubenswrapper[2565]: W0918 13:58:53.735863    2565 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 18 13:58:53.740708 ip-10-0-129-16 kubenswrapper[2565]: W0918 13:58:53.735866    2565 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 18 13:58:53.740708 ip-10-0-129-16 kubenswrapper[2565]: W0918 13:58:53.735869    2565 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 18 13:58:53.740708 ip-10-0-129-16 kubenswrapper[2565]: W0918 13:58:53.735872    2565 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 18 13:58:53.740708 ip-10-0-129-16 kubenswrapper[2565]: W0918 13:58:53.735880    2565 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 18 13:58:53.740708 ip-10-0-129-16 kubenswrapper[2565]: W0918 13:58:53.735882    2565 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 18 13:58:53.741212 ip-10-0-129-16 kubenswrapper[2565]: W0918 13:58:53.735886    2565 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 18 13:58:53.741212 ip-10-0-129-16 kubenswrapper[2565]: W0918 13:58:53.735890    2565 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 18 13:58:53.741212 ip-10-0-129-16 kubenswrapper[2565]: W0918 13:58:53.735893    2565 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 18 13:58:53.741212 ip-10-0-129-16 kubenswrapper[2565]: W0918 13:58:53.735896    2565 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 18 13:58:53.741212 ip-10-0-129-16 kubenswrapper[2565]: W0918 13:58:53.735898    2565 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 18 13:58:53.741212 ip-10-0-129-16 kubenswrapper[2565]: W0918 13:58:53.735901    2565 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 18 13:58:53.741212 ip-10-0-129-16 kubenswrapper[2565]: W0918 13:58:53.735904    2565 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 18 13:58:53.741212 ip-10-0-129-16 kubenswrapper[2565]: W0918 13:58:53.735907    2565 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 18 13:58:53.741212 ip-10-0-129-16 kubenswrapper[2565]: W0918 13:58:53.735909    2565 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 18 13:58:53.741212 ip-10-0-129-16 kubenswrapper[2565]: W0918 13:58:53.735912    2565 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 18 13:58:53.741212 ip-10-0-129-16 kubenswrapper[2565]: W0918 13:58:53.735914    2565 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 18 13:58:53.741212 ip-10-0-129-16 kubenswrapper[2565]: W0918 13:58:53.735917    2565 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 18 13:58:53.741212 ip-10-0-129-16 kubenswrapper[2565]: W0918 13:58:53.735920    2565 feature_gate.go:328] unrecognized feature gate: Example2
Sep 18 13:58:53.741212 ip-10-0-129-16 kubenswrapper[2565]: W0918 13:58:53.735923    2565 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 18 13:58:53.741212 ip-10-0-129-16 kubenswrapper[2565]: W0918 13:58:53.735926    2565 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 18 13:58:53.741212 ip-10-0-129-16 kubenswrapper[2565]: W0918 13:58:53.735930    2565 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 18 13:58:53.741212 ip-10-0-129-16 kubenswrapper[2565]: W0918 13:58:53.735933    2565 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 18 13:58:53.741212 ip-10-0-129-16 kubenswrapper[2565]: W0918 13:58:53.735936    2565 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 18 13:58:53.741212 ip-10-0-129-16 kubenswrapper[2565]: W0918 13:58:53.735939    2565 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 18 13:58:53.741684 ip-10-0-129-16 kubenswrapper[2565]: W0918 13:58:53.735942    2565 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 18 13:58:53.741684 ip-10-0-129-16 kubenswrapper[2565]: W0918 13:58:53.735945    2565 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 18 13:58:53.741684 ip-10-0-129-16 kubenswrapper[2565]: W0918 13:58:53.735949    2565 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 18 13:58:53.741684 ip-10-0-129-16 kubenswrapper[2565]: W0918 13:58:53.735951    2565 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 18 13:58:53.741684 ip-10-0-129-16 kubenswrapper[2565]: W0918 13:58:53.735955    2565 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 18 13:58:53.741684 ip-10-0-129-16 kubenswrapper[2565]: W0918 13:58:53.735958    2565 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 18 13:58:53.741684 ip-10-0-129-16 kubenswrapper[2565]: W0918 13:58:53.735961    2565 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 18 13:58:53.741684 ip-10-0-129-16 kubenswrapper[2565]: W0918 13:58:53.735964    2565 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 18 13:58:53.741684 ip-10-0-129-16 kubenswrapper[2565]: W0918 13:58:53.735967    2565 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 18 13:58:53.741684 ip-10-0-129-16 kubenswrapper[2565]: W0918 13:58:53.735969    2565 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 18 13:58:53.741684 ip-10-0-129-16 kubenswrapper[2565]: W0918 13:58:53.735972    2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 18 13:58:53.741684 ip-10-0-129-16 kubenswrapper[2565]: W0918 13:58:53.735975    2565 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 18 13:58:53.741684 ip-10-0-129-16 kubenswrapper[2565]: W0918 13:58:53.735983    2565 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 18 13:58:53.741684 ip-10-0-129-16 kubenswrapper[2565]: W0918 13:58:53.735986    2565 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 18 13:58:53.741684 ip-10-0-129-16 kubenswrapper[2565]: W0918 13:58:53.735995    2565 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 18 13:58:53.741684 ip-10-0-129-16 kubenswrapper[2565]: W0918 13:58:53.735997    2565 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 18 13:58:53.742089 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:58:53.736003    2565 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Sep 18 13:58:53.742089 ip-10-0-129-16 kubenswrapper[2565]: W0918 13:58:53.736168    2565 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 18 13:58:53.742089 ip-10-0-129-16 kubenswrapper[2565]: W0918 13:58:53.736174    2565 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 18 13:58:53.742089 ip-10-0-129-16 kubenswrapper[2565]: W0918 13:58:53.736177    2565 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 18 13:58:53.742089 ip-10-0-129-16 kubenswrapper[2565]: W0918 13:58:53.736180    2565 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 18 13:58:53.742089 ip-10-0-129-16 kubenswrapper[2565]: W0918 13:58:53.736183    2565 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 18 13:58:53.742089 ip-10-0-129-16 kubenswrapper[2565]: W0918 13:58:53.736187    2565 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 18 13:58:53.742089 ip-10-0-129-16 kubenswrapper[2565]: W0918 13:58:53.736190    2565 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 18 13:58:53.742089 ip-10-0-129-16 kubenswrapper[2565]: W0918 13:58:53.736193    2565 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 18 13:58:53.742089 ip-10-0-129-16 kubenswrapper[2565]: W0918 13:58:53.736196    2565 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 18 13:58:53.742089 ip-10-0-129-16 kubenswrapper[2565]: W0918 13:58:53.736199    2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 18 13:58:53.742089 ip-10-0-129-16 kubenswrapper[2565]: W0918 13:58:53.736202    2565 feature_gate.go:328] unrecognized feature gate: Example
Sep 18 13:58:53.742089 ip-10-0-129-16 kubenswrapper[2565]: W0918 13:58:53.736205    2565 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 18 13:58:53.742089 ip-10-0-129-16 kubenswrapper[2565]: W0918 13:58:53.736208    2565 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 18 13:58:53.742089 ip-10-0-129-16 kubenswrapper[2565]: W0918 13:58:53.736210    2565 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 18 13:58:53.742462 ip-10-0-129-16 kubenswrapper[2565]: W0918 13:58:53.736213    2565 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 18 13:58:53.742462 ip-10-0-129-16 kubenswrapper[2565]: W0918 13:58:53.736217    2565 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 18 13:58:53.742462 ip-10-0-129-16 kubenswrapper[2565]: W0918 13:58:53.736220    2565 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 18 13:58:53.742462 ip-10-0-129-16 kubenswrapper[2565]: W0918 13:58:53.736223    2565 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 18 13:58:53.742462 ip-10-0-129-16 kubenswrapper[2565]: W0918 13:58:53.736226    2565 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 18 13:58:53.742462 ip-10-0-129-16 kubenswrapper[2565]: W0918 13:58:53.736228    2565 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 18 13:58:53.742462 ip-10-0-129-16 kubenswrapper[2565]: W0918 13:58:53.736232    2565 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 18 13:58:53.742462 ip-10-0-129-16 kubenswrapper[2565]: W0918 13:58:53.736236    2565 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 18 13:58:53.742462 ip-10-0-129-16 kubenswrapper[2565]: W0918 13:58:53.736239    2565 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 18 13:58:53.742462 ip-10-0-129-16 kubenswrapper[2565]: W0918 13:58:53.736242    2565 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 18 13:58:53.742462 ip-10-0-129-16 kubenswrapper[2565]: W0918 13:58:53.736245    2565 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 18 13:58:53.742462 ip-10-0-129-16 kubenswrapper[2565]: W0918 13:58:53.736248    2565 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 18 13:58:53.742462 ip-10-0-129-16 kubenswrapper[2565]: W0918 13:58:53.736250    2565 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 18 13:58:53.742462 ip-10-0-129-16 kubenswrapper[2565]: W0918 13:58:53.736254    2565 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 18 13:58:53.742462 ip-10-0-129-16 kubenswrapper[2565]: W0918 13:58:53.736263    2565 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 18 13:58:53.742462 ip-10-0-129-16 kubenswrapper[2565]: W0918 13:58:53.736266    2565 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 18 13:58:53.742462 ip-10-0-129-16 kubenswrapper[2565]: W0918 13:58:53.736269    2565 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 18 13:58:53.742462 ip-10-0-129-16 kubenswrapper[2565]: W0918 13:58:53.736272    2565 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 18 13:58:53.742462 ip-10-0-129-16 kubenswrapper[2565]: W0918 13:58:53.736275    2565 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 18 13:58:53.742462 ip-10-0-129-16 kubenswrapper[2565]: W0918 13:58:53.736278    2565 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 18 13:58:53.742960 ip-10-0-129-16 kubenswrapper[2565]: W0918 13:58:53.736280    2565 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 18 13:58:53.742960 ip-10-0-129-16 kubenswrapper[2565]: W0918 13:58:53.736283    2565 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 18 13:58:53.742960 ip-10-0-129-16 kubenswrapper[2565]: W0918 13:58:53.736286    2565 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 18 13:58:53.742960 ip-10-0-129-16 kubenswrapper[2565]: W0918 13:58:53.736289    2565 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 18 13:58:53.742960 ip-10-0-129-16 kubenswrapper[2565]: W0918 13:58:53.736291    2565 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 18 13:58:53.742960 ip-10-0-129-16 kubenswrapper[2565]: W0918 13:58:53.736294    2565 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 18 13:58:53.742960 ip-10-0-129-16 kubenswrapper[2565]: W0918 13:58:53.736296    2565 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 18 13:58:53.742960 ip-10-0-129-16 kubenswrapper[2565]: W0918 13:58:53.736299    2565 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 18 13:58:53.742960 ip-10-0-129-16 kubenswrapper[2565]: W0918 13:58:53.736302    2565 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 18 13:58:53.742960 ip-10-0-129-16 kubenswrapper[2565]: W0918 13:58:53.736304    2565 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 18 13:58:53.742960 ip-10-0-129-16 kubenswrapper[2565]: W0918 13:58:53.736307    2565 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 18 13:58:53.742960 ip-10-0-129-16 kubenswrapper[2565]: W0918 13:58:53.736310    2565 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 18 13:58:53.742960 ip-10-0-129-16 kubenswrapper[2565]: W0918 13:58:53.736312    2565 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 18 13:58:53.742960 ip-10-0-129-16 kubenswrapper[2565]: W0918 13:58:53.736315    2565 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 18 13:58:53.742960 ip-10-0-129-16 kubenswrapper[2565]: W0918 13:58:53.736318    2565 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 18 13:58:53.742960 ip-10-0-129-16 kubenswrapper[2565]: W0918 13:58:53.736321    2565 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 18 13:58:53.742960 ip-10-0-129-16 kubenswrapper[2565]: W0918 13:58:53.736323    2565 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 18 13:58:53.742960 ip-10-0-129-16 kubenswrapper[2565]: W0918 13:58:53.736326    2565 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 18 13:58:53.742960 ip-10-0-129-16 kubenswrapper[2565]: W0918 13:58:53.736329    2565 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 18 13:58:53.742960 ip-10-0-129-16 kubenswrapper[2565]: W0918 13:58:53.736331    2565 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 18 13:58:53.743483 ip-10-0-129-16 kubenswrapper[2565]: W0918 13:58:53.736335    2565 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 18 13:58:53.743483 ip-10-0-129-16 kubenswrapper[2565]: W0918 13:58:53.736338    2565 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 18 13:58:53.743483 ip-10-0-129-16 kubenswrapper[2565]: W0918 13:58:53.736340    2565 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 18 13:58:53.743483 ip-10-0-129-16 kubenswrapper[2565]: W0918 13:58:53.736343    2565 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 18 13:58:53.743483 ip-10-0-129-16 kubenswrapper[2565]: W0918 13:58:53.736345    2565 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 18 13:58:53.743483 ip-10-0-129-16 kubenswrapper[2565]: W0918 13:58:53.736348    2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 18 13:58:53.743483 ip-10-0-129-16 kubenswrapper[2565]: W0918 13:58:53.736351    2565 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 18 13:58:53.743483 ip-10-0-129-16 kubenswrapper[2565]: W0918 13:58:53.736358    2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 18 13:58:53.743483 ip-10-0-129-16 kubenswrapper[2565]: W0918 13:58:53.736361    2565 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 18 13:58:53.743483 ip-10-0-129-16 kubenswrapper[2565]: W0918 13:58:53.736364    2565 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 18 13:58:53.743483 ip-10-0-129-16 kubenswrapper[2565]: W0918 13:58:53.736366    2565 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 18 13:58:53.743483 ip-10-0-129-16 kubenswrapper[2565]: W0918 13:58:53.736369    2565 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 18 13:58:53.743483 ip-10-0-129-16 kubenswrapper[2565]: W0918 13:58:53.736371    2565 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 18 13:58:53.743483 ip-10-0-129-16 kubenswrapper[2565]: W0918 13:58:53.736374    2565 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 18 13:58:53.743483 ip-10-0-129-16 kubenswrapper[2565]: W0918 13:58:53.736377    2565 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 18 13:58:53.743483 ip-10-0-129-16 kubenswrapper[2565]: W0918 13:58:53.736379    2565 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 18 13:58:53.743483 ip-10-0-129-16 kubenswrapper[2565]: W0918 13:58:53.736382    2565 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 18 13:58:53.743483 ip-10-0-129-16 kubenswrapper[2565]: W0918 13:58:53.736385    2565 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 18 13:58:53.743483 ip-10-0-129-16 kubenswrapper[2565]: W0918 13:58:53.736387    2565 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 18 13:58:53.743483 ip-10-0-129-16 kubenswrapper[2565]: W0918 13:58:53.736390    2565 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 18 13:58:53.744057 ip-10-0-129-16 kubenswrapper[2565]: W0918 13:58:53.736392    2565 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 18 13:58:53.744057 ip-10-0-129-16 kubenswrapper[2565]: W0918 13:58:53.736395    2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 18 13:58:53.744057 ip-10-0-129-16 kubenswrapper[2565]: W0918 13:58:53.736398    2565 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 18 13:58:53.744057 ip-10-0-129-16 kubenswrapper[2565]: W0918 13:58:53.736400    2565 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 18 13:58:53.744057 ip-10-0-129-16 kubenswrapper[2565]: W0918 13:58:53.736403    2565 feature_gate.go:328] unrecognized feature gate: Example2
Sep 18 13:58:53.744057 ip-10-0-129-16 kubenswrapper[2565]: W0918 13:58:53.736406    2565 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 18 13:58:53.744057 ip-10-0-129-16 kubenswrapper[2565]: W0918 13:58:53.736409    2565 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 18 13:58:53.744057 ip-10-0-129-16 kubenswrapper[2565]: W0918 13:58:53.736411    2565 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 18 13:58:53.744057 ip-10-0-129-16 kubenswrapper[2565]: W0918 13:58:53.736414    2565 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 18 13:58:53.744057 ip-10-0-129-16 kubenswrapper[2565]: W0918 13:58:53.736416    2565 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 18 13:58:53.744057 ip-10-0-129-16 kubenswrapper[2565]: W0918 13:58:53.736419    2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 18 13:58:53.744057 ip-10-0-129-16 kubenswrapper[2565]: W0918 13:58:53.736422    2565 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 18 13:58:53.744057 ip-10-0-129-16 kubenswrapper[2565]: W0918 13:58:53.736424    2565 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 18 13:58:53.744057 ip-10-0-129-16 kubenswrapper[2565]: W0918 13:58:53.736427    2565 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 18 13:58:53.744057 ip-10-0-129-16 kubenswrapper[2565]: W0918 13:58:53.736429    2565 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 18 13:58:53.744057 ip-10-0-129-16 kubenswrapper[2565]: W0918 13:58:53.736432    2565 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 18 13:58:53.744057 ip-10-0-129-16 kubenswrapper[2565]: W0918 13:58:53.736434    2565 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 18 13:58:53.744057 ip-10-0-129-16 kubenswrapper[2565]: W0918 13:58:53.736437    2565 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 18 13:58:53.744057 ip-10-0-129-16 kubenswrapper[2565]: W0918 13:58:53.736440    2565 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 18 13:58:53.744057 ip-10-0-129-16 kubenswrapper[2565]: W0918 13:58:53.736457    2565 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 18 13:58:53.744561 ip-10-0-129-16 kubenswrapper[2565]: W0918 13:58:53.736466    2565 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 18 13:58:53.744561 ip-10-0-129-16 kubenswrapper[2565]: W0918 13:58:53.736470    2565 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 18 13:58:53.744561 ip-10-0-129-16 kubenswrapper[2565]: W0918 13:58:53.736474    2565 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 18 13:58:53.744561 ip-10-0-129-16 kubenswrapper[2565]: W0918 13:58:53.736478    2565 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 18 13:58:53.744561 ip-10-0-129-16 kubenswrapper[2565]: W0918 13:58:53.736481    2565 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 18 13:58:53.744561 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:58:53.736486    2565 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Sep 18 13:58:53.744561 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:58:53.737885    2565 server.go:962] "Client rotation is on, will bootstrap in background"
Sep 18 13:58:53.744561 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:58:53.740070    2565 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir"
Sep 18 13:58:53.744561 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:58:53.741051    2565 server.go:1019] "Starting client certificate rotation"
Sep 18 13:58:53.744561 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:58:53.741153    2565 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet"
Sep 18 13:58:53.744561 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:58:53.741777    2565 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet"
Sep 18 13:58:53.744561 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:58:53.743443    2565 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Sep 18 13:58:53.744935 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:58:53.744670    2565 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Sep 18 13:58:53.748968 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:58:53.748950    2565 log.go:25] "Validated CRI v1 runtime API"
Sep 18 13:58:53.750540 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:58:53.750524    2565 log.go:25] "Validated CRI v1 image API"
Sep 18 13:58:53.751362 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:58:53.751347    2565 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd"
Sep 18 13:58:53.754429 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:58:53.754405    2565 fs.go:135] Filesystem UUIDs: map[416690ea-929d-4fb6-b725-1c5d7dcd0b0a:/dev/nvme0n1p3 7B77-95E7:/dev/nvme0n1p2 dd578f56-8d57-4b32-a98e-1af979333313:/dev/nvme0n1p4]
Sep 18 13:58:53.754504 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:58:53.754429    2565 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}]
Sep 18 13:58:53.760020 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:58:53.759918    2565 manager.go:216] Machine: {Timestamp:2026-09-18 13:58:53.759407103 +0000 UTC m=+1.933316776 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3068880 MemoryCapacity:33164480512 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec2b72765b19aa05f0b32923d5c8f877 SystemUUID:ec2b7276-5b19-aa05-f0b3-2923d5c8f877 BootID:ae57acdb-b443-45b1-bf31-cc2c0eea20a0 Filesystems:[{Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6632898560 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6111232 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16582242304 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true} {Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16582238208 Type:vfs Inodes:4048398 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:ff:d6:4a:e8:b1 Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:ff:d6:4a:e8:b1 Speed:0 Mtu:9001} {Name:ovs-system MacAddress:46:5d:30:e7:46:4b Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:33164480512 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:37486592 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None}
Sep 18 13:58:53.760020 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:58:53.760017    2565 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available.
Sep 18 13:58:53.760132 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:58:53.760106    2565 manager.go:232] Version: {KernelVersion:5.14.0-570.141.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260914-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:}
Sep 18 13:58:53.761778 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:58:53.761735    2565 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[]
Sep 18 13:58:53.761955 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:58:53.761791    2565 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-129-16.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2}
Sep 18 13:58:53.762451 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:58:53.762441    2565 topology_manager.go:138] "Creating topology manager with none policy"
Sep 18 13:58:53.762489 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:58:53.762453    2565 container_manager_linux.go:309] "Creating device plugin manager"
Sep 18 13:58:53.762489 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:58:53.762461    2565 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock"
Sep 18 13:58:53.762489 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:58:53.762482    2565 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock"
Sep 18 13:58:53.763479 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:58:53.763470    2565 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager"
Sep 18 13:58:53.767817 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:58:53.767803    2565 state_mem.go:36] "Initialized new in-memory state store"
Sep 18 13:58:53.767977 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:58:53.767967    2565 server.go:1240] "Using root directory" path="/var/lib/kubelet"
Sep 18 13:58:53.768025 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:58:53.768017    2565 kubelet.go:486] "Attempting to sync node with API server"
Sep 18 13:58:53.768076 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:58:53.768045    2565 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests"
Sep 18 13:58:53.768076 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:58:53.768062    2565 file.go:69] "Watching path" path="/etc/kubernetes/manifests"
Sep 18 13:58:53.768076 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:58:53.768072    2565 kubelet.go:398] "Adding apiserver pod source"
Sep 18 13:58:53.768158 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:58:53.768080    2565 apiserver.go:42] "Waiting for node sync before watching apiserver pods"
Sep 18 13:58:53.768498 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:58:53.768479    2565 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Sep 18 13:58:53.768930 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:58:53.768914    2565 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Sep 18 13:58:53.777258 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:58:53.777204    2565 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.13-6.rhaos4.21.git6253162.el9" apiVersion="v1"
Sep 18 13:58:53.779268 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:58:53.779249    2565 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled"
Sep 18 13:58:53.779366 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:58:53.779289    2565 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled"
Sep 18 13:58:53.780235 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:58:53.780220    2565 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume"
Sep 18 13:58:53.780301 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:58:53.780245    2565 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir"
Sep 18 13:58:53.780301 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:58:53.780255    2565 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo"
Sep 18 13:58:53.780301 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:58:53.780263    2565 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path"
Sep 18 13:58:53.780301 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:58:53.780276    2565 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs"
Sep 18 13:58:53.780301 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:58:53.780284    2565 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret"
Sep 18 13:58:53.780301 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:58:53.780303    2565 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi"
Sep 18 13:58:53.780530 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:58:53.780311    2565 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api"
Sep 18 13:58:53.780530 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:58:53.780321    2565 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc"
Sep 18 13:58:53.780530 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:58:53.780330    2565 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap"
Sep 18 13:58:53.780530 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:58:53.780343    2565 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected"
Sep 18 13:58:53.780530 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:58:53.780350    2565 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume"
Sep 18 13:58:53.780530 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:58:53.780384    2565 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi"
Sep 18 13:58:53.780530 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:58:53.780392    2565 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image"
Sep 18 13:58:53.784196 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:58:53.784181    2565 server.go:1268] "Started kubelet"
Sep 18 13:58:53.784279 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:58:53.784239    2565 server.go:180] "Starting to listen" address="0.0.0.0" port=10250
Sep 18 13:58:53.784336 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:58:53.784270    2565 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10
Sep 18 13:58:53.784379 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:58:53.784357    2565 server_v1.go:49] "podresources" method="list" useActivePods=true
Sep 18 13:58:53.785154 ip-10-0-129-16 systemd[1]: Started Kubernetes Kubelet.
Sep 18 13:58:53.785647 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:58:53.785342    2565 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock"
Sep 18 13:58:53.786361 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:58:53.786347    2565 server.go:310] "Adding debug handlers to kubelet server"
Sep 18 13:58:53.788255 ip-10-0-129-16 kubenswrapper[2565]: E0918 13:58:53.788201    2565 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-129-16.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node"
Sep 18 13:58:53.788339 ip-10-0-129-16 kubenswrapper[2565]: E0918 13:58:53.788270    2565 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service"
Sep 18 13:58:53.789981 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:58:53.789963    2565 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Sep 18 13:58:53.790252 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:58:53.790234    2565 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-129-16.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Sep 18 13:58:53.791843 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:58:53.791824    2565 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving"
Sep 18 13:58:53.791918 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:58:53.791826    2565 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer"
Sep 18 13:58:53.792315 ip-10-0-129-16 kubenswrapper[2565]: E0918 13:58:53.792300    2565 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-16.ec2.internal\" not found"
Sep 18 13:58:53.792502 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:58:53.792487    2565 volume_manager.go:311] "The desired_state_of_world populator starts"
Sep 18 13:58:53.792625 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:58:53.792506    2565 volume_manager.go:313] "Starting Kubelet Volume Manager"
Sep 18 13:58:53.792625 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:58:53.792617    2565 reconstruct.go:97] "Volume reconstruction finished"
Sep 18 13:58:53.792625 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:58:53.792623    2565 reconciler.go:29] "Reconciler: start to sync state"
Sep 18 13:58:53.792817 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:58:53.792804    2565 desired_state_of_world_populator.go:146] "Desired state populator starts to run"
Sep 18 13:58:53.793292 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:58:53.793274    2565 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory
Sep 18 13:58:53.793292 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:58:53.793293    2565 factory.go:55] Registering systemd factory
Sep 18 13:58:53.793424 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:58:53.793303    2565 factory.go:223] Registration of the systemd container factory successfully
Sep 18 13:58:53.793734 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:58:53.793715    2565 factory.go:153] Registering CRI-O factory
Sep 18 13:58:53.793734 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:58:53.793736    2565 factory.go:223] Registration of the crio container factory successfully
Sep 18 13:58:53.793855 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:58:53.793754    2565 factory.go:103] Registering Raw factory
Sep 18 13:58:53.793855 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:58:53.793764    2565 manager.go:1196] Started watching for new ooms in manager
Sep 18 13:58:53.794270 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:58:53.794254    2565 manager.go:349] Starting recovery of all containers
Sep 18 13:58:53.798333 ip-10-0-129-16 kubenswrapper[2565]: E0918 13:58:53.798307    2565 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache"
Sep 18 13:58:53.800360 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:58:53.800222    2565 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6"
Sep 18 13:58:53.819521 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:58:53.819495    2565 manager.go:354] Recovery completed
Sep 18 13:58:53.819661 ip-10-0-129-16 kubenswrapper[2565]: E0918 13:58:53.819570    2565 reflector.go:205] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver"
Sep 18 13:58:53.819777 ip-10-0-129-16 kubenswrapper[2565]: E0918 13:58:53.819734    2565 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-129-16.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms"
Sep 18 13:58:53.824697 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:58:53.824681    2565 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 18 13:58:53.826556 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:58:53.826539    2565 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-16.ec2.internal" event="NodeHasSufficientMemory"
Sep 18 13:58:53.826619 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:58:53.826568    2565 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-16.ec2.internal" event="NodeHasNoDiskPressure"
Sep 18 13:58:53.826619 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:58:53.826578    2565 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-16.ec2.internal" event="NodeHasSufficientPID"
Sep 18 13:58:53.827143 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:58:53.827130    2565 cpu_manager.go:222] "Starting CPU manager" policy="none"
Sep 18 13:58:53.827143 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:58:53.827141    2565 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s"
Sep 18 13:58:53.827239 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:58:53.827156    2565 state_mem.go:36] "Initialized new in-memory state store"
Sep 18 13:58:53.827239 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:58:53.827200    2565 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 18 13:58:53.828311 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:58:53.828291    2565 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-16.ec2.internal" event="NodeHasSufficientMemory"
Sep 18 13:58:53.828372 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:58:53.828323    2565 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-16.ec2.internal" event="NodeHasNoDiskPressure"
Sep 18 13:58:53.828372 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:58:53.828334    2565 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-16.ec2.internal" event="NodeHasSufficientPID"
Sep 18 13:58:53.829459 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:58:53.829448    2565 policy_none.go:49] "None policy: Start"
Sep 18 13:58:53.829494 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:58:53.829463    2565 memory_manager.go:187] "Starting memorymanager" policy="None"
Sep 18 13:58:53.829494 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:58:53.829473    2565 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint"
Sep 18 13:58:53.830709 ip-10-0-129-16 kubenswrapper[2565]: E0918 13:58:53.829640    2565 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-129-16.ec2.internal.18d66efcc189260b  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-129-16.ec2.internal,UID:ip-10-0-129-16.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-129-16.ec2.internal,},FirstTimestamp:2026-09-18 13:58:53.784155659 +0000 UTC m=+1.958065363,LastTimestamp:2026-09-18 13:58:53.784155659 +0000 UTC m=+1.958065363,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-129-16.ec2.internal,}"
Sep 18 13:58:53.831531 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:58:53.831520    2565 policy_none.go:47] "Start"
Sep 18 13:58:53.840070 ip-10-0-129-16 kubenswrapper[2565]: E0918 13:58:53.839987    2565 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-129-16.ec2.internal.18d66efcc4101f7d  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-129-16.ec2.internal,UID:ip-10-0-129-16.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientMemory,Message:Node ip-10-0-129-16.ec2.internal status is now: NodeHasSufficientMemory,Source:EventSource{Component:kubelet,Host:ip-10-0-129-16.ec2.internal,},FirstTimestamp:2026-09-18 13:58:53.826555773 +0000 UTC m=+2.000465423,LastTimestamp:2026-09-18 13:58:53.826555773 +0000 UTC m=+2.000465423,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-129-16.ec2.internal,}"
Sep 18 13:58:53.851800 ip-10-0-129-16 kubenswrapper[2565]: E0918 13:58:53.851733    2565 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-129-16.ec2.internal.18d66efcc4106177  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-129-16.ec2.internal,UID:ip-10-0-129-16.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasNoDiskPressure,Message:Node ip-10-0-129-16.ec2.internal status is now: NodeHasNoDiskPressure,Source:EventSource{Component:kubelet,Host:ip-10-0-129-16.ec2.internal,},FirstTimestamp:2026-09-18 13:58:53.826572663 +0000 UTC m=+2.000482313,LastTimestamp:2026-09-18 13:58:53.826572663 +0000 UTC m=+2.000482313,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-129-16.ec2.internal,}"
Sep 18 13:58:53.862608 ip-10-0-129-16 kubenswrapper[2565]: E0918 13:58:53.862519    2565 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-129-16.ec2.internal.18d66efcc4108560  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-129-16.ec2.internal,UID:ip-10-0-129-16.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientPID,Message:Node ip-10-0-129-16.ec2.internal status is now: NodeHasSufficientPID,Source:EventSource{Component:kubelet,Host:ip-10-0-129-16.ec2.internal,},FirstTimestamp:2026-09-18 13:58:53.826581856 +0000 UTC m=+2.000491506,LastTimestamp:2026-09-18 13:58:53.826581856 +0000 UTC m=+2.000491506,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-129-16.ec2.internal,}"
Sep 18 13:58:53.864822 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:58:53.864803    2565 manager.go:337] "Starting Device Plugin manager"
Sep 18 13:58:53.885265 ip-10-0-129-16 kubenswrapper[2565]: E0918 13:58:53.864850    2565 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint"
Sep 18 13:58:53.885265 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:58:53.864865    2565 server.go:87] "Starting device plugin registration server"
Sep 18 13:58:53.885265 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:58:53.865185    2565 eviction_manager.go:189] "Eviction manager: starting control loop"
Sep 18 13:58:53.885265 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:58:53.865213    2565 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s"
Sep 18 13:58:53.885265 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:58:53.865280    2565 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry"
Sep 18 13:58:53.885265 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:58:53.865361    2565 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts"
Sep 18 13:58:53.885265 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:58:53.865370    2565 plugin_manager.go:118] "Starting Kubelet Plugin Manager"
Sep 18 13:58:53.885265 ip-10-0-129-16 kubenswrapper[2565]: E0918 13:58:53.865922    2565 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\""
Sep 18 13:58:53.885265 ip-10-0-129-16 kubenswrapper[2565]: E0918 13:58:53.865969    2565 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-129-16.ec2.internal\" not found"
Sep 18 13:58:53.885265 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:58:53.870373    2565 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-x8pkn"
Sep 18 13:58:53.885265 ip-10-0-129-16 kubenswrapper[2565]: E0918 13:58:53.878018    2565 event.go:359] "Server rejected event (will not retry!)" err="events \"ip-10-0-129-16.ec2.internal.18d66efcc4101f7d\" is forbidden: User \"system:anonymous\" cannot patch resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-129-16.ec2.internal.18d66efcc4101f7d  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-129-16.ec2.internal,UID:ip-10-0-129-16.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientMemory,Message:Node ip-10-0-129-16.ec2.internal status is now: NodeHasSufficientMemory,Source:EventSource{Component:kubelet,Host:ip-10-0-129-16.ec2.internal,},FirstTimestamp:2026-09-18 13:58:53.826555773 +0000 UTC m=+2.000465423,LastTimestamp:2026-09-18 13:58:53.828309396 +0000 UTC m=+2.002219049,Count:2,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-129-16.ec2.internal,}"
Sep 18 13:58:53.886945 ip-10-0-129-16 kubenswrapper[2565]: E0918 13:58:53.886884    2565 event.go:359] "Server rejected event (will not retry!)" err="events \"ip-10-0-129-16.ec2.internal.18d66efcc4106177\" is forbidden: User \"system:anonymous\" cannot patch resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-129-16.ec2.internal.18d66efcc4106177  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-129-16.ec2.internal,UID:ip-10-0-129-16.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasNoDiskPressure,Message:Node ip-10-0-129-16.ec2.internal status is now: NodeHasNoDiskPressure,Source:EventSource{Component:kubelet,Host:ip-10-0-129-16.ec2.internal,},FirstTimestamp:2026-09-18 13:58:53.826572663 +0000 UTC m=+2.000482313,LastTimestamp:2026-09-18 13:58:53.828328384 +0000 UTC m=+2.002238035,Count:2,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-129-16.ec2.internal,}"
Sep 18 13:58:53.887040 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:58:53.887013    2565 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-x8pkn"
Sep 18 13:58:53.905805 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:58:53.905776    2565 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4"
Sep 18 13:58:53.905903 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:58:53.905832    2565 status_manager.go:244] "Starting to sync pod status with apiserver"
Sep 18 13:58:53.905903 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:58:53.905854    2565 kubelet.go:2448] "Starting kubelet main sync loop"
Sep 18 13:58:53.905982 ip-10-0-129-16 kubenswrapper[2565]: E0918 13:58:53.905890    2565 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful"
Sep 18 13:58:53.909201 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:58:53.909182    2565 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 18 13:58:53.966364 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:58:53.966336    2565 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 18 13:58:53.967498 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:58:53.967483    2565 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-16.ec2.internal" event="NodeHasSufficientMemory"
Sep 18 13:58:53.967579 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:58:53.967511    2565 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-16.ec2.internal" event="NodeHasNoDiskPressure"
Sep 18 13:58:53.967579 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:58:53.967521    2565 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-16.ec2.internal" event="NodeHasSufficientPID"
Sep 18 13:58:53.967579 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:58:53.967542    2565 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-129-16.ec2.internal"
Sep 18 13:58:53.978801 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:58:53.978757    2565 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-129-16.ec2.internal"
Sep 18 13:58:53.978801 ip-10-0-129-16 kubenswrapper[2565]: E0918 13:58:53.978780    2565 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-129-16.ec2.internal\": node \"ip-10-0-129-16.ec2.internal\" not found"
Sep 18 13:58:54.006796 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:58:54.006770    2565 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-16.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-129-16.ec2.internal"]
Sep 18 13:58:54.006902 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:58:54.006838    2565 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 18 13:58:54.007638 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:58:54.007621    2565 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-16.ec2.internal" event="NodeHasSufficientMemory"
Sep 18 13:58:54.007711 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:58:54.007650    2565 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-16.ec2.internal" event="NodeHasNoDiskPressure"
Sep 18 13:58:54.007711 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:58:54.007660    2565 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-16.ec2.internal" event="NodeHasSufficientPID"
Sep 18 13:58:54.008735 ip-10-0-129-16 kubenswrapper[2565]: E0918 13:58:54.008719    2565 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-16.ec2.internal\" not found"
Sep 18 13:58:54.009750 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:58:54.009738    2565 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 18 13:58:54.009903 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:58:54.009887    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-16.ec2.internal"
Sep 18 13:58:54.009966 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:58:54.009922    2565 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 18 13:58:54.010489 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:58:54.010473    2565 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-16.ec2.internal" event="NodeHasSufficientMemory"
Sep 18 13:58:54.010566 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:58:54.010505    2565 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-16.ec2.internal" event="NodeHasNoDiskPressure"
Sep 18 13:58:54.010566 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:58:54.010510    2565 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-16.ec2.internal" event="NodeHasSufficientMemory"
Sep 18 13:58:54.010566 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:58:54.010519    2565 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-16.ec2.internal" event="NodeHasSufficientPID"
Sep 18 13:58:54.010566 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:58:54.010534    2565 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-16.ec2.internal" event="NodeHasNoDiskPressure"
Sep 18 13:58:54.010566 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:58:54.010548    2565 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-16.ec2.internal" event="NodeHasSufficientPID"
Sep 18 13:58:54.012556 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:58:54.012542    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-129-16.ec2.internal"
Sep 18 13:58:54.012605 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:58:54.012568    2565 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 18 13:58:54.013236 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:58:54.013219    2565 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-16.ec2.internal" event="NodeHasSufficientMemory"
Sep 18 13:58:54.013295 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:58:54.013249    2565 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-16.ec2.internal" event="NodeHasNoDiskPressure"
Sep 18 13:58:54.013295 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:58:54.013262    2565 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-16.ec2.internal" event="NodeHasSufficientPID"
Sep 18 13:58:54.035816 ip-10-0-129-16 kubenswrapper[2565]: E0918 13:58:54.035782    2565 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-129-16.ec2.internal\" not found" node="ip-10-0-129-16.ec2.internal"
Sep 18 13:58:54.040186 ip-10-0-129-16 kubenswrapper[2565]: E0918 13:58:54.040168    2565 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-129-16.ec2.internal\" not found" node="ip-10-0-129-16.ec2.internal"
Sep 18 13:58:54.093729 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:58:54.093695    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/e264249022c87eb79103b2d4c06c5fb0-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-129-16.ec2.internal\" (UID: \"e264249022c87eb79103b2d4c06c5fb0\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-16.ec2.internal"
Sep 18 13:58:54.093729 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:58:54.093730    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/e264249022c87eb79103b2d4c06c5fb0-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-129-16.ec2.internal\" (UID: \"e264249022c87eb79103b2d4c06c5fb0\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-16.ec2.internal"
Sep 18 13:58:54.093916 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:58:54.093788    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/0092863d56373d9006f35b4f163e59f7-config\") pod \"kube-apiserver-proxy-ip-10-0-129-16.ec2.internal\" (UID: \"0092863d56373d9006f35b4f163e59f7\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-129-16.ec2.internal"
Sep 18 13:58:54.109158 ip-10-0-129-16 kubenswrapper[2565]: E0918 13:58:54.109136    2565 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-16.ec2.internal\" not found"
Sep 18 13:58:54.194257 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:58:54.194227    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/0092863d56373d9006f35b4f163e59f7-config\") pod \"kube-apiserver-proxy-ip-10-0-129-16.ec2.internal\" (UID: \"0092863d56373d9006f35b4f163e59f7\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-129-16.ec2.internal"
Sep 18 13:58:54.194257 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:58:54.194256    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/e264249022c87eb79103b2d4c06c5fb0-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-129-16.ec2.internal\" (UID: \"e264249022c87eb79103b2d4c06c5fb0\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-16.ec2.internal"
Sep 18 13:58:54.194438 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:58:54.194273    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/e264249022c87eb79103b2d4c06c5fb0-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-129-16.ec2.internal\" (UID: \"e264249022c87eb79103b2d4c06c5fb0\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-16.ec2.internal"
Sep 18 13:58:54.194438 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:58:54.194318    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/0092863d56373d9006f35b4f163e59f7-config\") pod \"kube-apiserver-proxy-ip-10-0-129-16.ec2.internal\" (UID: \"0092863d56373d9006f35b4f163e59f7\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-129-16.ec2.internal"
Sep 18 13:58:54.194438 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:58:54.194366    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/e264249022c87eb79103b2d4c06c5fb0-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-129-16.ec2.internal\" (UID: \"e264249022c87eb79103b2d4c06c5fb0\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-16.ec2.internal"
Sep 18 13:58:54.194438 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:58:54.194398    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/e264249022c87eb79103b2d4c06c5fb0-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-129-16.ec2.internal\" (UID: \"e264249022c87eb79103b2d4c06c5fb0\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-16.ec2.internal"
Sep 18 13:58:54.209363 ip-10-0-129-16 kubenswrapper[2565]: E0918 13:58:54.209329    2565 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-16.ec2.internal\" not found"
Sep 18 13:58:54.309654 ip-10-0-129-16 kubenswrapper[2565]: E0918 13:58:54.309573    2565 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-16.ec2.internal\" not found"
Sep 18 13:58:54.339290 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:58:54.339266    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-16.ec2.internal"
Sep 18 13:58:54.343214 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:58:54.343193    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-129-16.ec2.internal"
Sep 18 13:58:54.410113 ip-10-0-129-16 kubenswrapper[2565]: E0918 13:58:54.410079    2565 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-16.ec2.internal\" not found"
Sep 18 13:58:54.510603 ip-10-0-129-16 kubenswrapper[2565]: E0918 13:58:54.510572    2565 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-16.ec2.internal\" not found"
Sep 18 13:58:54.611190 ip-10-0-129-16 kubenswrapper[2565]: E0918 13:58:54.611127    2565 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-16.ec2.internal\" not found"
Sep 18 13:58:54.711868 ip-10-0-129-16 kubenswrapper[2565]: E0918 13:58:54.711836    2565 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-16.ec2.internal\" not found"
Sep 18 13:58:54.741045 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:58:54.741002    2565 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials"
Sep 18 13:58:54.741607 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:58:54.741191    2565 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received"
Sep 18 13:58:54.764767 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:58:54.764742    2565 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 18 13:58:54.811920 ip-10-0-129-16 kubenswrapper[2565]: E0918 13:58:54.811889    2565 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-16.ec2.internal\" not found"
Sep 18 13:58:54.892343 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:58:54.892230    2565 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-09-17 13:53:53 +0000 UTC" deadline="2028-05-13 21:42:26.872350373 +0000 UTC"
Sep 18 13:58:54.892343 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:58:54.892294    2565 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="14479h43m31.980060085s"
Sep 18 13:58:54.912044 ip-10-0-129-16 kubenswrapper[2565]: E0918 13:58:54.912004    2565 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-16.ec2.internal\" not found"
Sep 18 13:58:54.950426 ip-10-0-129-16 kubenswrapper[2565]: W0918 13:58:54.950378    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode264249022c87eb79103b2d4c06c5fb0.slice/crio-c534c4646c7c97f7616795fca51f913c5c2c1d5261358eb6987c8f22f9d52a71 WatchSource:0}: Error finding container c534c4646c7c97f7616795fca51f913c5c2c1d5261358eb6987c8f22f9d52a71: Status 404 returned error can't find the container with id c534c4646c7c97f7616795fca51f913c5c2c1d5261358eb6987c8f22f9d52a71
Sep 18 13:58:54.950903 ip-10-0-129-16 kubenswrapper[2565]: W0918 13:58:54.950885    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0092863d56373d9006f35b4f163e59f7.slice/crio-cd978145887cc0c2ee98d0198c2b564813353625009971706d5a70fed33ba63f WatchSource:0}: Error finding container cd978145887cc0c2ee98d0198c2b564813353625009971706d5a70fed33ba63f: Status 404 returned error can't find the container with id cd978145887cc0c2ee98d0198c2b564813353625009971706d5a70fed33ba63f
Sep 18 13:58:54.955590 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:58:54.955574    2565 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 18 13:58:54.991649 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:58:54.991626    2565 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 18 13:58:54.992798 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:58:54.992775    2565 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-16.ec2.internal" event="NodeHasSufficientMemory"
Sep 18 13:58:54.992876 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:58:54.992812    2565 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-16.ec2.internal" event="NodeHasNoDiskPressure"
Sep 18 13:58:54.992876 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:58:54.992825    2565 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-16.ec2.internal" event="NodeHasSufficientPID"
Sep 18 13:58:55.013026 ip-10-0-129-16 kubenswrapper[2565]: E0918 13:58:55.012999    2565 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-16.ec2.internal\" not found"
Sep 18 13:58:55.101705 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:58:55.101682    2565 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 18 13:58:55.113874 ip-10-0-129-16 kubenswrapper[2565]: E0918 13:58:55.113855    2565 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-16.ec2.internal\" not found"
Sep 18 13:58:55.214564 ip-10-0-129-16 kubenswrapper[2565]: E0918 13:58:55.214495    2565 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-16.ec2.internal\" not found"
Sep 18 13:58:55.245440 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:58:55.244126    2565 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 18 13:58:55.292752 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:58:55.292717    2565 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-16.ec2.internal"
Sep 18 13:58:55.306246 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:58:55.306218    2565 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Sep 18 13:58:55.307129 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:58:55.307116    2565 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-129-16.ec2.internal"
Sep 18 13:58:55.316308 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:58:55.316287    2565 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Sep 18 13:58:55.601936 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:58:55.601700    2565 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 18 13:58:55.769407 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:58:55.769330    2565 apiserver.go:52] "Watching apiserver"
Sep 18 13:58:55.779026 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:58:55.778849    2565 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66"
Sep 18 13:58:55.779330 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:58:55.779296    2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kube-system/global-pull-secret-syncer-xqrkr","openshift-cluster-node-tuning-operator/tuned-gck9k","openshift-dns/node-resolver-lqtdm","openshift-multus/multus-lqtzw","openshift-multus/network-metrics-daemon-vpgtc","openshift-network-diagnostics/network-check-target-d4rf2","openshift-ovn-kubernetes/ovnkube-node-bvqls","kube-system/konnectivity-agent-rzgjk","kube-system/kube-apiserver-proxy-ip-10-0-129-16.ec2.internal","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2jfng","openshift-image-registry/node-ca-hvhcs","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-16.ec2.internal","openshift-multus/multus-additional-cni-plugins-68wp2","openshift-network-operator/iptables-alerter-6nt4s"]
Sep 18 13:58:55.781941 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:58:55.781918    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-d4rf2"
Sep 18 13:58:55.782058 ip-10-0-129-16 kubenswrapper[2565]: E0918 13:58:55.782006    2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-d4rf2" podUID="46552e87-3933-4e45-97c0-257d31588c9d"
Sep 18 13:58:55.784316 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:58:55.784291    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-bvqls"
Sep 18 13:58:55.786858 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:58:55.786563    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-lqtdm"
Sep 18 13:58:55.789325 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:58:55.788976    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-lqtzw"
Sep 18 13:58:55.791236 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:58:55.791217    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-vpgtc"
Sep 18 13:58:55.791356 ip-10-0-129-16 kubenswrapper[2565]: E0918 13:58:55.791298    2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-vpgtc" podUID="7fe04bef-1706-4159-8a98-89a352d62c7a"
Sep 18 13:58:55.791549 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:58:55.791526    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\""
Sep 18 13:58:55.792092 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:58:55.792073    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\""
Sep 18 13:58:55.792551 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:58:55.792281    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\""
Sep 18 13:58:55.792551 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:58:55.792306    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\""
Sep 18 13:58:55.792551 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:58:55.792426    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\""
Sep 18 13:58:55.792551 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:58:55.792466    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-nb7gc\""
Sep 18 13:58:55.792551 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:58:55.792468    2565 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving"
Sep 18 13:58:55.792838 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:58:55.792565    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\""
Sep 18 13:58:55.792838 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:58:55.792716    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\""
Sep 18 13:58:55.792838 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:58:55.792727    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\""
Sep 18 13:58:55.792980 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:58:55.792848    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-g7mbb\""
Sep 18 13:58:55.792980 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:58:55.792885    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-27qgb\""
Sep 18 13:58:55.793089 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:58:55.792982    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\""
Sep 18 13:58:55.793645 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:58:55.793628    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\""
Sep 18 13:58:55.793870 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:58:55.793853    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\""
Sep 18 13:58:55.794049 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:58:55.794017    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\""
Sep 18 13:58:55.794241 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:58:55.794225    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-gck9k"
Sep 18 13:58:55.796925 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:58:55.796812    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-xqrkr"
Sep 18 13:58:55.796925 ip-10-0-129-16 kubenswrapper[2565]: E0918 13:58:55.796888    2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-xqrkr" podUID="d83ca837-1f57-4ff5-a513-a35f36198a99"
Sep 18 13:58:55.800290 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:58:55.800075    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-rzgjk"
Sep 18 13:58:55.803096 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:58:55.803079    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2jfng"
Sep 18 13:58:55.804170 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:58:55.803980    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\""
Sep 18 13:58:55.804170 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:58:55.804047    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-rd465\""
Sep 18 13:58:55.804170 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:58:55.804118    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-v6h9z\""
Sep 18 13:58:55.804377 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:58:55.804250    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\""
Sep 18 13:58:55.804432 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:58:55.804399    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\""
Sep 18 13:58:55.804590 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:58:55.804569    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\""
Sep 18 13:58:55.804983 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:58:55.804808    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/f2b3025c-cb1e-49f1-8cde-431f4b0810f0-host-var-lib-kubelet\") pod \"multus-lqtzw\" (UID: \"f2b3025c-cb1e-49f1-8cde-431f4b0810f0\") " pod="openshift-multus/multus-lqtzw"
Sep 18 13:58:55.804983 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:58:55.804839    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/f2b3025c-cb1e-49f1-8cde-431f4b0810f0-multus-conf-dir\") pod \"multus-lqtzw\" (UID: \"f2b3025c-cb1e-49f1-8cde-431f4b0810f0\") " pod="openshift-multus/multus-lqtzw"
Sep 18 13:58:55.804983 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:58:55.804854    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/f2b3025c-cb1e-49f1-8cde-431f4b0810f0-host-run-multus-certs\") pod \"multus-lqtzw\" (UID: \"f2b3025c-cb1e-49f1-8cde-431f4b0810f0\") " pod="openshift-multus/multus-lqtzw"
Sep 18 13:58:55.804983 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:58:55.804874    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-g6zrg\" (UniqueName: \"kubernetes.io/projected/f2b3025c-cb1e-49f1-8cde-431f4b0810f0-kube-api-access-g6zrg\") pod \"multus-lqtzw\" (UID: \"f2b3025c-cb1e-49f1-8cde-431f4b0810f0\") " pod="openshift-multus/multus-lqtzw"
Sep 18 13:58:55.804983 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:58:55.804901    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/5d21c619-2150-4bdc-9ce1-275f65d68d5a-var-lib-kubelet\") pod \"tuned-gck9k\" (UID: \"5d21c619-2150-4bdc-9ce1-275f65d68d5a\") " pod="openshift-cluster-node-tuning-operator/tuned-gck9k"
Sep 18 13:58:55.804983 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:58:55.804933    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/0b0463f5-d647-4a38-b752-44afba77daff-systemd-units\") pod \"ovnkube-node-bvqls\" (UID: \"0b0463f5-d647-4a38-b752-44afba77daff\") " pod="openshift-ovn-kubernetes/ovnkube-node-bvqls"
Sep 18 13:58:55.804983 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:58:55.804970    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/5d21c619-2150-4bdc-9ce1-275f65d68d5a-etc-modprobe-d\") pod \"tuned-gck9k\" (UID: \"5d21c619-2150-4bdc-9ce1-275f65d68d5a\") " pod="openshift-cluster-node-tuning-operator/tuned-gck9k"
Sep 18 13:58:55.805428 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:58:55.804997    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/5d21c619-2150-4bdc-9ce1-275f65d68d5a-etc-systemd\") pod \"tuned-gck9k\" (UID: \"5d21c619-2150-4bdc-9ce1-275f65d68d5a\") " pod="openshift-cluster-node-tuning-operator/tuned-gck9k"
Sep 18 13:58:55.805428 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:58:55.805020    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/5d21c619-2150-4bdc-9ce1-275f65d68d5a-lib-modules\") pod \"tuned-gck9k\" (UID: \"5d21c619-2150-4bdc-9ce1-275f65d68d5a\") " pod="openshift-cluster-node-tuning-operator/tuned-gck9k"
Sep 18 13:58:55.805428 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:58:55.805057    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gqntj\" (UniqueName: \"kubernetes.io/projected/46552e87-3933-4e45-97c0-257d31588c9d-kube-api-access-gqntj\") pod \"network-check-target-d4rf2\" (UID: \"46552e87-3933-4e45-97c0-257d31588c9d\") " pod="openshift-network-diagnostics/network-check-target-d4rf2"
Sep 18 13:58:55.805428 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:58:55.805082    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/0b0463f5-d647-4a38-b752-44afba77daff-host-slash\") pod \"ovnkube-node-bvqls\" (UID: \"0b0463f5-d647-4a38-b752-44afba77daff\") " pod="openshift-ovn-kubernetes/ovnkube-node-bvqls"
Sep 18 13:58:55.805428 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:58:55.805114    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/0b0463f5-d647-4a38-b752-44afba77daff-log-socket\") pod \"ovnkube-node-bvqls\" (UID: \"0b0463f5-d647-4a38-b752-44afba77daff\") " pod="openshift-ovn-kubernetes/ovnkube-node-bvqls"
Sep 18 13:58:55.805428 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:58:55.805138    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qhrxx\" (UniqueName: \"kubernetes.io/projected/2044301a-3777-480b-9c7e-cdca3c1eb61a-kube-api-access-qhrxx\") pod \"node-resolver-lqtdm\" (UID: \"2044301a-3777-480b-9c7e-cdca3c1eb61a\") " pod="openshift-dns/node-resolver-lqtdm"
Sep 18 13:58:55.805428 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:58:55.805168    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/f2b3025c-cb1e-49f1-8cde-431f4b0810f0-cni-binary-copy\") pod \"multus-lqtzw\" (UID: \"f2b3025c-cb1e-49f1-8cde-431f4b0810f0\") " pod="openshift-multus/multus-lqtzw"
Sep 18 13:58:55.805428 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:58:55.805195    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/5d21c619-2150-4bdc-9ce1-275f65d68d5a-etc-tuned\") pod \"tuned-gck9k\" (UID: \"5d21c619-2150-4bdc-9ce1-275f65d68d5a\") " pod="openshift-cluster-node-tuning-operator/tuned-gck9k"
Sep 18 13:58:55.805428 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:58:55.805222    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/f2b3025c-cb1e-49f1-8cde-431f4b0810f0-etc-kubernetes\") pod \"multus-lqtzw\" (UID: \"f2b3025c-cb1e-49f1-8cde-431f4b0810f0\") " pod="openshift-multus/multus-lqtzw"
Sep 18 13:58:55.805428 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:58:55.805247    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7fe04bef-1706-4159-8a98-89a352d62c7a-metrics-certs\") pod \"network-metrics-daemon-vpgtc\" (UID: \"7fe04bef-1706-4159-8a98-89a352d62c7a\") " pod="openshift-multus/network-metrics-daemon-vpgtc"
Sep 18 13:58:55.805428 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:58:55.805275    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/0b0463f5-d647-4a38-b752-44afba77daff-etc-openvswitch\") pod \"ovnkube-node-bvqls\" (UID: \"0b0463f5-d647-4a38-b752-44afba77daff\") " pod="openshift-ovn-kubernetes/ovnkube-node-bvqls"
Sep 18 13:58:55.805428 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:58:55.805299    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/f2b3025c-cb1e-49f1-8cde-431f4b0810f0-multus-socket-dir-parent\") pod \"multus-lqtzw\" (UID: \"f2b3025c-cb1e-49f1-8cde-431f4b0810f0\") " pod="openshift-multus/multus-lqtzw"
Sep 18 13:58:55.805428 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:58:55.805324    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/f2b3025c-cb1e-49f1-8cde-431f4b0810f0-host-var-lib-cni-multus\") pod \"multus-lqtzw\" (UID: \"f2b3025c-cb1e-49f1-8cde-431f4b0810f0\") " pod="openshift-multus/multus-lqtzw"
Sep 18 13:58:55.805428 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:58:55.805398    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/f2b3025c-cb1e-49f1-8cde-431f4b0810f0-multus-daemon-config\") pod \"multus-lqtzw\" (UID: \"f2b3025c-cb1e-49f1-8cde-431f4b0810f0\") " pod="openshift-multus/multus-lqtzw"
Sep 18 13:58:55.806067 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:58:55.805434    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/0b0463f5-d647-4a38-b752-44afba77daff-run-openvswitch\") pod \"ovnkube-node-bvqls\" (UID: \"0b0463f5-d647-4a38-b752-44afba77daff\") " pod="openshift-ovn-kubernetes/ovnkube-node-bvqls"
Sep 18 13:58:55.806067 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:58:55.805472    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/0b0463f5-d647-4a38-b752-44afba77daff-host-run-ovn-kubernetes\") pod \"ovnkube-node-bvqls\" (UID: \"0b0463f5-d647-4a38-b752-44afba77daff\") " pod="openshift-ovn-kubernetes/ovnkube-node-bvqls"
Sep 18 13:58:55.806067 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:58:55.805513    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/5d21c619-2150-4bdc-9ce1-275f65d68d5a-run\") pod \"tuned-gck9k\" (UID: \"5d21c619-2150-4bdc-9ce1-275f65d68d5a\") " pod="openshift-cluster-node-tuning-operator/tuned-gck9k"
Sep 18 13:58:55.806067 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:58:55.805543    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-d4xvb\" (UniqueName: \"kubernetes.io/projected/7fe04bef-1706-4159-8a98-89a352d62c7a-kube-api-access-d4xvb\") pod \"network-metrics-daemon-vpgtc\" (UID: \"7fe04bef-1706-4159-8a98-89a352d62c7a\") " pod="openshift-multus/network-metrics-daemon-vpgtc"
Sep 18 13:58:55.806067 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:58:55.805568    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/5d21c619-2150-4bdc-9ce1-275f65d68d5a-etc-sysctl-d\") pod \"tuned-gck9k\" (UID: \"5d21c619-2150-4bdc-9ce1-275f65d68d5a\") " pod="openshift-cluster-node-tuning-operator/tuned-gck9k"
Sep 18 13:58:55.806067 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:58:55.805591    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/5d21c619-2150-4bdc-9ce1-275f65d68d5a-etc-sysconfig\") pod \"tuned-gck9k\" (UID: \"5d21c619-2150-4bdc-9ce1-275f65d68d5a\") " pod="openshift-cluster-node-tuning-operator/tuned-gck9k"
Sep 18 13:58:55.806067 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:58:55.805616    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/5d21c619-2150-4bdc-9ce1-275f65d68d5a-etc-sysctl-conf\") pod \"tuned-gck9k\" (UID: \"5d21c619-2150-4bdc-9ce1-275f65d68d5a\") " pod="openshift-cluster-node-tuning-operator/tuned-gck9k"
Sep 18 13:58:55.806067 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:58:55.805658    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/5d21c619-2150-4bdc-9ce1-275f65d68d5a-host\") pod \"tuned-gck9k\" (UID: \"5d21c619-2150-4bdc-9ce1-275f65d68d5a\") " pod="openshift-cluster-node-tuning-operator/tuned-gck9k"
Sep 18 13:58:55.806067 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:58:55.805684    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/0b0463f5-d647-4a38-b752-44afba77daff-run-systemd\") pod \"ovnkube-node-bvqls\" (UID: \"0b0463f5-d647-4a38-b752-44afba77daff\") " pod="openshift-ovn-kubernetes/ovnkube-node-bvqls"
Sep 18 13:58:55.806067 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:58:55.805708    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/0b0463f5-d647-4a38-b752-44afba77daff-ovnkube-config\") pod \"ovnkube-node-bvqls\" (UID: \"0b0463f5-d647-4a38-b752-44afba77daff\") " pod="openshift-ovn-kubernetes/ovnkube-node-bvqls"
Sep 18 13:58:55.806067 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:58:55.805733    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/0b0463f5-d647-4a38-b752-44afba77daff-node-log\") pod \"ovnkube-node-bvqls\" (UID: \"0b0463f5-d647-4a38-b752-44afba77daff\") " pod="openshift-ovn-kubernetes/ovnkube-node-bvqls"
Sep 18 13:58:55.806067 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:58:55.805785    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/f2b3025c-cb1e-49f1-8cde-431f4b0810f0-system-cni-dir\") pod \"multus-lqtzw\" (UID: \"f2b3025c-cb1e-49f1-8cde-431f4b0810f0\") " pod="openshift-multus/multus-lqtzw"
Sep 18 13:58:55.806067 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:58:55.805817    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/f2b3025c-cb1e-49f1-8cde-431f4b0810f0-multus-cni-dir\") pod \"multus-lqtzw\" (UID: \"f2b3025c-cb1e-49f1-8cde-431f4b0810f0\") " pod="openshift-multus/multus-lqtzw"
Sep 18 13:58:55.806067 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:58:55.805852    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/f2b3025c-cb1e-49f1-8cde-431f4b0810f0-host-var-lib-cni-bin\") pod \"multus-lqtzw\" (UID: \"f2b3025c-cb1e-49f1-8cde-431f4b0810f0\") " pod="openshift-multus/multus-lqtzw"
Sep 18 13:58:55.806067 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:58:55.805882    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/5d21c619-2150-4bdc-9ce1-275f65d68d5a-tmp\") pod \"tuned-gck9k\" (UID: \"5d21c619-2150-4bdc-9ce1-275f65d68d5a\") " pod="openshift-cluster-node-tuning-operator/tuned-gck9k"
Sep 18 13:58:55.806067 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:58:55.805910    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/0b0463f5-d647-4a38-b752-44afba77daff-ovnkube-script-lib\") pod \"ovnkube-node-bvqls\" (UID: \"0b0463f5-d647-4a38-b752-44afba77daff\") " pod="openshift-ovn-kubernetes/ovnkube-node-bvqls"
Sep 18 13:58:55.806067 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:58:55.805933    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/0b0463f5-d647-4a38-b752-44afba77daff-host-cni-bin\") pod \"ovnkube-node-bvqls\" (UID: \"0b0463f5-d647-4a38-b752-44afba77daff\") " pod="openshift-ovn-kubernetes/ovnkube-node-bvqls"
Sep 18 13:58:55.806915 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:58:55.805952    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-hvhcs"
Sep 18 13:58:55.806915 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:58:55.805953    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/5d21c619-2150-4bdc-9ce1-275f65d68d5a-sys\") pod \"tuned-gck9k\" (UID: \"5d21c619-2150-4bdc-9ce1-275f65d68d5a\") " pod="openshift-cluster-node-tuning-operator/tuned-gck9k"
Sep 18 13:58:55.806915 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:58:55.806094    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/0b0463f5-d647-4a38-b752-44afba77daff-host-cni-netd\") pod \"ovnkube-node-bvqls\" (UID: \"0b0463f5-d647-4a38-b752-44afba77daff\") " pod="openshift-ovn-kubernetes/ovnkube-node-bvqls"
Sep 18 13:58:55.806915 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:58:55.806122    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/0b0463f5-d647-4a38-b752-44afba77daff-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-bvqls\" (UID: \"0b0463f5-d647-4a38-b752-44afba77daff\") " pod="openshift-ovn-kubernetes/ovnkube-node-bvqls"
Sep 18 13:58:55.806915 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:58:55.806151    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2044301a-3777-480b-9c7e-cdca3c1eb61a-tmp-dir\") pod \"node-resolver-lqtdm\" (UID: \"2044301a-3777-480b-9c7e-cdca3c1eb61a\") " pod="openshift-dns/node-resolver-lqtdm"
Sep 18 13:58:55.806915 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:58:55.806177    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/f2b3025c-cb1e-49f1-8cde-431f4b0810f0-host-run-netns\") pod \"multus-lqtzw\" (UID: \"f2b3025c-cb1e-49f1-8cde-431f4b0810f0\") " pod="openshift-multus/multus-lqtzw"
Sep 18 13:58:55.806915 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:58:55.806199    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/0b0463f5-d647-4a38-b752-44afba77daff-host-kubelet\") pod \"ovnkube-node-bvqls\" (UID: \"0b0463f5-d647-4a38-b752-44afba77daff\") " pod="openshift-ovn-kubernetes/ovnkube-node-bvqls"
Sep 18 13:58:55.806915 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:58:55.806222    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/0b0463f5-d647-4a38-b752-44afba77daff-var-lib-openvswitch\") pod \"ovnkube-node-bvqls\" (UID: \"0b0463f5-d647-4a38-b752-44afba77daff\") " pod="openshift-ovn-kubernetes/ovnkube-node-bvqls"
Sep 18 13:58:55.806915 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:58:55.806237    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/0b0463f5-d647-4a38-b752-44afba77daff-run-ovn\") pod \"ovnkube-node-bvqls\" (UID: \"0b0463f5-d647-4a38-b752-44afba77daff\") " pod="openshift-ovn-kubernetes/ovnkube-node-bvqls"
Sep 18 13:58:55.806915 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:58:55.806250    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/0b0463f5-d647-4a38-b752-44afba77daff-env-overrides\") pod \"ovnkube-node-bvqls\" (UID: \"0b0463f5-d647-4a38-b752-44afba77daff\") " pod="openshift-ovn-kubernetes/ovnkube-node-bvqls"
Sep 18 13:58:55.806915 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:58:55.806263    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/0b0463f5-d647-4a38-b752-44afba77daff-ovn-node-metrics-cert\") pod \"ovnkube-node-bvqls\" (UID: \"0b0463f5-d647-4a38-b752-44afba77daff\") " pod="openshift-ovn-kubernetes/ovnkube-node-bvqls"
Sep 18 13:58:55.806915 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:58:55.806292    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/2044301a-3777-480b-9c7e-cdca3c1eb61a-hosts-file\") pod \"node-resolver-lqtdm\" (UID: \"2044301a-3777-480b-9c7e-cdca3c1eb61a\") " pod="openshift-dns/node-resolver-lqtdm"
Sep 18 13:58:55.806915 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:58:55.806307    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hf6bl\" (UniqueName: \"kubernetes.io/projected/5d21c619-2150-4bdc-9ce1-275f65d68d5a-kube-api-access-hf6bl\") pod \"tuned-gck9k\" (UID: \"5d21c619-2150-4bdc-9ce1-275f65d68d5a\") " pod="openshift-cluster-node-tuning-operator/tuned-gck9k"
Sep 18 13:58:55.806915 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:58:55.806321    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/0b0463f5-d647-4a38-b752-44afba77daff-host-run-netns\") pod \"ovnkube-node-bvqls\" (UID: \"0b0463f5-d647-4a38-b752-44afba77daff\") " pod="openshift-ovn-kubernetes/ovnkube-node-bvqls"
Sep 18 13:58:55.806915 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:58:55.806350    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dltvg\" (UniqueName: \"kubernetes.io/projected/0b0463f5-d647-4a38-b752-44afba77daff-kube-api-access-dltvg\") pod \"ovnkube-node-bvqls\" (UID: \"0b0463f5-d647-4a38-b752-44afba77daff\") " pod="openshift-ovn-kubernetes/ovnkube-node-bvqls"
Sep 18 13:58:55.806915 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:58:55.806364    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/f2b3025c-cb1e-49f1-8cde-431f4b0810f0-cnibin\") pod \"multus-lqtzw\" (UID: \"f2b3025c-cb1e-49f1-8cde-431f4b0810f0\") " pod="openshift-multus/multus-lqtzw"
Sep 18 13:58:55.806915 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:58:55.806378    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/f2b3025c-cb1e-49f1-8cde-431f4b0810f0-hostroot\") pod \"multus-lqtzw\" (UID: \"f2b3025c-cb1e-49f1-8cde-431f4b0810f0\") " pod="openshift-multus/multus-lqtzw"
Sep 18 13:58:55.807641 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:58:55.806392    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/f2b3025c-cb1e-49f1-8cde-431f4b0810f0-host-run-k8s-cni-cncf-io\") pod \"multus-lqtzw\" (UID: \"f2b3025c-cb1e-49f1-8cde-431f4b0810f0\") " pod="openshift-multus/multus-lqtzw"
Sep 18 13:58:55.807641 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:58:55.806414    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/5d21c619-2150-4bdc-9ce1-275f65d68d5a-etc-kubernetes\") pod \"tuned-gck9k\" (UID: \"5d21c619-2150-4bdc-9ce1-275f65d68d5a\") " pod="openshift-cluster-node-tuning-operator/tuned-gck9k"
Sep 18 13:58:55.807641 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:58:55.806491    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\""
Sep 18 13:58:55.808551 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:58:55.808350    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-68wp2"
Sep 18 13:58:55.809800 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:58:55.809383    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\""
Sep 18 13:58:55.810182 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:58:55.810106    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-d8w8d\""
Sep 18 13:58:55.811771 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:58:55.810444    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\""
Sep 18 13:58:55.811771 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:58:55.810469    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\""
Sep 18 13:58:55.811771 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:58:55.810727    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\""
Sep 18 13:58:55.811771 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:58:55.810800    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\""
Sep 18 13:58:55.811771 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:58:55.810935    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\""
Sep 18 13:58:55.811771 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:58:55.811513    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-zcwsc\""
Sep 18 13:58:55.811771 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:58:55.811637    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-zlvr9\""
Sep 18 13:58:55.811771 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:58:55.811643    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\""
Sep 18 13:58:55.813163 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:58:55.812801    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-6nt4s"
Sep 18 13:58:55.820241 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:58:55.820218    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-h4j87\""
Sep 18 13:58:55.822284 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:58:55.822252    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\""
Sep 18 13:58:55.822396 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:58:55.822289    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\""
Sep 18 13:58:55.822584 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:58:55.822564    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\""
Sep 18 13:58:55.830629 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:58:55.830605    2565 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Sep 18 13:58:55.889564 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:58:55.889527    2565 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-kt858"
Sep 18 13:58:55.894322 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:58:55.894297    2565 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world"
Sep 18 13:58:55.906322 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:58:55.906294    2565 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-kt858"
Sep 18 13:58:55.907916 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:58:55.907580    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/32ca2a66-ab18-447c-8120-7324ae6f94e4-socket-dir\") pod \"aws-ebs-csi-driver-node-2jfng\" (UID: \"32ca2a66-ab18-447c-8120-7324ae6f94e4\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2jfng"
Sep 18 13:58:55.907916 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:58:55.907624    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/f2b3025c-cb1e-49f1-8cde-431f4b0810f0-host-run-netns\") pod \"multus-lqtzw\" (UID: \"f2b3025c-cb1e-49f1-8cde-431f4b0810f0\") " pod="openshift-multus/multus-lqtzw"
Sep 18 13:58:55.907916 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:58:55.907651    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/0b0463f5-d647-4a38-b752-44afba77daff-host-kubelet\") pod \"ovnkube-node-bvqls\" (UID: \"0b0463f5-d647-4a38-b752-44afba77daff\") " pod="openshift-ovn-kubernetes/ovnkube-node-bvqls"
Sep 18 13:58:55.907916 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:58:55.907674    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/0b0463f5-d647-4a38-b752-44afba77daff-var-lib-openvswitch\") pod \"ovnkube-node-bvqls\" (UID: \"0b0463f5-d647-4a38-b752-44afba77daff\") " pod="openshift-ovn-kubernetes/ovnkube-node-bvqls"
Sep 18 13:58:55.907916 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:58:55.907725    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/0b0463f5-d647-4a38-b752-44afba77daff-var-lib-openvswitch\") pod \"ovnkube-node-bvqls\" (UID: \"0b0463f5-d647-4a38-b752-44afba77daff\") " pod="openshift-ovn-kubernetes/ovnkube-node-bvqls"
Sep 18 13:58:55.907916 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:58:55.907728    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/0b0463f5-d647-4a38-b752-44afba77daff-run-ovn\") pod \"ovnkube-node-bvqls\" (UID: \"0b0463f5-d647-4a38-b752-44afba77daff\") " pod="openshift-ovn-kubernetes/ovnkube-node-bvqls"
Sep 18 13:58:55.907916 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:58:55.907767    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/f2b3025c-cb1e-49f1-8cde-431f4b0810f0-host-run-netns\") pod \"multus-lqtzw\" (UID: \"f2b3025c-cb1e-49f1-8cde-431f4b0810f0\") " pod="openshift-multus/multus-lqtzw"
Sep 18 13:58:55.907916 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:58:55.907792    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/0b0463f5-d647-4a38-b752-44afba77daff-host-kubelet\") pod \"ovnkube-node-bvqls\" (UID: \"0b0463f5-d647-4a38-b752-44afba77daff\") " pod="openshift-ovn-kubernetes/ovnkube-node-bvqls"
Sep 18 13:58:55.907916 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:58:55.907796    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/0b0463f5-d647-4a38-b752-44afba77daff-run-ovn\") pod \"ovnkube-node-bvqls\" (UID: \"0b0463f5-d647-4a38-b752-44afba77daff\") " pod="openshift-ovn-kubernetes/ovnkube-node-bvqls"
Sep 18 13:58:55.907916 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:58:55.907846    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/0b0463f5-d647-4a38-b752-44afba77daff-env-overrides\") pod \"ovnkube-node-bvqls\" (UID: \"0b0463f5-d647-4a38-b752-44afba77daff\") " pod="openshift-ovn-kubernetes/ovnkube-node-bvqls"
Sep 18 13:58:55.907916 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:58:55.907878    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/0b0463f5-d647-4a38-b752-44afba77daff-ovn-node-metrics-cert\") pod \"ovnkube-node-bvqls\" (UID: \"0b0463f5-d647-4a38-b752-44afba77daff\") " pod="openshift-ovn-kubernetes/ovnkube-node-bvqls"
Sep 18 13:58:55.907916 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:58:55.907905    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/2044301a-3777-480b-9c7e-cdca3c1eb61a-hosts-file\") pod \"node-resolver-lqtdm\" (UID: \"2044301a-3777-480b-9c7e-cdca3c1eb61a\") " pod="openshift-dns/node-resolver-lqtdm"
Sep 18 13:58:55.908582 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:58:55.907936    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/32ca2a66-ab18-447c-8120-7324ae6f94e4-sys-fs\") pod \"aws-ebs-csi-driver-node-2jfng\" (UID: \"32ca2a66-ab18-447c-8120-7324ae6f94e4\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2jfng"
Sep 18 13:58:55.908582 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:58:55.907966    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hf6bl\" (UniqueName: \"kubernetes.io/projected/5d21c619-2150-4bdc-9ce1-275f65d68d5a-kube-api-access-hf6bl\") pod \"tuned-gck9k\" (UID: \"5d21c619-2150-4bdc-9ce1-275f65d68d5a\") " pod="openshift-cluster-node-tuning-operator/tuned-gck9k"
Sep 18 13:58:55.908582 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:58:55.907996    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/0b0463f5-d647-4a38-b752-44afba77daff-host-run-netns\") pod \"ovnkube-node-bvqls\" (UID: \"0b0463f5-d647-4a38-b752-44afba77daff\") " pod="openshift-ovn-kubernetes/ovnkube-node-bvqls"
Sep 18 13:58:55.908582 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:58:55.908022    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dltvg\" (UniqueName: \"kubernetes.io/projected/0b0463f5-d647-4a38-b752-44afba77daff-kube-api-access-dltvg\") pod \"ovnkube-node-bvqls\" (UID: \"0b0463f5-d647-4a38-b752-44afba77daff\") " pod="openshift-ovn-kubernetes/ovnkube-node-bvqls"
Sep 18 13:58:55.908582 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:58:55.908051    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/2044301a-3777-480b-9c7e-cdca3c1eb61a-hosts-file\") pod \"node-resolver-lqtdm\" (UID: \"2044301a-3777-480b-9c7e-cdca3c1eb61a\") " pod="openshift-dns/node-resolver-lqtdm"
Sep 18 13:58:55.908582 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:58:55.908070    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/19a8e259-6225-41e6-9556-f2439e36171d-tuning-conf-dir\") pod \"multus-additional-cni-plugins-68wp2\" (UID: \"19a8e259-6225-41e6-9556-f2439e36171d\") " pod="openshift-multus/multus-additional-cni-plugins-68wp2"
Sep 18 13:58:55.908582 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:58:55.908099    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5tmmv\" (UniqueName: \"kubernetes.io/projected/32ca2a66-ab18-447c-8120-7324ae6f94e4-kube-api-access-5tmmv\") pod \"aws-ebs-csi-driver-node-2jfng\" (UID: \"32ca2a66-ab18-447c-8120-7324ae6f94e4\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2jfng"
Sep 18 13:58:55.908582 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:58:55.908136    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/f2b3025c-cb1e-49f1-8cde-431f4b0810f0-cnibin\") pod \"multus-lqtzw\" (UID: \"f2b3025c-cb1e-49f1-8cde-431f4b0810f0\") " pod="openshift-multus/multus-lqtzw"
Sep 18 13:58:55.908582 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:58:55.908175    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/f2b3025c-cb1e-49f1-8cde-431f4b0810f0-hostroot\") pod \"multus-lqtzw\" (UID: \"f2b3025c-cb1e-49f1-8cde-431f4b0810f0\") " pod="openshift-multus/multus-lqtzw"
Sep 18 13:58:55.908582 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:58:55.908225    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/f2b3025c-cb1e-49f1-8cde-431f4b0810f0-hostroot\") pod \"multus-lqtzw\" (UID: \"f2b3025c-cb1e-49f1-8cde-431f4b0810f0\") " pod="openshift-multus/multus-lqtzw"
Sep 18 13:58:55.908582 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:58:55.908235    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/0b0463f5-d647-4a38-b752-44afba77daff-host-run-netns\") pod \"ovnkube-node-bvqls\" (UID: \"0b0463f5-d647-4a38-b752-44afba77daff\") " pod="openshift-ovn-kubernetes/ovnkube-node-bvqls"
Sep 18 13:58:55.908582 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:58:55.908303    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/f2b3025c-cb1e-49f1-8cde-431f4b0810f0-cnibin\") pod \"multus-lqtzw\" (UID: \"f2b3025c-cb1e-49f1-8cde-431f4b0810f0\") " pod="openshift-multus/multus-lqtzw"
Sep 18 13:58:55.908582 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:58:55.908363    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/7623391e-952a-4649-81ad-9c59628e3cbf-iptables-alerter-script\") pod \"iptables-alerter-6nt4s\" (UID: \"7623391e-952a-4649-81ad-9c59628e3cbf\") " pod="openshift-network-operator/iptables-alerter-6nt4s"
Sep 18 13:58:55.908582 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:58:55.908417    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/f2b3025c-cb1e-49f1-8cde-431f4b0810f0-host-run-k8s-cni-cncf-io\") pod \"multus-lqtzw\" (UID: \"f2b3025c-cb1e-49f1-8cde-431f4b0810f0\") " pod="openshift-multus/multus-lqtzw"
Sep 18 13:58:55.908582 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:58:55.908420    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/0b0463f5-d647-4a38-b752-44afba77daff-env-overrides\") pod \"ovnkube-node-bvqls\" (UID: \"0b0463f5-d647-4a38-b752-44afba77daff\") " pod="openshift-ovn-kubernetes/ovnkube-node-bvqls"
Sep 18 13:58:55.908582 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:58:55.908455    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/5d21c619-2150-4bdc-9ce1-275f65d68d5a-etc-kubernetes\") pod \"tuned-gck9k\" (UID: \"5d21c619-2150-4bdc-9ce1-275f65d68d5a\") " pod="openshift-cluster-node-tuning-operator/tuned-gck9k"
Sep 18 13:58:55.908582 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:58:55.908484    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/f2b3025c-cb1e-49f1-8cde-431f4b0810f0-host-run-k8s-cni-cncf-io\") pod \"multus-lqtzw\" (UID: \"f2b3025c-cb1e-49f1-8cde-431f4b0810f0\") " pod="openshift-multus/multus-lqtzw"
Sep 18 13:58:55.909401 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:58:55.908531    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/19a8e259-6225-41e6-9556-f2439e36171d-system-cni-dir\") pod \"multus-additional-cni-plugins-68wp2\" (UID: \"19a8e259-6225-41e6-9556-f2439e36171d\") " pod="openshift-multus/multus-additional-cni-plugins-68wp2"
Sep 18 13:58:55.909401 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:58:55.908580    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/bff1008a-4c27-4278-969d-8974600e696f-agent-certs\") pod \"konnectivity-agent-rzgjk\" (UID: \"bff1008a-4c27-4278-969d-8974600e696f\") " pod="kube-system/konnectivity-agent-rzgjk"
Sep 18 13:58:55.909401 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:58:55.908585    2565 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory"
Sep 18 13:58:55.909401 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:58:55.908612    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/75c7d00d-fb1a-4716-9963-b7ef1c0b35bd-serviceca\") pod \"node-ca-hvhcs\" (UID: \"75c7d00d-fb1a-4716-9963-b7ef1c0b35bd\") " pod="openshift-image-registry/node-ca-hvhcs"
Sep 18 13:58:55.909401 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:58:55.908659    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/5d21c619-2150-4bdc-9ce1-275f65d68d5a-etc-kubernetes\") pod \"tuned-gck9k\" (UID: \"5d21c619-2150-4bdc-9ce1-275f65d68d5a\") " pod="openshift-cluster-node-tuning-operator/tuned-gck9k"
Sep 18 13:58:55.909401 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:58:55.908668    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/f2b3025c-cb1e-49f1-8cde-431f4b0810f0-host-var-lib-kubelet\") pod \"multus-lqtzw\" (UID: \"f2b3025c-cb1e-49f1-8cde-431f4b0810f0\") " pod="openshift-multus/multus-lqtzw"
Sep 18 13:58:55.909401 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:58:55.908697    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/f2b3025c-cb1e-49f1-8cde-431f4b0810f0-multus-conf-dir\") pod \"multus-lqtzw\" (UID: \"f2b3025c-cb1e-49f1-8cde-431f4b0810f0\") " pod="openshift-multus/multus-lqtzw"
Sep 18 13:58:55.909401 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:58:55.908721    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/f2b3025c-cb1e-49f1-8cde-431f4b0810f0-host-var-lib-kubelet\") pod \"multus-lqtzw\" (UID: \"f2b3025c-cb1e-49f1-8cde-431f4b0810f0\") " pod="openshift-multus/multus-lqtzw"
Sep 18 13:58:55.909401 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:58:55.908723    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/f2b3025c-cb1e-49f1-8cde-431f4b0810f0-host-run-multus-certs\") pod \"multus-lqtzw\" (UID: \"f2b3025c-cb1e-49f1-8cde-431f4b0810f0\") " pod="openshift-multus/multus-lqtzw"
Sep 18 13:58:55.909401 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:58:55.908758    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-g6zrg\" (UniqueName: \"kubernetes.io/projected/f2b3025c-cb1e-49f1-8cde-431f4b0810f0-kube-api-access-g6zrg\") pod \"multus-lqtzw\" (UID: \"f2b3025c-cb1e-49f1-8cde-431f4b0810f0\") " pod="openshift-multus/multus-lqtzw"
Sep 18 13:58:55.909401 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:58:55.908763    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/f2b3025c-cb1e-49f1-8cde-431f4b0810f0-multus-conf-dir\") pod \"multus-lqtzw\" (UID: \"f2b3025c-cb1e-49f1-8cde-431f4b0810f0\") " pod="openshift-multus/multus-lqtzw"
Sep 18 13:58:55.909401 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:58:55.908785    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/5d21c619-2150-4bdc-9ce1-275f65d68d5a-var-lib-kubelet\") pod \"tuned-gck9k\" (UID: \"5d21c619-2150-4bdc-9ce1-275f65d68d5a\") " pod="openshift-cluster-node-tuning-operator/tuned-gck9k"
Sep 18 13:58:55.909401 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:58:55.908810    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/f2b3025c-cb1e-49f1-8cde-431f4b0810f0-host-run-multus-certs\") pod \"multus-lqtzw\" (UID: \"f2b3025c-cb1e-49f1-8cde-431f4b0810f0\") " pod="openshift-multus/multus-lqtzw"
Sep 18 13:58:55.909401 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:58:55.908812    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/0b0463f5-d647-4a38-b752-44afba77daff-systemd-units\") pod \"ovnkube-node-bvqls\" (UID: \"0b0463f5-d647-4a38-b752-44afba77daff\") " pod="openshift-ovn-kubernetes/ovnkube-node-bvqls"
Sep 18 13:58:55.909401 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:58:55.908859    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/5d21c619-2150-4bdc-9ce1-275f65d68d5a-etc-modprobe-d\") pod \"tuned-gck9k\" (UID: \"5d21c619-2150-4bdc-9ce1-275f65d68d5a\") " pod="openshift-cluster-node-tuning-operator/tuned-gck9k"
Sep 18 13:58:55.909401 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:58:55.908861    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/5d21c619-2150-4bdc-9ce1-275f65d68d5a-var-lib-kubelet\") pod \"tuned-gck9k\" (UID: \"5d21c619-2150-4bdc-9ce1-275f65d68d5a\") " pod="openshift-cluster-node-tuning-operator/tuned-gck9k"
Sep 18 13:58:55.909401 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:58:55.908894    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/0b0463f5-d647-4a38-b752-44afba77daff-systemd-units\") pod \"ovnkube-node-bvqls\" (UID: \"0b0463f5-d647-4a38-b752-44afba77daff\") " pod="openshift-ovn-kubernetes/ovnkube-node-bvqls"
Sep 18 13:58:55.909401 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:58:55.909003    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/5d21c619-2150-4bdc-9ce1-275f65d68d5a-etc-systemd\") pod \"tuned-gck9k\" (UID: \"5d21c619-2150-4bdc-9ce1-275f65d68d5a\") " pod="openshift-cluster-node-tuning-operator/tuned-gck9k"
Sep 18 13:58:55.910216 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:58:55.909045    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/5d21c619-2150-4bdc-9ce1-275f65d68d5a-etc-modprobe-d\") pod \"tuned-gck9k\" (UID: \"5d21c619-2150-4bdc-9ce1-275f65d68d5a\") " pod="openshift-cluster-node-tuning-operator/tuned-gck9k"
Sep 18 13:58:55.910216 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:58:55.909053    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/5d21c619-2150-4bdc-9ce1-275f65d68d5a-lib-modules\") pod \"tuned-gck9k\" (UID: \"5d21c619-2150-4bdc-9ce1-275f65d68d5a\") " pod="openshift-cluster-node-tuning-operator/tuned-gck9k"
Sep 18 13:58:55.910216 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:58:55.909081    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gqntj\" (UniqueName: \"kubernetes.io/projected/46552e87-3933-4e45-97c0-257d31588c9d-kube-api-access-gqntj\") pod \"network-check-target-d4rf2\" (UID: \"46552e87-3933-4e45-97c0-257d31588c9d\") " pod="openshift-network-diagnostics/network-check-target-d4rf2"
Sep 18 13:58:55.910216 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:58:55.909094    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/5d21c619-2150-4bdc-9ce1-275f65d68d5a-etc-systemd\") pod \"tuned-gck9k\" (UID: \"5d21c619-2150-4bdc-9ce1-275f65d68d5a\") " pod="openshift-cluster-node-tuning-operator/tuned-gck9k"
Sep 18 13:58:55.910216 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:58:55.909104    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/0b0463f5-d647-4a38-b752-44afba77daff-host-slash\") pod \"ovnkube-node-bvqls\" (UID: \"0b0463f5-d647-4a38-b752-44afba77daff\") " pod="openshift-ovn-kubernetes/ovnkube-node-bvqls"
Sep 18 13:58:55.910216 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:58:55.909129    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/0b0463f5-d647-4a38-b752-44afba77daff-log-socket\") pod \"ovnkube-node-bvqls\" (UID: \"0b0463f5-d647-4a38-b752-44afba77daff\") " pod="openshift-ovn-kubernetes/ovnkube-node-bvqls"
Sep 18 13:58:55.910216 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:58:55.909152    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qhrxx\" (UniqueName: \"kubernetes.io/projected/2044301a-3777-480b-9c7e-cdca3c1eb61a-kube-api-access-qhrxx\") pod \"node-resolver-lqtdm\" (UID: \"2044301a-3777-480b-9c7e-cdca3c1eb61a\") " pod="openshift-dns/node-resolver-lqtdm"
Sep 18 13:58:55.910216 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:58:55.909162    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/5d21c619-2150-4bdc-9ce1-275f65d68d5a-lib-modules\") pod \"tuned-gck9k\" (UID: \"5d21c619-2150-4bdc-9ce1-275f65d68d5a\") " pod="openshift-cluster-node-tuning-operator/tuned-gck9k"
Sep 18 13:58:55.910216 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:58:55.909164    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/0b0463f5-d647-4a38-b752-44afba77daff-host-slash\") pod \"ovnkube-node-bvqls\" (UID: \"0b0463f5-d647-4a38-b752-44afba77daff\") " pod="openshift-ovn-kubernetes/ovnkube-node-bvqls"
Sep 18 13:58:55.910216 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:58:55.909202    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/32ca2a66-ab18-447c-8120-7324ae6f94e4-device-dir\") pod \"aws-ebs-csi-driver-node-2jfng\" (UID: \"32ca2a66-ab18-447c-8120-7324ae6f94e4\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2jfng"
Sep 18 13:58:55.910216 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:58:55.909215    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/0b0463f5-d647-4a38-b752-44afba77daff-log-socket\") pod \"ovnkube-node-bvqls\" (UID: \"0b0463f5-d647-4a38-b752-44afba77daff\") " pod="openshift-ovn-kubernetes/ovnkube-node-bvqls"
Sep 18 13:58:55.910216 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:58:55.909231    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/f2b3025c-cb1e-49f1-8cde-431f4b0810f0-cni-binary-copy\") pod \"multus-lqtzw\" (UID: \"f2b3025c-cb1e-49f1-8cde-431f4b0810f0\") " pod="openshift-multus/multus-lqtzw"
Sep 18 13:58:55.910216 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:58:55.909258    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/5d21c619-2150-4bdc-9ce1-275f65d68d5a-etc-tuned\") pod \"tuned-gck9k\" (UID: \"5d21c619-2150-4bdc-9ce1-275f65d68d5a\") " pod="openshift-cluster-node-tuning-operator/tuned-gck9k"
Sep 18 13:58:55.910216 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:58:55.909281    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/f2b3025c-cb1e-49f1-8cde-431f4b0810f0-etc-kubernetes\") pod \"multus-lqtzw\" (UID: \"f2b3025c-cb1e-49f1-8cde-431f4b0810f0\") " pod="openshift-multus/multus-lqtzw"
Sep 18 13:58:55.910216 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:58:55.909306    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7fe04bef-1706-4159-8a98-89a352d62c7a-metrics-certs\") pod \"network-metrics-daemon-vpgtc\" (UID: \"7fe04bef-1706-4159-8a98-89a352d62c7a\") " pod="openshift-multus/network-metrics-daemon-vpgtc"
Sep 18 13:58:55.910216 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:58:55.909331    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/0b0463f5-d647-4a38-b752-44afba77daff-etc-openvswitch\") pod \"ovnkube-node-bvqls\" (UID: \"0b0463f5-d647-4a38-b752-44afba77daff\") " pod="openshift-ovn-kubernetes/ovnkube-node-bvqls"
Sep 18 13:58:55.910216 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:58:55.909355    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/19a8e259-6225-41e6-9556-f2439e36171d-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-68wp2\" (UID: \"19a8e259-6225-41e6-9556-f2439e36171d\") " pod="openshift-multus/multus-additional-cni-plugins-68wp2"
Sep 18 13:58:55.911054 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:58:55.909400    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-52xxl\" (UniqueName: \"kubernetes.io/projected/7623391e-952a-4649-81ad-9c59628e3cbf-kube-api-access-52xxl\") pod \"iptables-alerter-6nt4s\" (UID: \"7623391e-952a-4649-81ad-9c59628e3cbf\") " pod="openshift-network-operator/iptables-alerter-6nt4s"
Sep 18 13:58:55.911054 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:58:55.909423    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/d83ca837-1f57-4ff5-a513-a35f36198a99-kubelet-config\") pod \"global-pull-secret-syncer-xqrkr\" (UID: \"d83ca837-1f57-4ff5-a513-a35f36198a99\") " pod="kube-system/global-pull-secret-syncer-xqrkr"
Sep 18 13:58:55.911054 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:58:55.909449    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/32ca2a66-ab18-447c-8120-7324ae6f94e4-etc-selinux\") pod \"aws-ebs-csi-driver-node-2jfng\" (UID: \"32ca2a66-ab18-447c-8120-7324ae6f94e4\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2jfng"
Sep 18 13:58:55.911054 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:58:55.909464    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/f2b3025c-cb1e-49f1-8cde-431f4b0810f0-etc-kubernetes\") pod \"multus-lqtzw\" (UID: \"f2b3025c-cb1e-49f1-8cde-431f4b0810f0\") " pod="openshift-multus/multus-lqtzw"
Sep 18 13:58:55.911054 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:58:55.909475    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/19a8e259-6225-41e6-9556-f2439e36171d-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-68wp2\" (UID: \"19a8e259-6225-41e6-9556-f2439e36171d\") " pod="openshift-multus/multus-additional-cni-plugins-68wp2"
Sep 18 13:58:55.911054 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:58:55.909515    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/0b0463f5-d647-4a38-b752-44afba77daff-etc-openvswitch\") pod \"ovnkube-node-bvqls\" (UID: \"0b0463f5-d647-4a38-b752-44afba77daff\") " pod="openshift-ovn-kubernetes/ovnkube-node-bvqls"
Sep 18 13:58:55.911054 ip-10-0-129-16 kubenswrapper[2565]: E0918 13:58:55.909424    2565 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 18 13:58:55.911054 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:58:55.909585    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8r2vh\" (UniqueName: \"kubernetes.io/projected/19a8e259-6225-41e6-9556-f2439e36171d-kube-api-access-8r2vh\") pod \"multus-additional-cni-plugins-68wp2\" (UID: \"19a8e259-6225-41e6-9556-f2439e36171d\") " pod="openshift-multus/multus-additional-cni-plugins-68wp2"
Sep 18 13:58:55.911054 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:58:55.909647    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/32ca2a66-ab18-447c-8120-7324ae6f94e4-kubelet-dir\") pod \"aws-ebs-csi-driver-node-2jfng\" (UID: \"32ca2a66-ab18-447c-8120-7324ae6f94e4\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2jfng"
Sep 18 13:58:55.911054 ip-10-0-129-16 kubenswrapper[2565]: E0918 13:58:55.909672    2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7fe04bef-1706-4159-8a98-89a352d62c7a-metrics-certs podName:7fe04bef-1706-4159-8a98-89a352d62c7a nodeName:}" failed. No retries permitted until 2026-09-18 13:58:56.409650494 +0000 UTC m=+4.583560142 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/7fe04bef-1706-4159-8a98-89a352d62c7a-metrics-certs") pod "network-metrics-daemon-vpgtc" (UID: "7fe04bef-1706-4159-8a98-89a352d62c7a") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 18 13:58:55.911054 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:58:55.909716    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/f2b3025c-cb1e-49f1-8cde-431f4b0810f0-multus-socket-dir-parent\") pod \"multus-lqtzw\" (UID: \"f2b3025c-cb1e-49f1-8cde-431f4b0810f0\") " pod="openshift-multus/multus-lqtzw"
Sep 18 13:58:55.911054 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:58:55.909742    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/f2b3025c-cb1e-49f1-8cde-431f4b0810f0-host-var-lib-cni-multus\") pod \"multus-lqtzw\" (UID: \"f2b3025c-cb1e-49f1-8cde-431f4b0810f0\") " pod="openshift-multus/multus-lqtzw"
Sep 18 13:58:55.911054 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:58:55.909773    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/f2b3025c-cb1e-49f1-8cde-431f4b0810f0-multus-daemon-config\") pod \"multus-lqtzw\" (UID: \"f2b3025c-cb1e-49f1-8cde-431f4b0810f0\") " pod="openshift-multus/multus-lqtzw"
Sep 18 13:58:55.911054 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:58:55.909789    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/f2b3025c-cb1e-49f1-8cde-431f4b0810f0-multus-socket-dir-parent\") pod \"multus-lqtzw\" (UID: \"f2b3025c-cb1e-49f1-8cde-431f4b0810f0\") " pod="openshift-multus/multus-lqtzw"
Sep 18 13:58:55.911054 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:58:55.909800    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/0b0463f5-d647-4a38-b752-44afba77daff-run-openvswitch\") pod \"ovnkube-node-bvqls\" (UID: \"0b0463f5-d647-4a38-b752-44afba77daff\") " pod="openshift-ovn-kubernetes/ovnkube-node-bvqls"
Sep 18 13:58:55.911054 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:58:55.909838    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/0b0463f5-d647-4a38-b752-44afba77daff-host-run-ovn-kubernetes\") pod \"ovnkube-node-bvqls\" (UID: \"0b0463f5-d647-4a38-b752-44afba77daff\") " pod="openshift-ovn-kubernetes/ovnkube-node-bvqls"
Sep 18 13:58:55.911805 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:58:55.909867    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/f2b3025c-cb1e-49f1-8cde-431f4b0810f0-host-var-lib-cni-multus\") pod \"multus-lqtzw\" (UID: \"f2b3025c-cb1e-49f1-8cde-431f4b0810f0\") " pod="openshift-multus/multus-lqtzw"
Sep 18 13:58:55.911805 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:58:55.909887    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/7623391e-952a-4649-81ad-9c59628e3cbf-host-slash\") pod \"iptables-alerter-6nt4s\" (UID: \"7623391e-952a-4649-81ad-9c59628e3cbf\") " pod="openshift-network-operator/iptables-alerter-6nt4s"
Sep 18 13:58:55.911805 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:58:55.909899    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/0b0463f5-d647-4a38-b752-44afba77daff-run-openvswitch\") pod \"ovnkube-node-bvqls\" (UID: \"0b0463f5-d647-4a38-b752-44afba77daff\") " pod="openshift-ovn-kubernetes/ovnkube-node-bvqls"
Sep 18 13:58:55.911805 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:58:55.909907    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/f2b3025c-cb1e-49f1-8cde-431f4b0810f0-cni-binary-copy\") pod \"multus-lqtzw\" (UID: \"f2b3025c-cb1e-49f1-8cde-431f4b0810f0\") " pod="openshift-multus/multus-lqtzw"
Sep 18 13:58:55.911805 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:58:55.909916    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/0b0463f5-d647-4a38-b752-44afba77daff-host-run-ovn-kubernetes\") pod \"ovnkube-node-bvqls\" (UID: \"0b0463f5-d647-4a38-b752-44afba77daff\") " pod="openshift-ovn-kubernetes/ovnkube-node-bvqls"
Sep 18 13:58:55.911805 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:58:55.909912    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/5d21c619-2150-4bdc-9ce1-275f65d68d5a-run\") pod \"tuned-gck9k\" (UID: \"5d21c619-2150-4bdc-9ce1-275f65d68d5a\") " pod="openshift-cluster-node-tuning-operator/tuned-gck9k"
Sep 18 13:58:55.911805 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:58:55.909954    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/5d21c619-2150-4bdc-9ce1-275f65d68d5a-run\") pod \"tuned-gck9k\" (UID: \"5d21c619-2150-4bdc-9ce1-275f65d68d5a\") " pod="openshift-cluster-node-tuning-operator/tuned-gck9k"
Sep 18 13:58:55.911805 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:58:55.909966    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-d4xvb\" (UniqueName: \"kubernetes.io/projected/7fe04bef-1706-4159-8a98-89a352d62c7a-kube-api-access-d4xvb\") pod \"network-metrics-daemon-vpgtc\" (UID: \"7fe04bef-1706-4159-8a98-89a352d62c7a\") " pod="openshift-multus/network-metrics-daemon-vpgtc"
Sep 18 13:58:55.911805 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:58:55.909995    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/5d21c619-2150-4bdc-9ce1-275f65d68d5a-etc-sysctl-d\") pod \"tuned-gck9k\" (UID: \"5d21c619-2150-4bdc-9ce1-275f65d68d5a\") " pod="openshift-cluster-node-tuning-operator/tuned-gck9k"
Sep 18 13:58:55.911805 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:58:55.910023    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/75c7d00d-fb1a-4716-9963-b7ef1c0b35bd-host\") pod \"node-ca-hvhcs\" (UID: \"75c7d00d-fb1a-4716-9963-b7ef1c0b35bd\") " pod="openshift-image-registry/node-ca-hvhcs"
Sep 18 13:58:55.911805 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:58:55.910067    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/32ca2a66-ab18-447c-8120-7324ae6f94e4-registration-dir\") pod \"aws-ebs-csi-driver-node-2jfng\" (UID: \"32ca2a66-ab18-447c-8120-7324ae6f94e4\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2jfng"
Sep 18 13:58:55.911805 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:58:55.910093    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/5d21c619-2150-4bdc-9ce1-275f65d68d5a-etc-sysconfig\") pod \"tuned-gck9k\" (UID: \"5d21c619-2150-4bdc-9ce1-275f65d68d5a\") " pod="openshift-cluster-node-tuning-operator/tuned-gck9k"
Sep 18 13:58:55.911805 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:58:55.910122    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/5d21c619-2150-4bdc-9ce1-275f65d68d5a-etc-sysctl-conf\") pod \"tuned-gck9k\" (UID: \"5d21c619-2150-4bdc-9ce1-275f65d68d5a\") " pod="openshift-cluster-node-tuning-operator/tuned-gck9k"
Sep 18 13:58:55.911805 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:58:55.910140    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/5d21c619-2150-4bdc-9ce1-275f65d68d5a-etc-sysctl-d\") pod \"tuned-gck9k\" (UID: \"5d21c619-2150-4bdc-9ce1-275f65d68d5a\") " pod="openshift-cluster-node-tuning-operator/tuned-gck9k"
Sep 18 13:58:55.911805 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:58:55.910181    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/5d21c619-2150-4bdc-9ce1-275f65d68d5a-host\") pod \"tuned-gck9k\" (UID: \"5d21c619-2150-4bdc-9ce1-275f65d68d5a\") " pod="openshift-cluster-node-tuning-operator/tuned-gck9k"
Sep 18 13:58:55.911805 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:58:55.910191    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/5d21c619-2150-4bdc-9ce1-275f65d68d5a-etc-sysconfig\") pod \"tuned-gck9k\" (UID: \"5d21c619-2150-4bdc-9ce1-275f65d68d5a\") " pod="openshift-cluster-node-tuning-operator/tuned-gck9k"
Sep 18 13:58:55.911805 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:58:55.910235    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/0b0463f5-d647-4a38-b752-44afba77daff-run-systemd\") pod \"ovnkube-node-bvqls\" (UID: \"0b0463f5-d647-4a38-b752-44afba77daff\") " pod="openshift-ovn-kubernetes/ovnkube-node-bvqls"
Sep 18 13:58:55.911805 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:58:55.910256    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/f2b3025c-cb1e-49f1-8cde-431f4b0810f0-multus-daemon-config\") pod \"multus-lqtzw\" (UID: \"f2b3025c-cb1e-49f1-8cde-431f4b0810f0\") " pod="openshift-multus/multus-lqtzw"
Sep 18 13:58:55.912680 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:58:55.910257    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/0b0463f5-d647-4a38-b752-44afba77daff-ovnkube-config\") pod \"ovnkube-node-bvqls\" (UID: \"0b0463f5-d647-4a38-b752-44afba77daff\") " pod="openshift-ovn-kubernetes/ovnkube-node-bvqls"
Sep 18 13:58:55.912680 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:58:55.910296    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/19a8e259-6225-41e6-9556-f2439e36171d-cnibin\") pod \"multus-additional-cni-plugins-68wp2\" (UID: \"19a8e259-6225-41e6-9556-f2439e36171d\") " pod="openshift-multus/multus-additional-cni-plugins-68wp2"
Sep 18 13:58:55.912680 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:58:55.910303    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/5d21c619-2150-4bdc-9ce1-275f65d68d5a-etc-sysctl-conf\") pod \"tuned-gck9k\" (UID: \"5d21c619-2150-4bdc-9ce1-275f65d68d5a\") " pod="openshift-cluster-node-tuning-operator/tuned-gck9k"
Sep 18 13:58:55.912680 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:58:55.910316    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/d83ca837-1f57-4ff5-a513-a35f36198a99-dbus\") pod \"global-pull-secret-syncer-xqrkr\" (UID: \"d83ca837-1f57-4ff5-a513-a35f36198a99\") " pod="kube-system/global-pull-secret-syncer-xqrkr"
Sep 18 13:58:55.912680 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:58:55.910324    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/5d21c619-2150-4bdc-9ce1-275f65d68d5a-host\") pod \"tuned-gck9k\" (UID: \"5d21c619-2150-4bdc-9ce1-275f65d68d5a\") " pod="openshift-cluster-node-tuning-operator/tuned-gck9k"
Sep 18 13:58:55.912680 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:58:55.910339    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/0b0463f5-d647-4a38-b752-44afba77daff-node-log\") pod \"ovnkube-node-bvqls\" (UID: \"0b0463f5-d647-4a38-b752-44afba77daff\") " pod="openshift-ovn-kubernetes/ovnkube-node-bvqls"
Sep 18 13:58:55.912680 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:58:55.910350    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/0b0463f5-d647-4a38-b752-44afba77daff-run-systemd\") pod \"ovnkube-node-bvqls\" (UID: \"0b0463f5-d647-4a38-b752-44afba77daff\") " pod="openshift-ovn-kubernetes/ovnkube-node-bvqls"
Sep 18 13:58:55.912680 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:58:55.910360    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/19a8e259-6225-41e6-9556-f2439e36171d-cni-binary-copy\") pod \"multus-additional-cni-plugins-68wp2\" (UID: \"19a8e259-6225-41e6-9556-f2439e36171d\") " pod="openshift-multus/multus-additional-cni-plugins-68wp2"
Sep 18 13:58:55.912680 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:58:55.910381    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/d83ca837-1f57-4ff5-a513-a35f36198a99-original-pull-secret\") pod \"global-pull-secret-syncer-xqrkr\" (UID: \"d83ca837-1f57-4ff5-a513-a35f36198a99\") " pod="kube-system/global-pull-secret-syncer-xqrkr"
Sep 18 13:58:55.912680 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:58:55.910381    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/0b0463f5-d647-4a38-b752-44afba77daff-node-log\") pod \"ovnkube-node-bvqls\" (UID: \"0b0463f5-d647-4a38-b752-44afba77daff\") " pod="openshift-ovn-kubernetes/ovnkube-node-bvqls"
Sep 18 13:58:55.912680 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:58:55.910403    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-df5ln\" (UniqueName: \"kubernetes.io/projected/75c7d00d-fb1a-4716-9963-b7ef1c0b35bd-kube-api-access-df5ln\") pod \"node-ca-hvhcs\" (UID: \"75c7d00d-fb1a-4716-9963-b7ef1c0b35bd\") " pod="openshift-image-registry/node-ca-hvhcs"
Sep 18 13:58:55.912680 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:58:55.910426    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/f2b3025c-cb1e-49f1-8cde-431f4b0810f0-system-cni-dir\") pod \"multus-lqtzw\" (UID: \"f2b3025c-cb1e-49f1-8cde-431f4b0810f0\") " pod="openshift-multus/multus-lqtzw"
Sep 18 13:58:55.912680 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:58:55.910445    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/f2b3025c-cb1e-49f1-8cde-431f4b0810f0-multus-cni-dir\") pod \"multus-lqtzw\" (UID: \"f2b3025c-cb1e-49f1-8cde-431f4b0810f0\") " pod="openshift-multus/multus-lqtzw"
Sep 18 13:58:55.912680 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:58:55.910466    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/f2b3025c-cb1e-49f1-8cde-431f4b0810f0-host-var-lib-cni-bin\") pod \"multus-lqtzw\" (UID: \"f2b3025c-cb1e-49f1-8cde-431f4b0810f0\") " pod="openshift-multus/multus-lqtzw"
Sep 18 13:58:55.912680 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:58:55.910500    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/5d21c619-2150-4bdc-9ce1-275f65d68d5a-tmp\") pod \"tuned-gck9k\" (UID: \"5d21c619-2150-4bdc-9ce1-275f65d68d5a\") " pod="openshift-cluster-node-tuning-operator/tuned-gck9k"
Sep 18 13:58:55.912680 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:58:55.910528    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/0b0463f5-d647-4a38-b752-44afba77daff-ovnkube-script-lib\") pod \"ovnkube-node-bvqls\" (UID: \"0b0463f5-d647-4a38-b752-44afba77daff\") " pod="openshift-ovn-kubernetes/ovnkube-node-bvqls"
Sep 18 13:58:55.912680 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:58:55.910554    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/0b0463f5-d647-4a38-b752-44afba77daff-host-cni-bin\") pod \"ovnkube-node-bvqls\" (UID: \"0b0463f5-d647-4a38-b752-44afba77daff\") " pod="openshift-ovn-kubernetes/ovnkube-node-bvqls"
Sep 18 13:58:55.913509 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:58:55.910576    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/5d21c619-2150-4bdc-9ce1-275f65d68d5a-sys\") pod \"tuned-gck9k\" (UID: \"5d21c619-2150-4bdc-9ce1-275f65d68d5a\") " pod="openshift-cluster-node-tuning-operator/tuned-gck9k"
Sep 18 13:58:55.913509 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:58:55.910566    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/f2b3025c-cb1e-49f1-8cde-431f4b0810f0-multus-cni-dir\") pod \"multus-lqtzw\" (UID: \"f2b3025c-cb1e-49f1-8cde-431f4b0810f0\") " pod="openshift-multus/multus-lqtzw"
Sep 18 13:58:55.913509 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:58:55.910622    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/0b0463f5-d647-4a38-b752-44afba77daff-host-cni-netd\") pod \"ovnkube-node-bvqls\" (UID: \"0b0463f5-d647-4a38-b752-44afba77daff\") " pod="openshift-ovn-kubernetes/ovnkube-node-bvqls"
Sep 18 13:58:55.913509 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:58:55.910648    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/0b0463f5-d647-4a38-b752-44afba77daff-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-bvqls\" (UID: \"0b0463f5-d647-4a38-b752-44afba77daff\") " pod="openshift-ovn-kubernetes/ovnkube-node-bvqls"
Sep 18 13:58:55.913509 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:58:55.910658    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/f2b3025c-cb1e-49f1-8cde-431f4b0810f0-host-var-lib-cni-bin\") pod \"multus-lqtzw\" (UID: \"f2b3025c-cb1e-49f1-8cde-431f4b0810f0\") " pod="openshift-multus/multus-lqtzw"
Sep 18 13:58:55.913509 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:58:55.910696    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/0b0463f5-d647-4a38-b752-44afba77daff-host-cni-bin\") pod \"ovnkube-node-bvqls\" (UID: \"0b0463f5-d647-4a38-b752-44afba77daff\") " pod="openshift-ovn-kubernetes/ovnkube-node-bvqls"
Sep 18 13:58:55.913509 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:58:55.910705    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/0b0463f5-d647-4a38-b752-44afba77daff-ovnkube-config\") pod \"ovnkube-node-bvqls\" (UID: \"0b0463f5-d647-4a38-b752-44afba77daff\") " pod="openshift-ovn-kubernetes/ovnkube-node-bvqls"
Sep 18 13:58:55.913509 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:58:55.910728    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2044301a-3777-480b-9c7e-cdca3c1eb61a-tmp-dir\") pod \"node-resolver-lqtdm\" (UID: \"2044301a-3777-480b-9c7e-cdca3c1eb61a\") " pod="openshift-dns/node-resolver-lqtdm"
Sep 18 13:58:55.913509 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:58:55.910738    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/0b0463f5-d647-4a38-b752-44afba77daff-host-cni-netd\") pod \"ovnkube-node-bvqls\" (UID: \"0b0463f5-d647-4a38-b752-44afba77daff\") " pod="openshift-ovn-kubernetes/ovnkube-node-bvqls"
Sep 18 13:58:55.913509 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:58:55.910763    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/0b0463f5-d647-4a38-b752-44afba77daff-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-bvqls\" (UID: \"0b0463f5-d647-4a38-b752-44afba77daff\") " pod="openshift-ovn-kubernetes/ovnkube-node-bvqls"
Sep 18 13:58:55.913509 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:58:55.910756    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/bff1008a-4c27-4278-969d-8974600e696f-konnectivity-ca\") pod \"konnectivity-agent-rzgjk\" (UID: \"bff1008a-4c27-4278-969d-8974600e696f\") " pod="kube-system/konnectivity-agent-rzgjk"
Sep 18 13:58:55.913509 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:58:55.910709    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/f2b3025c-cb1e-49f1-8cde-431f4b0810f0-system-cni-dir\") pod \"multus-lqtzw\" (UID: \"f2b3025c-cb1e-49f1-8cde-431f4b0810f0\") " pod="openshift-multus/multus-lqtzw"
Sep 18 13:58:55.913509 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:58:55.910873    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/5d21c619-2150-4bdc-9ce1-275f65d68d5a-sys\") pod \"tuned-gck9k\" (UID: \"5d21c619-2150-4bdc-9ce1-275f65d68d5a\") " pod="openshift-cluster-node-tuning-operator/tuned-gck9k"
Sep 18 13:58:55.913509 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:58:55.911054    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/0b0463f5-d647-4a38-b752-44afba77daff-ovnkube-script-lib\") pod \"ovnkube-node-bvqls\" (UID: \"0b0463f5-d647-4a38-b752-44afba77daff\") " pod="openshift-ovn-kubernetes/ovnkube-node-bvqls"
Sep 18 13:58:55.913509 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:58:55.911436    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2044301a-3777-480b-9c7e-cdca3c1eb61a-tmp-dir\") pod \"node-resolver-lqtdm\" (UID: \"2044301a-3777-480b-9c7e-cdca3c1eb61a\") " pod="openshift-dns/node-resolver-lqtdm"
Sep 18 13:58:55.913509 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:58:55.911778    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-16.ec2.internal" event={"ID":"e264249022c87eb79103b2d4c06c5fb0","Type":"ContainerStarted","Data":"c534c4646c7c97f7616795fca51f913c5c2c1d5261358eb6987c8f22f9d52a71"}
Sep 18 13:58:55.913509 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:58:55.912980    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-129-16.ec2.internal" event={"ID":"0092863d56373d9006f35b4f163e59f7","Type":"ContainerStarted","Data":"cd978145887cc0c2ee98d0198c2b564813353625009971706d5a70fed33ba63f"}
Sep 18 13:58:55.913509 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:58:55.913258    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/5d21c619-2150-4bdc-9ce1-275f65d68d5a-etc-tuned\") pod \"tuned-gck9k\" (UID: \"5d21c619-2150-4bdc-9ce1-275f65d68d5a\") " pod="openshift-cluster-node-tuning-operator/tuned-gck9k"
Sep 18 13:58:55.914364 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:58:55.913988    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/0b0463f5-d647-4a38-b752-44afba77daff-ovn-node-metrics-cert\") pod \"ovnkube-node-bvqls\" (UID: \"0b0463f5-d647-4a38-b752-44afba77daff\") " pod="openshift-ovn-kubernetes/ovnkube-node-bvqls"
Sep 18 13:58:55.914539 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:58:55.914515    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/5d21c619-2150-4bdc-9ce1-275f65d68d5a-tmp\") pod \"tuned-gck9k\" (UID: \"5d21c619-2150-4bdc-9ce1-275f65d68d5a\") " pod="openshift-cluster-node-tuning-operator/tuned-gck9k"
Sep 18 13:58:55.920527 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:58:55.920468    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hf6bl\" (UniqueName: \"kubernetes.io/projected/5d21c619-2150-4bdc-9ce1-275f65d68d5a-kube-api-access-hf6bl\") pod \"tuned-gck9k\" (UID: \"5d21c619-2150-4bdc-9ce1-275f65d68d5a\") " pod="openshift-cluster-node-tuning-operator/tuned-gck9k"
Sep 18 13:58:55.920800 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:58:55.920781    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dltvg\" (UniqueName: \"kubernetes.io/projected/0b0463f5-d647-4a38-b752-44afba77daff-kube-api-access-dltvg\") pod \"ovnkube-node-bvqls\" (UID: \"0b0463f5-d647-4a38-b752-44afba77daff\") " pod="openshift-ovn-kubernetes/ovnkube-node-bvqls"
Sep 18 13:58:55.921807 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:58:55.921787    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-g6zrg\" (UniqueName: \"kubernetes.io/projected/f2b3025c-cb1e-49f1-8cde-431f4b0810f0-kube-api-access-g6zrg\") pod \"multus-lqtzw\" (UID: \"f2b3025c-cb1e-49f1-8cde-431f4b0810f0\") " pod="openshift-multus/multus-lqtzw"
Sep 18 13:58:55.923645 ip-10-0-129-16 kubenswrapper[2565]: E0918 13:58:55.923624    2565 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 18 13:58:55.923765 ip-10-0-129-16 kubenswrapper[2565]: E0918 13:58:55.923650    2565 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 18 13:58:55.923765 ip-10-0-129-16 kubenswrapper[2565]: E0918 13:58:55.923661    2565 projected.go:196] Error preparing data for projected volume kube-api-access-gqntj for pod openshift-network-diagnostics/network-check-target-d4rf2: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 18 13:58:55.923765 ip-10-0-129-16 kubenswrapper[2565]: E0918 13:58:55.923761    2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/46552e87-3933-4e45-97c0-257d31588c9d-kube-api-access-gqntj podName:46552e87-3933-4e45-97c0-257d31588c9d nodeName:}" failed. No retries permitted until 2026-09-18 13:58:56.423745371 +0000 UTC m=+4.597655009 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-gqntj" (UniqueName: "kubernetes.io/projected/46552e87-3933-4e45-97c0-257d31588c9d-kube-api-access-gqntj") pod "network-check-target-d4rf2" (UID: "46552e87-3933-4e45-97c0-257d31588c9d") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 18 13:58:55.925709 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:58:55.925683    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-d4xvb\" (UniqueName: \"kubernetes.io/projected/7fe04bef-1706-4159-8a98-89a352d62c7a-kube-api-access-d4xvb\") pod \"network-metrics-daemon-vpgtc\" (UID: \"7fe04bef-1706-4159-8a98-89a352d62c7a\") " pod="openshift-multus/network-metrics-daemon-vpgtc"
Sep 18 13:58:55.926083 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:58:55.926062    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qhrxx\" (UniqueName: \"kubernetes.io/projected/2044301a-3777-480b-9c7e-cdca3c1eb61a-kube-api-access-qhrxx\") pod \"node-resolver-lqtdm\" (UID: \"2044301a-3777-480b-9c7e-cdca3c1eb61a\") " pod="openshift-dns/node-resolver-lqtdm"
Sep 18 13:58:56.011420 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:58:56.011385    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/19a8e259-6225-41e6-9556-f2439e36171d-cnibin\") pod \"multus-additional-cni-plugins-68wp2\" (UID: \"19a8e259-6225-41e6-9556-f2439e36171d\") " pod="openshift-multus/multus-additional-cni-plugins-68wp2"
Sep 18 13:58:56.011607 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:58:56.011428    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/d83ca837-1f57-4ff5-a513-a35f36198a99-dbus\") pod \"global-pull-secret-syncer-xqrkr\" (UID: \"d83ca837-1f57-4ff5-a513-a35f36198a99\") " pod="kube-system/global-pull-secret-syncer-xqrkr"
Sep 18 13:58:56.011607 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:58:56.011450    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/19a8e259-6225-41e6-9556-f2439e36171d-cni-binary-copy\") pod \"multus-additional-cni-plugins-68wp2\" (UID: \"19a8e259-6225-41e6-9556-f2439e36171d\") " pod="openshift-multus/multus-additional-cni-plugins-68wp2"
Sep 18 13:58:56.011607 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:58:56.011466    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/d83ca837-1f57-4ff5-a513-a35f36198a99-original-pull-secret\") pod \"global-pull-secret-syncer-xqrkr\" (UID: \"d83ca837-1f57-4ff5-a513-a35f36198a99\") " pod="kube-system/global-pull-secret-syncer-xqrkr"
Sep 18 13:58:56.011607 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:58:56.011480    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-df5ln\" (UniqueName: \"kubernetes.io/projected/75c7d00d-fb1a-4716-9963-b7ef1c0b35bd-kube-api-access-df5ln\") pod \"node-ca-hvhcs\" (UID: \"75c7d00d-fb1a-4716-9963-b7ef1c0b35bd\") " pod="openshift-image-registry/node-ca-hvhcs"
Sep 18 13:58:56.011607 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:58:56.011495    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/19a8e259-6225-41e6-9556-f2439e36171d-cnibin\") pod \"multus-additional-cni-plugins-68wp2\" (UID: \"19a8e259-6225-41e6-9556-f2439e36171d\") " pod="openshift-multus/multus-additional-cni-plugins-68wp2"
Sep 18 13:58:56.011607 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:58:56.011510    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/bff1008a-4c27-4278-969d-8974600e696f-konnectivity-ca\") pod \"konnectivity-agent-rzgjk\" (UID: \"bff1008a-4c27-4278-969d-8974600e696f\") " pod="kube-system/konnectivity-agent-rzgjk"
Sep 18 13:58:56.011607 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:58:56.011535    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/32ca2a66-ab18-447c-8120-7324ae6f94e4-socket-dir\") pod \"aws-ebs-csi-driver-node-2jfng\" (UID: \"32ca2a66-ab18-447c-8120-7324ae6f94e4\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2jfng"
Sep 18 13:58:56.011607 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:58:56.011566    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/32ca2a66-ab18-447c-8120-7324ae6f94e4-sys-fs\") pod \"aws-ebs-csi-driver-node-2jfng\" (UID: \"32ca2a66-ab18-447c-8120-7324ae6f94e4\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2jfng"
Sep 18 13:58:56.011607 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:58:56.011594    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/19a8e259-6225-41e6-9556-f2439e36171d-tuning-conf-dir\") pod \"multus-additional-cni-plugins-68wp2\" (UID: \"19a8e259-6225-41e6-9556-f2439e36171d\") " pod="openshift-multus/multus-additional-cni-plugins-68wp2"
Sep 18 13:58:56.011607 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:58:56.011613    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5tmmv\" (UniqueName: \"kubernetes.io/projected/32ca2a66-ab18-447c-8120-7324ae6f94e4-kube-api-access-5tmmv\") pod \"aws-ebs-csi-driver-node-2jfng\" (UID: \"32ca2a66-ab18-447c-8120-7324ae6f94e4\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2jfng"
Sep 18 13:58:56.012112 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:58:56.011637    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/7623391e-952a-4649-81ad-9c59628e3cbf-iptables-alerter-script\") pod \"iptables-alerter-6nt4s\" (UID: \"7623391e-952a-4649-81ad-9c59628e3cbf\") " pod="openshift-network-operator/iptables-alerter-6nt4s"
Sep 18 13:58:56.012112 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:58:56.011662    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/19a8e259-6225-41e6-9556-f2439e36171d-system-cni-dir\") pod \"multus-additional-cni-plugins-68wp2\" (UID: \"19a8e259-6225-41e6-9556-f2439e36171d\") " pod="openshift-multus/multus-additional-cni-plugins-68wp2"
Sep 18 13:58:56.012112 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:58:56.011671    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/d83ca837-1f57-4ff5-a513-a35f36198a99-dbus\") pod \"global-pull-secret-syncer-xqrkr\" (UID: \"d83ca837-1f57-4ff5-a513-a35f36198a99\") " pod="kube-system/global-pull-secret-syncer-xqrkr"
Sep 18 13:58:56.012112 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:58:56.011679    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/bff1008a-4c27-4278-969d-8974600e696f-agent-certs\") pod \"konnectivity-agent-rzgjk\" (UID: \"bff1008a-4c27-4278-969d-8974600e696f\") " pod="kube-system/konnectivity-agent-rzgjk"
Sep 18 13:58:56.012112 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:58:56.011721    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/75c7d00d-fb1a-4716-9963-b7ef1c0b35bd-serviceca\") pod \"node-ca-hvhcs\" (UID: \"75c7d00d-fb1a-4716-9963-b7ef1c0b35bd\") " pod="openshift-image-registry/node-ca-hvhcs"
Sep 18 13:58:56.012112 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:58:56.011778    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/32ca2a66-ab18-447c-8120-7324ae6f94e4-device-dir\") pod \"aws-ebs-csi-driver-node-2jfng\" (UID: \"32ca2a66-ab18-447c-8120-7324ae6f94e4\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2jfng"
Sep 18 13:58:56.012112 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:58:56.011822    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/19a8e259-6225-41e6-9556-f2439e36171d-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-68wp2\" (UID: \"19a8e259-6225-41e6-9556-f2439e36171d\") " pod="openshift-multus/multus-additional-cni-plugins-68wp2"
Sep 18 13:58:56.012112 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:58:56.011854    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-52xxl\" (UniqueName: \"kubernetes.io/projected/7623391e-952a-4649-81ad-9c59628e3cbf-kube-api-access-52xxl\") pod \"iptables-alerter-6nt4s\" (UID: \"7623391e-952a-4649-81ad-9c59628e3cbf\") " pod="openshift-network-operator/iptables-alerter-6nt4s"
Sep 18 13:58:56.012112 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:58:56.011881    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/d83ca837-1f57-4ff5-a513-a35f36198a99-kubelet-config\") pod \"global-pull-secret-syncer-xqrkr\" (UID: \"d83ca837-1f57-4ff5-a513-a35f36198a99\") " pod="kube-system/global-pull-secret-syncer-xqrkr"
Sep 18 13:58:56.012112 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:58:56.011907    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/32ca2a66-ab18-447c-8120-7324ae6f94e4-etc-selinux\") pod \"aws-ebs-csi-driver-node-2jfng\" (UID: \"32ca2a66-ab18-447c-8120-7324ae6f94e4\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2jfng"
Sep 18 13:58:56.012112 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:58:56.011935    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/19a8e259-6225-41e6-9556-f2439e36171d-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-68wp2\" (UID: \"19a8e259-6225-41e6-9556-f2439e36171d\") " pod="openshift-multus/multus-additional-cni-plugins-68wp2"
Sep 18 13:58:56.012112 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:58:56.011962    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8r2vh\" (UniqueName: \"kubernetes.io/projected/19a8e259-6225-41e6-9556-f2439e36171d-kube-api-access-8r2vh\") pod \"multus-additional-cni-plugins-68wp2\" (UID: \"19a8e259-6225-41e6-9556-f2439e36171d\") " pod="openshift-multus/multus-additional-cni-plugins-68wp2"
Sep 18 13:58:56.012112 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:58:56.011991    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/32ca2a66-ab18-447c-8120-7324ae6f94e4-kubelet-dir\") pod \"aws-ebs-csi-driver-node-2jfng\" (UID: \"32ca2a66-ab18-447c-8120-7324ae6f94e4\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2jfng"
Sep 18 13:58:56.012112 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:58:56.012026    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/7623391e-952a-4649-81ad-9c59628e3cbf-host-slash\") pod \"iptables-alerter-6nt4s\" (UID: \"7623391e-952a-4649-81ad-9c59628e3cbf\") " pod="openshift-network-operator/iptables-alerter-6nt4s"
Sep 18 13:58:56.012112 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:58:56.012052    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/32ca2a66-ab18-447c-8120-7324ae6f94e4-sys-fs\") pod \"aws-ebs-csi-driver-node-2jfng\" (UID: \"32ca2a66-ab18-447c-8120-7324ae6f94e4\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2jfng"
Sep 18 13:58:56.012112 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:58:56.012073    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/75c7d00d-fb1a-4716-9963-b7ef1c0b35bd-host\") pod \"node-ca-hvhcs\" (UID: \"75c7d00d-fb1a-4716-9963-b7ef1c0b35bd\") " pod="openshift-image-registry/node-ca-hvhcs"
Sep 18 13:58:56.012112 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:58:56.012099    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/32ca2a66-ab18-447c-8120-7324ae6f94e4-registration-dir\") pod \"aws-ebs-csi-driver-node-2jfng\" (UID: \"32ca2a66-ab18-447c-8120-7324ae6f94e4\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2jfng"
Sep 18 13:58:56.012772 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:58:56.012120    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/19a8e259-6225-41e6-9556-f2439e36171d-cni-binary-copy\") pod \"multus-additional-cni-plugins-68wp2\" (UID: \"19a8e259-6225-41e6-9556-f2439e36171d\") " pod="openshift-multus/multus-additional-cni-plugins-68wp2"
Sep 18 13:58:56.012772 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:58:56.012152    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/19a8e259-6225-41e6-9556-f2439e36171d-system-cni-dir\") pod \"multus-additional-cni-plugins-68wp2\" (UID: \"19a8e259-6225-41e6-9556-f2439e36171d\") " pod="openshift-multus/multus-additional-cni-plugins-68wp2"
Sep 18 13:58:56.012772 ip-10-0-129-16 kubenswrapper[2565]: E0918 13:58:56.012171    2565 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 18 13:58:56.012772 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:58:56.012180    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/32ca2a66-ab18-447c-8120-7324ae6f94e4-registration-dir\") pod \"aws-ebs-csi-driver-node-2jfng\" (UID: \"32ca2a66-ab18-447c-8120-7324ae6f94e4\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2jfng"
Sep 18 13:58:56.012772 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:58:56.012251    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/32ca2a66-ab18-447c-8120-7324ae6f94e4-etc-selinux\") pod \"aws-ebs-csi-driver-node-2jfng\" (UID: \"32ca2a66-ab18-447c-8120-7324ae6f94e4\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2jfng"
Sep 18 13:58:56.012772 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:58:56.012222    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/19a8e259-6225-41e6-9556-f2439e36171d-tuning-conf-dir\") pod \"multus-additional-cni-plugins-68wp2\" (UID: \"19a8e259-6225-41e6-9556-f2439e36171d\") " pod="openshift-multus/multus-additional-cni-plugins-68wp2"
Sep 18 13:58:56.013078 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:58:56.012830    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/7623391e-952a-4649-81ad-9c59628e3cbf-host-slash\") pod \"iptables-alerter-6nt4s\" (UID: \"7623391e-952a-4649-81ad-9c59628e3cbf\") " pod="openshift-network-operator/iptables-alerter-6nt4s"
Sep 18 13:58:56.013078 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:58:56.012923    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/75c7d00d-fb1a-4716-9963-b7ef1c0b35bd-host\") pod \"node-ca-hvhcs\" (UID: \"75c7d00d-fb1a-4716-9963-b7ef1c0b35bd\") " pod="openshift-image-registry/node-ca-hvhcs"
Sep 18 13:58:56.013172 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:58:56.013091    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/32ca2a66-ab18-447c-8120-7324ae6f94e4-kubelet-dir\") pod \"aws-ebs-csi-driver-node-2jfng\" (UID: \"32ca2a66-ab18-447c-8120-7324ae6f94e4\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2jfng"
Sep 18 13:58:56.013425 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:58:56.013390    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/7623391e-952a-4649-81ad-9c59628e3cbf-iptables-alerter-script\") pod \"iptables-alerter-6nt4s\" (UID: \"7623391e-952a-4649-81ad-9c59628e3cbf\") " pod="openshift-network-operator/iptables-alerter-6nt4s"
Sep 18 13:58:56.013626 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:58:56.013583    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/bff1008a-4c27-4278-969d-8974600e696f-konnectivity-ca\") pod \"konnectivity-agent-rzgjk\" (UID: \"bff1008a-4c27-4278-969d-8974600e696f\") " pod="kube-system/konnectivity-agent-rzgjk"
Sep 18 13:58:56.013626 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:58:56.013616    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/19a8e259-6225-41e6-9556-f2439e36171d-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-68wp2\" (UID: \"19a8e259-6225-41e6-9556-f2439e36171d\") " pod="openshift-multus/multus-additional-cni-plugins-68wp2"
Sep 18 13:58:56.013803 ip-10-0-129-16 kubenswrapper[2565]: E0918 13:58:56.012251    2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d83ca837-1f57-4ff5-a513-a35f36198a99-original-pull-secret podName:d83ca837-1f57-4ff5-a513-a35f36198a99 nodeName:}" failed. No retries permitted until 2026-09-18 13:58:56.512213526 +0000 UTC m=+4.686123180 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/d83ca837-1f57-4ff5-a513-a35f36198a99-original-pull-secret") pod "global-pull-secret-syncer-xqrkr" (UID: "d83ca837-1f57-4ff5-a513-a35f36198a99") : object "kube-system"/"original-pull-secret" not registered
Sep 18 13:58:56.013803 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:58:56.013668    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/75c7d00d-fb1a-4716-9963-b7ef1c0b35bd-serviceca\") pod \"node-ca-hvhcs\" (UID: \"75c7d00d-fb1a-4716-9963-b7ef1c0b35bd\") " pod="openshift-image-registry/node-ca-hvhcs"
Sep 18 13:58:56.013803 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:58:56.013692    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/32ca2a66-ab18-447c-8120-7324ae6f94e4-device-dir\") pod \"aws-ebs-csi-driver-node-2jfng\" (UID: \"32ca2a66-ab18-447c-8120-7324ae6f94e4\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2jfng"
Sep 18 13:58:56.013803 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:58:56.013754    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/d83ca837-1f57-4ff5-a513-a35f36198a99-kubelet-config\") pod \"global-pull-secret-syncer-xqrkr\" (UID: \"d83ca837-1f57-4ff5-a513-a35f36198a99\") " pod="kube-system/global-pull-secret-syncer-xqrkr"
Sep 18 13:58:56.014117 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:58:56.013860    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/32ca2a66-ab18-447c-8120-7324ae6f94e4-socket-dir\") pod \"aws-ebs-csi-driver-node-2jfng\" (UID: \"32ca2a66-ab18-447c-8120-7324ae6f94e4\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2jfng"
Sep 18 13:58:56.014117 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:58:56.014101    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/19a8e259-6225-41e6-9556-f2439e36171d-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-68wp2\" (UID: \"19a8e259-6225-41e6-9556-f2439e36171d\") " pod="openshift-multus/multus-additional-cni-plugins-68wp2"
Sep 18 13:58:56.015626 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:58:56.014792    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/bff1008a-4c27-4278-969d-8974600e696f-agent-certs\") pod \"konnectivity-agent-rzgjk\" (UID: \"bff1008a-4c27-4278-969d-8974600e696f\") " pod="kube-system/konnectivity-agent-rzgjk"
Sep 18 13:58:56.030651 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:58:56.030621    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-df5ln\" (UniqueName: \"kubernetes.io/projected/75c7d00d-fb1a-4716-9963-b7ef1c0b35bd-kube-api-access-df5ln\") pod \"node-ca-hvhcs\" (UID: \"75c7d00d-fb1a-4716-9963-b7ef1c0b35bd\") " pod="openshift-image-registry/node-ca-hvhcs"
Sep 18 13:58:56.030782 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:58:56.030694    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8r2vh\" (UniqueName: \"kubernetes.io/projected/19a8e259-6225-41e6-9556-f2439e36171d-kube-api-access-8r2vh\") pod \"multus-additional-cni-plugins-68wp2\" (UID: \"19a8e259-6225-41e6-9556-f2439e36171d\") " pod="openshift-multus/multus-additional-cni-plugins-68wp2"
Sep 18 13:58:56.030853 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:58:56.030786    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-52xxl\" (UniqueName: \"kubernetes.io/projected/7623391e-952a-4649-81ad-9c59628e3cbf-kube-api-access-52xxl\") pod \"iptables-alerter-6nt4s\" (UID: \"7623391e-952a-4649-81ad-9c59628e3cbf\") " pod="openshift-network-operator/iptables-alerter-6nt4s"
Sep 18 13:58:56.031138 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:58:56.031115    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5tmmv\" (UniqueName: \"kubernetes.io/projected/32ca2a66-ab18-447c-8120-7324ae6f94e4-kube-api-access-5tmmv\") pod \"aws-ebs-csi-driver-node-2jfng\" (UID: \"32ca2a66-ab18-447c-8120-7324ae6f94e4\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2jfng"
Sep 18 13:58:56.098267 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:58:56.098234    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-bvqls"
Sep 18 13:58:56.104571 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:58:56.104545    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-lqtdm"
Sep 18 13:58:56.115626 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:58:56.115603    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-lqtzw"
Sep 18 13:58:56.122346 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:58:56.122319    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-gck9k"
Sep 18 13:58:56.129300 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:58:56.129257    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-rzgjk"
Sep 18 13:58:56.136806 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:58:56.136788    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2jfng"
Sep 18 13:58:56.145425 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:58:56.145398    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-hvhcs"
Sep 18 13:58:56.153054 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:58:56.153022    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-68wp2"
Sep 18 13:58:56.160613 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:58:56.160594    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-6nt4s"
Sep 18 13:58:56.414688 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:58:56.414613    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7fe04bef-1706-4159-8a98-89a352d62c7a-metrics-certs\") pod \"network-metrics-daemon-vpgtc\" (UID: \"7fe04bef-1706-4159-8a98-89a352d62c7a\") " pod="openshift-multus/network-metrics-daemon-vpgtc"
Sep 18 13:58:56.414827 ip-10-0-129-16 kubenswrapper[2565]: E0918 13:58:56.414780    2565 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 18 13:58:56.414877 ip-10-0-129-16 kubenswrapper[2565]: E0918 13:58:56.414854    2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7fe04bef-1706-4159-8a98-89a352d62c7a-metrics-certs podName:7fe04bef-1706-4159-8a98-89a352d62c7a nodeName:}" failed. No retries permitted until 2026-09-18 13:58:57.414836758 +0000 UTC m=+5.588746419 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/7fe04bef-1706-4159-8a98-89a352d62c7a-metrics-certs") pod "network-metrics-daemon-vpgtc" (UID: "7fe04bef-1706-4159-8a98-89a352d62c7a") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 18 13:58:56.515179 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:58:56.515146    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/d83ca837-1f57-4ff5-a513-a35f36198a99-original-pull-secret\") pod \"global-pull-secret-syncer-xqrkr\" (UID: \"d83ca837-1f57-4ff5-a513-a35f36198a99\") " pod="kube-system/global-pull-secret-syncer-xqrkr"
Sep 18 13:58:56.515179 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:58:56.515197    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gqntj\" (UniqueName: \"kubernetes.io/projected/46552e87-3933-4e45-97c0-257d31588c9d-kube-api-access-gqntj\") pod \"network-check-target-d4rf2\" (UID: \"46552e87-3933-4e45-97c0-257d31588c9d\") " pod="openshift-network-diagnostics/network-check-target-d4rf2"
Sep 18 13:58:56.515427 ip-10-0-129-16 kubenswrapper[2565]: E0918 13:58:56.515319    2565 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 18 13:58:56.515427 ip-10-0-129-16 kubenswrapper[2565]: E0918 13:58:56.515402    2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d83ca837-1f57-4ff5-a513-a35f36198a99-original-pull-secret podName:d83ca837-1f57-4ff5-a513-a35f36198a99 nodeName:}" failed. No retries permitted until 2026-09-18 13:58:57.515381069 +0000 UTC m=+5.689290723 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/d83ca837-1f57-4ff5-a513-a35f36198a99-original-pull-secret") pod "global-pull-secret-syncer-xqrkr" (UID: "d83ca837-1f57-4ff5-a513-a35f36198a99") : object "kube-system"/"original-pull-secret" not registered
Sep 18 13:58:56.515533 ip-10-0-129-16 kubenswrapper[2565]: E0918 13:58:56.515449    2565 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 18 13:58:56.515533 ip-10-0-129-16 kubenswrapper[2565]: E0918 13:58:56.515469    2565 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 18 13:58:56.515533 ip-10-0-129-16 kubenswrapper[2565]: E0918 13:58:56.515479    2565 projected.go:196] Error preparing data for projected volume kube-api-access-gqntj for pod openshift-network-diagnostics/network-check-target-d4rf2: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 18 13:58:56.515533 ip-10-0-129-16 kubenswrapper[2565]: E0918 13:58:56.515518    2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/46552e87-3933-4e45-97c0-257d31588c9d-kube-api-access-gqntj podName:46552e87-3933-4e45-97c0-257d31588c9d nodeName:}" failed. No retries permitted until 2026-09-18 13:58:57.515507739 +0000 UTC m=+5.689417405 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-gqntj" (UniqueName: "kubernetes.io/projected/46552e87-3933-4e45-97c0-257d31588c9d-kube-api-access-gqntj") pod "network-check-target-d4rf2" (UID: "46552e87-3933-4e45-97c0-257d31588c9d") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 18 13:58:56.888857 ip-10-0-129-16 kubenswrapper[2565]: W0918 13:58:56.888833    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod32ca2a66_ab18_447c_8120_7324ae6f94e4.slice/crio-c86993ed3e157b15c016926e1ff293425106787e33182251dc68b1acf41be781 WatchSource:0}: Error finding container c86993ed3e157b15c016926e1ff293425106787e33182251dc68b1acf41be781: Status 404 returned error can't find the container with id c86993ed3e157b15c016926e1ff293425106787e33182251dc68b1acf41be781
Sep 18 13:58:56.890430 ip-10-0-129-16 kubenswrapper[2565]: W0918 13:58:56.890408    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf2b3025c_cb1e_49f1_8cde_431f4b0810f0.slice/crio-177f178e938e498268a07ad3c2084a424b5e070af5a21b8e93dafb0d9e35592c WatchSource:0}: Error finding container 177f178e938e498268a07ad3c2084a424b5e070af5a21b8e93dafb0d9e35592c: Status 404 returned error can't find the container with id 177f178e938e498268a07ad3c2084a424b5e070af5a21b8e93dafb0d9e35592c
Sep 18 13:58:56.894715 ip-10-0-129-16 kubenswrapper[2565]: W0918 13:58:56.894693    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0b0463f5_d647_4a38_b752_44afba77daff.slice/crio-fe96087c8a124c8e0cedd3d211605c577b5a0d93845d04d7893e9c5d9a680af9 WatchSource:0}: Error finding container fe96087c8a124c8e0cedd3d211605c577b5a0d93845d04d7893e9c5d9a680af9: Status 404 returned error can't find the container with id fe96087c8a124c8e0cedd3d211605c577b5a0d93845d04d7893e9c5d9a680af9
Sep 18 13:58:56.896280 ip-10-0-129-16 kubenswrapper[2565]: W0918 13:58:56.896251    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2044301a_3777_480b_9c7e_cdca3c1eb61a.slice/crio-c826c9c5fe9b159a6067212e8d9c8c36a1eab0b2dc25ef59798fd4da06b1e771 WatchSource:0}: Error finding container c826c9c5fe9b159a6067212e8d9c8c36a1eab0b2dc25ef59798fd4da06b1e771: Status 404 returned error can't find the container with id c826c9c5fe9b159a6067212e8d9c8c36a1eab0b2dc25ef59798fd4da06b1e771
Sep 18 13:58:56.910413 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:58:56.910378    2565 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-09-17 13:53:55 +0000 UTC" deadline="2028-04-28 17:51:04.62833132 +0000 UTC"
Sep 18 13:58:56.910413 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:58:56.910412    2565 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="14115h52m7.717923981s"
Sep 18 13:58:56.915833 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:58:56.915797    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-hvhcs" event={"ID":"75c7d00d-fb1a-4716-9963-b7ef1c0b35bd","Type":"ContainerStarted","Data":"2f2b40b1bf2e076702b670cfdc81aaa4579cecaab781a7f59b155f171ce45d8a"}
Sep 18 13:58:56.917047 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:58:56.917007    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-gck9k" event={"ID":"5d21c619-2150-4bdc-9ce1-275f65d68d5a","Type":"ContainerStarted","Data":"57a85a5bbd7a34f9520ebc231f33780f18640dbe05ccc2b58ee9c77910e02c1b"}
Sep 18 13:58:56.918297 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:58:56.918279    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-68wp2" event={"ID":"19a8e259-6225-41e6-9556-f2439e36171d","Type":"ContainerStarted","Data":"907b1c9bcf0f4bb616e957c5f9480b233fd5b98a36e932cca1b6d0bd42cd1db0"}
Sep 18 13:58:56.919582 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:58:56.919548    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-lqtdm" event={"ID":"2044301a-3777-480b-9c7e-cdca3c1eb61a","Type":"ContainerStarted","Data":"c826c9c5fe9b159a6067212e8d9c8c36a1eab0b2dc25ef59798fd4da06b1e771"}
Sep 18 13:58:56.920934 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:58:56.920908    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-6nt4s" event={"ID":"7623391e-952a-4649-81ad-9c59628e3cbf","Type":"ContainerStarted","Data":"6612cce8ff23795990bbca37a80d916b40f2106fdf6d6e98c9388724b656ee6e"}
Sep 18 13:58:56.922285 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:58:56.922261    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2jfng" event={"ID":"32ca2a66-ab18-447c-8120-7324ae6f94e4","Type":"ContainerStarted","Data":"c86993ed3e157b15c016926e1ff293425106787e33182251dc68b1acf41be781"}
Sep 18 13:58:56.923469 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:58:56.923438    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-rzgjk" event={"ID":"bff1008a-4c27-4278-969d-8974600e696f","Type":"ContainerStarted","Data":"5f01b06fc350bd11e96af690aa8483fb346d2b91debfbd20e74ac6686345d3ad"}
Sep 18 13:58:56.925011 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:58:56.924990    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-bvqls" event={"ID":"0b0463f5-d647-4a38-b752-44afba77daff","Type":"ContainerStarted","Data":"fe96087c8a124c8e0cedd3d211605c577b5a0d93845d04d7893e9c5d9a680af9"}
Sep 18 13:58:56.926242 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:58:56.926216    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-lqtzw" event={"ID":"f2b3025c-cb1e-49f1-8cde-431f4b0810f0","Type":"ContainerStarted","Data":"177f178e938e498268a07ad3c2084a424b5e070af5a21b8e93dafb0d9e35592c"}
Sep 18 13:58:57.422254 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:58:57.421585    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7fe04bef-1706-4159-8a98-89a352d62c7a-metrics-certs\") pod \"network-metrics-daemon-vpgtc\" (UID: \"7fe04bef-1706-4159-8a98-89a352d62c7a\") " pod="openshift-multus/network-metrics-daemon-vpgtc"
Sep 18 13:58:57.422254 ip-10-0-129-16 kubenswrapper[2565]: E0918 13:58:57.421728    2565 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 18 13:58:57.422254 ip-10-0-129-16 kubenswrapper[2565]: E0918 13:58:57.421794    2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7fe04bef-1706-4159-8a98-89a352d62c7a-metrics-certs podName:7fe04bef-1706-4159-8a98-89a352d62c7a nodeName:}" failed. No retries permitted until 2026-09-18 13:58:59.421773603 +0000 UTC m=+7.595683261 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/7fe04bef-1706-4159-8a98-89a352d62c7a-metrics-certs") pod "network-metrics-daemon-vpgtc" (UID: "7fe04bef-1706-4159-8a98-89a352d62c7a") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 18 13:58:57.522597 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:58:57.522555    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/d83ca837-1f57-4ff5-a513-a35f36198a99-original-pull-secret\") pod \"global-pull-secret-syncer-xqrkr\" (UID: \"d83ca837-1f57-4ff5-a513-a35f36198a99\") " pod="kube-system/global-pull-secret-syncer-xqrkr"
Sep 18 13:58:57.522777 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:58:57.522634    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gqntj\" (UniqueName: \"kubernetes.io/projected/46552e87-3933-4e45-97c0-257d31588c9d-kube-api-access-gqntj\") pod \"network-check-target-d4rf2\" (UID: \"46552e87-3933-4e45-97c0-257d31588c9d\") " pod="openshift-network-diagnostics/network-check-target-d4rf2"
Sep 18 13:58:57.522902 ip-10-0-129-16 kubenswrapper[2565]: E0918 13:58:57.522792    2565 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 18 13:58:57.522902 ip-10-0-129-16 kubenswrapper[2565]: E0918 13:58:57.522819    2565 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 18 13:58:57.522902 ip-10-0-129-16 kubenswrapper[2565]: E0918 13:58:57.522832    2565 projected.go:196] Error preparing data for projected volume kube-api-access-gqntj for pod openshift-network-diagnostics/network-check-target-d4rf2: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 18 13:58:57.522902 ip-10-0-129-16 kubenswrapper[2565]: E0918 13:58:57.522888    2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/46552e87-3933-4e45-97c0-257d31588c9d-kube-api-access-gqntj podName:46552e87-3933-4e45-97c0-257d31588c9d nodeName:}" failed. No retries permitted until 2026-09-18 13:58:59.522871113 +0000 UTC m=+7.696780756 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-gqntj" (UniqueName: "kubernetes.io/projected/46552e87-3933-4e45-97c0-257d31588c9d-kube-api-access-gqntj") pod "network-check-target-d4rf2" (UID: "46552e87-3933-4e45-97c0-257d31588c9d") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 18 13:58:57.523155 ip-10-0-129-16 kubenswrapper[2565]: E0918 13:58:57.523138    2565 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 18 13:58:57.523210 ip-10-0-129-16 kubenswrapper[2565]: E0918 13:58:57.523195    2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d83ca837-1f57-4ff5-a513-a35f36198a99-original-pull-secret podName:d83ca837-1f57-4ff5-a513-a35f36198a99 nodeName:}" failed. No retries permitted until 2026-09-18 13:58:59.523180909 +0000 UTC m=+7.697090554 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/d83ca837-1f57-4ff5-a513-a35f36198a99-original-pull-secret") pod "global-pull-secret-syncer-xqrkr" (UID: "d83ca837-1f57-4ff5-a513-a35f36198a99") : object "kube-system"/"original-pull-secret" not registered
Sep 18 13:58:57.907051 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:58:57.906957    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-vpgtc"
Sep 18 13:58:57.907485 ip-10-0-129-16 kubenswrapper[2565]: E0918 13:58:57.907110    2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-vpgtc" podUID="7fe04bef-1706-4159-8a98-89a352d62c7a"
Sep 18 13:58:57.907485 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:58:57.907127    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-d4rf2"
Sep 18 13:58:57.907485 ip-10-0-129-16 kubenswrapper[2565]: E0918 13:58:57.907227    2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-d4rf2" podUID="46552e87-3933-4e45-97c0-257d31588c9d"
Sep 18 13:58:57.908685 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:58:57.908664    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-xqrkr"
Sep 18 13:58:57.908799 ip-10-0-129-16 kubenswrapper[2565]: E0918 13:58:57.908779    2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-xqrkr" podUID="d83ca837-1f57-4ff5-a513-a35f36198a99"
Sep 18 13:58:57.910823 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:58:57.910775    2565 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-09-17 13:53:55 +0000 UTC" deadline="2028-04-01 08:38:37.11720748 +0000 UTC"
Sep 18 13:58:57.910823 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:58:57.910823    2565 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="13458h39m39.20638817s"
Sep 18 13:58:57.942451 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:58:57.942415    2565 generic.go:356] "Generic (PLEG): container finished" podID="e264249022c87eb79103b2d4c06c5fb0" containerID="832ca471050d2224e5df2de269019e46d7086972686b24302d140228c32e5baf" exitCode=0
Sep 18 13:58:57.942607 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:58:57.942516    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-16.ec2.internal" event={"ID":"e264249022c87eb79103b2d4c06c5fb0","Type":"ContainerDied","Data":"832ca471050d2224e5df2de269019e46d7086972686b24302d140228c32e5baf"}
Sep 18 13:58:57.951610 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:58:57.951574    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-129-16.ec2.internal" event={"ID":"0092863d56373d9006f35b4f163e59f7","Type":"ContainerStarted","Data":"e6b666db67f8624a7c8af7312dad3d0105f4729f12c6bde611eb342c8d75f20f"}
Sep 18 13:58:58.963610 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:58:58.963577    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-16.ec2.internal" event={"ID":"e264249022c87eb79103b2d4c06c5fb0","Type":"ContainerStarted","Data":"f6d951ba4cba3a3e163777da9c62439613f32c0dc8eb80312a85490130fd3339"}
Sep 18 13:58:58.988765 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:58:58.988695    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-129-16.ec2.internal" podStartSLOduration=3.988669765 podStartE2EDuration="3.988669765s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-18 13:58:55 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 13:58:57.979985942 +0000 UTC m=+6.153895619" watchObservedRunningTime="2026-09-18 13:58:58.988669765 +0000 UTC m=+7.162579427"
Sep 18 13:58:59.440827 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:58:59.440740    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7fe04bef-1706-4159-8a98-89a352d62c7a-metrics-certs\") pod \"network-metrics-daemon-vpgtc\" (UID: \"7fe04bef-1706-4159-8a98-89a352d62c7a\") " pod="openshift-multus/network-metrics-daemon-vpgtc"
Sep 18 13:58:59.441012 ip-10-0-129-16 kubenswrapper[2565]: E0918 13:58:59.440971    2565 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 18 13:58:59.441106 ip-10-0-129-16 kubenswrapper[2565]: E0918 13:58:59.441049    2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7fe04bef-1706-4159-8a98-89a352d62c7a-metrics-certs podName:7fe04bef-1706-4159-8a98-89a352d62c7a nodeName:}" failed. No retries permitted until 2026-09-18 13:59:03.441017234 +0000 UTC m=+11.614926879 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/7fe04bef-1706-4159-8a98-89a352d62c7a-metrics-certs") pod "network-metrics-daemon-vpgtc" (UID: "7fe04bef-1706-4159-8a98-89a352d62c7a") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 18 13:58:59.542280 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:58:59.542241    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/d83ca837-1f57-4ff5-a513-a35f36198a99-original-pull-secret\") pod \"global-pull-secret-syncer-xqrkr\" (UID: \"d83ca837-1f57-4ff5-a513-a35f36198a99\") " pod="kube-system/global-pull-secret-syncer-xqrkr"
Sep 18 13:58:59.542464 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:58:59.542338    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gqntj\" (UniqueName: \"kubernetes.io/projected/46552e87-3933-4e45-97c0-257d31588c9d-kube-api-access-gqntj\") pod \"network-check-target-d4rf2\" (UID: \"46552e87-3933-4e45-97c0-257d31588c9d\") " pod="openshift-network-diagnostics/network-check-target-d4rf2"
Sep 18 13:58:59.542550 ip-10-0-129-16 kubenswrapper[2565]: E0918 13:58:59.542523    2565 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 18 13:58:59.542606 ip-10-0-129-16 kubenswrapper[2565]: E0918 13:58:59.542553    2565 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 18 13:58:59.542606 ip-10-0-129-16 kubenswrapper[2565]: E0918 13:58:59.542567    2565 projected.go:196] Error preparing data for projected volume kube-api-access-gqntj for pod openshift-network-diagnostics/network-check-target-d4rf2: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 18 13:58:59.542704 ip-10-0-129-16 kubenswrapper[2565]: E0918 13:58:59.542622    2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/46552e87-3933-4e45-97c0-257d31588c9d-kube-api-access-gqntj podName:46552e87-3933-4e45-97c0-257d31588c9d nodeName:}" failed. No retries permitted until 2026-09-18 13:59:03.542605288 +0000 UTC m=+11.716514931 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-gqntj" (UniqueName: "kubernetes.io/projected/46552e87-3933-4e45-97c0-257d31588c9d-kube-api-access-gqntj") pod "network-check-target-d4rf2" (UID: "46552e87-3933-4e45-97c0-257d31588c9d") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 18 13:58:59.543058 ip-10-0-129-16 kubenswrapper[2565]: E0918 13:58:59.543009    2565 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 18 13:58:59.543151 ip-10-0-129-16 kubenswrapper[2565]: E0918 13:58:59.543081    2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d83ca837-1f57-4ff5-a513-a35f36198a99-original-pull-secret podName:d83ca837-1f57-4ff5-a513-a35f36198a99 nodeName:}" failed. No retries permitted until 2026-09-18 13:59:03.543067048 +0000 UTC m=+11.716976690 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/d83ca837-1f57-4ff5-a513-a35f36198a99-original-pull-secret") pod "global-pull-secret-syncer-xqrkr" (UID: "d83ca837-1f57-4ff5-a513-a35f36198a99") : object "kube-system"/"original-pull-secret" not registered
Sep 18 13:58:59.907164 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:58:59.907058    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-xqrkr"
Sep 18 13:58:59.907164 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:58:59.907130    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-vpgtc"
Sep 18 13:58:59.907164 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:58:59.907142    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-d4rf2"
Sep 18 13:58:59.907474 ip-10-0-129-16 kubenswrapper[2565]: E0918 13:58:59.907250    2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-vpgtc" podUID="7fe04bef-1706-4159-8a98-89a352d62c7a"
Sep 18 13:58:59.907474 ip-10-0-129-16 kubenswrapper[2565]: E0918 13:58:59.907334    2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-d4rf2" podUID="46552e87-3933-4e45-97c0-257d31588c9d"
Sep 18 13:58:59.907474 ip-10-0-129-16 kubenswrapper[2565]: E0918 13:58:59.907422    2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-xqrkr" podUID="d83ca837-1f57-4ff5-a513-a35f36198a99"
Sep 18 13:59:01.911298 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:59:01.911256    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-d4rf2"
Sep 18 13:59:01.911830 ip-10-0-129-16 kubenswrapper[2565]: E0918 13:59:01.911389    2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-d4rf2" podUID="46552e87-3933-4e45-97c0-257d31588c9d"
Sep 18 13:59:01.911830 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:59:01.911813    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-xqrkr"
Sep 18 13:59:01.911939 ip-10-0-129-16 kubenswrapper[2565]: E0918 13:59:01.911902    2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-xqrkr" podUID="d83ca837-1f57-4ff5-a513-a35f36198a99"
Sep 18 13:59:01.911994 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:59:01.911960    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-vpgtc"
Sep 18 13:59:01.912091 ip-10-0-129-16 kubenswrapper[2565]: E0918 13:59:01.912072    2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-vpgtc" podUID="7fe04bef-1706-4159-8a98-89a352d62c7a"
Sep 18 13:59:03.476117 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:59:03.476079    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7fe04bef-1706-4159-8a98-89a352d62c7a-metrics-certs\") pod \"network-metrics-daemon-vpgtc\" (UID: \"7fe04bef-1706-4159-8a98-89a352d62c7a\") " pod="openshift-multus/network-metrics-daemon-vpgtc"
Sep 18 13:59:03.476565 ip-10-0-129-16 kubenswrapper[2565]: E0918 13:59:03.476235    2565 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 18 13:59:03.476565 ip-10-0-129-16 kubenswrapper[2565]: E0918 13:59:03.476308    2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7fe04bef-1706-4159-8a98-89a352d62c7a-metrics-certs podName:7fe04bef-1706-4159-8a98-89a352d62c7a nodeName:}" failed. No retries permitted until 2026-09-18 13:59:11.476288138 +0000 UTC m=+19.650197790 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/7fe04bef-1706-4159-8a98-89a352d62c7a-metrics-certs") pod "network-metrics-daemon-vpgtc" (UID: "7fe04bef-1706-4159-8a98-89a352d62c7a") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 18 13:59:03.576994 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:59:03.576953    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/d83ca837-1f57-4ff5-a513-a35f36198a99-original-pull-secret\") pod \"global-pull-secret-syncer-xqrkr\" (UID: \"d83ca837-1f57-4ff5-a513-a35f36198a99\") " pod="kube-system/global-pull-secret-syncer-xqrkr"
Sep 18 13:59:03.577193 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:59:03.577044    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gqntj\" (UniqueName: \"kubernetes.io/projected/46552e87-3933-4e45-97c0-257d31588c9d-kube-api-access-gqntj\") pod \"network-check-target-d4rf2\" (UID: \"46552e87-3933-4e45-97c0-257d31588c9d\") " pod="openshift-network-diagnostics/network-check-target-d4rf2"
Sep 18 13:59:03.577260 ip-10-0-129-16 kubenswrapper[2565]: E0918 13:59:03.577241    2565 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 18 13:59:03.577311 ip-10-0-129-16 kubenswrapper[2565]: E0918 13:59:03.577268    2565 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 18 13:59:03.577311 ip-10-0-129-16 kubenswrapper[2565]: E0918 13:59:03.577282    2565 projected.go:196] Error preparing data for projected volume kube-api-access-gqntj for pod openshift-network-diagnostics/network-check-target-d4rf2: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 18 13:59:03.577397 ip-10-0-129-16 kubenswrapper[2565]: E0918 13:59:03.577338    2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/46552e87-3933-4e45-97c0-257d31588c9d-kube-api-access-gqntj podName:46552e87-3933-4e45-97c0-257d31588c9d nodeName:}" failed. No retries permitted until 2026-09-18 13:59:11.577321156 +0000 UTC m=+19.751230799 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-gqntj" (UniqueName: "kubernetes.io/projected/46552e87-3933-4e45-97c0-257d31588c9d-kube-api-access-gqntj") pod "network-check-target-d4rf2" (UID: "46552e87-3933-4e45-97c0-257d31588c9d") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 18 13:59:03.577810 ip-10-0-129-16 kubenswrapper[2565]: E0918 13:59:03.577728    2565 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 18 13:59:03.577810 ip-10-0-129-16 kubenswrapper[2565]: E0918 13:59:03.577786    2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d83ca837-1f57-4ff5-a513-a35f36198a99-original-pull-secret podName:d83ca837-1f57-4ff5-a513-a35f36198a99 nodeName:}" failed. No retries permitted until 2026-09-18 13:59:11.577770478 +0000 UTC m=+19.751680122 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/d83ca837-1f57-4ff5-a513-a35f36198a99-original-pull-secret") pod "global-pull-secret-syncer-xqrkr" (UID: "d83ca837-1f57-4ff5-a513-a35f36198a99") : object "kube-system"/"original-pull-secret" not registered
Sep 18 13:59:03.907515 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:59:03.906979    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-vpgtc"
Sep 18 13:59:03.907515 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:59:03.907001    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-xqrkr"
Sep 18 13:59:03.907515 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:59:03.906979    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-d4rf2"
Sep 18 13:59:03.907515 ip-10-0-129-16 kubenswrapper[2565]: E0918 13:59:03.907136    2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-vpgtc" podUID="7fe04bef-1706-4159-8a98-89a352d62c7a"
Sep 18 13:59:03.907515 ip-10-0-129-16 kubenswrapper[2565]: E0918 13:59:03.907240    2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-xqrkr" podUID="d83ca837-1f57-4ff5-a513-a35f36198a99"
Sep 18 13:59:03.908379 ip-10-0-129-16 kubenswrapper[2565]: E0918 13:59:03.908353    2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-d4rf2" podUID="46552e87-3933-4e45-97c0-257d31588c9d"
Sep 18 13:59:05.909956 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:59:05.909919    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-vpgtc"
Sep 18 13:59:05.910417 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:59:05.910060    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-xqrkr"
Sep 18 13:59:05.910417 ip-10-0-129-16 kubenswrapper[2565]: E0918 13:59:05.910177    2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-xqrkr" podUID="d83ca837-1f57-4ff5-a513-a35f36198a99"
Sep 18 13:59:05.910417 ip-10-0-129-16 kubenswrapper[2565]: E0918 13:59:05.910063    2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-vpgtc" podUID="7fe04bef-1706-4159-8a98-89a352d62c7a"
Sep 18 13:59:05.910417 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:59:05.910066    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-d4rf2"
Sep 18 13:59:05.910417 ip-10-0-129-16 kubenswrapper[2565]: E0918 13:59:05.910274    2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-d4rf2" podUID="46552e87-3933-4e45-97c0-257d31588c9d"
Sep 18 13:59:07.909895 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:59:07.909868    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-d4rf2"
Sep 18 13:59:07.910370 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:59:07.909868    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-xqrkr"
Sep 18 13:59:07.910370 ip-10-0-129-16 kubenswrapper[2565]: E0918 13:59:07.909989    2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-d4rf2" podUID="46552e87-3933-4e45-97c0-257d31588c9d"
Sep 18 13:59:07.910370 ip-10-0-129-16 kubenswrapper[2565]: E0918 13:59:07.910068    2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-xqrkr" podUID="d83ca837-1f57-4ff5-a513-a35f36198a99"
Sep 18 13:59:07.910370 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:59:07.909868    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-vpgtc"
Sep 18 13:59:07.910370 ip-10-0-129-16 kubenswrapper[2565]: E0918 13:59:07.910196    2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-vpgtc" podUID="7fe04bef-1706-4159-8a98-89a352d62c7a"
Sep 18 13:59:09.909706 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:59:09.909644    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-vpgtc"
Sep 18 13:59:09.909706 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:59:09.909679    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-xqrkr"
Sep 18 13:59:09.909706 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:59:09.909681    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-d4rf2"
Sep 18 13:59:09.910241 ip-10-0-129-16 kubenswrapper[2565]: E0918 13:59:09.909776    2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-vpgtc" podUID="7fe04bef-1706-4159-8a98-89a352d62c7a"
Sep 18 13:59:09.910241 ip-10-0-129-16 kubenswrapper[2565]: E0918 13:59:09.909863    2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-xqrkr" podUID="d83ca837-1f57-4ff5-a513-a35f36198a99"
Sep 18 13:59:09.910241 ip-10-0-129-16 kubenswrapper[2565]: E0918 13:59:09.909913    2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-d4rf2" podUID="46552e87-3933-4e45-97c0-257d31588c9d"
Sep 18 13:59:11.532254 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:59:11.532217    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7fe04bef-1706-4159-8a98-89a352d62c7a-metrics-certs\") pod \"network-metrics-daemon-vpgtc\" (UID: \"7fe04bef-1706-4159-8a98-89a352d62c7a\") " pod="openshift-multus/network-metrics-daemon-vpgtc"
Sep 18 13:59:11.532701 ip-10-0-129-16 kubenswrapper[2565]: E0918 13:59:11.532362    2565 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 18 13:59:11.532701 ip-10-0-129-16 kubenswrapper[2565]: E0918 13:59:11.532425    2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7fe04bef-1706-4159-8a98-89a352d62c7a-metrics-certs podName:7fe04bef-1706-4159-8a98-89a352d62c7a nodeName:}" failed. No retries permitted until 2026-09-18 13:59:27.532405679 +0000 UTC m=+35.706315329 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/7fe04bef-1706-4159-8a98-89a352d62c7a-metrics-certs") pod "network-metrics-daemon-vpgtc" (UID: "7fe04bef-1706-4159-8a98-89a352d62c7a") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 18 13:59:11.632759 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:59:11.632715    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gqntj\" (UniqueName: \"kubernetes.io/projected/46552e87-3933-4e45-97c0-257d31588c9d-kube-api-access-gqntj\") pod \"network-check-target-d4rf2\" (UID: \"46552e87-3933-4e45-97c0-257d31588c9d\") " pod="openshift-network-diagnostics/network-check-target-d4rf2"
Sep 18 13:59:11.632907 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:59:11.632789    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/d83ca837-1f57-4ff5-a513-a35f36198a99-original-pull-secret\") pod \"global-pull-secret-syncer-xqrkr\" (UID: \"d83ca837-1f57-4ff5-a513-a35f36198a99\") " pod="kube-system/global-pull-secret-syncer-xqrkr"
Sep 18 13:59:11.632907 ip-10-0-129-16 kubenswrapper[2565]: E0918 13:59:11.632845    2565 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 18 13:59:11.632907 ip-10-0-129-16 kubenswrapper[2565]: E0918 13:59:11.632875    2565 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 18 13:59:11.632907 ip-10-0-129-16 kubenswrapper[2565]: E0918 13:59:11.632889    2565 projected.go:196] Error preparing data for projected volume kube-api-access-gqntj for pod openshift-network-diagnostics/network-check-target-d4rf2: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 18 13:59:11.632907 ip-10-0-129-16 kubenswrapper[2565]: E0918 13:59:11.632901    2565 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 18 13:59:11.633151 ip-10-0-129-16 kubenswrapper[2565]: E0918 13:59:11.632946    2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/46552e87-3933-4e45-97c0-257d31588c9d-kube-api-access-gqntj podName:46552e87-3933-4e45-97c0-257d31588c9d nodeName:}" failed. No retries permitted until 2026-09-18 13:59:27.632929147 +0000 UTC m=+35.806838790 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-gqntj" (UniqueName: "kubernetes.io/projected/46552e87-3933-4e45-97c0-257d31588c9d-kube-api-access-gqntj") pod "network-check-target-d4rf2" (UID: "46552e87-3933-4e45-97c0-257d31588c9d") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 18 13:59:11.633151 ip-10-0-129-16 kubenswrapper[2565]: E0918 13:59:11.632963    2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d83ca837-1f57-4ff5-a513-a35f36198a99-original-pull-secret podName:d83ca837-1f57-4ff5-a513-a35f36198a99 nodeName:}" failed. No retries permitted until 2026-09-18 13:59:27.632956445 +0000 UTC m=+35.806866082 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/d83ca837-1f57-4ff5-a513-a35f36198a99-original-pull-secret") pod "global-pull-secret-syncer-xqrkr" (UID: "d83ca837-1f57-4ff5-a513-a35f36198a99") : object "kube-system"/"original-pull-secret" not registered
Sep 18 13:59:11.906995 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:59:11.906918    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-vpgtc"
Sep 18 13:59:11.906995 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:59:11.906950    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-xqrkr"
Sep 18 13:59:11.907254 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:59:11.906918    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-d4rf2"
Sep 18 13:59:11.907254 ip-10-0-129-16 kubenswrapper[2565]: E0918 13:59:11.907091    2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-vpgtc" podUID="7fe04bef-1706-4159-8a98-89a352d62c7a"
Sep 18 13:59:11.907254 ip-10-0-129-16 kubenswrapper[2565]: E0918 13:59:11.907144    2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-d4rf2" podUID="46552e87-3933-4e45-97c0-257d31588c9d"
Sep 18 13:59:11.907254 ip-10-0-129-16 kubenswrapper[2565]: E0918 13:59:11.907215    2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-xqrkr" podUID="d83ca837-1f57-4ff5-a513-a35f36198a99"
Sep 18 13:59:13.907848 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:59:13.907818    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-d4rf2"
Sep 18 13:59:13.908306 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:59:13.907922    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-vpgtc"
Sep 18 13:59:13.908306 ip-10-0-129-16 kubenswrapper[2565]: E0918 13:59:13.907927    2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-d4rf2" podUID="46552e87-3933-4e45-97c0-257d31588c9d"
Sep 18 13:59:13.908306 ip-10-0-129-16 kubenswrapper[2565]: E0918 13:59:13.908049    2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-vpgtc" podUID="7fe04bef-1706-4159-8a98-89a352d62c7a"
Sep 18 13:59:13.908306 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:59:13.908058    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-xqrkr"
Sep 18 13:59:13.908306 ip-10-0-129-16 kubenswrapper[2565]: E0918 13:59:13.908126    2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-xqrkr" podUID="d83ca837-1f57-4ff5-a513-a35f36198a99"
Sep 18 13:59:14.996930 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:59:14.996408    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-hvhcs" event={"ID":"75c7d00d-fb1a-4716-9963-b7ef1c0b35bd","Type":"ContainerStarted","Data":"4e0d030a4cf411a6feee36c095447c99211587ce7bacbf54021215e3bca4cdb6"}
Sep 18 13:59:14.998220 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:59:14.998192    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-gck9k" event={"ID":"5d21c619-2150-4bdc-9ce1-275f65d68d5a","Type":"ContainerStarted","Data":"f81a8ed95a0170700fcc56f39f39a2f6e9098898e4436b2d3bab7cb54eae7387"}
Sep 18 13:59:15.000049 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:59:14.999939    2565 generic.go:356] "Generic (PLEG): container finished" podID="19a8e259-6225-41e6-9556-f2439e36171d" containerID="101b699132aa0073512377d074c81b7a8f4bde30ff38104aa93a8215fb63d4d0" exitCode=0
Sep 18 13:59:15.000158 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:59:15.000060    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-68wp2" event={"ID":"19a8e259-6225-41e6-9556-f2439e36171d","Type":"ContainerDied","Data":"101b699132aa0073512377d074c81b7a8f4bde30ff38104aa93a8215fb63d4d0"}
Sep 18 13:59:15.002186 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:59:15.002159    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-lqtdm" event={"ID":"2044301a-3777-480b-9c7e-cdca3c1eb61a","Type":"ContainerStarted","Data":"2d5b527dc3ded7f00224c6d2bcfe98a4ad16ce4327269b6b38fd82173e226885"}
Sep 18 13:59:15.005373 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:59:15.005106    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2jfng" event={"ID":"32ca2a66-ab18-447c-8120-7324ae6f94e4","Type":"ContainerStarted","Data":"012d777b3ac12fb477dd57c2356a1665cc30ac02dd484112e43c59e6398547e1"}
Sep 18 13:59:15.007450 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:59:15.007426    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-rzgjk" event={"ID":"bff1008a-4c27-4278-969d-8974600e696f","Type":"ContainerStarted","Data":"85cf0455e15c0923f36a1d400b15e214898ff580a338189da3d73437e0a2ea23"}
Sep 18 13:59:15.010661 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:59:15.010639    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-bvqls_0b0463f5-d647-4a38-b752-44afba77daff/ovn-acl-logging/0.log"
Sep 18 13:59:15.011182 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:59:15.011147    2565 generic.go:356] "Generic (PLEG): container finished" podID="0b0463f5-d647-4a38-b752-44afba77daff" containerID="c76ad955c00fe81107bf5dda1a2af48ea1d7d8121f6570c363cba1bca9f37e39" exitCode=1
Sep 18 13:59:15.011268 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:59:15.011209    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-bvqls" event={"ID":"0b0463f5-d647-4a38-b752-44afba77daff","Type":"ContainerStarted","Data":"d7e3dcbaf94a5f93ea02a55d1da4cf91f4f28ae57e5a6eadc8ef786680cf0b43"}
Sep 18 13:59:15.011322 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:59:15.011284    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-bvqls" event={"ID":"0b0463f5-d647-4a38-b752-44afba77daff","Type":"ContainerStarted","Data":"33285d96e6b46113e167c92195928b5fccca0199a42e3c3bb1466a924e021f3b"}
Sep 18 13:59:15.011377 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:59:15.011336    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-bvqls" event={"ID":"0b0463f5-d647-4a38-b752-44afba77daff","Type":"ContainerStarted","Data":"e2ca2049f66a19ce3a1e46c196020bea212fb34f48b3d2b121b2e843e6904074"}
Sep 18 13:59:15.011430 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:59:15.011386    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-bvqls" event={"ID":"0b0463f5-d647-4a38-b752-44afba77daff","Type":"ContainerStarted","Data":"05edb28cc0841b12fc03fb533ccb5577a7d5f39fe738d407e00e4699ea499289"}
Sep 18 13:59:15.011483 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:59:15.011436    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-bvqls" event={"ID":"0b0463f5-d647-4a38-b752-44afba77daff","Type":"ContainerDied","Data":"c76ad955c00fe81107bf5dda1a2af48ea1d7d8121f6570c363cba1bca9f37e39"}
Sep 18 13:59:15.011536 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:59:15.011506    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-bvqls" event={"ID":"0b0463f5-d647-4a38-b752-44afba77daff","Type":"ContainerStarted","Data":"3f40992af5e2976e59cb7bb3c8bc5c2853e919dc4bda63d5754bf2711d9300c2"}
Sep 18 13:59:15.012884 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:59:15.012864    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-lqtzw" event={"ID":"f2b3025c-cb1e-49f1-8cde-431f4b0810f0","Type":"ContainerStarted","Data":"b1f39c6e224ca973f0bbc575cd4958fc17eef3e4d5b20f096653b25d78d66020"}
Sep 18 13:59:15.015597 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:59:15.015556    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-hvhcs" podStartSLOduration=3.938549955 podStartE2EDuration="21.015543545s" totalImagesPullingTime="17.07699359s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-18 13:58:54 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 13:59:15.015235 +0000 UTC m=+23.189144681" watchObservedRunningTime="2026-09-18 13:59:15.015543545 +0000 UTC m=+23.189453205"
Sep 18 13:59:15.015688 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:59:15.015628    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-16.ec2.internal" podStartSLOduration=20.015623511 podStartE2EDuration="20.015623511s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-18 13:58:55 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 13:58:58.989705025 +0000 UTC m=+7.163614688" watchObservedRunningTime="2026-09-18 13:59:15.015623511 +0000 UTC m=+23.189533169"
Sep 18 13:59:15.033877 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:59:15.033834    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-rzgjk" podStartSLOduration=3.9551061929999998 podStartE2EDuration="21.033815901s" totalImagesPullingTime="17.078709708s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 13:58:54 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 13:59:15.033650817 +0000 UTC m=+23.207560477" watchObservedRunningTime="2026-09-18 13:59:15.033815901 +0000 UTC m=+23.207725563"
Sep 18 13:59:15.079683 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:59:15.079642    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-lqtdm" podStartSLOduration=3.970500306 podStartE2EDuration="21.079627545s" totalImagesPullingTime="17.109127239s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-18 13:58:54 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 13:59:15.079066042 +0000 UTC m=+23.252975721" watchObservedRunningTime="2026-09-18 13:59:15.079627545 +0000 UTC m=+23.253537204"
Sep 18 13:59:15.097584 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:59:15.097536    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-lqtzw" podStartSLOduration=3.9662760649999997 podStartE2EDuration="21.097520733s" totalImagesPullingTime="17.131244668s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-18 13:58:54 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 13:59:15.097110983 +0000 UTC m=+23.271020662" watchObservedRunningTime="2026-09-18 13:59:15.097520733 +0000 UTC m=+23.271430428"
Sep 18 13:59:15.619248 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:59:15.619220    2565 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock"
Sep 18 13:59:15.708066 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:59:15.708020    2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-rzgjk"
Sep 18 13:59:15.708699 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:59:15.708679    2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-rzgjk"
Sep 18 13:59:15.724598 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:59:15.724553    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-gck9k" podStartSLOduration=4.647225347 podStartE2EDuration="21.724538154s" totalImagesPullingTime="17.077312807s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-18 13:58:54 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 13:59:15.115386036 +0000 UTC m=+23.289295696" watchObservedRunningTime="2026-09-18 13:59:15.724538154 +0000 UTC m=+23.898447814"
Sep 18 13:59:15.869636 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:59:15.869557    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-hvhcs_75c7d00d-fb1a-4716-9963-b7ef1c0b35bd/node-ca/0.log"
Sep 18 13:59:15.878775 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:59:15.878693    2565 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-09-18T13:59:15.61924493Z","UUID":"6e5a834d-69b3-4815-833c-f8a103d0a42f","Handler":null,"Name":"","Endpoint":""}
Sep 18 13:59:15.880694 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:59:15.880674    2565 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0
Sep 18 13:59:15.880803 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:59:15.880709    2565 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock
Sep 18 13:59:15.911016 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:59:15.910993    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-xqrkr"
Sep 18 13:59:15.911201 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:59:15.911115    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-vpgtc"
Sep 18 13:59:15.911201 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:59:15.911126    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-d4rf2"
Sep 18 13:59:15.911201 ip-10-0-129-16 kubenswrapper[2565]: E0918 13:59:15.911121    2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-xqrkr" podUID="d83ca837-1f57-4ff5-a513-a35f36198a99"
Sep 18 13:59:15.911201 ip-10-0-129-16 kubenswrapper[2565]: E0918 13:59:15.911204    2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-vpgtc" podUID="7fe04bef-1706-4159-8a98-89a352d62c7a"
Sep 18 13:59:15.911387 ip-10-0-129-16 kubenswrapper[2565]: E0918 13:59:15.911256    2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-d4rf2" podUID="46552e87-3933-4e45-97c0-257d31588c9d"
Sep 18 13:59:16.017439 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:59:16.017402    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-6nt4s" event={"ID":"7623391e-952a-4649-81ad-9c59628e3cbf","Type":"ContainerStarted","Data":"740e76e879ad4a8dc7f09341643651598479ee969d89f95f106e8565d19a74be"}
Sep 18 13:59:16.019362 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:59:16.019334    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2jfng" event={"ID":"32ca2a66-ab18-447c-8120-7324ae6f94e4","Type":"ContainerStarted","Data":"a8bc176d4843eb930d947415a23db2c7e71ab874d81cf48a993a5e45d61a6c51"}
Sep 18 13:59:16.019588 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:59:16.019571    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-rzgjk"
Sep 18 13:59:16.020061 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:59:16.020045    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-rzgjk"
Sep 18 13:59:16.037046 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:59:16.036991    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-6nt4s" podStartSLOduration=4.95558164 podStartE2EDuration="22.036974204s" totalImagesPullingTime="17.081392564s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-18 13:58:54 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 13:59:16.036548514 +0000 UTC m=+24.210458186" watchObservedRunningTime="2026-09-18 13:59:16.036974204 +0000 UTC m=+24.210883865"
Sep 18 13:59:17.024644 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:59:17.024603    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2jfng" event={"ID":"32ca2a66-ab18-447c-8120-7324ae6f94e4","Type":"ContainerStarted","Data":"0af165d2f41022e3ad93fd63f9f6414ed71cfa2223f929e1f3df99a5b10e4d9d"}
Sep 18 13:59:17.047004 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:59:17.046965    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2jfng" podStartSLOduration=3.926608012 podStartE2EDuration="23.046947111s" totalImagesPullingTime="19.120339099s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-18 13:58:54 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 13:59:17.045519146 +0000 UTC m=+25.219428854" watchObservedRunningTime="2026-09-18 13:59:17.046947111 +0000 UTC m=+25.220856772"
Sep 18 13:59:17.906870 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:59:17.906836    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-d4rf2"
Sep 18 13:59:17.907080 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:59:17.906836    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-vpgtc"
Sep 18 13:59:17.907080 ip-10-0-129-16 kubenswrapper[2565]: E0918 13:59:17.906949    2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-d4rf2" podUID="46552e87-3933-4e45-97c0-257d31588c9d"
Sep 18 13:59:17.907080 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:59:17.906843    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-xqrkr"
Sep 18 13:59:17.907313 ip-10-0-129-16 kubenswrapper[2565]: E0918 13:59:17.907065    2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-vpgtc" podUID="7fe04bef-1706-4159-8a98-89a352d62c7a"
Sep 18 13:59:17.907313 ip-10-0-129-16 kubenswrapper[2565]: E0918 13:59:17.907158    2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-xqrkr" podUID="d83ca837-1f57-4ff5-a513-a35f36198a99"
Sep 18 13:59:18.030476 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:59:18.030298    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-bvqls_0b0463f5-d647-4a38-b752-44afba77daff/ovn-acl-logging/0.log"
Sep 18 13:59:18.030916 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:59:18.030882    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-bvqls" event={"ID":"0b0463f5-d647-4a38-b752-44afba77daff","Type":"ContainerStarted","Data":"c90889b478caf7d8d674a0f7f7446a676fcf19484f8c9f40df647754699f1b51"}
Sep 18 13:59:19.906958 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:59:19.906761    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-d4rf2"
Sep 18 13:59:19.907713 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:59:19.906761    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-xqrkr"
Sep 18 13:59:19.907713 ip-10-0-129-16 kubenswrapper[2565]: E0918 13:59:19.907027    2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-d4rf2" podUID="46552e87-3933-4e45-97c0-257d31588c9d"
Sep 18 13:59:19.907713 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:59:19.906839    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-vpgtc"
Sep 18 13:59:19.907713 ip-10-0-129-16 kubenswrapper[2565]: E0918 13:59:19.907119    2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-xqrkr" podUID="d83ca837-1f57-4ff5-a513-a35f36198a99"
Sep 18 13:59:19.907713 ip-10-0-129-16 kubenswrapper[2565]: E0918 13:59:19.907214    2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-vpgtc" podUID="7fe04bef-1706-4159-8a98-89a352d62c7a"
Sep 18 13:59:20.039021 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:59:20.038995    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-bvqls_0b0463f5-d647-4a38-b752-44afba77daff/ovn-acl-logging/0.log"
Sep 18 13:59:20.039462 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:59:20.039430    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-bvqls" event={"ID":"0b0463f5-d647-4a38-b752-44afba77daff","Type":"ContainerStarted","Data":"72ebea80fac90718323cb7b417be1a0a80405ced77e63d511397310b9ad27fb2"}
Sep 18 13:59:20.039699 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:59:20.039676    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-bvqls"
Sep 18 13:59:20.039831 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:59:20.039743    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-bvqls"
Sep 18 13:59:20.039879 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:59:20.039852    2565 scope.go:117] "RemoveContainer" containerID="c76ad955c00fe81107bf5dda1a2af48ea1d7d8121f6570c363cba1bca9f37e39"
Sep 18 13:59:20.041440 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:59:20.041413    2565 generic.go:356] "Generic (PLEG): container finished" podID="19a8e259-6225-41e6-9556-f2439e36171d" containerID="bd27bd79268e9e69295f125a21cf313a5813c950de74dbb6277fadd2c7794891" exitCode=0
Sep 18 13:59:20.041547 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:59:20.041462    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-68wp2" event={"ID":"19a8e259-6225-41e6-9556-f2439e36171d","Type":"ContainerDied","Data":"bd27bd79268e9e69295f125a21cf313a5813c950de74dbb6277fadd2c7794891"}
Sep 18 13:59:20.056835 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:59:20.056815    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-bvqls"
Sep 18 13:59:21.047088 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:59:21.047064    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-bvqls_0b0463f5-d647-4a38-b752-44afba77daff/ovn-acl-logging/0.log"
Sep 18 13:59:21.047478 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:59:21.047440    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-bvqls" event={"ID":"0b0463f5-d647-4a38-b752-44afba77daff","Type":"ContainerStarted","Data":"20784380eea9aa003d941ab0336c261845eee6bb101518454589523ec000dba0"}
Sep 18 13:59:21.047652 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:59:21.047635    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-bvqls"
Sep 18 13:59:21.049623 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:59:21.049599    2565 generic.go:356] "Generic (PLEG): container finished" podID="19a8e259-6225-41e6-9556-f2439e36171d" containerID="d07b5d8d11551d0230272a2a0afe6914d53c51d4df4665e00dae8a632f341e29" exitCode=0
Sep 18 13:59:21.049700 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:59:21.049639    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-68wp2" event={"ID":"19a8e259-6225-41e6-9556-f2439e36171d","Type":"ContainerDied","Data":"d07b5d8d11551d0230272a2a0afe6914d53c51d4df4665e00dae8a632f341e29"}
Sep 18 13:59:21.064074 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:59:21.064053    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-bvqls"
Sep 18 13:59:21.077628 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:59:21.077587    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-bvqls" podStartSLOduration=9.668415308 podStartE2EDuration="27.077574659s" totalImagesPullingTime="17.409159351s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-18 13:58:54 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 13:59:21.075917351 +0000 UTC m=+29.249827031" watchObservedRunningTime="2026-09-18 13:59:21.077574659 +0000 UTC m=+29.251484332"
Sep 18 13:59:21.249026 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:59:21.248990    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-xqrkr"]
Sep 18 13:59:21.249191 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:59:21.249130    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-xqrkr"
Sep 18 13:59:21.249264 ip-10-0-129-16 kubenswrapper[2565]: E0918 13:59:21.249240    2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-xqrkr" podUID="d83ca837-1f57-4ff5-a513-a35f36198a99"
Sep 18 13:59:21.253306 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:59:21.253270    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-d4rf2"]
Sep 18 13:59:21.253441 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:59:21.253394    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-d4rf2"
Sep 18 13:59:21.253517 ip-10-0-129-16 kubenswrapper[2565]: E0918 13:59:21.253494    2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-d4rf2" podUID="46552e87-3933-4e45-97c0-257d31588c9d"
Sep 18 13:59:21.254003 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:59:21.253982    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-vpgtc"]
Sep 18 13:59:21.254119 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:59:21.254102    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-vpgtc"
Sep 18 13:59:21.254224 ip-10-0-129-16 kubenswrapper[2565]: E0918 13:59:21.254202    2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-vpgtc" podUID="7fe04bef-1706-4159-8a98-89a352d62c7a"
Sep 18 13:59:22.906524 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:59:22.906489    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-vpgtc"
Sep 18 13:59:22.906524 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:59:22.906528    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-d4rf2"
Sep 18 13:59:22.906943 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:59:22.906587    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-xqrkr"
Sep 18 13:59:22.906943 ip-10-0-129-16 kubenswrapper[2565]: E0918 13:59:22.906600    2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-vpgtc" podUID="7fe04bef-1706-4159-8a98-89a352d62c7a"
Sep 18 13:59:22.906943 ip-10-0-129-16 kubenswrapper[2565]: E0918 13:59:22.906652    2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-xqrkr" podUID="d83ca837-1f57-4ff5-a513-a35f36198a99"
Sep 18 13:59:22.906943 ip-10-0-129-16 kubenswrapper[2565]: E0918 13:59:22.906723    2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-d4rf2" podUID="46552e87-3933-4e45-97c0-257d31588c9d"
Sep 18 13:59:23.056763 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:59:23.056723    2565 generic.go:356] "Generic (PLEG): container finished" podID="19a8e259-6225-41e6-9556-f2439e36171d" containerID="ef0b37a1133b42749e35a1219582c9939a60c7608c5484bed3a6baf0574504e2" exitCode=0
Sep 18 13:59:23.056896 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:59:23.056797    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-68wp2" event={"ID":"19a8e259-6225-41e6-9556-f2439e36171d","Type":"ContainerDied","Data":"ef0b37a1133b42749e35a1219582c9939a60c7608c5484bed3a6baf0574504e2"}
Sep 18 13:59:24.906656 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:59:24.906432    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-d4rf2"
Sep 18 13:59:24.907115 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:59:24.906434    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-vpgtc"
Sep 18 13:59:24.907115 ip-10-0-129-16 kubenswrapper[2565]: E0918 13:59:24.906728    2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-d4rf2" podUID="46552e87-3933-4e45-97c0-257d31588c9d"
Sep 18 13:59:24.907115 ip-10-0-129-16 kubenswrapper[2565]: E0918 13:59:24.906854    2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-vpgtc" podUID="7fe04bef-1706-4159-8a98-89a352d62c7a"
Sep 18 13:59:24.907115 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:59:24.906471    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-xqrkr"
Sep 18 13:59:24.907115 ip-10-0-129-16 kubenswrapper[2565]: E0918 13:59:24.906997    2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-xqrkr" podUID="d83ca837-1f57-4ff5-a513-a35f36198a99"
Sep 18 13:59:26.906145 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:59:26.906110    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-xqrkr"
Sep 18 13:59:26.906145 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:59:26.906128    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-vpgtc"
Sep 18 13:59:26.906657 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:59:26.906127    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-d4rf2"
Sep 18 13:59:26.906657 ip-10-0-129-16 kubenswrapper[2565]: E0918 13:59:26.906233    2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-xqrkr" podUID="d83ca837-1f57-4ff5-a513-a35f36198a99"
Sep 18 13:59:26.906657 ip-10-0-129-16 kubenswrapper[2565]: E0918 13:59:26.906303    2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-vpgtc" podUID="7fe04bef-1706-4159-8a98-89a352d62c7a"
Sep 18 13:59:26.906657 ip-10-0-129-16 kubenswrapper[2565]: E0918 13:59:26.906376    2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-d4rf2" podUID="46552e87-3933-4e45-97c0-257d31588c9d"
Sep 18 13:59:27.069436 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:59:27.069407    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-68wp2" event={"ID":"19a8e259-6225-41e6-9556-f2439e36171d","Type":"ContainerStarted","Data":"cbd91d59a405f2ef295779109461f3ef054a49fb6c556a5c73af64f92e2ffd1c"}
Sep 18 13:59:27.548146 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:59:27.548117    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7fe04bef-1706-4159-8a98-89a352d62c7a-metrics-certs\") pod \"network-metrics-daemon-vpgtc\" (UID: \"7fe04bef-1706-4159-8a98-89a352d62c7a\") " pod="openshift-multus/network-metrics-daemon-vpgtc"
Sep 18 13:59:27.548301 ip-10-0-129-16 kubenswrapper[2565]: E0918 13:59:27.548237    2565 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 18 13:59:27.548301 ip-10-0-129-16 kubenswrapper[2565]: E0918 13:59:27.548290    2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7fe04bef-1706-4159-8a98-89a352d62c7a-metrics-certs podName:7fe04bef-1706-4159-8a98-89a352d62c7a nodeName:}" failed. No retries permitted until 2026-09-18 13:59:59.548276451 +0000 UTC m=+67.722186088 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/7fe04bef-1706-4159-8a98-89a352d62c7a-metrics-certs") pod "network-metrics-daemon-vpgtc" (UID: "7fe04bef-1706-4159-8a98-89a352d62c7a") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 18 13:59:27.648600 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:59:27.648558    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gqntj\" (UniqueName: \"kubernetes.io/projected/46552e87-3933-4e45-97c0-257d31588c9d-kube-api-access-gqntj\") pod \"network-check-target-d4rf2\" (UID: \"46552e87-3933-4e45-97c0-257d31588c9d\") " pod="openshift-network-diagnostics/network-check-target-d4rf2"
Sep 18 13:59:27.648754 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:59:27.648631    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/d83ca837-1f57-4ff5-a513-a35f36198a99-original-pull-secret\") pod \"global-pull-secret-syncer-xqrkr\" (UID: \"d83ca837-1f57-4ff5-a513-a35f36198a99\") " pod="kube-system/global-pull-secret-syncer-xqrkr"
Sep 18 13:59:27.648754 ip-10-0-129-16 kubenswrapper[2565]: E0918 13:59:27.648730    2565 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 18 13:59:27.648825 ip-10-0-129-16 kubenswrapper[2565]: E0918 13:59:27.648787    2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d83ca837-1f57-4ff5-a513-a35f36198a99-original-pull-secret podName:d83ca837-1f57-4ff5-a513-a35f36198a99 nodeName:}" failed. No retries permitted until 2026-09-18 13:59:59.648774394 +0000 UTC m=+67.822684045 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/d83ca837-1f57-4ff5-a513-a35f36198a99-original-pull-secret") pod "global-pull-secret-syncer-xqrkr" (UID: "d83ca837-1f57-4ff5-a513-a35f36198a99") : object "kube-system"/"original-pull-secret" not registered
Sep 18 13:59:27.648825 ip-10-0-129-16 kubenswrapper[2565]: E0918 13:59:27.648732    2565 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 18 13:59:27.648900 ip-10-0-129-16 kubenswrapper[2565]: E0918 13:59:27.648834    2565 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 18 13:59:27.648900 ip-10-0-129-16 kubenswrapper[2565]: E0918 13:59:27.648848    2565 projected.go:196] Error preparing data for projected volume kube-api-access-gqntj for pod openshift-network-diagnostics/network-check-target-d4rf2: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 18 13:59:27.648900 ip-10-0-129-16 kubenswrapper[2565]: E0918 13:59:27.648894    2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/46552e87-3933-4e45-97c0-257d31588c9d-kube-api-access-gqntj podName:46552e87-3933-4e45-97c0-257d31588c9d nodeName:}" failed. No retries permitted until 2026-09-18 13:59:59.648882194 +0000 UTC m=+67.822791842 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-gqntj" (UniqueName: "kubernetes.io/projected/46552e87-3933-4e45-97c0-257d31588c9d-kube-api-access-gqntj") pod "network-check-target-d4rf2" (UID: "46552e87-3933-4e45-97c0-257d31588c9d") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 18 13:59:28.074773 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:59:28.074743    2565 generic.go:356] "Generic (PLEG): container finished" podID="19a8e259-6225-41e6-9556-f2439e36171d" containerID="cbd91d59a405f2ef295779109461f3ef054a49fb6c556a5c73af64f92e2ffd1c" exitCode=0
Sep 18 13:59:28.075230 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:59:28.074790    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-68wp2" event={"ID":"19a8e259-6225-41e6-9556-f2439e36171d","Type":"ContainerDied","Data":"cbd91d59a405f2ef295779109461f3ef054a49fb6c556a5c73af64f92e2ffd1c"}
Sep 18 13:59:28.906295 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:59:28.906263    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-xqrkr"
Sep 18 13:59:28.906462 ip-10-0-129-16 kubenswrapper[2565]: E0918 13:59:28.906362    2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-xqrkr" podUID="d83ca837-1f57-4ff5-a513-a35f36198a99"
Sep 18 13:59:28.906462 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:59:28.906257    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-d4rf2"
Sep 18 13:59:28.906535 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:59:28.906270    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-vpgtc"
Sep 18 13:59:28.906571 ip-10-0-129-16 kubenswrapper[2565]: E0918 13:59:28.906555    2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-vpgtc" podUID="7fe04bef-1706-4159-8a98-89a352d62c7a"
Sep 18 13:59:28.906606 ip-10-0-129-16 kubenswrapper[2565]: E0918 13:59:28.906455    2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-d4rf2" podUID="46552e87-3933-4e45-97c0-257d31588c9d"
Sep 18 13:59:29.080425 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:59:29.080385    2565 generic.go:356] "Generic (PLEG): container finished" podID="19a8e259-6225-41e6-9556-f2439e36171d" containerID="c1bfe997b4a2641bce3a8af567be570500b0a4cdcd81e5ccd4c62e9dd34c2639" exitCode=0
Sep 18 13:59:29.080814 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:59:29.080428    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-68wp2" event={"ID":"19a8e259-6225-41e6-9556-f2439e36171d","Type":"ContainerDied","Data":"c1bfe997b4a2641bce3a8af567be570500b0a4cdcd81e5ccd4c62e9dd34c2639"}
Sep 18 13:59:30.085758 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:59:30.085723    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-68wp2" event={"ID":"19a8e259-6225-41e6-9556-f2439e36171d","Type":"ContainerStarted","Data":"5dbf347cc461a1ba20cd2163064d731501cf15b3ed6a03878448c4592e8a5be2"}
Sep 18 13:59:30.121183 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:59:30.121139    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-68wp2" podStartSLOduration=9.258864303 podStartE2EDuration="36.121123555s" totalImagesPullingTime="26.862259252s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-18 13:58:54 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 13:59:30.120548464 +0000 UTC m=+38.294458141" watchObservedRunningTime="2026-09-18 13:59:30.121123555 +0000 UTC m=+38.295033214"
Sep 18 13:59:30.906931 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:59:30.906892    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-vpgtc"
Sep 18 13:59:30.906931 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:59:30.906919    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-d4rf2"
Sep 18 13:59:30.907156 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:59:30.906919    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-xqrkr"
Sep 18 13:59:30.907156 ip-10-0-129-16 kubenswrapper[2565]: E0918 13:59:30.907053    2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-vpgtc" podUID="7fe04bef-1706-4159-8a98-89a352d62c7a"
Sep 18 13:59:30.907156 ip-10-0-129-16 kubenswrapper[2565]: E0918 13:59:30.907109    2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-d4rf2" podUID="46552e87-3933-4e45-97c0-257d31588c9d"
Sep 18 13:59:30.907253 ip-10-0-129-16 kubenswrapper[2565]: E0918 13:59:30.907181    2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-xqrkr" podUID="d83ca837-1f57-4ff5-a513-a35f36198a99"
Sep 18 13:59:32.906518 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:59:32.906485    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-vpgtc"
Sep 18 13:59:32.907060 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:59:32.906524    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-d4rf2"
Sep 18 13:59:32.907060 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:59:32.906580    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-xqrkr"
Sep 18 13:59:32.907060 ip-10-0-129-16 kubenswrapper[2565]: E0918 13:59:32.906593    2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-vpgtc" podUID="7fe04bef-1706-4159-8a98-89a352d62c7a"
Sep 18 13:59:32.907060 ip-10-0-129-16 kubenswrapper[2565]: E0918 13:59:32.906648    2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-xqrkr" podUID="d83ca837-1f57-4ff5-a513-a35f36198a99"
Sep 18 13:59:32.907060 ip-10-0-129-16 kubenswrapper[2565]: E0918 13:59:32.906703    2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-d4rf2" podUID="46552e87-3933-4e45-97c0-257d31588c9d"
Sep 18 13:59:34.906661 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:59:34.906618    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-vpgtc"
Sep 18 13:59:34.907092 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:59:34.906618    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-xqrkr"
Sep 18 13:59:34.907092 ip-10-0-129-16 kubenswrapper[2565]: E0918 13:59:34.906741    2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-vpgtc" podUID="7fe04bef-1706-4159-8a98-89a352d62c7a"
Sep 18 13:59:34.907092 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:59:34.906618    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-d4rf2"
Sep 18 13:59:34.907092 ip-10-0-129-16 kubenswrapper[2565]: E0918 13:59:34.906798    2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-xqrkr" podUID="d83ca837-1f57-4ff5-a513-a35f36198a99"
Sep 18 13:59:34.907092 ip-10-0-129-16 kubenswrapper[2565]: E0918 13:59:34.906881    2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-d4rf2" podUID="46552e87-3933-4e45-97c0-257d31588c9d"
Sep 18 13:59:36.906552 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:59:36.906513    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-d4rf2"
Sep 18 13:59:36.907141 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:59:36.906559    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-vpgtc"
Sep 18 13:59:36.907141 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:59:36.906513    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-xqrkr"
Sep 18 13:59:36.907141 ip-10-0-129-16 kubenswrapper[2565]: E0918 13:59:36.906649    2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-d4rf2" podUID="46552e87-3933-4e45-97c0-257d31588c9d"
Sep 18 13:59:36.907141 ip-10-0-129-16 kubenswrapper[2565]: E0918 13:59:36.906714    2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-xqrkr" podUID="d83ca837-1f57-4ff5-a513-a35f36198a99"
Sep 18 13:59:36.907141 ip-10-0-129-16 kubenswrapper[2565]: E0918 13:59:36.906775    2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-vpgtc" podUID="7fe04bef-1706-4159-8a98-89a352d62c7a"
Sep 18 13:59:38.906615 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:59:38.906578    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-vpgtc"
Sep 18 13:59:38.906615 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:59:38.906607    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-d4rf2"
Sep 18 13:59:38.906615 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:59:38.906613    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-xqrkr"
Sep 18 13:59:38.907205 ip-10-0-129-16 kubenswrapper[2565]: E0918 13:59:38.906721    2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-vpgtc" podUID="7fe04bef-1706-4159-8a98-89a352d62c7a"
Sep 18 13:59:38.907205 ip-10-0-129-16 kubenswrapper[2565]: E0918 13:59:38.906825    2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-d4rf2" podUID="46552e87-3933-4e45-97c0-257d31588c9d"
Sep 18 13:59:38.907205 ip-10-0-129-16 kubenswrapper[2565]: E0918 13:59:38.906897    2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-xqrkr" podUID="d83ca837-1f57-4ff5-a513-a35f36198a99"
Sep 18 13:59:40.906817 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:59:40.906778    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-vpgtc"
Sep 18 13:59:40.907238 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:59:40.906828    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-d4rf2"
Sep 18 13:59:40.907238 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:59:40.906791    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-xqrkr"
Sep 18 13:59:40.907238 ip-10-0-129-16 kubenswrapper[2565]: E0918 13:59:40.906902    2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-vpgtc" podUID="7fe04bef-1706-4159-8a98-89a352d62c7a"
Sep 18 13:59:40.907238 ip-10-0-129-16 kubenswrapper[2565]: E0918 13:59:40.906970    2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-xqrkr" podUID="d83ca837-1f57-4ff5-a513-a35f36198a99"
Sep 18 13:59:40.907238 ip-10-0-129-16 kubenswrapper[2565]: E0918 13:59:40.907057    2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-d4rf2" podUID="46552e87-3933-4e45-97c0-257d31588c9d"
Sep 18 13:59:42.906644 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:59:42.906609    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-vpgtc"
Sep 18 13:59:42.906644 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:59:42.906634    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-d4rf2"
Sep 18 13:59:42.907141 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:59:42.906721    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-xqrkr"
Sep 18 13:59:42.907141 ip-10-0-129-16 kubenswrapper[2565]: E0918 13:59:42.906723    2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-vpgtc" podUID="7fe04bef-1706-4159-8a98-89a352d62c7a"
Sep 18 13:59:42.907141 ip-10-0-129-16 kubenswrapper[2565]: E0918 13:59:42.906780    2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-d4rf2" podUID="46552e87-3933-4e45-97c0-257d31588c9d"
Sep 18 13:59:42.907141 ip-10-0-129-16 kubenswrapper[2565]: E0918 13:59:42.906858    2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-xqrkr" podUID="d83ca837-1f57-4ff5-a513-a35f36198a99"
Sep 18 13:59:44.225057 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:59:44.225013    2565 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-16.ec2.internal" event="NodeReady"
Sep 18 13:59:44.225428 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:59:44.225164    2565 kubelet_node_status.go:489] "Fast updating node status as it just became ready"
Sep 18 13:59:44.321951 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:59:44.321918    2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-msrpt"]
Sep 18 13:59:44.344924 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:59:44.344897    2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-54mm2"]
Sep 18 13:59:44.345108 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:59:44.345039    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-msrpt"
Sep 18 13:59:44.347891 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:59:44.347838    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\""
Sep 18 13:59:44.348049 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:59:44.347909    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\""
Sep 18 13:59:44.348049 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:59:44.347960    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\""
Sep 18 13:59:44.348301 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:59:44.348281    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\""
Sep 18 13:59:44.348710 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:59:44.348693    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-p5mbv\""
Sep 18 13:59:44.362937 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:59:44.362913    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-54mm2"]
Sep 18 13:59:44.363076 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:59:44.362951    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-msrpt"]
Sep 18 13:59:44.363076 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:59:44.362982    2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-bzgmq"]
Sep 18 13:59:44.363076 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:59:44.363023    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-54mm2"
Sep 18 13:59:44.365776 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:59:44.365755    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-fh88p\""
Sep 18 13:59:44.366047 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:59:44.366012    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\""
Sep 18 13:59:44.366343 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:59:44.366325    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\""
Sep 18 13:59:44.383394 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:59:44.383372    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-bzgmq"]
Sep 18 13:59:44.383500 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:59:44.383418    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-bzgmq"
Sep 18 13:59:44.387157 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:59:44.387135    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\""
Sep 18 13:59:44.387262 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:59:44.387174    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\""
Sep 18 13:59:44.387262 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:59:44.387223    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-5l9f5\""
Sep 18 13:59:44.393266 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:59:44.393250    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\""
Sep 18 13:59:44.460251 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:59:44.460215    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/cbf59139-0ecb-4f9c-bc28-0045adf52ecf-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-msrpt\" (UID: \"cbf59139-0ecb-4f9c-bc28-0045adf52ecf\") " pod="openshift-insights/insights-runtime-extractor-msrpt"
Sep 18 13:59:44.460251 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:59:44.460250    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/95c207c5-e264-4f7d-b4b4-9b48bee32aa6-metrics-tls\") pod \"dns-default-54mm2\" (UID: \"95c207c5-e264-4f7d-b4b4-9b48bee32aa6\") " pod="openshift-dns/dns-default-54mm2"
Sep 18 13:59:44.460522 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:59:44.460271    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/95c207c5-e264-4f7d-b4b4-9b48bee32aa6-config-volume\") pod \"dns-default-54mm2\" (UID: \"95c207c5-e264-4f7d-b4b4-9b48bee32aa6\") " pod="openshift-dns/dns-default-54mm2"
Sep 18 13:59:44.460522 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:59:44.460311    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/cbf59139-0ecb-4f9c-bc28-0045adf52ecf-crio-socket\") pod \"insights-runtime-extractor-msrpt\" (UID: \"cbf59139-0ecb-4f9c-bc28-0045adf52ecf\") " pod="openshift-insights/insights-runtime-extractor-msrpt"
Sep 18 13:59:44.460522 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:59:44.460382    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/cbf59139-0ecb-4f9c-bc28-0045adf52ecf-data-volume\") pod \"insights-runtime-extractor-msrpt\" (UID: \"cbf59139-0ecb-4f9c-bc28-0045adf52ecf\") " pod="openshift-insights/insights-runtime-extractor-msrpt"
Sep 18 13:59:44.460522 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:59:44.460404    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/cbf59139-0ecb-4f9c-bc28-0045adf52ecf-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-msrpt\" (UID: \"cbf59139-0ecb-4f9c-bc28-0045adf52ecf\") " pod="openshift-insights/insights-runtime-extractor-msrpt"
Sep 18 13:59:44.460522 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:59:44.460421    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jndw4\" (UniqueName: \"kubernetes.io/projected/cbf59139-0ecb-4f9c-bc28-0045adf52ecf-kube-api-access-jndw4\") pod \"insights-runtime-extractor-msrpt\" (UID: \"cbf59139-0ecb-4f9c-bc28-0045adf52ecf\") " pod="openshift-insights/insights-runtime-extractor-msrpt"
Sep 18 13:59:44.460522 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:59:44.460438    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/95c207c5-e264-4f7d-b4b4-9b48bee32aa6-tmp-dir\") pod \"dns-default-54mm2\" (UID: \"95c207c5-e264-4f7d-b4b4-9b48bee32aa6\") " pod="openshift-dns/dns-default-54mm2"
Sep 18 13:59:44.460522 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:59:44.460474    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vvw42\" (UniqueName: \"kubernetes.io/projected/95c207c5-e264-4f7d-b4b4-9b48bee32aa6-kube-api-access-vvw42\") pod \"dns-default-54mm2\" (UID: \"95c207c5-e264-4f7d-b4b4-9b48bee32aa6\") " pod="openshift-dns/dns-default-54mm2"
Sep 18 13:59:44.561564 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:59:44.561529    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/cbf59139-0ecb-4f9c-bc28-0045adf52ecf-data-volume\") pod \"insights-runtime-extractor-msrpt\" (UID: \"cbf59139-0ecb-4f9c-bc28-0045adf52ecf\") " pod="openshift-insights/insights-runtime-extractor-msrpt"
Sep 18 13:59:44.561748 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:59:44.561573    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/cbf59139-0ecb-4f9c-bc28-0045adf52ecf-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-msrpt\" (UID: \"cbf59139-0ecb-4f9c-bc28-0045adf52ecf\") " pod="openshift-insights/insights-runtime-extractor-msrpt"
Sep 18 13:59:44.561748 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:59:44.561604    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jndw4\" (UniqueName: \"kubernetes.io/projected/cbf59139-0ecb-4f9c-bc28-0045adf52ecf-kube-api-access-jndw4\") pod \"insights-runtime-extractor-msrpt\" (UID: \"cbf59139-0ecb-4f9c-bc28-0045adf52ecf\") " pod="openshift-insights/insights-runtime-extractor-msrpt"
Sep 18 13:59:44.561748 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:59:44.561631    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/95c207c5-e264-4f7d-b4b4-9b48bee32aa6-tmp-dir\") pod \"dns-default-54mm2\" (UID: \"95c207c5-e264-4f7d-b4b4-9b48bee32aa6\") " pod="openshift-dns/dns-default-54mm2"
Sep 18 13:59:44.561748 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:59:44.561662    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vvw42\" (UniqueName: \"kubernetes.io/projected/95c207c5-e264-4f7d-b4b4-9b48bee32aa6-kube-api-access-vvw42\") pod \"dns-default-54mm2\" (UID: \"95c207c5-e264-4f7d-b4b4-9b48bee32aa6\") " pod="openshift-dns/dns-default-54mm2"
Sep 18 13:59:44.561748 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:59:44.561693    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/cbf59139-0ecb-4f9c-bc28-0045adf52ecf-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-msrpt\" (UID: \"cbf59139-0ecb-4f9c-bc28-0045adf52ecf\") " pod="openshift-insights/insights-runtime-extractor-msrpt"
Sep 18 13:59:44.561748 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:59:44.561720    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/95c207c5-e264-4f7d-b4b4-9b48bee32aa6-metrics-tls\") pod \"dns-default-54mm2\" (UID: \"95c207c5-e264-4f7d-b4b4-9b48bee32aa6\") " pod="openshift-dns/dns-default-54mm2"
Sep 18 13:59:44.561748 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:59:44.561745    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/fc0c22c0-58aa-4c0e-9d86-ef684058c2ca-cert\") pod \"ingress-canary-bzgmq\" (UID: \"fc0c22c0-58aa-4c0e-9d86-ef684058c2ca\") " pod="openshift-ingress-canary/ingress-canary-bzgmq"
Sep 18 13:59:44.562139 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:59:44.561768    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nnjzw\" (UniqueName: \"kubernetes.io/projected/fc0c22c0-58aa-4c0e-9d86-ef684058c2ca-kube-api-access-nnjzw\") pod \"ingress-canary-bzgmq\" (UID: \"fc0c22c0-58aa-4c0e-9d86-ef684058c2ca\") " pod="openshift-ingress-canary/ingress-canary-bzgmq"
Sep 18 13:59:44.562139 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:59:44.561799    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/95c207c5-e264-4f7d-b4b4-9b48bee32aa6-config-volume\") pod \"dns-default-54mm2\" (UID: \"95c207c5-e264-4f7d-b4b4-9b48bee32aa6\") " pod="openshift-dns/dns-default-54mm2"
Sep 18 13:59:44.562139 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:59:44.561826    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/cbf59139-0ecb-4f9c-bc28-0045adf52ecf-crio-socket\") pod \"insights-runtime-extractor-msrpt\" (UID: \"cbf59139-0ecb-4f9c-bc28-0045adf52ecf\") " pod="openshift-insights/insights-runtime-extractor-msrpt"
Sep 18 13:59:44.562139 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:59:44.561930    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/cbf59139-0ecb-4f9c-bc28-0045adf52ecf-crio-socket\") pod \"insights-runtime-extractor-msrpt\" (UID: \"cbf59139-0ecb-4f9c-bc28-0045adf52ecf\") " pod="openshift-insights/insights-runtime-extractor-msrpt"
Sep 18 13:59:44.562139 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:59:44.561948    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/cbf59139-0ecb-4f9c-bc28-0045adf52ecf-data-volume\") pod \"insights-runtime-extractor-msrpt\" (UID: \"cbf59139-0ecb-4f9c-bc28-0045adf52ecf\") " pod="openshift-insights/insights-runtime-extractor-msrpt"
Sep 18 13:59:44.562316 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:59:44.562159    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/95c207c5-e264-4f7d-b4b4-9b48bee32aa6-tmp-dir\") pod \"dns-default-54mm2\" (UID: \"95c207c5-e264-4f7d-b4b4-9b48bee32aa6\") " pod="openshift-dns/dns-default-54mm2"
Sep 18 13:59:44.562394 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:59:44.562373    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/95c207c5-e264-4f7d-b4b4-9b48bee32aa6-config-volume\") pod \"dns-default-54mm2\" (UID: \"95c207c5-e264-4f7d-b4b4-9b48bee32aa6\") " pod="openshift-dns/dns-default-54mm2"
Sep 18 13:59:44.565705 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:59:44.565683    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/95c207c5-e264-4f7d-b4b4-9b48bee32aa6-metrics-tls\") pod \"dns-default-54mm2\" (UID: \"95c207c5-e264-4f7d-b4b4-9b48bee32aa6\") " pod="openshift-dns/dns-default-54mm2"
Sep 18 13:59:44.565781 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:59:44.565687    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/cbf59139-0ecb-4f9c-bc28-0045adf52ecf-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-msrpt\" (UID: \"cbf59139-0ecb-4f9c-bc28-0045adf52ecf\") " pod="openshift-insights/insights-runtime-extractor-msrpt"
Sep 18 13:59:44.579432 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:59:44.579412    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/cbf59139-0ecb-4f9c-bc28-0045adf52ecf-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-msrpt\" (UID: \"cbf59139-0ecb-4f9c-bc28-0045adf52ecf\") " pod="openshift-insights/insights-runtime-extractor-msrpt"
Sep 18 13:59:44.589346 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:59:44.589319    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jndw4\" (UniqueName: \"kubernetes.io/projected/cbf59139-0ecb-4f9c-bc28-0045adf52ecf-kube-api-access-jndw4\") pod \"insights-runtime-extractor-msrpt\" (UID: \"cbf59139-0ecb-4f9c-bc28-0045adf52ecf\") " pod="openshift-insights/insights-runtime-extractor-msrpt"
Sep 18 13:59:44.590381 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:59:44.590365    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vvw42\" (UniqueName: \"kubernetes.io/projected/95c207c5-e264-4f7d-b4b4-9b48bee32aa6-kube-api-access-vvw42\") pod \"dns-default-54mm2\" (UID: \"95c207c5-e264-4f7d-b4b4-9b48bee32aa6\") " pod="openshift-dns/dns-default-54mm2"
Sep 18 13:59:44.655895 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:59:44.655858    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-msrpt"
Sep 18 13:59:44.663132 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:59:44.663112    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/fc0c22c0-58aa-4c0e-9d86-ef684058c2ca-cert\") pod \"ingress-canary-bzgmq\" (UID: \"fc0c22c0-58aa-4c0e-9d86-ef684058c2ca\") " pod="openshift-ingress-canary/ingress-canary-bzgmq"
Sep 18 13:59:44.663208 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:59:44.663141    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nnjzw\" (UniqueName: \"kubernetes.io/projected/fc0c22c0-58aa-4c0e-9d86-ef684058c2ca-kube-api-access-nnjzw\") pod \"ingress-canary-bzgmq\" (UID: \"fc0c22c0-58aa-4c0e-9d86-ef684058c2ca\") " pod="openshift-ingress-canary/ingress-canary-bzgmq"
Sep 18 13:59:44.665574 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:59:44.665546    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/fc0c22c0-58aa-4c0e-9d86-ef684058c2ca-cert\") pod \"ingress-canary-bzgmq\" (UID: \"fc0c22c0-58aa-4c0e-9d86-ef684058c2ca\") " pod="openshift-ingress-canary/ingress-canary-bzgmq"
Sep 18 13:59:44.671133 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:59:44.671113    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-54mm2"
Sep 18 13:59:44.673279 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:59:44.673261    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nnjzw\" (UniqueName: \"kubernetes.io/projected/fc0c22c0-58aa-4c0e-9d86-ef684058c2ca-kube-api-access-nnjzw\") pod \"ingress-canary-bzgmq\" (UID: \"fc0c22c0-58aa-4c0e-9d86-ef684058c2ca\") " pod="openshift-ingress-canary/ingress-canary-bzgmq"
Sep 18 13:59:44.691799 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:59:44.691777    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-bzgmq"
Sep 18 13:59:44.907163 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:59:44.907073    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-xqrkr"
Sep 18 13:59:44.907163 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:59:44.907119    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-d4rf2"
Sep 18 13:59:44.907353 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:59:44.907210    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-vpgtc"
Sep 18 13:59:44.909823 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:59:44.909802    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-8cnh9\""
Sep 18 13:59:44.909990 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:59:44.909971    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Sep 18 13:59:44.910060 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:59:44.910045    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Sep 18 13:59:44.910194 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:59:44.910176    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Sep 18 13:59:44.910307 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:59:44.910290    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-z6j5p\""
Sep 18 13:59:44.910395 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:59:44.910381    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Sep 18 13:59:44.969304 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:59:44.969270    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-54mm2"]
Sep 18 13:59:44.973169 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:59:44.972796    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-msrpt"]
Sep 18 13:59:44.973377 ip-10-0-129-16 kubenswrapper[2565]: W0918 13:59:44.973347    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod95c207c5_e264_4f7d_b4b4_9b48bee32aa6.slice/crio-d09c47729751d9f476d1f5f9a314e683de45f8d4f6565aae49e8d597d279bc8c WatchSource:0}: Error finding container d09c47729751d9f476d1f5f9a314e683de45f8d4f6565aae49e8d597d279bc8c: Status 404 returned error can't find the container with id d09c47729751d9f476d1f5f9a314e683de45f8d4f6565aae49e8d597d279bc8c
Sep 18 13:59:44.996959 ip-10-0-129-16 kubenswrapper[2565]: W0918 13:59:44.996921    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfc0c22c0_58aa_4c0e_9d86_ef684058c2ca.slice/crio-f2c9eb8848c7ca914b63dd32dbb8f0945d4073bfcf5902320db1d290cfc8b761 WatchSource:0}: Error finding container f2c9eb8848c7ca914b63dd32dbb8f0945d4073bfcf5902320db1d290cfc8b761: Status 404 returned error can't find the container with id f2c9eb8848c7ca914b63dd32dbb8f0945d4073bfcf5902320db1d290cfc8b761
Sep 18 13:59:44.997974 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:59:44.997889    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-bzgmq"]
Sep 18 13:59:45.122704 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:59:45.122660    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-msrpt" event={"ID":"cbf59139-0ecb-4f9c-bc28-0045adf52ecf","Type":"ContainerStarted","Data":"bd410a7788932d49d7a27a0af52aebc9f1e1b95a537357e29ac356487b08ab34"}
Sep 18 13:59:45.122842 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:59:45.122746    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-msrpt" event={"ID":"cbf59139-0ecb-4f9c-bc28-0045adf52ecf","Type":"ContainerStarted","Data":"edc43bf4b036a7b648c9e7721d3224a223358d23f9a3b110ed60e791ab1b43ed"}
Sep 18 13:59:45.123898 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:59:45.123873    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-bzgmq" event={"ID":"fc0c22c0-58aa-4c0e-9d86-ef684058c2ca","Type":"ContainerStarted","Data":"f2c9eb8848c7ca914b63dd32dbb8f0945d4073bfcf5902320db1d290cfc8b761"}
Sep 18 13:59:45.124952 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:59:45.124933    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-54mm2" event={"ID":"95c207c5-e264-4f7d-b4b4-9b48bee32aa6","Type":"ContainerStarted","Data":"d09c47729751d9f476d1f5f9a314e683de45f8d4f6565aae49e8d597d279bc8c"}
Sep 18 13:59:46.880136 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:59:46.880097    2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-operator-7bd5d56d6-dbqgx"]
Sep 18 13:59:46.890828 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:59:46.890798    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-7bd5d56d6-dbqgx"
Sep 18 13:59:46.894517 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:59:46.894450    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-tls\""
Sep 18 13:59:46.894924 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:59:46.894905    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-kube-rbac-proxy-config\""
Sep 18 13:59:46.895102 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:59:46.894924    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\""
Sep 18 13:59:46.895631 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:59:46.895607    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\""
Sep 18 13:59:46.896142 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:59:46.895990    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-dockercfg-dn8jk\""
Sep 18 13:59:46.896142 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:59:46.896049    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\""
Sep 18 13:59:46.903664 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:59:46.903642    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-7bd5d56d6-dbqgx"]
Sep 18 13:59:46.982016 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:59:46.981981    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cgz7f\" (UniqueName: \"kubernetes.io/projected/8e0b8c6d-ac6c-4621-bc2b-3cc23dbee621-kube-api-access-cgz7f\") pod \"prometheus-operator-7bd5d56d6-dbqgx\" (UID: \"8e0b8c6d-ac6c-4621-bc2b-3cc23dbee621\") " pod="openshift-monitoring/prometheus-operator-7bd5d56d6-dbqgx"
Sep 18 13:59:46.982208 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:59:46.982117    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/8e0b8c6d-ac6c-4621-bc2b-3cc23dbee621-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-7bd5d56d6-dbqgx\" (UID: \"8e0b8c6d-ac6c-4621-bc2b-3cc23dbee621\") " pod="openshift-monitoring/prometheus-operator-7bd5d56d6-dbqgx"
Sep 18 13:59:46.982208 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:59:46.982157    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/8e0b8c6d-ac6c-4621-bc2b-3cc23dbee621-metrics-client-ca\") pod \"prometheus-operator-7bd5d56d6-dbqgx\" (UID: \"8e0b8c6d-ac6c-4621-bc2b-3cc23dbee621\") " pod="openshift-monitoring/prometheus-operator-7bd5d56d6-dbqgx"
Sep 18 13:59:46.982208 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:59:46.982193    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/8e0b8c6d-ac6c-4621-bc2b-3cc23dbee621-prometheus-operator-tls\") pod \"prometheus-operator-7bd5d56d6-dbqgx\" (UID: \"8e0b8c6d-ac6c-4621-bc2b-3cc23dbee621\") " pod="openshift-monitoring/prometheus-operator-7bd5d56d6-dbqgx"
Sep 18 13:59:47.083379 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:59:47.083295    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cgz7f\" (UniqueName: \"kubernetes.io/projected/8e0b8c6d-ac6c-4621-bc2b-3cc23dbee621-kube-api-access-cgz7f\") pod \"prometheus-operator-7bd5d56d6-dbqgx\" (UID: \"8e0b8c6d-ac6c-4621-bc2b-3cc23dbee621\") " pod="openshift-monitoring/prometheus-operator-7bd5d56d6-dbqgx"
Sep 18 13:59:47.083546 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:59:47.083380    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/8e0b8c6d-ac6c-4621-bc2b-3cc23dbee621-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-7bd5d56d6-dbqgx\" (UID: \"8e0b8c6d-ac6c-4621-bc2b-3cc23dbee621\") " pod="openshift-monitoring/prometheus-operator-7bd5d56d6-dbqgx"
Sep 18 13:59:47.083546 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:59:47.083409    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/8e0b8c6d-ac6c-4621-bc2b-3cc23dbee621-metrics-client-ca\") pod \"prometheus-operator-7bd5d56d6-dbqgx\" (UID: \"8e0b8c6d-ac6c-4621-bc2b-3cc23dbee621\") " pod="openshift-monitoring/prometheus-operator-7bd5d56d6-dbqgx"
Sep 18 13:59:47.083546 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:59:47.083438    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/8e0b8c6d-ac6c-4621-bc2b-3cc23dbee621-prometheus-operator-tls\") pod \"prometheus-operator-7bd5d56d6-dbqgx\" (UID: \"8e0b8c6d-ac6c-4621-bc2b-3cc23dbee621\") " pod="openshift-monitoring/prometheus-operator-7bd5d56d6-dbqgx"
Sep 18 13:59:47.084237 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:59:47.084213    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/8e0b8c6d-ac6c-4621-bc2b-3cc23dbee621-metrics-client-ca\") pod \"prometheus-operator-7bd5d56d6-dbqgx\" (UID: \"8e0b8c6d-ac6c-4621-bc2b-3cc23dbee621\") " pod="openshift-monitoring/prometheus-operator-7bd5d56d6-dbqgx"
Sep 18 13:59:47.086263 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:59:47.086236    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/8e0b8c6d-ac6c-4621-bc2b-3cc23dbee621-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-7bd5d56d6-dbqgx\" (UID: \"8e0b8c6d-ac6c-4621-bc2b-3cc23dbee621\") " pod="openshift-monitoring/prometheus-operator-7bd5d56d6-dbqgx"
Sep 18 13:59:47.086364 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:59:47.086241    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/8e0b8c6d-ac6c-4621-bc2b-3cc23dbee621-prometheus-operator-tls\") pod \"prometheus-operator-7bd5d56d6-dbqgx\" (UID: \"8e0b8c6d-ac6c-4621-bc2b-3cc23dbee621\") " pod="openshift-monitoring/prometheus-operator-7bd5d56d6-dbqgx"
Sep 18 13:59:47.093275 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:59:47.093255    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cgz7f\" (UniqueName: \"kubernetes.io/projected/8e0b8c6d-ac6c-4621-bc2b-3cc23dbee621-kube-api-access-cgz7f\") pod \"prometheus-operator-7bd5d56d6-dbqgx\" (UID: \"8e0b8c6d-ac6c-4621-bc2b-3cc23dbee621\") " pod="openshift-monitoring/prometheus-operator-7bd5d56d6-dbqgx"
Sep 18 13:59:47.201508 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:59:47.201472    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-7bd5d56d6-dbqgx"
Sep 18 13:59:48.136991 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:59:48.136292    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-bzgmq" event={"ID":"fc0c22c0-58aa-4c0e-9d86-ef684058c2ca","Type":"ContainerStarted","Data":"4d824bdda87690c8844343bf48970d8f1313aa63813d5b28f09d054e6c3cbcc2"}
Sep 18 13:59:48.140816 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:59:48.140785    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-54mm2" event={"ID":"95c207c5-e264-4f7d-b4b4-9b48bee32aa6","Type":"ContainerStarted","Data":"5699f264ee62023080c4db56872d55361c0a20baf777a397cf3517d78a5660d6"}
Sep 18 13:59:48.143024 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:59:48.142962    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-msrpt" event={"ID":"cbf59139-0ecb-4f9c-bc28-0045adf52ecf","Type":"ContainerStarted","Data":"b41ddd56a49a36e38314aee5de12bdb5047ccadcb2b66ef88fb2ac33df0f9d46"}
Sep 18 13:59:48.154566 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:59:48.154330    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-bzgmq" podStartSLOduration=1.203370605 podStartE2EDuration="4.154315697s" totalImagesPullingTime="2.950945092s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 13:59:44 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 13:59:48.153635231 +0000 UTC m=+56.327544912" watchObservedRunningTime="2026-09-18 13:59:48.154315697 +0000 UTC m=+56.328225348"
Sep 18 13:59:48.203096 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:59:48.203062    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-7bd5d56d6-dbqgx"]
Sep 18 13:59:48.211275 ip-10-0-129-16 kubenswrapper[2565]: W0918 13:59:48.211247    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8e0b8c6d_ac6c_4621_bc2b_3cc23dbee621.slice/crio-1c96e155b5ab98f0cd3dc1f1882c4502e8cb021765c902799feaabd020086948 WatchSource:0}: Error finding container 1c96e155b5ab98f0cd3dc1f1882c4502e8cb021765c902799feaabd020086948: Status 404 returned error can't find the container with id 1c96e155b5ab98f0cd3dc1f1882c4502e8cb021765c902799feaabd020086948
Sep 18 13:59:49.149477 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:59:49.149437    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-54mm2" event={"ID":"95c207c5-e264-4f7d-b4b4-9b48bee32aa6","Type":"ContainerStarted","Data":"7620306c4838be491a32222aa1d38c47a2447cbb21a34a7798a5e9371adc467d"}
Sep 18 13:59:49.149989 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:59:49.149555    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-54mm2"
Sep 18 13:59:49.151571 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:59:49.151544    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-7bd5d56d6-dbqgx" event={"ID":"8e0b8c6d-ac6c-4621-bc2b-3cc23dbee621","Type":"ContainerStarted","Data":"1c96e155b5ab98f0cd3dc1f1882c4502e8cb021765c902799feaabd020086948"}
Sep 18 13:59:49.167309 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:59:49.167257    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-54mm2" podStartSLOduration=2.202034127 podStartE2EDuration="5.167239681s" totalImagesPullingTime="2.965205554s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 13:59:44 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 13:59:49.165909446 +0000 UTC m=+57.339819134" watchObservedRunningTime="2026-09-18 13:59:49.167239681 +0000 UTC m=+57.341149341"
Sep 18 13:59:50.157520 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:59:50.157426    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-msrpt" event={"ID":"cbf59139-0ecb-4f9c-bc28-0045adf52ecf","Type":"ContainerStarted","Data":"c8f42ec3cd1c2f32271151677df4d9aafac41d6b187fd57943cc7187000b989d"}
Sep 18 13:59:50.159304 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:59:50.159276    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-7bd5d56d6-dbqgx" event={"ID":"8e0b8c6d-ac6c-4621-bc2b-3cc23dbee621","Type":"ContainerStarted","Data":"779bae487bb9521c61ccbebb8d58a4325662a1061229affe4912941f355a1c28"}
Sep 18 13:59:50.159425 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:59:50.159329    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-7bd5d56d6-dbqgx" event={"ID":"8e0b8c6d-ac6c-4621-bc2b-3cc23dbee621","Type":"ContainerStarted","Data":"1144a5cb89705b51860ddcdd59e2e420bfda32335eccd50d5f11328bd9e9384a"}
Sep 18 13:59:50.175348 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:59:50.175308    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-msrpt" podStartSLOduration=1.61667553 podStartE2EDuration="6.175293964s" totalImagesPullingTime="4.558618434s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-18 13:59:44 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 13:59:50.174799308 +0000 UTC m=+58.348708986" watchObservedRunningTime="2026-09-18 13:59:50.175293964 +0000 UTC m=+58.349203663"
Sep 18 13:59:50.192026 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:59:50.191979    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-operator-7bd5d56d6-dbqgx" podStartSLOduration=2.5991300710000003 podStartE2EDuration="4.191967394s" totalImagesPullingTime="1.592837323s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 13:59:46 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 13:59:50.190409998 +0000 UTC m=+58.364319657" watchObservedRunningTime="2026-09-18 13:59:50.191967394 +0000 UTC m=+58.365877064"
Sep 18 13:59:52.304219 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:59:52.304186    2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/openshift-state-metrics-86c5bc79d6-tgk6f"]
Sep 18 13:59:52.306143 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:59:52.306127    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/openshift-state-metrics-86c5bc79d6-tgk6f"
Sep 18 13:59:52.309231 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:59:52.309209    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-dockercfg-4rt9l\""
Sep 18 13:59:52.309231 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:59:52.309223    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-tls\""
Sep 18 13:59:52.309403 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:59:52.309301    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-kube-rbac-proxy-config\""
Sep 18 13:59:52.319139 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:59:52.319114    2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/kube-state-metrics-84c885658c-hwgkx"]
Sep 18 13:59:52.321104 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:59:52.321084    2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-psstj"]
Sep 18 13:59:52.321218 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:59:52.321201    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/kube-state-metrics-84c885658c-hwgkx"
Sep 18 13:59:52.322884 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:59:52.322867    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-psstj"
Sep 18 13:59:52.324460 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:59:52.324438    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-kube-rbac-proxy-config\""
Sep 18 13:59:52.324752 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:59:52.324709    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/5ac47b4f-a039-4016-a1df-9195a2124c08-node-exporter-wtmp\") pod \"node-exporter-psstj\" (UID: \"5ac47b4f-a039-4016-a1df-9195a2124c08\") " pod="openshift-monitoring/node-exporter-psstj"
Sep 18 13:59:52.324752 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:59:52.324745    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5dsqr\" (UniqueName: \"kubernetes.io/projected/98c35b3b-f121-4202-bf45-0e9c31a2d848-kube-api-access-5dsqr\") pod \"kube-state-metrics-84c885658c-hwgkx\" (UID: \"98c35b3b-f121-4202-bf45-0e9c31a2d848\") " pod="openshift-monitoring/kube-state-metrics-84c885658c-hwgkx"
Sep 18 13:59:52.324940 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:59:52.324816    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/98c35b3b-f121-4202-bf45-0e9c31a2d848-metrics-client-ca\") pod \"kube-state-metrics-84c885658c-hwgkx\" (UID: \"98c35b3b-f121-4202-bf45-0e9c31a2d848\") " pod="openshift-monitoring/kube-state-metrics-84c885658c-hwgkx"
Sep 18 13:59:52.324940 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:59:52.324846    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/5ac47b4f-a039-4016-a1df-9195a2124c08-node-exporter-tls\") pod \"node-exporter-psstj\" (UID: \"5ac47b4f-a039-4016-a1df-9195a2124c08\") " pod="openshift-monitoring/node-exporter-psstj"
Sep 18 13:59:52.324940 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:59:52.324880    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/98c35b3b-f121-4202-bf45-0e9c31a2d848-kube-state-metrics-tls\") pod \"kube-state-metrics-84c885658c-hwgkx\" (UID: \"98c35b3b-f121-4202-bf45-0e9c31a2d848\") " pod="openshift-monitoring/kube-state-metrics-84c885658c-hwgkx"
Sep 18 13:59:52.324940 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:59:52.324913    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/5ac47b4f-a039-4016-a1df-9195a2124c08-node-exporter-accelerators-collector-config\") pod \"node-exporter-psstj\" (UID: \"5ac47b4f-a039-4016-a1df-9195a2124c08\") " pod="openshift-monitoring/node-exporter-psstj"
Sep 18 13:59:52.325269 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:59:52.324965    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/98c35b3b-f121-4202-bf45-0e9c31a2d848-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-84c885658c-hwgkx\" (UID: \"98c35b3b-f121-4202-bf45-0e9c31a2d848\") " pod="openshift-monitoring/kube-state-metrics-84c885658c-hwgkx"
Sep 18 13:59:52.325269 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:59:52.324977    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-tls\""
Sep 18 13:59:52.325269 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:59:52.324987    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/5ac47b4f-a039-4016-a1df-9195a2124c08-node-exporter-textfile\") pod \"node-exporter-psstj\" (UID: \"5ac47b4f-a039-4016-a1df-9195a2124c08\") " pod="openshift-monitoring/node-exporter-psstj"
Sep 18 13:59:52.325269 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:59:52.325018    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/5ac47b4f-a039-4016-a1df-9195a2124c08-root\") pod \"node-exporter-psstj\" (UID: \"5ac47b4f-a039-4016-a1df-9195a2124c08\") " pod="openshift-monitoring/node-exporter-psstj"
Sep 18 13:59:52.325269 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:59:52.325063    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/27863c15-1a0a-4577-a57e-39db30081e27-openshift-state-metrics-tls\") pod \"openshift-state-metrics-86c5bc79d6-tgk6f\" (UID: \"27863c15-1a0a-4577-a57e-39db30081e27\") " pod="openshift-monitoring/openshift-state-metrics-86c5bc79d6-tgk6f"
Sep 18 13:59:52.325269 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:59:52.325105    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/27863c15-1a0a-4577-a57e-39db30081e27-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-86c5bc79d6-tgk6f\" (UID: \"27863c15-1a0a-4577-a57e-39db30081e27\") " pod="openshift-monitoring/openshift-state-metrics-86c5bc79d6-tgk6f"
Sep 18 13:59:52.325269 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:59:52.325135    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-s7lh2\" (UniqueName: \"kubernetes.io/projected/5ac47b4f-a039-4016-a1df-9195a2124c08-kube-api-access-s7lh2\") pod \"node-exporter-psstj\" (UID: \"5ac47b4f-a039-4016-a1df-9195a2124c08\") " pod="openshift-monitoring/node-exporter-psstj"
Sep 18 13:59:52.325269 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:59:52.325164    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-88n8l\" (UniqueName: \"kubernetes.io/projected/27863c15-1a0a-4577-a57e-39db30081e27-kube-api-access-88n8l\") pod \"openshift-state-metrics-86c5bc79d6-tgk6f\" (UID: \"27863c15-1a0a-4577-a57e-39db30081e27\") " pod="openshift-monitoring/openshift-state-metrics-86c5bc79d6-tgk6f"
Sep 18 13:59:52.325269 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:59:52.325188    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/5ac47b4f-a039-4016-a1df-9195a2124c08-metrics-client-ca\") pod \"node-exporter-psstj\" (UID: \"5ac47b4f-a039-4016-a1df-9195a2124c08\") " pod="openshift-monitoring/node-exporter-psstj"
Sep 18 13:59:52.325269 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:59:52.325219    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-dockercfg-92gbb\""
Sep 18 13:59:52.325269 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:59:52.325240    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\""
Sep 18 13:59:52.325269 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:59:52.325261    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/27863c15-1a0a-4577-a57e-39db30081e27-metrics-client-ca\") pod \"openshift-state-metrics-86c5bc79d6-tgk6f\" (UID: \"27863c15-1a0a-4577-a57e-39db30081e27\") " pod="openshift-monitoring/openshift-state-metrics-86c5bc79d6-tgk6f"
Sep 18 13:59:52.325765 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:59:52.325288    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/5ac47b4f-a039-4016-a1df-9195a2124c08-sys\") pod \"node-exporter-psstj\" (UID: \"5ac47b4f-a039-4016-a1df-9195a2124c08\") " pod="openshift-monitoring/node-exporter-psstj"
Sep 18 13:59:52.325765 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:59:52.325337    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-fvvcw\""
Sep 18 13:59:52.325765 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:59:52.325337    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/5ac47b4f-a039-4016-a1df-9195a2124c08-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-psstj\" (UID: \"5ac47b4f-a039-4016-a1df-9195a2124c08\") " pod="openshift-monitoring/node-exporter-psstj"
Sep 18 13:59:52.325765 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:59:52.325357    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\""
Sep 18 13:59:52.325765 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:59:52.325404    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/98c35b3b-f121-4202-bf45-0e9c31a2d848-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-84c885658c-hwgkx\" (UID: \"98c35b3b-f121-4202-bf45-0e9c31a2d848\") " pod="openshift-monitoring/kube-state-metrics-84c885658c-hwgkx"
Sep 18 13:59:52.325765 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:59:52.325433    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/98c35b3b-f121-4202-bf45-0e9c31a2d848-volume-directive-shadow\") pod \"kube-state-metrics-84c885658c-hwgkx\" (UID: \"98c35b3b-f121-4202-bf45-0e9c31a2d848\") " pod="openshift-monitoring/kube-state-metrics-84c885658c-hwgkx"
Sep 18 13:59:52.325765 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:59:52.325457    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\""
Sep 18 13:59:52.325765 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:59:52.325489    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-custom-resource-state-configmap\""
Sep 18 13:59:52.332313 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:59:52.332288    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/openshift-state-metrics-86c5bc79d6-tgk6f"]
Sep 18 13:59:52.348687 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:59:52.348654    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/kube-state-metrics-84c885658c-hwgkx"]
Sep 18 13:59:52.425916 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:59:52.425885    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/5ac47b4f-a039-4016-a1df-9195a2124c08-root\") pod \"node-exporter-psstj\" (UID: \"5ac47b4f-a039-4016-a1df-9195a2124c08\") " pod="openshift-monitoring/node-exporter-psstj"
Sep 18 13:59:52.425916 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:59:52.425920    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/27863c15-1a0a-4577-a57e-39db30081e27-openshift-state-metrics-tls\") pod \"openshift-state-metrics-86c5bc79d6-tgk6f\" (UID: \"27863c15-1a0a-4577-a57e-39db30081e27\") " pod="openshift-monitoring/openshift-state-metrics-86c5bc79d6-tgk6f"
Sep 18 13:59:52.426187 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:59:52.425991    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/5ac47b4f-a039-4016-a1df-9195a2124c08-root\") pod \"node-exporter-psstj\" (UID: \"5ac47b4f-a039-4016-a1df-9195a2124c08\") " pod="openshift-monitoring/node-exporter-psstj"
Sep 18 13:59:52.426187 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:59:52.426049    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/27863c15-1a0a-4577-a57e-39db30081e27-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-86c5bc79d6-tgk6f\" (UID: \"27863c15-1a0a-4577-a57e-39db30081e27\") " pod="openshift-monitoring/openshift-state-metrics-86c5bc79d6-tgk6f"
Sep 18 13:59:52.426187 ip-10-0-129-16 kubenswrapper[2565]: E0918 13:59:52.426064    2565 secret.go:189] Couldn't get secret openshift-monitoring/openshift-state-metrics-tls: secret "openshift-state-metrics-tls" not found
Sep 18 13:59:52.426187 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:59:52.426086    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-s7lh2\" (UniqueName: \"kubernetes.io/projected/5ac47b4f-a039-4016-a1df-9195a2124c08-kube-api-access-s7lh2\") pod \"node-exporter-psstj\" (UID: \"5ac47b4f-a039-4016-a1df-9195a2124c08\") " pod="openshift-monitoring/node-exporter-psstj"
Sep 18 13:59:52.426187 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:59:52.426116    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-88n8l\" (UniqueName: \"kubernetes.io/projected/27863c15-1a0a-4577-a57e-39db30081e27-kube-api-access-88n8l\") pod \"openshift-state-metrics-86c5bc79d6-tgk6f\" (UID: \"27863c15-1a0a-4577-a57e-39db30081e27\") " pod="openshift-monitoring/openshift-state-metrics-86c5bc79d6-tgk6f"
Sep 18 13:59:52.426187 ip-10-0-129-16 kubenswrapper[2565]: E0918 13:59:52.426129    2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/27863c15-1a0a-4577-a57e-39db30081e27-openshift-state-metrics-tls podName:27863c15-1a0a-4577-a57e-39db30081e27 nodeName:}" failed. No retries permitted until 2026-09-18 13:59:52.926110701 +0000 UTC m=+61.100020347 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "openshift-state-metrics-tls" (UniqueName: "kubernetes.io/secret/27863c15-1a0a-4577-a57e-39db30081e27-openshift-state-metrics-tls") pod "openshift-state-metrics-86c5bc79d6-tgk6f" (UID: "27863c15-1a0a-4577-a57e-39db30081e27") : secret "openshift-state-metrics-tls" not found
Sep 18 13:59:52.426187 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:59:52.426154    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/5ac47b4f-a039-4016-a1df-9195a2124c08-metrics-client-ca\") pod \"node-exporter-psstj\" (UID: \"5ac47b4f-a039-4016-a1df-9195a2124c08\") " pod="openshift-monitoring/node-exporter-psstj"
Sep 18 13:59:52.426540 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:59:52.426193    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/27863c15-1a0a-4577-a57e-39db30081e27-metrics-client-ca\") pod \"openshift-state-metrics-86c5bc79d6-tgk6f\" (UID: \"27863c15-1a0a-4577-a57e-39db30081e27\") " pod="openshift-monitoring/openshift-state-metrics-86c5bc79d6-tgk6f"
Sep 18 13:59:52.426540 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:59:52.426237    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/5ac47b4f-a039-4016-a1df-9195a2124c08-sys\") pod \"node-exporter-psstj\" (UID: \"5ac47b4f-a039-4016-a1df-9195a2124c08\") " pod="openshift-monitoring/node-exporter-psstj"
Sep 18 13:59:52.426540 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:59:52.426267    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/5ac47b4f-a039-4016-a1df-9195a2124c08-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-psstj\" (UID: \"5ac47b4f-a039-4016-a1df-9195a2124c08\") " pod="openshift-monitoring/node-exporter-psstj"
Sep 18 13:59:52.426540 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:59:52.426319    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/98c35b3b-f121-4202-bf45-0e9c31a2d848-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-84c885658c-hwgkx\" (UID: \"98c35b3b-f121-4202-bf45-0e9c31a2d848\") " pod="openshift-monitoring/kube-state-metrics-84c885658c-hwgkx"
Sep 18 13:59:52.426540 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:59:52.426348    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/98c35b3b-f121-4202-bf45-0e9c31a2d848-volume-directive-shadow\") pod \"kube-state-metrics-84c885658c-hwgkx\" (UID: \"98c35b3b-f121-4202-bf45-0e9c31a2d848\") " pod="openshift-monitoring/kube-state-metrics-84c885658c-hwgkx"
Sep 18 13:59:52.426540 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:59:52.426321    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/5ac47b4f-a039-4016-a1df-9195a2124c08-sys\") pod \"node-exporter-psstj\" (UID: \"5ac47b4f-a039-4016-a1df-9195a2124c08\") " pod="openshift-monitoring/node-exporter-psstj"
Sep 18 13:59:52.426540 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:59:52.426374    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/5ac47b4f-a039-4016-a1df-9195a2124c08-node-exporter-wtmp\") pod \"node-exporter-psstj\" (UID: \"5ac47b4f-a039-4016-a1df-9195a2124c08\") " pod="openshift-monitoring/node-exporter-psstj"
Sep 18 13:59:52.426540 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:59:52.426437    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5dsqr\" (UniqueName: \"kubernetes.io/projected/98c35b3b-f121-4202-bf45-0e9c31a2d848-kube-api-access-5dsqr\") pod \"kube-state-metrics-84c885658c-hwgkx\" (UID: \"98c35b3b-f121-4202-bf45-0e9c31a2d848\") " pod="openshift-monitoring/kube-state-metrics-84c885658c-hwgkx"
Sep 18 13:59:52.426540 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:59:52.426471    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/98c35b3b-f121-4202-bf45-0e9c31a2d848-metrics-client-ca\") pod \"kube-state-metrics-84c885658c-hwgkx\" (UID: \"98c35b3b-f121-4202-bf45-0e9c31a2d848\") " pod="openshift-monitoring/kube-state-metrics-84c885658c-hwgkx"
Sep 18 13:59:52.426540 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:59:52.426498    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/5ac47b4f-a039-4016-a1df-9195a2124c08-node-exporter-tls\") pod \"node-exporter-psstj\" (UID: \"5ac47b4f-a039-4016-a1df-9195a2124c08\") " pod="openshift-monitoring/node-exporter-psstj"
Sep 18 13:59:52.426540 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:59:52.426511    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/5ac47b4f-a039-4016-a1df-9195a2124c08-node-exporter-wtmp\") pod \"node-exporter-psstj\" (UID: \"5ac47b4f-a039-4016-a1df-9195a2124c08\") " pod="openshift-monitoring/node-exporter-psstj"
Sep 18 13:59:52.426540 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:59:52.426525    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/98c35b3b-f121-4202-bf45-0e9c31a2d848-kube-state-metrics-tls\") pod \"kube-state-metrics-84c885658c-hwgkx\" (UID: \"98c35b3b-f121-4202-bf45-0e9c31a2d848\") " pod="openshift-monitoring/kube-state-metrics-84c885658c-hwgkx"
Sep 18 13:59:52.427128 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:59:52.426558    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/5ac47b4f-a039-4016-a1df-9195a2124c08-node-exporter-accelerators-collector-config\") pod \"node-exporter-psstj\" (UID: \"5ac47b4f-a039-4016-a1df-9195a2124c08\") " pod="openshift-monitoring/node-exporter-psstj"
Sep 18 13:59:52.427128 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:59:52.426587    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/98c35b3b-f121-4202-bf45-0e9c31a2d848-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-84c885658c-hwgkx\" (UID: \"98c35b3b-f121-4202-bf45-0e9c31a2d848\") " pod="openshift-monitoring/kube-state-metrics-84c885658c-hwgkx"
Sep 18 13:59:52.427128 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:59:52.426614    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/5ac47b4f-a039-4016-a1df-9195a2124c08-node-exporter-textfile\") pod \"node-exporter-psstj\" (UID: \"5ac47b4f-a039-4016-a1df-9195a2124c08\") " pod="openshift-monitoring/node-exporter-psstj"
Sep 18 13:59:52.427128 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:59:52.426866    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/5ac47b4f-a039-4016-a1df-9195a2124c08-metrics-client-ca\") pod \"node-exporter-psstj\" (UID: \"5ac47b4f-a039-4016-a1df-9195a2124c08\") " pod="openshift-monitoring/node-exporter-psstj"
Sep 18 13:59:52.427128 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:59:52.426888    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/5ac47b4f-a039-4016-a1df-9195a2124c08-node-exporter-textfile\") pod \"node-exporter-psstj\" (UID: \"5ac47b4f-a039-4016-a1df-9195a2124c08\") " pod="openshift-monitoring/node-exporter-psstj"
Sep 18 13:59:52.427128 ip-10-0-129-16 kubenswrapper[2565]: E0918 13:59:52.426911    2565 secret.go:189] Couldn't get secret openshift-monitoring/kube-state-metrics-tls: secret "kube-state-metrics-tls" not found
Sep 18 13:59:52.427128 ip-10-0-129-16 kubenswrapper[2565]: E0918 13:59:52.426971    2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/98c35b3b-f121-4202-bf45-0e9c31a2d848-kube-state-metrics-tls podName:98c35b3b-f121-4202-bf45-0e9c31a2d848 nodeName:}" failed. No retries permitted until 2026-09-18 13:59:52.92695361 +0000 UTC m=+61.100863260 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-state-metrics-tls" (UniqueName: "kubernetes.io/secret/98c35b3b-f121-4202-bf45-0e9c31a2d848-kube-state-metrics-tls") pod "kube-state-metrics-84c885658c-hwgkx" (UID: "98c35b3b-f121-4202-bf45-0e9c31a2d848") : secret "kube-state-metrics-tls" not found
Sep 18 13:59:52.427128 ip-10-0-129-16 kubenswrapper[2565]: E0918 13:59:52.427063    2565 secret.go:189] Couldn't get secret openshift-monitoring/node-exporter-tls: secret "node-exporter-tls" not found
Sep 18 13:59:52.427128 ip-10-0-129-16 kubenswrapper[2565]: E0918 13:59:52.427101    2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5ac47b4f-a039-4016-a1df-9195a2124c08-node-exporter-tls podName:5ac47b4f-a039-4016-a1df-9195a2124c08 nodeName:}" failed. No retries permitted until 2026-09-18 13:59:52.927089418 +0000 UTC m=+61.100999056 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "node-exporter-tls" (UniqueName: "kubernetes.io/secret/5ac47b4f-a039-4016-a1df-9195a2124c08-node-exporter-tls") pod "node-exporter-psstj" (UID: "5ac47b4f-a039-4016-a1df-9195a2124c08") : secret "node-exporter-tls" not found
Sep 18 13:59:52.427562 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:59:52.427351    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/27863c15-1a0a-4577-a57e-39db30081e27-metrics-client-ca\") pod \"openshift-state-metrics-86c5bc79d6-tgk6f\" (UID: \"27863c15-1a0a-4577-a57e-39db30081e27\") " pod="openshift-monitoring/openshift-state-metrics-86c5bc79d6-tgk6f"
Sep 18 13:59:52.427562 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:59:52.427412    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/5ac47b4f-a039-4016-a1df-9195a2124c08-node-exporter-accelerators-collector-config\") pod \"node-exporter-psstj\" (UID: \"5ac47b4f-a039-4016-a1df-9195a2124c08\") " pod="openshift-monitoring/node-exporter-psstj"
Sep 18 13:59:52.427562 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:59:52.427423    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/98c35b3b-f121-4202-bf45-0e9c31a2d848-volume-directive-shadow\") pod \"kube-state-metrics-84c885658c-hwgkx\" (UID: \"98c35b3b-f121-4202-bf45-0e9c31a2d848\") " pod="openshift-monitoring/kube-state-metrics-84c885658c-hwgkx"
Sep 18 13:59:52.427562 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:59:52.427440    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/98c35b3b-f121-4202-bf45-0e9c31a2d848-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-84c885658c-hwgkx\" (UID: \"98c35b3b-f121-4202-bf45-0e9c31a2d848\") " pod="openshift-monitoring/kube-state-metrics-84c885658c-hwgkx"
Sep 18 13:59:52.427757 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:59:52.427666    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/98c35b3b-f121-4202-bf45-0e9c31a2d848-metrics-client-ca\") pod \"kube-state-metrics-84c885658c-hwgkx\" (UID: \"98c35b3b-f121-4202-bf45-0e9c31a2d848\") " pod="openshift-monitoring/kube-state-metrics-84c885658c-hwgkx"
Sep 18 13:59:52.428744 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:59:52.428721    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/27863c15-1a0a-4577-a57e-39db30081e27-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-86c5bc79d6-tgk6f\" (UID: \"27863c15-1a0a-4577-a57e-39db30081e27\") " pod="openshift-monitoring/openshift-state-metrics-86c5bc79d6-tgk6f"
Sep 18 13:59:52.429248 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:59:52.429230    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/5ac47b4f-a039-4016-a1df-9195a2124c08-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-psstj\" (UID: \"5ac47b4f-a039-4016-a1df-9195a2124c08\") " pod="openshift-monitoring/node-exporter-psstj"
Sep 18 13:59:52.429436 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:59:52.429418    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/98c35b3b-f121-4202-bf45-0e9c31a2d848-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-84c885658c-hwgkx\" (UID: \"98c35b3b-f121-4202-bf45-0e9c31a2d848\") " pod="openshift-monitoring/kube-state-metrics-84c885658c-hwgkx"
Sep 18 13:59:52.439911 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:59:52.439884    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-s7lh2\" (UniqueName: \"kubernetes.io/projected/5ac47b4f-a039-4016-a1df-9195a2124c08-kube-api-access-s7lh2\") pod \"node-exporter-psstj\" (UID: \"5ac47b4f-a039-4016-a1df-9195a2124c08\") " pod="openshift-monitoring/node-exporter-psstj"
Sep 18 13:59:52.440087 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:59:52.440070    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-88n8l\" (UniqueName: \"kubernetes.io/projected/27863c15-1a0a-4577-a57e-39db30081e27-kube-api-access-88n8l\") pod \"openshift-state-metrics-86c5bc79d6-tgk6f\" (UID: \"27863c15-1a0a-4577-a57e-39db30081e27\") " pod="openshift-monitoring/openshift-state-metrics-86c5bc79d6-tgk6f"
Sep 18 13:59:52.441220 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:59:52.441203    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5dsqr\" (UniqueName: \"kubernetes.io/projected/98c35b3b-f121-4202-bf45-0e9c31a2d848-kube-api-access-5dsqr\") pod \"kube-state-metrics-84c885658c-hwgkx\" (UID: \"98c35b3b-f121-4202-bf45-0e9c31a2d848\") " pod="openshift-monitoring/kube-state-metrics-84c885658c-hwgkx"
Sep 18 13:59:52.930973 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:59:52.930939    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/5ac47b4f-a039-4016-a1df-9195a2124c08-node-exporter-tls\") pod \"node-exporter-psstj\" (UID: \"5ac47b4f-a039-4016-a1df-9195a2124c08\") " pod="openshift-monitoring/node-exporter-psstj"
Sep 18 13:59:52.930973 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:59:52.930976    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/98c35b3b-f121-4202-bf45-0e9c31a2d848-kube-state-metrics-tls\") pod \"kube-state-metrics-84c885658c-hwgkx\" (UID: \"98c35b3b-f121-4202-bf45-0e9c31a2d848\") " pod="openshift-monitoring/kube-state-metrics-84c885658c-hwgkx"
Sep 18 13:59:52.931205 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:59:52.931003    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/27863c15-1a0a-4577-a57e-39db30081e27-openshift-state-metrics-tls\") pod \"openshift-state-metrics-86c5bc79d6-tgk6f\" (UID: \"27863c15-1a0a-4577-a57e-39db30081e27\") " pod="openshift-monitoring/openshift-state-metrics-86c5bc79d6-tgk6f"
Sep 18 13:59:52.931205 ip-10-0-129-16 kubenswrapper[2565]: E0918 13:59:52.931160    2565 secret.go:189] Couldn't get secret openshift-monitoring/openshift-state-metrics-tls: secret "openshift-state-metrics-tls" not found
Sep 18 13:59:52.931286 ip-10-0-129-16 kubenswrapper[2565]: E0918 13:59:52.931237    2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/27863c15-1a0a-4577-a57e-39db30081e27-openshift-state-metrics-tls podName:27863c15-1a0a-4577-a57e-39db30081e27 nodeName:}" failed. No retries permitted until 2026-09-18 13:59:53.931221741 +0000 UTC m=+62.105131382 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "openshift-state-metrics-tls" (UniqueName: "kubernetes.io/secret/27863c15-1a0a-4577-a57e-39db30081e27-openshift-state-metrics-tls") pod "openshift-state-metrics-86c5bc79d6-tgk6f" (UID: "27863c15-1a0a-4577-a57e-39db30081e27") : secret "openshift-state-metrics-tls" not found
Sep 18 13:59:52.933324 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:59:52.933297    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/5ac47b4f-a039-4016-a1df-9195a2124c08-node-exporter-tls\") pod \"node-exporter-psstj\" (UID: \"5ac47b4f-a039-4016-a1df-9195a2124c08\") " pod="openshift-monitoring/node-exporter-psstj"
Sep 18 13:59:52.933429 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:59:52.933330    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/98c35b3b-f121-4202-bf45-0e9c31a2d848-kube-state-metrics-tls\") pod \"kube-state-metrics-84c885658c-hwgkx\" (UID: \"98c35b3b-f121-4202-bf45-0e9c31a2d848\") " pod="openshift-monitoring/kube-state-metrics-84c885658c-hwgkx"
Sep 18 13:59:52.937256 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:59:52.937224    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-psstj"
Sep 18 13:59:52.946223 ip-10-0-129-16 kubenswrapper[2565]: W0918 13:59:52.946200    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5ac47b4f_a039_4016_a1df_9195a2124c08.slice/crio-ebb97e682622c87fccf73843621a10ef4d662ced8ed662ce3f85d7d5283be1f0 WatchSource:0}: Error finding container ebb97e682622c87fccf73843621a10ef4d662ced8ed662ce3f85d7d5283be1f0: Status 404 returned error can't find the container with id ebb97e682622c87fccf73843621a10ef4d662ced8ed662ce3f85d7d5283be1f0
Sep 18 13:59:53.067846 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:59:53.067821    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-bvqls"
Sep 18 13:59:53.172677 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:59:53.172639    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-psstj" event={"ID":"5ac47b4f-a039-4016-a1df-9195a2124c08","Type":"ContainerStarted","Data":"ebb97e682622c87fccf73843621a10ef4d662ced8ed662ce3f85d7d5283be1f0"}
Sep 18 13:59:53.231305 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:59:53.231226    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/kube-state-metrics-84c885658c-hwgkx"
Sep 18 13:59:53.455203 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:59:53.455169    2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Sep 18 13:59:53.458237 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:59:53.458212    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Sep 18 13:59:53.464600 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:59:53.464413    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls-assets-0\""
Sep 18 13:59:53.464600 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:59:53.464416    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-generated\""
Sep 18 13:59:53.464600 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:59:53.464475    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-cluster-tls-config\""
Sep 18 13:59:53.464600 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:59:53.464565    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-web-config\""
Sep 18 13:59:53.464970 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:59:53.464945    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-web\""
Sep 18 13:59:53.466640 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:59:53.466559    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy\""
Sep 18 13:59:53.466747 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:59:53.466731    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-metric\""
Sep 18 13:59:53.466817 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:59:53.466759    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"alertmanager-trusted-ca-bundle\""
Sep 18 13:59:53.466988 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:59:53.466959    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-dockercfg-2z746\""
Sep 18 13:59:53.467302 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:59:53.467281    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls\""
Sep 18 13:59:53.483696 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:59:53.483622    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Sep 18 13:59:53.520003 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:59:53.519975    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/kube-state-metrics-84c885658c-hwgkx"]
Sep 18 13:59:53.521921 ip-10-0-129-16 kubenswrapper[2565]: W0918 13:59:53.521892    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod98c35b3b_f121_4202_bf45_0e9c31a2d848.slice/crio-7d752662abecaae4eed8bc9b3f21c5e0b1ee247d9917750a1eaffab9da0f9701 WatchSource:0}: Error finding container 7d752662abecaae4eed8bc9b3f21c5e0b1ee247d9917750a1eaffab9da0f9701: Status 404 returned error can't find the container with id 7d752662abecaae4eed8bc9b3f21c5e0b1ee247d9917750a1eaffab9da0f9701
Sep 18 13:59:53.537996 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:59:53.537966    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/ee93bad9-7ad1-4dee-97fb-b16f6e1e1459-config-out\") pod \"alertmanager-main-0\" (UID: \"ee93bad9-7ad1-4dee-97fb-b16f6e1e1459\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 18 13:59:53.538119 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:59:53.538021    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/ee93bad9-7ad1-4dee-97fb-b16f6e1e1459-tls-assets\") pod \"alertmanager-main-0\" (UID: \"ee93bad9-7ad1-4dee-97fb-b16f6e1e1459\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 18 13:59:53.538119 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:59:53.538096    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/ee93bad9-7ad1-4dee-97fb-b16f6e1e1459-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"ee93bad9-7ad1-4dee-97fb-b16f6e1e1459\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 18 13:59:53.538226 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:59:53.538123    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gckwq\" (UniqueName: \"kubernetes.io/projected/ee93bad9-7ad1-4dee-97fb-b16f6e1e1459-kube-api-access-gckwq\") pod \"alertmanager-main-0\" (UID: \"ee93bad9-7ad1-4dee-97fb-b16f6e1e1459\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 18 13:59:53.538226 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:59:53.538183    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/ee93bad9-7ad1-4dee-97fb-b16f6e1e1459-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"ee93bad9-7ad1-4dee-97fb-b16f6e1e1459\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 18 13:59:53.538226 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:59:53.538216    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/ee93bad9-7ad1-4dee-97fb-b16f6e1e1459-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"ee93bad9-7ad1-4dee-97fb-b16f6e1e1459\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 18 13:59:53.538376 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:59:53.538262    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/ee93bad9-7ad1-4dee-97fb-b16f6e1e1459-config-volume\") pod \"alertmanager-main-0\" (UID: \"ee93bad9-7ad1-4dee-97fb-b16f6e1e1459\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 18 13:59:53.538376 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:59:53.538339    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/ee93bad9-7ad1-4dee-97fb-b16f6e1e1459-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"ee93bad9-7ad1-4dee-97fb-b16f6e1e1459\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 18 13:59:53.538456 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:59:53.538375    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/ee93bad9-7ad1-4dee-97fb-b16f6e1e1459-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"ee93bad9-7ad1-4dee-97fb-b16f6e1e1459\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 18 13:59:53.538505 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:59:53.538468    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/ee93bad9-7ad1-4dee-97fb-b16f6e1e1459-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"ee93bad9-7ad1-4dee-97fb-b16f6e1e1459\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 18 13:59:53.538554 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:59:53.538501    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/ee93bad9-7ad1-4dee-97fb-b16f6e1e1459-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"ee93bad9-7ad1-4dee-97fb-b16f6e1e1459\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 18 13:59:53.538554 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:59:53.538526    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/ee93bad9-7ad1-4dee-97fb-b16f6e1e1459-web-config\") pod \"alertmanager-main-0\" (UID: \"ee93bad9-7ad1-4dee-97fb-b16f6e1e1459\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 18 13:59:53.538648 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:59:53.538603    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ee93bad9-7ad1-4dee-97fb-b16f6e1e1459-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"ee93bad9-7ad1-4dee-97fb-b16f6e1e1459\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 18 13:59:53.639567 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:59:53.639526    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/ee93bad9-7ad1-4dee-97fb-b16f6e1e1459-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"ee93bad9-7ad1-4dee-97fb-b16f6e1e1459\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 18 13:59:53.639761 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:59:53.639580    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gckwq\" (UniqueName: \"kubernetes.io/projected/ee93bad9-7ad1-4dee-97fb-b16f6e1e1459-kube-api-access-gckwq\") pod \"alertmanager-main-0\" (UID: \"ee93bad9-7ad1-4dee-97fb-b16f6e1e1459\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 18 13:59:53.639761 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:59:53.639611    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/ee93bad9-7ad1-4dee-97fb-b16f6e1e1459-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"ee93bad9-7ad1-4dee-97fb-b16f6e1e1459\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 18 13:59:53.639761 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:59:53.639635    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/ee93bad9-7ad1-4dee-97fb-b16f6e1e1459-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"ee93bad9-7ad1-4dee-97fb-b16f6e1e1459\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 18 13:59:53.639761 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:59:53.639677    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/ee93bad9-7ad1-4dee-97fb-b16f6e1e1459-config-volume\") pod \"alertmanager-main-0\" (UID: \"ee93bad9-7ad1-4dee-97fb-b16f6e1e1459\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 18 13:59:53.639971 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:59:53.639877    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/ee93bad9-7ad1-4dee-97fb-b16f6e1e1459-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"ee93bad9-7ad1-4dee-97fb-b16f6e1e1459\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 18 13:59:53.639971 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:59:53.639941    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/ee93bad9-7ad1-4dee-97fb-b16f6e1e1459-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"ee93bad9-7ad1-4dee-97fb-b16f6e1e1459\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 18 13:59:53.640104 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:59:53.639997    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/ee93bad9-7ad1-4dee-97fb-b16f6e1e1459-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"ee93bad9-7ad1-4dee-97fb-b16f6e1e1459\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 18 13:59:53.640104 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:59:53.640027    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/ee93bad9-7ad1-4dee-97fb-b16f6e1e1459-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"ee93bad9-7ad1-4dee-97fb-b16f6e1e1459\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 18 13:59:53.640104 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:59:53.640064    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/ee93bad9-7ad1-4dee-97fb-b16f6e1e1459-web-config\") pod \"alertmanager-main-0\" (UID: \"ee93bad9-7ad1-4dee-97fb-b16f6e1e1459\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 18 13:59:53.640259 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:59:53.640135    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/ee93bad9-7ad1-4dee-97fb-b16f6e1e1459-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"ee93bad9-7ad1-4dee-97fb-b16f6e1e1459\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 18 13:59:53.640259 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:59:53.640228    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ee93bad9-7ad1-4dee-97fb-b16f6e1e1459-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"ee93bad9-7ad1-4dee-97fb-b16f6e1e1459\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 18 13:59:53.640354 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:59:53.640264    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/ee93bad9-7ad1-4dee-97fb-b16f6e1e1459-config-out\") pod \"alertmanager-main-0\" (UID: \"ee93bad9-7ad1-4dee-97fb-b16f6e1e1459\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 18 13:59:53.640354 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:59:53.640307    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/ee93bad9-7ad1-4dee-97fb-b16f6e1e1459-tls-assets\") pod \"alertmanager-main-0\" (UID: \"ee93bad9-7ad1-4dee-97fb-b16f6e1e1459\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 18 13:59:53.641999 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:59:53.641673    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/ee93bad9-7ad1-4dee-97fb-b16f6e1e1459-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"ee93bad9-7ad1-4dee-97fb-b16f6e1e1459\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 18 13:59:53.642918 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:59:53.642891    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ee93bad9-7ad1-4dee-97fb-b16f6e1e1459-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"ee93bad9-7ad1-4dee-97fb-b16f6e1e1459\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 18 13:59:53.643059 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:59:53.643022    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/ee93bad9-7ad1-4dee-97fb-b16f6e1e1459-config-volume\") pod \"alertmanager-main-0\" (UID: \"ee93bad9-7ad1-4dee-97fb-b16f6e1e1459\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 18 13:59:53.643365 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:59:53.643343    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/ee93bad9-7ad1-4dee-97fb-b16f6e1e1459-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"ee93bad9-7ad1-4dee-97fb-b16f6e1e1459\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 18 13:59:53.643707 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:59:53.643677    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/ee93bad9-7ad1-4dee-97fb-b16f6e1e1459-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"ee93bad9-7ad1-4dee-97fb-b16f6e1e1459\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 18 13:59:53.644443 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:59:53.644378    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/ee93bad9-7ad1-4dee-97fb-b16f6e1e1459-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"ee93bad9-7ad1-4dee-97fb-b16f6e1e1459\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 18 13:59:53.644601 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:59:53.644579    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/ee93bad9-7ad1-4dee-97fb-b16f6e1e1459-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"ee93bad9-7ad1-4dee-97fb-b16f6e1e1459\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 18 13:59:53.644682 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:59:53.644625    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/ee93bad9-7ad1-4dee-97fb-b16f6e1e1459-web-config\") pod \"alertmanager-main-0\" (UID: \"ee93bad9-7ad1-4dee-97fb-b16f6e1e1459\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 18 13:59:53.644805 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:59:53.644780    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/ee93bad9-7ad1-4dee-97fb-b16f6e1e1459-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"ee93bad9-7ad1-4dee-97fb-b16f6e1e1459\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 18 13:59:53.645104 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:59:53.645086    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/ee93bad9-7ad1-4dee-97fb-b16f6e1e1459-tls-assets\") pod \"alertmanager-main-0\" (UID: \"ee93bad9-7ad1-4dee-97fb-b16f6e1e1459\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 18 13:59:53.645755 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:59:53.645737    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/ee93bad9-7ad1-4dee-97fb-b16f6e1e1459-config-out\") pod \"alertmanager-main-0\" (UID: \"ee93bad9-7ad1-4dee-97fb-b16f6e1e1459\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 18 13:59:53.653344 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:59:53.653322    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gckwq\" (UniqueName: \"kubernetes.io/projected/ee93bad9-7ad1-4dee-97fb-b16f6e1e1459-kube-api-access-gckwq\") pod \"alertmanager-main-0\" (UID: \"ee93bad9-7ad1-4dee-97fb-b16f6e1e1459\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 18 13:59:53.772788 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:59:53.772710    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-dockercfg-2z746\""
Sep 18 13:59:53.780056 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:59:53.780014    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Sep 18 13:59:53.943790 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:59:53.943757    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/27863c15-1a0a-4577-a57e-39db30081e27-openshift-state-metrics-tls\") pod \"openshift-state-metrics-86c5bc79d6-tgk6f\" (UID: \"27863c15-1a0a-4577-a57e-39db30081e27\") " pod="openshift-monitoring/openshift-state-metrics-86c5bc79d6-tgk6f"
Sep 18 13:59:53.945989 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:59:53.945963    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/27863c15-1a0a-4577-a57e-39db30081e27-openshift-state-metrics-tls\") pod \"openshift-state-metrics-86c5bc79d6-tgk6f\" (UID: \"27863c15-1a0a-4577-a57e-39db30081e27\") " pod="openshift-monitoring/openshift-state-metrics-86c5bc79d6-tgk6f"
Sep 18 13:59:54.091339 ip-10-0-129-16 kubenswrapper[2565]: W0918 13:59:54.091310    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podee93bad9_7ad1_4dee_97fb_b16f6e1e1459.slice/crio-6be049ee68c7f5308f03194381a257f271cba6a654062f52b3e392ccb57f0796 WatchSource:0}: Error finding container 6be049ee68c7f5308f03194381a257f271cba6a654062f52b3e392ccb57f0796: Status 404 returned error can't find the container with id 6be049ee68c7f5308f03194381a257f271cba6a654062f52b3e392ccb57f0796
Sep 18 13:59:54.097435 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:59:54.097404    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Sep 18 13:59:54.118008 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:59:54.117972    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-dockercfg-4rt9l\""
Sep 18 13:59:54.125320 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:59:54.125290    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/openshift-state-metrics-86c5bc79d6-tgk6f"
Sep 18 13:59:54.183244 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:59:54.183165    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"ee93bad9-7ad1-4dee-97fb-b16f6e1e1459","Type":"ContainerStarted","Data":"6be049ee68c7f5308f03194381a257f271cba6a654062f52b3e392ccb57f0796"}
Sep 18 13:59:54.186499 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:59:54.186465    2565 generic.go:356] "Generic (PLEG): container finished" podID="5ac47b4f-a039-4016-a1df-9195a2124c08" containerID="76f4432069d7c81c3c56bbc68cbebfdc8cb58459e440d45c5edceeeb78795069" exitCode=0
Sep 18 13:59:54.186620 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:59:54.186554    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-psstj" event={"ID":"5ac47b4f-a039-4016-a1df-9195a2124c08","Type":"ContainerDied","Data":"76f4432069d7c81c3c56bbc68cbebfdc8cb58459e440d45c5edceeeb78795069"}
Sep 18 13:59:54.190604 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:59:54.190573    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-84c885658c-hwgkx" event={"ID":"98c35b3b-f121-4202-bf45-0e9c31a2d848","Type":"ContainerStarted","Data":"7d752662abecaae4eed8bc9b3f21c5e0b1ee247d9917750a1eaffab9da0f9701"}
Sep 18 13:59:54.390965 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:59:54.390877    2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/thanos-querier-5f599748b7-k2vqh"]
Sep 18 13:59:54.393953 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:59:54.393924    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/thanos-querier-5f599748b7-k2vqh"
Sep 18 13:59:54.396782 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:59:54.396759    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-rules\""
Sep 18 13:59:54.397091 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:59:54.397067    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-grpc-tls-8mkl56d4dlplg\""
Sep 18 13:59:54.397197 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:59:54.397125    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-tls\""
Sep 18 13:59:54.397197 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:59:54.397159    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-web\""
Sep 18 13:59:54.397323 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:59:54.397234    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-metrics\""
Sep 18 13:59:54.397374 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:59:54.397330    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy\""
Sep 18 13:59:54.397523 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:59:54.397502    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-dockercfg-pjzqb\""
Sep 18 13:59:54.405795 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:59:54.405771    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/openshift-state-metrics-86c5bc79d6-tgk6f"]
Sep 18 13:59:54.409477 ip-10-0-129-16 kubenswrapper[2565]: W0918 13:59:54.409457    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod27863c15_1a0a_4577_a57e_39db30081e27.slice/crio-c272ee09d7aa9270a6765db0c3d3e44867cf489d1ec106997cb2076cbf4c1c2e WatchSource:0}: Error finding container c272ee09d7aa9270a6765db0c3d3e44867cf489d1ec106997cb2076cbf4c1c2e: Status 404 returned error can't find the container with id c272ee09d7aa9270a6765db0c3d3e44867cf489d1ec106997cb2076cbf4c1c2e
Sep 18 13:59:54.412789 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:59:54.412771    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/thanos-querier-5f599748b7-k2vqh"]
Sep 18 13:59:54.448187 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:59:54.448157    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sxknc\" (UniqueName: \"kubernetes.io/projected/476315e9-c60b-485e-aa57-9ec551db2c80-kube-api-access-sxknc\") pod \"thanos-querier-5f599748b7-k2vqh\" (UID: \"476315e9-c60b-485e-aa57-9ec551db2c80\") " pod="openshift-monitoring/thanos-querier-5f599748b7-k2vqh"
Sep 18 13:59:54.448293 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:59:54.448198    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/476315e9-c60b-485e-aa57-9ec551db2c80-secret-thanos-querier-tls\") pod \"thanos-querier-5f599748b7-k2vqh\" (UID: \"476315e9-c60b-485e-aa57-9ec551db2c80\") " pod="openshift-monitoring/thanos-querier-5f599748b7-k2vqh"
Sep 18 13:59:54.448293 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:59:54.448239    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/476315e9-c60b-485e-aa57-9ec551db2c80-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-5f599748b7-k2vqh\" (UID: \"476315e9-c60b-485e-aa57-9ec551db2c80\") " pod="openshift-monitoring/thanos-querier-5f599748b7-k2vqh"
Sep 18 13:59:54.448293 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:59:54.448273    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/476315e9-c60b-485e-aa57-9ec551db2c80-secret-grpc-tls\") pod \"thanos-querier-5f599748b7-k2vqh\" (UID: \"476315e9-c60b-485e-aa57-9ec551db2c80\") " pod="openshift-monitoring/thanos-querier-5f599748b7-k2vqh"
Sep 18 13:59:54.448443 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:59:54.448408    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/476315e9-c60b-485e-aa57-9ec551db2c80-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-5f599748b7-k2vqh\" (UID: \"476315e9-c60b-485e-aa57-9ec551db2c80\") " pod="openshift-monitoring/thanos-querier-5f599748b7-k2vqh"
Sep 18 13:59:54.448505 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:59:54.448460    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/476315e9-c60b-485e-aa57-9ec551db2c80-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-5f599748b7-k2vqh\" (UID: \"476315e9-c60b-485e-aa57-9ec551db2c80\") " pod="openshift-monitoring/thanos-querier-5f599748b7-k2vqh"
Sep 18 13:59:54.448556 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:59:54.448515    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/476315e9-c60b-485e-aa57-9ec551db2c80-metrics-client-ca\") pod \"thanos-querier-5f599748b7-k2vqh\" (UID: \"476315e9-c60b-485e-aa57-9ec551db2c80\") " pod="openshift-monitoring/thanos-querier-5f599748b7-k2vqh"
Sep 18 13:59:54.448556 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:59:54.448546    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/476315e9-c60b-485e-aa57-9ec551db2c80-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-5f599748b7-k2vqh\" (UID: \"476315e9-c60b-485e-aa57-9ec551db2c80\") " pod="openshift-monitoring/thanos-querier-5f599748b7-k2vqh"
Sep 18 13:59:54.549818 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:59:54.549775    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/476315e9-c60b-485e-aa57-9ec551db2c80-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-5f599748b7-k2vqh\" (UID: \"476315e9-c60b-485e-aa57-9ec551db2c80\") " pod="openshift-monitoring/thanos-querier-5f599748b7-k2vqh"
Sep 18 13:59:54.550239 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:59:54.549827    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/476315e9-c60b-485e-aa57-9ec551db2c80-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-5f599748b7-k2vqh\" (UID: \"476315e9-c60b-485e-aa57-9ec551db2c80\") " pod="openshift-monitoring/thanos-querier-5f599748b7-k2vqh"
Sep 18 13:59:54.550239 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:59:54.549855    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/476315e9-c60b-485e-aa57-9ec551db2c80-metrics-client-ca\") pod \"thanos-querier-5f599748b7-k2vqh\" (UID: \"476315e9-c60b-485e-aa57-9ec551db2c80\") " pod="openshift-monitoring/thanos-querier-5f599748b7-k2vqh"
Sep 18 13:59:54.550239 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:59:54.549884    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/476315e9-c60b-485e-aa57-9ec551db2c80-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-5f599748b7-k2vqh\" (UID: \"476315e9-c60b-485e-aa57-9ec551db2c80\") " pod="openshift-monitoring/thanos-querier-5f599748b7-k2vqh"
Sep 18 13:59:54.550239 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:59:54.549929    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sxknc\" (UniqueName: \"kubernetes.io/projected/476315e9-c60b-485e-aa57-9ec551db2c80-kube-api-access-sxknc\") pod \"thanos-querier-5f599748b7-k2vqh\" (UID: \"476315e9-c60b-485e-aa57-9ec551db2c80\") " pod="openshift-monitoring/thanos-querier-5f599748b7-k2vqh"
Sep 18 13:59:54.550239 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:59:54.549959    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/476315e9-c60b-485e-aa57-9ec551db2c80-secret-thanos-querier-tls\") pod \"thanos-querier-5f599748b7-k2vqh\" (UID: \"476315e9-c60b-485e-aa57-9ec551db2c80\") " pod="openshift-monitoring/thanos-querier-5f599748b7-k2vqh"
Sep 18 13:59:54.550239 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:59:54.550043    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/476315e9-c60b-485e-aa57-9ec551db2c80-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-5f599748b7-k2vqh\" (UID: \"476315e9-c60b-485e-aa57-9ec551db2c80\") " pod="openshift-monitoring/thanos-querier-5f599748b7-k2vqh"
Sep 18 13:59:54.550239 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:59:54.550080    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/476315e9-c60b-485e-aa57-9ec551db2c80-secret-grpc-tls\") pod \"thanos-querier-5f599748b7-k2vqh\" (UID: \"476315e9-c60b-485e-aa57-9ec551db2c80\") " pod="openshift-monitoring/thanos-querier-5f599748b7-k2vqh"
Sep 18 13:59:54.551432 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:59:54.550970    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/476315e9-c60b-485e-aa57-9ec551db2c80-metrics-client-ca\") pod \"thanos-querier-5f599748b7-k2vqh\" (UID: \"476315e9-c60b-485e-aa57-9ec551db2c80\") " pod="openshift-monitoring/thanos-querier-5f599748b7-k2vqh"
Sep 18 13:59:54.552900 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:59:54.552724    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/476315e9-c60b-485e-aa57-9ec551db2c80-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-5f599748b7-k2vqh\" (UID: \"476315e9-c60b-485e-aa57-9ec551db2c80\") " pod="openshift-monitoring/thanos-querier-5f599748b7-k2vqh"
Sep 18 13:59:54.552900 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:59:54.552872    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/476315e9-c60b-485e-aa57-9ec551db2c80-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-5f599748b7-k2vqh\" (UID: \"476315e9-c60b-485e-aa57-9ec551db2c80\") " pod="openshift-monitoring/thanos-querier-5f599748b7-k2vqh"
Sep 18 13:59:54.553016 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:59:54.552901    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/476315e9-c60b-485e-aa57-9ec551db2c80-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-5f599748b7-k2vqh\" (UID: \"476315e9-c60b-485e-aa57-9ec551db2c80\") " pod="openshift-monitoring/thanos-querier-5f599748b7-k2vqh"
Sep 18 13:59:54.553016 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:59:54.552989    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/476315e9-c60b-485e-aa57-9ec551db2c80-secret-grpc-tls\") pod \"thanos-querier-5f599748b7-k2vqh\" (UID: \"476315e9-c60b-485e-aa57-9ec551db2c80\") " pod="openshift-monitoring/thanos-querier-5f599748b7-k2vqh"
Sep 18 13:59:54.553672 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:59:54.553650    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/476315e9-c60b-485e-aa57-9ec551db2c80-secret-thanos-querier-tls\") pod \"thanos-querier-5f599748b7-k2vqh\" (UID: \"476315e9-c60b-485e-aa57-9ec551db2c80\") " pod="openshift-monitoring/thanos-querier-5f599748b7-k2vqh"
Sep 18 13:59:54.553930 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:59:54.553907    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/476315e9-c60b-485e-aa57-9ec551db2c80-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-5f599748b7-k2vqh\" (UID: \"476315e9-c60b-485e-aa57-9ec551db2c80\") " pod="openshift-monitoring/thanos-querier-5f599748b7-k2vqh"
Sep 18 13:59:54.559190 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:59:54.559172    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-sxknc\" (UniqueName: \"kubernetes.io/projected/476315e9-c60b-485e-aa57-9ec551db2c80-kube-api-access-sxknc\") pod \"thanos-querier-5f599748b7-k2vqh\" (UID: \"476315e9-c60b-485e-aa57-9ec551db2c80\") " pod="openshift-monitoring/thanos-querier-5f599748b7-k2vqh"
Sep 18 13:59:54.705155 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:59:54.705070    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/thanos-querier-5f599748b7-k2vqh"
Sep 18 13:59:55.105763 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:59:55.105632    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/thanos-querier-5f599748b7-k2vqh"]
Sep 18 13:59:55.114458 ip-10-0-129-16 kubenswrapper[2565]: W0918 13:59:55.114424    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod476315e9_c60b_485e_aa57_9ec551db2c80.slice/crio-fe0a68c30e8b9af8cde322f45a37f222aafe298f36f73a27e4313480d302bde2 WatchSource:0}: Error finding container fe0a68c30e8b9af8cde322f45a37f222aafe298f36f73a27e4313480d302bde2: Status 404 returned error can't find the container with id fe0a68c30e8b9af8cde322f45a37f222aafe298f36f73a27e4313480d302bde2
Sep 18 13:59:55.198447 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:59:55.198408    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-5f599748b7-k2vqh" event={"ID":"476315e9-c60b-485e-aa57-9ec551db2c80","Type":"ContainerStarted","Data":"fe0a68c30e8b9af8cde322f45a37f222aafe298f36f73a27e4313480d302bde2"}
Sep 18 13:59:55.200996 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:59:55.200967    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-86c5bc79d6-tgk6f" event={"ID":"27863c15-1a0a-4577-a57e-39db30081e27","Type":"ContainerStarted","Data":"105885af006f6d0395971fd3ad4f78da8a1a5db9b47ec2e372215143ed15ca39"}
Sep 18 13:59:55.201142 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:59:55.201018    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-86c5bc79d6-tgk6f" event={"ID":"27863c15-1a0a-4577-a57e-39db30081e27","Type":"ContainerStarted","Data":"40797aa7cbb17fd6e3a00b1b20c6fb5952e675a1abc959d8b91c5163b0cd84a4"}
Sep 18 13:59:55.201142 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:59:55.201066    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-86c5bc79d6-tgk6f" event={"ID":"27863c15-1a0a-4577-a57e-39db30081e27","Type":"ContainerStarted","Data":"c272ee09d7aa9270a6765db0c3d3e44867cf489d1ec106997cb2076cbf4c1c2e"}
Sep 18 13:59:55.203578 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:59:55.203554    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-psstj" event={"ID":"5ac47b4f-a039-4016-a1df-9195a2124c08","Type":"ContainerStarted","Data":"77336dfa2fcfaeb0f14feb714379e173d815598f3f44e8aacee8d3a4cb6ce732"}
Sep 18 13:59:55.203686 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:59:55.203604    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-psstj" event={"ID":"5ac47b4f-a039-4016-a1df-9195a2124c08","Type":"ContainerStarted","Data":"3f4acfbf4eb69e579312154afd8edf9885a0c8b02e0ca8aecfcdc6ff868b047b"}
Sep 18 13:59:55.206025 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:59:55.205999    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-84c885658c-hwgkx" event={"ID":"98c35b3b-f121-4202-bf45-0e9c31a2d848","Type":"ContainerStarted","Data":"c0e0bbc71a2e740b598670defd7f8f5b1c13097149de9ace50822895419c3728"}
Sep 18 13:59:55.206164 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:59:55.206075    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-84c885658c-hwgkx" event={"ID":"98c35b3b-f121-4202-bf45-0e9c31a2d848","Type":"ContainerStarted","Data":"e2ba43446a6a50bea1a77ce32c821f65ade50b0fb449ef0199a695242b9e0950"}
Sep 18 13:59:55.206164 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:59:55.206110    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-84c885658c-hwgkx" event={"ID":"98c35b3b-f121-4202-bf45-0e9c31a2d848","Type":"ContainerStarted","Data":"343b6df492a9b0636fec015c17c81fe5b1ac076ef7c8738e415238232cd628bc"}
Sep 18 13:59:55.229348 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:59:55.229251    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-psstj" podStartSLOduration=2.295655498 podStartE2EDuration="3.229234924s" totalImagesPullingTime="869.33486ms" totalInitContainerRuntime="64.244566ms" isStatefulPod=true podCreationTimestamp="2026-09-18 13:59:52 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 13:59:55.22770539 +0000 UTC m=+63.401615074" watchObservedRunningTime="2026-09-18 13:59:55.229234924 +0000 UTC m=+63.403144581"
Sep 18 13:59:55.249814 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:59:55.249755    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/kube-state-metrics-84c885658c-hwgkx" podStartSLOduration=1.976306745 podStartE2EDuration="3.249733556s" totalImagesPullingTime="1.273426811s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 13:59:52 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 13:59:55.248767204 +0000 UTC m=+63.422676867" watchObservedRunningTime="2026-09-18 13:59:55.249733556 +0000 UTC m=+63.423643218"
Sep 18 13:59:56.212374 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:59:56.212332    2565 generic.go:356] "Generic (PLEG): container finished" podID="ee93bad9-7ad1-4dee-97fb-b16f6e1e1459" containerID="5335dd5531981afa3da021508066d3ee5575c0160e7a8230c9b9d1a8a94bc14c" exitCode=0
Sep 18 13:59:56.212780 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:59:56.212427    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"ee93bad9-7ad1-4dee-97fb-b16f6e1e1459","Type":"ContainerDied","Data":"5335dd5531981afa3da021508066d3ee5575c0160e7a8230c9b9d1a8a94bc14c"}
Sep 18 13:59:56.215578 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:59:56.215547    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-86c5bc79d6-tgk6f" event={"ID":"27863c15-1a0a-4577-a57e-39db30081e27","Type":"ContainerStarted","Data":"da77e5d3de11ccc37ca3547de8b2442446d2fda1377baf1c394156a06dd15f14"}
Sep 18 13:59:56.262212 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:59:56.261942    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/openshift-state-metrics-86c5bc79d6-tgk6f" podStartSLOduration=3.054794887 podStartE2EDuration="4.261922208s" totalImagesPullingTime="1.207127321s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 13:59:52 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 13:59:56.26109126 +0000 UTC m=+64.435000940" watchObservedRunningTime="2026-09-18 13:59:56.261922208 +0000 UTC m=+64.435831863"
Sep 18 13:59:56.910894 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:59:56.910857    2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/metrics-server-99bb6cbd5-4mdgm"]
Sep 18 13:59:56.913576 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:59:56.913553    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/metrics-server-99bb6cbd5-4mdgm"
Sep 18 13:59:56.916401 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:59:56.916378    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-server-audit-profiles\""
Sep 18 13:59:56.916533 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:59:56.916398    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kubelet-serving-ca-bundle\""
Sep 18 13:59:56.916592 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:59:56.916525    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-tls\""
Sep 18 13:59:56.917463 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:59:56.917447    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-4q4mjg1lgg8ar\""
Sep 18 13:59:56.917463 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:59:56.917458    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-client-certs\""
Sep 18 13:59:56.917636 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:59:56.917460    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-dockercfg-7ddkw\""
Sep 18 13:59:56.923880 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:59:56.923855    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/metrics-server-99bb6cbd5-4mdgm"]
Sep 18 13:59:56.975309 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:59:56.975280    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/9533560d-e6fc-4d8b-984e-a9cf6f0f0315-metrics-server-audit-profiles\") pod \"metrics-server-99bb6cbd5-4mdgm\" (UID: \"9533560d-e6fc-4d8b-984e-a9cf6f0f0315\") " pod="openshift-monitoring/metrics-server-99bb6cbd5-4mdgm"
Sep 18 13:59:56.975490 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:59:56.975348    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/9533560d-e6fc-4d8b-984e-a9cf6f0f0315-secret-metrics-server-client-certs\") pod \"metrics-server-99bb6cbd5-4mdgm\" (UID: \"9533560d-e6fc-4d8b-984e-a9cf6f0f0315\") " pod="openshift-monitoring/metrics-server-99bb6cbd5-4mdgm"
Sep 18 13:59:56.975490 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:59:56.975399    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/9533560d-e6fc-4d8b-984e-a9cf6f0f0315-secret-metrics-server-tls\") pod \"metrics-server-99bb6cbd5-4mdgm\" (UID: \"9533560d-e6fc-4d8b-984e-a9cf6f0f0315\") " pod="openshift-monitoring/metrics-server-99bb6cbd5-4mdgm"
Sep 18 13:59:56.975490 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:59:56.975463    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jhwxt\" (UniqueName: \"kubernetes.io/projected/9533560d-e6fc-4d8b-984e-a9cf6f0f0315-kube-api-access-jhwxt\") pod \"metrics-server-99bb6cbd5-4mdgm\" (UID: \"9533560d-e6fc-4d8b-984e-a9cf6f0f0315\") " pod="openshift-monitoring/metrics-server-99bb6cbd5-4mdgm"
Sep 18 13:59:56.975656 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:59:56.975503    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/9533560d-e6fc-4d8b-984e-a9cf6f0f0315-audit-log\") pod \"metrics-server-99bb6cbd5-4mdgm\" (UID: \"9533560d-e6fc-4d8b-984e-a9cf6f0f0315\") " pod="openshift-monitoring/metrics-server-99bb6cbd5-4mdgm"
Sep 18 13:59:56.975656 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:59:56.975546    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/9533560d-e6fc-4d8b-984e-a9cf6f0f0315-client-ca-bundle\") pod \"metrics-server-99bb6cbd5-4mdgm\" (UID: \"9533560d-e6fc-4d8b-984e-a9cf6f0f0315\") " pod="openshift-monitoring/metrics-server-99bb6cbd5-4mdgm"
Sep 18 13:59:56.975656 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:59:56.975609    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9533560d-e6fc-4d8b-984e-a9cf6f0f0315-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-99bb6cbd5-4mdgm\" (UID: \"9533560d-e6fc-4d8b-984e-a9cf6f0f0315\") " pod="openshift-monitoring/metrics-server-99bb6cbd5-4mdgm"
Sep 18 13:59:57.009897 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:59:57.009813    2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/monitoring-plugin-5984459ff8-p9gl7"]
Sep 18 13:59:57.012348 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:59:57.012329    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/monitoring-plugin-5984459ff8-p9gl7"
Sep 18 13:59:57.015071 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:59:57.014985    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"monitoring-plugin-dockercfg-kwfsg\""
Sep 18 13:59:57.015251 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:59:57.015233    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"monitoring-plugin-cert\""
Sep 18 13:59:57.023851 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:59:57.023762    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/monitoring-plugin-5984459ff8-p9gl7"]
Sep 18 13:59:57.076647 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:59:57.076613    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/9c851437-b7ed-4359-8335-7af07f726404-monitoring-plugin-cert\") pod \"monitoring-plugin-5984459ff8-p9gl7\" (UID: \"9c851437-b7ed-4359-8335-7af07f726404\") " pod="openshift-monitoring/monitoring-plugin-5984459ff8-p9gl7"
Sep 18 13:59:57.076840 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:59:57.076664    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hd2fr\" (UniqueName: \"kubernetes.io/projected/9c851437-b7ed-4359-8335-7af07f726404-kube-api-access-hd2fr\") pod \"monitoring-plugin-5984459ff8-p9gl7\" (UID: \"9c851437-b7ed-4359-8335-7af07f726404\") " pod="openshift-monitoring/monitoring-plugin-5984459ff8-p9gl7"
Sep 18 13:59:57.076840 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:59:57.076703    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jhwxt\" (UniqueName: \"kubernetes.io/projected/9533560d-e6fc-4d8b-984e-a9cf6f0f0315-kube-api-access-jhwxt\") pod \"metrics-server-99bb6cbd5-4mdgm\" (UID: \"9533560d-e6fc-4d8b-984e-a9cf6f0f0315\") " pod="openshift-monitoring/metrics-server-99bb6cbd5-4mdgm"
Sep 18 13:59:57.076840 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:59:57.076736    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/9533560d-e6fc-4d8b-984e-a9cf6f0f0315-audit-log\") pod \"metrics-server-99bb6cbd5-4mdgm\" (UID: \"9533560d-e6fc-4d8b-984e-a9cf6f0f0315\") " pod="openshift-monitoring/metrics-server-99bb6cbd5-4mdgm"
Sep 18 13:59:57.076840 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:59:57.076765    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/9533560d-e6fc-4d8b-984e-a9cf6f0f0315-client-ca-bundle\") pod \"metrics-server-99bb6cbd5-4mdgm\" (UID: \"9533560d-e6fc-4d8b-984e-a9cf6f0f0315\") " pod="openshift-monitoring/metrics-server-99bb6cbd5-4mdgm"
Sep 18 13:59:57.076840 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:59:57.076792    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9533560d-e6fc-4d8b-984e-a9cf6f0f0315-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-99bb6cbd5-4mdgm\" (UID: \"9533560d-e6fc-4d8b-984e-a9cf6f0f0315\") " pod="openshift-monitoring/metrics-server-99bb6cbd5-4mdgm"
Sep 18 13:59:57.077135 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:59:57.076861    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/9533560d-e6fc-4d8b-984e-a9cf6f0f0315-metrics-server-audit-profiles\") pod \"metrics-server-99bb6cbd5-4mdgm\" (UID: \"9533560d-e6fc-4d8b-984e-a9cf6f0f0315\") " pod="openshift-monitoring/metrics-server-99bb6cbd5-4mdgm"
Sep 18 13:59:57.077135 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:59:57.076916    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/9533560d-e6fc-4d8b-984e-a9cf6f0f0315-secret-metrics-server-client-certs\") pod \"metrics-server-99bb6cbd5-4mdgm\" (UID: \"9533560d-e6fc-4d8b-984e-a9cf6f0f0315\") " pod="openshift-monitoring/metrics-server-99bb6cbd5-4mdgm"
Sep 18 13:59:57.077135 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:59:57.076944    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/9533560d-e6fc-4d8b-984e-a9cf6f0f0315-secret-metrics-server-tls\") pod \"metrics-server-99bb6cbd5-4mdgm\" (UID: \"9533560d-e6fc-4d8b-984e-a9cf6f0f0315\") " pod="openshift-monitoring/metrics-server-99bb6cbd5-4mdgm"
Sep 18 13:59:57.077290 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:59:57.077222    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/9533560d-e6fc-4d8b-984e-a9cf6f0f0315-audit-log\") pod \"metrics-server-99bb6cbd5-4mdgm\" (UID: \"9533560d-e6fc-4d8b-984e-a9cf6f0f0315\") " pod="openshift-monitoring/metrics-server-99bb6cbd5-4mdgm"
Sep 18 13:59:57.078027 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:59:57.077999    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9533560d-e6fc-4d8b-984e-a9cf6f0f0315-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-99bb6cbd5-4mdgm\" (UID: \"9533560d-e6fc-4d8b-984e-a9cf6f0f0315\") " pod="openshift-monitoring/metrics-server-99bb6cbd5-4mdgm"
Sep 18 13:59:57.078403 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:59:57.078356    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/9533560d-e6fc-4d8b-984e-a9cf6f0f0315-metrics-server-audit-profiles\") pod \"metrics-server-99bb6cbd5-4mdgm\" (UID: \"9533560d-e6fc-4d8b-984e-a9cf6f0f0315\") " pod="openshift-monitoring/metrics-server-99bb6cbd5-4mdgm"
Sep 18 13:59:57.080139 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:59:57.080115    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/9533560d-e6fc-4d8b-984e-a9cf6f0f0315-client-ca-bundle\") pod \"metrics-server-99bb6cbd5-4mdgm\" (UID: \"9533560d-e6fc-4d8b-984e-a9cf6f0f0315\") " pod="openshift-monitoring/metrics-server-99bb6cbd5-4mdgm"
Sep 18 13:59:57.080244 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:59:57.080195    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/9533560d-e6fc-4d8b-984e-a9cf6f0f0315-secret-metrics-server-tls\") pod \"metrics-server-99bb6cbd5-4mdgm\" (UID: \"9533560d-e6fc-4d8b-984e-a9cf6f0f0315\") " pod="openshift-monitoring/metrics-server-99bb6cbd5-4mdgm"
Sep 18 13:59:57.080327 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:59:57.080305    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/9533560d-e6fc-4d8b-984e-a9cf6f0f0315-secret-metrics-server-client-certs\") pod \"metrics-server-99bb6cbd5-4mdgm\" (UID: \"9533560d-e6fc-4d8b-984e-a9cf6f0f0315\") " pod="openshift-monitoring/metrics-server-99bb6cbd5-4mdgm"
Sep 18 13:59:57.086094 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:59:57.086076    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jhwxt\" (UniqueName: \"kubernetes.io/projected/9533560d-e6fc-4d8b-984e-a9cf6f0f0315-kube-api-access-jhwxt\") pod \"metrics-server-99bb6cbd5-4mdgm\" (UID: \"9533560d-e6fc-4d8b-984e-a9cf6f0f0315\") " pod="openshift-monitoring/metrics-server-99bb6cbd5-4mdgm"
Sep 18 13:59:57.177551 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:59:57.177463    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/9c851437-b7ed-4359-8335-7af07f726404-monitoring-plugin-cert\") pod \"monitoring-plugin-5984459ff8-p9gl7\" (UID: \"9c851437-b7ed-4359-8335-7af07f726404\") " pod="openshift-monitoring/monitoring-plugin-5984459ff8-p9gl7"
Sep 18 13:59:57.177551 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:59:57.177525    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hd2fr\" (UniqueName: \"kubernetes.io/projected/9c851437-b7ed-4359-8335-7af07f726404-kube-api-access-hd2fr\") pod \"monitoring-plugin-5984459ff8-p9gl7\" (UID: \"9c851437-b7ed-4359-8335-7af07f726404\") " pod="openshift-monitoring/monitoring-plugin-5984459ff8-p9gl7"
Sep 18 13:59:57.181551 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:59:57.180835    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/9c851437-b7ed-4359-8335-7af07f726404-monitoring-plugin-cert\") pod \"monitoring-plugin-5984459ff8-p9gl7\" (UID: \"9c851437-b7ed-4359-8335-7af07f726404\") " pod="openshift-monitoring/monitoring-plugin-5984459ff8-p9gl7"
Sep 18 13:59:57.187942 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:59:57.187920    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hd2fr\" (UniqueName: \"kubernetes.io/projected/9c851437-b7ed-4359-8335-7af07f726404-kube-api-access-hd2fr\") pod \"monitoring-plugin-5984459ff8-p9gl7\" (UID: \"9c851437-b7ed-4359-8335-7af07f726404\") " pod="openshift-monitoring/monitoring-plugin-5984459ff8-p9gl7"
Sep 18 13:59:57.224022 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:59:57.223994    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/metrics-server-99bb6cbd5-4mdgm"
Sep 18 13:59:57.347471 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:59:57.347428    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/monitoring-plugin-5984459ff8-p9gl7"
Sep 18 13:59:57.667164 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:59:57.667127    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/monitoring-plugin-5984459ff8-p9gl7"]
Sep 18 13:59:57.670272 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:59:57.670198    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/metrics-server-99bb6cbd5-4mdgm"]
Sep 18 13:59:57.932901 ip-10-0-129-16 kubenswrapper[2565]: W0918 13:59:57.932784    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9c851437_b7ed_4359_8335_7af07f726404.slice/crio-723eb35d453212ca4c9b5047780ba1d403601e2f63691801edbb810ab6b4bba6 WatchSource:0}: Error finding container 723eb35d453212ca4c9b5047780ba1d403601e2f63691801edbb810ab6b4bba6: Status 404 returned error can't find the container with id 723eb35d453212ca4c9b5047780ba1d403601e2f63691801edbb810ab6b4bba6
Sep 18 13:59:57.933189 ip-10-0-129-16 kubenswrapper[2565]: W0918 13:59:57.933162    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9533560d_e6fc_4d8b_984e_a9cf6f0f0315.slice/crio-16c1c33642c95d3f6e1ca8e7e3efac70d208f201fe8a56042946cd0d2a8a3c61 WatchSource:0}: Error finding container 16c1c33642c95d3f6e1ca8e7e3efac70d208f201fe8a56042946cd0d2a8a3c61: Status 404 returned error can't find the container with id 16c1c33642c95d3f6e1ca8e7e3efac70d208f201fe8a56042946cd0d2a8a3c61
Sep 18 13:59:58.225490 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:59:58.225456    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"ee93bad9-7ad1-4dee-97fb-b16f6e1e1459","Type":"ContainerStarted","Data":"7e3e20f0b86177313aa96d896e92b7c76658bfb49d53645d127641ef6d417fe2"}
Sep 18 13:59:58.225839 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:59:58.225541    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"ee93bad9-7ad1-4dee-97fb-b16f6e1e1459","Type":"ContainerStarted","Data":"9ef5343c3bc1e1eac16f57250234906547bf9ec7e415b7ae962e87c858b5c233"}
Sep 18 13:59:58.225839 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:59:58.225585    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"ee93bad9-7ad1-4dee-97fb-b16f6e1e1459","Type":"ContainerStarted","Data":"4c64db841dd019fb9e270c5be74c0d03d3e853c11feceaabbba536e258c337d2"}
Sep 18 13:59:58.225839 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:59:58.225625    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"ee93bad9-7ad1-4dee-97fb-b16f6e1e1459","Type":"ContainerStarted","Data":"960a7ab1d6f00de8536ec945bf1691e2cf2b7b08ab9193b75ba0cc2faf286cb3"}
Sep 18 13:59:58.228639 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:59:58.228590    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-5f599748b7-k2vqh" event={"ID":"476315e9-c60b-485e-aa57-9ec551db2c80","Type":"ContainerStarted","Data":"e246a723d6918450f3b92bae18a89b4f230f231b4b1842ad1521d9c0c315d22b"}
Sep 18 13:59:58.228751 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:59:58.228671    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-5f599748b7-k2vqh" event={"ID":"476315e9-c60b-485e-aa57-9ec551db2c80","Type":"ContainerStarted","Data":"5b49293c2e46f067ea6338a9d0a2bebb51688e5868294381886c52a8618306c7"}
Sep 18 13:59:58.228751 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:59:58.228717    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-5f599748b7-k2vqh" event={"ID":"476315e9-c60b-485e-aa57-9ec551db2c80","Type":"ContainerStarted","Data":"fbb2001c13a591759da2276cce0700930d294743e320e0fbb2d75c3f0554c3b9"}
Sep 18 13:59:58.229860 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:59:58.229837    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/metrics-server-99bb6cbd5-4mdgm" event={"ID":"9533560d-e6fc-4d8b-984e-a9cf6f0f0315","Type":"ContainerStarted","Data":"16c1c33642c95d3f6e1ca8e7e3efac70d208f201fe8a56042946cd0d2a8a3c61"}
Sep 18 13:59:58.231155 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:59:58.231130    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/monitoring-plugin-5984459ff8-p9gl7" event={"ID":"9c851437-b7ed-4359-8335-7af07f726404","Type":"ContainerStarted","Data":"723eb35d453212ca4c9b5047780ba1d403601e2f63691801edbb810ab6b4bba6"}
Sep 18 13:59:58.812896 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:59:58.812215    2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Sep 18 13:59:58.815604 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:59:58.815582    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Sep 18 13:59:58.821318 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:59:58.821286    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-dockercfg-h5lqx\""
Sep 18 13:59:58.825334 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:59:58.822165    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-client-certs\""
Sep 18 13:59:58.825334 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:59:58.822459    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-web-config\""
Sep 18 13:59:58.825334 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:59:58.822674    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s\""
Sep 18 13:59:58.825334 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:59:58.822870    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-1\""
Sep 18 13:59:58.825334 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:59:58.823068    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-prometheus-http-client-file\""
Sep 18 13:59:58.825334 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:59:58.823504    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls-assets-0\""
Sep 18 13:59:58.825334 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:59:58.823992    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-2\""
Sep 18 13:59:58.825334 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:59:58.824224    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-rbac-proxy\""
Sep 18 13:59:58.825334 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:59:58.824578    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"serving-certs-ca-bundle\""
Sep 18 13:59:58.825334 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:59:58.824844    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-kube-rbac-proxy-web\""
Sep 18 13:59:58.838198 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:59:58.830581    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls\""
Sep 18 13:59:58.838198 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:59:58.831774    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-grpc-tls-hnmh6p546cld\""
Sep 18 13:59:58.838198 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:59:58.835771    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-sidecar-tls\""
Sep 18 13:59:58.840885 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:59:58.840860    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-0\""
Sep 18 13:59:58.845064 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:59:58.845025    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Sep 18 13:59:58.845767 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:59:58.845585    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-trusted-ca-bundle\""
Sep 18 13:59:58.893481 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:59:58.893445    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/1d0d5aec-4917-4b99-a58e-592c2863d81b-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"1d0d5aec-4917-4b99-a58e-592c2863d81b\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 18 13:59:58.893737 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:59:58.893496    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ccqdb\" (UniqueName: \"kubernetes.io/projected/1d0d5aec-4917-4b99-a58e-592c2863d81b-kube-api-access-ccqdb\") pod \"prometheus-k8s-0\" (UID: \"1d0d5aec-4917-4b99-a58e-592c2863d81b\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 18 13:59:58.893737 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:59:58.893526    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/1d0d5aec-4917-4b99-a58e-592c2863d81b-config\") pod \"prometheus-k8s-0\" (UID: \"1d0d5aec-4917-4b99-a58e-592c2863d81b\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 18 13:59:58.893737 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:59:58.893557    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/1d0d5aec-4917-4b99-a58e-592c2863d81b-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"1d0d5aec-4917-4b99-a58e-592c2863d81b\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 18 13:59:58.893737 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:59:58.893591    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/1d0d5aec-4917-4b99-a58e-592c2863d81b-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"1d0d5aec-4917-4b99-a58e-592c2863d81b\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 18 13:59:58.893737 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:59:58.893615    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1d0d5aec-4917-4b99-a58e-592c2863d81b-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"1d0d5aec-4917-4b99-a58e-592c2863d81b\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 18 13:59:58.893737 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:59:58.893641    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/1d0d5aec-4917-4b99-a58e-592c2863d81b-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"1d0d5aec-4917-4b99-a58e-592c2863d81b\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 18 13:59:58.893737 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:59:58.893695    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1d0d5aec-4917-4b99-a58e-592c2863d81b-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"1d0d5aec-4917-4b99-a58e-592c2863d81b\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 18 13:59:58.893737 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:59:58.893728    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/1d0d5aec-4917-4b99-a58e-592c2863d81b-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"1d0d5aec-4917-4b99-a58e-592c2863d81b\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 18 13:59:58.894242 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:59:58.893761    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/1d0d5aec-4917-4b99-a58e-592c2863d81b-config-out\") pod \"prometheus-k8s-0\" (UID: \"1d0d5aec-4917-4b99-a58e-592c2863d81b\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 18 13:59:58.894242 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:59:58.893790    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/1d0d5aec-4917-4b99-a58e-592c2863d81b-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"1d0d5aec-4917-4b99-a58e-592c2863d81b\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 18 13:59:58.894242 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:59:58.893826    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/1d0d5aec-4917-4b99-a58e-592c2863d81b-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"1d0d5aec-4917-4b99-a58e-592c2863d81b\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 18 13:59:58.894242 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:59:58.893873    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/1d0d5aec-4917-4b99-a58e-592c2863d81b-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"1d0d5aec-4917-4b99-a58e-592c2863d81b\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 18 13:59:58.894242 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:59:58.893900    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1d0d5aec-4917-4b99-a58e-592c2863d81b-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"1d0d5aec-4917-4b99-a58e-592c2863d81b\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 18 13:59:58.894242 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:59:58.893932    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/1d0d5aec-4917-4b99-a58e-592c2863d81b-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"1d0d5aec-4917-4b99-a58e-592c2863d81b\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 18 13:59:58.894242 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:59:58.893960    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/1d0d5aec-4917-4b99-a58e-592c2863d81b-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"1d0d5aec-4917-4b99-a58e-592c2863d81b\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 18 13:59:58.894242 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:59:58.894001    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/1d0d5aec-4917-4b99-a58e-592c2863d81b-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"1d0d5aec-4917-4b99-a58e-592c2863d81b\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 18 13:59:58.894242 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:59:58.894052    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/1d0d5aec-4917-4b99-a58e-592c2863d81b-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"1d0d5aec-4917-4b99-a58e-592c2863d81b\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 18 13:59:58.894242 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:59:58.894083    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/1d0d5aec-4917-4b99-a58e-592c2863d81b-web-config\") pod \"prometheus-k8s-0\" (UID: \"1d0d5aec-4917-4b99-a58e-592c2863d81b\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 18 13:59:58.894242 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:59:58.894115    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/1d0d5aec-4917-4b99-a58e-592c2863d81b-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"1d0d5aec-4917-4b99-a58e-592c2863d81b\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 18 13:59:58.995526 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:59:58.995437    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/1d0d5aec-4917-4b99-a58e-592c2863d81b-config-out\") pod \"prometheus-k8s-0\" (UID: \"1d0d5aec-4917-4b99-a58e-592c2863d81b\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 18 13:59:58.995526 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:59:58.995500    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/1d0d5aec-4917-4b99-a58e-592c2863d81b-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"1d0d5aec-4917-4b99-a58e-592c2863d81b\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 18 13:59:58.995736 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:59:58.995538    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/1d0d5aec-4917-4b99-a58e-592c2863d81b-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"1d0d5aec-4917-4b99-a58e-592c2863d81b\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 18 13:59:58.995736 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:59:58.995578    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/1d0d5aec-4917-4b99-a58e-592c2863d81b-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"1d0d5aec-4917-4b99-a58e-592c2863d81b\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 18 13:59:58.995736 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:59:58.995602    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1d0d5aec-4917-4b99-a58e-592c2863d81b-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"1d0d5aec-4917-4b99-a58e-592c2863d81b\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 18 13:59:58.995736 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:59:58.995627    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/1d0d5aec-4917-4b99-a58e-592c2863d81b-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"1d0d5aec-4917-4b99-a58e-592c2863d81b\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 18 13:59:58.995736 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:59:58.995654    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/1d0d5aec-4917-4b99-a58e-592c2863d81b-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"1d0d5aec-4917-4b99-a58e-592c2863d81b\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 18 13:59:58.995736 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:59:58.995698    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/1d0d5aec-4917-4b99-a58e-592c2863d81b-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"1d0d5aec-4917-4b99-a58e-592c2863d81b\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 18 13:59:58.995736 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:59:58.995726    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/1d0d5aec-4917-4b99-a58e-592c2863d81b-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"1d0d5aec-4917-4b99-a58e-592c2863d81b\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 18 13:59:58.996102 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:59:58.995754    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/1d0d5aec-4917-4b99-a58e-592c2863d81b-web-config\") pod \"prometheus-k8s-0\" (UID: \"1d0d5aec-4917-4b99-a58e-592c2863d81b\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 18 13:59:58.996102 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:59:58.995783    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/1d0d5aec-4917-4b99-a58e-592c2863d81b-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"1d0d5aec-4917-4b99-a58e-592c2863d81b\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 18 13:59:58.996102 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:59:58.995829    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/1d0d5aec-4917-4b99-a58e-592c2863d81b-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"1d0d5aec-4917-4b99-a58e-592c2863d81b\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 18 13:59:58.996102 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:59:58.995857    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ccqdb\" (UniqueName: \"kubernetes.io/projected/1d0d5aec-4917-4b99-a58e-592c2863d81b-kube-api-access-ccqdb\") pod \"prometheus-k8s-0\" (UID: \"1d0d5aec-4917-4b99-a58e-592c2863d81b\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 18 13:59:58.996102 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:59:58.995887    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/1d0d5aec-4917-4b99-a58e-592c2863d81b-config\") pod \"prometheus-k8s-0\" (UID: \"1d0d5aec-4917-4b99-a58e-592c2863d81b\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 18 13:59:58.996102 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:59:58.995916    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/1d0d5aec-4917-4b99-a58e-592c2863d81b-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"1d0d5aec-4917-4b99-a58e-592c2863d81b\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 18 13:59:58.996102 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:59:58.995950    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/1d0d5aec-4917-4b99-a58e-592c2863d81b-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"1d0d5aec-4917-4b99-a58e-592c2863d81b\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 18 13:59:58.996102 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:59:58.995976    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1d0d5aec-4917-4b99-a58e-592c2863d81b-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"1d0d5aec-4917-4b99-a58e-592c2863d81b\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 18 13:59:58.996102 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:59:58.996004    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/1d0d5aec-4917-4b99-a58e-592c2863d81b-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"1d0d5aec-4917-4b99-a58e-592c2863d81b\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 18 13:59:58.996102 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:59:58.996088    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1d0d5aec-4917-4b99-a58e-592c2863d81b-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"1d0d5aec-4917-4b99-a58e-592c2863d81b\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 18 13:59:58.996589 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:59:58.996127    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/1d0d5aec-4917-4b99-a58e-592c2863d81b-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"1d0d5aec-4917-4b99-a58e-592c2863d81b\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 18 13:59:58.996589 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:59:58.996132    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/1d0d5aec-4917-4b99-a58e-592c2863d81b-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"1d0d5aec-4917-4b99-a58e-592c2863d81b\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 18 13:59:58.996923 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:59:58.996895    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1d0d5aec-4917-4b99-a58e-592c2863d81b-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"1d0d5aec-4917-4b99-a58e-592c2863d81b\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 18 13:59:58.999606 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:59:58.999255    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/1d0d5aec-4917-4b99-a58e-592c2863d81b-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"1d0d5aec-4917-4b99-a58e-592c2863d81b\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 18 13:59:59.000449 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:59:59.000287    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/1d0d5aec-4917-4b99-a58e-592c2863d81b-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"1d0d5aec-4917-4b99-a58e-592c2863d81b\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 18 13:59:59.000933 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:59:59.000772    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1d0d5aec-4917-4b99-a58e-592c2863d81b-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"1d0d5aec-4917-4b99-a58e-592c2863d81b\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 18 13:59:59.000933 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:59:59.000884    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/1d0d5aec-4917-4b99-a58e-592c2863d81b-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"1d0d5aec-4917-4b99-a58e-592c2863d81b\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 18 13:59:59.001903 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:59:59.001281    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1d0d5aec-4917-4b99-a58e-592c2863d81b-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"1d0d5aec-4917-4b99-a58e-592c2863d81b\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 18 13:59:59.001903 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:59:59.001784    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/1d0d5aec-4917-4b99-a58e-592c2863d81b-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"1d0d5aec-4917-4b99-a58e-592c2863d81b\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 18 13:59:59.002091 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:59:59.001927    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/1d0d5aec-4917-4b99-a58e-592c2863d81b-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"1d0d5aec-4917-4b99-a58e-592c2863d81b\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 18 13:59:59.002699 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:59:59.002202    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/1d0d5aec-4917-4b99-a58e-592c2863d81b-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"1d0d5aec-4917-4b99-a58e-592c2863d81b\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 18 13:59:59.002699 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:59:59.002356    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/1d0d5aec-4917-4b99-a58e-592c2863d81b-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"1d0d5aec-4917-4b99-a58e-592c2863d81b\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 18 13:59:59.002699 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:59:59.002654    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/1d0d5aec-4917-4b99-a58e-592c2863d81b-web-config\") pod \"prometheus-k8s-0\" (UID: \"1d0d5aec-4917-4b99-a58e-592c2863d81b\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 18 13:59:59.002699 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:59:59.002660    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/1d0d5aec-4917-4b99-a58e-592c2863d81b-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"1d0d5aec-4917-4b99-a58e-592c2863d81b\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 18 13:59:59.003012 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:59:59.002981    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/1d0d5aec-4917-4b99-a58e-592c2863d81b-config-out\") pod \"prometheus-k8s-0\" (UID: \"1d0d5aec-4917-4b99-a58e-592c2863d81b\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 18 13:59:59.003262 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:59:59.003236    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/1d0d5aec-4917-4b99-a58e-592c2863d81b-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"1d0d5aec-4917-4b99-a58e-592c2863d81b\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 18 13:59:59.003610 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:59:59.003570    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/1d0d5aec-4917-4b99-a58e-592c2863d81b-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"1d0d5aec-4917-4b99-a58e-592c2863d81b\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 18 13:59:59.004309 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:59:59.004283    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/1d0d5aec-4917-4b99-a58e-592c2863d81b-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"1d0d5aec-4917-4b99-a58e-592c2863d81b\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 18 13:59:59.004497 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:59:59.004465    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/1d0d5aec-4917-4b99-a58e-592c2863d81b-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"1d0d5aec-4917-4b99-a58e-592c2863d81b\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 18 13:59:59.005240 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:59:59.005224    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/secret/1d0d5aec-4917-4b99-a58e-592c2863d81b-config\") pod \"prometheus-k8s-0\" (UID: \"1d0d5aec-4917-4b99-a58e-592c2863d81b\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 18 13:59:59.016050 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:59:59.015433    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ccqdb\" (UniqueName: \"kubernetes.io/projected/1d0d5aec-4917-4b99-a58e-592c2863d81b-kube-api-access-ccqdb\") pod \"prometheus-k8s-0\" (UID: \"1d0d5aec-4917-4b99-a58e-592c2863d81b\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 18 13:59:59.148819 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:59:59.148746    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Sep 18 13:59:59.163233 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:59:59.163137    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-54mm2"
Sep 18 13:59:59.246792 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:59:59.246703    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"ee93bad9-7ad1-4dee-97fb-b16f6e1e1459","Type":"ContainerStarted","Data":"db45ed57d9795a67392898795b4507f422b52ec0f9fad6a124941897d46655d8"}
Sep 18 13:59:59.247223 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:59:59.246792    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"ee93bad9-7ad1-4dee-97fb-b16f6e1e1459","Type":"ContainerStarted","Data":"35a37ab880ea6a325e056f361fe49df6ce1f73a8d9c42bea62b8b7ab37419157"}
Sep 18 13:59:59.253736 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:59:59.253696    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-5f599748b7-k2vqh" event={"ID":"476315e9-c60b-485e-aa57-9ec551db2c80","Type":"ContainerStarted","Data":"38283a8ea6dc53bb82d525c22654633a3a7818d0bf9d0b96c98217a311715806"}
Sep 18 13:59:59.253879 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:59:59.253781    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-5f599748b7-k2vqh" event={"ID":"476315e9-c60b-485e-aa57-9ec551db2c80","Type":"ContainerStarted","Data":"81fd79aa5feb0c7f39161d9e03a3612635351e161ed146570739fa4fb6271bc0"}
Sep 18 13:59:59.253879 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:59:59.253827    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-5f599748b7-k2vqh" event={"ID":"476315e9-c60b-485e-aa57-9ec551db2c80","Type":"ContainerStarted","Data":"912b1edec32ca1dd432063d7334d568e2b45d1f43ccc3720c95e87c025029ae7"}
Sep 18 13:59:59.256478 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:59:59.254681    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/thanos-querier-5f599748b7-k2vqh"
Sep 18 13:59:59.309732 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:59:59.309628    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/alertmanager-main-0" podStartSLOduration=2.703061657 podStartE2EDuration="6.30960713s" totalImagesPullingTime="3.590511s" totalInitContainerRuntime="16.034473ms" isStatefulPod=false podCreationTimestamp="2026-09-18 13:59:53 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 13:59:59.309319406 +0000 UTC m=+67.483229097" watchObservedRunningTime="2026-09-18 13:59:59.30960713 +0000 UTC m=+67.483516791"
Sep 18 13:59:59.362093 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:59:59.362021    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/thanos-querier-5f599748b7-k2vqh" podStartSLOduration=2.19696074 podStartE2EDuration="5.361999655s" totalImagesPullingTime="3.165038915s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 13:59:54 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 13:59:59.359512583 +0000 UTC m=+67.533422304" watchObservedRunningTime="2026-09-18 13:59:59.361999655 +0000 UTC m=+67.535909314"
Sep 18 13:59:59.603143 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:59:59.603020    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7fe04bef-1706-4159-8a98-89a352d62c7a-metrics-certs\") pod \"network-metrics-daemon-vpgtc\" (UID: \"7fe04bef-1706-4159-8a98-89a352d62c7a\") " pod="openshift-multus/network-metrics-daemon-vpgtc"
Sep 18 13:59:59.606106 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:59:59.606079    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Sep 18 13:59:59.615922 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:59:59.615894    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7fe04bef-1706-4159-8a98-89a352d62c7a-metrics-certs\") pod \"network-metrics-daemon-vpgtc\" (UID: \"7fe04bef-1706-4159-8a98-89a352d62c7a\") " pod="openshift-multus/network-metrics-daemon-vpgtc"
Sep 18 13:59:59.626099 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:59:59.626075    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-8cnh9\""
Sep 18 13:59:59.633762 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:59:59.633738    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-vpgtc"
Sep 18 13:59:59.703509 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:59:59.703473    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gqntj\" (UniqueName: \"kubernetes.io/projected/46552e87-3933-4e45-97c0-257d31588c9d-kube-api-access-gqntj\") pod \"network-check-target-d4rf2\" (UID: \"46552e87-3933-4e45-97c0-257d31588c9d\") " pod="openshift-network-diagnostics/network-check-target-d4rf2"
Sep 18 13:59:59.703712 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:59:59.703579    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/d83ca837-1f57-4ff5-a513-a35f36198a99-original-pull-secret\") pod \"global-pull-secret-syncer-xqrkr\" (UID: \"d83ca837-1f57-4ff5-a513-a35f36198a99\") " pod="kube-system/global-pull-secret-syncer-xqrkr"
Sep 18 13:59:59.706588 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:59:59.706562    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Sep 18 13:59:59.706887 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:59:59.706871    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Sep 18 13:59:59.716234 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:59:59.716214    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Sep 18 13:59:59.716496 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:59:59.716465    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/d83ca837-1f57-4ff5-a513-a35f36198a99-original-pull-secret\") pod \"global-pull-secret-syncer-xqrkr\" (UID: \"d83ca837-1f57-4ff5-a513-a35f36198a99\") " pod="kube-system/global-pull-secret-syncer-xqrkr"
Sep 18 13:59:59.726890 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:59:59.726865    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gqntj\" (UniqueName: \"kubernetes.io/projected/46552e87-3933-4e45-97c0-257d31588c9d-kube-api-access-gqntj\") pod \"network-check-target-d4rf2\" (UID: \"46552e87-3933-4e45-97c0-257d31588c9d\") " pod="openshift-network-diagnostics/network-check-target-d4rf2"
Sep 18 13:59:59.916856 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:59:59.916782    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-xqrkr"
Sep 18 13:59:59.930920 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:59:59.930893    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-z6j5p\""
Sep 18 13:59:59.938127 ip-10-0-129-16 kubenswrapper[2565]: I0918 13:59:59.938110    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-d4rf2"
Sep 18 14:00:00.516520 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:00:00.516382    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-xqrkr"]
Sep 18 14:00:00.517432 ip-10-0-129-16 kubenswrapper[2565]: W0918 14:00:00.517407    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd83ca837_1f57_4ff5_a513_a35f36198a99.slice/crio-2a556583987689124b4bc9c7feb0eff47c85a9bcba3e564389f297d830171c18 WatchSource:0}: Error finding container 2a556583987689124b4bc9c7feb0eff47c85a9bcba3e564389f297d830171c18: Status 404 returned error can't find the container with id 2a556583987689124b4bc9c7feb0eff47c85a9bcba3e564389f297d830171c18
Sep 18 14:00:00.517892 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:00:00.517865    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-vpgtc"]
Sep 18 14:00:00.518122 ip-10-0-129-16 kubenswrapper[2565]: W0918 14:00:00.518102    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7fe04bef_1706_4159_8a98_89a352d62c7a.slice/crio-71c83f5c3e9563d63759ce64bef8962f1d535cf59945fcbca250b5c8ccd160b5 WatchSource:0}: Error finding container 71c83f5c3e9563d63759ce64bef8962f1d535cf59945fcbca250b5c8ccd160b5: Status 404 returned error can't find the container with id 71c83f5c3e9563d63759ce64bef8962f1d535cf59945fcbca250b5c8ccd160b5
Sep 18 14:00:00.537903 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:00:00.537866    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Sep 18 14:00:00.538103 ip-10-0-129-16 kubenswrapper[2565]: W0918 14:00:00.538083    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1d0d5aec_4917_4b99_a58e_592c2863d81b.slice/crio-728a437efb3a9040b4513b80b8684937406827d98d6dd1f09e4fb751c98ab019 WatchSource:0}: Error finding container 728a437efb3a9040b4513b80b8684937406827d98d6dd1f09e4fb751c98ab019: Status 404 returned error can't find the container with id 728a437efb3a9040b4513b80b8684937406827d98d6dd1f09e4fb751c98ab019
Sep 18 14:00:00.551226 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:00:00.551122    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-d4rf2"]
Sep 18 14:00:00.553191 ip-10-0-129-16 kubenswrapper[2565]: W0918 14:00:00.553114    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod46552e87_3933_4e45_97c0_257d31588c9d.slice/crio-eb1c93f88590a3878ff1c6c32ebc10fbc224f4a862a01542ecd239e6c18287cc WatchSource:0}: Error finding container eb1c93f88590a3878ff1c6c32ebc10fbc224f4a862a01542ecd239e6c18287cc: Status 404 returned error can't find the container with id eb1c93f88590a3878ff1c6c32ebc10fbc224f4a862a01542ecd239e6c18287cc
Sep 18 14:00:01.269292 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:00:01.269253    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/metrics-server-99bb6cbd5-4mdgm" event={"ID":"9533560d-e6fc-4d8b-984e-a9cf6f0f0315","Type":"ContainerStarted","Data":"ea441b46466b4a643c88da4a5f9d9fe3ee3540b4d29c5f4808328234da7916e6"}
Sep 18 14:00:01.271713 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:00:01.271685    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-d4rf2" event={"ID":"46552e87-3933-4e45-97c0-257d31588c9d","Type":"ContainerStarted","Data":"eb1c93f88590a3878ff1c6c32ebc10fbc224f4a862a01542ecd239e6c18287cc"}
Sep 18 14:00:01.275351 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:00:01.275325    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-vpgtc" event={"ID":"7fe04bef-1706-4159-8a98-89a352d62c7a","Type":"ContainerStarted","Data":"71c83f5c3e9563d63759ce64bef8962f1d535cf59945fcbca250b5c8ccd160b5"}
Sep 18 14:00:01.278113 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:00:01.277320    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/monitoring-plugin-5984459ff8-p9gl7" event={"ID":"9c851437-b7ed-4359-8335-7af07f726404","Type":"ContainerStarted","Data":"5aeca3ab8ded230e2ab9d8989e23eff32ee7d1f844985048fa216b4486ddb44d"}
Sep 18 14:00:01.278113 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:00:01.278073    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/monitoring-plugin-5984459ff8-p9gl7"
Sep 18 14:00:01.281215 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:00:01.280975    2565 generic.go:356] "Generic (PLEG): container finished" podID="1d0d5aec-4917-4b99-a58e-592c2863d81b" containerID="afe8246960b14fecc377cb32454ce7ea7b78e02f2a486f7e64561689c2fa4068" exitCode=0
Sep 18 14:00:01.281215 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:00:01.281102    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"1d0d5aec-4917-4b99-a58e-592c2863d81b","Type":"ContainerDied","Data":"afe8246960b14fecc377cb32454ce7ea7b78e02f2a486f7e64561689c2fa4068"}
Sep 18 14:00:01.281215 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:00:01.281161    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"1d0d5aec-4917-4b99-a58e-592c2863d81b","Type":"ContainerStarted","Data":"728a437efb3a9040b4513b80b8684937406827d98d6dd1f09e4fb751c98ab019"}
Sep 18 14:00:01.286170 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:00:01.285230    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-xqrkr" event={"ID":"d83ca837-1f57-4ff5-a513-a35f36198a99","Type":"ContainerStarted","Data":"2a556583987689124b4bc9c7feb0eff47c85a9bcba3e564389f297d830171c18"}
Sep 18 14:00:01.286170 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:00:01.285408    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/monitoring-plugin-5984459ff8-p9gl7"
Sep 18 14:00:01.311656 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:00:01.311599    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/monitoring-plugin-5984459ff8-p9gl7" podStartSLOduration=3.023028474 podStartE2EDuration="5.311582093s" totalImagesPullingTime="2.288553619s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 13:59:56 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 14:00:01.310777102 +0000 UTC m=+69.484686789" watchObservedRunningTime="2026-09-18 14:00:01.311582093 +0000 UTC m=+69.485491784"
Sep 18 14:00:01.312543 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:00:01.312500    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/metrics-server-99bb6cbd5-4mdgm" podStartSLOduration=3.022295398 podStartE2EDuration="5.312488666s" totalImagesPullingTime="2.290193268s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 13:59:56 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 14:00:01.289805278 +0000 UTC m=+69.463714965" watchObservedRunningTime="2026-09-18 14:00:01.312488666 +0000 UTC m=+69.486398308"
Sep 18 14:00:02.293045 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:00:02.292985    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-vpgtc" event={"ID":"7fe04bef-1706-4159-8a98-89a352d62c7a","Type":"ContainerStarted","Data":"df686061b384faf8f63ea6fd0d41969f83286d866b1237da8a863db7a9f6c75c"}
Sep 18 14:00:02.293492 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:00:02.293068    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-vpgtc" event={"ID":"7fe04bef-1706-4159-8a98-89a352d62c7a","Type":"ContainerStarted","Data":"8b5b1e0c9b24396a572e0c1a0c0e13e393c2ffc31a9575f085bc82d45c6ffa18"}
Sep 18 14:00:02.309371 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:00:02.309263    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-vpgtc" podStartSLOduration=67.122727524 podStartE2EDuration="1m8.309243246s" totalImagesPullingTime="1.186515722s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 13:58:54 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 14:00:02.308842091 +0000 UTC m=+70.482751749" watchObservedRunningTime="2026-09-18 14:00:02.309243246 +0000 UTC m=+70.483152907"
Sep 18 14:00:06.292216 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:00:06.292181    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/thanos-querier-5f599748b7-k2vqh"
Sep 18 14:00:06.322312 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:00:06.322272    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-xqrkr" event={"ID":"d83ca837-1f57-4ff5-a513-a35f36198a99","Type":"ContainerStarted","Data":"d85fb602c93d3dd94c85bd3e8b28ceb1cc45d41c038ce7ea5eef01d65c61105a"}
Sep 18 14:00:06.324153 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:00:06.324120    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-d4rf2" event={"ID":"46552e87-3933-4e45-97c0-257d31588c9d","Type":"ContainerStarted","Data":"289565508f70ac31d9d7e7efb755110421717908c95a41313c77caceebed61a6"}
Sep 18 14:00:06.324304 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:00:06.324225    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-d4rf2"
Sep 18 14:00:06.344149 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:00:06.344094    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-xqrkr" podStartSLOduration=67.376332333 podStartE2EDuration="1m12.344069814s" totalImagesPullingTime="4.967737481s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-18 13:58:54 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 14:00:06.342449396 +0000 UTC m=+74.516359048" watchObservedRunningTime="2026-09-18 14:00:06.344069814 +0000 UTC m=+74.517979474"
Sep 18 14:00:06.360168 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:00:06.360073    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-d4rf2" podStartSLOduration=68.433777395 podStartE2EDuration="1m13.360053707s" totalImagesPullingTime="4.926276312s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 13:58:53 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 14:00:06.359656436 +0000 UTC m=+74.533566118" watchObservedRunningTime="2026-09-18 14:00:06.360053707 +0000 UTC m=+74.533963372"
Sep 18 14:00:07.333553 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:00:07.333459    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"1d0d5aec-4917-4b99-a58e-592c2863d81b","Type":"ContainerStarted","Data":"95b5dac5b5886a4314daf23ac8bc5b0b7d0394f08e45cf1e00b44c51f684f442"}
Sep 18 14:00:07.333553 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:00:07.333536    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"1d0d5aec-4917-4b99-a58e-592c2863d81b","Type":"ContainerStarted","Data":"5670e8703e3d33730e46c486e162dcf3e63ae3c170476bb196fbed419cec7acb"}
Sep 18 14:00:07.334070 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:00:07.333567    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"1d0d5aec-4917-4b99-a58e-592c2863d81b","Type":"ContainerStarted","Data":"269624523c0629bfe048e9478419b509f305b696807736fd2360b16221cd10fb"}
Sep 18 14:00:07.334070 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:00:07.333591    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"1d0d5aec-4917-4b99-a58e-592c2863d81b","Type":"ContainerStarted","Data":"a065761a0ad0ee010aaad6fdd3c024a40cf2122fe5bceb2f7c66f85220e3e521"}
Sep 18 14:00:07.334070 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:00:07.333623    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"1d0d5aec-4917-4b99-a58e-592c2863d81b","Type":"ContainerStarted","Data":"ed5877be28108db692a5a67e1989ebbbe9f45e5fcdf0bcc114bdd37a251053b1"}
Sep 18 14:00:07.334070 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:00:07.333647    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"1d0d5aec-4917-4b99-a58e-592c2863d81b","Type":"ContainerStarted","Data":"7a640029364c17829cc96ab4d5210590ce031338fd16245acbcc8c39c0729c64"}
Sep 18 14:00:07.368436 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:00:07.368382    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-k8s-0" podStartSLOduration=3.823478484 podStartE2EDuration="9.368361654s" totalImagesPullingTime="5.509889242s" totalInitContainerRuntime="34.993928ms" isStatefulPod=false podCreationTimestamp="2026-09-18 13:59:58 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 14:00:07.365619448 +0000 UTC m=+75.539529132" watchObservedRunningTime="2026-09-18 14:00:07.368361654 +0000 UTC m=+75.542271313"
Sep 18 14:00:09.149633 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:00:09.149592    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-k8s-0"
Sep 18 14:00:17.225191 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:00:17.225145    2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/metrics-server-99bb6cbd5-4mdgm"
Sep 18 14:00:17.225609 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:00:17.225247    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/metrics-server-99bb6cbd5-4mdgm"
Sep 18 14:00:37.233942 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:00:37.233900    2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/metrics-server-99bb6cbd5-4mdgm"
Sep 18 14:00:37.240120 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:00:37.240089    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/metrics-server-99bb6cbd5-4mdgm"
Sep 18 14:00:37.337221 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:00:37.336375    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-d4rf2"
Sep 18 14:00:59.149320 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:00:59.149275    2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/prometheus-k8s-0"
Sep 18 14:00:59.168761 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:00:59.168732    2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/prometheus-k8s-0"
Sep 18 14:00:59.562692 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:00:59.562668    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-k8s-0"
Sep 18 14:01:12.814447 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:01:12.814408    2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Sep 18 14:01:12.815122 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:01:12.814815    2565 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="ee93bad9-7ad1-4dee-97fb-b16f6e1e1459" containerName="alertmanager" containerID="cri-o://960a7ab1d6f00de8536ec945bf1691e2cf2b7b08ab9193b75ba0cc2faf286cb3" gracePeriod=120
Sep 18 14:01:12.815122 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:01:12.814847    2565 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="ee93bad9-7ad1-4dee-97fb-b16f6e1e1459" containerName="kube-rbac-proxy-metric" containerID="cri-o://35a37ab880ea6a325e056f361fe49df6ce1f73a8d9c42bea62b8b7ab37419157" gracePeriod=120
Sep 18 14:01:12.815122 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:01:12.814856    2565 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="ee93bad9-7ad1-4dee-97fb-b16f6e1e1459" containerName="prom-label-proxy" containerID="cri-o://db45ed57d9795a67392898795b4507f422b52ec0f9fad6a124941897d46655d8" gracePeriod=120
Sep 18 14:01:12.815122 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:01:12.814921    2565 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="ee93bad9-7ad1-4dee-97fb-b16f6e1e1459" containerName="kube-rbac-proxy-web" containerID="cri-o://9ef5343c3bc1e1eac16f57250234906547bf9ec7e415b7ae962e87c858b5c233" gracePeriod=120
Sep 18 14:01:12.815122 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:01:12.814945    2565 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="ee93bad9-7ad1-4dee-97fb-b16f6e1e1459" containerName="kube-rbac-proxy" containerID="cri-o://7e3e20f0b86177313aa96d896e92b7c76658bfb49d53645d127641ef6d417fe2" gracePeriod=120
Sep 18 14:01:12.815122 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:01:12.814950    2565 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="ee93bad9-7ad1-4dee-97fb-b16f6e1e1459" containerName="config-reloader" containerID="cri-o://4c64db841dd019fb9e270c5be74c0d03d3e853c11feceaabbba536e258c337d2" gracePeriod=120
Sep 18 14:01:13.605635 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:01:13.605599    2565 generic.go:356] "Generic (PLEG): container finished" podID="ee93bad9-7ad1-4dee-97fb-b16f6e1e1459" containerID="db45ed57d9795a67392898795b4507f422b52ec0f9fad6a124941897d46655d8" exitCode=0
Sep 18 14:01:13.605635 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:01:13.605626    2565 generic.go:356] "Generic (PLEG): container finished" podID="ee93bad9-7ad1-4dee-97fb-b16f6e1e1459" containerID="35a37ab880ea6a325e056f361fe49df6ce1f73a8d9c42bea62b8b7ab37419157" exitCode=0
Sep 18 14:01:13.605635 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:01:13.605634    2565 generic.go:356] "Generic (PLEG): container finished" podID="ee93bad9-7ad1-4dee-97fb-b16f6e1e1459" containerID="7e3e20f0b86177313aa96d896e92b7c76658bfb49d53645d127641ef6d417fe2" exitCode=0
Sep 18 14:01:13.605635 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:01:13.605640    2565 generic.go:356] "Generic (PLEG): container finished" podID="ee93bad9-7ad1-4dee-97fb-b16f6e1e1459" containerID="4c64db841dd019fb9e270c5be74c0d03d3e853c11feceaabbba536e258c337d2" exitCode=0
Sep 18 14:01:13.605635 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:01:13.605645    2565 generic.go:356] "Generic (PLEG): container finished" podID="ee93bad9-7ad1-4dee-97fb-b16f6e1e1459" containerID="960a7ab1d6f00de8536ec945bf1691e2cf2b7b08ab9193b75ba0cc2faf286cb3" exitCode=0
Sep 18 14:01:13.605920 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:01:13.605677    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"ee93bad9-7ad1-4dee-97fb-b16f6e1e1459","Type":"ContainerDied","Data":"db45ed57d9795a67392898795b4507f422b52ec0f9fad6a124941897d46655d8"}
Sep 18 14:01:13.605920 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:01:13.605771    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"ee93bad9-7ad1-4dee-97fb-b16f6e1e1459","Type":"ContainerDied","Data":"35a37ab880ea6a325e056f361fe49df6ce1f73a8d9c42bea62b8b7ab37419157"}
Sep 18 14:01:13.605920 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:01:13.605800    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"ee93bad9-7ad1-4dee-97fb-b16f6e1e1459","Type":"ContainerDied","Data":"7e3e20f0b86177313aa96d896e92b7c76658bfb49d53645d127641ef6d417fe2"}
Sep 18 14:01:13.605920 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:01:13.605825    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"ee93bad9-7ad1-4dee-97fb-b16f6e1e1459","Type":"ContainerDied","Data":"4c64db841dd019fb9e270c5be74c0d03d3e853c11feceaabbba536e258c337d2"}
Sep 18 14:01:13.605920 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:01:13.605849    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"ee93bad9-7ad1-4dee-97fb-b16f6e1e1459","Type":"ContainerDied","Data":"960a7ab1d6f00de8536ec945bf1691e2cf2b7b08ab9193b75ba0cc2faf286cb3"}
Sep 18 14:01:14.059458 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:01:14.059433    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Sep 18 14:01:14.182461 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:01:14.182382    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/ee93bad9-7ad1-4dee-97fb-b16f6e1e1459-secret-alertmanager-kube-rbac-proxy-metric\") pod \"ee93bad9-7ad1-4dee-97fb-b16f6e1e1459\" (UID: \"ee93bad9-7ad1-4dee-97fb-b16f6e1e1459\") "
Sep 18 14:01:14.182461 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:01:14.182428    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/ee93bad9-7ad1-4dee-97fb-b16f6e1e1459-config-out\") pod \"ee93bad9-7ad1-4dee-97fb-b16f6e1e1459\" (UID: \"ee93bad9-7ad1-4dee-97fb-b16f6e1e1459\") "
Sep 18 14:01:14.182629 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:01:14.182472    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/ee93bad9-7ad1-4dee-97fb-b16f6e1e1459-secret-alertmanager-kube-rbac-proxy-web\") pod \"ee93bad9-7ad1-4dee-97fb-b16f6e1e1459\" (UID: \"ee93bad9-7ad1-4dee-97fb-b16f6e1e1459\") "
Sep 18 14:01:14.182669 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:01:14.182628    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/ee93bad9-7ad1-4dee-97fb-b16f6e1e1459-tls-assets\") pod \"ee93bad9-7ad1-4dee-97fb-b16f6e1e1459\" (UID: \"ee93bad9-7ad1-4dee-97fb-b16f6e1e1459\") "
Sep 18 14:01:14.182669 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:01:14.182663    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-gckwq\" (UniqueName: \"kubernetes.io/projected/ee93bad9-7ad1-4dee-97fb-b16f6e1e1459-kube-api-access-gckwq\") pod \"ee93bad9-7ad1-4dee-97fb-b16f6e1e1459\" (UID: \"ee93bad9-7ad1-4dee-97fb-b16f6e1e1459\") "
Sep 18 14:01:14.182756 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:01:14.182694    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/ee93bad9-7ad1-4dee-97fb-b16f6e1e1459-alertmanager-main-db\") pod \"ee93bad9-7ad1-4dee-97fb-b16f6e1e1459\" (UID: \"ee93bad9-7ad1-4dee-97fb-b16f6e1e1459\") "
Sep 18 14:01:14.182756 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:01:14.182730    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ee93bad9-7ad1-4dee-97fb-b16f6e1e1459-alertmanager-trusted-ca-bundle\") pod \"ee93bad9-7ad1-4dee-97fb-b16f6e1e1459\" (UID: \"ee93bad9-7ad1-4dee-97fb-b16f6e1e1459\") "
Sep 18 14:01:14.182856 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:01:14.182786    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/ee93bad9-7ad1-4dee-97fb-b16f6e1e1459-cluster-tls-config\") pod \"ee93bad9-7ad1-4dee-97fb-b16f6e1e1459\" (UID: \"ee93bad9-7ad1-4dee-97fb-b16f6e1e1459\") "
Sep 18 14:01:14.182856 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:01:14.182811    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/ee93bad9-7ad1-4dee-97fb-b16f6e1e1459-web-config\") pod \"ee93bad9-7ad1-4dee-97fb-b16f6e1e1459\" (UID: \"ee93bad9-7ad1-4dee-97fb-b16f6e1e1459\") "
Sep 18 14:01:14.182856 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:01:14.182837    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/ee93bad9-7ad1-4dee-97fb-b16f6e1e1459-config-volume\") pod \"ee93bad9-7ad1-4dee-97fb-b16f6e1e1459\" (UID: \"ee93bad9-7ad1-4dee-97fb-b16f6e1e1459\") "
Sep 18 14:01:14.183018 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:01:14.182873    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/ee93bad9-7ad1-4dee-97fb-b16f6e1e1459-secret-alertmanager-main-tls\") pod \"ee93bad9-7ad1-4dee-97fb-b16f6e1e1459\" (UID: \"ee93bad9-7ad1-4dee-97fb-b16f6e1e1459\") "
Sep 18 14:01:14.183018 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:01:14.182907    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/ee93bad9-7ad1-4dee-97fb-b16f6e1e1459-metrics-client-ca\") pod \"ee93bad9-7ad1-4dee-97fb-b16f6e1e1459\" (UID: \"ee93bad9-7ad1-4dee-97fb-b16f6e1e1459\") "
Sep 18 14:01:14.183018 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:01:14.182941    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/ee93bad9-7ad1-4dee-97fb-b16f6e1e1459-secret-alertmanager-kube-rbac-proxy\") pod \"ee93bad9-7ad1-4dee-97fb-b16f6e1e1459\" (UID: \"ee93bad9-7ad1-4dee-97fb-b16f6e1e1459\") "
Sep 18 14:01:14.183379 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:01:14.183345    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ee93bad9-7ad1-4dee-97fb-b16f6e1e1459-alertmanager-main-db" (OuterVolumeSpecName: "alertmanager-main-db") pod "ee93bad9-7ad1-4dee-97fb-b16f6e1e1459" (UID: "ee93bad9-7ad1-4dee-97fb-b16f6e1e1459"). InnerVolumeSpecName "alertmanager-main-db". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 18 14:01:14.185602 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:01:14.185516    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ee93bad9-7ad1-4dee-97fb-b16f6e1e1459-kube-api-access-gckwq" (OuterVolumeSpecName: "kube-api-access-gckwq") pod "ee93bad9-7ad1-4dee-97fb-b16f6e1e1459" (UID: "ee93bad9-7ad1-4dee-97fb-b16f6e1e1459"). InnerVolumeSpecName "kube-api-access-gckwq". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 18 14:01:14.185602 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:01:14.185582    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ee93bad9-7ad1-4dee-97fb-b16f6e1e1459-secret-alertmanager-kube-rbac-proxy-web" (OuterVolumeSpecName: "secret-alertmanager-kube-rbac-proxy-web") pod "ee93bad9-7ad1-4dee-97fb-b16f6e1e1459" (UID: "ee93bad9-7ad1-4dee-97fb-b16f6e1e1459"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy-web". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 18 14:01:14.185781 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:01:14.185606    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ee93bad9-7ad1-4dee-97fb-b16f6e1e1459-config-out" (OuterVolumeSpecName: "config-out") pod "ee93bad9-7ad1-4dee-97fb-b16f6e1e1459" (UID: "ee93bad9-7ad1-4dee-97fb-b16f6e1e1459"). InnerVolumeSpecName "config-out". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 18 14:01:14.185906 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:01:14.185884    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ee93bad9-7ad1-4dee-97fb-b16f6e1e1459-tls-assets" (OuterVolumeSpecName: "tls-assets") pod "ee93bad9-7ad1-4dee-97fb-b16f6e1e1459" (UID: "ee93bad9-7ad1-4dee-97fb-b16f6e1e1459"). InnerVolumeSpecName "tls-assets". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 18 14:01:14.185962 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:01:14.185925    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ee93bad9-7ad1-4dee-97fb-b16f6e1e1459-metrics-client-ca" (OuterVolumeSpecName: "metrics-client-ca") pod "ee93bad9-7ad1-4dee-97fb-b16f6e1e1459" (UID: "ee93bad9-7ad1-4dee-97fb-b16f6e1e1459"). InnerVolumeSpecName "metrics-client-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 18 14:01:14.185962 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:01:14.185937    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ee93bad9-7ad1-4dee-97fb-b16f6e1e1459-alertmanager-trusted-ca-bundle" (OuterVolumeSpecName: "alertmanager-trusted-ca-bundle") pod "ee93bad9-7ad1-4dee-97fb-b16f6e1e1459" (UID: "ee93bad9-7ad1-4dee-97fb-b16f6e1e1459"). InnerVolumeSpecName "alertmanager-trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 18 14:01:14.186239 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:01:14.186214    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ee93bad9-7ad1-4dee-97fb-b16f6e1e1459-secret-alertmanager-kube-rbac-proxy-metric" (OuterVolumeSpecName: "secret-alertmanager-kube-rbac-proxy-metric") pod "ee93bad9-7ad1-4dee-97fb-b16f6e1e1459" (UID: "ee93bad9-7ad1-4dee-97fb-b16f6e1e1459"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy-metric". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 18 14:01:14.186324 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:01:14.186289    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ee93bad9-7ad1-4dee-97fb-b16f6e1e1459-config-volume" (OuterVolumeSpecName: "config-volume") pod "ee93bad9-7ad1-4dee-97fb-b16f6e1e1459" (UID: "ee93bad9-7ad1-4dee-97fb-b16f6e1e1459"). InnerVolumeSpecName "config-volume". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 18 14:01:14.186393 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:01:14.186375    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ee93bad9-7ad1-4dee-97fb-b16f6e1e1459-secret-alertmanager-kube-rbac-proxy" (OuterVolumeSpecName: "secret-alertmanager-kube-rbac-proxy") pod "ee93bad9-7ad1-4dee-97fb-b16f6e1e1459" (UID: "ee93bad9-7ad1-4dee-97fb-b16f6e1e1459"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 18 14:01:14.187773 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:01:14.187755    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ee93bad9-7ad1-4dee-97fb-b16f6e1e1459-secret-alertmanager-main-tls" (OuterVolumeSpecName: "secret-alertmanager-main-tls") pod "ee93bad9-7ad1-4dee-97fb-b16f6e1e1459" (UID: "ee93bad9-7ad1-4dee-97fb-b16f6e1e1459"). InnerVolumeSpecName "secret-alertmanager-main-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 18 14:01:14.190289 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:01:14.190265    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ee93bad9-7ad1-4dee-97fb-b16f6e1e1459-cluster-tls-config" (OuterVolumeSpecName: "cluster-tls-config") pod "ee93bad9-7ad1-4dee-97fb-b16f6e1e1459" (UID: "ee93bad9-7ad1-4dee-97fb-b16f6e1e1459"). InnerVolumeSpecName "cluster-tls-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 18 14:01:14.197419 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:01:14.197397    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ee93bad9-7ad1-4dee-97fb-b16f6e1e1459-web-config" (OuterVolumeSpecName: "web-config") pod "ee93bad9-7ad1-4dee-97fb-b16f6e1e1459" (UID: "ee93bad9-7ad1-4dee-97fb-b16f6e1e1459"). InnerVolumeSpecName "web-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 18 14:01:14.283895 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:01:14.283856    2565 reconciler_common.go:299] "Volume detached for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/ee93bad9-7ad1-4dee-97fb-b16f6e1e1459-alertmanager-main-db\") on node \"ip-10-0-129-16.ec2.internal\" DevicePath \"\""
Sep 18 14:01:14.283895 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:01:14.283889    2565 reconciler_common.go:299] "Volume detached for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ee93bad9-7ad1-4dee-97fb-b16f6e1e1459-alertmanager-trusted-ca-bundle\") on node \"ip-10-0-129-16.ec2.internal\" DevicePath \"\""
Sep 18 14:01:14.283895 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:01:14.283900    2565 reconciler_common.go:299] "Volume detached for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/ee93bad9-7ad1-4dee-97fb-b16f6e1e1459-cluster-tls-config\") on node \"ip-10-0-129-16.ec2.internal\" DevicePath \"\""
Sep 18 14:01:14.284132 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:01:14.283910    2565 reconciler_common.go:299] "Volume detached for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/ee93bad9-7ad1-4dee-97fb-b16f6e1e1459-web-config\") on node \"ip-10-0-129-16.ec2.internal\" DevicePath \"\""
Sep 18 14:01:14.284132 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:01:14.283919    2565 reconciler_common.go:299] "Volume detached for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/ee93bad9-7ad1-4dee-97fb-b16f6e1e1459-config-volume\") on node \"ip-10-0-129-16.ec2.internal\" DevicePath \"\""
Sep 18 14:01:14.284132 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:01:14.283927    2565 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/ee93bad9-7ad1-4dee-97fb-b16f6e1e1459-secret-alertmanager-main-tls\") on node \"ip-10-0-129-16.ec2.internal\" DevicePath \"\""
Sep 18 14:01:14.284132 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:01:14.283935    2565 reconciler_common.go:299] "Volume detached for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/ee93bad9-7ad1-4dee-97fb-b16f6e1e1459-metrics-client-ca\") on node \"ip-10-0-129-16.ec2.internal\" DevicePath \"\""
Sep 18 14:01:14.284132 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:01:14.283943    2565 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/ee93bad9-7ad1-4dee-97fb-b16f6e1e1459-secret-alertmanager-kube-rbac-proxy\") on node \"ip-10-0-129-16.ec2.internal\" DevicePath \"\""
Sep 18 14:01:14.284132 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:01:14.283952    2565 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/ee93bad9-7ad1-4dee-97fb-b16f6e1e1459-secret-alertmanager-kube-rbac-proxy-metric\") on node \"ip-10-0-129-16.ec2.internal\" DevicePath \"\""
Sep 18 14:01:14.284132 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:01:14.283963    2565 reconciler_common.go:299] "Volume detached for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/ee93bad9-7ad1-4dee-97fb-b16f6e1e1459-config-out\") on node \"ip-10-0-129-16.ec2.internal\" DevicePath \"\""
Sep 18 14:01:14.284132 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:01:14.283971    2565 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/ee93bad9-7ad1-4dee-97fb-b16f6e1e1459-secret-alertmanager-kube-rbac-proxy-web\") on node \"ip-10-0-129-16.ec2.internal\" DevicePath \"\""
Sep 18 14:01:14.284132 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:01:14.283980    2565 reconciler_common.go:299] "Volume detached for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/ee93bad9-7ad1-4dee-97fb-b16f6e1e1459-tls-assets\") on node \"ip-10-0-129-16.ec2.internal\" DevicePath \"\""
Sep 18 14:01:14.284132 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:01:14.283987    2565 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-gckwq\" (UniqueName: \"kubernetes.io/projected/ee93bad9-7ad1-4dee-97fb-b16f6e1e1459-kube-api-access-gckwq\") on node \"ip-10-0-129-16.ec2.internal\" DevicePath \"\""
Sep 18 14:01:14.612797 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:01:14.612758    2565 generic.go:356] "Generic (PLEG): container finished" podID="ee93bad9-7ad1-4dee-97fb-b16f6e1e1459" containerID="9ef5343c3bc1e1eac16f57250234906547bf9ec7e415b7ae962e87c858b5c233" exitCode=0
Sep 18 14:01:14.612971 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:01:14.612848    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Sep 18 14:01:14.612971 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:01:14.612845    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"ee93bad9-7ad1-4dee-97fb-b16f6e1e1459","Type":"ContainerDied","Data":"9ef5343c3bc1e1eac16f57250234906547bf9ec7e415b7ae962e87c858b5c233"}
Sep 18 14:01:14.612971 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:01:14.612917    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"ee93bad9-7ad1-4dee-97fb-b16f6e1e1459","Type":"ContainerDied","Data":"6be049ee68c7f5308f03194381a257f271cba6a654062f52b3e392ccb57f0796"}
Sep 18 14:01:14.612971 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:01:14.612952    2565 scope.go:117] "RemoveContainer" containerID="db45ed57d9795a67392898795b4507f422b52ec0f9fad6a124941897d46655d8"
Sep 18 14:01:14.621316 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:01:14.621297    2565 scope.go:117] "RemoveContainer" containerID="35a37ab880ea6a325e056f361fe49df6ce1f73a8d9c42bea62b8b7ab37419157"
Sep 18 14:01:14.628392 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:01:14.628372    2565 scope.go:117] "RemoveContainer" containerID="7e3e20f0b86177313aa96d896e92b7c76658bfb49d53645d127641ef6d417fe2"
Sep 18 14:01:14.634831 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:01:14.634814    2565 scope.go:117] "RemoveContainer" containerID="9ef5343c3bc1e1eac16f57250234906547bf9ec7e415b7ae962e87c858b5c233"
Sep 18 14:01:14.638890 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:01:14.638867    2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Sep 18 14:01:14.645157 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:01:14.645136    2565 scope.go:117] "RemoveContainer" containerID="4c64db841dd019fb9e270c5be74c0d03d3e853c11feceaabbba536e258c337d2"
Sep 18 14:01:14.646178 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:01:14.646157    2565 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Sep 18 14:01:14.652652 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:01:14.652628    2565 scope.go:117] "RemoveContainer" containerID="960a7ab1d6f00de8536ec945bf1691e2cf2b7b08ab9193b75ba0cc2faf286cb3"
Sep 18 14:01:14.659370 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:01:14.659352    2565 scope.go:117] "RemoveContainer" containerID="5335dd5531981afa3da021508066d3ee5575c0160e7a8230c9b9d1a8a94bc14c"
Sep 18 14:01:14.667190 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:01:14.667174    2565 scope.go:117] "RemoveContainer" containerID="db45ed57d9795a67392898795b4507f422b52ec0f9fad6a124941897d46655d8"
Sep 18 14:01:14.667478 ip-10-0-129-16 kubenswrapper[2565]: E0918 14:01:14.667456    2565 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"db45ed57d9795a67392898795b4507f422b52ec0f9fad6a124941897d46655d8\": container with ID starting with db45ed57d9795a67392898795b4507f422b52ec0f9fad6a124941897d46655d8 not found: ID does not exist" containerID="db45ed57d9795a67392898795b4507f422b52ec0f9fad6a124941897d46655d8"
Sep 18 14:01:14.667542 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:01:14.667488    2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"db45ed57d9795a67392898795b4507f422b52ec0f9fad6a124941897d46655d8"} err="failed to get container status \"db45ed57d9795a67392898795b4507f422b52ec0f9fad6a124941897d46655d8\": rpc error: code = NotFound desc = could not find container \"db45ed57d9795a67392898795b4507f422b52ec0f9fad6a124941897d46655d8\": container with ID starting with db45ed57d9795a67392898795b4507f422b52ec0f9fad6a124941897d46655d8 not found: ID does not exist"
Sep 18 14:01:14.667542 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:01:14.667519    2565 scope.go:117] "RemoveContainer" containerID="35a37ab880ea6a325e056f361fe49df6ce1f73a8d9c42bea62b8b7ab37419157"
Sep 18 14:01:14.667776 ip-10-0-129-16 kubenswrapper[2565]: E0918 14:01:14.667755    2565 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"35a37ab880ea6a325e056f361fe49df6ce1f73a8d9c42bea62b8b7ab37419157\": container with ID starting with 35a37ab880ea6a325e056f361fe49df6ce1f73a8d9c42bea62b8b7ab37419157 not found: ID does not exist" containerID="35a37ab880ea6a325e056f361fe49df6ce1f73a8d9c42bea62b8b7ab37419157"
Sep 18 14:01:14.667817 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:01:14.667783    2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"35a37ab880ea6a325e056f361fe49df6ce1f73a8d9c42bea62b8b7ab37419157"} err="failed to get container status \"35a37ab880ea6a325e056f361fe49df6ce1f73a8d9c42bea62b8b7ab37419157\": rpc error: code = NotFound desc = could not find container \"35a37ab880ea6a325e056f361fe49df6ce1f73a8d9c42bea62b8b7ab37419157\": container with ID starting with 35a37ab880ea6a325e056f361fe49df6ce1f73a8d9c42bea62b8b7ab37419157 not found: ID does not exist"
Sep 18 14:01:14.667817 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:01:14.667800    2565 scope.go:117] "RemoveContainer" containerID="7e3e20f0b86177313aa96d896e92b7c76658bfb49d53645d127641ef6d417fe2"
Sep 18 14:01:14.668070 ip-10-0-129-16 kubenswrapper[2565]: E0918 14:01:14.668057    2565 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7e3e20f0b86177313aa96d896e92b7c76658bfb49d53645d127641ef6d417fe2\": container with ID starting with 7e3e20f0b86177313aa96d896e92b7c76658bfb49d53645d127641ef6d417fe2 not found: ID does not exist" containerID="7e3e20f0b86177313aa96d896e92b7c76658bfb49d53645d127641ef6d417fe2"
Sep 18 14:01:14.668110 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:01:14.668073    2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7e3e20f0b86177313aa96d896e92b7c76658bfb49d53645d127641ef6d417fe2"} err="failed to get container status \"7e3e20f0b86177313aa96d896e92b7c76658bfb49d53645d127641ef6d417fe2\": rpc error: code = NotFound desc = could not find container \"7e3e20f0b86177313aa96d896e92b7c76658bfb49d53645d127641ef6d417fe2\": container with ID starting with 7e3e20f0b86177313aa96d896e92b7c76658bfb49d53645d127641ef6d417fe2 not found: ID does not exist"
Sep 18 14:01:14.668110 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:01:14.668087    2565 scope.go:117] "RemoveContainer" containerID="9ef5343c3bc1e1eac16f57250234906547bf9ec7e415b7ae962e87c858b5c233"
Sep 18 14:01:14.668320 ip-10-0-129-16 kubenswrapper[2565]: E0918 14:01:14.668303    2565 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9ef5343c3bc1e1eac16f57250234906547bf9ec7e415b7ae962e87c858b5c233\": container with ID starting with 9ef5343c3bc1e1eac16f57250234906547bf9ec7e415b7ae962e87c858b5c233 not found: ID does not exist" containerID="9ef5343c3bc1e1eac16f57250234906547bf9ec7e415b7ae962e87c858b5c233"
Sep 18 14:01:14.668373 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:01:14.668324    2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9ef5343c3bc1e1eac16f57250234906547bf9ec7e415b7ae962e87c858b5c233"} err="failed to get container status \"9ef5343c3bc1e1eac16f57250234906547bf9ec7e415b7ae962e87c858b5c233\": rpc error: code = NotFound desc = could not find container \"9ef5343c3bc1e1eac16f57250234906547bf9ec7e415b7ae962e87c858b5c233\": container with ID starting with 9ef5343c3bc1e1eac16f57250234906547bf9ec7e415b7ae962e87c858b5c233 not found: ID does not exist"
Sep 18 14:01:14.668373 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:01:14.668337    2565 scope.go:117] "RemoveContainer" containerID="4c64db841dd019fb9e270c5be74c0d03d3e853c11feceaabbba536e258c337d2"
Sep 18 14:01:14.668644 ip-10-0-129-16 kubenswrapper[2565]: E0918 14:01:14.668628    2565 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4c64db841dd019fb9e270c5be74c0d03d3e853c11feceaabbba536e258c337d2\": container with ID starting with 4c64db841dd019fb9e270c5be74c0d03d3e853c11feceaabbba536e258c337d2 not found: ID does not exist" containerID="4c64db841dd019fb9e270c5be74c0d03d3e853c11feceaabbba536e258c337d2"
Sep 18 14:01:14.668704 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:01:14.668647    2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4c64db841dd019fb9e270c5be74c0d03d3e853c11feceaabbba536e258c337d2"} err="failed to get container status \"4c64db841dd019fb9e270c5be74c0d03d3e853c11feceaabbba536e258c337d2\": rpc error: code = NotFound desc = could not find container \"4c64db841dd019fb9e270c5be74c0d03d3e853c11feceaabbba536e258c337d2\": container with ID starting with 4c64db841dd019fb9e270c5be74c0d03d3e853c11feceaabbba536e258c337d2 not found: ID does not exist"
Sep 18 14:01:14.668704 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:01:14.668659    2565 scope.go:117] "RemoveContainer" containerID="960a7ab1d6f00de8536ec945bf1691e2cf2b7b08ab9193b75ba0cc2faf286cb3"
Sep 18 14:01:14.668961 ip-10-0-129-16 kubenswrapper[2565]: E0918 14:01:14.668935    2565 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"960a7ab1d6f00de8536ec945bf1691e2cf2b7b08ab9193b75ba0cc2faf286cb3\": container with ID starting with 960a7ab1d6f00de8536ec945bf1691e2cf2b7b08ab9193b75ba0cc2faf286cb3 not found: ID does not exist" containerID="960a7ab1d6f00de8536ec945bf1691e2cf2b7b08ab9193b75ba0cc2faf286cb3"
Sep 18 14:01:14.669084 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:01:14.668967    2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"960a7ab1d6f00de8536ec945bf1691e2cf2b7b08ab9193b75ba0cc2faf286cb3"} err="failed to get container status \"960a7ab1d6f00de8536ec945bf1691e2cf2b7b08ab9193b75ba0cc2faf286cb3\": rpc error: code = NotFound desc = could not find container \"960a7ab1d6f00de8536ec945bf1691e2cf2b7b08ab9193b75ba0cc2faf286cb3\": container with ID starting with 960a7ab1d6f00de8536ec945bf1691e2cf2b7b08ab9193b75ba0cc2faf286cb3 not found: ID does not exist"
Sep 18 14:01:14.669084 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:01:14.668988    2565 scope.go:117] "RemoveContainer" containerID="5335dd5531981afa3da021508066d3ee5575c0160e7a8230c9b9d1a8a94bc14c"
Sep 18 14:01:14.669695 ip-10-0-129-16 kubenswrapper[2565]: E0918 14:01:14.669649    2565 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5335dd5531981afa3da021508066d3ee5575c0160e7a8230c9b9d1a8a94bc14c\": container with ID starting with 5335dd5531981afa3da021508066d3ee5575c0160e7a8230c9b9d1a8a94bc14c not found: ID does not exist" containerID="5335dd5531981afa3da021508066d3ee5575c0160e7a8230c9b9d1a8a94bc14c"
Sep 18 14:01:14.669768 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:01:14.669705    2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5335dd5531981afa3da021508066d3ee5575c0160e7a8230c9b9d1a8a94bc14c"} err="failed to get container status \"5335dd5531981afa3da021508066d3ee5575c0160e7a8230c9b9d1a8a94bc14c\": rpc error: code = NotFound desc = could not find container \"5335dd5531981afa3da021508066d3ee5575c0160e7a8230c9b9d1a8a94bc14c\": container with ID starting with 5335dd5531981afa3da021508066d3ee5575c0160e7a8230c9b9d1a8a94bc14c not found: ID does not exist"
Sep 18 14:01:14.682602 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:01:14.682578    2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Sep 18 14:01:14.682872 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:01:14.682858    2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ee93bad9-7ad1-4dee-97fb-b16f6e1e1459" containerName="alertmanager"
Sep 18 14:01:14.682872 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:01:14.682873    2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="ee93bad9-7ad1-4dee-97fb-b16f6e1e1459" containerName="alertmanager"
Sep 18 14:01:14.682995 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:01:14.682880    2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ee93bad9-7ad1-4dee-97fb-b16f6e1e1459" containerName="config-reloader"
Sep 18 14:01:14.682995 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:01:14.682886    2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="ee93bad9-7ad1-4dee-97fb-b16f6e1e1459" containerName="config-reloader"
Sep 18 14:01:14.682995 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:01:14.682894    2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ee93bad9-7ad1-4dee-97fb-b16f6e1e1459" containerName="kube-rbac-proxy-metric"
Sep 18 14:01:14.682995 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:01:14.682899    2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="ee93bad9-7ad1-4dee-97fb-b16f6e1e1459" containerName="kube-rbac-proxy-metric"
Sep 18 14:01:14.682995 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:01:14.682909    2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ee93bad9-7ad1-4dee-97fb-b16f6e1e1459" containerName="kube-rbac-proxy"
Sep 18 14:01:14.682995 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:01:14.682914    2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="ee93bad9-7ad1-4dee-97fb-b16f6e1e1459" containerName="kube-rbac-proxy"
Sep 18 14:01:14.682995 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:01:14.682923    2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ee93bad9-7ad1-4dee-97fb-b16f6e1e1459" containerName="prom-label-proxy"
Sep 18 14:01:14.682995 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:01:14.682929    2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="ee93bad9-7ad1-4dee-97fb-b16f6e1e1459" containerName="prom-label-proxy"
Sep 18 14:01:14.682995 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:01:14.682939    2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ee93bad9-7ad1-4dee-97fb-b16f6e1e1459" containerName="kube-rbac-proxy-web"
Sep 18 14:01:14.682995 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:01:14.682945    2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="ee93bad9-7ad1-4dee-97fb-b16f6e1e1459" containerName="kube-rbac-proxy-web"
Sep 18 14:01:14.682995 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:01:14.682954    2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ee93bad9-7ad1-4dee-97fb-b16f6e1e1459" containerName="init-config-reloader"
Sep 18 14:01:14.682995 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:01:14.682959    2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="ee93bad9-7ad1-4dee-97fb-b16f6e1e1459" containerName="init-config-reloader"
Sep 18 14:01:14.683499 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:01:14.683004    2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="ee93bad9-7ad1-4dee-97fb-b16f6e1e1459" containerName="kube-rbac-proxy-metric"
Sep 18 14:01:14.683499 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:01:14.683012    2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="ee93bad9-7ad1-4dee-97fb-b16f6e1e1459" containerName="alertmanager"
Sep 18 14:01:14.683499 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:01:14.683020    2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="ee93bad9-7ad1-4dee-97fb-b16f6e1e1459" containerName="kube-rbac-proxy-web"
Sep 18 14:01:14.683499 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:01:14.683025    2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="ee93bad9-7ad1-4dee-97fb-b16f6e1e1459" containerName="kube-rbac-proxy"
Sep 18 14:01:14.683499 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:01:14.683049    2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="ee93bad9-7ad1-4dee-97fb-b16f6e1e1459" containerName="config-reloader"
Sep 18 14:01:14.683499 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:01:14.683056    2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="ee93bad9-7ad1-4dee-97fb-b16f6e1e1459" containerName="prom-label-proxy"
Sep 18 14:01:14.686271 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:01:14.686256    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Sep 18 14:01:14.689380 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:01:14.689350    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-generated\""
Sep 18 14:01:14.689477 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:01:14.689390    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-cluster-tls-config\""
Sep 18 14:01:14.689477 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:01:14.689405    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-dockercfg-2z746\""
Sep 18 14:01:14.689678 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:01:14.689662    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls\""
Sep 18 14:01:14.689769 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:01:14.689665    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy\""
Sep 18 14:01:14.689948 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:01:14.689927    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-web-config\""
Sep 18 14:01:14.690093 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:01:14.689927    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-web\""
Sep 18 14:01:14.690093 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:01:14.690016    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls-assets-0\""
Sep 18 14:01:14.694411 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:01:14.694390    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-metric\""
Sep 18 14:01:14.696848 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:01:14.696833    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"alertmanager-trusted-ca-bundle\""
Sep 18 14:01:14.713430 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:01:14.713404    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Sep 18 14:01:14.787669 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:01:14.787627    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/400c6df4-8db3-4ad7-86fc-0948cee05b23-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"400c6df4-8db3-4ad7-86fc-0948cee05b23\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 18 14:01:14.787846 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:01:14.787676    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/400c6df4-8db3-4ad7-86fc-0948cee05b23-tls-assets\") pod \"alertmanager-main-0\" (UID: \"400c6df4-8db3-4ad7-86fc-0948cee05b23\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 18 14:01:14.787846 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:01:14.787707    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ngp5g\" (UniqueName: \"kubernetes.io/projected/400c6df4-8db3-4ad7-86fc-0948cee05b23-kube-api-access-ngp5g\") pod \"alertmanager-main-0\" (UID: \"400c6df4-8db3-4ad7-86fc-0948cee05b23\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 18 14:01:14.787846 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:01:14.787756    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/400c6df4-8db3-4ad7-86fc-0948cee05b23-web-config\") pod \"alertmanager-main-0\" (UID: \"400c6df4-8db3-4ad7-86fc-0948cee05b23\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 18 14:01:14.787846 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:01:14.787794    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/400c6df4-8db3-4ad7-86fc-0948cee05b23-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"400c6df4-8db3-4ad7-86fc-0948cee05b23\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 18 14:01:14.787846 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:01:14.787814    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/400c6df4-8db3-4ad7-86fc-0948cee05b23-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"400c6df4-8db3-4ad7-86fc-0948cee05b23\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 18 14:01:14.787846 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:01:14.787832    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/400c6df4-8db3-4ad7-86fc-0948cee05b23-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"400c6df4-8db3-4ad7-86fc-0948cee05b23\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 18 14:01:14.788062 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:01:14.787883    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/400c6df4-8db3-4ad7-86fc-0948cee05b23-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"400c6df4-8db3-4ad7-86fc-0948cee05b23\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 18 14:01:14.788062 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:01:14.787910    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/400c6df4-8db3-4ad7-86fc-0948cee05b23-config-out\") pod \"alertmanager-main-0\" (UID: \"400c6df4-8db3-4ad7-86fc-0948cee05b23\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 18 14:01:14.788062 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:01:14.787953    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/400c6df4-8db3-4ad7-86fc-0948cee05b23-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"400c6df4-8db3-4ad7-86fc-0948cee05b23\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 18 14:01:14.788062 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:01:14.787972    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/400c6df4-8db3-4ad7-86fc-0948cee05b23-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"400c6df4-8db3-4ad7-86fc-0948cee05b23\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 18 14:01:14.788062 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:01:14.787991    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/400c6df4-8db3-4ad7-86fc-0948cee05b23-config-volume\") pod \"alertmanager-main-0\" (UID: \"400c6df4-8db3-4ad7-86fc-0948cee05b23\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 18 14:01:14.788210 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:01:14.788059    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/400c6df4-8db3-4ad7-86fc-0948cee05b23-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"400c6df4-8db3-4ad7-86fc-0948cee05b23\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 18 14:01:14.889216 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:01:14.889190    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/400c6df4-8db3-4ad7-86fc-0948cee05b23-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"400c6df4-8db3-4ad7-86fc-0948cee05b23\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 18 14:01:14.889305 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:01:14.889229    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/400c6df4-8db3-4ad7-86fc-0948cee05b23-tls-assets\") pod \"alertmanager-main-0\" (UID: \"400c6df4-8db3-4ad7-86fc-0948cee05b23\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 18 14:01:14.889305 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:01:14.889246    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ngp5g\" (UniqueName: \"kubernetes.io/projected/400c6df4-8db3-4ad7-86fc-0948cee05b23-kube-api-access-ngp5g\") pod \"alertmanager-main-0\" (UID: \"400c6df4-8db3-4ad7-86fc-0948cee05b23\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 18 14:01:14.889396 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:01:14.889359    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/400c6df4-8db3-4ad7-86fc-0948cee05b23-web-config\") pod \"alertmanager-main-0\" (UID: \"400c6df4-8db3-4ad7-86fc-0948cee05b23\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 18 14:01:14.889446 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:01:14.889416    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/400c6df4-8db3-4ad7-86fc-0948cee05b23-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"400c6df4-8db3-4ad7-86fc-0948cee05b23\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 18 14:01:14.889552 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:01:14.889532    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/400c6df4-8db3-4ad7-86fc-0948cee05b23-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"400c6df4-8db3-4ad7-86fc-0948cee05b23\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 18 14:01:14.889624 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:01:14.889570    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/400c6df4-8db3-4ad7-86fc-0948cee05b23-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"400c6df4-8db3-4ad7-86fc-0948cee05b23\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 18 14:01:14.889624 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:01:14.889607    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/400c6df4-8db3-4ad7-86fc-0948cee05b23-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"400c6df4-8db3-4ad7-86fc-0948cee05b23\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 18 14:01:14.889726 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:01:14.889632    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/400c6df4-8db3-4ad7-86fc-0948cee05b23-config-out\") pod \"alertmanager-main-0\" (UID: \"400c6df4-8db3-4ad7-86fc-0948cee05b23\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 18 14:01:14.889726 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:01:14.889690    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/400c6df4-8db3-4ad7-86fc-0948cee05b23-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"400c6df4-8db3-4ad7-86fc-0948cee05b23\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 18 14:01:14.889828 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:01:14.889722    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/400c6df4-8db3-4ad7-86fc-0948cee05b23-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"400c6df4-8db3-4ad7-86fc-0948cee05b23\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 18 14:01:14.889828 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:01:14.889754    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/400c6df4-8db3-4ad7-86fc-0948cee05b23-config-volume\") pod \"alertmanager-main-0\" (UID: \"400c6df4-8db3-4ad7-86fc-0948cee05b23\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 18 14:01:14.889828 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:01:14.889791    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/400c6df4-8db3-4ad7-86fc-0948cee05b23-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"400c6df4-8db3-4ad7-86fc-0948cee05b23\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 18 14:01:14.890466 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:01:14.890431    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/400c6df4-8db3-4ad7-86fc-0948cee05b23-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"400c6df4-8db3-4ad7-86fc-0948cee05b23\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 18 14:01:14.890742 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:01:14.890717    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/400c6df4-8db3-4ad7-86fc-0948cee05b23-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"400c6df4-8db3-4ad7-86fc-0948cee05b23\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 18 14:01:14.891336 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:01:14.891308    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/400c6df4-8db3-4ad7-86fc-0948cee05b23-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"400c6df4-8db3-4ad7-86fc-0948cee05b23\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 18 14:01:14.892473 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:01:14.892347    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/400c6df4-8db3-4ad7-86fc-0948cee05b23-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"400c6df4-8db3-4ad7-86fc-0948cee05b23\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 18 14:01:14.892473 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:01:14.892401    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/400c6df4-8db3-4ad7-86fc-0948cee05b23-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"400c6df4-8db3-4ad7-86fc-0948cee05b23\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 18 14:01:14.892473 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:01:14.892458    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/400c6df4-8db3-4ad7-86fc-0948cee05b23-web-config\") pod \"alertmanager-main-0\" (UID: \"400c6df4-8db3-4ad7-86fc-0948cee05b23\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 18 14:01:14.892672 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:01:14.892509    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/400c6df4-8db3-4ad7-86fc-0948cee05b23-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"400c6df4-8db3-4ad7-86fc-0948cee05b23\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 18 14:01:14.892672 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:01:14.892561    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/400c6df4-8db3-4ad7-86fc-0948cee05b23-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"400c6df4-8db3-4ad7-86fc-0948cee05b23\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 18 14:01:14.892835 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:01:14.892819    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/400c6df4-8db3-4ad7-86fc-0948cee05b23-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"400c6df4-8db3-4ad7-86fc-0948cee05b23\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 18 14:01:14.893210 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:01:14.893187    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/400c6df4-8db3-4ad7-86fc-0948cee05b23-tls-assets\") pod \"alertmanager-main-0\" (UID: \"400c6df4-8db3-4ad7-86fc-0948cee05b23\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 18 14:01:14.893730 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:01:14.893710    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/400c6df4-8db3-4ad7-86fc-0948cee05b23-config-volume\") pod \"alertmanager-main-0\" (UID: \"400c6df4-8db3-4ad7-86fc-0948cee05b23\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 18 14:01:14.894122 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:01:14.894105    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/400c6df4-8db3-4ad7-86fc-0948cee05b23-config-out\") pod \"alertmanager-main-0\" (UID: \"400c6df4-8db3-4ad7-86fc-0948cee05b23\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 18 14:01:14.898418 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:01:14.898401    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ngp5g\" (UniqueName: \"kubernetes.io/projected/400c6df4-8db3-4ad7-86fc-0948cee05b23-kube-api-access-ngp5g\") pod \"alertmanager-main-0\" (UID: \"400c6df4-8db3-4ad7-86fc-0948cee05b23\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 18 14:01:14.996959 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:01:14.996922    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Sep 18 14:01:15.283490 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:01:15.282722    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Sep 18 14:01:15.619266 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:01:15.619235    2565 generic.go:356] "Generic (PLEG): container finished" podID="400c6df4-8db3-4ad7-86fc-0948cee05b23" containerID="b1dc06ee0e58acb65bfebacf3043b61fd2b001b79d6e3a791a887e89d9fbfa92" exitCode=0
Sep 18 14:01:15.619500 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:01:15.619293    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"400c6df4-8db3-4ad7-86fc-0948cee05b23","Type":"ContainerDied","Data":"b1dc06ee0e58acb65bfebacf3043b61fd2b001b79d6e3a791a887e89d9fbfa92"}
Sep 18 14:01:15.619500 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:01:15.619350    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"400c6df4-8db3-4ad7-86fc-0948cee05b23","Type":"ContainerStarted","Data":"d7210ac88ee553948a356c867d5b292122c97ddbbbb6bd563fd0cba6c2826b4c"}
Sep 18 14:01:15.913073 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:01:15.913026    2565 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ee93bad9-7ad1-4dee-97fb-b16f6e1e1459" path="/var/lib/kubelet/pods/ee93bad9-7ad1-4dee-97fb-b16f6e1e1459/volumes"
Sep 18 14:01:16.626840 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:01:16.626807    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"400c6df4-8db3-4ad7-86fc-0948cee05b23","Type":"ContainerStarted","Data":"54a7763ff5dba7d6823c3285ac46d6274cc5e4498d31598896dfa37a5863ee35"}
Sep 18 14:01:16.627244 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:01:16.626876    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"400c6df4-8db3-4ad7-86fc-0948cee05b23","Type":"ContainerStarted","Data":"f976ad76c8a45276cfbcadbce354c1f71711343551e4c04500e73f235a6e2cf4"}
Sep 18 14:01:16.627244 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:01:16.626906    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"400c6df4-8db3-4ad7-86fc-0948cee05b23","Type":"ContainerStarted","Data":"6a06856c7624d93c75e2e285138c74abee89f1d6a12903fd7b2adf6c03bbf911"}
Sep 18 14:01:16.627244 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:01:16.626934    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"400c6df4-8db3-4ad7-86fc-0948cee05b23","Type":"ContainerStarted","Data":"c0a3afcf813d01d10ebbc7544c327fd070d9cedb4c4e7bea36a0f8208d648c55"}
Sep 18 14:01:16.627244 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:01:16.626959    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"400c6df4-8db3-4ad7-86fc-0948cee05b23","Type":"ContainerStarted","Data":"020cfd44412636eb8ae2fe26fe14ab6e8536d73db6fe29f0f32bf04cfa684c06"}
Sep 18 14:01:16.627244 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:01:16.626983    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"400c6df4-8db3-4ad7-86fc-0948cee05b23","Type":"ContainerStarted","Data":"2c463f699b04bcae15a8c224e61869427343e979f25766786c6b9e05b95fd0ad"}
Sep 18 14:01:16.656216 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:01:16.656167    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/alertmanager-main-0" podStartSLOduration=2.64114598 podStartE2EDuration="2.656151142s" totalImagesPullingTime="0s" totalInitContainerRuntime="15.005162ms" isStatefulPod=false podCreationTimestamp="2026-09-18 14:01:14 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 14:01:16.655132419 +0000 UTC m=+144.829042105" watchObservedRunningTime="2026-09-18 14:01:16.656151142 +0000 UTC m=+144.830060796"
Sep 18 14:01:16.872188 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:01:16.872148    2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/telemeter-client-5c6545f8b5-c8mmm"]
Sep 18 14:01:16.875346 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:01:16.875321    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/telemeter-client-5c6545f8b5-c8mmm"
Sep 18 14:01:16.888015 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:01:16.887993    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-kube-rbac-proxy-config\""
Sep 18 14:01:16.888015 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:01:16.888011    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"federate-client-certs\""
Sep 18 14:01:16.888448 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:01:16.888435    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-dockercfg-ljx6h\""
Sep 18 14:01:16.888611 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:01:16.888601    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-tls\""
Sep 18 14:01:16.890080 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:01:16.890065    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client\""
Sep 18 14:01:16.903341 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:01:16.903325    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemeter-client-serving-certs-ca-bundle\""
Sep 18 14:01:16.909315 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:01:16.909297    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/20618239-3f82-4756-a0ad-d892a1bf13e8-secret-telemeter-client\") pod \"telemeter-client-5c6545f8b5-c8mmm\" (UID: \"20618239-3f82-4756-a0ad-d892a1bf13e8\") " pod="openshift-monitoring/telemeter-client-5c6545f8b5-c8mmm"
Sep 18 14:01:16.909375 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:01:16.909322    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/20618239-3f82-4756-a0ad-d892a1bf13e8-federate-client-tls\") pod \"telemeter-client-5c6545f8b5-c8mmm\" (UID: \"20618239-3f82-4756-a0ad-d892a1bf13e8\") " pod="openshift-monitoring/telemeter-client-5c6545f8b5-c8mmm"
Sep 18 14:01:16.909375 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:01:16.909354    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/20618239-3f82-4756-a0ad-d892a1bf13e8-telemeter-client-tls\") pod \"telemeter-client-5c6545f8b5-c8mmm\" (UID: \"20618239-3f82-4756-a0ad-d892a1bf13e8\") " pod="openshift-monitoring/telemeter-client-5c6545f8b5-c8mmm"
Sep 18 14:01:16.909443 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:01:16.909399    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/20618239-3f82-4756-a0ad-d892a1bf13e8-serving-certs-ca-bundle\") pod \"telemeter-client-5c6545f8b5-c8mmm\" (UID: \"20618239-3f82-4756-a0ad-d892a1bf13e8\") " pod="openshift-monitoring/telemeter-client-5c6545f8b5-c8mmm"
Sep 18 14:01:16.909443 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:01:16.909437    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/20618239-3f82-4756-a0ad-d892a1bf13e8-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-5c6545f8b5-c8mmm\" (UID: \"20618239-3f82-4756-a0ad-d892a1bf13e8\") " pod="openshift-monitoring/telemeter-client-5c6545f8b5-c8mmm"
Sep 18 14:01:16.909503 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:01:16.909477    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/20618239-3f82-4756-a0ad-d892a1bf13e8-telemeter-trusted-ca-bundle\") pod \"telemeter-client-5c6545f8b5-c8mmm\" (UID: \"20618239-3f82-4756-a0ad-d892a1bf13e8\") " pod="openshift-monitoring/telemeter-client-5c6545f8b5-c8mmm"
Sep 18 14:01:16.909535 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:01:16.909527    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/20618239-3f82-4756-a0ad-d892a1bf13e8-metrics-client-ca\") pod \"telemeter-client-5c6545f8b5-c8mmm\" (UID: \"20618239-3f82-4756-a0ad-d892a1bf13e8\") " pod="openshift-monitoring/telemeter-client-5c6545f8b5-c8mmm"
Sep 18 14:01:16.909568 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:01:16.909553    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-84bz9\" (UniqueName: \"kubernetes.io/projected/20618239-3f82-4756-a0ad-d892a1bf13e8-kube-api-access-84bz9\") pod \"telemeter-client-5c6545f8b5-c8mmm\" (UID: \"20618239-3f82-4756-a0ad-d892a1bf13e8\") " pod="openshift-monitoring/telemeter-client-5c6545f8b5-c8mmm"
Sep 18 14:01:16.925945 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:01:16.925921    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/telemeter-client-5c6545f8b5-c8mmm"]
Sep 18 14:01:16.943398 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:01:16.943196    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemeter-trusted-ca-bundle-8i12ta5c71j38\""
Sep 18 14:01:17.010429 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:01:17.010392    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/20618239-3f82-4756-a0ad-d892a1bf13e8-telemeter-trusted-ca-bundle\") pod \"telemeter-client-5c6545f8b5-c8mmm\" (UID: \"20618239-3f82-4756-a0ad-d892a1bf13e8\") " pod="openshift-monitoring/telemeter-client-5c6545f8b5-c8mmm"
Sep 18 14:01:17.010605 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:01:17.010451    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/20618239-3f82-4756-a0ad-d892a1bf13e8-metrics-client-ca\") pod \"telemeter-client-5c6545f8b5-c8mmm\" (UID: \"20618239-3f82-4756-a0ad-d892a1bf13e8\") " pod="openshift-monitoring/telemeter-client-5c6545f8b5-c8mmm"
Sep 18 14:01:17.010605 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:01:17.010515    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-84bz9\" (UniqueName: \"kubernetes.io/projected/20618239-3f82-4756-a0ad-d892a1bf13e8-kube-api-access-84bz9\") pod \"telemeter-client-5c6545f8b5-c8mmm\" (UID: \"20618239-3f82-4756-a0ad-d892a1bf13e8\") " pod="openshift-monitoring/telemeter-client-5c6545f8b5-c8mmm"
Sep 18 14:01:17.010605 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:01:17.010556    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/20618239-3f82-4756-a0ad-d892a1bf13e8-secret-telemeter-client\") pod \"telemeter-client-5c6545f8b5-c8mmm\" (UID: \"20618239-3f82-4756-a0ad-d892a1bf13e8\") " pod="openshift-monitoring/telemeter-client-5c6545f8b5-c8mmm"
Sep 18 14:01:17.010605 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:01:17.010583    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/20618239-3f82-4756-a0ad-d892a1bf13e8-federate-client-tls\") pod \"telemeter-client-5c6545f8b5-c8mmm\" (UID: \"20618239-3f82-4756-a0ad-d892a1bf13e8\") " pod="openshift-monitoring/telemeter-client-5c6545f8b5-c8mmm"
Sep 18 14:01:17.010810 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:01:17.010634    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/20618239-3f82-4756-a0ad-d892a1bf13e8-telemeter-client-tls\") pod \"telemeter-client-5c6545f8b5-c8mmm\" (UID: \"20618239-3f82-4756-a0ad-d892a1bf13e8\") " pod="openshift-monitoring/telemeter-client-5c6545f8b5-c8mmm"
Sep 18 14:01:17.010810 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:01:17.010668    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/20618239-3f82-4756-a0ad-d892a1bf13e8-serving-certs-ca-bundle\") pod \"telemeter-client-5c6545f8b5-c8mmm\" (UID: \"20618239-3f82-4756-a0ad-d892a1bf13e8\") " pod="openshift-monitoring/telemeter-client-5c6545f8b5-c8mmm"
Sep 18 14:01:17.010810 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:01:17.010698    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/20618239-3f82-4756-a0ad-d892a1bf13e8-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-5c6545f8b5-c8mmm\" (UID: \"20618239-3f82-4756-a0ad-d892a1bf13e8\") " pod="openshift-monitoring/telemeter-client-5c6545f8b5-c8mmm"
Sep 18 14:01:17.011476 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:01:17.011392    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/20618239-3f82-4756-a0ad-d892a1bf13e8-telemeter-trusted-ca-bundle\") pod \"telemeter-client-5c6545f8b5-c8mmm\" (UID: \"20618239-3f82-4756-a0ad-d892a1bf13e8\") " pod="openshift-monitoring/telemeter-client-5c6545f8b5-c8mmm"
Sep 18 14:01:17.012087 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:01:17.012063    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/20618239-3f82-4756-a0ad-d892a1bf13e8-metrics-client-ca\") pod \"telemeter-client-5c6545f8b5-c8mmm\" (UID: \"20618239-3f82-4756-a0ad-d892a1bf13e8\") " pod="openshift-monitoring/telemeter-client-5c6545f8b5-c8mmm"
Sep 18 14:01:17.012331 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:01:17.012306    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/20618239-3f82-4756-a0ad-d892a1bf13e8-serving-certs-ca-bundle\") pod \"telemeter-client-5c6545f8b5-c8mmm\" (UID: \"20618239-3f82-4756-a0ad-d892a1bf13e8\") " pod="openshift-monitoring/telemeter-client-5c6545f8b5-c8mmm"
Sep 18 14:01:17.014254 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:01:17.014219    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/20618239-3f82-4756-a0ad-d892a1bf13e8-federate-client-tls\") pod \"telemeter-client-5c6545f8b5-c8mmm\" (UID: \"20618239-3f82-4756-a0ad-d892a1bf13e8\") " pod="openshift-monitoring/telemeter-client-5c6545f8b5-c8mmm"
Sep 18 14:01:17.014356 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:01:17.014277    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/20618239-3f82-4756-a0ad-d892a1bf13e8-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-5c6545f8b5-c8mmm\" (UID: \"20618239-3f82-4756-a0ad-d892a1bf13e8\") " pod="openshift-monitoring/telemeter-client-5c6545f8b5-c8mmm"
Sep 18 14:01:17.014356 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:01:17.014343    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/20618239-3f82-4756-a0ad-d892a1bf13e8-telemeter-client-tls\") pod \"telemeter-client-5c6545f8b5-c8mmm\" (UID: \"20618239-3f82-4756-a0ad-d892a1bf13e8\") " pod="openshift-monitoring/telemeter-client-5c6545f8b5-c8mmm"
Sep 18 14:01:17.014675 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:01:17.014658    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/20618239-3f82-4756-a0ad-d892a1bf13e8-secret-telemeter-client\") pod \"telemeter-client-5c6545f8b5-c8mmm\" (UID: \"20618239-3f82-4756-a0ad-d892a1bf13e8\") " pod="openshift-monitoring/telemeter-client-5c6545f8b5-c8mmm"
Sep 18 14:01:17.021130 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:01:17.021103    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-84bz9\" (UniqueName: \"kubernetes.io/projected/20618239-3f82-4756-a0ad-d892a1bf13e8-kube-api-access-84bz9\") pod \"telemeter-client-5c6545f8b5-c8mmm\" (UID: \"20618239-3f82-4756-a0ad-d892a1bf13e8\") " pod="openshift-monitoring/telemeter-client-5c6545f8b5-c8mmm"
Sep 18 14:01:17.185281 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:01:17.185196    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/telemeter-client-5c6545f8b5-c8mmm"
Sep 18 14:01:17.236447 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:01:17.236238    2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Sep 18 14:01:17.237055 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:01:17.236993    2565 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="1d0d5aec-4917-4b99-a58e-592c2863d81b" containerName="prometheus" containerID="cri-o://7a640029364c17829cc96ab4d5210590ce031338fd16245acbcc8c39c0729c64" gracePeriod=600
Sep 18 14:01:17.237383 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:01:17.237360    2565 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="1d0d5aec-4917-4b99-a58e-592c2863d81b" containerName="kube-rbac-proxy-web" containerID="cri-o://269624523c0629bfe048e9478419b509f305b696807736fd2360b16221cd10fb" gracePeriod=600
Sep 18 14:01:17.237608 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:01:17.237351    2565 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="1d0d5aec-4917-4b99-a58e-592c2863d81b" containerName="kube-rbac-proxy" containerID="cri-o://5670e8703e3d33730e46c486e162dcf3e63ae3c170476bb196fbed419cec7acb" gracePeriod=600
Sep 18 14:01:17.237878 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:01:17.237351    2565 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="1d0d5aec-4917-4b99-a58e-592c2863d81b" containerName="kube-rbac-proxy-thanos" containerID="cri-o://95b5dac5b5886a4314daf23ac8bc5b0b7d0394f08e45cf1e00b44c51f684f442" gracePeriod=600
Sep 18 14:01:17.238061 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:01:17.237412    2565 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="1d0d5aec-4917-4b99-a58e-592c2863d81b" containerName="config-reloader" containerID="cri-o://ed5877be28108db692a5a67e1989ebbbe9f45e5fcdf0bcc114bdd37a251053b1" gracePeriod=600
Sep 18 14:01:17.238249 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:01:17.237428    2565 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="1d0d5aec-4917-4b99-a58e-592c2863d81b" containerName="thanos-sidecar" containerID="cri-o://a065761a0ad0ee010aaad6fdd3c024a40cf2122fe5bceb2f7c66f85220e3e521" gracePeriod=600
Sep 18 14:01:17.488365 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:01:17.488221    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/telemeter-client-5c6545f8b5-c8mmm"]
Sep 18 14:01:17.494296 ip-10-0-129-16 kubenswrapper[2565]: W0918 14:01:17.494256    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod20618239_3f82_4756_a0ad_d892a1bf13e8.slice/crio-0cf1f8a5533fac7caa5357733d8c667a9986cc8a8b9c432077b33e9b0fffec3d WatchSource:0}: Error finding container 0cf1f8a5533fac7caa5357733d8c667a9986cc8a8b9c432077b33e9b0fffec3d: Status 404 returned error can't find the container with id 0cf1f8a5533fac7caa5357733d8c667a9986cc8a8b9c432077b33e9b0fffec3d
Sep 18 14:01:17.634845 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:01:17.634807    2565 generic.go:356] "Generic (PLEG): container finished" podID="1d0d5aec-4917-4b99-a58e-592c2863d81b" containerID="269624523c0629bfe048e9478419b509f305b696807736fd2360b16221cd10fb" exitCode=0
Sep 18 14:01:17.634845 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:01:17.634841    2565 generic.go:356] "Generic (PLEG): container finished" podID="1d0d5aec-4917-4b99-a58e-592c2863d81b" containerID="a065761a0ad0ee010aaad6fdd3c024a40cf2122fe5bceb2f7c66f85220e3e521" exitCode=0
Sep 18 14:01:17.634845 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:01:17.634852    2565 generic.go:356] "Generic (PLEG): container finished" podID="1d0d5aec-4917-4b99-a58e-592c2863d81b" containerID="ed5877be28108db692a5a67e1989ebbbe9f45e5fcdf0bcc114bdd37a251053b1" exitCode=0
Sep 18 14:01:17.635374 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:01:17.634883    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"1d0d5aec-4917-4b99-a58e-592c2863d81b","Type":"ContainerDied","Data":"269624523c0629bfe048e9478419b509f305b696807736fd2360b16221cd10fb"}
Sep 18 14:01:17.635374 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:01:17.634969    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"1d0d5aec-4917-4b99-a58e-592c2863d81b","Type":"ContainerDied","Data":"a065761a0ad0ee010aaad6fdd3c024a40cf2122fe5bceb2f7c66f85220e3e521"}
Sep 18 14:01:17.635374 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:01:17.635002    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"1d0d5aec-4917-4b99-a58e-592c2863d81b","Type":"ContainerDied","Data":"ed5877be28108db692a5a67e1989ebbbe9f45e5fcdf0bcc114bdd37a251053b1"}
Sep 18 14:01:17.636392 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:01:17.636360    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-5c6545f8b5-c8mmm" event={"ID":"20618239-3f82-4756-a0ad-d892a1bf13e8","Type":"ContainerStarted","Data":"0cf1f8a5533fac7caa5357733d8c667a9986cc8a8b9c432077b33e9b0fffec3d"}
Sep 18 14:01:18.647565 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:01:18.647408    2565 generic.go:356] "Generic (PLEG): container finished" podID="1d0d5aec-4917-4b99-a58e-592c2863d81b" containerID="95b5dac5b5886a4314daf23ac8bc5b0b7d0394f08e45cf1e00b44c51f684f442" exitCode=0
Sep 18 14:01:18.647565 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:01:18.647440    2565 generic.go:356] "Generic (PLEG): container finished" podID="1d0d5aec-4917-4b99-a58e-592c2863d81b" containerID="5670e8703e3d33730e46c486e162dcf3e63ae3c170476bb196fbed419cec7acb" exitCode=0
Sep 18 14:01:18.647565 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:01:18.647452    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"1d0d5aec-4917-4b99-a58e-592c2863d81b","Type":"ContainerDied","Data":"95b5dac5b5886a4314daf23ac8bc5b0b7d0394f08e45cf1e00b44c51f684f442"}
Sep 18 14:01:18.647565 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:01:18.647537    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"1d0d5aec-4917-4b99-a58e-592c2863d81b","Type":"ContainerDied","Data":"5670e8703e3d33730e46c486e162dcf3e63ae3c170476bb196fbed419cec7acb"}
Sep 18 14:01:19.549392 ip-10-0-129-16 kubenswrapper[2565]: E0918 14:01:19.549355    2565 log.go:32] "ExecSync cmd from runtime service failed" err="rpc error: code = Unknown desc = command error: cannot register an exec PID: container is stopping, stdout: , stderr: , exit code -1" containerID="7a640029364c17829cc96ab4d5210590ce031338fd16245acbcc8c39c0729c64" cmd=["sh","-c","if [ -x \"$(command -v curl)\" ]; then exec curl --fail http://localhost:9090/-/ready; elif [ -x \"$(command -v wget)\" ]; then exec wget -q -O /dev/null http://localhost:9090/-/ready; else exit 1; fi"]
Sep 18 14:01:19.550298 ip-10-0-129-16 kubenswrapper[2565]: E0918 14:01:19.550268    2565 log.go:32] "ExecSync cmd from runtime service failed" err="rpc error: code = Unknown desc = command error: cannot register an exec PID: container is stopping, stdout: , stderr: , exit code -1" containerID="7a640029364c17829cc96ab4d5210590ce031338fd16245acbcc8c39c0729c64" cmd=["sh","-c","if [ -x \"$(command -v curl)\" ]; then exec curl --fail http://localhost:9090/-/ready; elif [ -x \"$(command -v wget)\" ]; then exec wget -q -O /dev/null http://localhost:9090/-/ready; else exit 1; fi"]
Sep 18 14:01:19.551238 ip-10-0-129-16 kubenswrapper[2565]: E0918 14:01:19.551208    2565 log.go:32] "ExecSync cmd from runtime service failed" err="rpc error: code = Unknown desc = command error: cannot register an exec PID: container is stopping, stdout: , stderr: , exit code -1" containerID="7a640029364c17829cc96ab4d5210590ce031338fd16245acbcc8c39c0729c64" cmd=["sh","-c","if [ -x \"$(command -v curl)\" ]; then exec curl --fail http://localhost:9090/-/ready; elif [ -x \"$(command -v wget)\" ]; then exec wget -q -O /dev/null http://localhost:9090/-/ready; else exit 1; fi"]
Sep 18 14:01:19.551330 ip-10-0-129-16 kubenswrapper[2565]: E0918 14:01:19.551252    2565 prober.go:104] "Probe errored" err="rpc error: code = Unknown desc = command error: cannot register an exec PID: container is stopping, stdout: , stderr: , exit code -1" probeType="Readiness" pod="openshift-monitoring/prometheus-k8s-0" podUID="1d0d5aec-4917-4b99-a58e-592c2863d81b" containerName="prometheus" probeResult="unknown"
Sep 18 14:01:19.654390 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:01:19.654353    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-5c6545f8b5-c8mmm" event={"ID":"20618239-3f82-4756-a0ad-d892a1bf13e8","Type":"ContainerStarted","Data":"e124f85e637749e1a006cab1887a68aac5e96ef9d7093abbb8783ed42d0ae82f"}
Sep 18 14:01:19.654783 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:01:19.654432    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-5c6545f8b5-c8mmm" event={"ID":"20618239-3f82-4756-a0ad-d892a1bf13e8","Type":"ContainerStarted","Data":"d503e3cda86bc3dbe8dbe56431a847413e345af789d8b61d73c663736afa6dde"}
Sep 18 14:01:19.654783 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:01:19.654466    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-5c6545f8b5-c8mmm" event={"ID":"20618239-3f82-4756-a0ad-d892a1bf13e8","Type":"ContainerStarted","Data":"a2165de1fcd7930f4401bb037f5e95d9cb160fc4f5307ef2759c57d0fb26d453"}
Sep 18 14:01:19.680599 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:01:19.680545    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/telemeter-client-5c6545f8b5-c8mmm" podStartSLOduration=1.996422264 podStartE2EDuration="3.68052509s" totalImagesPullingTime="1.684102826s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 14:01:16 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 14:01:19.680081741 +0000 UTC m=+147.853991421" watchObservedRunningTime="2026-09-18 14:01:19.68052509 +0000 UTC m=+147.854434749"
Sep 18 14:01:24.549236 ip-10-0-129-16 kubenswrapper[2565]: E0918 14:01:24.549189    2565 log.go:32] "ExecSync cmd from runtime service failed" err="rpc error: code = Unknown desc = command error: cannot register an exec PID: container is stopping, stdout: , stderr: , exit code -1" containerID="7a640029364c17829cc96ab4d5210590ce031338fd16245acbcc8c39c0729c64" cmd=["sh","-c","if [ -x \"$(command -v curl)\" ]; then exec curl --fail http://localhost:9090/-/ready; elif [ -x \"$(command -v wget)\" ]; then exec wget -q -O /dev/null http://localhost:9090/-/ready; else exit 1; fi"]
Sep 18 14:01:24.550246 ip-10-0-129-16 kubenswrapper[2565]: E0918 14:01:24.550218    2565 log.go:32] "ExecSync cmd from runtime service failed" err="rpc error: code = Unknown desc = command error: cannot register an exec PID: container is stopping, stdout: , stderr: , exit code -1" containerID="7a640029364c17829cc96ab4d5210590ce031338fd16245acbcc8c39c0729c64" cmd=["sh","-c","if [ -x \"$(command -v curl)\" ]; then exec curl --fail http://localhost:9090/-/ready; elif [ -x \"$(command -v wget)\" ]; then exec wget -q -O /dev/null http://localhost:9090/-/ready; else exit 1; fi"]
Sep 18 14:01:24.551109 ip-10-0-129-16 kubenswrapper[2565]: E0918 14:01:24.551073    2565 log.go:32] "ExecSync cmd from runtime service failed" err="rpc error: code = Unknown desc = command error: cannot register an exec PID: container is stopping, stdout: , stderr: , exit code -1" containerID="7a640029364c17829cc96ab4d5210590ce031338fd16245acbcc8c39c0729c64" cmd=["sh","-c","if [ -x \"$(command -v curl)\" ]; then exec curl --fail http://localhost:9090/-/ready; elif [ -x \"$(command -v wget)\" ]; then exec wget -q -O /dev/null http://localhost:9090/-/ready; else exit 1; fi"]
Sep 18 14:01:24.551201 ip-10-0-129-16 kubenswrapper[2565]: E0918 14:01:24.551123    2565 prober.go:104] "Probe errored" err="rpc error: code = Unknown desc = command error: cannot register an exec PID: container is stopping, stdout: , stderr: , exit code -1" probeType="Readiness" pod="openshift-monitoring/prometheus-k8s-0" podUID="1d0d5aec-4917-4b99-a58e-592c2863d81b" containerName="prometheus" probeResult="unknown"
Sep 18 14:01:27.085702 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:01:27.085675    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Sep 18 14:01:27.209090 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:01:27.208980    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/1d0d5aec-4917-4b99-a58e-592c2863d81b-secret-prometheus-k8s-tls\") pod \"1d0d5aec-4917-4b99-a58e-592c2863d81b\" (UID: \"1d0d5aec-4917-4b99-a58e-592c2863d81b\") "
Sep 18 14:01:27.209090 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:01:27.209018    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/1d0d5aec-4917-4b99-a58e-592c2863d81b-prometheus-k8s-db\") pod \"1d0d5aec-4917-4b99-a58e-592c2863d81b\" (UID: \"1d0d5aec-4917-4b99-a58e-592c2863d81b\") "
Sep 18 14:01:27.209090 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:01:27.209071    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/1d0d5aec-4917-4b99-a58e-592c2863d81b-secret-kube-rbac-proxy\") pod \"1d0d5aec-4917-4b99-a58e-592c2863d81b\" (UID: \"1d0d5aec-4917-4b99-a58e-592c2863d81b\") "
Sep 18 14:01:27.209354 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:01:27.209098    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/1d0d5aec-4917-4b99-a58e-592c2863d81b-prometheus-k8s-rulefiles-2\") pod \"1d0d5aec-4917-4b99-a58e-592c2863d81b\" (UID: \"1d0d5aec-4917-4b99-a58e-592c2863d81b\") "
Sep 18 14:01:27.209354 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:01:27.209121    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/1d0d5aec-4917-4b99-a58e-592c2863d81b-secret-grpc-tls\") pod \"1d0d5aec-4917-4b99-a58e-592c2863d81b\" (UID: \"1d0d5aec-4917-4b99-a58e-592c2863d81b\") "
Sep 18 14:01:27.209354 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:01:27.209146    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1d0d5aec-4917-4b99-a58e-592c2863d81b-configmap-kubelet-serving-ca-bundle\") pod \"1d0d5aec-4917-4b99-a58e-592c2863d81b\" (UID: \"1d0d5aec-4917-4b99-a58e-592c2863d81b\") "
Sep 18 14:01:27.209354 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:01:27.209181    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/1d0d5aec-4917-4b99-a58e-592c2863d81b-config\") pod \"1d0d5aec-4917-4b99-a58e-592c2863d81b\" (UID: \"1d0d5aec-4917-4b99-a58e-592c2863d81b\") "
Sep 18 14:01:27.211208 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:01:27.209578    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1d0d5aec-4917-4b99-a58e-592c2863d81b-prometheus-k8s-rulefiles-2" (OuterVolumeSpecName: "prometheus-k8s-rulefiles-2") pod "1d0d5aec-4917-4b99-a58e-592c2863d81b" (UID: "1d0d5aec-4917-4b99-a58e-592c2863d81b"). InnerVolumeSpecName "prometheus-k8s-rulefiles-2". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 18 14:01:27.211208 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:01:27.209621    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1d0d5aec-4917-4b99-a58e-592c2863d81b-configmap-kubelet-serving-ca-bundle" (OuterVolumeSpecName: "configmap-kubelet-serving-ca-bundle") pod "1d0d5aec-4917-4b99-a58e-592c2863d81b" (UID: "1d0d5aec-4917-4b99-a58e-592c2863d81b"). InnerVolumeSpecName "configmap-kubelet-serving-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 18 14:01:27.211208 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:01:27.209633    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/1d0d5aec-4917-4b99-a58e-592c2863d81b-prometheus-k8s-rulefiles-1\") pod \"1d0d5aec-4917-4b99-a58e-592c2863d81b\" (UID: \"1d0d5aec-4917-4b99-a58e-592c2863d81b\") "
Sep 18 14:01:27.211208 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:01:27.209724    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/1d0d5aec-4917-4b99-a58e-592c2863d81b-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"1d0d5aec-4917-4b99-a58e-592c2863d81b\" (UID: \"1d0d5aec-4917-4b99-a58e-592c2863d81b\") "
Sep 18 14:01:27.211208 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:01:27.209752    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/1d0d5aec-4917-4b99-a58e-592c2863d81b-thanos-prometheus-http-client-file\") pod \"1d0d5aec-4917-4b99-a58e-592c2863d81b\" (UID: \"1d0d5aec-4917-4b99-a58e-592c2863d81b\") "
Sep 18 14:01:27.211208 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:01:27.209783    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/1d0d5aec-4917-4b99-a58e-592c2863d81b-prometheus-k8s-rulefiles-0\") pod \"1d0d5aec-4917-4b99-a58e-592c2863d81b\" (UID: \"1d0d5aec-4917-4b99-a58e-592c2863d81b\") "
Sep 18 14:01:27.211208 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:01:27.209959    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1d0d5aec-4917-4b99-a58e-592c2863d81b-prometheus-k8s-rulefiles-1" (OuterVolumeSpecName: "prometheus-k8s-rulefiles-1") pod "1d0d5aec-4917-4b99-a58e-592c2863d81b" (UID: "1d0d5aec-4917-4b99-a58e-592c2863d81b"). InnerVolumeSpecName "prometheus-k8s-rulefiles-1". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 18 14:01:27.211208 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:01:27.210009    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/1d0d5aec-4917-4b99-a58e-592c2863d81b-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"1d0d5aec-4917-4b99-a58e-592c2863d81b\" (UID: \"1d0d5aec-4917-4b99-a58e-592c2863d81b\") "
Sep 18 14:01:27.211208 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:01:27.210087    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/1d0d5aec-4917-4b99-a58e-592c2863d81b-tls-assets\") pod \"1d0d5aec-4917-4b99-a58e-592c2863d81b\" (UID: \"1d0d5aec-4917-4b99-a58e-592c2863d81b\") "
Sep 18 14:01:27.211208 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:01:27.210115    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/1d0d5aec-4917-4b99-a58e-592c2863d81b-web-config\") pod \"1d0d5aec-4917-4b99-a58e-592c2863d81b\" (UID: \"1d0d5aec-4917-4b99-a58e-592c2863d81b\") "
Sep 18 14:01:27.211208 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:01:27.210324    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1d0d5aec-4917-4b99-a58e-592c2863d81b-prometheus-k8s-db" (OuterVolumeSpecName: "prometheus-k8s-db") pod "1d0d5aec-4917-4b99-a58e-592c2863d81b" (UID: "1d0d5aec-4917-4b99-a58e-592c2863d81b"). InnerVolumeSpecName "prometheus-k8s-db". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 18 14:01:27.211208 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:01:27.210400    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/1d0d5aec-4917-4b99-a58e-592c2863d81b-secret-metrics-client-certs\") pod \"1d0d5aec-4917-4b99-a58e-592c2863d81b\" (UID: \"1d0d5aec-4917-4b99-a58e-592c2863d81b\") "
Sep 18 14:01:27.211208 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:01:27.210449    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/1d0d5aec-4917-4b99-a58e-592c2863d81b-config-out\") pod \"1d0d5aec-4917-4b99-a58e-592c2863d81b\" (UID: \"1d0d5aec-4917-4b99-a58e-592c2863d81b\") "
Sep 18 14:01:27.211208 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:01:27.210476    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1d0d5aec-4917-4b99-a58e-592c2863d81b-prometheus-trusted-ca-bundle\") pod \"1d0d5aec-4917-4b99-a58e-592c2863d81b\" (UID: \"1d0d5aec-4917-4b99-a58e-592c2863d81b\") "
Sep 18 14:01:27.211208 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:01:27.210516    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1d0d5aec-4917-4b99-a58e-592c2863d81b-configmap-serving-certs-ca-bundle\") pod \"1d0d5aec-4917-4b99-a58e-592c2863d81b\" (UID: \"1d0d5aec-4917-4b99-a58e-592c2863d81b\") "
Sep 18 14:01:27.211208 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:01:27.210541    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/1d0d5aec-4917-4b99-a58e-592c2863d81b-configmap-metrics-client-ca\") pod \"1d0d5aec-4917-4b99-a58e-592c2863d81b\" (UID: \"1d0d5aec-4917-4b99-a58e-592c2863d81b\") "
Sep 18 14:01:27.211208 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:01:27.210579    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-ccqdb\" (UniqueName: \"kubernetes.io/projected/1d0d5aec-4917-4b99-a58e-592c2863d81b-kube-api-access-ccqdb\") pod \"1d0d5aec-4917-4b99-a58e-592c2863d81b\" (UID: \"1d0d5aec-4917-4b99-a58e-592c2863d81b\") "
Sep 18 14:01:27.212175 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:01:27.210840    2565 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/1d0d5aec-4917-4b99-a58e-592c2863d81b-prometheus-k8s-db\") on node \"ip-10-0-129-16.ec2.internal\" DevicePath \"\""
Sep 18 14:01:27.212175 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:01:27.210860    2565 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/1d0d5aec-4917-4b99-a58e-592c2863d81b-prometheus-k8s-rulefiles-2\") on node \"ip-10-0-129-16.ec2.internal\" DevicePath \"\""
Sep 18 14:01:27.212175 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:01:27.210875    2565 reconciler_common.go:299] "Volume detached for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1d0d5aec-4917-4b99-a58e-592c2863d81b-configmap-kubelet-serving-ca-bundle\") on node \"ip-10-0-129-16.ec2.internal\" DevicePath \"\""
Sep 18 14:01:27.212175 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:01:27.210890    2565 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/1d0d5aec-4917-4b99-a58e-592c2863d81b-prometheus-k8s-rulefiles-1\") on node \"ip-10-0-129-16.ec2.internal\" DevicePath \"\""
Sep 18 14:01:27.212175 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:01:27.211389    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1d0d5aec-4917-4b99-a58e-592c2863d81b-prometheus-k8s-rulefiles-0" (OuterVolumeSpecName: "prometheus-k8s-rulefiles-0") pod "1d0d5aec-4917-4b99-a58e-592c2863d81b" (UID: "1d0d5aec-4917-4b99-a58e-592c2863d81b"). InnerVolumeSpecName "prometheus-k8s-rulefiles-0". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 18 14:01:27.212647 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:01:27.212618    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1d0d5aec-4917-4b99-a58e-592c2863d81b-configmap-serving-certs-ca-bundle" (OuterVolumeSpecName: "configmap-serving-certs-ca-bundle") pod "1d0d5aec-4917-4b99-a58e-592c2863d81b" (UID: "1d0d5aec-4917-4b99-a58e-592c2863d81b"). InnerVolumeSpecName "configmap-serving-certs-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 18 14:01:27.213003 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:01:27.212977    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1d0d5aec-4917-4b99-a58e-592c2863d81b-configmap-metrics-client-ca" (OuterVolumeSpecName: "configmap-metrics-client-ca") pod "1d0d5aec-4917-4b99-a58e-592c2863d81b" (UID: "1d0d5aec-4917-4b99-a58e-592c2863d81b"). InnerVolumeSpecName "configmap-metrics-client-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 18 14:01:27.213568 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:01:27.213359    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1d0d5aec-4917-4b99-a58e-592c2863d81b-secret-grpc-tls" (OuterVolumeSpecName: "secret-grpc-tls") pod "1d0d5aec-4917-4b99-a58e-592c2863d81b" (UID: "1d0d5aec-4917-4b99-a58e-592c2863d81b"). InnerVolumeSpecName "secret-grpc-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 18 14:01:27.213568 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:01:27.213410    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1d0d5aec-4917-4b99-a58e-592c2863d81b-prometheus-trusted-ca-bundle" (OuterVolumeSpecName: "prometheus-trusted-ca-bundle") pod "1d0d5aec-4917-4b99-a58e-592c2863d81b" (UID: "1d0d5aec-4917-4b99-a58e-592c2863d81b"). InnerVolumeSpecName "prometheus-trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 18 14:01:27.213568 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:01:27.213535    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1d0d5aec-4917-4b99-a58e-592c2863d81b-secret-kube-rbac-proxy" (OuterVolumeSpecName: "secret-kube-rbac-proxy") pod "1d0d5aec-4917-4b99-a58e-592c2863d81b" (UID: "1d0d5aec-4917-4b99-a58e-592c2863d81b"). InnerVolumeSpecName "secret-kube-rbac-proxy". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 18 14:01:27.213933 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:01:27.213696    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1d0d5aec-4917-4b99-a58e-592c2863d81b-config" (OuterVolumeSpecName: "config") pod "1d0d5aec-4917-4b99-a58e-592c2863d81b" (UID: "1d0d5aec-4917-4b99-a58e-592c2863d81b"). InnerVolumeSpecName "config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 18 14:01:27.214169 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:01:27.213991    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1d0d5aec-4917-4b99-a58e-592c2863d81b-secret-prometheus-k8s-thanos-sidecar-tls" (OuterVolumeSpecName: "secret-prometheus-k8s-thanos-sidecar-tls") pod "1d0d5aec-4917-4b99-a58e-592c2863d81b" (UID: "1d0d5aec-4917-4b99-a58e-592c2863d81b"). InnerVolumeSpecName "secret-prometheus-k8s-thanos-sidecar-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 18 14:01:27.214169 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:01:27.214089    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1d0d5aec-4917-4b99-a58e-592c2863d81b-secret-prometheus-k8s-kube-rbac-proxy-web" (OuterVolumeSpecName: "secret-prometheus-k8s-kube-rbac-proxy-web") pod "1d0d5aec-4917-4b99-a58e-592c2863d81b" (UID: "1d0d5aec-4917-4b99-a58e-592c2863d81b"). InnerVolumeSpecName "secret-prometheus-k8s-kube-rbac-proxy-web". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 18 14:01:27.214401 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:01:27.214369    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1d0d5aec-4917-4b99-a58e-592c2863d81b-secret-prometheus-k8s-tls" (OuterVolumeSpecName: "secret-prometheus-k8s-tls") pod "1d0d5aec-4917-4b99-a58e-592c2863d81b" (UID: "1d0d5aec-4917-4b99-a58e-592c2863d81b"). InnerVolumeSpecName "secret-prometheus-k8s-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 18 14:01:27.214599 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:01:27.214579    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1d0d5aec-4917-4b99-a58e-592c2863d81b-secret-metrics-client-certs" (OuterVolumeSpecName: "secret-metrics-client-certs") pod "1d0d5aec-4917-4b99-a58e-592c2863d81b" (UID: "1d0d5aec-4917-4b99-a58e-592c2863d81b"). InnerVolumeSpecName "secret-metrics-client-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 18 14:01:27.214824 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:01:27.214802    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1d0d5aec-4917-4b99-a58e-592c2863d81b-tls-assets" (OuterVolumeSpecName: "tls-assets") pod "1d0d5aec-4917-4b99-a58e-592c2863d81b" (UID: "1d0d5aec-4917-4b99-a58e-592c2863d81b"). InnerVolumeSpecName "tls-assets". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 18 14:01:27.214951 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:01:27.214917    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1d0d5aec-4917-4b99-a58e-592c2863d81b-config-out" (OuterVolumeSpecName: "config-out") pod "1d0d5aec-4917-4b99-a58e-592c2863d81b" (UID: "1d0d5aec-4917-4b99-a58e-592c2863d81b"). InnerVolumeSpecName "config-out". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 18 14:01:27.215106 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:01:27.214951    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1d0d5aec-4917-4b99-a58e-592c2863d81b-kube-api-access-ccqdb" (OuterVolumeSpecName: "kube-api-access-ccqdb") pod "1d0d5aec-4917-4b99-a58e-592c2863d81b" (UID: "1d0d5aec-4917-4b99-a58e-592c2863d81b"). InnerVolumeSpecName "kube-api-access-ccqdb". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 18 14:01:27.215106 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:01:27.215053    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1d0d5aec-4917-4b99-a58e-592c2863d81b-thanos-prometheus-http-client-file" (OuterVolumeSpecName: "thanos-prometheus-http-client-file") pod "1d0d5aec-4917-4b99-a58e-592c2863d81b" (UID: "1d0d5aec-4917-4b99-a58e-592c2863d81b"). InnerVolumeSpecName "thanos-prometheus-http-client-file". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 18 14:01:27.224913 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:01:27.224888    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1d0d5aec-4917-4b99-a58e-592c2863d81b-web-config" (OuterVolumeSpecName: "web-config") pod "1d0d5aec-4917-4b99-a58e-592c2863d81b" (UID: "1d0d5aec-4917-4b99-a58e-592c2863d81b"). InnerVolumeSpecName "web-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 18 14:01:27.311961 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:01:27.311926    2565 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/1d0d5aec-4917-4b99-a58e-592c2863d81b-secret-prometheus-k8s-thanos-sidecar-tls\") on node \"ip-10-0-129-16.ec2.internal\" DevicePath \"\""
Sep 18 14:01:27.311961 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:01:27.311961    2565 reconciler_common.go:299] "Volume detached for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/1d0d5aec-4917-4b99-a58e-592c2863d81b-thanos-prometheus-http-client-file\") on node \"ip-10-0-129-16.ec2.internal\" DevicePath \"\""
Sep 18 14:01:27.312276 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:01:27.311973    2565 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/1d0d5aec-4917-4b99-a58e-592c2863d81b-prometheus-k8s-rulefiles-0\") on node \"ip-10-0-129-16.ec2.internal\" DevicePath \"\""
Sep 18 14:01:27.312276 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:01:27.311987    2565 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/1d0d5aec-4917-4b99-a58e-592c2863d81b-secret-prometheus-k8s-kube-rbac-proxy-web\") on node \"ip-10-0-129-16.ec2.internal\" DevicePath \"\""
Sep 18 14:01:27.312276 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:01:27.312000    2565 reconciler_common.go:299] "Volume detached for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/1d0d5aec-4917-4b99-a58e-592c2863d81b-tls-assets\") on node \"ip-10-0-129-16.ec2.internal\" DevicePath \"\""
Sep 18 14:01:27.312276 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:01:27.312012    2565 reconciler_common.go:299] "Volume detached for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/1d0d5aec-4917-4b99-a58e-592c2863d81b-web-config\") on node \"ip-10-0-129-16.ec2.internal\" DevicePath \"\""
Sep 18 14:01:27.312276 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:01:27.312023    2565 reconciler_common.go:299] "Volume detached for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/1d0d5aec-4917-4b99-a58e-592c2863d81b-secret-metrics-client-certs\") on node \"ip-10-0-129-16.ec2.internal\" DevicePath \"\""
Sep 18 14:01:27.312276 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:01:27.312057    2565 reconciler_common.go:299] "Volume detached for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/1d0d5aec-4917-4b99-a58e-592c2863d81b-config-out\") on node \"ip-10-0-129-16.ec2.internal\" DevicePath \"\""
Sep 18 14:01:27.312276 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:01:27.312069    2565 reconciler_common.go:299] "Volume detached for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1d0d5aec-4917-4b99-a58e-592c2863d81b-prometheus-trusted-ca-bundle\") on node \"ip-10-0-129-16.ec2.internal\" DevicePath \"\""
Sep 18 14:01:27.312276 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:01:27.312081    2565 reconciler_common.go:299] "Volume detached for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1d0d5aec-4917-4b99-a58e-592c2863d81b-configmap-serving-certs-ca-bundle\") on node \"ip-10-0-129-16.ec2.internal\" DevicePath \"\""
Sep 18 14:01:27.312276 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:01:27.312093    2565 reconciler_common.go:299] "Volume detached for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/1d0d5aec-4917-4b99-a58e-592c2863d81b-configmap-metrics-client-ca\") on node \"ip-10-0-129-16.ec2.internal\" DevicePath \"\""
Sep 18 14:01:27.312276 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:01:27.312104    2565 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ccqdb\" (UniqueName: \"kubernetes.io/projected/1d0d5aec-4917-4b99-a58e-592c2863d81b-kube-api-access-ccqdb\") on node \"ip-10-0-129-16.ec2.internal\" DevicePath \"\""
Sep 18 14:01:27.312276 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:01:27.312116    2565 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/1d0d5aec-4917-4b99-a58e-592c2863d81b-secret-prometheus-k8s-tls\") on node \"ip-10-0-129-16.ec2.internal\" DevicePath \"\""
Sep 18 14:01:27.312276 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:01:27.312128    2565 reconciler_common.go:299] "Volume detached for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/1d0d5aec-4917-4b99-a58e-592c2863d81b-secret-kube-rbac-proxy\") on node \"ip-10-0-129-16.ec2.internal\" DevicePath \"\""
Sep 18 14:01:27.312276 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:01:27.312140    2565 reconciler_common.go:299] "Volume detached for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/1d0d5aec-4917-4b99-a58e-592c2863d81b-secret-grpc-tls\") on node \"ip-10-0-129-16.ec2.internal\" DevicePath \"\""
Sep 18 14:01:27.312276 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:01:27.312151    2565 reconciler_common.go:299] "Volume detached for volume \"config\" (UniqueName: \"kubernetes.io/secret/1d0d5aec-4917-4b99-a58e-592c2863d81b-config\") on node \"ip-10-0-129-16.ec2.internal\" DevicePath \"\""
Sep 18 14:01:27.696782 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:01:27.696747    2565 generic.go:356] "Generic (PLEG): container finished" podID="1d0d5aec-4917-4b99-a58e-592c2863d81b" containerID="7a640029364c17829cc96ab4d5210590ce031338fd16245acbcc8c39c0729c64" exitCode=0
Sep 18 14:01:27.697023 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:01:27.696816    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Sep 18 14:01:27.697023 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:01:27.696867    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"1d0d5aec-4917-4b99-a58e-592c2863d81b","Type":"ContainerDied","Data":"7a640029364c17829cc96ab4d5210590ce031338fd16245acbcc8c39c0729c64"}
Sep 18 14:01:27.697023 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:01:27.696961    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"1d0d5aec-4917-4b99-a58e-592c2863d81b","Type":"ContainerDied","Data":"728a437efb3a9040b4513b80b8684937406827d98d6dd1f09e4fb751c98ab019"}
Sep 18 14:01:27.697023 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:01:27.696996    2565 scope.go:117] "RemoveContainer" containerID="95b5dac5b5886a4314daf23ac8bc5b0b7d0394f08e45cf1e00b44c51f684f442"
Sep 18 14:01:27.705235 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:01:27.705219    2565 scope.go:117] "RemoveContainer" containerID="5670e8703e3d33730e46c486e162dcf3e63ae3c170476bb196fbed419cec7acb"
Sep 18 14:01:27.712125 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:01:27.712110    2565 scope.go:117] "RemoveContainer" containerID="269624523c0629bfe048e9478419b509f305b696807736fd2360b16221cd10fb"
Sep 18 14:01:27.718672 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:01:27.718652    2565 scope.go:117] "RemoveContainer" containerID="a065761a0ad0ee010aaad6fdd3c024a40cf2122fe5bceb2f7c66f85220e3e521"
Sep 18 14:01:27.721617 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:01:27.721595    2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Sep 18 14:01:27.725488 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:01:27.725467    2565 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Sep 18 14:01:27.726704 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:01:27.726689    2565 scope.go:117] "RemoveContainer" containerID="ed5877be28108db692a5a67e1989ebbbe9f45e5fcdf0bcc114bdd37a251053b1"
Sep 18 14:01:27.733715 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:01:27.733694    2565 scope.go:117] "RemoveContainer" containerID="7a640029364c17829cc96ab4d5210590ce031338fd16245acbcc8c39c0729c64"
Sep 18 14:01:27.740747 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:01:27.740732    2565 scope.go:117] "RemoveContainer" containerID="afe8246960b14fecc377cb32454ce7ea7b78e02f2a486f7e64561689c2fa4068"
Sep 18 14:01:27.747512 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:01:27.747497    2565 scope.go:117] "RemoveContainer" containerID="95b5dac5b5886a4314daf23ac8bc5b0b7d0394f08e45cf1e00b44c51f684f442"
Sep 18 14:01:27.747779 ip-10-0-129-16 kubenswrapper[2565]: E0918 14:01:27.747759    2565 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"95b5dac5b5886a4314daf23ac8bc5b0b7d0394f08e45cf1e00b44c51f684f442\": container with ID starting with 95b5dac5b5886a4314daf23ac8bc5b0b7d0394f08e45cf1e00b44c51f684f442 not found: ID does not exist" containerID="95b5dac5b5886a4314daf23ac8bc5b0b7d0394f08e45cf1e00b44c51f684f442"
Sep 18 14:01:27.747827 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:01:27.747788    2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"95b5dac5b5886a4314daf23ac8bc5b0b7d0394f08e45cf1e00b44c51f684f442"} err="failed to get container status \"95b5dac5b5886a4314daf23ac8bc5b0b7d0394f08e45cf1e00b44c51f684f442\": rpc error: code = NotFound desc = could not find container \"95b5dac5b5886a4314daf23ac8bc5b0b7d0394f08e45cf1e00b44c51f684f442\": container with ID starting with 95b5dac5b5886a4314daf23ac8bc5b0b7d0394f08e45cf1e00b44c51f684f442 not found: ID does not exist"
Sep 18 14:01:27.747827 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:01:27.747808    2565 scope.go:117] "RemoveContainer" containerID="5670e8703e3d33730e46c486e162dcf3e63ae3c170476bb196fbed419cec7acb"
Sep 18 14:01:27.748046 ip-10-0-129-16 kubenswrapper[2565]: E0918 14:01:27.748013    2565 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5670e8703e3d33730e46c486e162dcf3e63ae3c170476bb196fbed419cec7acb\": container with ID starting with 5670e8703e3d33730e46c486e162dcf3e63ae3c170476bb196fbed419cec7acb not found: ID does not exist" containerID="5670e8703e3d33730e46c486e162dcf3e63ae3c170476bb196fbed419cec7acb"
Sep 18 14:01:27.748085 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:01:27.748053    2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5670e8703e3d33730e46c486e162dcf3e63ae3c170476bb196fbed419cec7acb"} err="failed to get container status \"5670e8703e3d33730e46c486e162dcf3e63ae3c170476bb196fbed419cec7acb\": rpc error: code = NotFound desc = could not find container \"5670e8703e3d33730e46c486e162dcf3e63ae3c170476bb196fbed419cec7acb\": container with ID starting with 5670e8703e3d33730e46c486e162dcf3e63ae3c170476bb196fbed419cec7acb not found: ID does not exist"
Sep 18 14:01:27.748085 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:01:27.748068    2565 scope.go:117] "RemoveContainer" containerID="269624523c0629bfe048e9478419b509f305b696807736fd2360b16221cd10fb"
Sep 18 14:01:27.748320 ip-10-0-129-16 kubenswrapper[2565]: E0918 14:01:27.748304    2565 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"269624523c0629bfe048e9478419b509f305b696807736fd2360b16221cd10fb\": container with ID starting with 269624523c0629bfe048e9478419b509f305b696807736fd2360b16221cd10fb not found: ID does not exist" containerID="269624523c0629bfe048e9478419b509f305b696807736fd2360b16221cd10fb"
Sep 18 14:01:27.748357 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:01:27.748325    2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"269624523c0629bfe048e9478419b509f305b696807736fd2360b16221cd10fb"} err="failed to get container status \"269624523c0629bfe048e9478419b509f305b696807736fd2360b16221cd10fb\": rpc error: code = NotFound desc = could not find container \"269624523c0629bfe048e9478419b509f305b696807736fd2360b16221cd10fb\": container with ID starting with 269624523c0629bfe048e9478419b509f305b696807736fd2360b16221cd10fb not found: ID does not exist"
Sep 18 14:01:27.748357 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:01:27.748338    2565 scope.go:117] "RemoveContainer" containerID="a065761a0ad0ee010aaad6fdd3c024a40cf2122fe5bceb2f7c66f85220e3e521"
Sep 18 14:01:27.748547 ip-10-0-129-16 kubenswrapper[2565]: E0918 14:01:27.748532    2565 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a065761a0ad0ee010aaad6fdd3c024a40cf2122fe5bceb2f7c66f85220e3e521\": container with ID starting with a065761a0ad0ee010aaad6fdd3c024a40cf2122fe5bceb2f7c66f85220e3e521 not found: ID does not exist" containerID="a065761a0ad0ee010aaad6fdd3c024a40cf2122fe5bceb2f7c66f85220e3e521"
Sep 18 14:01:27.748597 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:01:27.748551    2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a065761a0ad0ee010aaad6fdd3c024a40cf2122fe5bceb2f7c66f85220e3e521"} err="failed to get container status \"a065761a0ad0ee010aaad6fdd3c024a40cf2122fe5bceb2f7c66f85220e3e521\": rpc error: code = NotFound desc = could not find container \"a065761a0ad0ee010aaad6fdd3c024a40cf2122fe5bceb2f7c66f85220e3e521\": container with ID starting with a065761a0ad0ee010aaad6fdd3c024a40cf2122fe5bceb2f7c66f85220e3e521 not found: ID does not exist"
Sep 18 14:01:27.748597 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:01:27.748564    2565 scope.go:117] "RemoveContainer" containerID="ed5877be28108db692a5a67e1989ebbbe9f45e5fcdf0bcc114bdd37a251053b1"
Sep 18 14:01:27.748841 ip-10-0-129-16 kubenswrapper[2565]: E0918 14:01:27.748820    2565 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ed5877be28108db692a5a67e1989ebbbe9f45e5fcdf0bcc114bdd37a251053b1\": container with ID starting with ed5877be28108db692a5a67e1989ebbbe9f45e5fcdf0bcc114bdd37a251053b1 not found: ID does not exist" containerID="ed5877be28108db692a5a67e1989ebbbe9f45e5fcdf0bcc114bdd37a251053b1"
Sep 18 14:01:27.748913 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:01:27.748844    2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ed5877be28108db692a5a67e1989ebbbe9f45e5fcdf0bcc114bdd37a251053b1"} err="failed to get container status \"ed5877be28108db692a5a67e1989ebbbe9f45e5fcdf0bcc114bdd37a251053b1\": rpc error: code = NotFound desc = could not find container \"ed5877be28108db692a5a67e1989ebbbe9f45e5fcdf0bcc114bdd37a251053b1\": container with ID starting with ed5877be28108db692a5a67e1989ebbbe9f45e5fcdf0bcc114bdd37a251053b1 not found: ID does not exist"
Sep 18 14:01:27.748913 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:01:27.748861    2565 scope.go:117] "RemoveContainer" containerID="7a640029364c17829cc96ab4d5210590ce031338fd16245acbcc8c39c0729c64"
Sep 18 14:01:27.749134 ip-10-0-129-16 kubenswrapper[2565]: E0918 14:01:27.749117    2565 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7a640029364c17829cc96ab4d5210590ce031338fd16245acbcc8c39c0729c64\": container with ID starting with 7a640029364c17829cc96ab4d5210590ce031338fd16245acbcc8c39c0729c64 not found: ID does not exist" containerID="7a640029364c17829cc96ab4d5210590ce031338fd16245acbcc8c39c0729c64"
Sep 18 14:01:27.749194 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:01:27.749140    2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7a640029364c17829cc96ab4d5210590ce031338fd16245acbcc8c39c0729c64"} err="failed to get container status \"7a640029364c17829cc96ab4d5210590ce031338fd16245acbcc8c39c0729c64\": rpc error: code = NotFound desc = could not find container \"7a640029364c17829cc96ab4d5210590ce031338fd16245acbcc8c39c0729c64\": container with ID starting with 7a640029364c17829cc96ab4d5210590ce031338fd16245acbcc8c39c0729c64 not found: ID does not exist"
Sep 18 14:01:27.749194 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:01:27.749155    2565 scope.go:117] "RemoveContainer" containerID="afe8246960b14fecc377cb32454ce7ea7b78e02f2a486f7e64561689c2fa4068"
Sep 18 14:01:27.749398 ip-10-0-129-16 kubenswrapper[2565]: E0918 14:01:27.749384    2565 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"afe8246960b14fecc377cb32454ce7ea7b78e02f2a486f7e64561689c2fa4068\": container with ID starting with afe8246960b14fecc377cb32454ce7ea7b78e02f2a486f7e64561689c2fa4068 not found: ID does not exist" containerID="afe8246960b14fecc377cb32454ce7ea7b78e02f2a486f7e64561689c2fa4068"
Sep 18 14:01:27.749439 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:01:27.749401    2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"afe8246960b14fecc377cb32454ce7ea7b78e02f2a486f7e64561689c2fa4068"} err="failed to get container status \"afe8246960b14fecc377cb32454ce7ea7b78e02f2a486f7e64561689c2fa4068\": rpc error: code = NotFound desc = could not find container \"afe8246960b14fecc377cb32454ce7ea7b78e02f2a486f7e64561689c2fa4068\": container with ID starting with afe8246960b14fecc377cb32454ce7ea7b78e02f2a486f7e64561689c2fa4068 not found: ID does not exist"
Sep 18 14:01:27.754899 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:01:27.754880    2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Sep 18 14:01:27.755203 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:01:27.755190    2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1d0d5aec-4917-4b99-a58e-592c2863d81b" containerName="kube-rbac-proxy-web"
Sep 18 14:01:27.755245 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:01:27.755204    2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="1d0d5aec-4917-4b99-a58e-592c2863d81b" containerName="kube-rbac-proxy-web"
Sep 18 14:01:27.755245 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:01:27.755216    2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1d0d5aec-4917-4b99-a58e-592c2863d81b" containerName="prometheus"
Sep 18 14:01:27.755245 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:01:27.755223    2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="1d0d5aec-4917-4b99-a58e-592c2863d81b" containerName="prometheus"
Sep 18 14:01:27.755245 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:01:27.755242    2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1d0d5aec-4917-4b99-a58e-592c2863d81b" containerName="init-config-reloader"
Sep 18 14:01:27.755349 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:01:27.755247    2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="1d0d5aec-4917-4b99-a58e-592c2863d81b" containerName="init-config-reloader"
Sep 18 14:01:27.755349 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:01:27.755254    2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1d0d5aec-4917-4b99-a58e-592c2863d81b" containerName="kube-rbac-proxy-thanos"
Sep 18 14:01:27.755349 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:01:27.755259    2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="1d0d5aec-4917-4b99-a58e-592c2863d81b" containerName="kube-rbac-proxy-thanos"
Sep 18 14:01:27.755349 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:01:27.755267    2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1d0d5aec-4917-4b99-a58e-592c2863d81b" containerName="thanos-sidecar"
Sep 18 14:01:27.755349 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:01:27.755271    2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="1d0d5aec-4917-4b99-a58e-592c2863d81b" containerName="thanos-sidecar"
Sep 18 14:01:27.755349 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:01:27.755278    2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1d0d5aec-4917-4b99-a58e-592c2863d81b" containerName="config-reloader"
Sep 18 14:01:27.755349 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:01:27.755284    2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="1d0d5aec-4917-4b99-a58e-592c2863d81b" containerName="config-reloader"
Sep 18 14:01:27.755349 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:01:27.755289    2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1d0d5aec-4917-4b99-a58e-592c2863d81b" containerName="kube-rbac-proxy"
Sep 18 14:01:27.755349 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:01:27.755294    2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="1d0d5aec-4917-4b99-a58e-592c2863d81b" containerName="kube-rbac-proxy"
Sep 18 14:01:27.755349 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:01:27.755339    2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="1d0d5aec-4917-4b99-a58e-592c2863d81b" containerName="kube-rbac-proxy"
Sep 18 14:01:27.755349 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:01:27.755349    2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="1d0d5aec-4917-4b99-a58e-592c2863d81b" containerName="kube-rbac-proxy-web"
Sep 18 14:01:27.755649 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:01:27.755355    2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="1d0d5aec-4917-4b99-a58e-592c2863d81b" containerName="thanos-sidecar"
Sep 18 14:01:27.755649 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:01:27.755364    2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="1d0d5aec-4917-4b99-a58e-592c2863d81b" containerName="config-reloader"
Sep 18 14:01:27.755649 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:01:27.755368    2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="1d0d5aec-4917-4b99-a58e-592c2863d81b" containerName="prometheus"
Sep 18 14:01:27.755649 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:01:27.755373    2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="1d0d5aec-4917-4b99-a58e-592c2863d81b" containerName="kube-rbac-proxy-thanos"
Sep 18 14:01:27.758799 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:01:27.758782    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Sep 18 14:01:27.761881 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:01:27.761864    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"serving-certs-ca-bundle\""
Sep 18 14:01:27.761971 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:01:27.761906    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-client-certs\""
Sep 18 14:01:27.762882 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:01:27.762862    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-web-config\""
Sep 18 14:01:27.763003 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:01:27.762895    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-2\""
Sep 18 14:01:27.763003 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:01:27.762899    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-rbac-proxy\""
Sep 18 14:01:27.763136 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:01:27.763003    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-kube-rbac-proxy-web\""
Sep 18 14:01:27.763199 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:01:27.763185    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-dockercfg-h5lqx\""
Sep 18 14:01:27.763253 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:01:27.763193    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-sidecar-tls\""
Sep 18 14:01:27.764387 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:01:27.764374    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls-assets-0\""
Sep 18 14:01:27.765001 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:01:27.764798    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-grpc-tls-hnmh6p546cld\""
Sep 18 14:01:27.765106 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:01:27.765004    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-prometheus-http-client-file\""
Sep 18 14:01:27.765210 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:01:27.765196    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-1\""
Sep 18 14:01:27.765953 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:01:27.765870    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s\""
Sep 18 14:01:27.766395 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:01:27.766374    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls\""
Sep 18 14:01:27.767977 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:01:27.767955    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-0\""
Sep 18 14:01:27.771308 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:01:27.771290    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-trusted-ca-bundle\""
Sep 18 14:01:27.778722 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:01:27.778701    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Sep 18 14:01:27.912384 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:01:27.912337    2565 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="1d0d5aec-4917-4b99-a58e-592c2863d81b" path="/var/lib/kubelet/pods/1d0d5aec-4917-4b99-a58e-592c2863d81b/volumes"
Sep 18 14:01:27.916968 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:01:27.916940    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/bbc5c0d4-5395-4791-a3e5-65ba18efef52-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"bbc5c0d4-5395-4791-a3e5-65ba18efef52\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 18 14:01:27.916968 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:01:27.916968    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-r5mlp\" (UniqueName: \"kubernetes.io/projected/bbc5c0d4-5395-4791-a3e5-65ba18efef52-kube-api-access-r5mlp\") pod \"prometheus-k8s-0\" (UID: \"bbc5c0d4-5395-4791-a3e5-65ba18efef52\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 18 14:01:27.917190 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:01:27.916988    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/bbc5c0d4-5395-4791-a3e5-65ba18efef52-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"bbc5c0d4-5395-4791-a3e5-65ba18efef52\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 18 14:01:27.917190 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:01:27.917003    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/bbc5c0d4-5395-4791-a3e5-65ba18efef52-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"bbc5c0d4-5395-4791-a3e5-65ba18efef52\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 18 14:01:27.917190 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:01:27.917101    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/bbc5c0d4-5395-4791-a3e5-65ba18efef52-config-out\") pod \"prometheus-k8s-0\" (UID: \"bbc5c0d4-5395-4791-a3e5-65ba18efef52\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 18 14:01:27.917190 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:01:27.917128    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/bbc5c0d4-5395-4791-a3e5-65ba18efef52-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"bbc5c0d4-5395-4791-a3e5-65ba18efef52\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 18 14:01:27.917190 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:01:27.917150    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/bbc5c0d4-5395-4791-a3e5-65ba18efef52-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"bbc5c0d4-5395-4791-a3e5-65ba18efef52\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 18 14:01:27.917190 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:01:27.917174    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/bbc5c0d4-5395-4791-a3e5-65ba18efef52-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"bbc5c0d4-5395-4791-a3e5-65ba18efef52\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 18 14:01:27.917481 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:01:27.917193    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/bbc5c0d4-5395-4791-a3e5-65ba18efef52-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"bbc5c0d4-5395-4791-a3e5-65ba18efef52\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 18 14:01:27.917481 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:01:27.917238    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/bbc5c0d4-5395-4791-a3e5-65ba18efef52-config\") pod \"prometheus-k8s-0\" (UID: \"bbc5c0d4-5395-4791-a3e5-65ba18efef52\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 18 14:01:27.917481 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:01:27.917256    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/bbc5c0d4-5395-4791-a3e5-65ba18efef52-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"bbc5c0d4-5395-4791-a3e5-65ba18efef52\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 18 14:01:27.917481 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:01:27.917277    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/bbc5c0d4-5395-4791-a3e5-65ba18efef52-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"bbc5c0d4-5395-4791-a3e5-65ba18efef52\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 18 14:01:27.917481 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:01:27.917297    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/bbc5c0d4-5395-4791-a3e5-65ba18efef52-web-config\") pod \"prometheus-k8s-0\" (UID: \"bbc5c0d4-5395-4791-a3e5-65ba18efef52\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 18 14:01:27.917481 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:01:27.917315    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/bbc5c0d4-5395-4791-a3e5-65ba18efef52-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"bbc5c0d4-5395-4791-a3e5-65ba18efef52\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 18 14:01:27.917481 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:01:27.917345    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/bbc5c0d4-5395-4791-a3e5-65ba18efef52-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"bbc5c0d4-5395-4791-a3e5-65ba18efef52\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 18 14:01:27.917481 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:01:27.917374    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/bbc5c0d4-5395-4791-a3e5-65ba18efef52-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"bbc5c0d4-5395-4791-a3e5-65ba18efef52\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 18 14:01:27.917481 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:01:27.917395    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/bbc5c0d4-5395-4791-a3e5-65ba18efef52-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"bbc5c0d4-5395-4791-a3e5-65ba18efef52\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 18 14:01:27.917481 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:01:27.917414    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/bbc5c0d4-5395-4791-a3e5-65ba18efef52-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"bbc5c0d4-5395-4791-a3e5-65ba18efef52\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 18 14:01:27.917481 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:01:27.917434    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/bbc5c0d4-5395-4791-a3e5-65ba18efef52-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"bbc5c0d4-5395-4791-a3e5-65ba18efef52\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 18 14:01:27.917994 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:01:27.917535    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/bbc5c0d4-5395-4791-a3e5-65ba18efef52-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"bbc5c0d4-5395-4791-a3e5-65ba18efef52\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 18 14:01:28.018004 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:01:28.017927    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/bbc5c0d4-5395-4791-a3e5-65ba18efef52-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"bbc5c0d4-5395-4791-a3e5-65ba18efef52\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 18 14:01:28.018004 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:01:28.017964    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/bbc5c0d4-5395-4791-a3e5-65ba18efef52-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"bbc5c0d4-5395-4791-a3e5-65ba18efef52\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 18 14:01:28.018004 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:01:28.017984    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-r5mlp\" (UniqueName: \"kubernetes.io/projected/bbc5c0d4-5395-4791-a3e5-65ba18efef52-kube-api-access-r5mlp\") pod \"prometheus-k8s-0\" (UID: \"bbc5c0d4-5395-4791-a3e5-65ba18efef52\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 18 14:01:28.018231 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:01:28.018099    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/bbc5c0d4-5395-4791-a3e5-65ba18efef52-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"bbc5c0d4-5395-4791-a3e5-65ba18efef52\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 18 14:01:28.018231 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:01:28.018136    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/bbc5c0d4-5395-4791-a3e5-65ba18efef52-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"bbc5c0d4-5395-4791-a3e5-65ba18efef52\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 18 14:01:28.018231 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:01:28.018187    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/bbc5c0d4-5395-4791-a3e5-65ba18efef52-config-out\") pod \"prometheus-k8s-0\" (UID: \"bbc5c0d4-5395-4791-a3e5-65ba18efef52\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 18 14:01:28.018231 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:01:28.018221    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/bbc5c0d4-5395-4791-a3e5-65ba18efef52-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"bbc5c0d4-5395-4791-a3e5-65ba18efef52\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 18 14:01:28.018426 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:01:28.018256    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/bbc5c0d4-5395-4791-a3e5-65ba18efef52-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"bbc5c0d4-5395-4791-a3e5-65ba18efef52\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 18 14:01:28.018426 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:01:28.018283    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/bbc5c0d4-5395-4791-a3e5-65ba18efef52-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"bbc5c0d4-5395-4791-a3e5-65ba18efef52\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 18 14:01:28.018426 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:01:28.018309    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/bbc5c0d4-5395-4791-a3e5-65ba18efef52-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"bbc5c0d4-5395-4791-a3e5-65ba18efef52\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 18 14:01:28.018426 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:01:28.018343    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/bbc5c0d4-5395-4791-a3e5-65ba18efef52-config\") pod \"prometheus-k8s-0\" (UID: \"bbc5c0d4-5395-4791-a3e5-65ba18efef52\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 18 14:01:28.018426 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:01:28.018372    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/bbc5c0d4-5395-4791-a3e5-65ba18efef52-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"bbc5c0d4-5395-4791-a3e5-65ba18efef52\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 18 14:01:28.018426 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:01:28.018403    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/bbc5c0d4-5395-4791-a3e5-65ba18efef52-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"bbc5c0d4-5395-4791-a3e5-65ba18efef52\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 18 14:01:28.018426 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:01:28.018423    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/bbc5c0d4-5395-4791-a3e5-65ba18efef52-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"bbc5c0d4-5395-4791-a3e5-65ba18efef52\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 18 14:01:28.018766 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:01:28.018427    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/bbc5c0d4-5395-4791-a3e5-65ba18efef52-web-config\") pod \"prometheus-k8s-0\" (UID: \"bbc5c0d4-5395-4791-a3e5-65ba18efef52\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 18 14:01:28.018766 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:01:28.018504    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/bbc5c0d4-5395-4791-a3e5-65ba18efef52-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"bbc5c0d4-5395-4791-a3e5-65ba18efef52\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 18 14:01:28.018766 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:01:28.018551    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/bbc5c0d4-5395-4791-a3e5-65ba18efef52-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"bbc5c0d4-5395-4791-a3e5-65ba18efef52\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 18 14:01:28.018766 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:01:28.018581    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/bbc5c0d4-5395-4791-a3e5-65ba18efef52-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"bbc5c0d4-5395-4791-a3e5-65ba18efef52\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 18 14:01:28.018766 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:01:28.018608    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/bbc5c0d4-5395-4791-a3e5-65ba18efef52-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"bbc5c0d4-5395-4791-a3e5-65ba18efef52\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 18 14:01:28.018766 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:01:28.018641    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/bbc5c0d4-5395-4791-a3e5-65ba18efef52-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"bbc5c0d4-5395-4791-a3e5-65ba18efef52\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 18 14:01:28.018766 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:01:28.018671    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/bbc5c0d4-5395-4791-a3e5-65ba18efef52-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"bbc5c0d4-5395-4791-a3e5-65ba18efef52\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 18 14:01:28.018766 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:01:28.018736    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/bbc5c0d4-5395-4791-a3e5-65ba18efef52-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"bbc5c0d4-5395-4791-a3e5-65ba18efef52\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 18 14:01:28.019198 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:01:28.018898    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/bbc5c0d4-5395-4791-a3e5-65ba18efef52-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"bbc5c0d4-5395-4791-a3e5-65ba18efef52\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 18 14:01:28.021564 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:01:28.021234    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/bbc5c0d4-5395-4791-a3e5-65ba18efef52-config-out\") pod \"prometheus-k8s-0\" (UID: \"bbc5c0d4-5395-4791-a3e5-65ba18efef52\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 18 14:01:28.021564 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:01:28.021240    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/bbc5c0d4-5395-4791-a3e5-65ba18efef52-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"bbc5c0d4-5395-4791-a3e5-65ba18efef52\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 18 14:01:28.021564 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:01:28.021329    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/bbc5c0d4-5395-4791-a3e5-65ba18efef52-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"bbc5c0d4-5395-4791-a3e5-65ba18efef52\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 18 14:01:28.021564 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:01:28.021379    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/bbc5c0d4-5395-4791-a3e5-65ba18efef52-web-config\") pod \"prometheus-k8s-0\" (UID: \"bbc5c0d4-5395-4791-a3e5-65ba18efef52\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 18 14:01:28.021564 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:01:28.021392    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/bbc5c0d4-5395-4791-a3e5-65ba18efef52-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"bbc5c0d4-5395-4791-a3e5-65ba18efef52\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 18 14:01:28.021564 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:01:28.021482    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/secret/bbc5c0d4-5395-4791-a3e5-65ba18efef52-config\") pod \"prometheus-k8s-0\" (UID: \"bbc5c0d4-5395-4791-a3e5-65ba18efef52\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 18 14:01:28.021935 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:01:28.021807    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/bbc5c0d4-5395-4791-a3e5-65ba18efef52-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"bbc5c0d4-5395-4791-a3e5-65ba18efef52\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 18 14:01:28.021991 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:01:28.021949    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/bbc5c0d4-5395-4791-a3e5-65ba18efef52-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"bbc5c0d4-5395-4791-a3e5-65ba18efef52\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 18 14:01:28.022431 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:01:28.022358    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/bbc5c0d4-5395-4791-a3e5-65ba18efef52-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"bbc5c0d4-5395-4791-a3e5-65ba18efef52\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 18 14:01:28.022534 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:01:28.022486    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/bbc5c0d4-5395-4791-a3e5-65ba18efef52-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"bbc5c0d4-5395-4791-a3e5-65ba18efef52\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 18 14:01:28.022941 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:01:28.022915    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/bbc5c0d4-5395-4791-a3e5-65ba18efef52-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"bbc5c0d4-5395-4791-a3e5-65ba18efef52\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 18 14:01:28.023841 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:01:28.023814    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/bbc5c0d4-5395-4791-a3e5-65ba18efef52-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"bbc5c0d4-5395-4791-a3e5-65ba18efef52\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 18 14:01:28.023973 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:01:28.023949    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/bbc5c0d4-5395-4791-a3e5-65ba18efef52-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"bbc5c0d4-5395-4791-a3e5-65ba18efef52\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 18 14:01:28.024152 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:01:28.024126    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/bbc5c0d4-5395-4791-a3e5-65ba18efef52-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"bbc5c0d4-5395-4791-a3e5-65ba18efef52\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 18 14:01:28.024320 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:01:28.024302    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/bbc5c0d4-5395-4791-a3e5-65ba18efef52-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"bbc5c0d4-5395-4791-a3e5-65ba18efef52\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 18 14:01:28.024464 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:01:28.024448    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/bbc5c0d4-5395-4791-a3e5-65ba18efef52-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"bbc5c0d4-5395-4791-a3e5-65ba18efef52\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 18 14:01:28.032614 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:01:28.032591    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-r5mlp\" (UniqueName: \"kubernetes.io/projected/bbc5c0d4-5395-4791-a3e5-65ba18efef52-kube-api-access-r5mlp\") pod \"prometheus-k8s-0\" (UID: \"bbc5c0d4-5395-4791-a3e5-65ba18efef52\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 18 14:01:28.069252 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:01:28.069214    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Sep 18 14:01:28.349455 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:01:28.349426    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Sep 18 14:01:28.350380 ip-10-0-129-16 kubenswrapper[2565]: W0918 14:01:28.350353    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbbc5c0d4_5395_4791_a3e5_65ba18efef52.slice/crio-2f1808f4980cbaae3df1fbb85cde35bef3daa14c2b1f0d090406d391b4e3e00e WatchSource:0}: Error finding container 2f1808f4980cbaae3df1fbb85cde35bef3daa14c2b1f0d090406d391b4e3e00e: Status 404 returned error can't find the container with id 2f1808f4980cbaae3df1fbb85cde35bef3daa14c2b1f0d090406d391b4e3e00e
Sep 18 14:01:28.702681 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:01:28.702588    2565 generic.go:356] "Generic (PLEG): container finished" podID="bbc5c0d4-5395-4791-a3e5-65ba18efef52" containerID="41aedd28326bc0a06c858fe72f0c481d3220dd168d7e1df8c9f1a66befe5b060" exitCode=0
Sep 18 14:01:28.702836 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:01:28.702670    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"bbc5c0d4-5395-4791-a3e5-65ba18efef52","Type":"ContainerDied","Data":"41aedd28326bc0a06c858fe72f0c481d3220dd168d7e1df8c9f1a66befe5b060"}
Sep 18 14:01:28.702836 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:01:28.702772    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"bbc5c0d4-5395-4791-a3e5-65ba18efef52","Type":"ContainerStarted","Data":"2f1808f4980cbaae3df1fbb85cde35bef3daa14c2b1f0d090406d391b4e3e00e"}
Sep 18 14:01:29.711348 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:01:29.711315    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"bbc5c0d4-5395-4791-a3e5-65ba18efef52","Type":"ContainerStarted","Data":"b53de8de43efa8c9cd320f630c8edcf6613deb333ac359e96e5a6cc8f7f39cbb"}
Sep 18 14:01:29.711709 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:01:29.711377    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"bbc5c0d4-5395-4791-a3e5-65ba18efef52","Type":"ContainerStarted","Data":"f36129f07e72b4712257cbcc36c5c10c9a1093e73f76390fcd44d48e818f143a"}
Sep 18 14:01:29.711709 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:01:29.711404    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"bbc5c0d4-5395-4791-a3e5-65ba18efef52","Type":"ContainerStarted","Data":"7fc3ecb9a3ae8e942221d5569ca09a1337b77e091648087f50a373f2b8f18bf0"}
Sep 18 14:01:29.711709 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:01:29.711428    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"bbc5c0d4-5395-4791-a3e5-65ba18efef52","Type":"ContainerStarted","Data":"9d3e8e7cc30db4c56f42c41232147e44ec41b58101b796decf1370748931868d"}
Sep 18 14:01:29.711709 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:01:29.711450    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"bbc5c0d4-5395-4791-a3e5-65ba18efef52","Type":"ContainerStarted","Data":"6e6afc919be2edfbee52ccfed268315de0cbfd27e4c6f13be6bdf1626ddc8d86"}
Sep 18 14:01:29.711709 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:01:29.711474    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"bbc5c0d4-5395-4791-a3e5-65ba18efef52","Type":"ContainerStarted","Data":"7d4d3407c758aa9b422b07517702741f456d748068fac125178573aa3744f2b8"}
Sep 18 14:01:29.746464 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:01:29.746411    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-k8s-0" podStartSLOduration=2.727316493 podStartE2EDuration="2.746394463s" totalImagesPullingTime="0s" totalInitContainerRuntime="19.07797ms" isStatefulPod=false podCreationTimestamp="2026-09-18 14:01:27 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 14:01:29.744165229 +0000 UTC m=+157.918074910" watchObservedRunningTime="2026-09-18 14:01:29.746394463 +0000 UTC m=+157.920304123"
Sep 18 14:01:33.070145 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:01:33.070095    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-k8s-0"
Sep 18 14:02:28.069425 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:02:28.069387    2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/prometheus-k8s-0"
Sep 18 14:02:28.084804 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:02:28.084779    2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/prometheus-k8s-0"
Sep 18 14:02:28.957255 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:02:28.957232    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-k8s-0"
Sep 18 14:02:56.859228 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:02:56.859187    2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-webhook-6486ffd997-fg2zx"]
Sep 18 14:02:56.862657 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:02:56.862635    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-webhook-6486ffd997-fg2zx"
Sep 18 14:02:56.871101 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:02:56.871080    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"openshift-service-ca.crt\""
Sep 18 14:02:56.871892 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:02:56.871875    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"kube-root-ca.crt\""
Sep 18 14:02:56.871992 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:02:56.871894    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-webhook-dockercfg-7dnzj\""
Sep 18 14:02:56.900122 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:02:56.900086    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-webhook-6486ffd997-fg2zx"]
Sep 18 14:02:56.978053 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:02:56.978000    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/0555d562-e8ff-472c-9d65-bda2fc7dd26e-bound-sa-token\") pod \"cert-manager-webhook-6486ffd997-fg2zx\" (UID: \"0555d562-e8ff-472c-9d65-bda2fc7dd26e\") " pod="cert-manager/cert-manager-webhook-6486ffd997-fg2zx"
Sep 18 14:02:56.978238 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:02:56.978071    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jzfdj\" (UniqueName: \"kubernetes.io/projected/0555d562-e8ff-472c-9d65-bda2fc7dd26e-kube-api-access-jzfdj\") pod \"cert-manager-webhook-6486ffd997-fg2zx\" (UID: \"0555d562-e8ff-472c-9d65-bda2fc7dd26e\") " pod="cert-manager/cert-manager-webhook-6486ffd997-fg2zx"
Sep 18 14:02:57.078819 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:02:57.078784    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/0555d562-e8ff-472c-9d65-bda2fc7dd26e-bound-sa-token\") pod \"cert-manager-webhook-6486ffd997-fg2zx\" (UID: \"0555d562-e8ff-472c-9d65-bda2fc7dd26e\") " pod="cert-manager/cert-manager-webhook-6486ffd997-fg2zx"
Sep 18 14:02:57.078819 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:02:57.078827    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jzfdj\" (UniqueName: \"kubernetes.io/projected/0555d562-e8ff-472c-9d65-bda2fc7dd26e-kube-api-access-jzfdj\") pod \"cert-manager-webhook-6486ffd997-fg2zx\" (UID: \"0555d562-e8ff-472c-9d65-bda2fc7dd26e\") " pod="cert-manager/cert-manager-webhook-6486ffd997-fg2zx"
Sep 18 14:02:57.099838 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:02:57.099804    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/0555d562-e8ff-472c-9d65-bda2fc7dd26e-bound-sa-token\") pod \"cert-manager-webhook-6486ffd997-fg2zx\" (UID: \"0555d562-e8ff-472c-9d65-bda2fc7dd26e\") " pod="cert-manager/cert-manager-webhook-6486ffd997-fg2zx"
Sep 18 14:02:57.112773 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:02:57.112708    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jzfdj\" (UniqueName: \"kubernetes.io/projected/0555d562-e8ff-472c-9d65-bda2fc7dd26e-kube-api-access-jzfdj\") pod \"cert-manager-webhook-6486ffd997-fg2zx\" (UID: \"0555d562-e8ff-472c-9d65-bda2fc7dd26e\") " pod="cert-manager/cert-manager-webhook-6486ffd997-fg2zx"
Sep 18 14:02:57.190850 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:02:57.190814    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-webhook-6486ffd997-fg2zx"
Sep 18 14:02:57.467905 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:02:57.467881    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-webhook-6486ffd997-fg2zx"]
Sep 18 14:02:57.470366 ip-10-0-129-16 kubenswrapper[2565]: W0918 14:02:57.470335    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod0555d562_e8ff_472c_9d65_bda2fc7dd26e.slice/crio-c53e73b8f7ebcf1c5d321d838291e2300d29b290977ceeca03bb3ea10c33802e WatchSource:0}: Error finding container c53e73b8f7ebcf1c5d321d838291e2300d29b290977ceeca03bb3ea10c33802e: Status 404 returned error can't find the container with id c53e73b8f7ebcf1c5d321d838291e2300d29b290977ceeca03bb3ea10c33802e
Sep 18 14:02:58.058902 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:02:58.058867    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-webhook-6486ffd997-fg2zx" event={"ID":"0555d562-e8ff-472c-9d65-bda2fc7dd26e","Type":"ContainerStarted","Data":"c53e73b8f7ebcf1c5d321d838291e2300d29b290977ceeca03bb3ea10c33802e"}
Sep 18 14:02:59.426717 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:02:59.426676    2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-cainjector-559cdd8c87-nqnj9"]
Sep 18 14:02:59.430679 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:02:59.430656    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-cainjector-559cdd8c87-nqnj9"
Sep 18 14:02:59.433490 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:02:59.433465    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-cainjector-dockercfg-vzcz9\""
Sep 18 14:02:59.443247 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:02:59.443214    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-cainjector-559cdd8c87-nqnj9"]
Sep 18 14:02:59.504635 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:02:59.504583    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/998ca240-0392-4a83-9ae2-624b09f6c604-bound-sa-token\") pod \"cert-manager-cainjector-559cdd8c87-nqnj9\" (UID: \"998ca240-0392-4a83-9ae2-624b09f6c604\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-nqnj9"
Sep 18 14:02:59.504811 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:02:59.504745    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tx2fl\" (UniqueName: \"kubernetes.io/projected/998ca240-0392-4a83-9ae2-624b09f6c604-kube-api-access-tx2fl\") pod \"cert-manager-cainjector-559cdd8c87-nqnj9\" (UID: \"998ca240-0392-4a83-9ae2-624b09f6c604\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-nqnj9"
Sep 18 14:02:59.605428 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:02:59.605388    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tx2fl\" (UniqueName: \"kubernetes.io/projected/998ca240-0392-4a83-9ae2-624b09f6c604-kube-api-access-tx2fl\") pod \"cert-manager-cainjector-559cdd8c87-nqnj9\" (UID: \"998ca240-0392-4a83-9ae2-624b09f6c604\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-nqnj9"
Sep 18 14:02:59.605592 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:02:59.605441    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/998ca240-0392-4a83-9ae2-624b09f6c604-bound-sa-token\") pod \"cert-manager-cainjector-559cdd8c87-nqnj9\" (UID: \"998ca240-0392-4a83-9ae2-624b09f6c604\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-nqnj9"
Sep 18 14:02:59.614885 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:02:59.614859    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/998ca240-0392-4a83-9ae2-624b09f6c604-bound-sa-token\") pod \"cert-manager-cainjector-559cdd8c87-nqnj9\" (UID: \"998ca240-0392-4a83-9ae2-624b09f6c604\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-nqnj9"
Sep 18 14:02:59.615074 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:02:59.615015    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tx2fl\" (UniqueName: \"kubernetes.io/projected/998ca240-0392-4a83-9ae2-624b09f6c604-kube-api-access-tx2fl\") pod \"cert-manager-cainjector-559cdd8c87-nqnj9\" (UID: \"998ca240-0392-4a83-9ae2-624b09f6c604\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-nqnj9"
Sep 18 14:02:59.740944 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:02:59.740857    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-cainjector-559cdd8c87-nqnj9"
Sep 18 14:03:00.569702 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:03:00.569676    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-cainjector-559cdd8c87-nqnj9"]
Sep 18 14:03:00.572315 ip-10-0-129-16 kubenswrapper[2565]: W0918 14:03:00.572285    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod998ca240_0392_4a83_9ae2_624b09f6c604.slice/crio-4c9661b6ffc0825d60e105afe0c03c14c480af25dedd4b770aa818a39a211e33 WatchSource:0}: Error finding container 4c9661b6ffc0825d60e105afe0c03c14c480af25dedd4b770aa818a39a211e33: Status 404 returned error can't find the container with id 4c9661b6ffc0825d60e105afe0c03c14c480af25dedd4b770aa818a39a211e33
Sep 18 14:03:01.076695 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:03:01.076656    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-webhook-6486ffd997-fg2zx" event={"ID":"0555d562-e8ff-472c-9d65-bda2fc7dd26e","Type":"ContainerStarted","Data":"7f4b538c9331452d53dd31ff61935fd2e5123286e4549cb3d6b9b74ae1bf7635"}
Sep 18 14:03:01.077089 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:03:01.076716    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="cert-manager/cert-manager-webhook-6486ffd997-fg2zx"
Sep 18 14:03:01.078353 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:03:01.078324    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-cainjector-559cdd8c87-nqnj9" event={"ID":"998ca240-0392-4a83-9ae2-624b09f6c604","Type":"ContainerStarted","Data":"1a4f922868b7045d8bcd2d22b3796abb38f45e3141499400f7ba15a533d56c2b"}
Sep 18 14:03:01.078472 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:03:01.078365    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-cainjector-559cdd8c87-nqnj9" event={"ID":"998ca240-0392-4a83-9ae2-624b09f6c604","Type":"ContainerStarted","Data":"4c9661b6ffc0825d60e105afe0c03c14c480af25dedd4b770aa818a39a211e33"}
Sep 18 14:03:01.096843 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:03:01.096791    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-webhook-6486ffd997-fg2zx" podStartSLOduration=2.190205497 podStartE2EDuration="5.096773213s" totalImagesPullingTime="2.906567716s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 14:02:56 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 14:03:01.094680958 +0000 UTC m=+249.268590619" watchObservedRunningTime="2026-09-18 14:03:01.096773213 +0000 UTC m=+249.270682872"
Sep 18 14:03:01.111962 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:03:01.111911    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-cainjector-559cdd8c87-nqnj9" podStartSLOduration=2.111895165 podStartE2EDuration="2.111895165s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 14:02:59 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 14:03:01.11043441 +0000 UTC m=+249.284344071" watchObservedRunningTime="2026-09-18 14:03:01.111895165 +0000 UTC m=+249.285804825"
Sep 18 14:03:07.085388 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:03:07.085307    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="cert-manager/cert-manager-webhook-6486ffd997-fg2zx"
Sep 18 14:03:29.132906 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:03:29.132866    2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-74d96fdcf9-hvj2l"]
Sep 18 14:03:29.136294 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:03:29.136277    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-74d96fdcf9-hvj2l"
Sep 18 14:03:29.140595 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:03:29.140573    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\""
Sep 18 14:03:29.140902 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:03:29.140888    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\""
Sep 18 14:03:29.141133 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:03:29.141111    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-ms7h5\""
Sep 18 14:03:29.141246 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:03:29.141228    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\""
Sep 18 14:03:29.141396 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:03:29.141382    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\""
Sep 18 14:03:29.159140 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:03:29.159118    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-74d96fdcf9-hvj2l"]
Sep 18 14:03:29.270992 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:03:29.270957    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jl7ph\" (UniqueName: \"kubernetes.io/projected/7b2fb2cd-9edd-4856-aeaf-342eec275da1-kube-api-access-jl7ph\") pod \"opendatahub-operator-controller-manager-74d96fdcf9-hvj2l\" (UID: \"7b2fb2cd-9edd-4856-aeaf-342eec275da1\") " pod="opendatahub/opendatahub-operator-controller-manager-74d96fdcf9-hvj2l"
Sep 18 14:03:29.271199 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:03:29.271084    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/7b2fb2cd-9edd-4856-aeaf-342eec275da1-webhook-cert\") pod \"opendatahub-operator-controller-manager-74d96fdcf9-hvj2l\" (UID: \"7b2fb2cd-9edd-4856-aeaf-342eec275da1\") " pod="opendatahub/opendatahub-operator-controller-manager-74d96fdcf9-hvj2l"
Sep 18 14:03:29.271199 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:03:29.271134    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/7b2fb2cd-9edd-4856-aeaf-342eec275da1-tmp\") pod \"opendatahub-operator-controller-manager-74d96fdcf9-hvj2l\" (UID: \"7b2fb2cd-9edd-4856-aeaf-342eec275da1\") " pod="opendatahub/opendatahub-operator-controller-manager-74d96fdcf9-hvj2l"
Sep 18 14:03:29.271199 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:03:29.271158    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/7b2fb2cd-9edd-4856-aeaf-342eec275da1-manifests\") pod \"opendatahub-operator-controller-manager-74d96fdcf9-hvj2l\" (UID: \"7b2fb2cd-9edd-4856-aeaf-342eec275da1\") " pod="opendatahub/opendatahub-operator-controller-manager-74d96fdcf9-hvj2l"
Sep 18 14:03:29.271308 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:03:29.271206    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/7b2fb2cd-9edd-4856-aeaf-342eec275da1-apiservice-cert\") pod \"opendatahub-operator-controller-manager-74d96fdcf9-hvj2l\" (UID: \"7b2fb2cd-9edd-4856-aeaf-342eec275da1\") " pod="opendatahub/opendatahub-operator-controller-manager-74d96fdcf9-hvj2l"
Sep 18 14:03:29.372346 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:03:29.372306    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/7b2fb2cd-9edd-4856-aeaf-342eec275da1-apiservice-cert\") pod \"opendatahub-operator-controller-manager-74d96fdcf9-hvj2l\" (UID: \"7b2fb2cd-9edd-4856-aeaf-342eec275da1\") " pod="opendatahub/opendatahub-operator-controller-manager-74d96fdcf9-hvj2l"
Sep 18 14:03:29.372346 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:03:29.372350    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jl7ph\" (UniqueName: \"kubernetes.io/projected/7b2fb2cd-9edd-4856-aeaf-342eec275da1-kube-api-access-jl7ph\") pod \"opendatahub-operator-controller-manager-74d96fdcf9-hvj2l\" (UID: \"7b2fb2cd-9edd-4856-aeaf-342eec275da1\") " pod="opendatahub/opendatahub-operator-controller-manager-74d96fdcf9-hvj2l"
Sep 18 14:03:29.372615 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:03:29.372430    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/7b2fb2cd-9edd-4856-aeaf-342eec275da1-webhook-cert\") pod \"opendatahub-operator-controller-manager-74d96fdcf9-hvj2l\" (UID: \"7b2fb2cd-9edd-4856-aeaf-342eec275da1\") " pod="opendatahub/opendatahub-operator-controller-manager-74d96fdcf9-hvj2l"
Sep 18 14:03:29.372615 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:03:29.372472    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/7b2fb2cd-9edd-4856-aeaf-342eec275da1-tmp\") pod \"opendatahub-operator-controller-manager-74d96fdcf9-hvj2l\" (UID: \"7b2fb2cd-9edd-4856-aeaf-342eec275da1\") " pod="opendatahub/opendatahub-operator-controller-manager-74d96fdcf9-hvj2l"
Sep 18 14:03:29.372615 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:03:29.372502    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/7b2fb2cd-9edd-4856-aeaf-342eec275da1-manifests\") pod \"opendatahub-operator-controller-manager-74d96fdcf9-hvj2l\" (UID: \"7b2fb2cd-9edd-4856-aeaf-342eec275da1\") " pod="opendatahub/opendatahub-operator-controller-manager-74d96fdcf9-hvj2l"
Sep 18 14:03:29.372921 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:03:29.372891    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/7b2fb2cd-9edd-4856-aeaf-342eec275da1-manifests\") pod \"opendatahub-operator-controller-manager-74d96fdcf9-hvj2l\" (UID: \"7b2fb2cd-9edd-4856-aeaf-342eec275da1\") " pod="opendatahub/opendatahub-operator-controller-manager-74d96fdcf9-hvj2l"
Sep 18 14:03:29.374722 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:03:29.374695    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/7b2fb2cd-9edd-4856-aeaf-342eec275da1-tmp\") pod \"opendatahub-operator-controller-manager-74d96fdcf9-hvj2l\" (UID: \"7b2fb2cd-9edd-4856-aeaf-342eec275da1\") " pod="opendatahub/opendatahub-operator-controller-manager-74d96fdcf9-hvj2l"
Sep 18 14:03:29.374926 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:03:29.374905    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/7b2fb2cd-9edd-4856-aeaf-342eec275da1-webhook-cert\") pod \"opendatahub-operator-controller-manager-74d96fdcf9-hvj2l\" (UID: \"7b2fb2cd-9edd-4856-aeaf-342eec275da1\") " pod="opendatahub/opendatahub-operator-controller-manager-74d96fdcf9-hvj2l"
Sep 18 14:03:29.374994 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:03:29.374905    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/7b2fb2cd-9edd-4856-aeaf-342eec275da1-apiservice-cert\") pod \"opendatahub-operator-controller-manager-74d96fdcf9-hvj2l\" (UID: \"7b2fb2cd-9edd-4856-aeaf-342eec275da1\") " pod="opendatahub/opendatahub-operator-controller-manager-74d96fdcf9-hvj2l"
Sep 18 14:03:29.388904 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:03:29.388841    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jl7ph\" (UniqueName: \"kubernetes.io/projected/7b2fb2cd-9edd-4856-aeaf-342eec275da1-kube-api-access-jl7ph\") pod \"opendatahub-operator-controller-manager-74d96fdcf9-hvj2l\" (UID: \"7b2fb2cd-9edd-4856-aeaf-342eec275da1\") " pod="opendatahub/opendatahub-operator-controller-manager-74d96fdcf9-hvj2l"
Sep 18 14:03:29.447174 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:03:29.447128    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-74d96fdcf9-hvj2l"
Sep 18 14:03:29.724063 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:03:29.724020    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-74d96fdcf9-hvj2l"]
Sep 18 14:03:29.726598 ip-10-0-129-16 kubenswrapper[2565]: W0918 14:03:29.726570    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7b2fb2cd_9edd_4856_aeaf_342eec275da1.slice/crio-8479ee1cd9bee42782bcce2c958f42c99592ef9937079a669fe0c3a2d90c0ccd WatchSource:0}: Error finding container 8479ee1cd9bee42782bcce2c958f42c99592ef9937079a669fe0c3a2d90c0ccd: Status 404 returned error can't find the container with id 8479ee1cd9bee42782bcce2c958f42c99592ef9937079a669fe0c3a2d90c0ccd
Sep 18 14:03:30.199973 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:03:30.199935    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-74d96fdcf9-hvj2l" event={"ID":"7b2fb2cd-9edd-4856-aeaf-342eec275da1","Type":"ContainerStarted","Data":"8479ee1cd9bee42782bcce2c958f42c99592ef9937079a669fe0c3a2d90c0ccd"}
Sep 18 14:03:33.216976 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:03:33.216942    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-74d96fdcf9-hvj2l" event={"ID":"7b2fb2cd-9edd-4856-aeaf-342eec275da1","Type":"ContainerStarted","Data":"04128733d5807dca768e3e9cb6bb34cfd70b1c618fc6949172b868df32e7f7e3"}
Sep 18 14:03:34.228026 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:03:34.227990    2565 generic.go:356] "Generic (PLEG): container finished" podID="7b2fb2cd-9edd-4856-aeaf-342eec275da1" containerID="04128733d5807dca768e3e9cb6bb34cfd70b1c618fc6949172b868df32e7f7e3" exitCode=0
Sep 18 14:03:34.228402 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:03:34.228073    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-74d96fdcf9-hvj2l" event={"ID":"7b2fb2cd-9edd-4856-aeaf-342eec275da1","Type":"ContainerDied","Data":"04128733d5807dca768e3e9cb6bb34cfd70b1c618fc6949172b868df32e7f7e3"}
Sep 18 14:03:35.234656 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:03:35.234621    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-74d96fdcf9-hvj2l" event={"ID":"7b2fb2cd-9edd-4856-aeaf-342eec275da1","Type":"ContainerStarted","Data":"933c052f16fa66e50576cf4d881f1abb5eb4dc5add8562d9ed06a71f3ddf299c"}
Sep 18 14:03:35.235079 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:03:35.234729    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-74d96fdcf9-hvj2l"
Sep 18 14:03:35.265588 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:03:35.265535    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-74d96fdcf9-hvj2l" podStartSLOduration=1.9149197550000001 podStartE2EDuration="6.265520587s" totalImagesPullingTime="3.294504366s" totalInitContainerRuntime="1.056096466s" isStatefulPod=false podCreationTimestamp="2026-09-18 14:03:29 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 14:03:35.26386046 +0000 UTC m=+283.437770120" watchObservedRunningTime="2026-09-18 14:03:35.265520587 +0000 UTC m=+283.439430247"
Sep 18 14:03:46.240696 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:03:46.240660    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-74d96fdcf9-hvj2l"
Sep 18 14:03:50.782117 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:03:50.782078    2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/lws-controller-manager-ff9cccccb-ntpsv"]
Sep 18 14:03:50.789454 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:03:50.789428    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-ff9cccccb-ntpsv"
Sep 18 14:03:50.792228 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:03:50.792202    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"lws-manager-config\""
Sep 18 14:03:50.793291 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:03:50.793252    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"webhook-server-cert\""
Sep 18 14:03:50.793291 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:03:50.793277    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"metrics-server-cert\""
Sep 18 14:03:50.793480 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:03:50.793319    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"openshift-service-ca.crt\""
Sep 18 14:03:50.793480 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:03:50.793338    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"kube-root-ca.crt\""
Sep 18 14:03:50.793624 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:03:50.793611    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"lws-controller-manager-dockercfg-mmxrk\""
Sep 18 14:03:50.796425 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:03:50.796400    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-ff9cccccb-ntpsv"]
Sep 18 14:03:50.850831 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:03:50.850790    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/5ce17b25-ef8c-4736-b9d4-319e627c1d9d-cert\") pod \"lws-controller-manager-ff9cccccb-ntpsv\" (UID: \"5ce17b25-ef8c-4736-b9d4-319e627c1d9d\") " pod="openshift-lws-operator/lws-controller-manager-ff9cccccb-ntpsv"
Sep 18 14:03:50.851128 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:03:50.851100    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/5ce17b25-ef8c-4736-b9d4-319e627c1d9d-manager-config\") pod \"lws-controller-manager-ff9cccccb-ntpsv\" (UID: \"5ce17b25-ef8c-4736-b9d4-319e627c1d9d\") " pod="openshift-lws-operator/lws-controller-manager-ff9cccccb-ntpsv"
Sep 18 14:03:50.851269 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:03:50.851218    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/5ce17b25-ef8c-4736-b9d4-319e627c1d9d-metrics-cert\") pod \"lws-controller-manager-ff9cccccb-ntpsv\" (UID: \"5ce17b25-ef8c-4736-b9d4-319e627c1d9d\") " pod="openshift-lws-operator/lws-controller-manager-ff9cccccb-ntpsv"
Sep 18 14:03:50.851269 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:03:50.851246    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7k8p2\" (UniqueName: \"kubernetes.io/projected/5ce17b25-ef8c-4736-b9d4-319e627c1d9d-kube-api-access-7k8p2\") pod \"lws-controller-manager-ff9cccccb-ntpsv\" (UID: \"5ce17b25-ef8c-4736-b9d4-319e627c1d9d\") " pod="openshift-lws-operator/lws-controller-manager-ff9cccccb-ntpsv"
Sep 18 14:03:50.952176 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:03:50.952145    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/5ce17b25-ef8c-4736-b9d4-319e627c1d9d-metrics-cert\") pod \"lws-controller-manager-ff9cccccb-ntpsv\" (UID: \"5ce17b25-ef8c-4736-b9d4-319e627c1d9d\") " pod="openshift-lws-operator/lws-controller-manager-ff9cccccb-ntpsv"
Sep 18 14:03:50.952176 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:03:50.952180    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7k8p2\" (UniqueName: \"kubernetes.io/projected/5ce17b25-ef8c-4736-b9d4-319e627c1d9d-kube-api-access-7k8p2\") pod \"lws-controller-manager-ff9cccccb-ntpsv\" (UID: \"5ce17b25-ef8c-4736-b9d4-319e627c1d9d\") " pod="openshift-lws-operator/lws-controller-manager-ff9cccccb-ntpsv"
Sep 18 14:03:50.952416 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:03:50.952399    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/5ce17b25-ef8c-4736-b9d4-319e627c1d9d-cert\") pod \"lws-controller-manager-ff9cccccb-ntpsv\" (UID: \"5ce17b25-ef8c-4736-b9d4-319e627c1d9d\") " pod="openshift-lws-operator/lws-controller-manager-ff9cccccb-ntpsv"
Sep 18 14:03:50.952466 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:03:50.952450    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/5ce17b25-ef8c-4736-b9d4-319e627c1d9d-manager-config\") pod \"lws-controller-manager-ff9cccccb-ntpsv\" (UID: \"5ce17b25-ef8c-4736-b9d4-319e627c1d9d\") " pod="openshift-lws-operator/lws-controller-manager-ff9cccccb-ntpsv"
Sep 18 14:03:50.953057 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:03:50.953011    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/5ce17b25-ef8c-4736-b9d4-319e627c1d9d-manager-config\") pod \"lws-controller-manager-ff9cccccb-ntpsv\" (UID: \"5ce17b25-ef8c-4736-b9d4-319e627c1d9d\") " pod="openshift-lws-operator/lws-controller-manager-ff9cccccb-ntpsv"
Sep 18 14:03:50.954692 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:03:50.954665    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/5ce17b25-ef8c-4736-b9d4-319e627c1d9d-metrics-cert\") pod \"lws-controller-manager-ff9cccccb-ntpsv\" (UID: \"5ce17b25-ef8c-4736-b9d4-319e627c1d9d\") " pod="openshift-lws-operator/lws-controller-manager-ff9cccccb-ntpsv"
Sep 18 14:03:50.954790 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:03:50.954707    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/5ce17b25-ef8c-4736-b9d4-319e627c1d9d-cert\") pod \"lws-controller-manager-ff9cccccb-ntpsv\" (UID: \"5ce17b25-ef8c-4736-b9d4-319e627c1d9d\") " pod="openshift-lws-operator/lws-controller-manager-ff9cccccb-ntpsv"
Sep 18 14:03:50.963127 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:03:50.963108    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7k8p2\" (UniqueName: \"kubernetes.io/projected/5ce17b25-ef8c-4736-b9d4-319e627c1d9d-kube-api-access-7k8p2\") pod \"lws-controller-manager-ff9cccccb-ntpsv\" (UID: \"5ce17b25-ef8c-4736-b9d4-319e627c1d9d\") " pod="openshift-lws-operator/lws-controller-manager-ff9cccccb-ntpsv"
Sep 18 14:03:51.100840 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:03:51.100803    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-ff9cccccb-ntpsv"
Sep 18 14:03:51.379968 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:03:51.379874    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-ff9cccccb-ntpsv"]
Sep 18 14:03:51.382662 ip-10-0-129-16 kubenswrapper[2565]: W0918 14:03:51.382632    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5ce17b25_ef8c_4736_b9d4_319e627c1d9d.slice/crio-f3e91366c4e474a459c4a810f37623d05c53daa26d0c5828b37092d1a4b7ec98 WatchSource:0}: Error finding container f3e91366c4e474a459c4a810f37623d05c53daa26d0c5828b37092d1a4b7ec98: Status 404 returned error can't find the container with id f3e91366c4e474a459c4a810f37623d05c53daa26d0c5828b37092d1a4b7ec98
Sep 18 14:03:52.315435 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:03:52.315395    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-ff9cccccb-ntpsv" event={"ID":"5ce17b25-ef8c-4736-b9d4-319e627c1d9d","Type":"ContainerStarted","Data":"f3e91366c4e474a459c4a810f37623d05c53daa26d0c5828b37092d1a4b7ec98"}
Sep 18 14:03:53.827006 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:03:53.826978    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-bvqls_0b0463f5-d647-4a38-b752-44afba77daff/ovn-acl-logging/0.log"
Sep 18 14:03:53.827006 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:03:53.827004    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-bvqls_0b0463f5-d647-4a38-b752-44afba77daff/ovn-acl-logging/0.log"
Sep 18 14:03:53.831350 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:03:53.831326    2565 kubelet.go:1648] "Image garbage collection succeeded"
Sep 18 14:03:55.366650 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:03:55.366620    2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-kl5k8"]
Sep 18 14:03:55.374975 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:03:55.374952    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-kl5k8"
Sep 18 14:03:55.393830 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:03:55.393797    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\""
Sep 18 14:03:55.393972 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:03:55.393956    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\""
Sep 18 14:03:55.394914 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:03:55.394887    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\""
Sep 18 14:03:55.395027 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:03:55.394935    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"data-science-gateway-data-science-gateway-class-dockercfg-w4v85\""
Sep 18 14:03:55.435715 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:03:55.435674    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-kl5k8"]
Sep 18 14:03:55.496444 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:03:55.496405    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/17bd7b6a-c1c7-4c39-972d-78c6b3f9d996-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-kl5k8\" (UID: \"17bd7b6a-c1c7-4c39-972d-78c6b3f9d996\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-kl5k8"
Sep 18 14:03:55.496652 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:03:55.496470    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/17bd7b6a-c1c7-4c39-972d-78c6b3f9d996-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-kl5k8\" (UID: \"17bd7b6a-c1c7-4c39-972d-78c6b3f9d996\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-kl5k8"
Sep 18 14:03:55.496652 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:03:55.496603    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/17bd7b6a-c1c7-4c39-972d-78c6b3f9d996-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-kl5k8\" (UID: \"17bd7b6a-c1c7-4c39-972d-78c6b3f9d996\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-kl5k8"
Sep 18 14:03:55.496652 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:03:55.496634    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/17bd7b6a-c1c7-4c39-972d-78c6b3f9d996-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-kl5k8\" (UID: \"17bd7b6a-c1c7-4c39-972d-78c6b3f9d996\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-kl5k8"
Sep 18 14:03:55.496828 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:03:55.496656    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/17bd7b6a-c1c7-4c39-972d-78c6b3f9d996-istio-data\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-kl5k8\" (UID: \"17bd7b6a-c1c7-4c39-972d-78c6b3f9d996\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-kl5k8"
Sep 18 14:03:55.496828 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:03:55.496677    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5plx9\" (UniqueName: \"kubernetes.io/projected/17bd7b6a-c1c7-4c39-972d-78c6b3f9d996-kube-api-access-5plx9\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-kl5k8\" (UID: \"17bd7b6a-c1c7-4c39-972d-78c6b3f9d996\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-kl5k8"
Sep 18 14:03:55.496828 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:03:55.496750    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/17bd7b6a-c1c7-4c39-972d-78c6b3f9d996-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-kl5k8\" (UID: \"17bd7b6a-c1c7-4c39-972d-78c6b3f9d996\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-kl5k8"
Sep 18 14:03:55.496828 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:03:55.496791    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/17bd7b6a-c1c7-4c39-972d-78c6b3f9d996-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-kl5k8\" (UID: \"17bd7b6a-c1c7-4c39-972d-78c6b3f9d996\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-kl5k8"
Sep 18 14:03:55.496828 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:03:55.496823    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/17bd7b6a-c1c7-4c39-972d-78c6b3f9d996-istio-token\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-kl5k8\" (UID: \"17bd7b6a-c1c7-4c39-972d-78c6b3f9d996\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-kl5k8"
Sep 18 14:03:55.597510 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:03:55.597480    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/17bd7b6a-c1c7-4c39-972d-78c6b3f9d996-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-kl5k8\" (UID: \"17bd7b6a-c1c7-4c39-972d-78c6b3f9d996\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-kl5k8"
Sep 18 14:03:55.597707 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:03:55.597529    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/17bd7b6a-c1c7-4c39-972d-78c6b3f9d996-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-kl5k8\" (UID: \"17bd7b6a-c1c7-4c39-972d-78c6b3f9d996\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-kl5k8"
Sep 18 14:03:55.597707 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:03:55.597574    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/17bd7b6a-c1c7-4c39-972d-78c6b3f9d996-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-kl5k8\" (UID: \"17bd7b6a-c1c7-4c39-972d-78c6b3f9d996\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-kl5k8"
Sep 18 14:03:55.597707 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:03:55.597596    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/17bd7b6a-c1c7-4c39-972d-78c6b3f9d996-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-kl5k8\" (UID: \"17bd7b6a-c1c7-4c39-972d-78c6b3f9d996\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-kl5k8"
Sep 18 14:03:55.597707 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:03:55.597620    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/17bd7b6a-c1c7-4c39-972d-78c6b3f9d996-istio-data\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-kl5k8\" (UID: \"17bd7b6a-c1c7-4c39-972d-78c6b3f9d996\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-kl5k8"
Sep 18 14:03:55.597707 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:03:55.597645    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5plx9\" (UniqueName: \"kubernetes.io/projected/17bd7b6a-c1c7-4c39-972d-78c6b3f9d996-kube-api-access-5plx9\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-kl5k8\" (UID: \"17bd7b6a-c1c7-4c39-972d-78c6b3f9d996\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-kl5k8"
Sep 18 14:03:55.597707 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:03:55.597694    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/17bd7b6a-c1c7-4c39-972d-78c6b3f9d996-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-kl5k8\" (UID: \"17bd7b6a-c1c7-4c39-972d-78c6b3f9d996\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-kl5k8"
Sep 18 14:03:55.598011 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:03:55.597731    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/17bd7b6a-c1c7-4c39-972d-78c6b3f9d996-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-kl5k8\" (UID: \"17bd7b6a-c1c7-4c39-972d-78c6b3f9d996\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-kl5k8"
Sep 18 14:03:55.598011 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:03:55.597773    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/17bd7b6a-c1c7-4c39-972d-78c6b3f9d996-istio-token\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-kl5k8\" (UID: \"17bd7b6a-c1c7-4c39-972d-78c6b3f9d996\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-kl5k8"
Sep 18 14:03:55.598011 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:03:55.597945    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/17bd7b6a-c1c7-4c39-972d-78c6b3f9d996-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-kl5k8\" (UID: \"17bd7b6a-c1c7-4c39-972d-78c6b3f9d996\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-kl5k8"
Sep 18 14:03:55.598011 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:03:55.597961    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/17bd7b6a-c1c7-4c39-972d-78c6b3f9d996-istio-data\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-kl5k8\" (UID: \"17bd7b6a-c1c7-4c39-972d-78c6b3f9d996\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-kl5k8"
Sep 18 14:03:55.598240 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:03:55.598127    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/17bd7b6a-c1c7-4c39-972d-78c6b3f9d996-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-kl5k8\" (UID: \"17bd7b6a-c1c7-4c39-972d-78c6b3f9d996\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-kl5k8"
Sep 18 14:03:55.598347 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:03:55.598319    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/17bd7b6a-c1c7-4c39-972d-78c6b3f9d996-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-kl5k8\" (UID: \"17bd7b6a-c1c7-4c39-972d-78c6b3f9d996\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-kl5k8"
Sep 18 14:03:55.598538 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:03:55.598507    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/17bd7b6a-c1c7-4c39-972d-78c6b3f9d996-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-kl5k8\" (UID: \"17bd7b6a-c1c7-4c39-972d-78c6b3f9d996\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-kl5k8"
Sep 18 14:03:55.600100 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:03:55.600080    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/17bd7b6a-c1c7-4c39-972d-78c6b3f9d996-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-kl5k8\" (UID: \"17bd7b6a-c1c7-4c39-972d-78c6b3f9d996\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-kl5k8"
Sep 18 14:03:55.600207 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:03:55.600189    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/17bd7b6a-c1c7-4c39-972d-78c6b3f9d996-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-kl5k8\" (UID: \"17bd7b6a-c1c7-4c39-972d-78c6b3f9d996\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-kl5k8"
Sep 18 14:03:55.624133 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:03:55.624066    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/17bd7b6a-c1c7-4c39-972d-78c6b3f9d996-istio-token\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-kl5k8\" (UID: \"17bd7b6a-c1c7-4c39-972d-78c6b3f9d996\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-kl5k8"
Sep 18 14:03:55.624355 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:03:55.624334    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5plx9\" (UniqueName: \"kubernetes.io/projected/17bd7b6a-c1c7-4c39-972d-78c6b3f9d996-kube-api-access-5plx9\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-kl5k8\" (UID: \"17bd7b6a-c1c7-4c39-972d-78c6b3f9d996\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-kl5k8"
Sep 18 14:03:55.687193 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:03:55.687159    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-kl5k8"
Sep 18 14:03:56.242536 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:03:56.242511    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-kl5k8"]
Sep 18 14:03:56.244739 ip-10-0-129-16 kubenswrapper[2565]: W0918 14:03:56.244709    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod17bd7b6a_c1c7_4c39_972d_78c6b3f9d996.slice/crio-9072a38a40f5c7cd3bbbfec4641c64348cc26ba1b7e895f584ff796440eedb42 WatchSource:0}: Error finding container 9072a38a40f5c7cd3bbbfec4641c64348cc26ba1b7e895f584ff796440eedb42: Status 404 returned error can't find the container with id 9072a38a40f5c7cd3bbbfec4641c64348cc26ba1b7e895f584ff796440eedb42
Sep 18 14:03:56.246791 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:03:56.246775    2565 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 18 14:03:56.336800 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:03:56.336762    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-kl5k8" event={"ID":"17bd7b6a-c1c7-4c39-972d-78c6b3f9d996","Type":"ContainerStarted","Data":"9072a38a40f5c7cd3bbbfec4641c64348cc26ba1b7e895f584ff796440eedb42"}
Sep 18 14:03:56.338382 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:03:56.338356    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-ff9cccccb-ntpsv" event={"ID":"5ce17b25-ef8c-4736-b9d4-319e627c1d9d","Type":"ContainerStarted","Data":"62edb2ba95c7708bd19c49ad34a91fc3e2271197d629a14f04fd72420b8d113a"}
Sep 18 14:03:56.338511 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:03:56.338497    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-lws-operator/lws-controller-manager-ff9cccccb-ntpsv"
Sep 18 14:03:56.357561 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:03:56.357510    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-lws-operator/lws-controller-manager-ff9cccccb-ntpsv" podStartSLOduration=1.709045844 podStartE2EDuration="6.357493835s" totalImagesPullingTime="4.648447991s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 14:03:50 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 14:03:56.357133015 +0000 UTC m=+304.531042697" watchObservedRunningTime="2026-09-18 14:03:56.357493835 +0000 UTC m=+304.531403543"
Sep 18 14:03:59.087823 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:03:59.087787    2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/odh-model-controller-7998c67dc4-jjhv5"]
Sep 18 14:03:59.091429 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:03:59.091409    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/odh-model-controller-7998c67dc4-jjhv5"
Sep 18 14:03:59.093826 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:03:59.093806    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"odh-model-controller-webhook-cert\""
Sep 18 14:03:59.094113 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:03:59.094092    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"odh-model-controller-dockercfg-nf4xw\""
Sep 18 14:03:59.110938 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:03:59.110913    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/odh-model-controller-7998c67dc4-jjhv5"]
Sep 18 14:03:59.153589 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:03:59.153555    2565 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 18 14:03:59.153677 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:03:59.153625    2565 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 18 14:03:59.153677 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:03:59.153652    2565 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 18 14:03:59.235114 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:03:59.235089    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/2c6f3c8f-9a24-4688-9ce5-8f6d2469cc37-cert\") pod \"odh-model-controller-7998c67dc4-jjhv5\" (UID: \"2c6f3c8f-9a24-4688-9ce5-8f6d2469cc37\") " pod="opendatahub/odh-model-controller-7998c67dc4-jjhv5"
Sep 18 14:03:59.235233 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:03:59.235129    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-m6gll\" (UniqueName: \"kubernetes.io/projected/2c6f3c8f-9a24-4688-9ce5-8f6d2469cc37-kube-api-access-m6gll\") pod \"odh-model-controller-7998c67dc4-jjhv5\" (UID: \"2c6f3c8f-9a24-4688-9ce5-8f6d2469cc37\") " pod="opendatahub/odh-model-controller-7998c67dc4-jjhv5"
Sep 18 14:03:59.336439 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:03:59.336409    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/2c6f3c8f-9a24-4688-9ce5-8f6d2469cc37-cert\") pod \"odh-model-controller-7998c67dc4-jjhv5\" (UID: \"2c6f3c8f-9a24-4688-9ce5-8f6d2469cc37\") " pod="opendatahub/odh-model-controller-7998c67dc4-jjhv5"
Sep 18 14:03:59.336644 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:03:59.336453    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-m6gll\" (UniqueName: \"kubernetes.io/projected/2c6f3c8f-9a24-4688-9ce5-8f6d2469cc37-kube-api-access-m6gll\") pod \"odh-model-controller-7998c67dc4-jjhv5\" (UID: \"2c6f3c8f-9a24-4688-9ce5-8f6d2469cc37\") " pod="opendatahub/odh-model-controller-7998c67dc4-jjhv5"
Sep 18 14:03:59.339419 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:03:59.339352    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/2c6f3c8f-9a24-4688-9ce5-8f6d2469cc37-cert\") pod \"odh-model-controller-7998c67dc4-jjhv5\" (UID: \"2c6f3c8f-9a24-4688-9ce5-8f6d2469cc37\") " pod="opendatahub/odh-model-controller-7998c67dc4-jjhv5"
Sep 18 14:03:59.346416 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:03:59.346387    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-m6gll\" (UniqueName: \"kubernetes.io/projected/2c6f3c8f-9a24-4688-9ce5-8f6d2469cc37-kube-api-access-m6gll\") pod \"odh-model-controller-7998c67dc4-jjhv5\" (UID: \"2c6f3c8f-9a24-4688-9ce5-8f6d2469cc37\") " pod="opendatahub/odh-model-controller-7998c67dc4-jjhv5"
Sep 18 14:03:59.357094 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:03:59.357065    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-kl5k8" event={"ID":"17bd7b6a-c1c7-4c39-972d-78c6b3f9d996","Type":"ContainerStarted","Data":"9be99cd2157cbc3d2fbe0bd82bd6d934dec1d1567c503c475b928bfe44dc34c6"}
Sep 18 14:03:59.379128 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:03:59.379080    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-kl5k8" podStartSLOduration=1.472729028 podStartE2EDuration="4.37906193s" totalImagesPullingTime="2.906332902s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 14:03:55 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 14:03:59.378325758 +0000 UTC m=+307.552235411" watchObservedRunningTime="2026-09-18 14:03:59.37906193 +0000 UTC m=+307.552971584"
Sep 18 14:03:59.402637 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:03:59.402607    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/odh-model-controller-7998c67dc4-jjhv5"
Sep 18 14:03:59.676397 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:03:59.676310    2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-rgjv6"]
Sep 18 14:03:59.680868 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:03:59.680842    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/model-serving-api-54d89bd5fc-rgjv6"
Sep 18 14:03:59.683859 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:03:59.683840    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"model-serving-api-tls\""
Sep 18 14:03:59.687280 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:03:59.687250    2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-kl5k8"
Sep 18 14:03:59.689992 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:03:59.689902    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/odh-model-controller-7998c67dc4-jjhv5"]
Sep 18 14:03:59.693642 ip-10-0-129-16 kubenswrapper[2565]: W0918 14:03:59.693608    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2c6f3c8f_9a24_4688_9ce5_8f6d2469cc37.slice/crio-f7396fee28df7bbc0149ee143c4d9d5ea4b963de6d6dbeb65550efe1821b1a33 WatchSource:0}: Error finding container f7396fee28df7bbc0149ee143c4d9d5ea4b963de6d6dbeb65550efe1821b1a33: Status 404 returned error can't find the container with id f7396fee28df7bbc0149ee143c4d9d5ea4b963de6d6dbeb65550efe1821b1a33
Sep 18 14:03:59.693913 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:03:59.693893    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-rgjv6"]
Sep 18 14:03:59.839928 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:03:59.839894    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e4fbc7b3-7e66-404b-a98b-231f84048b51-tls-certs\") pod \"model-serving-api-54d89bd5fc-rgjv6\" (UID: \"e4fbc7b3-7e66-404b-a98b-231f84048b51\") " pod="opendatahub/model-serving-api-54d89bd5fc-rgjv6"
Sep 18 14:03:59.840139 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:03:59.839942    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7vcf2\" (UniqueName: \"kubernetes.io/projected/e4fbc7b3-7e66-404b-a98b-231f84048b51-kube-api-access-7vcf2\") pod \"model-serving-api-54d89bd5fc-rgjv6\" (UID: \"e4fbc7b3-7e66-404b-a98b-231f84048b51\") " pod="opendatahub/model-serving-api-54d89bd5fc-rgjv6"
Sep 18 14:03:59.940816 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:03:59.940729    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e4fbc7b3-7e66-404b-a98b-231f84048b51-tls-certs\") pod \"model-serving-api-54d89bd5fc-rgjv6\" (UID: \"e4fbc7b3-7e66-404b-a98b-231f84048b51\") " pod="opendatahub/model-serving-api-54d89bd5fc-rgjv6"
Sep 18 14:03:59.940816 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:03:59.940782    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7vcf2\" (UniqueName: \"kubernetes.io/projected/e4fbc7b3-7e66-404b-a98b-231f84048b51-kube-api-access-7vcf2\") pod \"model-serving-api-54d89bd5fc-rgjv6\" (UID: \"e4fbc7b3-7e66-404b-a98b-231f84048b51\") " pod="opendatahub/model-serving-api-54d89bd5fc-rgjv6"
Sep 18 14:03:59.941074 ip-10-0-129-16 kubenswrapper[2565]: E0918 14:03:59.940873    2565 secret.go:189] Couldn't get secret opendatahub/model-serving-api-tls: secret "model-serving-api-tls" not found
Sep 18 14:03:59.941074 ip-10-0-129-16 kubenswrapper[2565]: E0918 14:03:59.940936    2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e4fbc7b3-7e66-404b-a98b-231f84048b51-tls-certs podName:e4fbc7b3-7e66-404b-a98b-231f84048b51 nodeName:}" failed. No retries permitted until 2026-09-18 14:04:00.440917966 +0000 UTC m=+308.614827618 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "tls-certs" (UniqueName: "kubernetes.io/secret/e4fbc7b3-7e66-404b-a98b-231f84048b51-tls-certs") pod "model-serving-api-54d89bd5fc-rgjv6" (UID: "e4fbc7b3-7e66-404b-a98b-231f84048b51") : secret "model-serving-api-tls" not found
Sep 18 14:03:59.952398 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:03:59.952369    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7vcf2\" (UniqueName: \"kubernetes.io/projected/e4fbc7b3-7e66-404b-a98b-231f84048b51-kube-api-access-7vcf2\") pod \"model-serving-api-54d89bd5fc-rgjv6\" (UID: \"e4fbc7b3-7e66-404b-a98b-231f84048b51\") " pod="opendatahub/model-serving-api-54d89bd5fc-rgjv6"
Sep 18 14:04:00.364741 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:04:00.364680    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/odh-model-controller-7998c67dc4-jjhv5" event={"ID":"2c6f3c8f-9a24-4688-9ce5-8f6d2469cc37","Type":"ContainerStarted","Data":"f7396fee28df7bbc0149ee143c4d9d5ea4b963de6d6dbeb65550efe1821b1a33"}
Sep 18 14:04:00.445748 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:04:00.445369    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e4fbc7b3-7e66-404b-a98b-231f84048b51-tls-certs\") pod \"model-serving-api-54d89bd5fc-rgjv6\" (UID: \"e4fbc7b3-7e66-404b-a98b-231f84048b51\") " pod="opendatahub/model-serving-api-54d89bd5fc-rgjv6"
Sep 18 14:04:00.448655 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:04:00.448622    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e4fbc7b3-7e66-404b-a98b-231f84048b51-tls-certs\") pod \"model-serving-api-54d89bd5fc-rgjv6\" (UID: \"e4fbc7b3-7e66-404b-a98b-231f84048b51\") " pod="opendatahub/model-serving-api-54d89bd5fc-rgjv6"
Sep 18 14:04:00.591839 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:04:00.591803    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/model-serving-api-54d89bd5fc-rgjv6"
Sep 18 14:04:00.693994 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:04:00.693966    2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-kl5k8"
Sep 18 14:04:00.875891 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:04:00.875865    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-rgjv6"]
Sep 18 14:04:00.877701 ip-10-0-129-16 kubenswrapper[2565]: W0918 14:04:00.877671    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode4fbc7b3_7e66_404b_a98b_231f84048b51.slice/crio-4ad18884cb532eb6f7090c688923e33dbbedc54fe99ed79b40340db87c5b5b33 WatchSource:0}: Error finding container 4ad18884cb532eb6f7090c688923e33dbbedc54fe99ed79b40340db87c5b5b33: Status 404 returned error can't find the container with id 4ad18884cb532eb6f7090c688923e33dbbedc54fe99ed79b40340db87c5b5b33
Sep 18 14:04:01.372313 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:04:01.372271    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/model-serving-api-54d89bd5fc-rgjv6" event={"ID":"e4fbc7b3-7e66-404b-a98b-231f84048b51","Type":"ContainerStarted","Data":"4ad18884cb532eb6f7090c688923e33dbbedc54fe99ed79b40340db87c5b5b33"}
Sep 18 14:04:01.372788 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:04:01.372532    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-kl5k8"
Sep 18 14:04:01.373676 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:04:01.373652    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-kl5k8"
Sep 18 14:04:03.384469 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:04:03.384431    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/odh-model-controller-7998c67dc4-jjhv5" event={"ID":"2c6f3c8f-9a24-4688-9ce5-8f6d2469cc37","Type":"ContainerStarted","Data":"b922937a64b8b1d9adfca5d41aa21decc70d24238939aba4a73f560497f8c0dd"}
Sep 18 14:04:03.403060 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:04:03.402992    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/odh-model-controller-7998c67dc4-jjhv5" podStartSLOduration=1.6976605999999999 podStartE2EDuration="4.402970842s" totalImagesPullingTime="2.705310242s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 14:03:59 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 14:04:03.400908976 +0000 UTC m=+311.574818660" watchObservedRunningTime="2026-09-18 14:04:03.402970842 +0000 UTC m=+311.576880503"
Sep 18 14:04:04.390711 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:04:04.390672    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/model-serving-api-54d89bd5fc-rgjv6" event={"ID":"e4fbc7b3-7e66-404b-a98b-231f84048b51","Type":"ContainerStarted","Data":"13b7166ece0d9433dd9757b440088b0adbefb31bf341886a5698e5381d640cab"}
Sep 18 14:04:04.391136 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:04:04.390830    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/model-serving-api-54d89bd5fc-rgjv6"
Sep 18 14:04:04.391136 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:04:04.390873    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/odh-model-controller-7998c67dc4-jjhv5"
Sep 18 14:04:04.424865 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:04:04.424811    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/model-serving-api-54d89bd5fc-rgjv6" podStartSLOduration=2.064821371 podStartE2EDuration="5.424795513s" totalImagesPullingTime="3.359974142s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 14:03:59 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 14:04:04.42427045 +0000 UTC m=+312.598180123" watchObservedRunningTime="2026-09-18 14:04:04.424795513 +0000 UTC m=+312.598705173"
Sep 18 14:04:07.346112 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:04:07.346083    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-lws-operator/lws-controller-manager-ff9cccccb-ntpsv"
Sep 18 14:04:09.454721 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:04:09.454693    2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/kserve-controller-manager-d6f4b65b8-8rpmc"]
Sep 18 14:04:09.461784 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:04:09.461761    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/kserve-controller-manager-d6f4b65b8-8rpmc"
Sep 18 14:04:09.464397 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:04:09.464369    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"kserve-webhook-server-cert\""
Sep 18 14:04:09.466380 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:04:09.466360    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"kserve-controller-manager-dockercfg-6xscp\""
Sep 18 14:04:09.480470 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:04:09.480447    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/kserve-controller-manager-d6f4b65b8-8rpmc"]
Sep 18 14:04:09.634503 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:04:09.634470    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cw9mq\" (UniqueName: \"kubernetes.io/projected/38a34857-f87d-4673-b963-15634897fc08-kube-api-access-cw9mq\") pod \"kserve-controller-manager-d6f4b65b8-8rpmc\" (UID: \"38a34857-f87d-4673-b963-15634897fc08\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-8rpmc"
Sep 18 14:04:09.634503 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:04:09.634510    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/38a34857-f87d-4673-b963-15634897fc08-cert\") pod \"kserve-controller-manager-d6f4b65b8-8rpmc\" (UID: \"38a34857-f87d-4673-b963-15634897fc08\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-8rpmc"
Sep 18 14:04:09.736088 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:04:09.735965    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cw9mq\" (UniqueName: \"kubernetes.io/projected/38a34857-f87d-4673-b963-15634897fc08-kube-api-access-cw9mq\") pod \"kserve-controller-manager-d6f4b65b8-8rpmc\" (UID: \"38a34857-f87d-4673-b963-15634897fc08\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-8rpmc"
Sep 18 14:04:09.736088 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:04:09.736013    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/38a34857-f87d-4673-b963-15634897fc08-cert\") pod \"kserve-controller-manager-d6f4b65b8-8rpmc\" (UID: \"38a34857-f87d-4673-b963-15634897fc08\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-8rpmc"
Sep 18 14:04:09.738569 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:04:09.738535    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/38a34857-f87d-4673-b963-15634897fc08-cert\") pod \"kserve-controller-manager-d6f4b65b8-8rpmc\" (UID: \"38a34857-f87d-4673-b963-15634897fc08\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-8rpmc"
Sep 18 14:04:09.751074 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:04:09.751050    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cw9mq\" (UniqueName: \"kubernetes.io/projected/38a34857-f87d-4673-b963-15634897fc08-kube-api-access-cw9mq\") pod \"kserve-controller-manager-d6f4b65b8-8rpmc\" (UID: \"38a34857-f87d-4673-b963-15634897fc08\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-8rpmc"
Sep 18 14:04:09.772192 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:04:09.772163    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/kserve-controller-manager-d6f4b65b8-8rpmc"
Sep 18 14:04:10.062789 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:04:10.062762    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/kserve-controller-manager-d6f4b65b8-8rpmc"]
Sep 18 14:04:10.064488 ip-10-0-129-16 kubenswrapper[2565]: W0918 14:04:10.064458    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod38a34857_f87d_4673_b963_15634897fc08.slice/crio-03c3c729104dc6179c9b7e5640fd7ac0da19f5071c7116a237c25633fb7058fb WatchSource:0}: Error finding container 03c3c729104dc6179c9b7e5640fd7ac0da19f5071c7116a237c25633fb7058fb: Status 404 returned error can't find the container with id 03c3c729104dc6179c9b7e5640fd7ac0da19f5071c7116a237c25633fb7058fb
Sep 18 14:04:10.420488 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:04:10.420405    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/kserve-controller-manager-d6f4b65b8-8rpmc" event={"ID":"38a34857-f87d-4673-b963-15634897fc08","Type":"ContainerStarted","Data":"03c3c729104dc6179c9b7e5640fd7ac0da19f5071c7116a237c25633fb7058fb"}
Sep 18 14:04:12.431533 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:04:12.431497    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/kserve-controller-manager-d6f4b65b8-8rpmc" event={"ID":"38a34857-f87d-4673-b963-15634897fc08","Type":"ContainerStarted","Data":"ccb90068458485ba469ec34631f476365d1c8bf06f03500980f229222644af9b"}
Sep 18 14:04:12.431941 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:04:12.431628    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/kserve-controller-manager-d6f4b65b8-8rpmc"
Sep 18 14:04:12.479544 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:04:12.479496    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/kserve-controller-manager-d6f4b65b8-8rpmc" podStartSLOduration=1.234692517 podStartE2EDuration="3.479478756s" totalImagesPullingTime="2.244786239s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 14:04:09 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 14:04:12.477836264 +0000 UTC m=+320.651745926" watchObservedRunningTime="2026-09-18 14:04:12.479478756 +0000 UTC m=+320.653388416"
Sep 18 14:04:15.397982 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:04:15.397954    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/odh-model-controller-7998c67dc4-jjhv5"
Sep 18 14:04:15.400044 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:04:15.400011    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/model-serving-api-54d89bd5fc-rgjv6"
Sep 18 14:04:43.441457 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:04:43.441382    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/kserve-controller-manager-d6f4b65b8-8rpmc"
Sep 18 14:05:23.427779 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:05:23.427742    2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-74495c69f8-89kt4"]
Sep 18 14:05:23.430756 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:05:23.430740    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-89kt4"
Sep 18 14:05:23.433979 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:05:23.433962    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\""
Sep 18 14:05:23.433979 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:05:23.433971    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"limitador-operator-controller-manager-dockercfg-znllt\""
Sep 18 14:05:23.434449 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:05:23.434434    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\""
Sep 18 14:05:23.444954 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:05:23.444932    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-74495c69f8-89kt4"]
Sep 18 14:05:23.490876 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:05:23.490841    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qbwvd\" (UniqueName: \"kubernetes.io/projected/18f5ffd7-6621-4e7c-bb6d-6f26ee55e37a-kube-api-access-qbwvd\") pod \"limitador-operator-controller-manager-74495c69f8-89kt4\" (UID: \"18f5ffd7-6621-4e7c-bb6d-6f26ee55e37a\") " pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-89kt4"
Sep 18 14:05:23.591798 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:05:23.591761    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qbwvd\" (UniqueName: \"kubernetes.io/projected/18f5ffd7-6621-4e7c-bb6d-6f26ee55e37a-kube-api-access-qbwvd\") pod \"limitador-operator-controller-manager-74495c69f8-89kt4\" (UID: \"18f5ffd7-6621-4e7c-bb6d-6f26ee55e37a\") " pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-89kt4"
Sep 18 14:05:23.605995 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:05:23.605968    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qbwvd\" (UniqueName: \"kubernetes.io/projected/18f5ffd7-6621-4e7c-bb6d-6f26ee55e37a-kube-api-access-qbwvd\") pod \"limitador-operator-controller-manager-74495c69f8-89kt4\" (UID: \"18f5ffd7-6621-4e7c-bb6d-6f26ee55e37a\") " pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-89kt4"
Sep 18 14:05:23.742313 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:05:23.742218    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-89kt4"
Sep 18 14:05:24.016535 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:05:24.016449    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-74495c69f8-89kt4"]
Sep 18 14:05:24.019906 ip-10-0-129-16 kubenswrapper[2565]: W0918 14:05:24.019875    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod18f5ffd7_6621_4e7c_bb6d_6f26ee55e37a.slice/crio-1bc54c04fbbc87bec734a75d82dea276df4a2453e096b5ba2bcd25781e982faf WatchSource:0}: Error finding container 1bc54c04fbbc87bec734a75d82dea276df4a2453e096b5ba2bcd25781e982faf: Status 404 returned error can't find the container with id 1bc54c04fbbc87bec734a75d82dea276df4a2453e096b5ba2bcd25781e982faf
Sep 18 14:05:24.770475 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:05:24.770438    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-89kt4" event={"ID":"18f5ffd7-6621-4e7c-bb6d-6f26ee55e37a","Type":"ContainerStarted","Data":"1bc54c04fbbc87bec734a75d82dea276df4a2453e096b5ba2bcd25781e982faf"}
Sep 18 14:05:27.792579 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:05:27.792549    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-89kt4" event={"ID":"18f5ffd7-6621-4e7c-bb6d-6f26ee55e37a","Type":"ContainerStarted","Data":"11b9ad09720cdad510d7e014b898e9ae9d76b4f614ac262c062a6a3c3972c3ee"}
Sep 18 14:05:27.793000 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:05:27.792686    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-89kt4"
Sep 18 14:05:27.811672 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:05:27.811613    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-89kt4" podStartSLOduration=2.003432594 podStartE2EDuration="4.811593686s" totalImagesPullingTime="2.808161092s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 14:05:23 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 14:05:27.811446583 +0000 UTC m=+395.985356256" watchObservedRunningTime="2026-09-18 14:05:27.811593686 +0000 UTC m=+395.985503368"
Sep 18 14:05:27.863480 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:05:27.863439    2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-8tttn"]
Sep 18 14:05:27.866689 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:05:27.866668    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-8tttn"
Sep 18 14:05:27.870458 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:05:27.870437    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-q692n\""
Sep 18 14:05:27.884468 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:05:27.884429    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-8tttn"]
Sep 18 14:05:27.938398 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:05:27.938363    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hvffg\" (UniqueName: \"kubernetes.io/projected/7aca40a5-1d85-4a2a-a4b5-3ea2182b7190-kube-api-access-hvffg\") pod \"kuadrant-operator-controller-manager-76f8f686bf-8tttn\" (UID: \"7aca40a5-1d85-4a2a-a4b5-3ea2182b7190\") " pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-8tttn"
Sep 18 14:05:27.938610 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:05:27.938500    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/7aca40a5-1d85-4a2a-a4b5-3ea2182b7190-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-76f8f686bf-8tttn\" (UID: \"7aca40a5-1d85-4a2a-a4b5-3ea2182b7190\") " pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-8tttn"
Sep 18 14:05:28.040041 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:05:28.039995    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/7aca40a5-1d85-4a2a-a4b5-3ea2182b7190-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-76f8f686bf-8tttn\" (UID: \"7aca40a5-1d85-4a2a-a4b5-3ea2182b7190\") " pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-8tttn"
Sep 18 14:05:28.040041 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:05:28.040044    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hvffg\" (UniqueName: \"kubernetes.io/projected/7aca40a5-1d85-4a2a-a4b5-3ea2182b7190-kube-api-access-hvffg\") pod \"kuadrant-operator-controller-manager-76f8f686bf-8tttn\" (UID: \"7aca40a5-1d85-4a2a-a4b5-3ea2182b7190\") " pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-8tttn"
Sep 18 14:05:28.040482 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:05:28.040429    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/7aca40a5-1d85-4a2a-a4b5-3ea2182b7190-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-76f8f686bf-8tttn\" (UID: \"7aca40a5-1d85-4a2a-a4b5-3ea2182b7190\") " pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-8tttn"
Sep 18 14:05:28.050364 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:05:28.050279    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hvffg\" (UniqueName: \"kubernetes.io/projected/7aca40a5-1d85-4a2a-a4b5-3ea2182b7190-kube-api-access-hvffg\") pod \"kuadrant-operator-controller-manager-76f8f686bf-8tttn\" (UID: \"7aca40a5-1d85-4a2a-a4b5-3ea2182b7190\") " pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-8tttn"
Sep 18 14:05:28.177930 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:05:28.177887    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-8tttn"
Sep 18 14:05:28.458948 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:05:28.458868    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-8tttn"]
Sep 18 14:05:28.464467 ip-10-0-129-16 kubenswrapper[2565]: W0918 14:05:28.464435    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7aca40a5_1d85_4a2a_a4b5_3ea2182b7190.slice/crio-177d72ae388ee4ed8fdc55845a86662ac71c1b214b31fc43e6d73b7af1a6fbe0 WatchSource:0}: Error finding container 177d72ae388ee4ed8fdc55845a86662ac71c1b214b31fc43e6d73b7af1a6fbe0: Status 404 returned error can't find the container with id 177d72ae388ee4ed8fdc55845a86662ac71c1b214b31fc43e6d73b7af1a6fbe0
Sep 18 14:05:28.799245 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:05:28.799206    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-8tttn" event={"ID":"7aca40a5-1d85-4a2a-a4b5-3ea2182b7190","Type":"ContainerStarted","Data":"177d72ae388ee4ed8fdc55845a86662ac71c1b214b31fc43e6d73b7af1a6fbe0"}
Sep 18 14:05:32.822943 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:05:32.822908    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-8tttn" event={"ID":"7aca40a5-1d85-4a2a-a4b5-3ea2182b7190","Type":"ContainerStarted","Data":"eee69d9d82ef80fe7f2c2fccdcc55f3aa628550a34d8d3c2f520a179cc052c46"}
Sep 18 14:05:32.823372 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:05:32.823054    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-8tttn"
Sep 18 14:05:32.848560 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:05:32.848511    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-8tttn" podStartSLOduration=2.117383484 podStartE2EDuration="5.848493667s" totalImagesPullingTime="3.731110183s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 14:05:27 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 14:05:32.847115788 +0000 UTC m=+401.021025468" watchObservedRunningTime="2026-09-18 14:05:32.848493667 +0000 UTC m=+401.022403329"
Sep 18 14:05:38.802054 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:05:38.802005    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-89kt4"
Sep 18 14:05:43.829998 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:05:43.829961    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-8tttn"
Sep 18 14:05:45.920010 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:05:45.919970    2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-8tttn"]
Sep 18 14:05:45.920452 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:05:45.920230    2565 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-8tttn" podUID="7aca40a5-1d85-4a2a-a4b5-3ea2182b7190" containerName="manager" containerID="cri-o://eee69d9d82ef80fe7f2c2fccdcc55f3aa628550a34d8d3c2f520a179cc052c46" gracePeriod=2
Sep 18 14:05:45.930943 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:05:45.930912    2565 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-8tttn"]
Sep 18 14:05:45.941485 ip-10-0-129-16 kubenswrapper[2565]: E0918 14:05:45.941441    2565 status_manager.go:1041] "Failed to update status for pod" err="failed to patch status \"{\\\"metadata\\\":{\\\"uid\\\":\\\"7aca40a5-1d85-4a2a-a4b5-3ea2182b7190\\\"},\\\"status\\\":{\\\"$setElementOrder/conditions\\\":[{\\\"type\\\":\\\"PodReadyToStartContainers\\\"},{\\\"type\\\":\\\"Initialized\\\"},{\\\"type\\\":\\\"Ready\\\"},{\\\"type\\\":\\\"ContainersReady\\\"},{\\\"type\\\":\\\"PodScheduled\\\"}],\\\"conditions\\\":[{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"PodReadyToStartContainers\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"Initialized\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"Ready\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"ContainersReady\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"PodScheduled\\\"}],\\\"observedGeneration\\\":2}}\" for pod \"kuadrant-system\"/\"kuadrant-operator-controller-manager-76f8f686bf-8tttn\": pods \"kuadrant-operator-controller-manager-76f8f686bf-8tttn\" not found" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-8tttn"
Sep 18 14:05:45.958389 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:05:45.958351    2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-tpjwq"]
Sep 18 14:05:45.958769 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:05:45.958754    2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7aca40a5-1d85-4a2a-a4b5-3ea2182b7190" containerName="manager"
Sep 18 14:05:45.958769 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:05:45.958769    2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="7aca40a5-1d85-4a2a-a4b5-3ea2182b7190" containerName="manager"
Sep 18 14:05:45.958912 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:05:45.958855    2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="7aca40a5-1d85-4a2a-a4b5-3ea2182b7190" containerName="manager"
Sep 18 14:05:45.974380 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:05:45.974354    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-tpjwq"
Sep 18 14:05:45.980629 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:05:45.980590    2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-74495c69f8-89kt4"]
Sep 18 14:05:45.980814 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:05:45.980793    2565 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-89kt4" podUID="18f5ffd7-6621-4e7c-bb6d-6f26ee55e37a" containerName="manager" containerID="cri-o://11b9ad09720cdad510d7e014b898e9ae9d76b4f614ac262c062a6a3c3972c3ee" gracePeriod=2
Sep 18 14:05:45.982672 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:05:45.982641    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-tpjwq"]
Sep 18 14:05:45.988469 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:05:45.988436    2565 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-74495c69f8-89kt4"]
Sep 18 14:05:46.011270 ip-10-0-129-16 kubenswrapper[2565]: E0918 14:05:46.011062    2565 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-76f8f686bf-8tttn\" is forbidden: User \"system:node:ip-10-0-129-16.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-129-16.ec2.internal' and this object" podUID="7aca40a5-1d85-4a2a-a4b5-3ea2182b7190" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-8tttn"
Sep 18 14:05:46.013977 ip-10-0-129-16 kubenswrapper[2565]: E0918 14:05:46.013942    2565 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-74495c69f8-89kt4\" is forbidden: User \"system:node:ip-10-0-129-16.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-129-16.ec2.internal' and this object" podUID="18f5ffd7-6621-4e7c-bb6d-6f26ee55e37a" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-89kt4"
Sep 18 14:05:46.014857 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:05:46.014832    2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-74495c69f8-llhlr"]
Sep 18 14:05:46.015248 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:05:46.015234    2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="18f5ffd7-6621-4e7c-bb6d-6f26ee55e37a" containerName="manager"
Sep 18 14:05:46.015248 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:05:46.015249    2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="18f5ffd7-6621-4e7c-bb6d-6f26ee55e37a" containerName="manager"
Sep 18 14:05:46.015346 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:05:46.015322    2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="18f5ffd7-6621-4e7c-bb6d-6f26ee55e37a" containerName="manager"
Sep 18 14:05:46.017435 ip-10-0-129-16 kubenswrapper[2565]: E0918 14:05:46.017410    2565 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-76f8f686bf-8tttn\" is forbidden: User \"system:node:ip-10-0-129-16.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-129-16.ec2.internal' and this object" podUID="7aca40a5-1d85-4a2a-a4b5-3ea2182b7190" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-8tttn"
Sep 18 14:05:46.035646 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:05:46.035611    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-74495c69f8-llhlr"]
Sep 18 14:05:46.035745 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:05:46.035634    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-llhlr"
Sep 18 14:05:46.072253 ip-10-0-129-16 kubenswrapper[2565]: E0918 14:05:46.071918    2565 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-74495c69f8-89kt4\" is forbidden: User \"system:node:ip-10-0-129-16.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-129-16.ec2.internal' and this object" podUID="18f5ffd7-6621-4e7c-bb6d-6f26ee55e37a" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-89kt4"
Sep 18 14:05:46.074484 ip-10-0-129-16 kubenswrapper[2565]: E0918 14:05:46.074460    2565 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-76f8f686bf-8tttn\" is forbidden: User \"system:node:ip-10-0-129-16.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-129-16.ec2.internal' and this object" podUID="7aca40a5-1d85-4a2a-a4b5-3ea2182b7190" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-8tttn"
Sep 18 14:05:46.112807 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:05:46.112770    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6qt4g\" (UniqueName: \"kubernetes.io/projected/191da26b-c57c-425b-9727-8709bafd68e0-kube-api-access-6qt4g\") pod \"kuadrant-operator-controller-manager-76f8f686bf-tpjwq\" (UID: \"191da26b-c57c-425b-9727-8709bafd68e0\") " pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-tpjwq"
Sep 18 14:05:46.112954 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:05:46.112893    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/191da26b-c57c-425b-9727-8709bafd68e0-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-76f8f686bf-tpjwq\" (UID: \"191da26b-c57c-425b-9727-8709bafd68e0\") " pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-tpjwq"
Sep 18 14:05:46.113026 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:05:46.112983    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kxthl\" (UniqueName: \"kubernetes.io/projected/1f5c710a-07d4-4029-b265-6d73b6fb1c53-kube-api-access-kxthl\") pod \"limitador-operator-controller-manager-74495c69f8-llhlr\" (UID: \"1f5c710a-07d4-4029-b265-6d73b6fb1c53\") " pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-llhlr"
Sep 18 14:05:46.213631 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:05:46.213539    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6qt4g\" (UniqueName: \"kubernetes.io/projected/191da26b-c57c-425b-9727-8709bafd68e0-kube-api-access-6qt4g\") pod \"kuadrant-operator-controller-manager-76f8f686bf-tpjwq\" (UID: \"191da26b-c57c-425b-9727-8709bafd68e0\") " pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-tpjwq"
Sep 18 14:05:46.213631 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:05:46.213616    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/191da26b-c57c-425b-9727-8709bafd68e0-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-76f8f686bf-tpjwq\" (UID: \"191da26b-c57c-425b-9727-8709bafd68e0\") " pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-tpjwq"
Sep 18 14:05:46.213842 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:05:46.213658    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kxthl\" (UniqueName: \"kubernetes.io/projected/1f5c710a-07d4-4029-b265-6d73b6fb1c53-kube-api-access-kxthl\") pod \"limitador-operator-controller-manager-74495c69f8-llhlr\" (UID: \"1f5c710a-07d4-4029-b265-6d73b6fb1c53\") " pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-llhlr"
Sep 18 14:05:46.214135 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:05:46.214109    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/191da26b-c57c-425b-9727-8709bafd68e0-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-76f8f686bf-tpjwq\" (UID: \"191da26b-c57c-425b-9727-8709bafd68e0\") " pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-tpjwq"
Sep 18 14:05:46.223116 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:05:46.223093    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kxthl\" (UniqueName: \"kubernetes.io/projected/1f5c710a-07d4-4029-b265-6d73b6fb1c53-kube-api-access-kxthl\") pod \"limitador-operator-controller-manager-74495c69f8-llhlr\" (UID: \"1f5c710a-07d4-4029-b265-6d73b6fb1c53\") " pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-llhlr"
Sep 18 14:05:46.224690 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:05:46.224671    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6qt4g\" (UniqueName: \"kubernetes.io/projected/191da26b-c57c-425b-9727-8709bafd68e0-kube-api-access-6qt4g\") pod \"kuadrant-operator-controller-manager-76f8f686bf-tpjwq\" (UID: \"191da26b-c57c-425b-9727-8709bafd68e0\") " pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-tpjwq"
Sep 18 14:05:46.272575 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:05:46.272548    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-8tttn"
Sep 18 14:05:46.274960 ip-10-0-129-16 kubenswrapper[2565]: E0918 14:05:46.274930    2565 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-74495c69f8-89kt4\" is forbidden: User \"system:node:ip-10-0-129-16.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-129-16.ec2.internal' and this object" podUID="18f5ffd7-6621-4e7c-bb6d-6f26ee55e37a" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-89kt4"
Sep 18 14:05:46.277107 ip-10-0-129-16 kubenswrapper[2565]: E0918 14:05:46.277084    2565 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-76f8f686bf-8tttn\" is forbidden: User \"system:node:ip-10-0-129-16.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-129-16.ec2.internal' and this object" podUID="7aca40a5-1d85-4a2a-a4b5-3ea2182b7190" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-8tttn"
Sep 18 14:05:46.286335 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:05:46.286316    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-89kt4"
Sep 18 14:05:46.288345 ip-10-0-129-16 kubenswrapper[2565]: E0918 14:05:46.288324    2565 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-74495c69f8-89kt4\" is forbidden: User \"system:node:ip-10-0-129-16.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-129-16.ec2.internal' and this object" podUID="18f5ffd7-6621-4e7c-bb6d-6f26ee55e37a" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-89kt4"
Sep 18 14:05:46.290224 ip-10-0-129-16 kubenswrapper[2565]: E0918 14:05:46.290202    2565 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-76f8f686bf-8tttn\" is forbidden: User \"system:node:ip-10-0-129-16.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-129-16.ec2.internal' and this object" podUID="7aca40a5-1d85-4a2a-a4b5-3ea2182b7190" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-8tttn"
Sep 18 14:05:46.407009 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:05:46.406976    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-tpjwq"
Sep 18 14:05:46.414150 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:05:46.414119    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-llhlr"
Sep 18 14:05:46.415341 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:05:46.415321    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-qbwvd\" (UniqueName: \"kubernetes.io/projected/18f5ffd7-6621-4e7c-bb6d-6f26ee55e37a-kube-api-access-qbwvd\") pod \"18f5ffd7-6621-4e7c-bb6d-6f26ee55e37a\" (UID: \"18f5ffd7-6621-4e7c-bb6d-6f26ee55e37a\") "
Sep 18 14:05:46.415396 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:05:46.415383    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/7aca40a5-1d85-4a2a-a4b5-3ea2182b7190-extensions-socket-volume\") pod \"7aca40a5-1d85-4a2a-a4b5-3ea2182b7190\" (UID: \"7aca40a5-1d85-4a2a-a4b5-3ea2182b7190\") "
Sep 18 14:05:46.415507 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:05:46.415486    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-hvffg\" (UniqueName: \"kubernetes.io/projected/7aca40a5-1d85-4a2a-a4b5-3ea2182b7190-kube-api-access-hvffg\") pod \"7aca40a5-1d85-4a2a-a4b5-3ea2182b7190\" (UID: \"7aca40a5-1d85-4a2a-a4b5-3ea2182b7190\") "
Sep 18 14:05:46.415855 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:05:46.415830    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7aca40a5-1d85-4a2a-a4b5-3ea2182b7190-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "7aca40a5-1d85-4a2a-a4b5-3ea2182b7190" (UID: "7aca40a5-1d85-4a2a-a4b5-3ea2182b7190"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 18 14:05:46.417643 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:05:46.417625    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7aca40a5-1d85-4a2a-a4b5-3ea2182b7190-kube-api-access-hvffg" (OuterVolumeSpecName: "kube-api-access-hvffg") pod "7aca40a5-1d85-4a2a-a4b5-3ea2182b7190" (UID: "7aca40a5-1d85-4a2a-a4b5-3ea2182b7190"). InnerVolumeSpecName "kube-api-access-hvffg". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 18 14:05:46.417742 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:05:46.417679    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/18f5ffd7-6621-4e7c-bb6d-6f26ee55e37a-kube-api-access-qbwvd" (OuterVolumeSpecName: "kube-api-access-qbwvd") pod "18f5ffd7-6621-4e7c-bb6d-6f26ee55e37a" (UID: "18f5ffd7-6621-4e7c-bb6d-6f26ee55e37a"). InnerVolumeSpecName "kube-api-access-qbwvd". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 18 14:05:46.516629 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:05:46.516601    2565 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hvffg\" (UniqueName: \"kubernetes.io/projected/7aca40a5-1d85-4a2a-a4b5-3ea2182b7190-kube-api-access-hvffg\") on node \"ip-10-0-129-16.ec2.internal\" DevicePath \"\""
Sep 18 14:05:46.516629 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:05:46.516629    2565 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qbwvd\" (UniqueName: \"kubernetes.io/projected/18f5ffd7-6621-4e7c-bb6d-6f26ee55e37a-kube-api-access-qbwvd\") on node \"ip-10-0-129-16.ec2.internal\" DevicePath \"\""
Sep 18 14:05:46.516780 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:05:46.516640    2565 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/7aca40a5-1d85-4a2a-a4b5-3ea2182b7190-extensions-socket-volume\") on node \"ip-10-0-129-16.ec2.internal\" DevicePath \"\""
Sep 18 14:05:46.708484 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:05:46.708459    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-tpjwq"]
Sep 18 14:05:46.710353 ip-10-0-129-16 kubenswrapper[2565]: W0918 14:05:46.710320    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod191da26b_c57c_425b_9727_8709bafd68e0.slice/crio-caa748f0d138fcf41b5826e4d2346fe311d2c212b05c65c8f9473a859d9e0bb0 WatchSource:0}: Error finding container caa748f0d138fcf41b5826e4d2346fe311d2c212b05c65c8f9473a859d9e0bb0: Status 404 returned error can't find the container with id caa748f0d138fcf41b5826e4d2346fe311d2c212b05c65c8f9473a859d9e0bb0
Sep 18 14:05:46.733225 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:05:46.733191    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-74495c69f8-llhlr"]
Sep 18 14:05:46.744237 ip-10-0-129-16 kubenswrapper[2565]: W0918 14:05:46.744196    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1f5c710a_07d4_4029_b265_6d73b6fb1c53.slice/crio-1e753375e2b926d685ad8fac1c78840025ccf9dad33d672a78f02b5a5fcdc4d2 WatchSource:0}: Error finding container 1e753375e2b926d685ad8fac1c78840025ccf9dad33d672a78f02b5a5fcdc4d2: Status 404 returned error can't find the container with id 1e753375e2b926d685ad8fac1c78840025ccf9dad33d672a78f02b5a5fcdc4d2
Sep 18 14:05:46.892023 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:05:46.891979    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-tpjwq" event={"ID":"191da26b-c57c-425b-9727-8709bafd68e0","Type":"ContainerStarted","Data":"6192c16b5744b23e40e371141539337cc8c207cb894e18bcee3785ea042fa5b5"}
Sep 18 14:05:46.892219 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:05:46.892076    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-tpjwq"
Sep 18 14:05:46.892219 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:05:46.892102    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-tpjwq" event={"ID":"191da26b-c57c-425b-9727-8709bafd68e0","Type":"ContainerStarted","Data":"caa748f0d138fcf41b5826e4d2346fe311d2c212b05c65c8f9473a859d9e0bb0"}
Sep 18 14:05:46.893421 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:05:46.893396    2565 generic.go:356] "Generic (PLEG): container finished" podID="18f5ffd7-6621-4e7c-bb6d-6f26ee55e37a" containerID="11b9ad09720cdad510d7e014b898e9ae9d76b4f614ac262c062a6a3c3972c3ee" exitCode=0
Sep 18 14:05:46.893537 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:05:46.893432    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-89kt4"
Sep 18 14:05:46.893537 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:05:46.893487    2565 scope.go:117] "RemoveContainer" containerID="11b9ad09720cdad510d7e014b898e9ae9d76b4f614ac262c062a6a3c3972c3ee"
Sep 18 14:05:46.894603 ip-10-0-129-16 kubenswrapper[2565]: E0918 14:05:46.894582    2565 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-74495c69f8-89kt4\" is forbidden: User \"system:node:ip-10-0-129-16.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-129-16.ec2.internal' and this object" podUID="18f5ffd7-6621-4e7c-bb6d-6f26ee55e37a" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-89kt4"
Sep 18 14:05:46.895397 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:05:46.895372    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-llhlr" event={"ID":"1f5c710a-07d4-4029-b265-6d73b6fb1c53","Type":"ContainerStarted","Data":"941308ea1f6258f01895bfa4f8bc88033bd199bade22cd3204f88ccad904cb31"}
Sep 18 14:05:46.895505 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:05:46.895422    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-llhlr" event={"ID":"1f5c710a-07d4-4029-b265-6d73b6fb1c53","Type":"ContainerStarted","Data":"1e753375e2b926d685ad8fac1c78840025ccf9dad33d672a78f02b5a5fcdc4d2"}
Sep 18 14:05:46.895505 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:05:46.895455    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-llhlr"
Sep 18 14:05:46.896793 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:05:46.896773    2565 generic.go:356] "Generic (PLEG): container finished" podID="7aca40a5-1d85-4a2a-a4b5-3ea2182b7190" containerID="eee69d9d82ef80fe7f2c2fccdcc55f3aa628550a34d8d3c2f520a179cc052c46" exitCode=0
Sep 18 14:05:46.896882 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:05:46.896827    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-8tttn"
Sep 18 14:05:46.901756 ip-10-0-129-16 kubenswrapper[2565]: E0918 14:05:46.901725    2565 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-76f8f686bf-8tttn\" is forbidden: User \"system:node:ip-10-0-129-16.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-129-16.ec2.internal' and this object" podUID="7aca40a5-1d85-4a2a-a4b5-3ea2182b7190" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-8tttn"
Sep 18 14:05:46.905762 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:05:46.905739    2565 scope.go:117] "RemoveContainer" containerID="11b9ad09720cdad510d7e014b898e9ae9d76b4f614ac262c062a6a3c3972c3ee"
Sep 18 14:05:46.906360 ip-10-0-129-16 kubenswrapper[2565]: E0918 14:05:46.906333    2565 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"11b9ad09720cdad510d7e014b898e9ae9d76b4f614ac262c062a6a3c3972c3ee\": container with ID starting with 11b9ad09720cdad510d7e014b898e9ae9d76b4f614ac262c062a6a3c3972c3ee not found: ID does not exist" containerID="11b9ad09720cdad510d7e014b898e9ae9d76b4f614ac262c062a6a3c3972c3ee"
Sep 18 14:05:46.906443 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:05:46.906372    2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"11b9ad09720cdad510d7e014b898e9ae9d76b4f614ac262c062a6a3c3972c3ee"} err="failed to get container status \"11b9ad09720cdad510d7e014b898e9ae9d76b4f614ac262c062a6a3c3972c3ee\": rpc error: code = NotFound desc = could not find container \"11b9ad09720cdad510d7e014b898e9ae9d76b4f614ac262c062a6a3c3972c3ee\": container with ID starting with 11b9ad09720cdad510d7e014b898e9ae9d76b4f614ac262c062a6a3c3972c3ee not found: ID does not exist"
Sep 18 14:05:46.906443 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:05:46.906395    2565 scope.go:117] "RemoveContainer" containerID="eee69d9d82ef80fe7f2c2fccdcc55f3aa628550a34d8d3c2f520a179cc052c46"
Sep 18 14:05:46.914806 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:05:46.914785    2565 scope.go:117] "RemoveContainer" containerID="eee69d9d82ef80fe7f2c2fccdcc55f3aa628550a34d8d3c2f520a179cc052c46"
Sep 18 14:05:46.915127 ip-10-0-129-16 kubenswrapper[2565]: E0918 14:05:46.915102    2565 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"eee69d9d82ef80fe7f2c2fccdcc55f3aa628550a34d8d3c2f520a179cc052c46\": container with ID starting with eee69d9d82ef80fe7f2c2fccdcc55f3aa628550a34d8d3c2f520a179cc052c46 not found: ID does not exist" containerID="eee69d9d82ef80fe7f2c2fccdcc55f3aa628550a34d8d3c2f520a179cc052c46"
Sep 18 14:05:46.915195 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:05:46.915130    2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"eee69d9d82ef80fe7f2c2fccdcc55f3aa628550a34d8d3c2f520a179cc052c46"} err="failed to get container status \"eee69d9d82ef80fe7f2c2fccdcc55f3aa628550a34d8d3c2f520a179cc052c46\": rpc error: code = NotFound desc = could not find container \"eee69d9d82ef80fe7f2c2fccdcc55f3aa628550a34d8d3c2f520a179cc052c46\": container with ID starting with eee69d9d82ef80fe7f2c2fccdcc55f3aa628550a34d8d3c2f520a179cc052c46 not found: ID does not exist"
Sep 18 14:05:46.927658 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:05:46.927615    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-tpjwq" podStartSLOduration=1.927601705 podStartE2EDuration="1.927601705s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 14:05:45 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 14:05:46.925902343 +0000 UTC m=+415.099812033" watchObservedRunningTime="2026-09-18 14:05:46.927601705 +0000 UTC m=+415.101511364"
Sep 18 14:05:46.929359 ip-10-0-129-16 kubenswrapper[2565]: E0918 14:05:46.929334    2565 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-74495c69f8-89kt4\" is forbidden: User \"system:node:ip-10-0-129-16.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-129-16.ec2.internal' and this object" podUID="18f5ffd7-6621-4e7c-bb6d-6f26ee55e37a" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-89kt4"
Sep 18 14:05:46.931287 ip-10-0-129-16 kubenswrapper[2565]: E0918 14:05:46.931268    2565 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-76f8f686bf-8tttn\" is forbidden: User \"system:node:ip-10-0-129-16.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-129-16.ec2.internal' and this object" podUID="7aca40a5-1d85-4a2a-a4b5-3ea2182b7190" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-8tttn"
Sep 18 14:05:46.958492 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:05:46.958438    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-llhlr" podStartSLOduration=1.958420761 podStartE2EDuration="1.958420761s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 14:05:45 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 14:05:46.955590434 +0000 UTC m=+415.129500118" watchObservedRunningTime="2026-09-18 14:05:46.958420761 +0000 UTC m=+415.132330421"
Sep 18 14:05:47.912384 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:05:47.912355    2565 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="18f5ffd7-6621-4e7c-bb6d-6f26ee55e37a" path="/var/lib/kubelet/pods/18f5ffd7-6621-4e7c-bb6d-6f26ee55e37a/volumes"
Sep 18 14:05:47.912730 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:05:47.912717    2565 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7aca40a5-1d85-4a2a-a4b5-3ea2182b7190" path="/var/lib/kubelet/pods/7aca40a5-1d85-4a2a-a4b5-3ea2182b7190/volumes"
Sep 18 14:05:57.911721 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:05:57.911695    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-tpjwq"
Sep 18 14:05:57.912172 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:05:57.911750    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-llhlr"
Sep 18 14:06:01.717987 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:06:01.717946    2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-tpjwq"]
Sep 18 14:06:01.718472 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:06:01.718201    2565 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-tpjwq" podUID="191da26b-c57c-425b-9727-8709bafd68e0" containerName="manager" containerID="cri-o://6192c16b5744b23e40e371141539337cc8c207cb894e18bcee3785ea042fa5b5" gracePeriod=10
Sep 18 14:06:01.968317 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:06:01.968253    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-tpjwq"
Sep 18 14:06:01.980595 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:06:01.980557    2565 generic.go:356] "Generic (PLEG): container finished" podID="191da26b-c57c-425b-9727-8709bafd68e0" containerID="6192c16b5744b23e40e371141539337cc8c207cb894e18bcee3785ea042fa5b5" exitCode=0
Sep 18 14:06:01.980736 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:06:01.980611    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-tpjwq"
Sep 18 14:06:01.980736 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:06:01.980645    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-tpjwq" event={"ID":"191da26b-c57c-425b-9727-8709bafd68e0","Type":"ContainerDied","Data":"6192c16b5744b23e40e371141539337cc8c207cb894e18bcee3785ea042fa5b5"}
Sep 18 14:06:01.980736 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:06:01.980704    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-tpjwq" event={"ID":"191da26b-c57c-425b-9727-8709bafd68e0","Type":"ContainerDied","Data":"caa748f0d138fcf41b5826e4d2346fe311d2c212b05c65c8f9473a859d9e0bb0"}
Sep 18 14:06:01.980736 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:06:01.980726    2565 scope.go:117] "RemoveContainer" containerID="6192c16b5744b23e40e371141539337cc8c207cb894e18bcee3785ea042fa5b5"
Sep 18 14:06:01.989049 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:06:01.989019    2565 scope.go:117] "RemoveContainer" containerID="6192c16b5744b23e40e371141539337cc8c207cb894e18bcee3785ea042fa5b5"
Sep 18 14:06:01.989352 ip-10-0-129-16 kubenswrapper[2565]: E0918 14:06:01.989327    2565 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6192c16b5744b23e40e371141539337cc8c207cb894e18bcee3785ea042fa5b5\": container with ID starting with 6192c16b5744b23e40e371141539337cc8c207cb894e18bcee3785ea042fa5b5 not found: ID does not exist" containerID="6192c16b5744b23e40e371141539337cc8c207cb894e18bcee3785ea042fa5b5"
Sep 18 14:06:01.989402 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:06:01.989363    2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6192c16b5744b23e40e371141539337cc8c207cb894e18bcee3785ea042fa5b5"} err="failed to get container status \"6192c16b5744b23e40e371141539337cc8c207cb894e18bcee3785ea042fa5b5\": rpc error: code = NotFound desc = could not find container \"6192c16b5744b23e40e371141539337cc8c207cb894e18bcee3785ea042fa5b5\": container with ID starting with 6192c16b5744b23e40e371141539337cc8c207cb894e18bcee3785ea042fa5b5 not found: ID does not exist"
Sep 18 14:06:02.049027 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:06:02.048992    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-6qt4g\" (UniqueName: \"kubernetes.io/projected/191da26b-c57c-425b-9727-8709bafd68e0-kube-api-access-6qt4g\") pod \"191da26b-c57c-425b-9727-8709bafd68e0\" (UID: \"191da26b-c57c-425b-9727-8709bafd68e0\") "
Sep 18 14:06:02.049027 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:06:02.049058    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/191da26b-c57c-425b-9727-8709bafd68e0-extensions-socket-volume\") pod \"191da26b-c57c-425b-9727-8709bafd68e0\" (UID: \"191da26b-c57c-425b-9727-8709bafd68e0\") "
Sep 18 14:06:02.049478 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:06:02.049454    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/191da26b-c57c-425b-9727-8709bafd68e0-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "191da26b-c57c-425b-9727-8709bafd68e0" (UID: "191da26b-c57c-425b-9727-8709bafd68e0"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 18 14:06:02.051164 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:06:02.051142    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/191da26b-c57c-425b-9727-8709bafd68e0-kube-api-access-6qt4g" (OuterVolumeSpecName: "kube-api-access-6qt4g") pod "191da26b-c57c-425b-9727-8709bafd68e0" (UID: "191da26b-c57c-425b-9727-8709bafd68e0"). InnerVolumeSpecName "kube-api-access-6qt4g". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 18 14:06:02.150721 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:06:02.150680    2565 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6qt4g\" (UniqueName: \"kubernetes.io/projected/191da26b-c57c-425b-9727-8709bafd68e0-kube-api-access-6qt4g\") on node \"ip-10-0-129-16.ec2.internal\" DevicePath \"\""
Sep 18 14:06:02.150721 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:06:02.150714    2565 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/191da26b-c57c-425b-9727-8709bafd68e0-extensions-socket-volume\") on node \"ip-10-0-129-16.ec2.internal\" DevicePath \"\""
Sep 18 14:06:02.303500 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:06:02.303466    2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-tpjwq"]
Sep 18 14:06:02.312139 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:06:02.312111    2565 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-tpjwq"]
Sep 18 14:06:03.912582 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:06:03.912546    2565 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="191da26b-c57c-425b-9727-8709bafd68e0" path="/var/lib/kubelet/pods/191da26b-c57c-425b-9727-8709bafd68e0/volumes"
Sep 18 14:06:18.072983 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:06:18.072897    2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-phnkt"]
Sep 18 14:06:18.073454 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:06:18.073388    2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="191da26b-c57c-425b-9727-8709bafd68e0" containerName="manager"
Sep 18 14:06:18.073533 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:06:18.073460    2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="191da26b-c57c-425b-9727-8709bafd68e0" containerName="manager"
Sep 18 14:06:18.073597 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:06:18.073581    2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="191da26b-c57c-425b-9727-8709bafd68e0" containerName="manager"
Sep 18 14:06:18.075733 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:06:18.075714    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-phnkt"
Sep 18 14:06:18.078239 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:06:18.078215    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"maas-default-gateway-openshift-default-dockercfg-lxkpg\""
Sep 18 14:06:18.092198 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:06:18.092172    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-phnkt"]
Sep 18 14:06:18.191219 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:06:18.191187    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/77ad5822-b925-43b7-8337-db3d1dd680cb-workload-socket\") pod \"maas-default-gateway-openshift-default-78bb78ff78-phnkt\" (UID: \"77ad5822-b925-43b7-8337-db3d1dd680cb\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-phnkt"
Sep 18 14:06:18.191219 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:06:18.191222    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/77ad5822-b925-43b7-8337-db3d1dd680cb-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-78bb78ff78-phnkt\" (UID: \"77ad5822-b925-43b7-8337-db3d1dd680cb\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-phnkt"
Sep 18 14:06:18.191443 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:06:18.191305    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/77ad5822-b925-43b7-8337-db3d1dd680cb-istio-envoy\") pod \"maas-default-gateway-openshift-default-78bb78ff78-phnkt\" (UID: \"77ad5822-b925-43b7-8337-db3d1dd680cb\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-phnkt"
Sep 18 14:06:18.191443 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:06:18.191339    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9zlg9\" (UniqueName: \"kubernetes.io/projected/77ad5822-b925-43b7-8337-db3d1dd680cb-kube-api-access-9zlg9\") pod \"maas-default-gateway-openshift-default-78bb78ff78-phnkt\" (UID: \"77ad5822-b925-43b7-8337-db3d1dd680cb\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-phnkt"
Sep 18 14:06:18.191443 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:06:18.191362    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/77ad5822-b925-43b7-8337-db3d1dd680cb-credential-socket\") pod \"maas-default-gateway-openshift-default-78bb78ff78-phnkt\" (UID: \"77ad5822-b925-43b7-8337-db3d1dd680cb\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-phnkt"
Sep 18 14:06:18.191443 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:06:18.191386    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/77ad5822-b925-43b7-8337-db3d1dd680cb-istio-data\") pod \"maas-default-gateway-openshift-default-78bb78ff78-phnkt\" (UID: \"77ad5822-b925-43b7-8337-db3d1dd680cb\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-phnkt"
Sep 18 14:06:18.191443 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:06:18.191441    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/77ad5822-b925-43b7-8337-db3d1dd680cb-istio-token\") pod \"maas-default-gateway-openshift-default-78bb78ff78-phnkt\" (UID: \"77ad5822-b925-43b7-8337-db3d1dd680cb\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-phnkt"
Sep 18 14:06:18.191625 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:06:18.191462    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/77ad5822-b925-43b7-8337-db3d1dd680cb-workload-certs\") pod \"maas-default-gateway-openshift-default-78bb78ff78-phnkt\" (UID: \"77ad5822-b925-43b7-8337-db3d1dd680cb\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-phnkt"
Sep 18 14:06:18.191625 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:06:18.191485    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/77ad5822-b925-43b7-8337-db3d1dd680cb-istio-podinfo\") pod \"maas-default-gateway-openshift-default-78bb78ff78-phnkt\" (UID: \"77ad5822-b925-43b7-8337-db3d1dd680cb\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-phnkt"
Sep 18 14:06:18.292766 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:06:18.292726    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/77ad5822-b925-43b7-8337-db3d1dd680cb-istio-data\") pod \"maas-default-gateway-openshift-default-78bb78ff78-phnkt\" (UID: \"77ad5822-b925-43b7-8337-db3d1dd680cb\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-phnkt"
Sep 18 14:06:18.292948 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:06:18.292779    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/77ad5822-b925-43b7-8337-db3d1dd680cb-istio-token\") pod \"maas-default-gateway-openshift-default-78bb78ff78-phnkt\" (UID: \"77ad5822-b925-43b7-8337-db3d1dd680cb\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-phnkt"
Sep 18 14:06:18.292948 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:06:18.292801    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/77ad5822-b925-43b7-8337-db3d1dd680cb-workload-certs\") pod \"maas-default-gateway-openshift-default-78bb78ff78-phnkt\" (UID: \"77ad5822-b925-43b7-8337-db3d1dd680cb\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-phnkt"
Sep 18 14:06:18.292948 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:06:18.292816    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/77ad5822-b925-43b7-8337-db3d1dd680cb-istio-podinfo\") pod \"maas-default-gateway-openshift-default-78bb78ff78-phnkt\" (UID: \"77ad5822-b925-43b7-8337-db3d1dd680cb\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-phnkt"
Sep 18 14:06:18.292948 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:06:18.292843    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/77ad5822-b925-43b7-8337-db3d1dd680cb-workload-socket\") pod \"maas-default-gateway-openshift-default-78bb78ff78-phnkt\" (UID: \"77ad5822-b925-43b7-8337-db3d1dd680cb\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-phnkt"
Sep 18 14:06:18.292948 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:06:18.292859    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/77ad5822-b925-43b7-8337-db3d1dd680cb-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-78bb78ff78-phnkt\" (UID: \"77ad5822-b925-43b7-8337-db3d1dd680cb\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-phnkt"
Sep 18 14:06:18.292948 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:06:18.292923    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/77ad5822-b925-43b7-8337-db3d1dd680cb-istio-envoy\") pod \"maas-default-gateway-openshift-default-78bb78ff78-phnkt\" (UID: \"77ad5822-b925-43b7-8337-db3d1dd680cb\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-phnkt"
Sep 18 14:06:18.293290 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:06:18.292970    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9zlg9\" (UniqueName: \"kubernetes.io/projected/77ad5822-b925-43b7-8337-db3d1dd680cb-kube-api-access-9zlg9\") pod \"maas-default-gateway-openshift-default-78bb78ff78-phnkt\" (UID: \"77ad5822-b925-43b7-8337-db3d1dd680cb\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-phnkt"
Sep 18 14:06:18.293290 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:06:18.293003    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/77ad5822-b925-43b7-8337-db3d1dd680cb-credential-socket\") pod \"maas-default-gateway-openshift-default-78bb78ff78-phnkt\" (UID: \"77ad5822-b925-43b7-8337-db3d1dd680cb\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-phnkt"
Sep 18 14:06:18.293290 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:06:18.293225    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/77ad5822-b925-43b7-8337-db3d1dd680cb-istio-data\") pod \"maas-default-gateway-openshift-default-78bb78ff78-phnkt\" (UID: \"77ad5822-b925-43b7-8337-db3d1dd680cb\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-phnkt"
Sep 18 14:06:18.293440 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:06:18.293285    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/77ad5822-b925-43b7-8337-db3d1dd680cb-workload-socket\") pod \"maas-default-gateway-openshift-default-78bb78ff78-phnkt\" (UID: \"77ad5822-b925-43b7-8337-db3d1dd680cb\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-phnkt"
Sep 18 14:06:18.293440 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:06:18.293428    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/77ad5822-b925-43b7-8337-db3d1dd680cb-credential-socket\") pod \"maas-default-gateway-openshift-default-78bb78ff78-phnkt\" (UID: \"77ad5822-b925-43b7-8337-db3d1dd680cb\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-phnkt"
Sep 18 14:06:18.293537 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:06:18.293473    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/77ad5822-b925-43b7-8337-db3d1dd680cb-workload-certs\") pod \"maas-default-gateway-openshift-default-78bb78ff78-phnkt\" (UID: \"77ad5822-b925-43b7-8337-db3d1dd680cb\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-phnkt"
Sep 18 14:06:18.293717 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:06:18.293686    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/77ad5822-b925-43b7-8337-db3d1dd680cb-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-78bb78ff78-phnkt\" (UID: \"77ad5822-b925-43b7-8337-db3d1dd680cb\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-phnkt"
Sep 18 14:06:18.295252 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:06:18.295227    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/77ad5822-b925-43b7-8337-db3d1dd680cb-istio-envoy\") pod \"maas-default-gateway-openshift-default-78bb78ff78-phnkt\" (UID: \"77ad5822-b925-43b7-8337-db3d1dd680cb\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-phnkt"
Sep 18 14:06:18.295509 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:06:18.295491    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/77ad5822-b925-43b7-8337-db3d1dd680cb-istio-podinfo\") pod \"maas-default-gateway-openshift-default-78bb78ff78-phnkt\" (UID: \"77ad5822-b925-43b7-8337-db3d1dd680cb\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-phnkt"
Sep 18 14:06:18.301697 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:06:18.301674    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/77ad5822-b925-43b7-8337-db3d1dd680cb-istio-token\") pod \"maas-default-gateway-openshift-default-78bb78ff78-phnkt\" (UID: \"77ad5822-b925-43b7-8337-db3d1dd680cb\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-phnkt"
Sep 18 14:06:18.301950 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:06:18.301930    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9zlg9\" (UniqueName: \"kubernetes.io/projected/77ad5822-b925-43b7-8337-db3d1dd680cb-kube-api-access-9zlg9\") pod \"maas-default-gateway-openshift-default-78bb78ff78-phnkt\" (UID: \"77ad5822-b925-43b7-8337-db3d1dd680cb\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-phnkt"
Sep 18 14:06:18.386063 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:06:18.385953    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-phnkt"
Sep 18 14:06:18.662854 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:06:18.662828    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-phnkt"]
Sep 18 14:06:18.665265 ip-10-0-129-16 kubenswrapper[2565]: W0918 14:06:18.665232    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod77ad5822_b925_43b7_8337_db3d1dd680cb.slice/crio-ecb44f64ce4653466ba11049dab6faf0e0b807aa1417681724fb4b2c89b3f9a2 WatchSource:0}: Error finding container ecb44f64ce4653466ba11049dab6faf0e0b807aa1417681724fb4b2c89b3f9a2: Status 404 returned error can't find the container with id ecb44f64ce4653466ba11049dab6faf0e0b807aa1417681724fb4b2c89b3f9a2
Sep 18 14:06:18.667437 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:06:18.667403    2565 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 18 14:06:18.667507 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:06:18.667482    2565 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 18 14:06:18.667544 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:06:18.667515    2565 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 18 14:06:19.062045 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:06:19.062006    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-phnkt" event={"ID":"77ad5822-b925-43b7-8337-db3d1dd680cb","Type":"ContainerStarted","Data":"f331a0263d5a34b834dffc954331d07436c871af93cd0ce8d27fc2749254d030"}
Sep 18 14:06:19.062256 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:06:19.062085    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-phnkt" event={"ID":"77ad5822-b925-43b7-8337-db3d1dd680cb","Type":"ContainerStarted","Data":"ecb44f64ce4653466ba11049dab6faf0e0b807aa1417681724fb4b2c89b3f9a2"}
Sep 18 14:06:19.085257 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:06:19.085203    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-phnkt" podStartSLOduration=1.085184837 podStartE2EDuration="1.085184837s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 14:06:18 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 14:06:19.08343447 +0000 UTC m=+447.257344134" watchObservedRunningTime="2026-09-18 14:06:19.085184837 +0000 UTC m=+447.259094497"
Sep 18 14:06:19.386706 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:06:19.386612    2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-phnkt"
Sep 18 14:06:19.391440 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:06:19.391414    2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-phnkt"
Sep 18 14:06:20.049305 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:06:20.049274    2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-limitador-8588749977-kc6px"]
Sep 18 14:06:20.052063 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:06:20.052020    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-8588749977-kc6px"
Sep 18 14:06:20.054495 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:06:20.054471    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"limitador-limits-config-limitador\""
Sep 18 14:06:20.054495 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:06:20.054488    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-tm7c4\""
Sep 18 14:06:20.062140 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:06:20.062115    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-8588749977-kc6px"]
Sep 18 14:06:20.068196 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:06:20.068168    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-phnkt"
Sep 18 14:06:20.069338 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:06:20.069313    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-phnkt"
Sep 18 14:06:20.110179 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:06:20.110149    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/df156db3-0534-4423-be5c-fd7c65293c99-config-file\") pod \"limitador-limitador-8588749977-kc6px\" (UID: \"df156db3-0534-4423-be5c-fd7c65293c99\") " pod="kuadrant-system/limitador-limitador-8588749977-kc6px"
Sep 18 14:06:20.110655 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:06:20.110189    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7jk84\" (UniqueName: \"kubernetes.io/projected/df156db3-0534-4423-be5c-fd7c65293c99-kube-api-access-7jk84\") pod \"limitador-limitador-8588749977-kc6px\" (UID: \"df156db3-0534-4423-be5c-fd7c65293c99\") " pod="kuadrant-system/limitador-limitador-8588749977-kc6px"
Sep 18 14:06:20.143210 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:06:20.143176    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-8588749977-kc6px"]
Sep 18 14:06:20.211824 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:06:20.211791    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/df156db3-0534-4423-be5c-fd7c65293c99-config-file\") pod \"limitador-limitador-8588749977-kc6px\" (UID: \"df156db3-0534-4423-be5c-fd7c65293c99\") " pod="kuadrant-system/limitador-limitador-8588749977-kc6px"
Sep 18 14:06:20.211824 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:06:20.211834    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7jk84\" (UniqueName: \"kubernetes.io/projected/df156db3-0534-4423-be5c-fd7c65293c99-kube-api-access-7jk84\") pod \"limitador-limitador-8588749977-kc6px\" (UID: \"df156db3-0534-4423-be5c-fd7c65293c99\") " pod="kuadrant-system/limitador-limitador-8588749977-kc6px"
Sep 18 14:06:20.212471 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:06:20.212452    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/df156db3-0534-4423-be5c-fd7c65293c99-config-file\") pod \"limitador-limitador-8588749977-kc6px\" (UID: \"df156db3-0534-4423-be5c-fd7c65293c99\") " pod="kuadrant-system/limitador-limitador-8588749977-kc6px"
Sep 18 14:06:20.219898 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:06:20.219860    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7jk84\" (UniqueName: \"kubernetes.io/projected/df156db3-0534-4423-be5c-fd7c65293c99-kube-api-access-7jk84\") pod \"limitador-limitador-8588749977-kc6px\" (UID: \"df156db3-0534-4423-be5c-fd7c65293c99\") " pod="kuadrant-system/limitador-limitador-8588749977-kc6px"
Sep 18 14:06:20.363628 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:06:20.363530    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-8588749977-kc6px"
Sep 18 14:06:20.633949 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:06:20.633921    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-8588749977-kc6px"]
Sep 18 14:06:20.636238 ip-10-0-129-16 kubenswrapper[2565]: W0918 14:06:20.636213    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddf156db3_0534_4423_be5c_fd7c65293c99.slice/crio-5c86e218852135a0d78fd6b097e31f2e3534cceedb6999a96aef0d2daf7eb255 WatchSource:0}: Error finding container 5c86e218852135a0d78fd6b097e31f2e3534cceedb6999a96aef0d2daf7eb255: Status 404 returned error can't find the container with id 5c86e218852135a0d78fd6b097e31f2e3534cceedb6999a96aef0d2daf7eb255
Sep 18 14:06:21.074333 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:06:21.074299    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-8588749977-kc6px" event={"ID":"df156db3-0534-4423-be5c-fd7c65293c99","Type":"ContainerStarted","Data":"5c86e218852135a0d78fd6b097e31f2e3534cceedb6999a96aef0d2daf7eb255"}
Sep 18 14:06:22.080649 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:06:22.080616    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-8588749977-kc6px" event={"ID":"df156db3-0534-4423-be5c-fd7c65293c99","Type":"ContainerStarted","Data":"70015825cf710730854be35de703257c36f9755b3620c5b4f542ae809b61d0a2"}
Sep 18 14:06:22.098050 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:06:22.097993    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-limitador-8588749977-kc6px" podStartSLOduration=0.777377725 podStartE2EDuration="2.097975612s" totalImagesPullingTime="1.320597887s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 14:06:20 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 14:06:22.096768073 +0000 UTC m=+450.270677767" watchObservedRunningTime="2026-09-18 14:06:22.097975612 +0000 UTC m=+450.271885271"
Sep 18 14:06:23.085635 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:06:23.085569    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-limitador-8588749977-kc6px"
Sep 18 14:06:34.092419 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:06:34.092382    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-limitador-8588749977-kc6px"
Sep 18 14:06:35.727587 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:06:35.727550    2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-limitador-8588749977-kc6px"]
Sep 18 14:06:35.728082 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:06:35.727827    2565 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/limitador-limitador-8588749977-kc6px" podUID="df156db3-0534-4423-be5c-fd7c65293c99" containerName="limitador" containerID="cri-o://70015825cf710730854be35de703257c36f9755b3620c5b4f542ae809b61d0a2" gracePeriod=30
Sep 18 14:06:36.149581 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:06:36.149550    2565 generic.go:356] "Generic (PLEG): container finished" podID="df156db3-0534-4423-be5c-fd7c65293c99" containerID="70015825cf710730854be35de703257c36f9755b3620c5b4f542ae809b61d0a2" exitCode=0
Sep 18 14:06:36.149767 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:06:36.149590    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-8588749977-kc6px" event={"ID":"df156db3-0534-4423-be5c-fd7c65293c99","Type":"ContainerDied","Data":"70015825cf710730854be35de703257c36f9755b3620c5b4f542ae809b61d0a2"}
Sep 18 14:06:36.675988 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:06:36.675965    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-8588749977-kc6px"
Sep 18 14:06:36.773347 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:06:36.773315    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-7jk84\" (UniqueName: \"kubernetes.io/projected/df156db3-0534-4423-be5c-fd7c65293c99-kube-api-access-7jk84\") pod \"df156db3-0534-4423-be5c-fd7c65293c99\" (UID: \"df156db3-0534-4423-be5c-fd7c65293c99\") "
Sep 18 14:06:36.773726 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:06:36.773377    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/df156db3-0534-4423-be5c-fd7c65293c99-config-file\") pod \"df156db3-0534-4423-be5c-fd7c65293c99\" (UID: \"df156db3-0534-4423-be5c-fd7c65293c99\") "
Sep 18 14:06:36.773766 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:06:36.773749    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/df156db3-0534-4423-be5c-fd7c65293c99-config-file" (OuterVolumeSpecName: "config-file") pod "df156db3-0534-4423-be5c-fd7c65293c99" (UID: "df156db3-0534-4423-be5c-fd7c65293c99"). InnerVolumeSpecName "config-file". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 18 14:06:36.775405 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:06:36.775368    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/df156db3-0534-4423-be5c-fd7c65293c99-kube-api-access-7jk84" (OuterVolumeSpecName: "kube-api-access-7jk84") pod "df156db3-0534-4423-be5c-fd7c65293c99" (UID: "df156db3-0534-4423-be5c-fd7c65293c99"). InnerVolumeSpecName "kube-api-access-7jk84". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 18 14:06:36.874796 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:06:36.874713    2565 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7jk84\" (UniqueName: \"kubernetes.io/projected/df156db3-0534-4423-be5c-fd7c65293c99-kube-api-access-7jk84\") on node \"ip-10-0-129-16.ec2.internal\" DevicePath \"\""
Sep 18 14:06:36.874796 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:06:36.874742    2565 reconciler_common.go:299] "Volume detached for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/df156db3-0534-4423-be5c-fd7c65293c99-config-file\") on node \"ip-10-0-129-16.ec2.internal\" DevicePath \"\""
Sep 18 14:06:37.155676 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:06:37.155599    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-8588749977-kc6px"
Sep 18 14:06:37.155676 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:06:37.155613    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-8588749977-kc6px" event={"ID":"df156db3-0534-4423-be5c-fd7c65293c99","Type":"ContainerDied","Data":"5c86e218852135a0d78fd6b097e31f2e3534cceedb6999a96aef0d2daf7eb255"}
Sep 18 14:06:37.155676 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:06:37.155665    2565 scope.go:117] "RemoveContainer" containerID="70015825cf710730854be35de703257c36f9755b3620c5b4f542ae809b61d0a2"
Sep 18 14:06:37.177799 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:06:37.177770    2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-limitador-8588749977-kc6px"]
Sep 18 14:06:37.183961 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:06:37.183935    2565 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/limitador-limitador-8588749977-kc6px"]
Sep 18 14:06:37.912727 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:06:37.912694    2565 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="df156db3-0534-4423-be5c-fd7c65293c99" path="/var/lib/kubelet/pods/df156db3-0534-4423-be5c-fd7c65293c99/volumes"
Sep 18 14:06:41.755419 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:06:41.755379    2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/postgres-69fb4594cf-mx9q7"]
Sep 18 14:06:41.755889 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:06:41.755771    2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="df156db3-0534-4423-be5c-fd7c65293c99" containerName="limitador"
Sep 18 14:06:41.755889 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:06:41.755781    2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="df156db3-0534-4423-be5c-fd7c65293c99" containerName="limitador"
Sep 18 14:06:41.755889 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:06:41.755859    2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="df156db3-0534-4423-be5c-fd7c65293c99" containerName="limitador"
Sep 18 14:06:41.760396 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:06:41.760375    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/postgres-69fb4594cf-mx9q7"
Sep 18 14:06:41.763538 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:06:41.763519    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"postgres-creds\""
Sep 18 14:06:41.763538 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:06:41.763529    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 18 14:06:41.763697 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:06:41.763577    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\""
Sep 18 14:06:41.764515 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:06:41.764498    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"default-dockercfg-67pxf\""
Sep 18 14:06:41.771070 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:06:41.771017    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/postgres-69fb4594cf-mx9q7"]
Sep 18 14:06:41.821205 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:06:41.821169    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/8107a555-0c1e-453d-b2ca-2dcb8aa64914-data\") pod \"postgres-69fb4594cf-mx9q7\" (UID: \"8107a555-0c1e-453d-b2ca-2dcb8aa64914\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-mx9q7"
Sep 18 14:06:41.821396 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:06:41.821227    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-96wcz\" (UniqueName: \"kubernetes.io/projected/8107a555-0c1e-453d-b2ca-2dcb8aa64914-kube-api-access-96wcz\") pod \"postgres-69fb4594cf-mx9q7\" (UID: \"8107a555-0c1e-453d-b2ca-2dcb8aa64914\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-mx9q7"
Sep 18 14:06:41.922158 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:06:41.922127    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-96wcz\" (UniqueName: \"kubernetes.io/projected/8107a555-0c1e-453d-b2ca-2dcb8aa64914-kube-api-access-96wcz\") pod \"postgres-69fb4594cf-mx9q7\" (UID: \"8107a555-0c1e-453d-b2ca-2dcb8aa64914\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-mx9q7"
Sep 18 14:06:41.922362 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:06:41.922255    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/8107a555-0c1e-453d-b2ca-2dcb8aa64914-data\") pod \"postgres-69fb4594cf-mx9q7\" (UID: \"8107a555-0c1e-453d-b2ca-2dcb8aa64914\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-mx9q7"
Sep 18 14:06:41.922652 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:06:41.922634    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/8107a555-0c1e-453d-b2ca-2dcb8aa64914-data\") pod \"postgres-69fb4594cf-mx9q7\" (UID: \"8107a555-0c1e-453d-b2ca-2dcb8aa64914\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-mx9q7"
Sep 18 14:06:41.930982 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:06:41.930950    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-96wcz\" (UniqueName: \"kubernetes.io/projected/8107a555-0c1e-453d-b2ca-2dcb8aa64914-kube-api-access-96wcz\") pod \"postgres-69fb4594cf-mx9q7\" (UID: \"8107a555-0c1e-453d-b2ca-2dcb8aa64914\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-mx9q7"
Sep 18 14:06:42.071809 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:06:42.071776    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/postgres-69fb4594cf-mx9q7"
Sep 18 14:06:42.342257 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:06:42.342233    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/postgres-69fb4594cf-mx9q7"]
Sep 18 14:06:42.343803 ip-10-0-129-16 kubenswrapper[2565]: W0918 14:06:42.343777    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8107a555_0c1e_453d_b2ca_2dcb8aa64914.slice/crio-2bba288e9669d6d953e3b85b845670d30c7a305a03741aa4abc90caaf9072268 WatchSource:0}: Error finding container 2bba288e9669d6d953e3b85b845670d30c7a305a03741aa4abc90caaf9072268: Status 404 returned error can't find the container with id 2bba288e9669d6d953e3b85b845670d30c7a305a03741aa4abc90caaf9072268
Sep 18 14:06:43.190734 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:06:43.190694    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/postgres-69fb4594cf-mx9q7" event={"ID":"8107a555-0c1e-453d-b2ca-2dcb8aa64914","Type":"ContainerStarted","Data":"2bba288e9669d6d953e3b85b845670d30c7a305a03741aa4abc90caaf9072268"}
Sep 18 14:06:47.216023 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:06:47.215974    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/postgres-69fb4594cf-mx9q7" event={"ID":"8107a555-0c1e-453d-b2ca-2dcb8aa64914","Type":"ContainerStarted","Data":"89bd310cfbd4de8ebed9faa8940ce0a0e71ef2b24a321deb86adc330965017e1"}
Sep 18 14:06:47.217135 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:06:47.217102    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/postgres-69fb4594cf-mx9q7"
Sep 18 14:06:47.255850 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:06:47.255801    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/postgres-69fb4594cf-mx9q7" podStartSLOduration=1.492878436 podStartE2EDuration="6.255783122s" totalImagesPullingTime="4.762904686s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 14:06:41 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 14:06:47.254474283 +0000 UTC m=+475.428383965" watchObservedRunningTime="2026-09-18 14:06:47.255783122 +0000 UTC m=+475.429692781"
Sep 18 14:06:54.254807 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:06:54.254728    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/postgres-69fb4594cf-mx9q7"
Sep 18 14:07:01.861009 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:07:01.860975    2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-765845fd55-t69bn"]
Sep 18 14:07:01.866954 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:07:01.866936    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-765845fd55-t69bn"
Sep 18 14:07:01.870256 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:07:01.870210    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-metrics-tls\""
Sep 18 14:07:01.871122 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:07:01.871104    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\""
Sep 18 14:07:01.871217 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:07:01.871150    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\""
Sep 18 14:07:01.871279 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:07:01.871228    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-svcgm\""
Sep 18 14:07:01.879181 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:07:01.879162    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-765845fd55-t69bn"]
Sep 18 14:07:02.014901 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:07:02.014865    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-k988t\" (UniqueName: \"kubernetes.io/projected/d24cf07c-4ff3-4dd9-b06f-8cc8ac4c12f8-kube-api-access-k988t\") pod \"maas-controller-765845fd55-t69bn\" (UID: \"d24cf07c-4ff3-4dd9-b06f-8cc8ac4c12f8\") " pod="opendatahub/maas-controller-765845fd55-t69bn"
Sep 18 14:07:02.015114 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:07:02.014921    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/d24cf07c-4ff3-4dd9-b06f-8cc8ac4c12f8-webhook-cert\") pod \"maas-controller-765845fd55-t69bn\" (UID: \"d24cf07c-4ff3-4dd9-b06f-8cc8ac4c12f8\") " pod="opendatahub/maas-controller-765845fd55-t69bn"
Sep 18 14:07:02.015114 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:07:02.014985    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d24cf07c-4ff3-4dd9-b06f-8cc8ac4c12f8-metrics-certs\") pod \"maas-controller-765845fd55-t69bn\" (UID: \"d24cf07c-4ff3-4dd9-b06f-8cc8ac4c12f8\") " pod="opendatahub/maas-controller-765845fd55-t69bn"
Sep 18 14:07:02.116482 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:07:02.116389    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/d24cf07c-4ff3-4dd9-b06f-8cc8ac4c12f8-webhook-cert\") pod \"maas-controller-765845fd55-t69bn\" (UID: \"d24cf07c-4ff3-4dd9-b06f-8cc8ac4c12f8\") " pod="opendatahub/maas-controller-765845fd55-t69bn"
Sep 18 14:07:02.116482 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:07:02.116449    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d24cf07c-4ff3-4dd9-b06f-8cc8ac4c12f8-metrics-certs\") pod \"maas-controller-765845fd55-t69bn\" (UID: \"d24cf07c-4ff3-4dd9-b06f-8cc8ac4c12f8\") " pod="opendatahub/maas-controller-765845fd55-t69bn"
Sep 18 14:07:02.116783 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:07:02.116527    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-k988t\" (UniqueName: \"kubernetes.io/projected/d24cf07c-4ff3-4dd9-b06f-8cc8ac4c12f8-kube-api-access-k988t\") pod \"maas-controller-765845fd55-t69bn\" (UID: \"d24cf07c-4ff3-4dd9-b06f-8cc8ac4c12f8\") " pod="opendatahub/maas-controller-765845fd55-t69bn"
Sep 18 14:07:02.118886 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:07:02.118862    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d24cf07c-4ff3-4dd9-b06f-8cc8ac4c12f8-metrics-certs\") pod \"maas-controller-765845fd55-t69bn\" (UID: \"d24cf07c-4ff3-4dd9-b06f-8cc8ac4c12f8\") " pod="opendatahub/maas-controller-765845fd55-t69bn"
Sep 18 14:07:02.119001 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:07:02.118889    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/d24cf07c-4ff3-4dd9-b06f-8cc8ac4c12f8-webhook-cert\") pod \"maas-controller-765845fd55-t69bn\" (UID: \"d24cf07c-4ff3-4dd9-b06f-8cc8ac4c12f8\") " pod="opendatahub/maas-controller-765845fd55-t69bn"
Sep 18 14:07:02.125158 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:07:02.125133    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-k988t\" (UniqueName: \"kubernetes.io/projected/d24cf07c-4ff3-4dd9-b06f-8cc8ac4c12f8-kube-api-access-k988t\") pod \"maas-controller-765845fd55-t69bn\" (UID: \"d24cf07c-4ff3-4dd9-b06f-8cc8ac4c12f8\") " pod="opendatahub/maas-controller-765845fd55-t69bn"
Sep 18 14:07:02.177719 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:07:02.177661    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-765845fd55-t69bn"
Sep 18 14:07:02.455408 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:07:02.455385    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-765845fd55-t69bn"]
Sep 18 14:07:02.457381 ip-10-0-129-16 kubenswrapper[2565]: W0918 14:07:02.457346    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd24cf07c_4ff3_4dd9_b06f_8cc8ac4c12f8.slice/crio-36db87eb71458d563cd0065679f6d6e5abce4dbc32cc4d0a066ef2c2ef565a0e WatchSource:0}: Error finding container 36db87eb71458d563cd0065679f6d6e5abce4dbc32cc4d0a066ef2c2ef565a0e: Status 404 returned error can't find the container with id 36db87eb71458d563cd0065679f6d6e5abce4dbc32cc4d0a066ef2c2ef565a0e
Sep 18 14:07:03.297564 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:07:03.297524    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-765845fd55-t69bn" event={"ID":"d24cf07c-4ff3-4dd9-b06f-8cc8ac4c12f8","Type":"ContainerStarted","Data":"36db87eb71458d563cd0065679f6d6e5abce4dbc32cc4d0a066ef2c2ef565a0e"}
Sep 18 14:07:05.308298 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:07:05.308266    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-765845fd55-t69bn" event={"ID":"d24cf07c-4ff3-4dd9-b06f-8cc8ac4c12f8","Type":"ContainerStarted","Data":"c93f47c21dde70d9a522d79f94ff2fd0c4476e0175408f91c659eed8d70b14a7"}
Sep 18 14:07:05.308746 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:07:05.308400    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-765845fd55-t69bn"
Sep 18 14:07:05.327294 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:07:05.327249    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-765845fd55-t69bn" podStartSLOduration=1.909000472 podStartE2EDuration="4.32723515s" totalImagesPullingTime="2.418234678s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 14:07:01 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 14:07:05.326276347 +0000 UTC m=+493.500186028" watchObservedRunningTime="2026-09-18 14:07:05.32723515 +0000 UTC m=+493.501144809"
Sep 18 14:07:16.315182 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:07:16.315142    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-765845fd55-t69bn"
Sep 18 14:07:39.688254 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:07:39.688174    2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-59fbc47bf8-8fw8g"]
Sep 18 14:07:39.693173 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:07:39.693153    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-59fbc47bf8-8fw8g"
Sep 18 14:07:39.696265 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:07:39.696247    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-dockercfg-lx2qf\""
Sep 18 14:07:39.696459 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:07:39.696446    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins\""
Sep 18 14:07:39.708375 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:07:39.708348    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-59fbc47bf8-8fw8g"]
Sep 18 14:07:39.756101 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:07:39.756064    2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-5b54b7fbb7-9v8fj"]
Sep 18 14:07:39.758486 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:07:39.758459    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-5b54b7fbb7-9v8fj"
Sep 18 14:07:39.770270 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:07:39.770242    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-5b54b7fbb7-9v8fj"]
Sep 18 14:07:39.776982 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:07:39.776954    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c9eadd96-985e-4259-b048-3adac3d7c5c1-plugins-config-volume\") pod \"payload-pre-processing-59fbc47bf8-8fw8g\" (UID: \"c9eadd96-985e-4259-b048-3adac3d7c5c1\") " pod="openshift-ingress/payload-pre-processing-59fbc47bf8-8fw8g"
Sep 18 14:07:39.776982 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:07:39.776986    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qj2rd\" (UniqueName: \"kubernetes.io/projected/c9eadd96-985e-4259-b048-3adac3d7c5c1-kube-api-access-qj2rd\") pod \"payload-pre-processing-59fbc47bf8-8fw8g\" (UID: \"c9eadd96-985e-4259-b048-3adac3d7c5c1\") " pod="openshift-ingress/payload-pre-processing-59fbc47bf8-8fw8g"
Sep 18 14:07:39.878513 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:07:39.878486    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c9eadd96-985e-4259-b048-3adac3d7c5c1-plugins-config-volume\") pod \"payload-pre-processing-59fbc47bf8-8fw8g\" (UID: \"c9eadd96-985e-4259-b048-3adac3d7c5c1\") " pod="openshift-ingress/payload-pre-processing-59fbc47bf8-8fw8g"
Sep 18 14:07:39.878513 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:07:39.878516    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qj2rd\" (UniqueName: \"kubernetes.io/projected/c9eadd96-985e-4259-b048-3adac3d7c5c1-kube-api-access-qj2rd\") pod \"payload-pre-processing-59fbc47bf8-8fw8g\" (UID: \"c9eadd96-985e-4259-b048-3adac3d7c5c1\") " pod="openshift-ingress/payload-pre-processing-59fbc47bf8-8fw8g"
Sep 18 14:07:39.878730 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:07:39.878541    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5bdm9\" (UniqueName: \"kubernetes.io/projected/5b843100-3abb-415e-80d7-fd7cff369593-kube-api-access-5bdm9\") pod \"payload-processing-5b54b7fbb7-9v8fj\" (UID: \"5b843100-3abb-415e-80d7-fd7cff369593\") " pod="openshift-ingress/payload-processing-5b54b7fbb7-9v8fj"
Sep 18 14:07:39.878730 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:07:39.878564    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/5b843100-3abb-415e-80d7-fd7cff369593-plugins-config-volume\") pod \"payload-processing-5b54b7fbb7-9v8fj\" (UID: \"5b843100-3abb-415e-80d7-fd7cff369593\") " pod="openshift-ingress/payload-processing-5b54b7fbb7-9v8fj"
Sep 18 14:07:39.879209 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:07:39.879180    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c9eadd96-985e-4259-b048-3adac3d7c5c1-plugins-config-volume\") pod \"payload-pre-processing-59fbc47bf8-8fw8g\" (UID: \"c9eadd96-985e-4259-b048-3adac3d7c5c1\") " pod="openshift-ingress/payload-pre-processing-59fbc47bf8-8fw8g"
Sep 18 14:07:39.887585 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:07:39.887564    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qj2rd\" (UniqueName: \"kubernetes.io/projected/c9eadd96-985e-4259-b048-3adac3d7c5c1-kube-api-access-qj2rd\") pod \"payload-pre-processing-59fbc47bf8-8fw8g\" (UID: \"c9eadd96-985e-4259-b048-3adac3d7c5c1\") " pod="openshift-ingress/payload-pre-processing-59fbc47bf8-8fw8g"
Sep 18 14:07:39.979429 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:07:39.979338    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5bdm9\" (UniqueName: \"kubernetes.io/projected/5b843100-3abb-415e-80d7-fd7cff369593-kube-api-access-5bdm9\") pod \"payload-processing-5b54b7fbb7-9v8fj\" (UID: \"5b843100-3abb-415e-80d7-fd7cff369593\") " pod="openshift-ingress/payload-processing-5b54b7fbb7-9v8fj"
Sep 18 14:07:39.979429 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:07:39.979376    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/5b843100-3abb-415e-80d7-fd7cff369593-plugins-config-volume\") pod \"payload-processing-5b54b7fbb7-9v8fj\" (UID: \"5b843100-3abb-415e-80d7-fd7cff369593\") " pod="openshift-ingress/payload-processing-5b54b7fbb7-9v8fj"
Sep 18 14:07:39.980078 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:07:39.980061    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/5b843100-3abb-415e-80d7-fd7cff369593-plugins-config-volume\") pod \"payload-processing-5b54b7fbb7-9v8fj\" (UID: \"5b843100-3abb-415e-80d7-fd7cff369593\") " pod="openshift-ingress/payload-processing-5b54b7fbb7-9v8fj"
Sep 18 14:07:39.987587 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:07:39.987563    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5bdm9\" (UniqueName: \"kubernetes.io/projected/5b843100-3abb-415e-80d7-fd7cff369593-kube-api-access-5bdm9\") pod \"payload-processing-5b54b7fbb7-9v8fj\" (UID: \"5b843100-3abb-415e-80d7-fd7cff369593\") " pod="openshift-ingress/payload-processing-5b54b7fbb7-9v8fj"
Sep 18 14:07:40.003414 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:07:40.003383    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-59fbc47bf8-8fw8g"
Sep 18 14:07:40.071395 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:07:40.071361    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-5b54b7fbb7-9v8fj"
Sep 18 14:07:40.277055 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:07:40.276933    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-59fbc47bf8-8fw8g"]
Sep 18 14:07:40.279921 ip-10-0-129-16 kubenswrapper[2565]: W0918 14:07:40.279889    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc9eadd96_985e_4259_b048_3adac3d7c5c1.slice/crio-25009753d2e814cb7bfc3b0594d8c0bd3dfc18543288ba28a2c053923a17fecf WatchSource:0}: Error finding container 25009753d2e814cb7bfc3b0594d8c0bd3dfc18543288ba28a2c053923a17fecf: Status 404 returned error can't find the container with id 25009753d2e814cb7bfc3b0594d8c0bd3dfc18543288ba28a2c053923a17fecf
Sep 18 14:07:40.355756 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:07:40.355729    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-5b54b7fbb7-9v8fj"]
Sep 18 14:07:40.357912 ip-10-0-129-16 kubenswrapper[2565]: W0918 14:07:40.357875    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5b843100_3abb_415e_80d7_fd7cff369593.slice/crio-4282c638bc8fa0d8fb60265e526bd5d91332a3e66ebaf8fcd17c07bb121e5319 WatchSource:0}: Error finding container 4282c638bc8fa0d8fb60265e526bd5d91332a3e66ebaf8fcd17c07bb121e5319: Status 404 returned error can't find the container with id 4282c638bc8fa0d8fb60265e526bd5d91332a3e66ebaf8fcd17c07bb121e5319
Sep 18 14:07:40.483114 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:07:40.483075    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-59fbc47bf8-8fw8g" event={"ID":"c9eadd96-985e-4259-b048-3adac3d7c5c1","Type":"ContainerStarted","Data":"25009753d2e814cb7bfc3b0594d8c0bd3dfc18543288ba28a2c053923a17fecf"}
Sep 18 14:07:40.484460 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:07:40.484431    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-5b54b7fbb7-9v8fj" event={"ID":"5b843100-3abb-415e-80d7-fd7cff369593","Type":"ContainerStarted","Data":"4282c638bc8fa0d8fb60265e526bd5d91332a3e66ebaf8fcd17c07bb121e5319"}
Sep 18 14:07:42.497919 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:07:42.497883    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-59fbc47bf8-8fw8g" event={"ID":"c9eadd96-985e-4259-b048-3adac3d7c5c1","Type":"ContainerStarted","Data":"055cde684faa37420f2a7db96bacce2cbfe6f0c6bbd99b5e3e45c0d671a48f5b"}
Sep 18 14:07:42.498384 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:07:42.498082    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-59fbc47bf8-8fw8g"
Sep 18 14:07:42.499600 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:07:42.499576    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-5b54b7fbb7-9v8fj" event={"ID":"5b843100-3abb-415e-80d7-fd7cff369593","Type":"ContainerStarted","Data":"10d160ad49d0f6b402551906e3e88e611234dcadd2dc7d327d7906c24499e899"}
Sep 18 14:07:42.499847 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:07:42.499621    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-5b54b7fbb7-9v8fj"
Sep 18 14:07:42.516599 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:07:42.516553    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-59fbc47bf8-8fw8g" podStartSLOduration=1.9718365599999998 podStartE2EDuration="3.516536174s" totalImagesPullingTime="1.544699614s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 14:07:39 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 14:07:42.513697445 +0000 UTC m=+530.687607118" watchObservedRunningTime="2026-09-18 14:07:42.516536174 +0000 UTC m=+530.690445834"
Sep 18 14:07:42.530896 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:07:42.530847    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-5b54b7fbb7-9v8fj" podStartSLOduration=2.062488218 podStartE2EDuration="3.530831983s" totalImagesPullingTime="1.468343765s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 14:07:39 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 14:07:42.52948945 +0000 UTC m=+530.703399140" watchObservedRunningTime="2026-09-18 14:07:42.530831983 +0000 UTC m=+530.704741680"
Sep 18 14:07:49.536708 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:07:49.536613    2565 generic.go:356] "Generic (PLEG): container finished" podID="77ad5822-b925-43b7-8337-db3d1dd680cb" containerID="f331a0263d5a34b834dffc954331d07436c871af93cd0ce8d27fc2749254d030" exitCode=0
Sep 18 14:07:49.536708 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:07:49.536679    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-phnkt" event={"ID":"77ad5822-b925-43b7-8337-db3d1dd680cb","Type":"ContainerDied","Data":"f331a0263d5a34b834dffc954331d07436c871af93cd0ce8d27fc2749254d030"}
Sep 18 14:07:49.537228 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:07:49.537081    2565 scope.go:117] "RemoveContainer" containerID="f331a0263d5a34b834dffc954331d07436c871af93cd0ce8d27fc2749254d030"
Sep 18 14:07:49.538058 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:07:49.538019    2565 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 18 14:07:49.538113 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:07:49.538097    2565 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 18 14:07:49.538149 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:07:49.538127    2565 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 18 14:07:50.069101 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:07:50.069057    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-phnkt"
Sep 18 14:07:50.546923 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:07:50.546875    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-phnkt" event={"ID":"77ad5822-b925-43b7-8337-db3d1dd680cb","Type":"ContainerStarted","Data":"37bad78f04717ea5abaf1609448b535bc80f1268966895344016fa1bae9e7454"}
Sep 18 14:07:51.386393 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:07:51.386360    2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-phnkt"
Sep 18 14:07:51.387541 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:07:51.387517    2565 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-78bb78ff78-phnkt container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.32:15021/healthz/ready\": dial tcp 10.134.0.32:15021: connect: connection refused" start-of-body=
Sep 18 14:07:51.387642 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:07:51.387562    2565 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-phnkt" podUID="77ad5822-b925-43b7-8337-db3d1dd680cb" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.32:15021/healthz/ready\": dial tcp 10.134.0.32:15021: connect: connection refused"
Sep 18 14:07:52.386478 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:07:52.386437    2565 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-78bb78ff78-phnkt container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.32:15021/healthz/ready\": dial tcp 10.134.0.32:15021: connect: connection refused" start-of-body=
Sep 18 14:07:52.386876 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:07:52.386500    2565 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-phnkt" podUID="77ad5822-b925-43b7-8337-db3d1dd680cb" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.32:15021/healthz/ready\": dial tcp 10.134.0.32:15021: connect: connection refused"
Sep 18 14:07:53.386496 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:07:53.386462    2565 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-78bb78ff78-phnkt container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.32:15021/healthz/ready\": dial tcp 10.134.0.32:15021: connect: connection refused" start-of-body=
Sep 18 14:07:53.386867 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:07:53.386517    2565 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-phnkt" podUID="77ad5822-b925-43b7-8337-db3d1dd680cb" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.32:15021/healthz/ready\": dial tcp 10.134.0.32:15021: connect: connection refused"
Sep 18 14:07:53.506668 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:07:53.506633    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-5b54b7fbb7-9v8fj"
Sep 18 14:07:53.506835 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:07:53.506726    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-59fbc47bf8-8fw8g"
Sep 18 14:07:54.386718 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:07:54.386675    2565 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-78bb78ff78-phnkt container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.32:15021/healthz/ready\": dial tcp 10.134.0.32:15021: connect: connection refused" start-of-body=
Sep 18 14:07:54.387122 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:07:54.386752    2565 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-phnkt" podUID="77ad5822-b925-43b7-8337-db3d1dd680cb" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.32:15021/healthz/ready\": dial tcp 10.134.0.32:15021: connect: connection refused"
Sep 18 14:07:55.386803 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:07:55.386767    2565 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-78bb78ff78-phnkt container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.32:15021/healthz/ready\": dial tcp 10.134.0.32:15021: connect: connection refused" start-of-body=
Sep 18 14:07:55.387211 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:07:55.386825    2565 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-phnkt" podUID="77ad5822-b925-43b7-8337-db3d1dd680cb" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.32:15021/healthz/ready\": dial tcp 10.134.0.32:15021: connect: connection refused"
Sep 18 14:07:56.390233 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:07:56.390204    2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-phnkt"
Sep 18 14:07:56.390602 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:07:56.390487    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-phnkt"
Sep 18 14:07:56.391117 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:07:56.391101    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-phnkt"
Sep 18 14:08:39.600295 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:08:39.600263    2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-distinct-simulated-kserve-59975d69f9-r5l5k"]
Sep 18 14:08:39.602778 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:08:39.602759    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-r5l5k"
Sep 18 14:08:39.607586 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:08:39.607565    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-5zz6x\""
Sep 18 14:08:39.607703 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:08:39.607598    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-distinct-simulated-kserve-self-signed-certs\""
Sep 18 14:08:39.607901 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:08:39.607882    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\""
Sep 18 14:08:39.608020 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:08:39.607935    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\""
Sep 18 14:08:39.649380 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:08:39.649349    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-simulated-kserve-59975d69f9-r5l5k"]
Sep 18 14:08:39.651601 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:08:39.651579    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d7732a87-9d53-48f9-aced-33692ad8a442-tmp-dir\") pod \"e2e-distinct-simulated-kserve-59975d69f9-r5l5k\" (UID: \"d7732a87-9d53-48f9-aced-33692ad8a442\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-r5l5k"
Sep 18 14:08:39.651721 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:08:39.651621    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lj8p2\" (UniqueName: \"kubernetes.io/projected/d7732a87-9d53-48f9-aced-33692ad8a442-kube-api-access-lj8p2\") pod \"e2e-distinct-simulated-kserve-59975d69f9-r5l5k\" (UID: \"d7732a87-9d53-48f9-aced-33692ad8a442\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-r5l5k"
Sep 18 14:08:39.651721 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:08:39.651656    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d7732a87-9d53-48f9-aced-33692ad8a442-dshm\") pod \"e2e-distinct-simulated-kserve-59975d69f9-r5l5k\" (UID: \"d7732a87-9d53-48f9-aced-33692ad8a442\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-r5l5k"
Sep 18 14:08:39.651811 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:08:39.651750    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d7732a87-9d53-48f9-aced-33692ad8a442-tls-certs\") pod \"e2e-distinct-simulated-kserve-59975d69f9-r5l5k\" (UID: \"d7732a87-9d53-48f9-aced-33692ad8a442\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-r5l5k"
Sep 18 14:08:39.651811 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:08:39.651807    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d7732a87-9d53-48f9-aced-33692ad8a442-home\") pod \"e2e-distinct-simulated-kserve-59975d69f9-r5l5k\" (UID: \"d7732a87-9d53-48f9-aced-33692ad8a442\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-r5l5k"
Sep 18 14:08:39.651908 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:08:39.651853    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d7732a87-9d53-48f9-aced-33692ad8a442-model-cache\") pod \"e2e-distinct-simulated-kserve-59975d69f9-r5l5k\" (UID: \"d7732a87-9d53-48f9-aced-33692ad8a442\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-r5l5k"
Sep 18 14:08:39.752880 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:08:39.752846    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d7732a87-9d53-48f9-aced-33692ad8a442-home\") pod \"e2e-distinct-simulated-kserve-59975d69f9-r5l5k\" (UID: \"d7732a87-9d53-48f9-aced-33692ad8a442\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-r5l5k"
Sep 18 14:08:39.753145 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:08:39.752887    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d7732a87-9d53-48f9-aced-33692ad8a442-model-cache\") pod \"e2e-distinct-simulated-kserve-59975d69f9-r5l5k\" (UID: \"d7732a87-9d53-48f9-aced-33692ad8a442\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-r5l5k"
Sep 18 14:08:39.753145 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:08:39.752936    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d7732a87-9d53-48f9-aced-33692ad8a442-tmp-dir\") pod \"e2e-distinct-simulated-kserve-59975d69f9-r5l5k\" (UID: \"d7732a87-9d53-48f9-aced-33692ad8a442\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-r5l5k"
Sep 18 14:08:39.753145 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:08:39.752975    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lj8p2\" (UniqueName: \"kubernetes.io/projected/d7732a87-9d53-48f9-aced-33692ad8a442-kube-api-access-lj8p2\") pod \"e2e-distinct-simulated-kserve-59975d69f9-r5l5k\" (UID: \"d7732a87-9d53-48f9-aced-33692ad8a442\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-r5l5k"
Sep 18 14:08:39.753145 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:08:39.753011    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d7732a87-9d53-48f9-aced-33692ad8a442-dshm\") pod \"e2e-distinct-simulated-kserve-59975d69f9-r5l5k\" (UID: \"d7732a87-9d53-48f9-aced-33692ad8a442\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-r5l5k"
Sep 18 14:08:39.753145 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:08:39.753083    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d7732a87-9d53-48f9-aced-33692ad8a442-tls-certs\") pod \"e2e-distinct-simulated-kserve-59975d69f9-r5l5k\" (UID: \"d7732a87-9d53-48f9-aced-33692ad8a442\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-r5l5k"
Sep 18 14:08:39.753419 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:08:39.753328    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d7732a87-9d53-48f9-aced-33692ad8a442-home\") pod \"e2e-distinct-simulated-kserve-59975d69f9-r5l5k\" (UID: \"d7732a87-9d53-48f9-aced-33692ad8a442\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-r5l5k"
Sep 18 14:08:39.753419 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:08:39.753398    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d7732a87-9d53-48f9-aced-33692ad8a442-model-cache\") pod \"e2e-distinct-simulated-kserve-59975d69f9-r5l5k\" (UID: \"d7732a87-9d53-48f9-aced-33692ad8a442\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-r5l5k"
Sep 18 14:08:39.753547 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:08:39.753516    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d7732a87-9d53-48f9-aced-33692ad8a442-tmp-dir\") pod \"e2e-distinct-simulated-kserve-59975d69f9-r5l5k\" (UID: \"d7732a87-9d53-48f9-aced-33692ad8a442\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-r5l5k"
Sep 18 14:08:39.755503 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:08:39.755476    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d7732a87-9d53-48f9-aced-33692ad8a442-dshm\") pod \"e2e-distinct-simulated-kserve-59975d69f9-r5l5k\" (UID: \"d7732a87-9d53-48f9-aced-33692ad8a442\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-r5l5k"
Sep 18 14:08:39.755801 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:08:39.755780    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d7732a87-9d53-48f9-aced-33692ad8a442-tls-certs\") pod \"e2e-distinct-simulated-kserve-59975d69f9-r5l5k\" (UID: \"d7732a87-9d53-48f9-aced-33692ad8a442\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-r5l5k"
Sep 18 14:08:39.782771 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:08:39.782737    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lj8p2\" (UniqueName: \"kubernetes.io/projected/d7732a87-9d53-48f9-aced-33692ad8a442-kube-api-access-lj8p2\") pod \"e2e-distinct-simulated-kserve-59975d69f9-r5l5k\" (UID: \"d7732a87-9d53-48f9-aced-33692ad8a442\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-r5l5k"
Sep 18 14:08:39.913289 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:08:39.913211    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-r5l5k"
Sep 18 14:08:40.212138 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:08:40.212087    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-simulated-kserve-59975d69f9-r5l5k"]
Sep 18 14:08:40.214373 ip-10-0-129-16 kubenswrapper[2565]: W0918 14:08:40.214348    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd7732a87_9d53_48f9_aced_33692ad8a442.slice/crio-5a9335b9c05a0038b258fe562ebab8207e3810f485bf10ae9b579aed9cf75b18 WatchSource:0}: Error finding container 5a9335b9c05a0038b258fe562ebab8207e3810f485bf10ae9b579aed9cf75b18: Status 404 returned error can't find the container with id 5a9335b9c05a0038b258fe562ebab8207e3810f485bf10ae9b579aed9cf75b18
Sep 18 14:08:40.815050 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:08:40.814992    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-r5l5k" event={"ID":"d7732a87-9d53-48f9-aced-33692ad8a442","Type":"ContainerStarted","Data":"5a9335b9c05a0038b258fe562ebab8207e3810f485bf10ae9b579aed9cf75b18"}
Sep 18 14:08:41.821833 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:08:41.821798    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-r5l5k" event={"ID":"d7732a87-9d53-48f9-aced-33692ad8a442","Type":"ContainerStarted","Data":"ab6cdbe3d1aefc36318f14704cdb3ed43673e43c3dfc2cb1605ed3870f12a195"}
Sep 18 14:08:41.859837 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:08:41.859789    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-r5l5k" podStartSLOduration=1.849148072 podStartE2EDuration="2.859771383s" totalImagesPullingTime="1.010623311s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 14:08:39 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 14:08:41.858983558 +0000 UTC m=+590.032893218" watchObservedRunningTime="2026-09-18 14:08:41.859771383 +0000 UTC m=+590.033681043"
Sep 18 14:08:49.913707 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:08:49.913672    2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-r5l5k"
Sep 18 14:08:49.914107 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:08:49.913729    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-r5l5k"
Sep 18 14:08:49.926213 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:08:49.926187    2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-r5l5k"
Sep 18 14:08:50.879269 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:08:50.879243    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-r5l5k"
Sep 18 14:08:53.862919 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:08:53.862895    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-bvqls_0b0463f5-d647-4a38-b752-44afba77daff/ovn-acl-logging/0.log"
Sep 18 14:08:53.863365 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:08:53.863342    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-bvqls_0b0463f5-d647-4a38-b752-44afba77daff/ovn-acl-logging/0.log"
Sep 18 14:09:25.537727 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:09:25.537694    2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-598466548d-s69nq"]
Sep 18 14:09:25.540627 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:09:25.540605    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-s69nq"
Sep 18 14:09:25.545138 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:09:25.545118    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-trlp-test-simulated-kserve-self-signed-certs\""
Sep 18 14:09:25.555783 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:09:25.555757    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-598466548d-s69nq"]
Sep 18 14:09:25.589918 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:09:25.589881    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xthxf\" (UniqueName: \"kubernetes.io/projected/87bb6eb6-8c3f-4e67-8e6e-39dd729b30a8-kube-api-access-xthxf\") pod \"e2e-trlp-test-simulated-kserve-598466548d-s69nq\" (UID: \"87bb6eb6-8c3f-4e67-8e6e-39dd729b30a8\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-s69nq"
Sep 18 14:09:25.590088 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:09:25.589929    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/87bb6eb6-8c3f-4e67-8e6e-39dd729b30a8-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-598466548d-s69nq\" (UID: \"87bb6eb6-8c3f-4e67-8e6e-39dd729b30a8\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-s69nq"
Sep 18 14:09:25.590088 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:09:25.589956    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/87bb6eb6-8c3f-4e67-8e6e-39dd729b30a8-dshm\") pod \"e2e-trlp-test-simulated-kserve-598466548d-s69nq\" (UID: \"87bb6eb6-8c3f-4e67-8e6e-39dd729b30a8\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-s69nq"
Sep 18 14:09:25.590088 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:09:25.590027    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/87bb6eb6-8c3f-4e67-8e6e-39dd729b30a8-model-cache\") pod \"e2e-trlp-test-simulated-kserve-598466548d-s69nq\" (UID: \"87bb6eb6-8c3f-4e67-8e6e-39dd729b30a8\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-s69nq"
Sep 18 14:09:25.590204 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:09:25.590093    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/87bb6eb6-8c3f-4e67-8e6e-39dd729b30a8-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-598466548d-s69nq\" (UID: \"87bb6eb6-8c3f-4e67-8e6e-39dd729b30a8\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-s69nq"
Sep 18 14:09:25.590204 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:09:25.590150    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/87bb6eb6-8c3f-4e67-8e6e-39dd729b30a8-home\") pod \"e2e-trlp-test-simulated-kserve-598466548d-s69nq\" (UID: \"87bb6eb6-8c3f-4e67-8e6e-39dd729b30a8\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-s69nq"
Sep 18 14:09:25.690809 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:09:25.690776    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xthxf\" (UniqueName: \"kubernetes.io/projected/87bb6eb6-8c3f-4e67-8e6e-39dd729b30a8-kube-api-access-xthxf\") pod \"e2e-trlp-test-simulated-kserve-598466548d-s69nq\" (UID: \"87bb6eb6-8c3f-4e67-8e6e-39dd729b30a8\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-s69nq"
Sep 18 14:09:25.690990 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:09:25.690823    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/87bb6eb6-8c3f-4e67-8e6e-39dd729b30a8-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-598466548d-s69nq\" (UID: \"87bb6eb6-8c3f-4e67-8e6e-39dd729b30a8\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-s69nq"
Sep 18 14:09:25.690990 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:09:25.690868    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/87bb6eb6-8c3f-4e67-8e6e-39dd729b30a8-dshm\") pod \"e2e-trlp-test-simulated-kserve-598466548d-s69nq\" (UID: \"87bb6eb6-8c3f-4e67-8e6e-39dd729b30a8\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-s69nq"
Sep 18 14:09:25.690990 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:09:25.690915    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/87bb6eb6-8c3f-4e67-8e6e-39dd729b30a8-model-cache\") pod \"e2e-trlp-test-simulated-kserve-598466548d-s69nq\" (UID: \"87bb6eb6-8c3f-4e67-8e6e-39dd729b30a8\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-s69nq"
Sep 18 14:09:25.690990 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:09:25.690973    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/87bb6eb6-8c3f-4e67-8e6e-39dd729b30a8-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-598466548d-s69nq\" (UID: \"87bb6eb6-8c3f-4e67-8e6e-39dd729b30a8\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-s69nq"
Sep 18 14:09:25.691218 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:09:25.691080    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/87bb6eb6-8c3f-4e67-8e6e-39dd729b30a8-home\") pod \"e2e-trlp-test-simulated-kserve-598466548d-s69nq\" (UID: \"87bb6eb6-8c3f-4e67-8e6e-39dd729b30a8\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-s69nq"
Sep 18 14:09:25.691454 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:09:25.691420    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/87bb6eb6-8c3f-4e67-8e6e-39dd729b30a8-model-cache\") pod \"e2e-trlp-test-simulated-kserve-598466548d-s69nq\" (UID: \"87bb6eb6-8c3f-4e67-8e6e-39dd729b30a8\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-s69nq"
Sep 18 14:09:25.691585 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:09:25.691452    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/87bb6eb6-8c3f-4e67-8e6e-39dd729b30a8-home\") pod \"e2e-trlp-test-simulated-kserve-598466548d-s69nq\" (UID: \"87bb6eb6-8c3f-4e67-8e6e-39dd729b30a8\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-s69nq"
Sep 18 14:09:25.691585 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:09:25.691431    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/87bb6eb6-8c3f-4e67-8e6e-39dd729b30a8-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-598466548d-s69nq\" (UID: \"87bb6eb6-8c3f-4e67-8e6e-39dd729b30a8\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-s69nq"
Sep 18 14:09:25.693237 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:09:25.693209    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/87bb6eb6-8c3f-4e67-8e6e-39dd729b30a8-dshm\") pod \"e2e-trlp-test-simulated-kserve-598466548d-s69nq\" (UID: \"87bb6eb6-8c3f-4e67-8e6e-39dd729b30a8\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-s69nq"
Sep 18 14:09:25.693527 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:09:25.693509    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/87bb6eb6-8c3f-4e67-8e6e-39dd729b30a8-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-598466548d-s69nq\" (UID: \"87bb6eb6-8c3f-4e67-8e6e-39dd729b30a8\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-s69nq"
Sep 18 14:09:25.699255 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:09:25.699235    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xthxf\" (UniqueName: \"kubernetes.io/projected/87bb6eb6-8c3f-4e67-8e6e-39dd729b30a8-kube-api-access-xthxf\") pod \"e2e-trlp-test-simulated-kserve-598466548d-s69nq\" (UID: \"87bb6eb6-8c3f-4e67-8e6e-39dd729b30a8\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-s69nq"
Sep 18 14:09:25.853481 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:09:25.853381    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-s69nq"
Sep 18 14:09:26.130534 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:09:26.130503    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-598466548d-s69nq"]
Sep 18 14:09:26.132235 ip-10-0-129-16 kubenswrapper[2565]: W0918 14:09:26.132207    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod87bb6eb6_8c3f_4e67_8e6e_39dd729b30a8.slice/crio-4f4bb9c75469b4211a78e9be1c3d2a43b7caa4e6de153516a8352be5f5a6c7ef WatchSource:0}: Error finding container 4f4bb9c75469b4211a78e9be1c3d2a43b7caa4e6de153516a8352be5f5a6c7ef: Status 404 returned error can't find the container with id 4f4bb9c75469b4211a78e9be1c3d2a43b7caa4e6de153516a8352be5f5a6c7ef
Sep 18 14:09:26.133508 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:09:26.133490    2565 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 18 14:09:27.051611 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:09:27.051576    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-s69nq" event={"ID":"87bb6eb6-8c3f-4e67-8e6e-39dd729b30a8","Type":"ContainerStarted","Data":"669f685094de651f1c79c8af121f6781212a86a1f4a33594d3a000cf80d113ff"}
Sep 18 14:09:27.052106 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:09:27.051625    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-s69nq" event={"ID":"87bb6eb6-8c3f-4e67-8e6e-39dd729b30a8","Type":"ContainerStarted","Data":"4f4bb9c75469b4211a78e9be1c3d2a43b7caa4e6de153516a8352be5f5a6c7ef"}
Sep 18 14:09:27.085553 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:09:27.085506    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-s69nq" podStartSLOduration=1.9141764810000002 podStartE2EDuration="2.085491874s" totalImagesPullingTime="171.315393ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 14:09:25 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 14:09:27.084240793 +0000 UTC m=+635.258150453" watchObservedRunningTime="2026-09-18 14:09:27.085491874 +0000 UTC m=+635.259401533"
Sep 18 14:09:35.853613 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:09:35.853582    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-s69nq"
Sep 18 14:09:35.854090 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:09:35.853667    2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-s69nq"
Sep 18 14:09:35.872142 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:09:35.872113    2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-s69nq"
Sep 18 14:09:36.106951 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:09:36.106860    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-s69nq"
Sep 18 14:09:37.354527 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:09:37.354494    2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-74b68447df-jnzsl"]
Sep 18 14:09:37.357800 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:09:37.357777    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-jnzsl"
Sep 18 14:09:37.360645 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:09:37.360618    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"facebook-opt-125m-simulated-kserve-self-signed-certs\""
Sep 18 14:09:37.422129 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:09:37.422097    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-74b68447df-jnzsl"]
Sep 18 14:09:37.507375 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:09:37.507334    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/5c48326f-2106-4e18-840d-119ec89289de-home\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-jnzsl\" (UID: \"5c48326f-2106-4e18-840d-119ec89289de\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-jnzsl"
Sep 18 14:09:37.507577 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:09:37.507451    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/5c48326f-2106-4e18-840d-119ec89289de-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-jnzsl\" (UID: \"5c48326f-2106-4e18-840d-119ec89289de\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-jnzsl"
Sep 18 14:09:37.507577 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:09:37.507484    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/5c48326f-2106-4e18-840d-119ec89289de-dshm\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-jnzsl\" (UID: \"5c48326f-2106-4e18-840d-119ec89289de\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-jnzsl"
Sep 18 14:09:37.507577 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:09:37.507512    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/5c48326f-2106-4e18-840d-119ec89289de-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-jnzsl\" (UID: \"5c48326f-2106-4e18-840d-119ec89289de\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-jnzsl"
Sep 18 14:09:37.507577 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:09:37.507534    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sth9m\" (UniqueName: \"kubernetes.io/projected/5c48326f-2106-4e18-840d-119ec89289de-kube-api-access-sth9m\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-jnzsl\" (UID: \"5c48326f-2106-4e18-840d-119ec89289de\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-jnzsl"
Sep 18 14:09:37.507990 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:09:37.507621    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/5c48326f-2106-4e18-840d-119ec89289de-model-cache\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-jnzsl\" (UID: \"5c48326f-2106-4e18-840d-119ec89289de\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-jnzsl"
Sep 18 14:09:37.608410 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:09:37.608318    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/5c48326f-2106-4e18-840d-119ec89289de-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-jnzsl\" (UID: \"5c48326f-2106-4e18-840d-119ec89289de\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-jnzsl"
Sep 18 14:09:37.608410 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:09:37.608377    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/5c48326f-2106-4e18-840d-119ec89289de-dshm\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-jnzsl\" (UID: \"5c48326f-2106-4e18-840d-119ec89289de\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-jnzsl"
Sep 18 14:09:37.608645 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:09:37.608432    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/5c48326f-2106-4e18-840d-119ec89289de-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-jnzsl\" (UID: \"5c48326f-2106-4e18-840d-119ec89289de\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-jnzsl"
Sep 18 14:09:37.608645 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:09:37.608463    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sth9m\" (UniqueName: \"kubernetes.io/projected/5c48326f-2106-4e18-840d-119ec89289de-kube-api-access-sth9m\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-jnzsl\" (UID: \"5c48326f-2106-4e18-840d-119ec89289de\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-jnzsl"
Sep 18 14:09:37.608645 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:09:37.608509    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/5c48326f-2106-4e18-840d-119ec89289de-model-cache\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-jnzsl\" (UID: \"5c48326f-2106-4e18-840d-119ec89289de\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-jnzsl"
Sep 18 14:09:37.608645 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:09:37.608561    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/5c48326f-2106-4e18-840d-119ec89289de-home\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-jnzsl\" (UID: \"5c48326f-2106-4e18-840d-119ec89289de\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-jnzsl"
Sep 18 14:09:37.608893 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:09:37.608863    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/5c48326f-2106-4e18-840d-119ec89289de-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-jnzsl\" (UID: \"5c48326f-2106-4e18-840d-119ec89289de\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-jnzsl"
Sep 18 14:09:37.608977 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:09:37.608898    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/5c48326f-2106-4e18-840d-119ec89289de-model-cache\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-jnzsl\" (UID: \"5c48326f-2106-4e18-840d-119ec89289de\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-jnzsl"
Sep 18 14:09:37.609068 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:09:37.609015    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/5c48326f-2106-4e18-840d-119ec89289de-home\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-jnzsl\" (UID: \"5c48326f-2106-4e18-840d-119ec89289de\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-jnzsl"
Sep 18 14:09:37.610826 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:09:37.610797    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/5c48326f-2106-4e18-840d-119ec89289de-dshm\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-jnzsl\" (UID: \"5c48326f-2106-4e18-840d-119ec89289de\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-jnzsl"
Sep 18 14:09:37.610996 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:09:37.610979    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/5c48326f-2106-4e18-840d-119ec89289de-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-jnzsl\" (UID: \"5c48326f-2106-4e18-840d-119ec89289de\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-jnzsl"
Sep 18 14:09:37.625918 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:09:37.625880    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-sth9m\" (UniqueName: \"kubernetes.io/projected/5c48326f-2106-4e18-840d-119ec89289de-kube-api-access-sth9m\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-jnzsl\" (UID: \"5c48326f-2106-4e18-840d-119ec89289de\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-jnzsl"
Sep 18 14:09:37.669930 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:09:37.669896    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-jnzsl"
Sep 18 14:09:37.973462 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:09:37.973431    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-74b68447df-jnzsl"]
Sep 18 14:09:37.974829 ip-10-0-129-16 kubenswrapper[2565]: W0918 14:09:37.974796    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5c48326f_2106_4e18_840d_119ec89289de.slice/crio-3898c1f50a3aa3a31835c6a68e7a83b256e8706121a5be3b2880a0c716d95e5e WatchSource:0}: Error finding container 3898c1f50a3aa3a31835c6a68e7a83b256e8706121a5be3b2880a0c716d95e5e: Status 404 returned error can't find the container with id 3898c1f50a3aa3a31835c6a68e7a83b256e8706121a5be3b2880a0c716d95e5e
Sep 18 14:09:38.107058 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:09:38.107006    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-jnzsl" event={"ID":"5c48326f-2106-4e18-840d-119ec89289de","Type":"ContainerStarted","Data":"3898c1f50a3aa3a31835c6a68e7a83b256e8706121a5be3b2880a0c716d95e5e"}
Sep 18 14:09:39.113771 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:09:39.113736    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-jnzsl" event={"ID":"5c48326f-2106-4e18-840d-119ec89289de","Type":"ContainerStarted","Data":"b315a06dae7f98e06c846c5af6477c30927167322326039ff39033d07a13e825"}
Sep 18 14:09:39.155062 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:09:39.154991    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-jnzsl" podStartSLOduration=1.984000811 podStartE2EDuration="2.15497565s" totalImagesPullingTime="170.974839ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 14:09:37 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 14:09:39.153641417 +0000 UTC m=+647.327551092" watchObservedRunningTime="2026-09-18 14:09:39.15497565 +0000 UTC m=+647.328885310"
Sep 18 14:09:47.670695 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:09:47.670653    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-jnzsl"
Sep 18 14:09:47.671091 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:09:47.670763    2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-jnzsl"
Sep 18 14:09:47.685458 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:09:47.685427    2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-jnzsl"
Sep 18 14:09:48.177167 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:09:48.177139    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-jnzsl"
Sep 18 14:11:16.351389 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:11:16.351355    2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w0-4d9222-openshift-default-849bc99f86-v5mg4"]
Sep 18 14:11:16.355293 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:11:16.355273    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-4d9222-openshift-default-849bc99f86-v5mg4"
Sep 18 14:11:16.357842 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:11:16.357822    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-worker-w0-4d9222-openshift-default-dockercfg-5n7nv\""
Sep 18 14:11:16.369170 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:11:16.369147    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w0-4d9222-openshift-default-849bc99f86-v5mg4"]
Sep 18 14:11:16.383705 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:11:16.383672    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f8ff7559-3e54-44b4-a9ce-50f69e2b1908-workload-certs\") pod \"e2e-worker-w0-4d9222-openshift-default-849bc99f86-v5mg4\" (UID: \"f8ff7559-3e54-44b4-a9ce-50f69e2b1908\") " pod="openshift-ingress/e2e-worker-w0-4d9222-openshift-default-849bc99f86-v5mg4"
Sep 18 14:11:16.383842 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:11:16.383746    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f8ff7559-3e54-44b4-a9ce-50f69e2b1908-istio-data\") pod \"e2e-worker-w0-4d9222-openshift-default-849bc99f86-v5mg4\" (UID: \"f8ff7559-3e54-44b4-a9ce-50f69e2b1908\") " pod="openshift-ingress/e2e-worker-w0-4d9222-openshift-default-849bc99f86-v5mg4"
Sep 18 14:11:16.383842 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:11:16.383795    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-d6h64\" (UniqueName: \"kubernetes.io/projected/f8ff7559-3e54-44b4-a9ce-50f69e2b1908-kube-api-access-d6h64\") pod \"e2e-worker-w0-4d9222-openshift-default-849bc99f86-v5mg4\" (UID: \"f8ff7559-3e54-44b4-a9ce-50f69e2b1908\") " pod="openshift-ingress/e2e-worker-w0-4d9222-openshift-default-849bc99f86-v5mg4"
Sep 18 14:11:16.383842 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:11:16.383830    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f8ff7559-3e54-44b4-a9ce-50f69e2b1908-istiod-ca-cert\") pod \"e2e-worker-w0-4d9222-openshift-default-849bc99f86-v5mg4\" (UID: \"f8ff7559-3e54-44b4-a9ce-50f69e2b1908\") " pod="openshift-ingress/e2e-worker-w0-4d9222-openshift-default-849bc99f86-v5mg4"
Sep 18 14:11:16.384015 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:11:16.383956    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f8ff7559-3e54-44b4-a9ce-50f69e2b1908-istio-envoy\") pod \"e2e-worker-w0-4d9222-openshift-default-849bc99f86-v5mg4\" (UID: \"f8ff7559-3e54-44b4-a9ce-50f69e2b1908\") " pod="openshift-ingress/e2e-worker-w0-4d9222-openshift-default-849bc99f86-v5mg4"
Sep 18 14:11:16.384015 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:11:16.384004    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f8ff7559-3e54-44b4-a9ce-50f69e2b1908-istio-podinfo\") pod \"e2e-worker-w0-4d9222-openshift-default-849bc99f86-v5mg4\" (UID: \"f8ff7559-3e54-44b4-a9ce-50f69e2b1908\") " pod="openshift-ingress/e2e-worker-w0-4d9222-openshift-default-849bc99f86-v5mg4"
Sep 18 14:11:16.384173 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:11:16.384087    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f8ff7559-3e54-44b4-a9ce-50f69e2b1908-istio-token\") pod \"e2e-worker-w0-4d9222-openshift-default-849bc99f86-v5mg4\" (UID: \"f8ff7559-3e54-44b4-a9ce-50f69e2b1908\") " pod="openshift-ingress/e2e-worker-w0-4d9222-openshift-default-849bc99f86-v5mg4"
Sep 18 14:11:16.384173 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:11:16.384165    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f8ff7559-3e54-44b4-a9ce-50f69e2b1908-credential-socket\") pod \"e2e-worker-w0-4d9222-openshift-default-849bc99f86-v5mg4\" (UID: \"f8ff7559-3e54-44b4-a9ce-50f69e2b1908\") " pod="openshift-ingress/e2e-worker-w0-4d9222-openshift-default-849bc99f86-v5mg4"
Sep 18 14:11:16.384285 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:11:16.384200    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f8ff7559-3e54-44b4-a9ce-50f69e2b1908-workload-socket\") pod \"e2e-worker-w0-4d9222-openshift-default-849bc99f86-v5mg4\" (UID: \"f8ff7559-3e54-44b4-a9ce-50f69e2b1908\") " pod="openshift-ingress/e2e-worker-w0-4d9222-openshift-default-849bc99f86-v5mg4"
Sep 18 14:11:16.428118 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:11:16.428083    2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w4-93827d-openshift-default-6c5bbfcf98-7nrxb"]
Sep 18 14:11:16.432431 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:11:16.432405    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-93827d-openshift-default-6c5bbfcf98-7nrxb"
Sep 18 14:11:16.436876 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:11:16.436857    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-worker-w4-93827d-openshift-default-dockercfg-5lq2c\""
Sep 18 14:11:16.445937 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:11:16.445907    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w4-93827d-openshift-default-6c5bbfcf98-7nrxb"]
Sep 18 14:11:16.485115 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:11:16.485078    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/97394e77-67c5-435f-9457-f9af6436ad5c-workload-certs\") pod \"e2e-worker-w4-93827d-openshift-default-6c5bbfcf98-7nrxb\" (UID: \"97394e77-67c5-435f-9457-f9af6436ad5c\") " pod="openshift-ingress/e2e-worker-w4-93827d-openshift-default-6c5bbfcf98-7nrxb"
Sep 18 14:11:16.485301 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:11:16.485132    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f8ff7559-3e54-44b4-a9ce-50f69e2b1908-istio-data\") pod \"e2e-worker-w0-4d9222-openshift-default-849bc99f86-v5mg4\" (UID: \"f8ff7559-3e54-44b4-a9ce-50f69e2b1908\") " pod="openshift-ingress/e2e-worker-w0-4d9222-openshift-default-849bc99f86-v5mg4"
Sep 18 14:11:16.485301 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:11:16.485196    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-d6h64\" (UniqueName: \"kubernetes.io/projected/f8ff7559-3e54-44b4-a9ce-50f69e2b1908-kube-api-access-d6h64\") pod \"e2e-worker-w0-4d9222-openshift-default-849bc99f86-v5mg4\" (UID: \"f8ff7559-3e54-44b4-a9ce-50f69e2b1908\") " pod="openshift-ingress/e2e-worker-w0-4d9222-openshift-default-849bc99f86-v5mg4"
Sep 18 14:11:16.485301 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:11:16.485224    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f8ff7559-3e54-44b4-a9ce-50f69e2b1908-istiod-ca-cert\") pod \"e2e-worker-w0-4d9222-openshift-default-849bc99f86-v5mg4\" (UID: \"f8ff7559-3e54-44b4-a9ce-50f69e2b1908\") " pod="openshift-ingress/e2e-worker-w0-4d9222-openshift-default-849bc99f86-v5mg4"
Sep 18 14:11:16.485301 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:11:16.485253    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/97394e77-67c5-435f-9457-f9af6436ad5c-workload-socket\") pod \"e2e-worker-w4-93827d-openshift-default-6c5bbfcf98-7nrxb\" (UID: \"97394e77-67c5-435f-9457-f9af6436ad5c\") " pod="openshift-ingress/e2e-worker-w4-93827d-openshift-default-6c5bbfcf98-7nrxb"
Sep 18 14:11:16.485301 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:11:16.485277    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/97394e77-67c5-435f-9457-f9af6436ad5c-istio-data\") pod \"e2e-worker-w4-93827d-openshift-default-6c5bbfcf98-7nrxb\" (UID: \"97394e77-67c5-435f-9457-f9af6436ad5c\") " pod="openshift-ingress/e2e-worker-w4-93827d-openshift-default-6c5bbfcf98-7nrxb"
Sep 18 14:11:16.485560 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:11:16.485409    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f8ff7559-3e54-44b4-a9ce-50f69e2b1908-istio-envoy\") pod \"e2e-worker-w0-4d9222-openshift-default-849bc99f86-v5mg4\" (UID: \"f8ff7559-3e54-44b4-a9ce-50f69e2b1908\") " pod="openshift-ingress/e2e-worker-w0-4d9222-openshift-default-849bc99f86-v5mg4"
Sep 18 14:11:16.485560 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:11:16.485447    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/97394e77-67c5-435f-9457-f9af6436ad5c-istiod-ca-cert\") pod \"e2e-worker-w4-93827d-openshift-default-6c5bbfcf98-7nrxb\" (UID: \"97394e77-67c5-435f-9457-f9af6436ad5c\") " pod="openshift-ingress/e2e-worker-w4-93827d-openshift-default-6c5bbfcf98-7nrxb"
Sep 18 14:11:16.485560 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:11:16.485486    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f8ff7559-3e54-44b4-a9ce-50f69e2b1908-istio-podinfo\") pod \"e2e-worker-w0-4d9222-openshift-default-849bc99f86-v5mg4\" (UID: \"f8ff7559-3e54-44b4-a9ce-50f69e2b1908\") " pod="openshift-ingress/e2e-worker-w0-4d9222-openshift-default-849bc99f86-v5mg4"
Sep 18 14:11:16.485560 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:11:16.485516    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f8ff7559-3e54-44b4-a9ce-50f69e2b1908-istio-token\") pod \"e2e-worker-w0-4d9222-openshift-default-849bc99f86-v5mg4\" (UID: \"f8ff7559-3e54-44b4-a9ce-50f69e2b1908\") " pod="openshift-ingress/e2e-worker-w0-4d9222-openshift-default-849bc99f86-v5mg4"
Sep 18 14:11:16.485560 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:11:16.485542    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2ghg8\" (UniqueName: \"kubernetes.io/projected/97394e77-67c5-435f-9457-f9af6436ad5c-kube-api-access-2ghg8\") pod \"e2e-worker-w4-93827d-openshift-default-6c5bbfcf98-7nrxb\" (UID: \"97394e77-67c5-435f-9457-f9af6436ad5c\") " pod="openshift-ingress/e2e-worker-w4-93827d-openshift-default-6c5bbfcf98-7nrxb"
Sep 18 14:11:16.485748 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:11:16.485583    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f8ff7559-3e54-44b4-a9ce-50f69e2b1908-istio-data\") pod \"e2e-worker-w0-4d9222-openshift-default-849bc99f86-v5mg4\" (UID: \"f8ff7559-3e54-44b4-a9ce-50f69e2b1908\") " pod="openshift-ingress/e2e-worker-w0-4d9222-openshift-default-849bc99f86-v5mg4"
Sep 18 14:11:16.485748 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:11:16.485590    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/97394e77-67c5-435f-9457-f9af6436ad5c-credential-socket\") pod \"e2e-worker-w4-93827d-openshift-default-6c5bbfcf98-7nrxb\" (UID: \"97394e77-67c5-435f-9457-f9af6436ad5c\") " pod="openshift-ingress/e2e-worker-w4-93827d-openshift-default-6c5bbfcf98-7nrxb"
Sep 18 14:11:16.485748 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:11:16.485620    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/97394e77-67c5-435f-9457-f9af6436ad5c-istio-envoy\") pod \"e2e-worker-w4-93827d-openshift-default-6c5bbfcf98-7nrxb\" (UID: \"97394e77-67c5-435f-9457-f9af6436ad5c\") " pod="openshift-ingress/e2e-worker-w4-93827d-openshift-default-6c5bbfcf98-7nrxb"
Sep 18 14:11:16.485901 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:11:16.485886    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f8ff7559-3e54-44b4-a9ce-50f69e2b1908-credential-socket\") pod \"e2e-worker-w0-4d9222-openshift-default-849bc99f86-v5mg4\" (UID: \"f8ff7559-3e54-44b4-a9ce-50f69e2b1908\") " pod="openshift-ingress/e2e-worker-w0-4d9222-openshift-default-849bc99f86-v5mg4"
Sep 18 14:11:16.485954 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:11:16.485922    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/97394e77-67c5-435f-9457-f9af6436ad5c-istio-token\") pod \"e2e-worker-w4-93827d-openshift-default-6c5bbfcf98-7nrxb\" (UID: \"97394e77-67c5-435f-9457-f9af6436ad5c\") " pod="openshift-ingress/e2e-worker-w4-93827d-openshift-default-6c5bbfcf98-7nrxb"
Sep 18 14:11:16.486020 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:11:16.485968    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f8ff7559-3e54-44b4-a9ce-50f69e2b1908-workload-socket\") pod \"e2e-worker-w0-4d9222-openshift-default-849bc99f86-v5mg4\" (UID: \"f8ff7559-3e54-44b4-a9ce-50f69e2b1908\") " pod="openshift-ingress/e2e-worker-w0-4d9222-openshift-default-849bc99f86-v5mg4"
Sep 18 14:11:16.486170 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:11:16.486024    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f8ff7559-3e54-44b4-a9ce-50f69e2b1908-istiod-ca-cert\") pod \"e2e-worker-w0-4d9222-openshift-default-849bc99f86-v5mg4\" (UID: \"f8ff7559-3e54-44b4-a9ce-50f69e2b1908\") " pod="openshift-ingress/e2e-worker-w0-4d9222-openshift-default-849bc99f86-v5mg4"
Sep 18 14:11:16.486258 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:11:16.486013    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/97394e77-67c5-435f-9457-f9af6436ad5c-istio-podinfo\") pod \"e2e-worker-w4-93827d-openshift-default-6c5bbfcf98-7nrxb\" (UID: \"97394e77-67c5-435f-9457-f9af6436ad5c\") " pod="openshift-ingress/e2e-worker-w4-93827d-openshift-default-6c5bbfcf98-7nrxb"
Sep 18 14:11:16.486344 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:11:16.486256    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f8ff7559-3e54-44b4-a9ce-50f69e2b1908-credential-socket\") pod \"e2e-worker-w0-4d9222-openshift-default-849bc99f86-v5mg4\" (UID: \"f8ff7559-3e54-44b4-a9ce-50f69e2b1908\") " pod="openshift-ingress/e2e-worker-w0-4d9222-openshift-default-849bc99f86-v5mg4"
Sep 18 14:11:16.487977 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:11:16.487756    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f8ff7559-3e54-44b4-a9ce-50f69e2b1908-workload-certs\") pod \"e2e-worker-w0-4d9222-openshift-default-849bc99f86-v5mg4\" (UID: \"f8ff7559-3e54-44b4-a9ce-50f69e2b1908\") " pod="openshift-ingress/e2e-worker-w0-4d9222-openshift-default-849bc99f86-v5mg4"
Sep 18 14:11:16.487977 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:11:16.487923    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f8ff7559-3e54-44b4-a9ce-50f69e2b1908-workload-socket\") pod \"e2e-worker-w0-4d9222-openshift-default-849bc99f86-v5mg4\" (UID: \"f8ff7559-3e54-44b4-a9ce-50f69e2b1908\") " pod="openshift-ingress/e2e-worker-w0-4d9222-openshift-default-849bc99f86-v5mg4"
Sep 18 14:11:16.488233 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:11:16.488152    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f8ff7559-3e54-44b4-a9ce-50f69e2b1908-workload-certs\") pod \"e2e-worker-w0-4d9222-openshift-default-849bc99f86-v5mg4\" (UID: \"f8ff7559-3e54-44b4-a9ce-50f69e2b1908\") " pod="openshift-ingress/e2e-worker-w0-4d9222-openshift-default-849bc99f86-v5mg4"
Sep 18 14:11:16.488935 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:11:16.488900    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f8ff7559-3e54-44b4-a9ce-50f69e2b1908-istio-podinfo\") pod \"e2e-worker-w0-4d9222-openshift-default-849bc99f86-v5mg4\" (UID: \"f8ff7559-3e54-44b4-a9ce-50f69e2b1908\") " pod="openshift-ingress/e2e-worker-w0-4d9222-openshift-default-849bc99f86-v5mg4"
Sep 18 14:11:16.489721 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:11:16.489690    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f8ff7559-3e54-44b4-a9ce-50f69e2b1908-istio-envoy\") pod \"e2e-worker-w0-4d9222-openshift-default-849bc99f86-v5mg4\" (UID: \"f8ff7559-3e54-44b4-a9ce-50f69e2b1908\") " pod="openshift-ingress/e2e-worker-w0-4d9222-openshift-default-849bc99f86-v5mg4"
Sep 18 14:11:16.507136 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:11:16.507100    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f8ff7559-3e54-44b4-a9ce-50f69e2b1908-istio-token\") pod \"e2e-worker-w0-4d9222-openshift-default-849bc99f86-v5mg4\" (UID: \"f8ff7559-3e54-44b4-a9ce-50f69e2b1908\") " pod="openshift-ingress/e2e-worker-w0-4d9222-openshift-default-849bc99f86-v5mg4"
Sep 18 14:11:16.507258 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:11:16.507163    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-d6h64\" (UniqueName: \"kubernetes.io/projected/f8ff7559-3e54-44b4-a9ce-50f69e2b1908-kube-api-access-d6h64\") pod \"e2e-worker-w0-4d9222-openshift-default-849bc99f86-v5mg4\" (UID: \"f8ff7559-3e54-44b4-a9ce-50f69e2b1908\") " pod="openshift-ingress/e2e-worker-w0-4d9222-openshift-default-849bc99f86-v5mg4"
Sep 18 14:11:16.589203 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:11:16.589171    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/97394e77-67c5-435f-9457-f9af6436ad5c-istio-token\") pod \"e2e-worker-w4-93827d-openshift-default-6c5bbfcf98-7nrxb\" (UID: \"97394e77-67c5-435f-9457-f9af6436ad5c\") " pod="openshift-ingress/e2e-worker-w4-93827d-openshift-default-6c5bbfcf98-7nrxb"
Sep 18 14:11:16.589203 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:11:16.589208    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/97394e77-67c5-435f-9457-f9af6436ad5c-istio-podinfo\") pod \"e2e-worker-w4-93827d-openshift-default-6c5bbfcf98-7nrxb\" (UID: \"97394e77-67c5-435f-9457-f9af6436ad5c\") " pod="openshift-ingress/e2e-worker-w4-93827d-openshift-default-6c5bbfcf98-7nrxb"
Sep 18 14:11:16.589466 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:11:16.589258    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/97394e77-67c5-435f-9457-f9af6436ad5c-workload-certs\") pod \"e2e-worker-w4-93827d-openshift-default-6c5bbfcf98-7nrxb\" (UID: \"97394e77-67c5-435f-9457-f9af6436ad5c\") " pod="openshift-ingress/e2e-worker-w4-93827d-openshift-default-6c5bbfcf98-7nrxb"
Sep 18 14:11:16.589466 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:11:16.589294    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/97394e77-67c5-435f-9457-f9af6436ad5c-workload-socket\") pod \"e2e-worker-w4-93827d-openshift-default-6c5bbfcf98-7nrxb\" (UID: \"97394e77-67c5-435f-9457-f9af6436ad5c\") " pod="openshift-ingress/e2e-worker-w4-93827d-openshift-default-6c5bbfcf98-7nrxb"
Sep 18 14:11:16.589466 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:11:16.589311    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/97394e77-67c5-435f-9457-f9af6436ad5c-istio-data\") pod \"e2e-worker-w4-93827d-openshift-default-6c5bbfcf98-7nrxb\" (UID: \"97394e77-67c5-435f-9457-f9af6436ad5c\") " pod="openshift-ingress/e2e-worker-w4-93827d-openshift-default-6c5bbfcf98-7nrxb"
Sep 18 14:11:16.589466 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:11:16.589383    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/97394e77-67c5-435f-9457-f9af6436ad5c-istiod-ca-cert\") pod \"e2e-worker-w4-93827d-openshift-default-6c5bbfcf98-7nrxb\" (UID: \"97394e77-67c5-435f-9457-f9af6436ad5c\") " pod="openshift-ingress/e2e-worker-w4-93827d-openshift-default-6c5bbfcf98-7nrxb"
Sep 18 14:11:16.589466 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:11:16.589418    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2ghg8\" (UniqueName: \"kubernetes.io/projected/97394e77-67c5-435f-9457-f9af6436ad5c-kube-api-access-2ghg8\") pod \"e2e-worker-w4-93827d-openshift-default-6c5bbfcf98-7nrxb\" (UID: \"97394e77-67c5-435f-9457-f9af6436ad5c\") " pod="openshift-ingress/e2e-worker-w4-93827d-openshift-default-6c5bbfcf98-7nrxb"
Sep 18 14:11:16.589466 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:11:16.589456    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/97394e77-67c5-435f-9457-f9af6436ad5c-credential-socket\") pod \"e2e-worker-w4-93827d-openshift-default-6c5bbfcf98-7nrxb\" (UID: \"97394e77-67c5-435f-9457-f9af6436ad5c\") " pod="openshift-ingress/e2e-worker-w4-93827d-openshift-default-6c5bbfcf98-7nrxb"
Sep 18 14:11:16.589823 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:11:16.589483    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/97394e77-67c5-435f-9457-f9af6436ad5c-istio-envoy\") pod \"e2e-worker-w4-93827d-openshift-default-6c5bbfcf98-7nrxb\" (UID: \"97394e77-67c5-435f-9457-f9af6436ad5c\") " pod="openshift-ingress/e2e-worker-w4-93827d-openshift-default-6c5bbfcf98-7nrxb"
Sep 18 14:11:16.589823 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:11:16.589713    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/97394e77-67c5-435f-9457-f9af6436ad5c-workload-certs\") pod \"e2e-worker-w4-93827d-openshift-default-6c5bbfcf98-7nrxb\" (UID: \"97394e77-67c5-435f-9457-f9af6436ad5c\") " pod="openshift-ingress/e2e-worker-w4-93827d-openshift-default-6c5bbfcf98-7nrxb"
Sep 18 14:11:16.589823 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:11:16.589782    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/97394e77-67c5-435f-9457-f9af6436ad5c-workload-socket\") pod \"e2e-worker-w4-93827d-openshift-default-6c5bbfcf98-7nrxb\" (UID: \"97394e77-67c5-435f-9457-f9af6436ad5c\") " pod="openshift-ingress/e2e-worker-w4-93827d-openshift-default-6c5bbfcf98-7nrxb"
Sep 18 14:11:16.589997 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:11:16.589889    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/97394e77-67c5-435f-9457-f9af6436ad5c-istio-data\") pod \"e2e-worker-w4-93827d-openshift-default-6c5bbfcf98-7nrxb\" (UID: \"97394e77-67c5-435f-9457-f9af6436ad5c\") " pod="openshift-ingress/e2e-worker-w4-93827d-openshift-default-6c5bbfcf98-7nrxb"
Sep 18 14:11:16.589997 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:11:16.589950    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/97394e77-67c5-435f-9457-f9af6436ad5c-credential-socket\") pod \"e2e-worker-w4-93827d-openshift-default-6c5bbfcf98-7nrxb\" (UID: \"97394e77-67c5-435f-9457-f9af6436ad5c\") " pod="openshift-ingress/e2e-worker-w4-93827d-openshift-default-6c5bbfcf98-7nrxb"
Sep 18 14:11:16.590224 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:11:16.590205    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/97394e77-67c5-435f-9457-f9af6436ad5c-istiod-ca-cert\") pod \"e2e-worker-w4-93827d-openshift-default-6c5bbfcf98-7nrxb\" (UID: \"97394e77-67c5-435f-9457-f9af6436ad5c\") " pod="openshift-ingress/e2e-worker-w4-93827d-openshift-default-6c5bbfcf98-7nrxb"
Sep 18 14:11:16.591746 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:11:16.591725    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/97394e77-67c5-435f-9457-f9af6436ad5c-istio-envoy\") pod \"e2e-worker-w4-93827d-openshift-default-6c5bbfcf98-7nrxb\" (UID: \"97394e77-67c5-435f-9457-f9af6436ad5c\") " pod="openshift-ingress/e2e-worker-w4-93827d-openshift-default-6c5bbfcf98-7nrxb"
Sep 18 14:11:16.592045 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:11:16.592010    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/97394e77-67c5-435f-9457-f9af6436ad5c-istio-podinfo\") pod \"e2e-worker-w4-93827d-openshift-default-6c5bbfcf98-7nrxb\" (UID: \"97394e77-67c5-435f-9457-f9af6436ad5c\") " pod="openshift-ingress/e2e-worker-w4-93827d-openshift-default-6c5bbfcf98-7nrxb"
Sep 18 14:11:16.601517 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:11:16.601460    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2ghg8\" (UniqueName: \"kubernetes.io/projected/97394e77-67c5-435f-9457-f9af6436ad5c-kube-api-access-2ghg8\") pod \"e2e-worker-w4-93827d-openshift-default-6c5bbfcf98-7nrxb\" (UID: \"97394e77-67c5-435f-9457-f9af6436ad5c\") " pod="openshift-ingress/e2e-worker-w4-93827d-openshift-default-6c5bbfcf98-7nrxb"
Sep 18 14:11:16.602483 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:11:16.602466    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/97394e77-67c5-435f-9457-f9af6436ad5c-istio-token\") pod \"e2e-worker-w4-93827d-openshift-default-6c5bbfcf98-7nrxb\" (UID: \"97394e77-67c5-435f-9457-f9af6436ad5c\") " pod="openshift-ingress/e2e-worker-w4-93827d-openshift-default-6c5bbfcf98-7nrxb"
Sep 18 14:11:16.666397 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:11:16.666364    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-4d9222-openshift-default-849bc99f86-v5mg4"
Sep 18 14:11:16.746421 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:11:16.746389    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-93827d-openshift-default-6c5bbfcf98-7nrxb"
Sep 18 14:11:16.955471 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:11:16.955442    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w0-4d9222-openshift-default-849bc99f86-v5mg4"]
Sep 18 14:11:16.957234 ip-10-0-129-16 kubenswrapper[2565]: W0918 14:11:16.957211    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf8ff7559_3e54_44b4_a9ce_50f69e2b1908.slice/crio-c6fb05d56dc5f22a17092f6bd8caf731e2f9d66755402e415b5dad3af8551927 WatchSource:0}: Error finding container c6fb05d56dc5f22a17092f6bd8caf731e2f9d66755402e415b5dad3af8551927: Status 404 returned error can't find the container with id c6fb05d56dc5f22a17092f6bd8caf731e2f9d66755402e415b5dad3af8551927
Sep 18 14:11:16.959320 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:11:16.959291    2565 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 18 14:11:16.959419 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:11:16.959350    2565 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 18 14:11:16.959419 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:11:16.959378    2565 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 18 14:11:17.033260 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:11:17.033230    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w4-93827d-openshift-default-6c5bbfcf98-7nrxb"]
Sep 18 14:11:17.036925 ip-10-0-129-16 kubenswrapper[2565]: W0918 14:11:17.036892    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod97394e77_67c5_435f_9457_f9af6436ad5c.slice/crio-bf8f522046d319b170f150aadbbeb99575d9bbceb8042cce4bbb6e4275107327 WatchSource:0}: Error finding container bf8f522046d319b170f150aadbbeb99575d9bbceb8042cce4bbb6e4275107327: Status 404 returned error can't find the container with id bf8f522046d319b170f150aadbbeb99575d9bbceb8042cce4bbb6e4275107327
Sep 18 14:11:17.039574 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:11:17.039528    2565 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 18 14:11:17.039668 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:11:17.039649    2565 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 18 14:11:17.039748 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:11:17.039697    2565 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 18 14:11:17.621824 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:11:17.621788    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-93827d-openshift-default-6c5bbfcf98-7nrxb" event={"ID":"97394e77-67c5-435f-9457-f9af6436ad5c","Type":"ContainerStarted","Data":"cf019763542150ecaef6b5767da1064e3fe353cd4c2e357badc0cb9b908aaad0"}
Sep 18 14:11:17.622295 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:11:17.621845    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-93827d-openshift-default-6c5bbfcf98-7nrxb" event={"ID":"97394e77-67c5-435f-9457-f9af6436ad5c","Type":"ContainerStarted","Data":"bf8f522046d319b170f150aadbbeb99575d9bbceb8042cce4bbb6e4275107327"}
Sep 18 14:11:17.623500 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:11:17.623471    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-4d9222-openshift-default-849bc99f86-v5mg4" event={"ID":"f8ff7559-3e54-44b4-a9ce-50f69e2b1908","Type":"ContainerStarted","Data":"3470f50403473d7e198a40e9941623dc0840c6d203553afd39d44680ac8ed2cd"}
Sep 18 14:11:17.623619 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:11:17.623532    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-4d9222-openshift-default-849bc99f86-v5mg4" event={"ID":"f8ff7559-3e54-44b4-a9ce-50f69e2b1908","Type":"ContainerStarted","Data":"c6fb05d56dc5f22a17092f6bd8caf731e2f9d66755402e415b5dad3af8551927"}
Sep 18 14:11:17.650743 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:11:17.650691    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w4-93827d-openshift-default-6c5bbfcf98-7nrxb" podStartSLOduration=1.65067516 podStartE2EDuration="1.65067516s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 14:11:16 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 14:11:17.650494106 +0000 UTC m=+745.824403786" watchObservedRunningTime="2026-09-18 14:11:17.65067516 +0000 UTC m=+745.824584820"
Sep 18 14:11:17.667257 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:11:17.667228    2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w0-4d9222-openshift-default-849bc99f86-v5mg4"
Sep 18 14:11:17.671412 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:11:17.671364    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w0-4d9222-openshift-default-849bc99f86-v5mg4" podStartSLOduration=1.671349342 podStartE2EDuration="1.671349342s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 14:11:16 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 14:11:17.670429931 +0000 UTC m=+745.844339611" watchObservedRunningTime="2026-09-18 14:11:17.671349342 +0000 UTC m=+745.845259001"
Sep 18 14:11:17.747483 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:11:17.747450    2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w4-93827d-openshift-default-6c5bbfcf98-7nrxb"
Sep 18 14:11:18.672578 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:11:18.672543    2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w0-4d9222-openshift-default-849bc99f86-v5mg4"
Sep 18 14:11:18.752302 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:11:18.752268    2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w4-93827d-openshift-default-6c5bbfcf98-7nrxb"
Sep 18 14:11:19.635338 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:11:19.635300    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w0-4d9222-openshift-default-849bc99f86-v5mg4"
Sep 18 14:11:19.635554 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:11:19.635479    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w4-93827d-openshift-default-6c5bbfcf98-7nrxb"
Sep 18 14:11:19.636288 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:11:19.636261    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w4-93827d-openshift-default-6c5bbfcf98-7nrxb"
Sep 18 14:11:19.636428 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:11:19.636401    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w0-4d9222-openshift-default-849bc99f86-v5mg4"
Sep 18 14:11:22.276050 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:11:22.276009    2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w0-4d9222-67d45969f8-gwf8b"]
Sep 18 14:11:22.284789 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:11:22.284755    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-4d9222-67d45969f8-gwf8b"
Sep 18 14:11:22.288441 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:11:22.288163    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-worker-w0-4d9222-dockercfg-2jpks\""
Sep 18 14:11:22.289245 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:11:22.289089    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-worker-w0-4d9222\""
Sep 18 14:11:22.292009 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:11:22.291983    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w0-4d9222-67d45969f8-gwf8b"]
Sep 18 14:11:22.352882 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:11:22.352846    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/609ab3f8-150f-40fe-956c-139806aee3ce-plugins-config-volume\") pod \"payload-pre-processing-e2e-worker-w0-4d9222-67d45969f8-gwf8b\" (UID: \"609ab3f8-150f-40fe-956c-139806aee3ce\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-4d9222-67d45969f8-gwf8b"
Sep 18 14:11:22.353075 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:11:22.352943    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-m6znr\" (UniqueName: \"kubernetes.io/projected/609ab3f8-150f-40fe-956c-139806aee3ce-kube-api-access-m6znr\") pod \"payload-pre-processing-e2e-worker-w0-4d9222-67d45969f8-gwf8b\" (UID: \"609ab3f8-150f-40fe-956c-139806aee3ce\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-4d9222-67d45969f8-gwf8b"
Sep 18 14:11:22.453826 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:11:22.453790    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/609ab3f8-150f-40fe-956c-139806aee3ce-plugins-config-volume\") pod \"payload-pre-processing-e2e-worker-w0-4d9222-67d45969f8-gwf8b\" (UID: \"609ab3f8-150f-40fe-956c-139806aee3ce\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-4d9222-67d45969f8-gwf8b"
Sep 18 14:11:22.454002 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:11:22.453856    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-m6znr\" (UniqueName: \"kubernetes.io/projected/609ab3f8-150f-40fe-956c-139806aee3ce-kube-api-access-m6znr\") pod \"payload-pre-processing-e2e-worker-w0-4d9222-67d45969f8-gwf8b\" (UID: \"609ab3f8-150f-40fe-956c-139806aee3ce\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-4d9222-67d45969f8-gwf8b"
Sep 18 14:11:22.454568 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:11:22.454547    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/609ab3f8-150f-40fe-956c-139806aee3ce-plugins-config-volume\") pod \"payload-pre-processing-e2e-worker-w0-4d9222-67d45969f8-gwf8b\" (UID: \"609ab3f8-150f-40fe-956c-139806aee3ce\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-4d9222-67d45969f8-gwf8b"
Sep 18 14:11:22.496842 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:11:22.496808    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-m6znr\" (UniqueName: \"kubernetes.io/projected/609ab3f8-150f-40fe-956c-139806aee3ce-kube-api-access-m6znr\") pod \"payload-pre-processing-e2e-worker-w0-4d9222-67d45969f8-gwf8b\" (UID: \"609ab3f8-150f-40fe-956c-139806aee3ce\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-4d9222-67d45969f8-gwf8b"
Sep 18 14:11:22.598518 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:11:22.598480    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-4d9222-67d45969f8-gwf8b"
Sep 18 14:11:22.903406 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:11:22.903383    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w0-4d9222-67d45969f8-gwf8b"]
Sep 18 14:11:22.905672 ip-10-0-129-16 kubenswrapper[2565]: W0918 14:11:22.905638    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod609ab3f8_150f_40fe_956c_139806aee3ce.slice/crio-80d9a6f343564abcd086f602d8623ee746c20bd8a487ff51bf7d0226c8a6df07 WatchSource:0}: Error finding container 80d9a6f343564abcd086f602d8623ee746c20bd8a487ff51bf7d0226c8a6df07: Status 404 returned error can't find the container with id 80d9a6f343564abcd086f602d8623ee746c20bd8a487ff51bf7d0226c8a6df07
Sep 18 14:11:23.646298 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:11:23.646265    2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w4-93827d-d6d7cb69c-m4ztv"]
Sep 18 14:11:23.650441 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:11:23.650420    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-worker-w4-93827d-d6d7cb69c-m4ztv"
Sep 18 14:11:23.653793 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:11:23.653770    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-worker-w4-93827d-dockercfg-s9zql\""
Sep 18 14:11:23.653890 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:11:23.653777    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-worker-w4-93827d\""
Sep 18 14:11:23.663595 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:11:23.663559    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-4d9222-67d45969f8-gwf8b" event={"ID":"609ab3f8-150f-40fe-956c-139806aee3ce","Type":"ContainerStarted","Data":"e61330fad1730e830efaf350fd60aea2b2c56292d992579f4ea9ef889c7eaac5"}
Sep 18 14:11:23.663730 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:11:23.663623    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-4d9222-67d45969f8-gwf8b" event={"ID":"609ab3f8-150f-40fe-956c-139806aee3ce","Type":"ContainerStarted","Data":"80d9a6f343564abcd086f602d8623ee746c20bd8a487ff51bf7d0226c8a6df07"}
Sep 18 14:11:23.663730 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:11:23.663645    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w4-93827d-d6d7cb69c-m4ztv"]
Sep 18 14:11:23.663730 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:11:23.663704    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-4d9222-67d45969f8-gwf8b"
Sep 18 14:11:23.702304 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:11:23.702252    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-4d9222-67d45969f8-gwf8b" podStartSLOduration=1.7022357590000001 podStartE2EDuration="1.702235759s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 14:11:22 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 14:11:23.700242182 +0000 UTC m=+751.874152059" watchObservedRunningTime="2026-09-18 14:11:23.702235759 +0000 UTC m=+751.876145483"
Sep 18 14:11:23.716321 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:11:23.716284    2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w4-93827d-7ff5557fbb-lbszk"]
Sep 18 14:11:23.720256 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:11:23.720237    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-w4-93827d-7ff5557fbb-lbszk"
Sep 18 14:11:23.734909 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:11:23.734886    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w4-93827d-7ff5557fbb-lbszk"]
Sep 18 14:11:23.767435 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:11:23.767402    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e2ac808b-b832-4228-a2e3-ac84d0bfc436-plugins-config-volume\") pod \"payload-processing-e2e-worker-w4-93827d-d6d7cb69c-m4ztv\" (UID: \"e2ac808b-b832-4228-a2e3-ac84d0bfc436\") " pod="openshift-ingress/payload-processing-e2e-worker-w4-93827d-d6d7cb69c-m4ztv"
Sep 18 14:11:23.767602 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:11:23.767464    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5zgg8\" (UniqueName: \"kubernetes.io/projected/d3515ddb-013d-488f-a714-efbb75884469-kube-api-access-5zgg8\") pod \"payload-pre-processing-e2e-worker-w4-93827d-7ff5557fbb-lbszk\" (UID: \"d3515ddb-013d-488f-a714-efbb75884469\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w4-93827d-7ff5557fbb-lbszk"
Sep 18 14:11:23.767602 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:11:23.767520    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-l4qn9\" (UniqueName: \"kubernetes.io/projected/e2ac808b-b832-4228-a2e3-ac84d0bfc436-kube-api-access-l4qn9\") pod \"payload-processing-e2e-worker-w4-93827d-d6d7cb69c-m4ztv\" (UID: \"e2ac808b-b832-4228-a2e3-ac84d0bfc436\") " pod="openshift-ingress/payload-processing-e2e-worker-w4-93827d-d6d7cb69c-m4ztv"
Sep 18 14:11:23.767693 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:11:23.767674    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d3515ddb-013d-488f-a714-efbb75884469-plugins-config-volume\") pod \"payload-pre-processing-e2e-worker-w4-93827d-7ff5557fbb-lbszk\" (UID: \"d3515ddb-013d-488f-a714-efbb75884469\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w4-93827d-7ff5557fbb-lbszk"
Sep 18 14:11:23.869027 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:11:23.868990    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e2ac808b-b832-4228-a2e3-ac84d0bfc436-plugins-config-volume\") pod \"payload-processing-e2e-worker-w4-93827d-d6d7cb69c-m4ztv\" (UID: \"e2ac808b-b832-4228-a2e3-ac84d0bfc436\") " pod="openshift-ingress/payload-processing-e2e-worker-w4-93827d-d6d7cb69c-m4ztv"
Sep 18 14:11:23.869234 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:11:23.869076    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5zgg8\" (UniqueName: \"kubernetes.io/projected/d3515ddb-013d-488f-a714-efbb75884469-kube-api-access-5zgg8\") pod \"payload-pre-processing-e2e-worker-w4-93827d-7ff5557fbb-lbszk\" (UID: \"d3515ddb-013d-488f-a714-efbb75884469\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w4-93827d-7ff5557fbb-lbszk"
Sep 18 14:11:23.869234 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:11:23.869106    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-l4qn9\" (UniqueName: \"kubernetes.io/projected/e2ac808b-b832-4228-a2e3-ac84d0bfc436-kube-api-access-l4qn9\") pod \"payload-processing-e2e-worker-w4-93827d-d6d7cb69c-m4ztv\" (UID: \"e2ac808b-b832-4228-a2e3-ac84d0bfc436\") " pod="openshift-ingress/payload-processing-e2e-worker-w4-93827d-d6d7cb69c-m4ztv"
Sep 18 14:11:23.869395 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:11:23.869371    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d3515ddb-013d-488f-a714-efbb75884469-plugins-config-volume\") pod \"payload-pre-processing-e2e-worker-w4-93827d-7ff5557fbb-lbszk\" (UID: \"d3515ddb-013d-488f-a714-efbb75884469\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w4-93827d-7ff5557fbb-lbszk"
Sep 18 14:11:23.869826 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:11:23.869807    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e2ac808b-b832-4228-a2e3-ac84d0bfc436-plugins-config-volume\") pod \"payload-processing-e2e-worker-w4-93827d-d6d7cb69c-m4ztv\" (UID: \"e2ac808b-b832-4228-a2e3-ac84d0bfc436\") " pod="openshift-ingress/payload-processing-e2e-worker-w4-93827d-d6d7cb69c-m4ztv"
Sep 18 14:11:23.870000 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:11:23.869983    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d3515ddb-013d-488f-a714-efbb75884469-plugins-config-volume\") pod \"payload-pre-processing-e2e-worker-w4-93827d-7ff5557fbb-lbszk\" (UID: \"d3515ddb-013d-488f-a714-efbb75884469\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w4-93827d-7ff5557fbb-lbszk"
Sep 18 14:11:23.880610 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:11:23.880572    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5zgg8\" (UniqueName: \"kubernetes.io/projected/d3515ddb-013d-488f-a714-efbb75884469-kube-api-access-5zgg8\") pod \"payload-pre-processing-e2e-worker-w4-93827d-7ff5557fbb-lbszk\" (UID: \"d3515ddb-013d-488f-a714-efbb75884469\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w4-93827d-7ff5557fbb-lbszk"
Sep 18 14:11:23.880894 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:11:23.880873    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-l4qn9\" (UniqueName: \"kubernetes.io/projected/e2ac808b-b832-4228-a2e3-ac84d0bfc436-kube-api-access-l4qn9\") pod \"payload-processing-e2e-worker-w4-93827d-d6d7cb69c-m4ztv\" (UID: \"e2ac808b-b832-4228-a2e3-ac84d0bfc436\") " pod="openshift-ingress/payload-processing-e2e-worker-w4-93827d-d6d7cb69c-m4ztv"
Sep 18 14:11:23.963140 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:11:23.963057    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-worker-w4-93827d-d6d7cb69c-m4ztv"
Sep 18 14:11:24.030650 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:11:24.030607    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-w4-93827d-7ff5557fbb-lbszk"
Sep 18 14:11:24.254975 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:11:24.254948    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w4-93827d-d6d7cb69c-m4ztv"]
Sep 18 14:11:24.256637 ip-10-0-129-16 kubenswrapper[2565]: W0918 14:11:24.256614    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode2ac808b_b832_4228_a2e3_ac84d0bfc436.slice/crio-07fd8a6b46b66d31facdd440644d6b276d44cf1581d8e1bee40f11c9d9016f41 WatchSource:0}: Error finding container 07fd8a6b46b66d31facdd440644d6b276d44cf1581d8e1bee40f11c9d9016f41: Status 404 returned error can't find the container with id 07fd8a6b46b66d31facdd440644d6b276d44cf1581d8e1bee40f11c9d9016f41
Sep 18 14:11:24.319131 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:11:24.319106    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w4-93827d-7ff5557fbb-lbszk"]
Sep 18 14:11:24.320709 ip-10-0-129-16 kubenswrapper[2565]: W0918 14:11:24.320684    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd3515ddb_013d_488f_a714_efbb75884469.slice/crio-381ae5e69f99e9e3e248e98730f5926e344ec3d307790952233eff6d7ef440a0 WatchSource:0}: Error finding container 381ae5e69f99e9e3e248e98730f5926e344ec3d307790952233eff6d7ef440a0: Status 404 returned error can't find the container with id 381ae5e69f99e9e3e248e98730f5926e344ec3d307790952233eff6d7ef440a0
Sep 18 14:11:24.671025 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:11:24.670986    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-w4-93827d-7ff5557fbb-lbszk" event={"ID":"d3515ddb-013d-488f-a714-efbb75884469","Type":"ContainerStarted","Data":"f17a2ce76a9fcf82b9835bb5b8d0a3b751d8527a3fb9f88b4d2985de6988caba"}
Sep 18 14:11:24.671512 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:11:24.671191    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-worker-w4-93827d-7ff5557fbb-lbszk"
Sep 18 14:11:24.671512 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:11:24.671278    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-w4-93827d-7ff5557fbb-lbszk" event={"ID":"d3515ddb-013d-488f-a714-efbb75884469","Type":"ContainerStarted","Data":"381ae5e69f99e9e3e248e98730f5926e344ec3d307790952233eff6d7ef440a0"}
Sep 18 14:11:24.672868 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:11:24.672841    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-worker-w4-93827d-d6d7cb69c-m4ztv" event={"ID":"e2ac808b-b832-4228-a2e3-ac84d0bfc436","Type":"ContainerStarted","Data":"70f6cbc1ccb3e079a917b877f73fc483b59524cf03bcb6adcbfe5fc44793952b"}
Sep 18 14:11:24.672994 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:11:24.672884    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-worker-w4-93827d-d6d7cb69c-m4ztv" event={"ID":"e2ac808b-b832-4228-a2e3-ac84d0bfc436","Type":"ContainerStarted","Data":"07fd8a6b46b66d31facdd440644d6b276d44cf1581d8e1bee40f11c9d9016f41"}
Sep 18 14:11:24.672994 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:11:24.672922    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-worker-w4-93827d-d6d7cb69c-m4ztv"
Sep 18 14:11:24.693647 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:11:24.693591    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-worker-w4-93827d-7ff5557fbb-lbszk" podStartSLOduration=1.693576308 podStartE2EDuration="1.693576308s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 14:11:23 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 14:11:24.690948172 +0000 UTC m=+752.864857832" watchObservedRunningTime="2026-09-18 14:11:24.693576308 +0000 UTC m=+752.867485967"
Sep 18 14:11:24.717514 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:11:24.717458    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-worker-w4-93827d-d6d7cb69c-m4ztv" podStartSLOduration=1.7174421579999999 podStartE2EDuration="1.717442158s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 14:11:23 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 14:11:24.713742253 +0000 UTC m=+752.887651931" watchObservedRunningTime="2026-09-18 14:11:24.717442158 +0000 UTC m=+752.891351818"
Sep 18 14:11:26.206324 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:11:26.206285    2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-a-dc2cee-79685b4ffd-lqfbh"]
Sep 18 14:11:26.211136 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:11:26.211106    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-dc2cee-79685b4ffd-lqfbh"
Sep 18 14:11:26.214110 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:11:26.214079    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-inf-a-dc2cee\""
Sep 18 14:11:26.214656 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:11:26.214630    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-inf-a-dc2cee-dockercfg-qc9dw\""
Sep 18 14:11:26.232445 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:11:26.232417    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-a-dc2cee-79685b4ffd-lqfbh"]
Sep 18 14:11:26.274589 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:11:26.274550    2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-a-dc2cee-647f88fd5c-6qkz8"]
Sep 18 14:11:26.278347 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:11:26.278330    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-a-dc2cee-647f88fd5c-6qkz8"
Sep 18 14:11:26.295085 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:11:26.295057    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-a-dc2cee-647f88fd5c-6qkz8"]
Sep 18 14:11:26.295543 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:11:26.295519    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/10d1626f-264e-4036-be5d-4a6a6dbaaa69-plugins-config-volume\") pod \"payload-pre-processing-e2e-inf-a-dc2cee-79685b4ffd-lqfbh\" (UID: \"10d1626f-264e-4036-be5d-4a6a6dbaaa69\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-a-dc2cee-79685b4ffd-lqfbh"
Sep 18 14:11:26.295605 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:11:26.295548    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vmfhz\" (UniqueName: \"kubernetes.io/projected/10d1626f-264e-4036-be5d-4a6a6dbaaa69-kube-api-access-vmfhz\") pod \"payload-pre-processing-e2e-inf-a-dc2cee-79685b4ffd-lqfbh\" (UID: \"10d1626f-264e-4036-be5d-4a6a6dbaaa69\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-a-dc2cee-79685b4ffd-lqfbh"
Sep 18 14:11:26.396263 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:11:26.396231    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6tmqs\" (UniqueName: \"kubernetes.io/projected/490e2772-09af-4a35-834b-fccc3f178137-kube-api-access-6tmqs\") pod \"payload-processing-e2e-inf-a-dc2cee-647f88fd5c-6qkz8\" (UID: \"490e2772-09af-4a35-834b-fccc3f178137\") " pod="openshift-ingress/payload-processing-e2e-inf-a-dc2cee-647f88fd5c-6qkz8"
Sep 18 14:11:26.396263 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:11:26.396264    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/490e2772-09af-4a35-834b-fccc3f178137-plugins-config-volume\") pod \"payload-processing-e2e-inf-a-dc2cee-647f88fd5c-6qkz8\" (UID: \"490e2772-09af-4a35-834b-fccc3f178137\") " pod="openshift-ingress/payload-processing-e2e-inf-a-dc2cee-647f88fd5c-6qkz8"
Sep 18 14:11:26.396507 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:11:26.396310    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/10d1626f-264e-4036-be5d-4a6a6dbaaa69-plugins-config-volume\") pod \"payload-pre-processing-e2e-inf-a-dc2cee-79685b4ffd-lqfbh\" (UID: \"10d1626f-264e-4036-be5d-4a6a6dbaaa69\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-a-dc2cee-79685b4ffd-lqfbh"
Sep 18 14:11:26.396507 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:11:26.396329    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vmfhz\" (UniqueName: \"kubernetes.io/projected/10d1626f-264e-4036-be5d-4a6a6dbaaa69-kube-api-access-vmfhz\") pod \"payload-pre-processing-e2e-inf-a-dc2cee-79685b4ffd-lqfbh\" (UID: \"10d1626f-264e-4036-be5d-4a6a6dbaaa69\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-a-dc2cee-79685b4ffd-lqfbh"
Sep 18 14:11:26.396990 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:11:26.396969    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/10d1626f-264e-4036-be5d-4a6a6dbaaa69-plugins-config-volume\") pod \"payload-pre-processing-e2e-inf-a-dc2cee-79685b4ffd-lqfbh\" (UID: \"10d1626f-264e-4036-be5d-4a6a6dbaaa69\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-a-dc2cee-79685b4ffd-lqfbh"
Sep 18 14:11:26.415025 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:11:26.414997    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vmfhz\" (UniqueName: \"kubernetes.io/projected/10d1626f-264e-4036-be5d-4a6a6dbaaa69-kube-api-access-vmfhz\") pod \"payload-pre-processing-e2e-inf-a-dc2cee-79685b4ffd-lqfbh\" (UID: \"10d1626f-264e-4036-be5d-4a6a6dbaaa69\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-a-dc2cee-79685b4ffd-lqfbh"
Sep 18 14:11:26.497008 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:11:26.496913    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6tmqs\" (UniqueName: \"kubernetes.io/projected/490e2772-09af-4a35-834b-fccc3f178137-kube-api-access-6tmqs\") pod \"payload-processing-e2e-inf-a-dc2cee-647f88fd5c-6qkz8\" (UID: \"490e2772-09af-4a35-834b-fccc3f178137\") " pod="openshift-ingress/payload-processing-e2e-inf-a-dc2cee-647f88fd5c-6qkz8"
Sep 18 14:11:26.497008 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:11:26.496956    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/490e2772-09af-4a35-834b-fccc3f178137-plugins-config-volume\") pod \"payload-processing-e2e-inf-a-dc2cee-647f88fd5c-6qkz8\" (UID: \"490e2772-09af-4a35-834b-fccc3f178137\") " pod="openshift-ingress/payload-processing-e2e-inf-a-dc2cee-647f88fd5c-6qkz8"
Sep 18 14:11:26.497678 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:11:26.497659    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/490e2772-09af-4a35-834b-fccc3f178137-plugins-config-volume\") pod \"payload-processing-e2e-inf-a-dc2cee-647f88fd5c-6qkz8\" (UID: \"490e2772-09af-4a35-834b-fccc3f178137\") " pod="openshift-ingress/payload-processing-e2e-inf-a-dc2cee-647f88fd5c-6qkz8"
Sep 18 14:11:26.521984 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:11:26.521956    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6tmqs\" (UniqueName: \"kubernetes.io/projected/490e2772-09af-4a35-834b-fccc3f178137-kube-api-access-6tmqs\") pod \"payload-processing-e2e-inf-a-dc2cee-647f88fd5c-6qkz8\" (UID: \"490e2772-09af-4a35-834b-fccc3f178137\") " pod="openshift-ingress/payload-processing-e2e-inf-a-dc2cee-647f88fd5c-6qkz8"
Sep 18 14:11:26.525749 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:11:26.525727    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-dc2cee-79685b4ffd-lqfbh"
Sep 18 14:11:26.591329 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:11:26.591304    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-a-dc2cee-647f88fd5c-6qkz8"
Sep 18 14:11:26.837092 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:11:26.837064    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-a-dc2cee-79685b4ffd-lqfbh"]
Sep 18 14:11:26.838018 ip-10-0-129-16 kubenswrapper[2565]: W0918 14:11:26.837985    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod10d1626f_264e_4036_be5d_4a6a6dbaaa69.slice/crio-e4f942e4e7dc0259f0f34e044256363bf7b1a463277006806a63d2a992b103f7 WatchSource:0}: Error finding container e4f942e4e7dc0259f0f34e044256363bf7b1a463277006806a63d2a992b103f7: Status 404 returned error can't find the container with id e4f942e4e7dc0259f0f34e044256363bf7b1a463277006806a63d2a992b103f7
Sep 18 14:11:26.916277 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:11:26.916247    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-a-dc2cee-647f88fd5c-6qkz8"]
Sep 18 14:11:26.917528 ip-10-0-129-16 kubenswrapper[2565]: W0918 14:11:26.917500    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod490e2772_09af_4a35_834b_fccc3f178137.slice/crio-8640c697227cf46a7f3cc7ba38fbb7ee70ed618239b1b0567f9a424985f91a8a WatchSource:0}: Error finding container 8640c697227cf46a7f3cc7ba38fbb7ee70ed618239b1b0567f9a424985f91a8a: Status 404 returned error can't find the container with id 8640c697227cf46a7f3cc7ba38fbb7ee70ed618239b1b0567f9a424985f91a8a
Sep 18 14:11:27.696198 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:11:27.696163    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-a-dc2cee-647f88fd5c-6qkz8" event={"ID":"490e2772-09af-4a35-834b-fccc3f178137","Type":"ContainerStarted","Data":"11eb7de5b5d6c335d94f98b18bf28663edeed6a55854b1a5c048b7db3388194d"}
Sep 18 14:11:27.696703 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:11:27.696223    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-a-dc2cee-647f88fd5c-6qkz8" event={"ID":"490e2772-09af-4a35-834b-fccc3f178137","Type":"ContainerStarted","Data":"8640c697227cf46a7f3cc7ba38fbb7ee70ed618239b1b0567f9a424985f91a8a"}
Sep 18 14:11:27.696703 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:11:27.696287    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-inf-a-dc2cee-647f88fd5c-6qkz8"
Sep 18 14:11:27.698107 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:11:27.698001    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-dc2cee-79685b4ffd-lqfbh" event={"ID":"10d1626f-264e-4036-be5d-4a6a6dbaaa69","Type":"ContainerStarted","Data":"d0214ca02a961afb125a10e2ab29ceabdc8cdaddefe6dcac1cdd2f199b4beb68"}
Sep 18 14:11:27.698107 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:11:27.698082    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-dc2cee-79685b4ffd-lqfbh" event={"ID":"10d1626f-264e-4036-be5d-4a6a6dbaaa69","Type":"ContainerStarted","Data":"e4f942e4e7dc0259f0f34e044256363bf7b1a463277006806a63d2a992b103f7"}
Sep 18 14:11:27.698602 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:11:27.698432    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-dc2cee-79685b4ffd-lqfbh"
Sep 18 14:11:27.760130 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:11:27.760069    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-inf-a-dc2cee-647f88fd5c-6qkz8" podStartSLOduration=1.760047508 podStartE2EDuration="1.760047508s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 14:11:26 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 14:11:27.756573986 +0000 UTC m=+755.930483684" watchObservedRunningTime="2026-09-18 14:11:27.760047508 +0000 UTC m=+755.933957171"
Sep 18 14:11:27.831877 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:11:27.831807    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-dc2cee-79685b4ffd-lqfbh" podStartSLOduration=1.831785421 podStartE2EDuration="1.831785421s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 14:11:26 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 14:11:27.831220013 +0000 UTC m=+756.005129692" watchObservedRunningTime="2026-09-18 14:11:27.831785421 +0000 UTC m=+756.005695083"
Sep 18 14:11:33.084778 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:11:33.084734    2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-eeb0a3f4-595b879cbc-7zh7w"]
Sep 18 14:11:33.099380 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:11:33.099345    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-eeb0a3f4-595b879cbc-7zh7w"
Sep 18 14:11:33.104529 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:11:33.104084    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-eeb0a3f4-595b879cbc-7zh7w"]
Sep 18 14:11:33.104529 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:11:33.104398    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-ait-eeb0a3f4-dockercfg-qqwqr\""
Sep 18 14:11:33.104762 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:11:33.104622    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-ait-eeb0a3f4\""
Sep 18 14:11:33.213571 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:11:33.213538    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f61bfe0e-7506-446a-a810-23403f108a1a-plugins-config-volume\") pod \"payload-processing-e2e-ait-eeb0a3f4-595b879cbc-7zh7w\" (UID: \"f61bfe0e-7506-446a-a810-23403f108a1a\") " pod="openshift-ingress/payload-processing-e2e-ait-eeb0a3f4-595b879cbc-7zh7w"
Sep 18 14:11:33.213777 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:11:33.213760    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xb987\" (UniqueName: \"kubernetes.io/projected/f61bfe0e-7506-446a-a810-23403f108a1a-kube-api-access-xb987\") pod \"payload-processing-e2e-ait-eeb0a3f4-595b879cbc-7zh7w\" (UID: \"f61bfe0e-7506-446a-a810-23403f108a1a\") " pod="openshift-ingress/payload-processing-e2e-ait-eeb0a3f4-595b879cbc-7zh7w"
Sep 18 14:11:33.314628 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:11:33.314589    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xb987\" (UniqueName: \"kubernetes.io/projected/f61bfe0e-7506-446a-a810-23403f108a1a-kube-api-access-xb987\") pod \"payload-processing-e2e-ait-eeb0a3f4-595b879cbc-7zh7w\" (UID: \"f61bfe0e-7506-446a-a810-23403f108a1a\") " pod="openshift-ingress/payload-processing-e2e-ait-eeb0a3f4-595b879cbc-7zh7w"
Sep 18 14:11:33.314783 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:11:33.314727    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f61bfe0e-7506-446a-a810-23403f108a1a-plugins-config-volume\") pod \"payload-processing-e2e-ait-eeb0a3f4-595b879cbc-7zh7w\" (UID: \"f61bfe0e-7506-446a-a810-23403f108a1a\") " pod="openshift-ingress/payload-processing-e2e-ait-eeb0a3f4-595b879cbc-7zh7w"
Sep 18 14:11:33.315484 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:11:33.315464    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f61bfe0e-7506-446a-a810-23403f108a1a-plugins-config-volume\") pod \"payload-processing-e2e-ait-eeb0a3f4-595b879cbc-7zh7w\" (UID: \"f61bfe0e-7506-446a-a810-23403f108a1a\") " pod="openshift-ingress/payload-processing-e2e-ait-eeb0a3f4-595b879cbc-7zh7w"
Sep 18 14:11:33.326124 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:11:33.326094    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xb987\" (UniqueName: \"kubernetes.io/projected/f61bfe0e-7506-446a-a810-23403f108a1a-kube-api-access-xb987\") pod \"payload-processing-e2e-ait-eeb0a3f4-595b879cbc-7zh7w\" (UID: \"f61bfe0e-7506-446a-a810-23403f108a1a\") " pod="openshift-ingress/payload-processing-e2e-ait-eeb0a3f4-595b879cbc-7zh7w"
Sep 18 14:11:33.414399 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:11:33.414319    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-eeb0a3f4-595b879cbc-7zh7w"
Sep 18 14:11:33.745108 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:11:33.743522    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-eeb0a3f4-595b879cbc-7zh7w"]
Sep 18 14:11:34.675500 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:11:34.675460    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-4d9222-67d45969f8-gwf8b"
Sep 18 14:11:34.788772 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:11:34.788727    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-eeb0a3f4-595b879cbc-7zh7w" event={"ID":"f61bfe0e-7506-446a-a810-23403f108a1a","Type":"ContainerStarted","Data":"438facf2ffc16fce103dd679f1f53ebd78600e0a3ed04744f29ef94da7bd750b"}
Sep 18 14:11:34.788976 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:11:34.788830    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-eeb0a3f4-595b879cbc-7zh7w" event={"ID":"f61bfe0e-7506-446a-a810-23403f108a1a","Type":"ContainerStarted","Data":"4784f8d9e59efb42d6f8555e2da1c8f37f043af17d31a12c79e631ac9a80b263"}
Sep 18 14:11:34.788976 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:11:34.788948    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-ait-eeb0a3f4-595b879cbc-7zh7w"
Sep 18 14:11:34.808145 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:11:34.808093    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-ait-eeb0a3f4-595b879cbc-7zh7w" podStartSLOduration=1.808025166 podStartE2EDuration="1.808025166s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 14:11:33 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 14:11:34.807318968 +0000 UTC m=+762.981228651" watchObservedRunningTime="2026-09-18 14:11:34.808025166 +0000 UTC m=+762.981934826"
Sep 18 14:11:35.681753 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:11:35.681719    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-worker-w4-93827d-d6d7cb69c-m4ztv"
Sep 18 14:11:35.682265 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:11:35.681904    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-worker-w4-93827d-7ff5557fbb-lbszk"
Sep 18 14:11:38.706046 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:11:38.706009    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-inf-a-dc2cee-647f88fd5c-6qkz8"
Sep 18 14:11:38.706474 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:11:38.706095    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-dc2cee-79685b4ffd-lqfbh"
Sep 18 14:11:38.817014 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:11:38.816976    2565 generic.go:356] "Generic (PLEG): container finished" podID="97394e77-67c5-435f-9457-f9af6436ad5c" containerID="cf019763542150ecaef6b5767da1064e3fe353cd4c2e357badc0cb9b908aaad0" exitCode=0
Sep 18 14:11:38.817229 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:11:38.817062    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-93827d-openshift-default-6c5bbfcf98-7nrxb" event={"ID":"97394e77-67c5-435f-9457-f9af6436ad5c","Type":"ContainerDied","Data":"cf019763542150ecaef6b5767da1064e3fe353cd4c2e357badc0cb9b908aaad0"}
Sep 18 14:11:38.817542 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:11:38.817523    2565 scope.go:117] "RemoveContainer" containerID="cf019763542150ecaef6b5767da1064e3fe353cd4c2e357badc0cb9b908aaad0"
Sep 18 14:11:38.818759 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:11:38.818732    2565 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 18 14:11:38.818868 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:11:38.818814    2565 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 18 14:11:38.818933 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:11:38.818864    2565 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 18 14:11:38.819210 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:11:38.819189    2565 generic.go:356] "Generic (PLEG): container finished" podID="f8ff7559-3e54-44b4-a9ce-50f69e2b1908" containerID="3470f50403473d7e198a40e9941623dc0840c6d203553afd39d44680ac8ed2cd" exitCode=0
Sep 18 14:11:38.819268 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:11:38.819238    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-4d9222-openshift-default-849bc99f86-v5mg4" event={"ID":"f8ff7559-3e54-44b4-a9ce-50f69e2b1908","Type":"ContainerDied","Data":"3470f50403473d7e198a40e9941623dc0840c6d203553afd39d44680ac8ed2cd"}
Sep 18 14:11:38.819556 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:11:38.819544    2565 scope.go:117] "RemoveContainer" containerID="3470f50403473d7e198a40e9941623dc0840c6d203553afd39d44680ac8ed2cd"
Sep 18 14:11:38.820599 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:11:38.820566    2565 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 18 14:11:38.820699 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:11:38.820637    2565 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 18 14:11:38.820699 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:11:38.820667    2565 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 18 14:11:39.826928 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:11:39.826890    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-4d9222-openshift-default-849bc99f86-v5mg4" event={"ID":"f8ff7559-3e54-44b4-a9ce-50f69e2b1908","Type":"ContainerStarted","Data":"85338d552b54e5d29fdab34775193237cbfae43d10bfd7511cbab811953a8a7f"}
Sep 18 14:11:39.829063 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:11:39.829018    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-93827d-openshift-default-6c5bbfcf98-7nrxb" event={"ID":"97394e77-67c5-435f-9457-f9af6436ad5c","Type":"ContainerStarted","Data":"3afd768339cfe21115a68520d406c6e364c9d56db530769f00873cd1b1eb0561"}
Sep 18 14:11:40.666830 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:11:40.666785    2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w0-4d9222-openshift-default-849bc99f86-v5mg4"
Sep 18 14:11:40.667656 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:11:40.667629    2565 patch_prober.go:28] interesting pod/e2e-worker-w0-4d9222-openshift-default-849bc99f86-v5mg4 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.41:15021/healthz/ready\": dial tcp 10.134.0.41:15021: connect: connection refused" start-of-body=
Sep 18 14:11:40.667878 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:11:40.667837    2565 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-4d9222-openshift-default-849bc99f86-v5mg4" podUID="f8ff7559-3e54-44b4-a9ce-50f69e2b1908" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.41:15021/healthz/ready\": dial tcp 10.134.0.41:15021: connect: connection refused"
Sep 18 14:11:40.747229 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:11:40.747186    2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w4-93827d-openshift-default-6c5bbfcf98-7nrxb"
Sep 18 14:11:40.747443 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:11:40.747328    2565 patch_prober.go:28] interesting pod/e2e-worker-w4-93827d-openshift-default-6c5bbfcf98-7nrxb container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.42:15021/healthz/ready\": dial tcp 10.134.0.42:15021: connect: connection refused" start-of-body=
Sep 18 14:11:40.747443 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:11:40.747373    2565 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-93827d-openshift-default-6c5bbfcf98-7nrxb" podUID="97394e77-67c5-435f-9457-f9af6436ad5c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.42:15021/healthz/ready\": dial tcp 10.134.0.42:15021: connect: connection refused"
Sep 18 14:11:41.666997 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:11:41.666949    2565 patch_prober.go:28] interesting pod/e2e-worker-w0-4d9222-openshift-default-849bc99f86-v5mg4 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.41:15021/healthz/ready\": dial tcp 10.134.0.41:15021: connect: connection refused" start-of-body=
Sep 18 14:11:41.667545 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:11:41.667024    2565 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-4d9222-openshift-default-849bc99f86-v5mg4" podUID="f8ff7559-3e54-44b4-a9ce-50f69e2b1908" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.41:15021/healthz/ready\": dial tcp 10.134.0.41:15021: connect: connection refused"
Sep 18 14:11:41.746963 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:11:41.746925    2565 patch_prober.go:28] interesting pod/e2e-worker-w4-93827d-openshift-default-6c5bbfcf98-7nrxb container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.42:15021/healthz/ready\": dial tcp 10.134.0.42:15021: connect: connection refused" start-of-body=
Sep 18 14:11:41.747661 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:11:41.747262    2565 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-93827d-openshift-default-6c5bbfcf98-7nrxb" podUID="97394e77-67c5-435f-9457-f9af6436ad5c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.42:15021/healthz/ready\": dial tcp 10.134.0.42:15021: connect: connection refused"
Sep 18 14:11:42.667579 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:11:42.667531    2565 patch_prober.go:28] interesting pod/e2e-worker-w0-4d9222-openshift-default-849bc99f86-v5mg4 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.41:15021/healthz/ready\": dial tcp 10.134.0.41:15021: connect: connection refused" start-of-body=
Sep 18 14:11:42.668079 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:11:42.667618    2565 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-4d9222-openshift-default-849bc99f86-v5mg4" podUID="f8ff7559-3e54-44b4-a9ce-50f69e2b1908" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.41:15021/healthz/ready\": dial tcp 10.134.0.41:15021: connect: connection refused"
Sep 18 14:11:42.747509 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:11:42.747461    2565 patch_prober.go:28] interesting pod/e2e-worker-w4-93827d-openshift-default-6c5bbfcf98-7nrxb container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.42:15021/healthz/ready\": dial tcp 10.134.0.42:15021: connect: connection refused" start-of-body=
Sep 18 14:11:42.747741 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:11:42.747544    2565 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-93827d-openshift-default-6c5bbfcf98-7nrxb" podUID="97394e77-67c5-435f-9457-f9af6436ad5c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.42:15021/healthz/ready\": dial tcp 10.134.0.42:15021: connect: connection refused"
Sep 18 14:11:43.667256 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:11:43.667213    2565 patch_prober.go:28] interesting pod/e2e-worker-w0-4d9222-openshift-default-849bc99f86-v5mg4 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.41:15021/healthz/ready\": dial tcp 10.134.0.41:15021: connect: connection refused" start-of-body=
Sep 18 14:11:43.667478 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:11:43.667283    2565 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-4d9222-openshift-default-849bc99f86-v5mg4" podUID="f8ff7559-3e54-44b4-a9ce-50f69e2b1908" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.41:15021/healthz/ready\": dial tcp 10.134.0.41:15021: connect: connection refused"
Sep 18 14:11:43.747198 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:11:43.747161    2565 patch_prober.go:28] interesting pod/e2e-worker-w4-93827d-openshift-default-6c5bbfcf98-7nrxb container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.42:15021/healthz/ready\": dial tcp 10.134.0.42:15021: connect: connection refused" start-of-body=
Sep 18 14:11:43.747776 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:11:43.747747    2565 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-93827d-openshift-default-6c5bbfcf98-7nrxb" podUID="97394e77-67c5-435f-9457-f9af6436ad5c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.42:15021/healthz/ready\": dial tcp 10.134.0.42:15021: connect: connection refused"
Sep 18 14:11:44.667822 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:11:44.667774    2565 patch_prober.go:28] interesting pod/e2e-worker-w0-4d9222-openshift-default-849bc99f86-v5mg4 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.41:15021/healthz/ready\": dial tcp 10.134.0.41:15021: connect: connection refused" start-of-body=
Sep 18 14:11:44.668024 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:11:44.667857    2565 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-4d9222-openshift-default-849bc99f86-v5mg4" podUID="f8ff7559-3e54-44b4-a9ce-50f69e2b1908" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.41:15021/healthz/ready\": dial tcp 10.134.0.41:15021: connect: connection refused"
Sep 18 14:11:44.750053 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:11:44.747509    2565 patch_prober.go:28] interesting pod/e2e-worker-w4-93827d-openshift-default-6c5bbfcf98-7nrxb container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.42:15021/healthz/ready\": dial tcp 10.134.0.42:15021: connect: connection refused" start-of-body=
Sep 18 14:11:44.750053 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:11:44.747583    2565 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-93827d-openshift-default-6c5bbfcf98-7nrxb" podUID="97394e77-67c5-435f-9457-f9af6436ad5c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.42:15021/healthz/ready\": dial tcp 10.134.0.42:15021: connect: connection refused"
Sep 18 14:11:45.746851 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:11:45.746806    2565 patch_prober.go:28] interesting pod/e2e-worker-w4-93827d-openshift-default-6c5bbfcf98-7nrxb container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.42:15021/healthz/ready\": dial tcp 10.134.0.42:15021: connect: connection refused" start-of-body=
Sep 18 14:11:45.747072 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:11:45.746880    2565 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-93827d-openshift-default-6c5bbfcf98-7nrxb" podUID="97394e77-67c5-435f-9457-f9af6436ad5c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.42:15021/healthz/ready\": dial tcp 10.134.0.42:15021: connect: connection refused"
Sep 18 14:11:45.768434 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:11:45.768343    2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w0-4d9222-openshift-default-849bc99f86-v5mg4"
Sep 18 14:11:45.769064 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:11:45.769014    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w0-4d9222-openshift-default-849bc99f86-v5mg4"
Sep 18 14:11:45.770168 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:11:45.770133    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w0-4d9222-openshift-default-849bc99f86-v5mg4"
Sep 18 14:11:45.799092 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:11:45.799062    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-ait-eeb0a3f4-595b879cbc-7zh7w"
Sep 18 14:11:46.752084 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:11:46.752020    2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w4-93827d-openshift-default-6c5bbfcf98-7nrxb"
Sep 18 14:11:46.752417 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:11:46.752396    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w4-93827d-openshift-default-6c5bbfcf98-7nrxb"
Sep 18 14:11:46.753201 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:11:46.753183    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w4-93827d-openshift-default-6c5bbfcf98-7nrxb"
Sep 18 14:11:50.209167 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:11:50.209127    2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-eeb0a3f4-frw22"]
Sep 18 14:11:50.214685 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:11:50.214643    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-eeb0a3f4-frw22"
Sep 18 14:11:50.219565 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:11:50.219541    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-l4fn9\""
Sep 18 14:11:50.245445 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:11:50.245407    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-eeb0a3f4-frw22"]
Sep 18 14:11:50.335517 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:11:50.335471    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/e0949a12-e8b4-45de-8b8c-840e57646d04-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-eeb0a3f4-frw22\" (UID: \"e0949a12-e8b4-45de-8b8c-840e57646d04\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-eeb0a3f4-frw22"
Sep 18 14:11:50.436249 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:11:50.436215    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/e0949a12-e8b4-45de-8b8c-840e57646d04-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-eeb0a3f4-frw22\" (UID: \"e0949a12-e8b4-45de-8b8c-840e57646d04\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-eeb0a3f4-frw22"
Sep 18 14:11:50.436849 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:11:50.436828    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/e0949a12-e8b4-45de-8b8c-840e57646d04-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-eeb0a3f4-frw22\" (UID: \"e0949a12-e8b4-45de-8b8c-840e57646d04\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-eeb0a3f4-frw22"
Sep 18 14:11:50.525491 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:11:50.525412    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-eeb0a3f4-frw22"
Sep 18 14:11:50.811556 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:11:50.811516    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-eeb0a3f4-frw22"]
Sep 18 14:11:50.813288 ip-10-0-129-16 kubenswrapper[2565]: W0918 14:11:50.813252    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode0949a12_e8b4_45de_8b8c_840e57646d04.slice/crio-2e5e3da1982ddca801e0650c52ff0df604ddd55267d881088cd8fe350fd56142 WatchSource:0}: Error finding container 2e5e3da1982ddca801e0650c52ff0df604ddd55267d881088cd8fe350fd56142: Status 404 returned error can't find the container with id 2e5e3da1982ddca801e0650c52ff0df604ddd55267d881088cd8fe350fd56142
Sep 18 14:11:50.913917 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:11:50.913879    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-eeb0a3f4-frw22" event={"ID":"e0949a12-e8b4-45de-8b8c-840e57646d04","Type":"ContainerStarted","Data":"2e5e3da1982ddca801e0650c52ff0df604ddd55267d881088cd8fe350fd56142"}
Sep 18 14:11:53.933184 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:11:53.933077    2565 generic.go:356] "Generic (PLEG): container finished" podID="e0949a12-e8b4-45de-8b8c-840e57646d04" containerID="c0daf7e53f83b114c1765f7dfa2169ce33b44cf7a5f5e9542a43e3a13496272c" exitCode=0
Sep 18 14:11:53.933184 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:11:53.933118    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-eeb0a3f4-frw22" event={"ID":"e0949a12-e8b4-45de-8b8c-840e57646d04","Type":"ContainerDied","Data":"c0daf7e53f83b114c1765f7dfa2169ce33b44cf7a5f5e9542a43e3a13496272c"}
Sep 18 14:11:55.084205 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:11:55.084181    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-eeb0a3f4-frw22"
Sep 18 14:11:55.184640 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:11:55.184608    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/e0949a12-e8b4-45de-8b8c-840e57646d04-maas-api-service-ca\") pod \"e0949a12-e8b4-45de-8b8c-840e57646d04\" (UID: \"e0949a12-e8b4-45de-8b8c-840e57646d04\") "
Sep 18 14:11:55.184989 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:11:55.184958    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e0949a12-e8b4-45de-8b8c-840e57646d04-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "e0949a12-e8b4-45de-8b8c-840e57646d04" (UID: "e0949a12-e8b4-45de-8b8c-840e57646d04"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 18 14:11:55.286765 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:11:55.286681    2565 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/e0949a12-e8b4-45de-8b8c-840e57646d04-maas-api-service-ca\") on node \"ip-10-0-129-16.ec2.internal\" DevicePath \"\""
Sep 18 14:11:55.946763 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:11:55.946736    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-eeb0a3f4-frw22"
Sep 18 14:11:55.946924 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:11:55.946729    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-eeb0a3f4-frw22" event={"ID":"e0949a12-e8b4-45de-8b8c-840e57646d04","Type":"ContainerDied","Data":"2e5e3da1982ddca801e0650c52ff0df604ddd55267d881088cd8fe350fd56142"}
Sep 18 14:11:55.946924 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:11:55.946856    2565 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="2e5e3da1982ddca801e0650c52ff0df604ddd55267d881088cd8fe350fd56142"
Sep 18 14:12:06.071829 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:12:06.071739    2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-eeb0a3f4-595b879cbc-7zh7w"]
Sep 18 14:12:06.072318 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:12:06.072021    2565 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-ait-eeb0a3f4-595b879cbc-7zh7w" podUID="f61bfe0e-7506-446a-a810-23403f108a1a" containerName="payload-processing" containerID="cri-o://438facf2ffc16fce103dd679f1f53ebd78600e0a3ed04744f29ef94da7bd750b" gracePeriod=30
Sep 18 14:12:06.311602 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:12:06.311568    2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-eeb0a3f4-frw22"]
Sep 18 14:12:06.318286 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:12:06.318260    2565 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-eeb0a3f4-frw22"]
Sep 18 14:12:06.352102 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:12:06.352077    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-eeb0a3f4-595b879cbc-7zh7w"
Sep 18 14:12:06.407742 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:12:06.407713    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-xb987\" (UniqueName: \"kubernetes.io/projected/f61bfe0e-7506-446a-a810-23403f108a1a-kube-api-access-xb987\") pod \"f61bfe0e-7506-446a-a810-23403f108a1a\" (UID: \"f61bfe0e-7506-446a-a810-23403f108a1a\") "
Sep 18 14:12:06.407924 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:12:06.407824    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f61bfe0e-7506-446a-a810-23403f108a1a-plugins-config-volume\") pod \"f61bfe0e-7506-446a-a810-23403f108a1a\" (UID: \"f61bfe0e-7506-446a-a810-23403f108a1a\") "
Sep 18 14:12:06.408259 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:12:06.408234    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f61bfe0e-7506-446a-a810-23403f108a1a-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "f61bfe0e-7506-446a-a810-23403f108a1a" (UID: "f61bfe0e-7506-446a-a810-23403f108a1a"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 18 14:12:06.410107 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:12:06.410075    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f61bfe0e-7506-446a-a810-23403f108a1a-kube-api-access-xb987" (OuterVolumeSpecName: "kube-api-access-xb987") pod "f61bfe0e-7506-446a-a810-23403f108a1a" (UID: "f61bfe0e-7506-446a-a810-23403f108a1a"). InnerVolumeSpecName "kube-api-access-xb987". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 18 14:12:06.509564 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:12:06.509533    2565 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f61bfe0e-7506-446a-a810-23403f108a1a-plugins-config-volume\") on node \"ip-10-0-129-16.ec2.internal\" DevicePath \"\""
Sep 18 14:12:06.509564 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:12:06.509563    2565 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xb987\" (UniqueName: \"kubernetes.io/projected/f61bfe0e-7506-446a-a810-23403f108a1a-kube-api-access-xb987\") on node \"ip-10-0-129-16.ec2.internal\" DevicePath \"\""
Sep 18 14:12:07.014051 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:12:07.013985    2565 generic.go:356] "Generic (PLEG): container finished" podID="f61bfe0e-7506-446a-a810-23403f108a1a" containerID="438facf2ffc16fce103dd679f1f53ebd78600e0a3ed04744f29ef94da7bd750b" exitCode=0
Sep 18 14:12:07.014215 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:12:07.014063    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-eeb0a3f4-595b879cbc-7zh7w"
Sep 18 14:12:07.014215 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:12:07.014092    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-eeb0a3f4-595b879cbc-7zh7w" event={"ID":"f61bfe0e-7506-446a-a810-23403f108a1a","Type":"ContainerDied","Data":"438facf2ffc16fce103dd679f1f53ebd78600e0a3ed04744f29ef94da7bd750b"}
Sep 18 14:12:07.014215 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:12:07.014151    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-eeb0a3f4-595b879cbc-7zh7w" event={"ID":"f61bfe0e-7506-446a-a810-23403f108a1a","Type":"ContainerDied","Data":"4784f8d9e59efb42d6f8555e2da1c8f37f043af17d31a12c79e631ac9a80b263"}
Sep 18 14:12:07.014215 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:12:07.014176    2565 scope.go:117] "RemoveContainer" containerID="438facf2ffc16fce103dd679f1f53ebd78600e0a3ed04744f29ef94da7bd750b"
Sep 18 14:12:07.023796 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:12:07.023778    2565 scope.go:117] "RemoveContainer" containerID="438facf2ffc16fce103dd679f1f53ebd78600e0a3ed04744f29ef94da7bd750b"
Sep 18 14:12:07.024202 ip-10-0-129-16 kubenswrapper[2565]: E0918 14:12:07.024173    2565 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"438facf2ffc16fce103dd679f1f53ebd78600e0a3ed04744f29ef94da7bd750b\": container with ID starting with 438facf2ffc16fce103dd679f1f53ebd78600e0a3ed04744f29ef94da7bd750b not found: ID does not exist" containerID="438facf2ffc16fce103dd679f1f53ebd78600e0a3ed04744f29ef94da7bd750b"
Sep 18 14:12:07.024321 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:12:07.024213    2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"438facf2ffc16fce103dd679f1f53ebd78600e0a3ed04744f29ef94da7bd750b"} err="failed to get container status \"438facf2ffc16fce103dd679f1f53ebd78600e0a3ed04744f29ef94da7bd750b\": rpc error: code = NotFound desc = could not find container \"438facf2ffc16fce103dd679f1f53ebd78600e0a3ed04744f29ef94da7bd750b\": container with ID starting with 438facf2ffc16fce103dd679f1f53ebd78600e0a3ed04744f29ef94da7bd750b not found: ID does not exist"
Sep 18 14:12:07.077424 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:12:07.077385    2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-eeb0a3f4-595b879cbc-7zh7w"]
Sep 18 14:12:07.110653 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:12:07.110625    2565 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-eeb0a3f4-595b879cbc-7zh7w"]
Sep 18 14:12:07.761799 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:12:07.761765    2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-migrate-5f7e430d-gw-openshift-default-56c9bfbb8-ghlx9"]
Sep 18 14:12:07.762273 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:12:07.762260    2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e0949a12-e8b4-45de-8b8c-840e57646d04" containerName="revoke-keys"
Sep 18 14:12:07.762273 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:12:07.762273    2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="e0949a12-e8b4-45de-8b8c-840e57646d04" containerName="revoke-keys"
Sep 18 14:12:07.762362 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:12:07.762291    2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f61bfe0e-7506-446a-a810-23403f108a1a" containerName="payload-processing"
Sep 18 14:12:07.762362 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:12:07.762297    2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="f61bfe0e-7506-446a-a810-23403f108a1a" containerName="payload-processing"
Sep 18 14:12:07.762428 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:12:07.762368    2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="e0949a12-e8b4-45de-8b8c-840e57646d04" containerName="revoke-keys"
Sep 18 14:12:07.762428 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:12:07.762380    2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="f61bfe0e-7506-446a-a810-23403f108a1a" containerName="payload-processing"
Sep 18 14:12:07.765654 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:12:07.765639    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-migrate-5f7e430d-gw-openshift-default-56c9bfbb8-ghlx9"
Sep 18 14:12:07.769152 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:12:07.769131    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-migrate-5f7e430d-gw-openshift-default-dockercfg-zfxtn\""
Sep 18 14:12:07.819679 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:12:07.819650    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-migrate-5f7e430d-gw-openshift-default-56c9bfbb8-ghlx9"]
Sep 18 14:12:07.822428 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:12:07.822392    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2e89e3fa-fe49-4ddf-8cab-dee2fc29e5dd-istiod-ca-cert\") pod \"e2e-migrate-5f7e430d-gw-openshift-default-56c9bfbb8-ghlx9\" (UID: \"2e89e3fa-fe49-4ddf-8cab-dee2fc29e5dd\") " pod="openshift-ingress/e2e-migrate-5f7e430d-gw-openshift-default-56c9bfbb8-ghlx9"
Sep 18 14:12:07.822578 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:12:07.822533    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2e89e3fa-fe49-4ddf-8cab-dee2fc29e5dd-credential-socket\") pod \"e2e-migrate-5f7e430d-gw-openshift-default-56c9bfbb8-ghlx9\" (UID: \"2e89e3fa-fe49-4ddf-8cab-dee2fc29e5dd\") " pod="openshift-ingress/e2e-migrate-5f7e430d-gw-openshift-default-56c9bfbb8-ghlx9"
Sep 18 14:12:07.822654 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:12:07.822586    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-z9m97\" (UniqueName: \"kubernetes.io/projected/2e89e3fa-fe49-4ddf-8cab-dee2fc29e5dd-kube-api-access-z9m97\") pod \"e2e-migrate-5f7e430d-gw-openshift-default-56c9bfbb8-ghlx9\" (UID: \"2e89e3fa-fe49-4ddf-8cab-dee2fc29e5dd\") " pod="openshift-ingress/e2e-migrate-5f7e430d-gw-openshift-default-56c9bfbb8-ghlx9"
Sep 18 14:12:07.822771 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:12:07.822737    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2e89e3fa-fe49-4ddf-8cab-dee2fc29e5dd-istio-data\") pod \"e2e-migrate-5f7e430d-gw-openshift-default-56c9bfbb8-ghlx9\" (UID: \"2e89e3fa-fe49-4ddf-8cab-dee2fc29e5dd\") " pod="openshift-ingress/e2e-migrate-5f7e430d-gw-openshift-default-56c9bfbb8-ghlx9"
Sep 18 14:12:07.822894 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:12:07.822778    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2e89e3fa-fe49-4ddf-8cab-dee2fc29e5dd-workload-certs\") pod \"e2e-migrate-5f7e430d-gw-openshift-default-56c9bfbb8-ghlx9\" (UID: \"2e89e3fa-fe49-4ddf-8cab-dee2fc29e5dd\") " pod="openshift-ingress/e2e-migrate-5f7e430d-gw-openshift-default-56c9bfbb8-ghlx9"
Sep 18 14:12:07.822894 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:12:07.822797    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2e89e3fa-fe49-4ddf-8cab-dee2fc29e5dd-istio-envoy\") pod \"e2e-migrate-5f7e430d-gw-openshift-default-56c9bfbb8-ghlx9\" (UID: \"2e89e3fa-fe49-4ddf-8cab-dee2fc29e5dd\") " pod="openshift-ingress/e2e-migrate-5f7e430d-gw-openshift-default-56c9bfbb8-ghlx9"
Sep 18 14:12:07.822894 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:12:07.822835    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2e89e3fa-fe49-4ddf-8cab-dee2fc29e5dd-workload-socket\") pod \"e2e-migrate-5f7e430d-gw-openshift-default-56c9bfbb8-ghlx9\" (UID: \"2e89e3fa-fe49-4ddf-8cab-dee2fc29e5dd\") " pod="openshift-ingress/e2e-migrate-5f7e430d-gw-openshift-default-56c9bfbb8-ghlx9"
Sep 18 14:12:07.823089 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:12:07.822904    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2e89e3fa-fe49-4ddf-8cab-dee2fc29e5dd-istio-token\") pod \"e2e-migrate-5f7e430d-gw-openshift-default-56c9bfbb8-ghlx9\" (UID: \"2e89e3fa-fe49-4ddf-8cab-dee2fc29e5dd\") " pod="openshift-ingress/e2e-migrate-5f7e430d-gw-openshift-default-56c9bfbb8-ghlx9"
Sep 18 14:12:07.823089 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:12:07.822952    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2e89e3fa-fe49-4ddf-8cab-dee2fc29e5dd-istio-podinfo\") pod \"e2e-migrate-5f7e430d-gw-openshift-default-56c9bfbb8-ghlx9\" (UID: \"2e89e3fa-fe49-4ddf-8cab-dee2fc29e5dd\") " pod="openshift-ingress/e2e-migrate-5f7e430d-gw-openshift-default-56c9bfbb8-ghlx9"
Sep 18 14:12:07.913249 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:12:07.913217    2565 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e0949a12-e8b4-45de-8b8c-840e57646d04" path="/var/lib/kubelet/pods/e0949a12-e8b4-45de-8b8c-840e57646d04/volumes"
Sep 18 14:12:07.913669 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:12:07.913654    2565 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f61bfe0e-7506-446a-a810-23403f108a1a" path="/var/lib/kubelet/pods/f61bfe0e-7506-446a-a810-23403f108a1a/volumes"
Sep 18 14:12:07.924074 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:12:07.924051    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2e89e3fa-fe49-4ddf-8cab-dee2fc29e5dd-istio-data\") pod \"e2e-migrate-5f7e430d-gw-openshift-default-56c9bfbb8-ghlx9\" (UID: \"2e89e3fa-fe49-4ddf-8cab-dee2fc29e5dd\") " pod="openshift-ingress/e2e-migrate-5f7e430d-gw-openshift-default-56c9bfbb8-ghlx9"
Sep 18 14:12:07.924210 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:12:07.924082    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2e89e3fa-fe49-4ddf-8cab-dee2fc29e5dd-workload-certs\") pod \"e2e-migrate-5f7e430d-gw-openshift-default-56c9bfbb8-ghlx9\" (UID: \"2e89e3fa-fe49-4ddf-8cab-dee2fc29e5dd\") " pod="openshift-ingress/e2e-migrate-5f7e430d-gw-openshift-default-56c9bfbb8-ghlx9"
Sep 18 14:12:07.924210 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:12:07.924100    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2e89e3fa-fe49-4ddf-8cab-dee2fc29e5dd-istio-envoy\") pod \"e2e-migrate-5f7e430d-gw-openshift-default-56c9bfbb8-ghlx9\" (UID: \"2e89e3fa-fe49-4ddf-8cab-dee2fc29e5dd\") " pod="openshift-ingress/e2e-migrate-5f7e430d-gw-openshift-default-56c9bfbb8-ghlx9"
Sep 18 14:12:07.924210 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:12:07.924127    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2e89e3fa-fe49-4ddf-8cab-dee2fc29e5dd-workload-socket\") pod \"e2e-migrate-5f7e430d-gw-openshift-default-56c9bfbb8-ghlx9\" (UID: \"2e89e3fa-fe49-4ddf-8cab-dee2fc29e5dd\") " pod="openshift-ingress/e2e-migrate-5f7e430d-gw-openshift-default-56c9bfbb8-ghlx9"
Sep 18 14:12:07.924210 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:12:07.924160    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2e89e3fa-fe49-4ddf-8cab-dee2fc29e5dd-istio-token\") pod \"e2e-migrate-5f7e430d-gw-openshift-default-56c9bfbb8-ghlx9\" (UID: \"2e89e3fa-fe49-4ddf-8cab-dee2fc29e5dd\") " pod="openshift-ingress/e2e-migrate-5f7e430d-gw-openshift-default-56c9bfbb8-ghlx9"
Sep 18 14:12:07.924210 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:12:07.924194    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2e89e3fa-fe49-4ddf-8cab-dee2fc29e5dd-istio-podinfo\") pod \"e2e-migrate-5f7e430d-gw-openshift-default-56c9bfbb8-ghlx9\" (UID: \"2e89e3fa-fe49-4ddf-8cab-dee2fc29e5dd\") " pod="openshift-ingress/e2e-migrate-5f7e430d-gw-openshift-default-56c9bfbb8-ghlx9"
Sep 18 14:12:07.924455 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:12:07.924232    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2e89e3fa-fe49-4ddf-8cab-dee2fc29e5dd-istiod-ca-cert\") pod \"e2e-migrate-5f7e430d-gw-openshift-default-56c9bfbb8-ghlx9\" (UID: \"2e89e3fa-fe49-4ddf-8cab-dee2fc29e5dd\") " pod="openshift-ingress/e2e-migrate-5f7e430d-gw-openshift-default-56c9bfbb8-ghlx9"
Sep 18 14:12:07.924455 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:12:07.924290    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2e89e3fa-fe49-4ddf-8cab-dee2fc29e5dd-credential-socket\") pod \"e2e-migrate-5f7e430d-gw-openshift-default-56c9bfbb8-ghlx9\" (UID: \"2e89e3fa-fe49-4ddf-8cab-dee2fc29e5dd\") " pod="openshift-ingress/e2e-migrate-5f7e430d-gw-openshift-default-56c9bfbb8-ghlx9"
Sep 18 14:12:07.924455 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:12:07.924334    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-z9m97\" (UniqueName: \"kubernetes.io/projected/2e89e3fa-fe49-4ddf-8cab-dee2fc29e5dd-kube-api-access-z9m97\") pod \"e2e-migrate-5f7e430d-gw-openshift-default-56c9bfbb8-ghlx9\" (UID: \"2e89e3fa-fe49-4ddf-8cab-dee2fc29e5dd\") " pod="openshift-ingress/e2e-migrate-5f7e430d-gw-openshift-default-56c9bfbb8-ghlx9"
Sep 18 14:12:07.924455 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:12:07.924435    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2e89e3fa-fe49-4ddf-8cab-dee2fc29e5dd-workload-certs\") pod \"e2e-migrate-5f7e430d-gw-openshift-default-56c9bfbb8-ghlx9\" (UID: \"2e89e3fa-fe49-4ddf-8cab-dee2fc29e5dd\") " pod="openshift-ingress/e2e-migrate-5f7e430d-gw-openshift-default-56c9bfbb8-ghlx9"
Sep 18 14:12:07.924756 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:12:07.924735    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2e89e3fa-fe49-4ddf-8cab-dee2fc29e5dd-istio-data\") pod \"e2e-migrate-5f7e430d-gw-openshift-default-56c9bfbb8-ghlx9\" (UID: \"2e89e3fa-fe49-4ddf-8cab-dee2fc29e5dd\") " pod="openshift-ingress/e2e-migrate-5f7e430d-gw-openshift-default-56c9bfbb8-ghlx9"
Sep 18 14:12:07.924822 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:12:07.924776    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2e89e3fa-fe49-4ddf-8cab-dee2fc29e5dd-workload-socket\") pod \"e2e-migrate-5f7e430d-gw-openshift-default-56c9bfbb8-ghlx9\" (UID: \"2e89e3fa-fe49-4ddf-8cab-dee2fc29e5dd\") " pod="openshift-ingress/e2e-migrate-5f7e430d-gw-openshift-default-56c9bfbb8-ghlx9"
Sep 18 14:12:07.924884 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:12:07.924822    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2e89e3fa-fe49-4ddf-8cab-dee2fc29e5dd-credential-socket\") pod \"e2e-migrate-5f7e430d-gw-openshift-default-56c9bfbb8-ghlx9\" (UID: \"2e89e3fa-fe49-4ddf-8cab-dee2fc29e5dd\") " pod="openshift-ingress/e2e-migrate-5f7e430d-gw-openshift-default-56c9bfbb8-ghlx9"
Sep 18 14:12:07.924977 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:12:07.924956    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2e89e3fa-fe49-4ddf-8cab-dee2fc29e5dd-istiod-ca-cert\") pod \"e2e-migrate-5f7e430d-gw-openshift-default-56c9bfbb8-ghlx9\" (UID: \"2e89e3fa-fe49-4ddf-8cab-dee2fc29e5dd\") " pod="openshift-ingress/e2e-migrate-5f7e430d-gw-openshift-default-56c9bfbb8-ghlx9"
Sep 18 14:12:07.926416 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:12:07.926394    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2e89e3fa-fe49-4ddf-8cab-dee2fc29e5dd-istio-envoy\") pod \"e2e-migrate-5f7e430d-gw-openshift-default-56c9bfbb8-ghlx9\" (UID: \"2e89e3fa-fe49-4ddf-8cab-dee2fc29e5dd\") " pod="openshift-ingress/e2e-migrate-5f7e430d-gw-openshift-default-56c9bfbb8-ghlx9"
Sep 18 14:12:07.926648 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:12:07.926632    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2e89e3fa-fe49-4ddf-8cab-dee2fc29e5dd-istio-podinfo\") pod \"e2e-migrate-5f7e430d-gw-openshift-default-56c9bfbb8-ghlx9\" (UID: \"2e89e3fa-fe49-4ddf-8cab-dee2fc29e5dd\") " pod="openshift-ingress/e2e-migrate-5f7e430d-gw-openshift-default-56c9bfbb8-ghlx9"
Sep 18 14:12:07.963431 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:12:07.963406    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2e89e3fa-fe49-4ddf-8cab-dee2fc29e5dd-istio-token\") pod \"e2e-migrate-5f7e430d-gw-openshift-default-56c9bfbb8-ghlx9\" (UID: \"2e89e3fa-fe49-4ddf-8cab-dee2fc29e5dd\") " pod="openshift-ingress/e2e-migrate-5f7e430d-gw-openshift-default-56c9bfbb8-ghlx9"
Sep 18 14:12:07.963606 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:12:07.963584    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-z9m97\" (UniqueName: \"kubernetes.io/projected/2e89e3fa-fe49-4ddf-8cab-dee2fc29e5dd-kube-api-access-z9m97\") pod \"e2e-migrate-5f7e430d-gw-openshift-default-56c9bfbb8-ghlx9\" (UID: \"2e89e3fa-fe49-4ddf-8cab-dee2fc29e5dd\") " pod="openshift-ingress/e2e-migrate-5f7e430d-gw-openshift-default-56c9bfbb8-ghlx9"
Sep 18 14:12:08.076165 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:12:08.076127    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-migrate-5f7e430d-gw-openshift-default-56c9bfbb8-ghlx9"
Sep 18 14:12:08.376126 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:12:08.376100    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-migrate-5f7e430d-gw-openshift-default-56c9bfbb8-ghlx9"]
Sep 18 14:12:08.378601 ip-10-0-129-16 kubenswrapper[2565]: W0918 14:12:08.378572    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2e89e3fa_fe49_4ddf_8cab_dee2fc29e5dd.slice/crio-6dd6cb22fb6281af8e51d4bcbf068276cfca3f61cf8fa9f926ad88ea3c38cfe3 WatchSource:0}: Error finding container 6dd6cb22fb6281af8e51d4bcbf068276cfca3f61cf8fa9f926ad88ea3c38cfe3: Status 404 returned error can't find the container with id 6dd6cb22fb6281af8e51d4bcbf068276cfca3f61cf8fa9f926ad88ea3c38cfe3
Sep 18 14:12:08.381014 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:12:08.380978    2565 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 18 14:12:08.381145 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:12:08.381115    2565 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 18 14:12:08.381186 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:12:08.381148    2565 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 18 14:12:09.030212 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:12:09.030174    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-5f7e430d-gw-openshift-default-56c9bfbb8-ghlx9" event={"ID":"2e89e3fa-fe49-4ddf-8cab-dee2fc29e5dd","Type":"ContainerStarted","Data":"82a16a09bb466b390f48e3635757cb4fa9f107e66d33555e07aa5072f2e0338f"}
Sep 18 14:12:09.030413 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:12:09.030240    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-5f7e430d-gw-openshift-default-56c9bfbb8-ghlx9" event={"ID":"2e89e3fa-fe49-4ddf-8cab-dee2fc29e5dd","Type":"ContainerStarted","Data":"6dd6cb22fb6281af8e51d4bcbf068276cfca3f61cf8fa9f926ad88ea3c38cfe3"}
Sep 18 14:12:09.075790 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:12:09.075740    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-migrate-5f7e430d-gw-openshift-default-56c9bfbb8-ghlx9" podStartSLOduration=2.075725332 podStartE2EDuration="2.075725332s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 14:12:07 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 14:12:09.073561134 +0000 UTC m=+797.247470822" watchObservedRunningTime="2026-09-18 14:12:09.075725332 +0000 UTC m=+797.249634993"
Sep 18 14:12:09.076698 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:12:09.076660    2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-migrate-5f7e430d-gw-openshift-default-56c9bfbb8-ghlx9"
Sep 18 14:12:09.078379 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:12:09.078358    2565 patch_prober.go:28] interesting pod/e2e-migrate-5f7e430d-gw-openshift-default-56c9bfbb8-ghlx9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.50:15021/healthz/ready\": dial tcp 10.134.0.50:15021: connect: connection refused" start-of-body=
Sep 18 14:12:09.078474 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:12:09.078407    2565 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-migrate-5f7e430d-gw-openshift-default-56c9bfbb8-ghlx9" podUID="2e89e3fa-fe49-4ddf-8cab-dee2fc29e5dd" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.50:15021/healthz/ready\": dial tcp 10.134.0.50:15021: connect: connection refused"
Sep 18 14:12:10.081210 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:12:10.081180    2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-migrate-5f7e430d-gw-openshift-default-56c9bfbb8-ghlx9"
Sep 18 14:12:11.042896 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:12:11.042861    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-migrate-5f7e430d-gw-openshift-default-56c9bfbb8-ghlx9"
Sep 18 14:12:11.043905 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:12:11.043875    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-migrate-5f7e430d-gw-openshift-default-56c9bfbb8-ghlx9"
Sep 18 14:12:15.534640 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:12:15.534604    2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-migrate-5f7e430d-96fbcbdd6-2qm99"]
Sep 18 14:12:15.537574 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:12:15.537558    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-migrate-5f7e430d-96fbcbdd6-2qm99"
Sep 18 14:12:15.540283 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:12:15.540260    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-migrate-5f7e430d\""
Sep 18 14:12:15.565629 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:12:15.565596    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-migrate-5f7e430d-96fbcbdd6-2qm99"]
Sep 18 14:12:15.586316 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:12:15.586285    2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-migrate-5f7e430d-67b67c94f4-vlwmn"]
Sep 18 14:12:15.589091 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:12:15.589070    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-migrate-5f7e430d-67b67c94f4-vlwmn"
Sep 18 14:12:15.591903 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:12:15.591883    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-migrate-5f7e430d-dockercfg-n2vj9\""
Sep 18 14:12:15.603434 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:12:15.603410    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-migrate-5f7e430d-67b67c94f4-vlwmn"]
Sep 18 14:12:15.611528 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:12:15.611503    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-56t8w\" (UniqueName: \"kubernetes.io/projected/4d819468-a5ee-4df5-8fb4-bf1a63b37a21-kube-api-access-56t8w\") pod \"payload-processing-e2e-migrate-5f7e430d-96fbcbdd6-2qm99\" (UID: \"4d819468-a5ee-4df5-8fb4-bf1a63b37a21\") " pod="openshift-ingress/payload-processing-e2e-migrate-5f7e430d-96fbcbdd6-2qm99"
Sep 18 14:12:15.611662 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:12:15.611644    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/4d819468-a5ee-4df5-8fb4-bf1a63b37a21-plugins-config-volume\") pod \"payload-processing-e2e-migrate-5f7e430d-96fbcbdd6-2qm99\" (UID: \"4d819468-a5ee-4df5-8fb4-bf1a63b37a21\") " pod="openshift-ingress/payload-processing-e2e-migrate-5f7e430d-96fbcbdd6-2qm99"
Sep 18 14:12:15.712529 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:12:15.712496    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/4d819468-a5ee-4df5-8fb4-bf1a63b37a21-plugins-config-volume\") pod \"payload-processing-e2e-migrate-5f7e430d-96fbcbdd6-2qm99\" (UID: \"4d819468-a5ee-4df5-8fb4-bf1a63b37a21\") " pod="openshift-ingress/payload-processing-e2e-migrate-5f7e430d-96fbcbdd6-2qm99"
Sep 18 14:12:15.712797 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:12:15.712593    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/4b34a994-8cfb-43f2-8c04-e57dbef010a8-plugins-config-volume\") pod \"payload-pre-processing-e2e-migrate-5f7e430d-67b67c94f4-vlwmn\" (UID: \"4b34a994-8cfb-43f2-8c04-e57dbef010a8\") " pod="openshift-ingress/payload-pre-processing-e2e-migrate-5f7e430d-67b67c94f4-vlwmn"
Sep 18 14:12:15.712797 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:12:15.712632    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-56t8w\" (UniqueName: \"kubernetes.io/projected/4d819468-a5ee-4df5-8fb4-bf1a63b37a21-kube-api-access-56t8w\") pod \"payload-processing-e2e-migrate-5f7e430d-96fbcbdd6-2qm99\" (UID: \"4d819468-a5ee-4df5-8fb4-bf1a63b37a21\") " pod="openshift-ingress/payload-processing-e2e-migrate-5f7e430d-96fbcbdd6-2qm99"
Sep 18 14:12:15.712797 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:12:15.712672    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qwzms\" (UniqueName: \"kubernetes.io/projected/4b34a994-8cfb-43f2-8c04-e57dbef010a8-kube-api-access-qwzms\") pod \"payload-pre-processing-e2e-migrate-5f7e430d-67b67c94f4-vlwmn\" (UID: \"4b34a994-8cfb-43f2-8c04-e57dbef010a8\") " pod="openshift-ingress/payload-pre-processing-e2e-migrate-5f7e430d-67b67c94f4-vlwmn"
Sep 18 14:12:15.713337 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:12:15.713315    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/4d819468-a5ee-4df5-8fb4-bf1a63b37a21-plugins-config-volume\") pod \"payload-processing-e2e-migrate-5f7e430d-96fbcbdd6-2qm99\" (UID: \"4d819468-a5ee-4df5-8fb4-bf1a63b37a21\") " pod="openshift-ingress/payload-processing-e2e-migrate-5f7e430d-96fbcbdd6-2qm99"
Sep 18 14:12:15.722533 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:12:15.722512    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-56t8w\" (UniqueName: \"kubernetes.io/projected/4d819468-a5ee-4df5-8fb4-bf1a63b37a21-kube-api-access-56t8w\") pod \"payload-processing-e2e-migrate-5f7e430d-96fbcbdd6-2qm99\" (UID: \"4d819468-a5ee-4df5-8fb4-bf1a63b37a21\") " pod="openshift-ingress/payload-processing-e2e-migrate-5f7e430d-96fbcbdd6-2qm99"
Sep 18 14:12:15.814426 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:12:15.814390    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/4b34a994-8cfb-43f2-8c04-e57dbef010a8-plugins-config-volume\") pod \"payload-pre-processing-e2e-migrate-5f7e430d-67b67c94f4-vlwmn\" (UID: \"4b34a994-8cfb-43f2-8c04-e57dbef010a8\") " pod="openshift-ingress/payload-pre-processing-e2e-migrate-5f7e430d-67b67c94f4-vlwmn"
Sep 18 14:12:15.814587 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:12:15.814448    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qwzms\" (UniqueName: \"kubernetes.io/projected/4b34a994-8cfb-43f2-8c04-e57dbef010a8-kube-api-access-qwzms\") pod \"payload-pre-processing-e2e-migrate-5f7e430d-67b67c94f4-vlwmn\" (UID: \"4b34a994-8cfb-43f2-8c04-e57dbef010a8\") " pod="openshift-ingress/payload-pre-processing-e2e-migrate-5f7e430d-67b67c94f4-vlwmn"
Sep 18 14:12:15.815178 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:12:15.815157    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/4b34a994-8cfb-43f2-8c04-e57dbef010a8-plugins-config-volume\") pod \"payload-pre-processing-e2e-migrate-5f7e430d-67b67c94f4-vlwmn\" (UID: \"4b34a994-8cfb-43f2-8c04-e57dbef010a8\") " pod="openshift-ingress/payload-pre-processing-e2e-migrate-5f7e430d-67b67c94f4-vlwmn"
Sep 18 14:12:15.824249 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:12:15.824220    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qwzms\" (UniqueName: \"kubernetes.io/projected/4b34a994-8cfb-43f2-8c04-e57dbef010a8-kube-api-access-qwzms\") pod \"payload-pre-processing-e2e-migrate-5f7e430d-67b67c94f4-vlwmn\" (UID: \"4b34a994-8cfb-43f2-8c04-e57dbef010a8\") " pod="openshift-ingress/payload-pre-processing-e2e-migrate-5f7e430d-67b67c94f4-vlwmn"
Sep 18 14:12:15.848147 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:12:15.848113    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-migrate-5f7e430d-96fbcbdd6-2qm99"
Sep 18 14:12:15.899467 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:12:15.899062    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-migrate-5f7e430d-67b67c94f4-vlwmn"
Sep 18 14:12:16.130205 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:12:16.130172    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-migrate-5f7e430d-96fbcbdd6-2qm99"]
Sep 18 14:12:16.130554 ip-10-0-129-16 kubenswrapper[2565]: W0918 14:12:16.130522    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4d819468_a5ee_4df5_8fb4_bf1a63b37a21.slice/crio-af3a02b34f9914d0ab3691e60fadaf7443464ef4f77f7f56951ee37ceea5d8cf WatchSource:0}: Error finding container af3a02b34f9914d0ab3691e60fadaf7443464ef4f77f7f56951ee37ceea5d8cf: Status 404 returned error can't find the container with id af3a02b34f9914d0ab3691e60fadaf7443464ef4f77f7f56951ee37ceea5d8cf
Sep 18 14:12:16.200294 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:12:16.200268    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-migrate-5f7e430d-67b67c94f4-vlwmn"]
Sep 18 14:12:16.202198 ip-10-0-129-16 kubenswrapper[2565]: W0918 14:12:16.202171    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4b34a994_8cfb_43f2_8c04_e57dbef010a8.slice/crio-cbc00d0f6415c11db3ed2774fd363042fe26cba8dd922b9dee6d74a32dd0c410 WatchSource:0}: Error finding container cbc00d0f6415c11db3ed2774fd363042fe26cba8dd922b9dee6d74a32dd0c410: Status 404 returned error can't find the container with id cbc00d0f6415c11db3ed2774fd363042fe26cba8dd922b9dee6d74a32dd0c410
Sep 18 14:12:17.079315 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:12:17.079274    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-migrate-5f7e430d-67b67c94f4-vlwmn" event={"ID":"4b34a994-8cfb-43f2-8c04-e57dbef010a8","Type":"ContainerStarted","Data":"7dbc34cff3dcd147fd96ccaf288e0f3773eaf76530bc908c6001ee4c6c9f1691"}
Sep 18 14:12:17.079812 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:12:17.079338    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-migrate-5f7e430d-67b67c94f4-vlwmn" event={"ID":"4b34a994-8cfb-43f2-8c04-e57dbef010a8","Type":"ContainerStarted","Data":"cbc00d0f6415c11db3ed2774fd363042fe26cba8dd922b9dee6d74a32dd0c410"}
Sep 18 14:12:17.079812 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:12:17.079469    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-migrate-5f7e430d-67b67c94f4-vlwmn"
Sep 18 14:12:17.080943 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:12:17.080918    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-migrate-5f7e430d-96fbcbdd6-2qm99" event={"ID":"4d819468-a5ee-4df5-8fb4-bf1a63b37a21","Type":"ContainerStarted","Data":"2d756864f75736f49c81f566a2398a3ccda32c0185439f385e80ba34c296a031"}
Sep 18 14:12:17.081026 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:12:17.080963    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-migrate-5f7e430d-96fbcbdd6-2qm99" event={"ID":"4d819468-a5ee-4df5-8fb4-bf1a63b37a21","Type":"ContainerStarted","Data":"af3a02b34f9914d0ab3691e60fadaf7443464ef4f77f7f56951ee37ceea5d8cf"}
Sep 18 14:12:17.081026 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:12:17.081000    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-migrate-5f7e430d-96fbcbdd6-2qm99"
Sep 18 14:12:17.121991 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:12:17.121933    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-migrate-5f7e430d-67b67c94f4-vlwmn" podStartSLOduration=2.12191544 podStartE2EDuration="2.12191544s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 14:12:15 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 14:12:17.121624852 +0000 UTC m=+805.295534541" watchObservedRunningTime="2026-09-18 14:12:17.12191544 +0000 UTC m=+805.295825101"
Sep 18 14:12:17.160311 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:12:17.160252    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-migrate-5f7e430d-96fbcbdd6-2qm99" podStartSLOduration=2.160235233 podStartE2EDuration="2.160235233s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 14:12:15 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 14:12:17.15993445 +0000 UTC m=+805.333844130" watchObservedRunningTime="2026-09-18 14:12:17.160235233 +0000 UTC m=+805.334144893"
Sep 18 14:12:28.088990 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:12:28.088963    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-migrate-5f7e430d-96fbcbdd6-2qm99"
Sep 18 14:12:28.089469 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:12:28.089048    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-migrate-5f7e430d-67b67c94f4-vlwmn"
Sep 18 14:12:34.195071 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:12:34.195020    2565 generic.go:356] "Generic (PLEG): container finished" podID="2e89e3fa-fe49-4ddf-8cab-dee2fc29e5dd" containerID="82a16a09bb466b390f48e3635757cb4fa9f107e66d33555e07aa5072f2e0338f" exitCode=0
Sep 18 14:12:34.195480 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:12:34.195095    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-5f7e430d-gw-openshift-default-56c9bfbb8-ghlx9" event={"ID":"2e89e3fa-fe49-4ddf-8cab-dee2fc29e5dd","Type":"ContainerDied","Data":"82a16a09bb466b390f48e3635757cb4fa9f107e66d33555e07aa5072f2e0338f"}
Sep 18 14:12:34.195561 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:12:34.195546    2565 scope.go:117] "RemoveContainer" containerID="82a16a09bb466b390f48e3635757cb4fa9f107e66d33555e07aa5072f2e0338f"
Sep 18 14:12:34.196824 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:12:34.196800    2565 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 18 14:12:34.196886 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:12:34.196864    2565 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 18 14:12:34.196920 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:12:34.196896    2565 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 18 14:12:35.202522 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:12:35.202487    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-5f7e430d-gw-openshift-default-56c9bfbb8-ghlx9" event={"ID":"2e89e3fa-fe49-4ddf-8cab-dee2fc29e5dd","Type":"ContainerStarted","Data":"bc05e14195fbfbda2bfd9fd7901d3093b46b99fadd7ec9bdd5740847735aa92a"}
Sep 18 14:12:36.077302 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:12:36.077264    2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-migrate-5f7e430d-gw-openshift-default-56c9bfbb8-ghlx9"
Sep 18 14:12:36.077491 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:12:36.077452    2565 patch_prober.go:28] interesting pod/e2e-migrate-5f7e430d-gw-openshift-default-56c9bfbb8-ghlx9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.50:15021/healthz/ready\": dial tcp 10.134.0.50:15021: connect: connection refused" start-of-body=
Sep 18 14:12:36.077539 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:12:36.077503    2565 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-migrate-5f7e430d-gw-openshift-default-56c9bfbb8-ghlx9" podUID="2e89e3fa-fe49-4ddf-8cab-dee2fc29e5dd" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.50:15021/healthz/ready\": dial tcp 10.134.0.50:15021: connect: connection refused"
Sep 18 14:12:37.077482 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:12:37.077440    2565 patch_prober.go:28] interesting pod/e2e-migrate-5f7e430d-gw-openshift-default-56c9bfbb8-ghlx9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.50:15021/healthz/ready\": dial tcp 10.134.0.50:15021: connect: connection refused" start-of-body=
Sep 18 14:12:37.077957 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:12:37.077528    2565 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-migrate-5f7e430d-gw-openshift-default-56c9bfbb8-ghlx9" podUID="2e89e3fa-fe49-4ddf-8cab-dee2fc29e5dd" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.50:15021/healthz/ready\": dial tcp 10.134.0.50:15021: connect: connection refused"
Sep 18 14:12:38.076676 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:12:38.076644    2565 patch_prober.go:28] interesting pod/e2e-migrate-5f7e430d-gw-openshift-default-56c9bfbb8-ghlx9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.50:15021/healthz/ready\": dial tcp 10.134.0.50:15021: connect: connection refused" start-of-body=
Sep 18 14:12:38.076852 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:12:38.076697    2565 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-migrate-5f7e430d-gw-openshift-default-56c9bfbb8-ghlx9" podUID="2e89e3fa-fe49-4ddf-8cab-dee2fc29e5dd" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.50:15021/healthz/ready\": dial tcp 10.134.0.50:15021: connect: connection refused"
Sep 18 14:12:39.076986 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:12:39.076949    2565 patch_prober.go:28] interesting pod/e2e-migrate-5f7e430d-gw-openshift-default-56c9bfbb8-ghlx9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.50:15021/healthz/ready\": dial tcp 10.134.0.50:15021: connect: connection refused" start-of-body=
Sep 18 14:12:39.077403 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:12:39.077013    2565 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-migrate-5f7e430d-gw-openshift-default-56c9bfbb8-ghlx9" podUID="2e89e3fa-fe49-4ddf-8cab-dee2fc29e5dd" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.50:15021/healthz/ready\": dial tcp 10.134.0.50:15021: connect: connection refused"
Sep 18 14:12:40.076736 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:12:40.076705    2565 patch_prober.go:28] interesting pod/e2e-migrate-5f7e430d-gw-openshift-default-56c9bfbb8-ghlx9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.50:15021/healthz/ready\": dial tcp 10.134.0.50:15021: connect: connection refused" start-of-body=
Sep 18 14:12:40.076908 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:12:40.076773    2565 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-migrate-5f7e430d-gw-openshift-default-56c9bfbb8-ghlx9" podUID="2e89e3fa-fe49-4ddf-8cab-dee2fc29e5dd" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.50:15021/healthz/ready\": dial tcp 10.134.0.50:15021: connect: connection refused"
Sep 18 14:12:41.043140 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:12:41.043103    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-migrate-5f7e430d-gw-openshift-default-56c9bfbb8-ghlx9"
Sep 18 14:12:41.081217 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:12:41.081186    2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-migrate-5f7e430d-gw-openshift-default-56c9bfbb8-ghlx9"
Sep 18 14:12:41.241101 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:12:41.241073    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-migrate-5f7e430d-gw-openshift-default-56c9bfbb8-ghlx9"
Sep 18 14:12:52.232226 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:12:52.232185    2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-migrate-5f7e430d-96fbcbdd6-2qm99"]
Sep 18 14:12:52.233977 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:12:52.232425    2565 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-migrate-5f7e430d-96fbcbdd6-2qm99" podUID="4d819468-a5ee-4df5-8fb4-bf1a63b37a21" containerName="payload-processing" containerID="cri-o://2d756864f75736f49c81f566a2398a3ccda32c0185439f385e80ba34c296a031" gracePeriod=30
Sep 18 14:12:52.239496 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:12:52.239466    2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-migrate-5f7e430d-67b67c94f4-vlwmn"]
Sep 18 14:12:52.239764 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:12:52.239721    2565 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-migrate-5f7e430d-67b67c94f4-vlwmn" podUID="4b34a994-8cfb-43f2-8c04-e57dbef010a8" containerName="payload-pre-processing" containerID="cri-o://7dbc34cff3dcd147fd96ccaf288e0f3773eaf76530bc908c6001ee4c6c9f1691" gracePeriod=30
Sep 18 14:12:52.532353 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:12:52.532326    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-migrate-5f7e430d-67b67c94f4-vlwmn"
Sep 18 14:12:52.535672 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:12:52.535650    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-migrate-5f7e430d-96fbcbdd6-2qm99"
Sep 18 14:12:52.572220 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:12:52.572182    2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-migrate-5f7e430d-gw-openshift-default-56c9bfbb8-ghlx9"]
Sep 18 14:12:52.572460 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:12:52.572434    2565 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-migrate-5f7e430d-gw-openshift-default-56c9bfbb8-ghlx9" podUID="2e89e3fa-fe49-4ddf-8cab-dee2fc29e5dd" containerName="istio-proxy" containerID="cri-o://bc05e14195fbfbda2bfd9fd7901d3093b46b99fadd7ec9bdd5740847735aa92a" gracePeriod=30
Sep 18 14:12:52.621600 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:12:52.621565    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/4d819468-a5ee-4df5-8fb4-bf1a63b37a21-plugins-config-volume\") pod \"4d819468-a5ee-4df5-8fb4-bf1a63b37a21\" (UID: \"4d819468-a5ee-4df5-8fb4-bf1a63b37a21\") "
Sep 18 14:12:52.621781 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:12:52.621643    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-56t8w\" (UniqueName: \"kubernetes.io/projected/4d819468-a5ee-4df5-8fb4-bf1a63b37a21-kube-api-access-56t8w\") pod \"4d819468-a5ee-4df5-8fb4-bf1a63b37a21\" (UID: \"4d819468-a5ee-4df5-8fb4-bf1a63b37a21\") "
Sep 18 14:12:52.621781 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:12:52.621691    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/4b34a994-8cfb-43f2-8c04-e57dbef010a8-plugins-config-volume\") pod \"4b34a994-8cfb-43f2-8c04-e57dbef010a8\" (UID: \"4b34a994-8cfb-43f2-8c04-e57dbef010a8\") "
Sep 18 14:12:52.621781 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:12:52.621735    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-qwzms\" (UniqueName: \"kubernetes.io/projected/4b34a994-8cfb-43f2-8c04-e57dbef010a8-kube-api-access-qwzms\") pod \"4b34a994-8cfb-43f2-8c04-e57dbef010a8\" (UID: \"4b34a994-8cfb-43f2-8c04-e57dbef010a8\") "
Sep 18 14:12:52.622050 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:12:52.621921    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4d819468-a5ee-4df5-8fb4-bf1a63b37a21-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "4d819468-a5ee-4df5-8fb4-bf1a63b37a21" (UID: "4d819468-a5ee-4df5-8fb4-bf1a63b37a21"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 18 14:12:52.622193 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:12:52.622169    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4b34a994-8cfb-43f2-8c04-e57dbef010a8-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "4b34a994-8cfb-43f2-8c04-e57dbef010a8" (UID: "4b34a994-8cfb-43f2-8c04-e57dbef010a8"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 18 14:12:52.623869 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:12:52.623848    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4d819468-a5ee-4df5-8fb4-bf1a63b37a21-kube-api-access-56t8w" (OuterVolumeSpecName: "kube-api-access-56t8w") pod "4d819468-a5ee-4df5-8fb4-bf1a63b37a21" (UID: "4d819468-a5ee-4df5-8fb4-bf1a63b37a21"). InnerVolumeSpecName "kube-api-access-56t8w". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 18 14:12:52.623937 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:12:52.623865    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4b34a994-8cfb-43f2-8c04-e57dbef010a8-kube-api-access-qwzms" (OuterVolumeSpecName: "kube-api-access-qwzms") pod "4b34a994-8cfb-43f2-8c04-e57dbef010a8" (UID: "4b34a994-8cfb-43f2-8c04-e57dbef010a8"). InnerVolumeSpecName "kube-api-access-qwzms". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 18 14:12:52.722932 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:12:52.722898    2565 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-56t8w\" (UniqueName: \"kubernetes.io/projected/4d819468-a5ee-4df5-8fb4-bf1a63b37a21-kube-api-access-56t8w\") on node \"ip-10-0-129-16.ec2.internal\" DevicePath \"\""
Sep 18 14:12:52.722932 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:12:52.722929    2565 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/4b34a994-8cfb-43f2-8c04-e57dbef010a8-plugins-config-volume\") on node \"ip-10-0-129-16.ec2.internal\" DevicePath \"\""
Sep 18 14:12:52.722932 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:12:52.722938    2565 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qwzms\" (UniqueName: \"kubernetes.io/projected/4b34a994-8cfb-43f2-8c04-e57dbef010a8-kube-api-access-qwzms\") on node \"ip-10-0-129-16.ec2.internal\" DevicePath \"\""
Sep 18 14:12:52.723220 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:12:52.722946    2565 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/4d819468-a5ee-4df5-8fb4-bf1a63b37a21-plugins-config-volume\") on node \"ip-10-0-129-16.ec2.internal\" DevicePath \"\""
Sep 18 14:12:53.322404 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:12:53.322367    2565 generic.go:356] "Generic (PLEG): container finished" podID="4b34a994-8cfb-43f2-8c04-e57dbef010a8" containerID="7dbc34cff3dcd147fd96ccaf288e0f3773eaf76530bc908c6001ee4c6c9f1691" exitCode=0
Sep 18 14:12:53.322875 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:12:53.322442    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-migrate-5f7e430d-67b67c94f4-vlwmn"
Sep 18 14:12:53.322875 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:12:53.322453    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-migrate-5f7e430d-67b67c94f4-vlwmn" event={"ID":"4b34a994-8cfb-43f2-8c04-e57dbef010a8","Type":"ContainerDied","Data":"7dbc34cff3dcd147fd96ccaf288e0f3773eaf76530bc908c6001ee4c6c9f1691"}
Sep 18 14:12:53.322875 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:12:53.322510    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-migrate-5f7e430d-67b67c94f4-vlwmn" event={"ID":"4b34a994-8cfb-43f2-8c04-e57dbef010a8","Type":"ContainerDied","Data":"cbc00d0f6415c11db3ed2774fd363042fe26cba8dd922b9dee6d74a32dd0c410"}
Sep 18 14:12:53.322875 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:12:53.322533    2565 scope.go:117] "RemoveContainer" containerID="7dbc34cff3dcd147fd96ccaf288e0f3773eaf76530bc908c6001ee4c6c9f1691"
Sep 18 14:12:53.324341 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:12:53.324078    2565 generic.go:356] "Generic (PLEG): container finished" podID="4d819468-a5ee-4df5-8fb4-bf1a63b37a21" containerID="2d756864f75736f49c81f566a2398a3ccda32c0185439f385e80ba34c296a031" exitCode=0
Sep 18 14:12:53.324341 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:12:53.324176    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-migrate-5f7e430d-96fbcbdd6-2qm99" event={"ID":"4d819468-a5ee-4df5-8fb4-bf1a63b37a21","Type":"ContainerDied","Data":"2d756864f75736f49c81f566a2398a3ccda32c0185439f385e80ba34c296a031"}
Sep 18 14:12:53.324341 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:12:53.324220    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-migrate-5f7e430d-96fbcbdd6-2qm99" event={"ID":"4d819468-a5ee-4df5-8fb4-bf1a63b37a21","Type":"ContainerDied","Data":"af3a02b34f9914d0ab3691e60fadaf7443464ef4f77f7f56951ee37ceea5d8cf"}
Sep 18 14:12:53.324341 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:12:53.324287    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-migrate-5f7e430d-96fbcbdd6-2qm99"
Sep 18 14:12:53.336386 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:12:53.336356    2565 scope.go:117] "RemoveContainer" containerID="7dbc34cff3dcd147fd96ccaf288e0f3773eaf76530bc908c6001ee4c6c9f1691"
Sep 18 14:12:53.336996 ip-10-0-129-16 kubenswrapper[2565]: E0918 14:12:53.336965    2565 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7dbc34cff3dcd147fd96ccaf288e0f3773eaf76530bc908c6001ee4c6c9f1691\": container with ID starting with 7dbc34cff3dcd147fd96ccaf288e0f3773eaf76530bc908c6001ee4c6c9f1691 not found: ID does not exist" containerID="7dbc34cff3dcd147fd96ccaf288e0f3773eaf76530bc908c6001ee4c6c9f1691"
Sep 18 14:12:53.337160 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:12:53.337003    2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7dbc34cff3dcd147fd96ccaf288e0f3773eaf76530bc908c6001ee4c6c9f1691"} err="failed to get container status \"7dbc34cff3dcd147fd96ccaf288e0f3773eaf76530bc908c6001ee4c6c9f1691\": rpc error: code = NotFound desc = could not find container \"7dbc34cff3dcd147fd96ccaf288e0f3773eaf76530bc908c6001ee4c6c9f1691\": container with ID starting with 7dbc34cff3dcd147fd96ccaf288e0f3773eaf76530bc908c6001ee4c6c9f1691 not found: ID does not exist"
Sep 18 14:12:53.337160 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:12:53.337051    2565 scope.go:117] "RemoveContainer" containerID="2d756864f75736f49c81f566a2398a3ccda32c0185439f385e80ba34c296a031"
Sep 18 14:12:53.347462 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:12:53.347438    2565 scope.go:117] "RemoveContainer" containerID="2d756864f75736f49c81f566a2398a3ccda32c0185439f385e80ba34c296a031"
Sep 18 14:12:53.347763 ip-10-0-129-16 kubenswrapper[2565]: E0918 14:12:53.347743    2565 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2d756864f75736f49c81f566a2398a3ccda32c0185439f385e80ba34c296a031\": container with ID starting with 2d756864f75736f49c81f566a2398a3ccda32c0185439f385e80ba34c296a031 not found: ID does not exist" containerID="2d756864f75736f49c81f566a2398a3ccda32c0185439f385e80ba34c296a031"
Sep 18 14:12:53.347817 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:12:53.347769    2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2d756864f75736f49c81f566a2398a3ccda32c0185439f385e80ba34c296a031"} err="failed to get container status \"2d756864f75736f49c81f566a2398a3ccda32c0185439f385e80ba34c296a031\": rpc error: code = NotFound desc = could not find container \"2d756864f75736f49c81f566a2398a3ccda32c0185439f385e80ba34c296a031\": container with ID starting with 2d756864f75736f49c81f566a2398a3ccda32c0185439f385e80ba34c296a031 not found: ID does not exist"
Sep 18 14:12:53.360092 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:12:53.358068    2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-migrate-5f7e430d-67b67c94f4-vlwmn"]
Sep 18 14:12:53.368103 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:12:53.368063    2565 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-migrate-5f7e430d-67b67c94f4-vlwmn"]
Sep 18 14:12:53.388633 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:12:53.388589    2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-migrate-5f7e430d-96fbcbdd6-2qm99"]
Sep 18 14:12:53.390796 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:12:53.390769    2565 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-migrate-5f7e430d-96fbcbdd6-2qm99"]
Sep 18 14:12:53.915320 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:12:53.915151    2565 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4b34a994-8cfb-43f2-8c04-e57dbef010a8" path="/var/lib/kubelet/pods/4b34a994-8cfb-43f2-8c04-e57dbef010a8/volumes"
Sep 18 14:12:53.915725 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:12:53.915704    2565 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4d819468-a5ee-4df5-8fb4-bf1a63b37a21" path="/var/lib/kubelet/pods/4d819468-a5ee-4df5-8fb4-bf1a63b37a21/volumes"
Sep 18 14:12:56.242266 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:12:56.242229    2565 patch_prober.go:28] interesting pod/e2e-migrate-5f7e430d-gw-openshift-default-56c9bfbb8-ghlx9 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Sep 18 14:12:56.242633 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:12:56.242283    2565 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-migrate-5f7e430d-gw-openshift-default-56c9bfbb8-ghlx9" podUID="2e89e3fa-fe49-4ddf-8cab-dee2fc29e5dd" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Sep 18 14:12:57.838001 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:12:57.837934    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-migrate-5f7e430d-gw-openshift-default-56c9bfbb8-ghlx9"
Sep 18 14:12:57.980765 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:12:57.980732    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2e89e3fa-fe49-4ddf-8cab-dee2fc29e5dd-istio-data\") pod \"2e89e3fa-fe49-4ddf-8cab-dee2fc29e5dd\" (UID: \"2e89e3fa-fe49-4ddf-8cab-dee2fc29e5dd\") "
Sep 18 14:12:57.980765 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:12:57.980768    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2e89e3fa-fe49-4ddf-8cab-dee2fc29e5dd-workload-socket\") pod \"2e89e3fa-fe49-4ddf-8cab-dee2fc29e5dd\" (UID: \"2e89e3fa-fe49-4ddf-8cab-dee2fc29e5dd\") "
Sep 18 14:12:57.981005 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:12:57.980785    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2e89e3fa-fe49-4ddf-8cab-dee2fc29e5dd-istio-podinfo\") pod \"2e89e3fa-fe49-4ddf-8cab-dee2fc29e5dd\" (UID: \"2e89e3fa-fe49-4ddf-8cab-dee2fc29e5dd\") "
Sep 18 14:12:57.981005 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:12:57.980843    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2e89e3fa-fe49-4ddf-8cab-dee2fc29e5dd-credential-socket\") pod \"2e89e3fa-fe49-4ddf-8cab-dee2fc29e5dd\" (UID: \"2e89e3fa-fe49-4ddf-8cab-dee2fc29e5dd\") "
Sep 18 14:12:57.981005 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:12:57.980873    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2e89e3fa-fe49-4ddf-8cab-dee2fc29e5dd-istio-envoy\") pod \"2e89e3fa-fe49-4ddf-8cab-dee2fc29e5dd\" (UID: \"2e89e3fa-fe49-4ddf-8cab-dee2fc29e5dd\") "
Sep 18 14:12:57.981005 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:12:57.980923    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2e89e3fa-fe49-4ddf-8cab-dee2fc29e5dd-istiod-ca-cert\") pod \"2e89e3fa-fe49-4ddf-8cab-dee2fc29e5dd\" (UID: \"2e89e3fa-fe49-4ddf-8cab-dee2fc29e5dd\") "
Sep 18 14:12:57.981005 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:12:57.980958    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-z9m97\" (UniqueName: \"kubernetes.io/projected/2e89e3fa-fe49-4ddf-8cab-dee2fc29e5dd-kube-api-access-z9m97\") pod \"2e89e3fa-fe49-4ddf-8cab-dee2fc29e5dd\" (UID: \"2e89e3fa-fe49-4ddf-8cab-dee2fc29e5dd\") "
Sep 18 14:12:57.981310 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:12:57.981085    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2e89e3fa-fe49-4ddf-8cab-dee2fc29e5dd-istio-data" (OuterVolumeSpecName: "istio-data") pod "2e89e3fa-fe49-4ddf-8cab-dee2fc29e5dd" (UID: "2e89e3fa-fe49-4ddf-8cab-dee2fc29e5dd"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 18 14:12:57.981310 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:12:57.981119    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2e89e3fa-fe49-4ddf-8cab-dee2fc29e5dd-istio-token\") pod \"2e89e3fa-fe49-4ddf-8cab-dee2fc29e5dd\" (UID: \"2e89e3fa-fe49-4ddf-8cab-dee2fc29e5dd\") "
Sep 18 14:12:57.981310 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:12:57.981121    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2e89e3fa-fe49-4ddf-8cab-dee2fc29e5dd-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "2e89e3fa-fe49-4ddf-8cab-dee2fc29e5dd" (UID: "2e89e3fa-fe49-4ddf-8cab-dee2fc29e5dd"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 18 14:12:57.981310 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:12:57.981148    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2e89e3fa-fe49-4ddf-8cab-dee2fc29e5dd-workload-certs\") pod \"2e89e3fa-fe49-4ddf-8cab-dee2fc29e5dd\" (UID: \"2e89e3fa-fe49-4ddf-8cab-dee2fc29e5dd\") "
Sep 18 14:12:57.981310 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:12:57.981206    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2e89e3fa-fe49-4ddf-8cab-dee2fc29e5dd-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "2e89e3fa-fe49-4ddf-8cab-dee2fc29e5dd" (UID: "2e89e3fa-fe49-4ddf-8cab-dee2fc29e5dd"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 18 14:12:57.981576 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:12:57.981446    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2e89e3fa-fe49-4ddf-8cab-dee2fc29e5dd-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "2e89e3fa-fe49-4ddf-8cab-dee2fc29e5dd" (UID: "2e89e3fa-fe49-4ddf-8cab-dee2fc29e5dd"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 18 14:12:57.981576 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:12:57.981451    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2e89e3fa-fe49-4ddf-8cab-dee2fc29e5dd-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "2e89e3fa-fe49-4ddf-8cab-dee2fc29e5dd" (UID: "2e89e3fa-fe49-4ddf-8cab-dee2fc29e5dd"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 18 14:12:57.981686 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:12:57.981631    2565 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2e89e3fa-fe49-4ddf-8cab-dee2fc29e5dd-istio-data\") on node \"ip-10-0-129-16.ec2.internal\" DevicePath \"\""
Sep 18 14:12:57.981686 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:12:57.981652    2565 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2e89e3fa-fe49-4ddf-8cab-dee2fc29e5dd-workload-socket\") on node \"ip-10-0-129-16.ec2.internal\" DevicePath \"\""
Sep 18 14:12:57.981686 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:12:57.981665    2565 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2e89e3fa-fe49-4ddf-8cab-dee2fc29e5dd-credential-socket\") on node \"ip-10-0-129-16.ec2.internal\" DevicePath \"\""
Sep 18 14:12:57.981686 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:12:57.981674    2565 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2e89e3fa-fe49-4ddf-8cab-dee2fc29e5dd-istiod-ca-cert\") on node \"ip-10-0-129-16.ec2.internal\" DevicePath \"\""
Sep 18 14:12:57.981686 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:12:57.981682    2565 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2e89e3fa-fe49-4ddf-8cab-dee2fc29e5dd-workload-certs\") on node \"ip-10-0-129-16.ec2.internal\" DevicePath \"\""
Sep 18 14:12:57.983289 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:12:57.983260    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2e89e3fa-fe49-4ddf-8cab-dee2fc29e5dd-istio-token" (OuterVolumeSpecName: "istio-token") pod "2e89e3fa-fe49-4ddf-8cab-dee2fc29e5dd" (UID: "2e89e3fa-fe49-4ddf-8cab-dee2fc29e5dd"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 18 14:12:57.983430 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:12:57.983385    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2e89e3fa-fe49-4ddf-8cab-dee2fc29e5dd-kube-api-access-z9m97" (OuterVolumeSpecName: "kube-api-access-z9m97") pod "2e89e3fa-fe49-4ddf-8cab-dee2fc29e5dd" (UID: "2e89e3fa-fe49-4ddf-8cab-dee2fc29e5dd"). InnerVolumeSpecName "kube-api-access-z9m97". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 18 14:12:57.983660 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:12:57.983637    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/2e89e3fa-fe49-4ddf-8cab-dee2fc29e5dd-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "2e89e3fa-fe49-4ddf-8cab-dee2fc29e5dd" (UID: "2e89e3fa-fe49-4ddf-8cab-dee2fc29e5dd"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 18 14:12:57.983750 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:12:57.983666    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2e89e3fa-fe49-4ddf-8cab-dee2fc29e5dd-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "2e89e3fa-fe49-4ddf-8cab-dee2fc29e5dd" (UID: "2e89e3fa-fe49-4ddf-8cab-dee2fc29e5dd"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 18 14:12:58.082539 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:12:58.082429    2565 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2e89e3fa-fe49-4ddf-8cab-dee2fc29e5dd-istio-podinfo\") on node \"ip-10-0-129-16.ec2.internal\" DevicePath \"\""
Sep 18 14:12:58.082539 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:12:58.082462    2565 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2e89e3fa-fe49-4ddf-8cab-dee2fc29e5dd-istio-envoy\") on node \"ip-10-0-129-16.ec2.internal\" DevicePath \"\""
Sep 18 14:12:58.082539 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:12:58.082476    2565 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-z9m97\" (UniqueName: \"kubernetes.io/projected/2e89e3fa-fe49-4ddf-8cab-dee2fc29e5dd-kube-api-access-z9m97\") on node \"ip-10-0-129-16.ec2.internal\" DevicePath \"\""
Sep 18 14:12:58.082539 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:12:58.082492    2565 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2e89e3fa-fe49-4ddf-8cab-dee2fc29e5dd-istio-token\") on node \"ip-10-0-129-16.ec2.internal\" DevicePath \"\""
Sep 18 14:12:58.358630 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:12:58.358593    2565 generic.go:356] "Generic (PLEG): container finished" podID="2e89e3fa-fe49-4ddf-8cab-dee2fc29e5dd" containerID="bc05e14195fbfbda2bfd9fd7901d3093b46b99fadd7ec9bdd5740847735aa92a" exitCode=0
Sep 18 14:12:58.358833 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:12:58.358666    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-migrate-5f7e430d-gw-openshift-default-56c9bfbb8-ghlx9"
Sep 18 14:12:58.358833 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:12:58.358673    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-5f7e430d-gw-openshift-default-56c9bfbb8-ghlx9" event={"ID":"2e89e3fa-fe49-4ddf-8cab-dee2fc29e5dd","Type":"ContainerDied","Data":"bc05e14195fbfbda2bfd9fd7901d3093b46b99fadd7ec9bdd5740847735aa92a"}
Sep 18 14:12:58.358833 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:12:58.358730    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-5f7e430d-gw-openshift-default-56c9bfbb8-ghlx9" event={"ID":"2e89e3fa-fe49-4ddf-8cab-dee2fc29e5dd","Type":"ContainerDied","Data":"6dd6cb22fb6281af8e51d4bcbf068276cfca3f61cf8fa9f926ad88ea3c38cfe3"}
Sep 18 14:12:58.358833 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:12:58.358763    2565 scope.go:117] "RemoveContainer" containerID="bc05e14195fbfbda2bfd9fd7901d3093b46b99fadd7ec9bdd5740847735aa92a"
Sep 18 14:12:58.372120 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:12:58.372099    2565 scope.go:117] "RemoveContainer" containerID="82a16a09bb466b390f48e3635757cb4fa9f107e66d33555e07aa5072f2e0338f"
Sep 18 14:12:58.382247 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:12:58.382228    2565 scope.go:117] "RemoveContainer" containerID="bc05e14195fbfbda2bfd9fd7901d3093b46b99fadd7ec9bdd5740847735aa92a"
Sep 18 14:12:58.382546 ip-10-0-129-16 kubenswrapper[2565]: E0918 14:12:58.382525    2565 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"bc05e14195fbfbda2bfd9fd7901d3093b46b99fadd7ec9bdd5740847735aa92a\": container with ID starting with bc05e14195fbfbda2bfd9fd7901d3093b46b99fadd7ec9bdd5740847735aa92a not found: ID does not exist" containerID="bc05e14195fbfbda2bfd9fd7901d3093b46b99fadd7ec9bdd5740847735aa92a"
Sep 18 14:12:58.382620 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:12:58.382553    2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"bc05e14195fbfbda2bfd9fd7901d3093b46b99fadd7ec9bdd5740847735aa92a"} err="failed to get container status \"bc05e14195fbfbda2bfd9fd7901d3093b46b99fadd7ec9bdd5740847735aa92a\": rpc error: code = NotFound desc = could not find container \"bc05e14195fbfbda2bfd9fd7901d3093b46b99fadd7ec9bdd5740847735aa92a\": container with ID starting with bc05e14195fbfbda2bfd9fd7901d3093b46b99fadd7ec9bdd5740847735aa92a not found: ID does not exist"
Sep 18 14:12:58.382620 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:12:58.382570    2565 scope.go:117] "RemoveContainer" containerID="82a16a09bb466b390f48e3635757cb4fa9f107e66d33555e07aa5072f2e0338f"
Sep 18 14:12:58.382857 ip-10-0-129-16 kubenswrapper[2565]: E0918 14:12:58.382841    2565 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"82a16a09bb466b390f48e3635757cb4fa9f107e66d33555e07aa5072f2e0338f\": container with ID starting with 82a16a09bb466b390f48e3635757cb4fa9f107e66d33555e07aa5072f2e0338f not found: ID does not exist" containerID="82a16a09bb466b390f48e3635757cb4fa9f107e66d33555e07aa5072f2e0338f"
Sep 18 14:12:58.382902 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:12:58.382863    2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"82a16a09bb466b390f48e3635757cb4fa9f107e66d33555e07aa5072f2e0338f"} err="failed to get container status \"82a16a09bb466b390f48e3635757cb4fa9f107e66d33555e07aa5072f2e0338f\": rpc error: code = NotFound desc = could not find container \"82a16a09bb466b390f48e3635757cb4fa9f107e66d33555e07aa5072f2e0338f\": container with ID starting with 82a16a09bb466b390f48e3635757cb4fa9f107e66d33555e07aa5072f2e0338f not found: ID does not exist"
Sep 18 14:12:58.399867 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:12:58.399839    2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-migrate-5f7e430d-gw-openshift-default-56c9bfbb8-ghlx9"]
Sep 18 14:12:58.411908 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:12:58.411882    2565 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-migrate-5f7e430d-gw-openshift-default-56c9bfbb8-ghlx9"]
Sep 18 14:12:59.146248 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:12:59.146214    2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-35b94378-5688bcc74b-dqxqz"]
Sep 18 14:12:59.146719 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:12:59.146706    2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4d819468-a5ee-4df5-8fb4-bf1a63b37a21" containerName="payload-processing"
Sep 18 14:12:59.146768 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:12:59.146720    2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="4d819468-a5ee-4df5-8fb4-bf1a63b37a21" containerName="payload-processing"
Sep 18 14:12:59.146768 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:12:59.146735    2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2e89e3fa-fe49-4ddf-8cab-dee2fc29e5dd" containerName="istio-proxy"
Sep 18 14:12:59.146768 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:12:59.146740    2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="2e89e3fa-fe49-4ddf-8cab-dee2fc29e5dd" containerName="istio-proxy"
Sep 18 14:12:59.146768 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:12:59.146763    2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4b34a994-8cfb-43f2-8c04-e57dbef010a8" containerName="payload-pre-processing"
Sep 18 14:12:59.146890 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:12:59.146770    2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="4b34a994-8cfb-43f2-8c04-e57dbef010a8" containerName="payload-pre-processing"
Sep 18 14:12:59.146890 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:12:59.146831    2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="2e89e3fa-fe49-4ddf-8cab-dee2fc29e5dd" containerName="istio-proxy"
Sep 18 14:12:59.146890 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:12:59.146844    2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="4d819468-a5ee-4df5-8fb4-bf1a63b37a21" containerName="payload-processing"
Sep 18 14:12:59.146890 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:12:59.146850    2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="4b34a994-8cfb-43f2-8c04-e57dbef010a8" containerName="payload-pre-processing"
Sep 18 14:12:59.150192 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:12:59.150175    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-35b94378-5688bcc74b-dqxqz"
Sep 18 14:12:59.154312 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:12:59.154292    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-ait-35b94378\""
Sep 18 14:12:59.154831 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:12:59.154816    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-ait-35b94378-dockercfg-pwvqf\""
Sep 18 14:12:59.172269 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:12:59.172237    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-35b94378-5688bcc74b-dqxqz"]
Sep 18 14:12:59.248756 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:12:59.248704    2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-35b94378-6cddc4947c-44bf7"]
Sep 18 14:12:59.249482 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:12:59.249455    2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2e89e3fa-fe49-4ddf-8cab-dee2fc29e5dd" containerName="istio-proxy"
Sep 18 14:12:59.249630 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:12:59.249497    2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="2e89e3fa-fe49-4ddf-8cab-dee2fc29e5dd" containerName="istio-proxy"
Sep 18 14:12:59.249732 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:12:59.249715    2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="2e89e3fa-fe49-4ddf-8cab-dee2fc29e5dd" containerName="istio-proxy"
Sep 18 14:12:59.252289 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:12:59.252261    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-35b94378-6cddc4947c-44bf7"
Sep 18 14:12:59.266445 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:12:59.265259    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-35b94378-6cddc4947c-44bf7"]
Sep 18 14:12:59.294267 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:12:59.294231    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/2448d2e1-10b4-4a3f-9237-c99f3701c786-plugins-config-volume\") pod \"payload-pre-processing-e2e-ait-35b94378-5688bcc74b-dqxqz\" (UID: \"2448d2e1-10b4-4a3f-9237-c99f3701c786\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-35b94378-5688bcc74b-dqxqz"
Sep 18 14:12:59.294437 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:12:59.294401    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-c98gl\" (UniqueName: \"kubernetes.io/projected/2448d2e1-10b4-4a3f-9237-c99f3701c786-kube-api-access-c98gl\") pod \"payload-pre-processing-e2e-ait-35b94378-5688bcc74b-dqxqz\" (UID: \"2448d2e1-10b4-4a3f-9237-c99f3701c786\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-35b94378-5688bcc74b-dqxqz"
Sep 18 14:12:59.395896 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:12:59.395861    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9slsg\" (UniqueName: \"kubernetes.io/projected/c73ad8f8-faff-44c3-b07b-5c888886409e-kube-api-access-9slsg\") pod \"payload-processing-e2e-ait-35b94378-6cddc4947c-44bf7\" (UID: \"c73ad8f8-faff-44c3-b07b-5c888886409e\") " pod="openshift-ingress/payload-processing-e2e-ait-35b94378-6cddc4947c-44bf7"
Sep 18 14:12:59.396133 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:12:59.395905    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/2448d2e1-10b4-4a3f-9237-c99f3701c786-plugins-config-volume\") pod \"payload-pre-processing-e2e-ait-35b94378-5688bcc74b-dqxqz\" (UID: \"2448d2e1-10b4-4a3f-9237-c99f3701c786\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-35b94378-5688bcc74b-dqxqz"
Sep 18 14:12:59.396133 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:12:59.395974    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c73ad8f8-faff-44c3-b07b-5c888886409e-plugins-config-volume\") pod \"payload-processing-e2e-ait-35b94378-6cddc4947c-44bf7\" (UID: \"c73ad8f8-faff-44c3-b07b-5c888886409e\") " pod="openshift-ingress/payload-processing-e2e-ait-35b94378-6cddc4947c-44bf7"
Sep 18 14:12:59.396133 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:12:59.396068    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-c98gl\" (UniqueName: \"kubernetes.io/projected/2448d2e1-10b4-4a3f-9237-c99f3701c786-kube-api-access-c98gl\") pod \"payload-pre-processing-e2e-ait-35b94378-5688bcc74b-dqxqz\" (UID: \"2448d2e1-10b4-4a3f-9237-c99f3701c786\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-35b94378-5688bcc74b-dqxqz"
Sep 18 14:12:59.396669 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:12:59.396617    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/2448d2e1-10b4-4a3f-9237-c99f3701c786-plugins-config-volume\") pod \"payload-pre-processing-e2e-ait-35b94378-5688bcc74b-dqxqz\" (UID: \"2448d2e1-10b4-4a3f-9237-c99f3701c786\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-35b94378-5688bcc74b-dqxqz"
Sep 18 14:12:59.423604 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:12:59.423571    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-c98gl\" (UniqueName: \"kubernetes.io/projected/2448d2e1-10b4-4a3f-9237-c99f3701c786-kube-api-access-c98gl\") pod \"payload-pre-processing-e2e-ait-35b94378-5688bcc74b-dqxqz\" (UID: \"2448d2e1-10b4-4a3f-9237-c99f3701c786\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-35b94378-5688bcc74b-dqxqz"
Sep 18 14:12:59.460266 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:12:59.460230    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-35b94378-5688bcc74b-dqxqz"
Sep 18 14:12:59.497553 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:12:59.497502    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c73ad8f8-faff-44c3-b07b-5c888886409e-plugins-config-volume\") pod \"payload-processing-e2e-ait-35b94378-6cddc4947c-44bf7\" (UID: \"c73ad8f8-faff-44c3-b07b-5c888886409e\") " pod="openshift-ingress/payload-processing-e2e-ait-35b94378-6cddc4947c-44bf7"
Sep 18 14:12:59.497732 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:12:59.497686    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9slsg\" (UniqueName: \"kubernetes.io/projected/c73ad8f8-faff-44c3-b07b-5c888886409e-kube-api-access-9slsg\") pod \"payload-processing-e2e-ait-35b94378-6cddc4947c-44bf7\" (UID: \"c73ad8f8-faff-44c3-b07b-5c888886409e\") " pod="openshift-ingress/payload-processing-e2e-ait-35b94378-6cddc4947c-44bf7"
Sep 18 14:12:59.498305 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:12:59.498280    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c73ad8f8-faff-44c3-b07b-5c888886409e-plugins-config-volume\") pod \"payload-processing-e2e-ait-35b94378-6cddc4947c-44bf7\" (UID: \"c73ad8f8-faff-44c3-b07b-5c888886409e\") " pod="openshift-ingress/payload-processing-e2e-ait-35b94378-6cddc4947c-44bf7"
Sep 18 14:12:59.515507 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:12:59.515419    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9slsg\" (UniqueName: \"kubernetes.io/projected/c73ad8f8-faff-44c3-b07b-5c888886409e-kube-api-access-9slsg\") pod \"payload-processing-e2e-ait-35b94378-6cddc4947c-44bf7\" (UID: \"c73ad8f8-faff-44c3-b07b-5c888886409e\") " pod="openshift-ingress/payload-processing-e2e-ait-35b94378-6cddc4947c-44bf7"
Sep 18 14:12:59.570134 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:12:59.570106    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-35b94378-6cddc4947c-44bf7"
Sep 18 14:12:59.739837 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:12:59.739806    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-35b94378-5688bcc74b-dqxqz"]
Sep 18 14:12:59.741385 ip-10-0-129-16 kubenswrapper[2565]: W0918 14:12:59.741361    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2448d2e1_10b4_4a3f_9237_c99f3701c786.slice/crio-9e9c1d66dae424656edc095285f9d413df40fbb69fcae125221f657293069f4b WatchSource:0}: Error finding container 9e9c1d66dae424656edc095285f9d413df40fbb69fcae125221f657293069f4b: Status 404 returned error can't find the container with id 9e9c1d66dae424656edc095285f9d413df40fbb69fcae125221f657293069f4b
Sep 18 14:12:59.845793 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:12:59.845761    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-35b94378-6cddc4947c-44bf7"]
Sep 18 14:12:59.847497 ip-10-0-129-16 kubenswrapper[2565]: W0918 14:12:59.847467    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc73ad8f8_faff_44c3_b07b_5c888886409e.slice/crio-d1100050f4aaa6cf725aae45fb32c6f19d6e29e8a97af1ba6b2685e3a774c4ff WatchSource:0}: Error finding container d1100050f4aaa6cf725aae45fb32c6f19d6e29e8a97af1ba6b2685e3a774c4ff: Status 404 returned error can't find the container with id d1100050f4aaa6cf725aae45fb32c6f19d6e29e8a97af1ba6b2685e3a774c4ff
Sep 18 14:12:59.913820 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:12:59.913791    2565 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2e89e3fa-fe49-4ddf-8cab-dee2fc29e5dd" path="/var/lib/kubelet/pods/2e89e3fa-fe49-4ddf-8cab-dee2fc29e5dd/volumes"
Sep 18 14:13:00.374286 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:13:00.374249    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-35b94378-6cddc4947c-44bf7" event={"ID":"c73ad8f8-faff-44c3-b07b-5c888886409e","Type":"ContainerStarted","Data":"3f50df8613897a99079557c678496e71de53468768f4542c349afbebad2d83e8"}
Sep 18 14:13:00.374776 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:13:00.374315    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-ait-35b94378-6cddc4947c-44bf7"
Sep 18 14:13:00.374776 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:13:00.374341    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-35b94378-6cddc4947c-44bf7" event={"ID":"c73ad8f8-faff-44c3-b07b-5c888886409e","Type":"ContainerStarted","Data":"d1100050f4aaa6cf725aae45fb32c6f19d6e29e8a97af1ba6b2685e3a774c4ff"}
Sep 18 14:13:00.376216 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:13:00.376186    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-35b94378-5688bcc74b-dqxqz" event={"ID":"2448d2e1-10b4-4a3f-9237-c99f3701c786","Type":"ContainerStarted","Data":"95e5e558861132a13b553c5802ffd8310b85d4c54cb093a0cb2c09acaeca53e6"}
Sep 18 14:13:00.376328 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:13:00.376241    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-35b94378-5688bcc74b-dqxqz" event={"ID":"2448d2e1-10b4-4a3f-9237-c99f3701c786","Type":"ContainerStarted","Data":"9e9c1d66dae424656edc095285f9d413df40fbb69fcae125221f657293069f4b"}
Sep 18 14:13:00.376387 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:13:00.376357    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-ait-35b94378-5688bcc74b-dqxqz"
Sep 18 14:13:00.392769 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:13:00.392726    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-ait-35b94378-6cddc4947c-44bf7" podStartSLOduration=1.392713188 podStartE2EDuration="1.392713188s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 14:12:59 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 14:13:00.391074913 +0000 UTC m=+848.564984596" watchObservedRunningTime="2026-09-18 14:13:00.392713188 +0000 UTC m=+848.566622847"
Sep 18 14:13:00.409967 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:13:00.409921    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-ait-35b94378-5688bcc74b-dqxqz" podStartSLOduration=1.4099058119999999 podStartE2EDuration="1.409905812s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 14:12:59 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 14:13:00.406819507 +0000 UTC m=+848.580729184" watchObservedRunningTime="2026-09-18 14:13:00.409905812 +0000 UTC m=+848.583815471"
Sep 18 14:13:08.060882 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:13:08.060841    2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-autores-ae7021-bd2hs"]
Sep 18 14:13:08.063764 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:13:08.063740    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-autores-ae7021-bd2hs"
Sep 18 14:13:08.066542 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:13:08.066518    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-l4fn9\""
Sep 18 14:13:08.073118 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:13:08.073091    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-autores-ae7021-bd2hs"]
Sep 18 14:13:08.086559 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:13:08.086527    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/5bbaeb5d-af08-4177-8762-f25afa54dccd-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-autores-ae7021-bd2hs\" (UID: \"5bbaeb5d-af08-4177-8762-f25afa54dccd\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-autores-ae7021-bd2hs"
Sep 18 14:13:08.187216 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:13:08.187186    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/5bbaeb5d-af08-4177-8762-f25afa54dccd-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-autores-ae7021-bd2hs\" (UID: \"5bbaeb5d-af08-4177-8762-f25afa54dccd\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-autores-ae7021-bd2hs"
Sep 18 14:13:08.187866 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:13:08.187838    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/5bbaeb5d-af08-4177-8762-f25afa54dccd-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-autores-ae7021-bd2hs\" (UID: \"5bbaeb5d-af08-4177-8762-f25afa54dccd\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-autores-ae7021-bd2hs"
Sep 18 14:13:08.378167 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:13:08.378079    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-autores-ae7021-bd2hs"
Sep 18 14:13:08.651127 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:13:08.650851    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-autores-ae7021-bd2hs"]
Sep 18 14:13:08.652679 ip-10-0-129-16 kubenswrapper[2565]: W0918 14:13:08.652650    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5bbaeb5d_af08_4177_8762_f25afa54dccd.slice/crio-fd482be1d8b7e91ba3c2680fc3f67d7370770ea2eae11baece9a6f43ed891020 WatchSource:0}: Error finding container fd482be1d8b7e91ba3c2680fc3f67d7370770ea2eae11baece9a6f43ed891020: Status 404 returned error can't find the container with id fd482be1d8b7e91ba3c2680fc3f67d7370770ea2eae11baece9a6f43ed891020
Sep 18 14:13:09.433411 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:13:09.433375    2565 generic.go:356] "Generic (PLEG): container finished" podID="5bbaeb5d-af08-4177-8762-f25afa54dccd" containerID="1868f276911a84dec91efcc375edcc1042fd3852709c3000ef0aaf8e13d346cb" exitCode=0
Sep 18 14:13:09.433782 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:13:09.433463    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-autores-ae7021-bd2hs" event={"ID":"5bbaeb5d-af08-4177-8762-f25afa54dccd","Type":"ContainerDied","Data":"1868f276911a84dec91efcc375edcc1042fd3852709c3000ef0aaf8e13d346cb"}
Sep 18 14:13:09.433782 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:13:09.433522    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-autores-ae7021-bd2hs" event={"ID":"5bbaeb5d-af08-4177-8762-f25afa54dccd","Type":"ContainerStarted","Data":"fd482be1d8b7e91ba3c2680fc3f67d7370770ea2eae11baece9a6f43ed891020"}
Sep 18 14:13:10.587863 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:13:10.587832    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-autores-ae7021-bd2hs"
Sep 18 14:13:10.611630 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:13:10.611600    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/5bbaeb5d-af08-4177-8762-f25afa54dccd-maas-api-service-ca\") pod \"5bbaeb5d-af08-4177-8762-f25afa54dccd\" (UID: \"5bbaeb5d-af08-4177-8762-f25afa54dccd\") "
Sep 18 14:13:10.611959 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:13:10.611936    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5bbaeb5d-af08-4177-8762-f25afa54dccd-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "5bbaeb5d-af08-4177-8762-f25afa54dccd" (UID: "5bbaeb5d-af08-4177-8762-f25afa54dccd"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 18 14:13:10.612099 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:13:10.612086    2565 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/5bbaeb5d-af08-4177-8762-f25afa54dccd-maas-api-service-ca\") on node \"ip-10-0-129-16.ec2.internal\" DevicePath \"\""
Sep 18 14:13:11.383229 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:13:11.383201    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-ait-35b94378-6cddc4947c-44bf7"
Sep 18 14:13:11.383509 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:13:11.383490    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-ait-35b94378-5688bcc74b-dqxqz"
Sep 18 14:13:11.450722 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:13:11.450694    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-autores-ae7021-bd2hs"
Sep 18 14:13:11.450722 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:13:11.450704    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-autores-ae7021-bd2hs" event={"ID":"5bbaeb5d-af08-4177-8762-f25afa54dccd","Type":"ContainerDied","Data":"fd482be1d8b7e91ba3c2680fc3f67d7370770ea2eae11baece9a6f43ed891020"}
Sep 18 14:13:11.450926 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:13:11.450752    2565 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="fd482be1d8b7e91ba3c2680fc3f67d7370770ea2eae11baece9a6f43ed891020"
Sep 18 14:13:16.555007 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:13:16.554973    2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-35b94378-8jn2g"]
Sep 18 14:13:16.555532 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:13:16.555517    2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5bbaeb5d-af08-4177-8762-f25afa54dccd" containerName="revoke-keys"
Sep 18 14:13:16.555532 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:13:16.555533    2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="5bbaeb5d-af08-4177-8762-f25afa54dccd" containerName="revoke-keys"
Sep 18 14:13:16.555628 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:13:16.555623    2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="5bbaeb5d-af08-4177-8762-f25afa54dccd" containerName="revoke-keys"
Sep 18 14:13:16.558061 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:13:16.558011    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-35b94378-8jn2g"
Sep 18 14:13:16.560896 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:13:16.560873    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-l4fn9\""
Sep 18 14:13:16.572879 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:13:16.572852    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-35b94378-8jn2g"]
Sep 18 14:13:16.673225 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:13:16.673192    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/6506201e-0d05-4393-8a16-e4c9ad014f1f-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-35b94378-8jn2g\" (UID: \"6506201e-0d05-4393-8a16-e4c9ad014f1f\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-35b94378-8jn2g"
Sep 18 14:13:16.774560 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:13:16.774525    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/6506201e-0d05-4393-8a16-e4c9ad014f1f-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-35b94378-8jn2g\" (UID: \"6506201e-0d05-4393-8a16-e4c9ad014f1f\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-35b94378-8jn2g"
Sep 18 14:13:16.775125 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:13:16.775106    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/6506201e-0d05-4393-8a16-e4c9ad014f1f-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-35b94378-8jn2g\" (UID: \"6506201e-0d05-4393-8a16-e4c9ad014f1f\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-35b94378-8jn2g"
Sep 18 14:13:16.877054 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:13:16.876945    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-35b94378-8jn2g"
Sep 18 14:13:17.148822 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:13:17.148790    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-35b94378-8jn2g"]
Sep 18 14:13:17.150702 ip-10-0-129-16 kubenswrapper[2565]: W0918 14:13:17.150671    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6506201e_0d05_4393_8a16_e4c9ad014f1f.slice/crio-d892612ed1b4452829640aa5538eb5a4bd2c4210aa48bbe07e63b9461438db38 WatchSource:0}: Error finding container d892612ed1b4452829640aa5538eb5a4bd2c4210aa48bbe07e63b9461438db38: Status 404 returned error can't find the container with id d892612ed1b4452829640aa5538eb5a4bd2c4210aa48bbe07e63b9461438db38
Sep 18 14:13:17.488276 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:13:17.488220    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-35b94378-8jn2g" event={"ID":"6506201e-0d05-4393-8a16-e4c9ad014f1f","Type":"ContainerStarted","Data":"8025c40be09f8d47e082416544e182bb0031bca6be01b9e6e89fbf45c7765c8b"}
Sep 18 14:13:17.488484 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:13:17.488317    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-35b94378-8jn2g" event={"ID":"6506201e-0d05-4393-8a16-e4c9ad014f1f","Type":"ContainerStarted","Data":"d892612ed1b4452829640aa5538eb5a4bd2c4210aa48bbe07e63b9461438db38"}
Sep 18 14:13:17.506201 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:13:17.506151    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-35b94378-8jn2g" podStartSLOduration=1.506130786 podStartE2EDuration="1.506130786s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 14:13:16 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 14:13:17.505123164 +0000 UTC m=+865.679032850" watchObservedRunningTime="2026-09-18 14:13:17.506130786 +0000 UTC m=+865.680040447"
Sep 18 14:13:18.495440 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:13:18.495401    2565 generic.go:356] "Generic (PLEG): container finished" podID="6506201e-0d05-4393-8a16-e4c9ad014f1f" containerID="8025c40be09f8d47e082416544e182bb0031bca6be01b9e6e89fbf45c7765c8b" exitCode=0
Sep 18 14:13:18.495819 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:13:18.495486    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-35b94378-8jn2g" event={"ID":"6506201e-0d05-4393-8a16-e4c9ad014f1f","Type":"ContainerDied","Data":"8025c40be09f8d47e082416544e182bb0031bca6be01b9e6e89fbf45c7765c8b"}
Sep 18 14:13:19.679112 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:13:19.679087    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-35b94378-8jn2g"
Sep 18 14:13:19.785085 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:13:19.784968    2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-autores-ae7021-bd2hs"]
Sep 18 14:13:19.788177 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:13:19.788150    2565 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-autores-ae7021-bd2hs"]
Sep 18 14:13:19.808568 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:13:19.808541    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/6506201e-0d05-4393-8a16-e4c9ad014f1f-maas-api-service-ca\") pod \"6506201e-0d05-4393-8a16-e4c9ad014f1f\" (UID: \"6506201e-0d05-4393-8a16-e4c9ad014f1f\") "
Sep 18 14:13:19.808942 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:13:19.808919    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6506201e-0d05-4393-8a16-e4c9ad014f1f-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "6506201e-0d05-4393-8a16-e4c9ad014f1f" (UID: "6506201e-0d05-4393-8a16-e4c9ad014f1f"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 18 14:13:19.909311 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:13:19.909269    2565 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/6506201e-0d05-4393-8a16-e4c9ad014f1f-maas-api-service-ca\") on node \"ip-10-0-129-16.ec2.internal\" DevicePath \"\""
Sep 18 14:13:19.913469 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:13:19.913441    2565 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5bbaeb5d-af08-4177-8762-f25afa54dccd" path="/var/lib/kubelet/pods/5bbaeb5d-af08-4177-8762-f25afa54dccd/volumes"
Sep 18 14:13:20.512183 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:13:20.512150    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-35b94378-8jn2g"
Sep 18 14:13:20.512336 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:13:20.512178    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-35b94378-8jn2g" event={"ID":"6506201e-0d05-4393-8a16-e4c9ad014f1f","Type":"ContainerDied","Data":"d892612ed1b4452829640aa5538eb5a4bd2c4210aa48bbe07e63b9461438db38"}
Sep 18 14:13:20.512336 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:13:20.512229    2565 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="d892612ed1b4452829640aa5538eb5a4bd2c4210aa48bbe07e63b9461438db38"
Sep 18 14:13:27.112201 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:13:27.112162    2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-35b94378-6cddc4947c-44bf7"]
Sep 18 14:13:27.112697 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:13:27.112398    2565 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-ait-35b94378-6cddc4947c-44bf7" podUID="c73ad8f8-faff-44c3-b07b-5c888886409e" containerName="payload-processing" containerID="cri-o://3f50df8613897a99079557c678496e71de53468768f4542c349afbebad2d83e8" gracePeriod=30
Sep 18 14:13:27.119998 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:13:27.119964    2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-35b94378-5688bcc74b-dqxqz"]
Sep 18 14:13:27.120877 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:13:27.120253    2565 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-ait-35b94378-5688bcc74b-dqxqz" podUID="2448d2e1-10b4-4a3f-9237-c99f3701c786" containerName="payload-pre-processing" containerID="cri-o://95e5e558861132a13b553c5802ffd8310b85d4c54cb093a0cb2c09acaeca53e6" gracePeriod=30
Sep 18 14:13:27.324103 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:13:27.324065    2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-35b94378-8jn2g"]
Sep 18 14:13:27.331392 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:13:27.328840    2565 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-35b94378-8jn2g"]
Sep 18 14:13:27.432963 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:13:27.432939    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-35b94378-6cddc4947c-44bf7"
Sep 18 14:13:27.436209 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:13:27.436188    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-35b94378-5688bcc74b-dqxqz"
Sep 18 14:13:27.561988 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:13:27.561953    2565 generic.go:356] "Generic (PLEG): container finished" podID="2448d2e1-10b4-4a3f-9237-c99f3701c786" containerID="95e5e558861132a13b553c5802ffd8310b85d4c54cb093a0cb2c09acaeca53e6" exitCode=0
Sep 18 14:13:27.562179 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:13:27.561999    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-35b94378-5688bcc74b-dqxqz"
Sep 18 14:13:27.562179 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:13:27.562080    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-35b94378-5688bcc74b-dqxqz" event={"ID":"2448d2e1-10b4-4a3f-9237-c99f3701c786","Type":"ContainerDied","Data":"95e5e558861132a13b553c5802ffd8310b85d4c54cb093a0cb2c09acaeca53e6"}
Sep 18 14:13:27.562179 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:13:27.562137    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-35b94378-5688bcc74b-dqxqz" event={"ID":"2448d2e1-10b4-4a3f-9237-c99f3701c786","Type":"ContainerDied","Data":"9e9c1d66dae424656edc095285f9d413df40fbb69fcae125221f657293069f4b"}
Sep 18 14:13:27.562179 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:13:27.562161    2565 scope.go:117] "RemoveContainer" containerID="95e5e558861132a13b553c5802ffd8310b85d4c54cb093a0cb2c09acaeca53e6"
Sep 18 14:13:27.563675 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:13:27.563655    2565 generic.go:356] "Generic (PLEG): container finished" podID="c73ad8f8-faff-44c3-b07b-5c888886409e" containerID="3f50df8613897a99079557c678496e71de53468768f4542c349afbebad2d83e8" exitCode=0
Sep 18 14:13:27.563781 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:13:27.563698    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-35b94378-6cddc4947c-44bf7"
Sep 18 14:13:27.563781 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:13:27.563708    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-35b94378-6cddc4947c-44bf7" event={"ID":"c73ad8f8-faff-44c3-b07b-5c888886409e","Type":"ContainerDied","Data":"3f50df8613897a99079557c678496e71de53468768f4542c349afbebad2d83e8"}
Sep 18 14:13:27.563781 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:13:27.563745    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-35b94378-6cddc4947c-44bf7" event={"ID":"c73ad8f8-faff-44c3-b07b-5c888886409e","Type":"ContainerDied","Data":"d1100050f4aaa6cf725aae45fb32c6f19d6e29e8a97af1ba6b2685e3a774c4ff"}
Sep 18 14:13:27.571828 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:13:27.571813    2565 scope.go:117] "RemoveContainer" containerID="95e5e558861132a13b553c5802ffd8310b85d4c54cb093a0cb2c09acaeca53e6"
Sep 18 14:13:27.572173 ip-10-0-129-16 kubenswrapper[2565]: E0918 14:13:27.572152    2565 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"95e5e558861132a13b553c5802ffd8310b85d4c54cb093a0cb2c09acaeca53e6\": container with ID starting with 95e5e558861132a13b553c5802ffd8310b85d4c54cb093a0cb2c09acaeca53e6 not found: ID does not exist" containerID="95e5e558861132a13b553c5802ffd8310b85d4c54cb093a0cb2c09acaeca53e6"
Sep 18 14:13:27.572264 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:13:27.572186    2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"95e5e558861132a13b553c5802ffd8310b85d4c54cb093a0cb2c09acaeca53e6"} err="failed to get container status \"95e5e558861132a13b553c5802ffd8310b85d4c54cb093a0cb2c09acaeca53e6\": rpc error: code = NotFound desc = could not find container \"95e5e558861132a13b553c5802ffd8310b85d4c54cb093a0cb2c09acaeca53e6\": container with ID starting with 95e5e558861132a13b553c5802ffd8310b85d4c54cb093a0cb2c09acaeca53e6 not found: ID does not exist"
Sep 18 14:13:27.572264 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:13:27.572210    2565 scope.go:117] "RemoveContainer" containerID="3f50df8613897a99079557c678496e71de53468768f4542c349afbebad2d83e8"
Sep 18 14:13:27.582137 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:13:27.582112    2565 scope.go:117] "RemoveContainer" containerID="3f50df8613897a99079557c678496e71de53468768f4542c349afbebad2d83e8"
Sep 18 14:13:27.582539 ip-10-0-129-16 kubenswrapper[2565]: E0918 14:13:27.582512    2565 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3f50df8613897a99079557c678496e71de53468768f4542c349afbebad2d83e8\": container with ID starting with 3f50df8613897a99079557c678496e71de53468768f4542c349afbebad2d83e8 not found: ID does not exist" containerID="3f50df8613897a99079557c678496e71de53468768f4542c349afbebad2d83e8"
Sep 18 14:13:27.582599 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:13:27.582549    2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3f50df8613897a99079557c678496e71de53468768f4542c349afbebad2d83e8"} err="failed to get container status \"3f50df8613897a99079557c678496e71de53468768f4542c349afbebad2d83e8\": rpc error: code = NotFound desc = could not find container \"3f50df8613897a99079557c678496e71de53468768f4542c349afbebad2d83e8\": container with ID starting with 3f50df8613897a99079557c678496e71de53468768f4542c349afbebad2d83e8 not found: ID does not exist"
Sep 18 14:13:27.600540 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:13:27.600518    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-9slsg\" (UniqueName: \"kubernetes.io/projected/c73ad8f8-faff-44c3-b07b-5c888886409e-kube-api-access-9slsg\") pod \"c73ad8f8-faff-44c3-b07b-5c888886409e\" (UID: \"c73ad8f8-faff-44c3-b07b-5c888886409e\") "
Sep 18 14:13:27.600689 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:13:27.600608    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/2448d2e1-10b4-4a3f-9237-c99f3701c786-plugins-config-volume\") pod \"2448d2e1-10b4-4a3f-9237-c99f3701c786\" (UID: \"2448d2e1-10b4-4a3f-9237-c99f3701c786\") "
Sep 18 14:13:27.600689 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:13:27.600652    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-c98gl\" (UniqueName: \"kubernetes.io/projected/2448d2e1-10b4-4a3f-9237-c99f3701c786-kube-api-access-c98gl\") pod \"2448d2e1-10b4-4a3f-9237-c99f3701c786\" (UID: \"2448d2e1-10b4-4a3f-9237-c99f3701c786\") "
Sep 18 14:13:27.600689 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:13:27.600689    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c73ad8f8-faff-44c3-b07b-5c888886409e-plugins-config-volume\") pod \"c73ad8f8-faff-44c3-b07b-5c888886409e\" (UID: \"c73ad8f8-faff-44c3-b07b-5c888886409e\") "
Sep 18 14:13:27.601025 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:13:27.600999    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2448d2e1-10b4-4a3f-9237-c99f3701c786-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "2448d2e1-10b4-4a3f-9237-c99f3701c786" (UID: "2448d2e1-10b4-4a3f-9237-c99f3701c786"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 18 14:13:27.601172 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:13:27.601149    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c73ad8f8-faff-44c3-b07b-5c888886409e-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "c73ad8f8-faff-44c3-b07b-5c888886409e" (UID: "c73ad8f8-faff-44c3-b07b-5c888886409e"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 18 14:13:27.602750 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:13:27.602723    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2448d2e1-10b4-4a3f-9237-c99f3701c786-kube-api-access-c98gl" (OuterVolumeSpecName: "kube-api-access-c98gl") pod "2448d2e1-10b4-4a3f-9237-c99f3701c786" (UID: "2448d2e1-10b4-4a3f-9237-c99f3701c786"). InnerVolumeSpecName "kube-api-access-c98gl". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 18 14:13:27.602834 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:13:27.602767    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c73ad8f8-faff-44c3-b07b-5c888886409e-kube-api-access-9slsg" (OuterVolumeSpecName: "kube-api-access-9slsg") pod "c73ad8f8-faff-44c3-b07b-5c888886409e" (UID: "c73ad8f8-faff-44c3-b07b-5c888886409e"). InnerVolumeSpecName "kube-api-access-9slsg". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 18 14:13:27.702197 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:13:27.702111    2565 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9slsg\" (UniqueName: \"kubernetes.io/projected/c73ad8f8-faff-44c3-b07b-5c888886409e-kube-api-access-9slsg\") on node \"ip-10-0-129-16.ec2.internal\" DevicePath \"\""
Sep 18 14:13:27.702197 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:13:27.702141    2565 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/2448d2e1-10b4-4a3f-9237-c99f3701c786-plugins-config-volume\") on node \"ip-10-0-129-16.ec2.internal\" DevicePath \"\""
Sep 18 14:13:27.702197 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:13:27.702151    2565 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-c98gl\" (UniqueName: \"kubernetes.io/projected/2448d2e1-10b4-4a3f-9237-c99f3701c786-kube-api-access-c98gl\") on node \"ip-10-0-129-16.ec2.internal\" DevicePath \"\""
Sep 18 14:13:27.702197 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:13:27.702160    2565 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c73ad8f8-faff-44c3-b07b-5c888886409e-plugins-config-volume\") on node \"ip-10-0-129-16.ec2.internal\" DevicePath \"\""
Sep 18 14:13:27.887808 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:13:27.887768    2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-35b94378-6cddc4947c-44bf7"]
Sep 18 14:13:27.891480 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:13:27.891453    2565 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-35b94378-6cddc4947c-44bf7"]
Sep 18 14:13:27.902628 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:13:27.902603    2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-35b94378-5688bcc74b-dqxqz"]
Sep 18 14:13:27.917191 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:13:27.917159    2565 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6506201e-0d05-4393-8a16-e4c9ad014f1f" path="/var/lib/kubelet/pods/6506201e-0d05-4393-8a16-e4c9ad014f1f/volumes"
Sep 18 14:13:27.917554 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:13:27.917540    2565 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c73ad8f8-faff-44c3-b07b-5c888886409e" path="/var/lib/kubelet/pods/c73ad8f8-faff-44c3-b07b-5c888886409e/volumes"
Sep 18 14:13:27.917804 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:13:27.917792    2565 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-35b94378-5688bcc74b-dqxqz"]
Sep 18 14:13:29.148499 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:13:29.148464    2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-derive-a721b5af-openshift-default-7c7cdb57c6-lqd4l"]
Sep 18 14:13:29.149257 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:13:29.149233    2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c73ad8f8-faff-44c3-b07b-5c888886409e" containerName="payload-processing"
Sep 18 14:13:29.149257 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:13:29.149257    2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="c73ad8f8-faff-44c3-b07b-5c888886409e" containerName="payload-processing"
Sep 18 14:13:29.149411 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:13:29.149277    2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2448d2e1-10b4-4a3f-9237-c99f3701c786" containerName="payload-pre-processing"
Sep 18 14:13:29.149411 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:13:29.149284    2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="2448d2e1-10b4-4a3f-9237-c99f3701c786" containerName="payload-pre-processing"
Sep 18 14:13:29.149411 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:13:29.149301    2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6506201e-0d05-4393-8a16-e4c9ad014f1f" containerName="revoke-keys"
Sep 18 14:13:29.149411 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:13:29.149337    2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="6506201e-0d05-4393-8a16-e4c9ad014f1f" containerName="revoke-keys"
Sep 18 14:13:29.149543 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:13:29.149428    2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="6506201e-0d05-4393-8a16-e4c9ad014f1f" containerName="revoke-keys"
Sep 18 14:13:29.149543 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:13:29.149438    2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="2448d2e1-10b4-4a3f-9237-c99f3701c786" containerName="payload-pre-processing"
Sep 18 14:13:29.149543 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:13:29.149445    2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="c73ad8f8-faff-44c3-b07b-5c888886409e" containerName="payload-processing"
Sep 18 14:13:29.152737 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:13:29.152720    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-a721b5af-openshift-default-7c7cdb57c6-lqd4l"
Sep 18 14:13:29.156259 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:13:29.156240    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-derive-a721b5af-openshift-default-dockercfg-7kq5w\""
Sep 18 14:13:29.172934 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:13:29.172912    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-derive-a721b5af-openshift-default-7c7cdb57c6-lqd4l"]
Sep 18 14:13:29.318820 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:13:29.318774    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a9263603-24b9-44cf-8295-6e42afb3158b-istiod-ca-cert\") pod \"e2e-derive-a721b5af-openshift-default-7c7cdb57c6-lqd4l\" (UID: \"a9263603-24b9-44cf-8295-6e42afb3158b\") " pod="openshift-ingress/e2e-derive-a721b5af-openshift-default-7c7cdb57c6-lqd4l"
Sep 18 14:13:29.319020 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:13:29.318824    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a9263603-24b9-44cf-8295-6e42afb3158b-istio-envoy\") pod \"e2e-derive-a721b5af-openshift-default-7c7cdb57c6-lqd4l\" (UID: \"a9263603-24b9-44cf-8295-6e42afb3158b\") " pod="openshift-ingress/e2e-derive-a721b5af-openshift-default-7c7cdb57c6-lqd4l"
Sep 18 14:13:29.319020 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:13:29.318868    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a9263603-24b9-44cf-8295-6e42afb3158b-credential-socket\") pod \"e2e-derive-a721b5af-openshift-default-7c7cdb57c6-lqd4l\" (UID: \"a9263603-24b9-44cf-8295-6e42afb3158b\") " pod="openshift-ingress/e2e-derive-a721b5af-openshift-default-7c7cdb57c6-lqd4l"
Sep 18 14:13:29.319020 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:13:29.318890    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a9263603-24b9-44cf-8295-6e42afb3158b-workload-certs\") pod \"e2e-derive-a721b5af-openshift-default-7c7cdb57c6-lqd4l\" (UID: \"a9263603-24b9-44cf-8295-6e42afb3158b\") " pod="openshift-ingress/e2e-derive-a721b5af-openshift-default-7c7cdb57c6-lqd4l"
Sep 18 14:13:29.319020 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:13:29.318964    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a9263603-24b9-44cf-8295-6e42afb3158b-istio-podinfo\") pod \"e2e-derive-a721b5af-openshift-default-7c7cdb57c6-lqd4l\" (UID: \"a9263603-24b9-44cf-8295-6e42afb3158b\") " pod="openshift-ingress/e2e-derive-a721b5af-openshift-default-7c7cdb57c6-lqd4l"
Sep 18 14:13:29.319020 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:13:29.318993    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a9263603-24b9-44cf-8295-6e42afb3158b-istio-data\") pod \"e2e-derive-a721b5af-openshift-default-7c7cdb57c6-lqd4l\" (UID: \"a9263603-24b9-44cf-8295-6e42afb3158b\") " pod="openshift-ingress/e2e-derive-a721b5af-openshift-default-7c7cdb57c6-lqd4l"
Sep 18 14:13:29.319357 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:13:29.319023    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4tzv8\" (UniqueName: \"kubernetes.io/projected/a9263603-24b9-44cf-8295-6e42afb3158b-kube-api-access-4tzv8\") pod \"e2e-derive-a721b5af-openshift-default-7c7cdb57c6-lqd4l\" (UID: \"a9263603-24b9-44cf-8295-6e42afb3158b\") " pod="openshift-ingress/e2e-derive-a721b5af-openshift-default-7c7cdb57c6-lqd4l"
Sep 18 14:13:29.319357 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:13:29.319142    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a9263603-24b9-44cf-8295-6e42afb3158b-istio-token\") pod \"e2e-derive-a721b5af-openshift-default-7c7cdb57c6-lqd4l\" (UID: \"a9263603-24b9-44cf-8295-6e42afb3158b\") " pod="openshift-ingress/e2e-derive-a721b5af-openshift-default-7c7cdb57c6-lqd4l"
Sep 18 14:13:29.319357 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:13:29.319190    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a9263603-24b9-44cf-8295-6e42afb3158b-workload-socket\") pod \"e2e-derive-a721b5af-openshift-default-7c7cdb57c6-lqd4l\" (UID: \"a9263603-24b9-44cf-8295-6e42afb3158b\") " pod="openshift-ingress/e2e-derive-a721b5af-openshift-default-7c7cdb57c6-lqd4l"
Sep 18 14:13:29.420165 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:13:29.420081    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a9263603-24b9-44cf-8295-6e42afb3158b-istio-data\") pod \"e2e-derive-a721b5af-openshift-default-7c7cdb57c6-lqd4l\" (UID: \"a9263603-24b9-44cf-8295-6e42afb3158b\") " pod="openshift-ingress/e2e-derive-a721b5af-openshift-default-7c7cdb57c6-lqd4l"
Sep 18 14:13:29.420165 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:13:29.420126    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4tzv8\" (UniqueName: \"kubernetes.io/projected/a9263603-24b9-44cf-8295-6e42afb3158b-kube-api-access-4tzv8\") pod \"e2e-derive-a721b5af-openshift-default-7c7cdb57c6-lqd4l\" (UID: \"a9263603-24b9-44cf-8295-6e42afb3158b\") " pod="openshift-ingress/e2e-derive-a721b5af-openshift-default-7c7cdb57c6-lqd4l"
Sep 18 14:13:29.420400 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:13:29.420174    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a9263603-24b9-44cf-8295-6e42afb3158b-istio-token\") pod \"e2e-derive-a721b5af-openshift-default-7c7cdb57c6-lqd4l\" (UID: \"a9263603-24b9-44cf-8295-6e42afb3158b\") " pod="openshift-ingress/e2e-derive-a721b5af-openshift-default-7c7cdb57c6-lqd4l"
Sep 18 14:13:29.420400 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:13:29.420198    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a9263603-24b9-44cf-8295-6e42afb3158b-workload-socket\") pod \"e2e-derive-a721b5af-openshift-default-7c7cdb57c6-lqd4l\" (UID: \"a9263603-24b9-44cf-8295-6e42afb3158b\") " pod="openshift-ingress/e2e-derive-a721b5af-openshift-default-7c7cdb57c6-lqd4l"
Sep 18 14:13:29.420400 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:13:29.420329    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a9263603-24b9-44cf-8295-6e42afb3158b-istiod-ca-cert\") pod \"e2e-derive-a721b5af-openshift-default-7c7cdb57c6-lqd4l\" (UID: \"a9263603-24b9-44cf-8295-6e42afb3158b\") " pod="openshift-ingress/e2e-derive-a721b5af-openshift-default-7c7cdb57c6-lqd4l"
Sep 18 14:13:29.420400 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:13:29.420383    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a9263603-24b9-44cf-8295-6e42afb3158b-istio-envoy\") pod \"e2e-derive-a721b5af-openshift-default-7c7cdb57c6-lqd4l\" (UID: \"a9263603-24b9-44cf-8295-6e42afb3158b\") " pod="openshift-ingress/e2e-derive-a721b5af-openshift-default-7c7cdb57c6-lqd4l"
Sep 18 14:13:29.420626 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:13:29.420435    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a9263603-24b9-44cf-8295-6e42afb3158b-credential-socket\") pod \"e2e-derive-a721b5af-openshift-default-7c7cdb57c6-lqd4l\" (UID: \"a9263603-24b9-44cf-8295-6e42afb3158b\") " pod="openshift-ingress/e2e-derive-a721b5af-openshift-default-7c7cdb57c6-lqd4l"
Sep 18 14:13:29.420626 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:13:29.420461    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a9263603-24b9-44cf-8295-6e42afb3158b-workload-certs\") pod \"e2e-derive-a721b5af-openshift-default-7c7cdb57c6-lqd4l\" (UID: \"a9263603-24b9-44cf-8295-6e42afb3158b\") " pod="openshift-ingress/e2e-derive-a721b5af-openshift-default-7c7cdb57c6-lqd4l"
Sep 18 14:13:29.420626 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:13:29.420499    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a9263603-24b9-44cf-8295-6e42afb3158b-istio-data\") pod \"e2e-derive-a721b5af-openshift-default-7c7cdb57c6-lqd4l\" (UID: \"a9263603-24b9-44cf-8295-6e42afb3158b\") " pod="openshift-ingress/e2e-derive-a721b5af-openshift-default-7c7cdb57c6-lqd4l"
Sep 18 14:13:29.420626 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:13:29.420517    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a9263603-24b9-44cf-8295-6e42afb3158b-istio-podinfo\") pod \"e2e-derive-a721b5af-openshift-default-7c7cdb57c6-lqd4l\" (UID: \"a9263603-24b9-44cf-8295-6e42afb3158b\") " pod="openshift-ingress/e2e-derive-a721b5af-openshift-default-7c7cdb57c6-lqd4l"
Sep 18 14:13:29.420626 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:13:29.420536    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a9263603-24b9-44cf-8295-6e42afb3158b-workload-socket\") pod \"e2e-derive-a721b5af-openshift-default-7c7cdb57c6-lqd4l\" (UID: \"a9263603-24b9-44cf-8295-6e42afb3158b\") " pod="openshift-ingress/e2e-derive-a721b5af-openshift-default-7c7cdb57c6-lqd4l"
Sep 18 14:13:29.420849 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:13:29.420795    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a9263603-24b9-44cf-8295-6e42afb3158b-credential-socket\") pod \"e2e-derive-a721b5af-openshift-default-7c7cdb57c6-lqd4l\" (UID: \"a9263603-24b9-44cf-8295-6e42afb3158b\") " pod="openshift-ingress/e2e-derive-a721b5af-openshift-default-7c7cdb57c6-lqd4l"
Sep 18 14:13:29.420849 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:13:29.420838    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a9263603-24b9-44cf-8295-6e42afb3158b-workload-certs\") pod \"e2e-derive-a721b5af-openshift-default-7c7cdb57c6-lqd4l\" (UID: \"a9263603-24b9-44cf-8295-6e42afb3158b\") " pod="openshift-ingress/e2e-derive-a721b5af-openshift-default-7c7cdb57c6-lqd4l"
Sep 18 14:13:29.420993 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:13:29.420974    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a9263603-24b9-44cf-8295-6e42afb3158b-istiod-ca-cert\") pod \"e2e-derive-a721b5af-openshift-default-7c7cdb57c6-lqd4l\" (UID: \"a9263603-24b9-44cf-8295-6e42afb3158b\") " pod="openshift-ingress/e2e-derive-a721b5af-openshift-default-7c7cdb57c6-lqd4l"
Sep 18 14:13:29.422785 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:13:29.422763    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a9263603-24b9-44cf-8295-6e42afb3158b-istio-envoy\") pod \"e2e-derive-a721b5af-openshift-default-7c7cdb57c6-lqd4l\" (UID: \"a9263603-24b9-44cf-8295-6e42afb3158b\") " pod="openshift-ingress/e2e-derive-a721b5af-openshift-default-7c7cdb57c6-lqd4l"
Sep 18 14:13:29.422887 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:13:29.422848    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a9263603-24b9-44cf-8295-6e42afb3158b-istio-podinfo\") pod \"e2e-derive-a721b5af-openshift-default-7c7cdb57c6-lqd4l\" (UID: \"a9263603-24b9-44cf-8295-6e42afb3158b\") " pod="openshift-ingress/e2e-derive-a721b5af-openshift-default-7c7cdb57c6-lqd4l"
Sep 18 14:13:29.429592 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:13:29.429569    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4tzv8\" (UniqueName: \"kubernetes.io/projected/a9263603-24b9-44cf-8295-6e42afb3158b-kube-api-access-4tzv8\") pod \"e2e-derive-a721b5af-openshift-default-7c7cdb57c6-lqd4l\" (UID: \"a9263603-24b9-44cf-8295-6e42afb3158b\") " pod="openshift-ingress/e2e-derive-a721b5af-openshift-default-7c7cdb57c6-lqd4l"
Sep 18 14:13:29.429679 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:13:29.429574    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a9263603-24b9-44cf-8295-6e42afb3158b-istio-token\") pod \"e2e-derive-a721b5af-openshift-default-7c7cdb57c6-lqd4l\" (UID: \"a9263603-24b9-44cf-8295-6e42afb3158b\") " pod="openshift-ingress/e2e-derive-a721b5af-openshift-default-7c7cdb57c6-lqd4l"
Sep 18 14:13:29.493268 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:13:29.493237    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-a721b5af-openshift-default-7c7cdb57c6-lqd4l"
Sep 18 14:13:29.782716 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:13:29.782633    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-derive-a721b5af-openshift-default-7c7cdb57c6-lqd4l"]
Sep 18 14:13:29.785765 ip-10-0-129-16 kubenswrapper[2565]: W0918 14:13:29.785739    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda9263603_24b9_44cf_8295_6e42afb3158b.slice/crio-9573381140ccacf2e3fe7d7796b2d79e5416187c645fd34a76532c91927e32fc WatchSource:0}: Error finding container 9573381140ccacf2e3fe7d7796b2d79e5416187c645fd34a76532c91927e32fc: Status 404 returned error can't find the container with id 9573381140ccacf2e3fe7d7796b2d79e5416187c645fd34a76532c91927e32fc
Sep 18 14:13:29.787961 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:13:29.787928    2565 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 18 14:13:29.788056 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:13:29.787991    2565 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 18 14:13:29.788056 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:13:29.788020    2565 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 18 14:13:29.916460 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:13:29.916426    2565 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2448d2e1-10b4-4a3f-9237-c99f3701c786" path="/var/lib/kubelet/pods/2448d2e1-10b4-4a3f-9237-c99f3701c786/volumes"
Sep 18 14:13:30.587026 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:13:30.586991    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-a721b5af-openshift-default-7c7cdb57c6-lqd4l" event={"ID":"a9263603-24b9-44cf-8295-6e42afb3158b","Type":"ContainerStarted","Data":"6596ac7e60eb1a0d9a30022246bae2b9e66967b38cf9f83d81c5b84d0f0761c6"}
Sep 18 14:13:30.587421 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:13:30.587090    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-a721b5af-openshift-default-7c7cdb57c6-lqd4l" event={"ID":"a9263603-24b9-44cf-8295-6e42afb3158b","Type":"ContainerStarted","Data":"9573381140ccacf2e3fe7d7796b2d79e5416187c645fd34a76532c91927e32fc"}
Sep 18 14:13:30.621219 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:13:30.621163    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-derive-a721b5af-openshift-default-7c7cdb57c6-lqd4l" podStartSLOduration=1.621146376 podStartE2EDuration="1.621146376s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 14:13:29 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 14:13:30.619943824 +0000 UTC m=+878.793853517" watchObservedRunningTime="2026-09-18 14:13:30.621146376 +0000 UTC m=+878.795056036"
Sep 18 14:13:31.493960 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:13:31.493921    2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-derive-a721b5af-openshift-default-7c7cdb57c6-lqd4l"
Sep 18 14:13:31.498966 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:13:31.498941    2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-derive-a721b5af-openshift-default-7c7cdb57c6-lqd4l"
Sep 18 14:13:31.592828 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:13:31.592798    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-derive-a721b5af-openshift-default-7c7cdb57c6-lqd4l"
Sep 18 14:13:31.593803 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:13:31.593785    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-derive-a721b5af-openshift-default-7c7cdb57c6-lqd4l"
Sep 18 14:13:36.076791 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:13:36.076715    2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-derive-a721b5af-5bcf9c6877-7qmm9"]
Sep 18 14:13:36.079521 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:13:36.079505    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-derive-a721b5af-5bcf9c6877-7qmm9"
Sep 18 14:13:36.085713 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:13:36.085689    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-derive-a721b5af\""
Sep 18 14:13:36.085834 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:13:36.085810    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-derive-a721b5af-dockercfg-ls5l4\""
Sep 18 14:13:36.095264 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:13:36.095235    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-derive-a721b5af-5bcf9c6877-7qmm9"]
Sep 18 14:13:36.120114 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:13:36.120078    2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-derive-a721b5af-858fff4c46-lhcg9"]
Sep 18 14:13:36.122736 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:13:36.122712    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-derive-a721b5af-858fff4c46-lhcg9"
Sep 18 14:13:36.134461 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:13:36.134434    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-derive-a721b5af-858fff4c46-lhcg9"]
Sep 18 14:13:36.193798 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:13:36.193759    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-clkxd\" (UniqueName: \"kubernetes.io/projected/6a27bdb8-3bfe-4f7e-afdb-ff20c1874d2d-kube-api-access-clkxd\") pod \"payload-processing-e2e-derive-a721b5af-5bcf9c6877-7qmm9\" (UID: \"6a27bdb8-3bfe-4f7e-afdb-ff20c1874d2d\") " pod="openshift-ingress/payload-processing-e2e-derive-a721b5af-5bcf9c6877-7qmm9"
Sep 18 14:13:36.193994 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:13:36.193905    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/6a27bdb8-3bfe-4f7e-afdb-ff20c1874d2d-plugins-config-volume\") pod \"payload-processing-e2e-derive-a721b5af-5bcf9c6877-7qmm9\" (UID: \"6a27bdb8-3bfe-4f7e-afdb-ff20c1874d2d\") " pod="openshift-ingress/payload-processing-e2e-derive-a721b5af-5bcf9c6877-7qmm9"
Sep 18 14:13:36.294658 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:13:36.294623    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-clkxd\" (UniqueName: \"kubernetes.io/projected/6a27bdb8-3bfe-4f7e-afdb-ff20c1874d2d-kube-api-access-clkxd\") pod \"payload-processing-e2e-derive-a721b5af-5bcf9c6877-7qmm9\" (UID: \"6a27bdb8-3bfe-4f7e-afdb-ff20c1874d2d\") " pod="openshift-ingress/payload-processing-e2e-derive-a721b5af-5bcf9c6877-7qmm9"
Sep 18 14:13:36.294852 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:13:36.294682    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/6a27bdb8-3bfe-4f7e-afdb-ff20c1874d2d-plugins-config-volume\") pod \"payload-processing-e2e-derive-a721b5af-5bcf9c6877-7qmm9\" (UID: \"6a27bdb8-3bfe-4f7e-afdb-ff20c1874d2d\") " pod="openshift-ingress/payload-processing-e2e-derive-a721b5af-5bcf9c6877-7qmm9"
Sep 18 14:13:36.294852 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:13:36.294720    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/38140408-473f-42d3-bf00-201cbb33c0e6-plugins-config-volume\") pod \"payload-pre-processing-e2e-derive-a721b5af-858fff4c46-lhcg9\" (UID: \"38140408-473f-42d3-bf00-201cbb33c0e6\") " pod="openshift-ingress/payload-pre-processing-e2e-derive-a721b5af-858fff4c46-lhcg9"
Sep 18 14:13:36.294852 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:13:36.294752    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pghx8\" (UniqueName: \"kubernetes.io/projected/38140408-473f-42d3-bf00-201cbb33c0e6-kube-api-access-pghx8\") pod \"payload-pre-processing-e2e-derive-a721b5af-858fff4c46-lhcg9\" (UID: \"38140408-473f-42d3-bf00-201cbb33c0e6\") " pod="openshift-ingress/payload-pre-processing-e2e-derive-a721b5af-858fff4c46-lhcg9"
Sep 18 14:13:36.295388 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:13:36.295368    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/6a27bdb8-3bfe-4f7e-afdb-ff20c1874d2d-plugins-config-volume\") pod \"payload-processing-e2e-derive-a721b5af-5bcf9c6877-7qmm9\" (UID: \"6a27bdb8-3bfe-4f7e-afdb-ff20c1874d2d\") " pod="openshift-ingress/payload-processing-e2e-derive-a721b5af-5bcf9c6877-7qmm9"
Sep 18 14:13:36.304008 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:13:36.303983    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-clkxd\" (UniqueName: \"kubernetes.io/projected/6a27bdb8-3bfe-4f7e-afdb-ff20c1874d2d-kube-api-access-clkxd\") pod \"payload-processing-e2e-derive-a721b5af-5bcf9c6877-7qmm9\" (UID: \"6a27bdb8-3bfe-4f7e-afdb-ff20c1874d2d\") " pod="openshift-ingress/payload-processing-e2e-derive-a721b5af-5bcf9c6877-7qmm9"
Sep 18 14:13:36.391003 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:13:36.390915    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-derive-a721b5af-5bcf9c6877-7qmm9"
Sep 18 14:13:36.395520 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:13:36.395485    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pghx8\" (UniqueName: \"kubernetes.io/projected/38140408-473f-42d3-bf00-201cbb33c0e6-kube-api-access-pghx8\") pod \"payload-pre-processing-e2e-derive-a721b5af-858fff4c46-lhcg9\" (UID: \"38140408-473f-42d3-bf00-201cbb33c0e6\") " pod="openshift-ingress/payload-pre-processing-e2e-derive-a721b5af-858fff4c46-lhcg9"
Sep 18 14:13:36.395658 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:13:36.395620    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/38140408-473f-42d3-bf00-201cbb33c0e6-plugins-config-volume\") pod \"payload-pre-processing-e2e-derive-a721b5af-858fff4c46-lhcg9\" (UID: \"38140408-473f-42d3-bf00-201cbb33c0e6\") " pod="openshift-ingress/payload-pre-processing-e2e-derive-a721b5af-858fff4c46-lhcg9"
Sep 18 14:13:36.396478 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:13:36.396452    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/38140408-473f-42d3-bf00-201cbb33c0e6-plugins-config-volume\") pod \"payload-pre-processing-e2e-derive-a721b5af-858fff4c46-lhcg9\" (UID: \"38140408-473f-42d3-bf00-201cbb33c0e6\") " pod="openshift-ingress/payload-pre-processing-e2e-derive-a721b5af-858fff4c46-lhcg9"
Sep 18 14:13:36.404092 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:13:36.404070    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pghx8\" (UniqueName: \"kubernetes.io/projected/38140408-473f-42d3-bf00-201cbb33c0e6-kube-api-access-pghx8\") pod \"payload-pre-processing-e2e-derive-a721b5af-858fff4c46-lhcg9\" (UID: \"38140408-473f-42d3-bf00-201cbb33c0e6\") " pod="openshift-ingress/payload-pre-processing-e2e-derive-a721b5af-858fff4c46-lhcg9"
Sep 18 14:13:36.436256 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:13:36.436224    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-derive-a721b5af-858fff4c46-lhcg9"
Sep 18 14:13:36.671675 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:13:36.671650    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-derive-a721b5af-5bcf9c6877-7qmm9"]
Sep 18 14:13:36.673077 ip-10-0-129-16 kubenswrapper[2565]: W0918 14:13:36.673048    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6a27bdb8_3bfe_4f7e_afdb_ff20c1874d2d.slice/crio-f4931274b067cf1d453e6daaa0c0b2eb55a2983442d7ee8f01669f59ffd47cda WatchSource:0}: Error finding container f4931274b067cf1d453e6daaa0c0b2eb55a2983442d7ee8f01669f59ffd47cda: Status 404 returned error can't find the container with id f4931274b067cf1d453e6daaa0c0b2eb55a2983442d7ee8f01669f59ffd47cda
Sep 18 14:13:36.725219 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:13:36.725193    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-derive-a721b5af-858fff4c46-lhcg9"]
Sep 18 14:13:36.730428 ip-10-0-129-16 kubenswrapper[2565]: W0918 14:13:36.730406    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod38140408_473f_42d3_bf00_201cbb33c0e6.slice/crio-880cb1cd22e6dfb60352b703682bc55193756bd8a10f260deb96f9ed074c0fdb WatchSource:0}: Error finding container 880cb1cd22e6dfb60352b703682bc55193756bd8a10f260deb96f9ed074c0fdb: Status 404 returned error can't find the container with id 880cb1cd22e6dfb60352b703682bc55193756bd8a10f260deb96f9ed074c0fdb
Sep 18 14:13:37.629175 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:13:37.629133    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-derive-a721b5af-5bcf9c6877-7qmm9" event={"ID":"6a27bdb8-3bfe-4f7e-afdb-ff20c1874d2d","Type":"ContainerStarted","Data":"bb744c520826a045a320f78f9d6f70b84ebe8617e50cdd9b45cf85b67331ec26"}
Sep 18 14:13:37.629691 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:13:37.629202    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-derive-a721b5af-5bcf9c6877-7qmm9" event={"ID":"6a27bdb8-3bfe-4f7e-afdb-ff20c1874d2d","Type":"ContainerStarted","Data":"f4931274b067cf1d453e6daaa0c0b2eb55a2983442d7ee8f01669f59ffd47cda"}
Sep 18 14:13:37.629691 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:13:37.629319    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-derive-a721b5af-5bcf9c6877-7qmm9"
Sep 18 14:13:37.630873 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:13:37.630848    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-derive-a721b5af-858fff4c46-lhcg9" event={"ID":"38140408-473f-42d3-bf00-201cbb33c0e6","Type":"ContainerStarted","Data":"33aa4bf200d80ec82f5b0dc5722ebe009a37d64da59c09a70aca5c7331fff4b9"}
Sep 18 14:13:37.631004 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:13:37.630888    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-derive-a721b5af-858fff4c46-lhcg9" event={"ID":"38140408-473f-42d3-bf00-201cbb33c0e6","Type":"ContainerStarted","Data":"880cb1cd22e6dfb60352b703682bc55193756bd8a10f260deb96f9ed074c0fdb"}
Sep 18 14:13:37.631004 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:13:37.630927    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-derive-a721b5af-858fff4c46-lhcg9"
Sep 18 14:13:37.676751 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:13:37.676696    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-derive-a721b5af-5bcf9c6877-7qmm9" podStartSLOduration=1.676677256 podStartE2EDuration="1.676677256s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 14:13:36 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 14:13:37.674832806 +0000 UTC m=+885.848742485" watchObservedRunningTime="2026-09-18 14:13:37.676677256 +0000 UTC m=+885.850586918"
Sep 18 14:13:48.641246 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:13:48.641208    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-derive-a721b5af-858fff4c46-lhcg9"
Sep 18 14:13:48.641780 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:13:48.641620    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-derive-a721b5af-5bcf9c6877-7qmm9"
Sep 18 14:13:48.666882 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:13:48.666826    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-derive-a721b5af-858fff4c46-lhcg9" podStartSLOduration=12.666810389 podStartE2EDuration="12.666810389s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 14:13:36 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 14:13:37.71242923 +0000 UTC m=+885.886338890" watchObservedRunningTime="2026-09-18 14:13:48.666810389 +0000 UTC m=+896.840720049"
Sep 18 14:13:53.732422 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:13:53.732378    2565 generic.go:356] "Generic (PLEG): container finished" podID="a9263603-24b9-44cf-8295-6e42afb3158b" containerID="6596ac7e60eb1a0d9a30022246bae2b9e66967b38cf9f83d81c5b84d0f0761c6" exitCode=0
Sep 18 14:13:53.732874 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:13:53.732452    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-a721b5af-openshift-default-7c7cdb57c6-lqd4l" event={"ID":"a9263603-24b9-44cf-8295-6e42afb3158b","Type":"ContainerDied","Data":"6596ac7e60eb1a0d9a30022246bae2b9e66967b38cf9f83d81c5b84d0f0761c6"}
Sep 18 14:13:53.732938 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:13:53.732908    2565 scope.go:117] "RemoveContainer" containerID="6596ac7e60eb1a0d9a30022246bae2b9e66967b38cf9f83d81c5b84d0f0761c6"
Sep 18 14:13:53.734200 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:13:53.734172    2565 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 18 14:13:53.734276 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:13:53.734250    2565 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 18 14:13:53.734326 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:13:53.734283    2565 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 18 14:13:53.908489 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:13:53.908457    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-bvqls_0b0463f5-d647-4a38-b752-44afba77daff/ovn-acl-logging/0.log"
Sep 18 14:13:53.910186 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:13:53.910161    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-bvqls_0b0463f5-d647-4a38-b752-44afba77daff/ovn-acl-logging/0.log"
Sep 18 14:13:54.742984 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:13:54.742945    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-a721b5af-openshift-default-7c7cdb57c6-lqd4l" event={"ID":"a9263603-24b9-44cf-8295-6e42afb3158b","Type":"ContainerStarted","Data":"a2f6de37badbf23b15357c61ea80f60b1e3f645890d0ca33a41691e372ffde76"}
Sep 18 14:13:55.493543 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:13:55.493501    2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-derive-a721b5af-openshift-default-7c7cdb57c6-lqd4l"
Sep 18 14:13:55.493774 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:13:55.493751    2565 patch_prober.go:28] interesting pod/e2e-derive-a721b5af-openshift-default-7c7cdb57c6-lqd4l container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.57:15021/healthz/ready\": dial tcp 10.134.0.57:15021: connect: connection refused" start-of-body=
Sep 18 14:13:55.493819 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:13:55.493801    2565 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-derive-a721b5af-openshift-default-7c7cdb57c6-lqd4l" podUID="a9263603-24b9-44cf-8295-6e42afb3158b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.57:15021/healthz/ready\": dial tcp 10.134.0.57:15021: connect: connection refused"
Sep 18 14:13:56.493585 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:13:56.493554    2565 patch_prober.go:28] interesting pod/e2e-derive-a721b5af-openshift-default-7c7cdb57c6-lqd4l container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.57:15021/healthz/ready\": dial tcp 10.134.0.57:15021: connect: connection refused" start-of-body=
Sep 18 14:13:56.494019 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:13:56.493608    2565 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-derive-a721b5af-openshift-default-7c7cdb57c6-lqd4l" podUID="a9263603-24b9-44cf-8295-6e42afb3158b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.57:15021/healthz/ready\": dial tcp 10.134.0.57:15021: connect: connection refused"
Sep 18 14:13:57.493877 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:13:57.493839    2565 patch_prober.go:28] interesting pod/e2e-derive-a721b5af-openshift-default-7c7cdb57c6-lqd4l container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.57:15021/healthz/ready\": dial tcp 10.134.0.57:15021: connect: connection refused" start-of-body=
Sep 18 14:13:57.494302 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:13:57.493923    2565 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-derive-a721b5af-openshift-default-7c7cdb57c6-lqd4l" podUID="a9263603-24b9-44cf-8295-6e42afb3158b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.57:15021/healthz/ready\": dial tcp 10.134.0.57:15021: connect: connection refused"
Sep 18 14:13:58.493823 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:13:58.493786    2565 patch_prober.go:28] interesting pod/e2e-derive-a721b5af-openshift-default-7c7cdb57c6-lqd4l container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.57:15021/healthz/ready\": dial tcp 10.134.0.57:15021: connect: connection refused" start-of-body=
Sep 18 14:13:58.494006 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:13:58.493843    2565 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-derive-a721b5af-openshift-default-7c7cdb57c6-lqd4l" podUID="a9263603-24b9-44cf-8295-6e42afb3158b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.57:15021/healthz/ready\": dial tcp 10.134.0.57:15021: connect: connection refused"
Sep 18 14:13:59.494136 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:13:59.494101    2565 patch_prober.go:28] interesting pod/e2e-derive-a721b5af-openshift-default-7c7cdb57c6-lqd4l container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.57:15021/healthz/ready\": dial tcp 10.134.0.57:15021: connect: connection refused" start-of-body=
Sep 18 14:13:59.494558 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:13:59.494168    2565 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-derive-a721b5af-openshift-default-7c7cdb57c6-lqd4l" podUID="a9263603-24b9-44cf-8295-6e42afb3158b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.57:15021/healthz/ready\": dial tcp 10.134.0.57:15021: connect: connection refused"
Sep 18 14:14:00.498493 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:14:00.498462    2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-derive-a721b5af-openshift-default-7c7cdb57c6-lqd4l"
Sep 18 14:14:00.498984 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:14:00.498733    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-derive-a721b5af-openshift-default-7c7cdb57c6-lqd4l"
Sep 18 14:14:00.499534 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:14:00.499519    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-derive-a721b5af-openshift-default-7c7cdb57c6-lqd4l"
Sep 18 14:14:13.914833 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:14:13.914803    2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-explicit-e11b4c-6876b46f7c-xxvlx"]
Sep 18 14:14:13.918711 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:14:13.918691    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-explicit-e11b4c-6876b46f7c-xxvlx"
Sep 18 14:14:13.922064 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:14:13.922026    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-explicit-e11b4c\""
Sep 18 14:14:13.922187 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:14:13.922124    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-explicit-e11b4c-dockercfg-lzt98\""
Sep 18 14:14:13.935767 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:14:13.935742    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-explicit-e11b4c-6876b46f7c-xxvlx"]
Sep 18 14:14:13.995813 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:14:13.995779    2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-explicit-e11b4c-676cdf8446-67jbs"]
Sep 18 14:14:13.999783 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:14:13.999762    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-explicit-e11b4c-676cdf8446-67jbs"
Sep 18 14:14:14.015016 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:14:14.014980    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-explicit-e11b4c-676cdf8446-67jbs"]
Sep 18 14:14:14.054524 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:14:14.054471    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e2db5cb4-9648-441c-9808-dd89944fd867-plugins-config-volume\") pod \"payload-processing-e2e-explicit-e11b4c-6876b46f7c-xxvlx\" (UID: \"e2db5cb4-9648-441c-9808-dd89944fd867\") " pod="openshift-ingress/payload-processing-e2e-explicit-e11b4c-6876b46f7c-xxvlx"
Sep 18 14:14:14.054753 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:14:14.054585    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cfh8g\" (UniqueName: \"kubernetes.io/projected/e2db5cb4-9648-441c-9808-dd89944fd867-kube-api-access-cfh8g\") pod \"payload-processing-e2e-explicit-e11b4c-6876b46f7c-xxvlx\" (UID: \"e2db5cb4-9648-441c-9808-dd89944fd867\") " pod="openshift-ingress/payload-processing-e2e-explicit-e11b4c-6876b46f7c-xxvlx"
Sep 18 14:14:14.155429 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:14:14.155381    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e2db5cb4-9648-441c-9808-dd89944fd867-plugins-config-volume\") pod \"payload-processing-e2e-explicit-e11b4c-6876b46f7c-xxvlx\" (UID: \"e2db5cb4-9648-441c-9808-dd89944fd867\") " pod="openshift-ingress/payload-processing-e2e-explicit-e11b4c-6876b46f7c-xxvlx"
Sep 18 14:14:14.155612 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:14:14.155515    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-66jrg\" (UniqueName: \"kubernetes.io/projected/5abb7bbd-f229-4637-8e52-b7e2041c0a22-kube-api-access-66jrg\") pod \"payload-pre-processing-e2e-explicit-e11b4c-676cdf8446-67jbs\" (UID: \"5abb7bbd-f229-4637-8e52-b7e2041c0a22\") " pod="openshift-ingress/payload-pre-processing-e2e-explicit-e11b4c-676cdf8446-67jbs"
Sep 18 14:14:14.155612 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:14:14.155547    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cfh8g\" (UniqueName: \"kubernetes.io/projected/e2db5cb4-9648-441c-9808-dd89944fd867-kube-api-access-cfh8g\") pod \"payload-processing-e2e-explicit-e11b4c-6876b46f7c-xxvlx\" (UID: \"e2db5cb4-9648-441c-9808-dd89944fd867\") " pod="openshift-ingress/payload-processing-e2e-explicit-e11b4c-6876b46f7c-xxvlx"
Sep 18 14:14:14.155746 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:14:14.155677    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/5abb7bbd-f229-4637-8e52-b7e2041c0a22-plugins-config-volume\") pod \"payload-pre-processing-e2e-explicit-e11b4c-676cdf8446-67jbs\" (UID: \"5abb7bbd-f229-4637-8e52-b7e2041c0a22\") " pod="openshift-ingress/payload-pre-processing-e2e-explicit-e11b4c-676cdf8446-67jbs"
Sep 18 14:14:14.156305 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:14:14.156282    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e2db5cb4-9648-441c-9808-dd89944fd867-plugins-config-volume\") pod \"payload-processing-e2e-explicit-e11b4c-6876b46f7c-xxvlx\" (UID: \"e2db5cb4-9648-441c-9808-dd89944fd867\") " pod="openshift-ingress/payload-processing-e2e-explicit-e11b4c-6876b46f7c-xxvlx"
Sep 18 14:14:14.170194 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:14:14.170106    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cfh8g\" (UniqueName: \"kubernetes.io/projected/e2db5cb4-9648-441c-9808-dd89944fd867-kube-api-access-cfh8g\") pod \"payload-processing-e2e-explicit-e11b4c-6876b46f7c-xxvlx\" (UID: \"e2db5cb4-9648-441c-9808-dd89944fd867\") " pod="openshift-ingress/payload-processing-e2e-explicit-e11b4c-6876b46f7c-xxvlx"
Sep 18 14:14:14.195804 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:14:14.195770    2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-derive-a721b5af-5bcf9c6877-7qmm9"]
Sep 18 14:14:14.196119 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:14:14.196089    2565 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-derive-a721b5af-5bcf9c6877-7qmm9" podUID="6a27bdb8-3bfe-4f7e-afdb-ff20c1874d2d" containerName="payload-processing" containerID="cri-o://bb744c520826a045a320f78f9d6f70b84ebe8617e50cdd9b45cf85b67331ec26" gracePeriod=30
Sep 18 14:14:14.206400 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:14:14.206206    2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-derive-a721b5af-858fff4c46-lhcg9"]
Sep 18 14:14:14.206528 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:14:14.206434    2565 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-derive-a721b5af-858fff4c46-lhcg9" podUID="38140408-473f-42d3-bf00-201cbb33c0e6" containerName="payload-pre-processing" containerID="cri-o://33aa4bf200d80ec82f5b0dc5722ebe009a37d64da59c09a70aca5c7331fff4b9" gracePeriod=30
Sep 18 14:14:14.229632 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:14:14.229608    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-explicit-e11b4c-6876b46f7c-xxvlx"
Sep 18 14:14:14.256824 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:14:14.256787    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/5abb7bbd-f229-4637-8e52-b7e2041c0a22-plugins-config-volume\") pod \"payload-pre-processing-e2e-explicit-e11b4c-676cdf8446-67jbs\" (UID: \"5abb7bbd-f229-4637-8e52-b7e2041c0a22\") " pod="openshift-ingress/payload-pre-processing-e2e-explicit-e11b4c-676cdf8446-67jbs"
Sep 18 14:14:14.257021 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:14:14.256949    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-66jrg\" (UniqueName: \"kubernetes.io/projected/5abb7bbd-f229-4637-8e52-b7e2041c0a22-kube-api-access-66jrg\") pod \"payload-pre-processing-e2e-explicit-e11b4c-676cdf8446-67jbs\" (UID: \"5abb7bbd-f229-4637-8e52-b7e2041c0a22\") " pod="openshift-ingress/payload-pre-processing-e2e-explicit-e11b4c-676cdf8446-67jbs"
Sep 18 14:14:14.257633 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:14:14.257611    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/5abb7bbd-f229-4637-8e52-b7e2041c0a22-plugins-config-volume\") pod \"payload-pre-processing-e2e-explicit-e11b4c-676cdf8446-67jbs\" (UID: \"5abb7bbd-f229-4637-8e52-b7e2041c0a22\") " pod="openshift-ingress/payload-pre-processing-e2e-explicit-e11b4c-676cdf8446-67jbs"
Sep 18 14:14:14.269574 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:14:14.269545    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-66jrg\" (UniqueName: \"kubernetes.io/projected/5abb7bbd-f229-4637-8e52-b7e2041c0a22-kube-api-access-66jrg\") pod \"payload-pre-processing-e2e-explicit-e11b4c-676cdf8446-67jbs\" (UID: \"5abb7bbd-f229-4637-8e52-b7e2041c0a22\") " pod="openshift-ingress/payload-pre-processing-e2e-explicit-e11b4c-676cdf8446-67jbs"
Sep 18 14:14:14.310580 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:14:14.310384    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-explicit-e11b4c-676cdf8446-67jbs"
Sep 18 14:14:14.475372 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:14:14.475344    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-derive-a721b5af-5bcf9c6877-7qmm9"
Sep 18 14:14:14.515300 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:14:14.515263    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-explicit-e11b4c-6876b46f7c-xxvlx"]
Sep 18 14:14:14.516750 ip-10-0-129-16 kubenswrapper[2565]: W0918 14:14:14.516722    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode2db5cb4_9648_441c_9808_dd89944fd867.slice/crio-a43ca13bb813db4a0fada037650cbee309acc38f5cbfea6ff0cf6d7adae0091b WatchSource:0}: Error finding container a43ca13bb813db4a0fada037650cbee309acc38f5cbfea6ff0cf6d7adae0091b: Status 404 returned error can't find the container with id a43ca13bb813db4a0fada037650cbee309acc38f5cbfea6ff0cf6d7adae0091b
Sep 18 14:14:14.526560 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:14:14.526491    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-derive-a721b5af-858fff4c46-lhcg9"
Sep 18 14:14:14.564065 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:14:14.562506    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/6a27bdb8-3bfe-4f7e-afdb-ff20c1874d2d-plugins-config-volume\") pod \"6a27bdb8-3bfe-4f7e-afdb-ff20c1874d2d\" (UID: \"6a27bdb8-3bfe-4f7e-afdb-ff20c1874d2d\") "
Sep 18 14:14:14.564065 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:14:14.562726    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-clkxd\" (UniqueName: \"kubernetes.io/projected/6a27bdb8-3bfe-4f7e-afdb-ff20c1874d2d-kube-api-access-clkxd\") pod \"6a27bdb8-3bfe-4f7e-afdb-ff20c1874d2d\" (UID: \"6a27bdb8-3bfe-4f7e-afdb-ff20c1874d2d\") "
Sep 18 14:14:14.564065 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:14:14.562967    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6a27bdb8-3bfe-4f7e-afdb-ff20c1874d2d-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "6a27bdb8-3bfe-4f7e-afdb-ff20c1874d2d" (UID: "6a27bdb8-3bfe-4f7e-afdb-ff20c1874d2d"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 18 14:14:14.564065 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:14:14.563120    2565 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/6a27bdb8-3bfe-4f7e-afdb-ff20c1874d2d-plugins-config-volume\") on node \"ip-10-0-129-16.ec2.internal\" DevicePath \"\""
Sep 18 14:14:14.568368 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:14:14.568232    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6a27bdb8-3bfe-4f7e-afdb-ff20c1874d2d-kube-api-access-clkxd" (OuterVolumeSpecName: "kube-api-access-clkxd") pod "6a27bdb8-3bfe-4f7e-afdb-ff20c1874d2d" (UID: "6a27bdb8-3bfe-4f7e-afdb-ff20c1874d2d"). InnerVolumeSpecName "kube-api-access-clkxd". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 18 14:14:14.611503 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:14:14.611477    2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-derive-a721b5af-openshift-default-7c7cdb57c6-lqd4l"]
Sep 18 14:14:14.611904 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:14:14.611877    2565 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-derive-a721b5af-openshift-default-7c7cdb57c6-lqd4l" podUID="a9263603-24b9-44cf-8295-6e42afb3158b" containerName="istio-proxy" containerID="cri-o://a2f6de37badbf23b15357c61ea80f60b1e3f645890d0ca33a41691e372ffde76" gracePeriod=30
Sep 18 14:14:14.633277 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:14:14.633248    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-explicit-e11b4c-676cdf8446-67jbs"]
Sep 18 14:14:14.637125 ip-10-0-129-16 kubenswrapper[2565]: W0918 14:14:14.636843    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5abb7bbd_f229_4637_8e52_b7e2041c0a22.slice/crio-c9fce9008b2d4f78071235084c9e83ea23cf173034bd32c2f6924391c3008ccd WatchSource:0}: Error finding container c9fce9008b2d4f78071235084c9e83ea23cf173034bd32c2f6924391c3008ccd: Status 404 returned error can't find the container with id c9fce9008b2d4f78071235084c9e83ea23cf173034bd32c2f6924391c3008ccd
Sep 18 14:14:14.664226 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:14:14.664192    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/38140408-473f-42d3-bf00-201cbb33c0e6-plugins-config-volume\") pod \"38140408-473f-42d3-bf00-201cbb33c0e6\" (UID: \"38140408-473f-42d3-bf00-201cbb33c0e6\") "
Sep 18 14:14:14.664386 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:14:14.664371    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-pghx8\" (UniqueName: \"kubernetes.io/projected/38140408-473f-42d3-bf00-201cbb33c0e6-kube-api-access-pghx8\") pod \"38140408-473f-42d3-bf00-201cbb33c0e6\" (UID: \"38140408-473f-42d3-bf00-201cbb33c0e6\") "
Sep 18 14:14:14.664558 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:14:14.664531    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/38140408-473f-42d3-bf00-201cbb33c0e6-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "38140408-473f-42d3-bf00-201cbb33c0e6" (UID: "38140408-473f-42d3-bf00-201cbb33c0e6"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 18 14:14:14.664756 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:14:14.664740    2565 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-clkxd\" (UniqueName: \"kubernetes.io/projected/6a27bdb8-3bfe-4f7e-afdb-ff20c1874d2d-kube-api-access-clkxd\") on node \"ip-10-0-129-16.ec2.internal\" DevicePath \"\""
Sep 18 14:14:14.666254 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:14:14.666233    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/38140408-473f-42d3-bf00-201cbb33c0e6-kube-api-access-pghx8" (OuterVolumeSpecName: "kube-api-access-pghx8") pod "38140408-473f-42d3-bf00-201cbb33c0e6" (UID: "38140408-473f-42d3-bf00-201cbb33c0e6"). InnerVolumeSpecName "kube-api-access-pghx8". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 18 14:14:14.765409 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:14:14.765377    2565 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/38140408-473f-42d3-bf00-201cbb33c0e6-plugins-config-volume\") on node \"ip-10-0-129-16.ec2.internal\" DevicePath \"\""
Sep 18 14:14:14.765409 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:14:14.765404    2565 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-pghx8\" (UniqueName: \"kubernetes.io/projected/38140408-473f-42d3-bf00-201cbb33c0e6-kube-api-access-pghx8\") on node \"ip-10-0-129-16.ec2.internal\" DevicePath \"\""
Sep 18 14:14:14.880590 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:14:14.880550    2565 generic.go:356] "Generic (PLEG): container finished" podID="38140408-473f-42d3-bf00-201cbb33c0e6" containerID="33aa4bf200d80ec82f5b0dc5722ebe009a37d64da59c09a70aca5c7331fff4b9" exitCode=0
Sep 18 14:14:14.880789 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:14:14.880606    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-derive-a721b5af-858fff4c46-lhcg9"
Sep 18 14:14:14.880789 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:14:14.880647    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-derive-a721b5af-858fff4c46-lhcg9" event={"ID":"38140408-473f-42d3-bf00-201cbb33c0e6","Type":"ContainerDied","Data":"33aa4bf200d80ec82f5b0dc5722ebe009a37d64da59c09a70aca5c7331fff4b9"}
Sep 18 14:14:14.880789 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:14:14.880700    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-derive-a721b5af-858fff4c46-lhcg9" event={"ID":"38140408-473f-42d3-bf00-201cbb33c0e6","Type":"ContainerDied","Data":"880cb1cd22e6dfb60352b703682bc55193756bd8a10f260deb96f9ed074c0fdb"}
Sep 18 14:14:14.880789 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:14:14.880731    2565 scope.go:117] "RemoveContainer" containerID="33aa4bf200d80ec82f5b0dc5722ebe009a37d64da59c09a70aca5c7331fff4b9"
Sep 18 14:14:14.882655 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:14:14.882627    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-explicit-e11b4c-676cdf8446-67jbs" event={"ID":"5abb7bbd-f229-4637-8e52-b7e2041c0a22","Type":"ContainerStarted","Data":"f8185aa32f74ac08cfa5e8376e3fc8f88ccf6a9c35619793b51df5812b2bc08f"}
Sep 18 14:14:14.882778 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:14:14.882680    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-explicit-e11b4c-676cdf8446-67jbs" event={"ID":"5abb7bbd-f229-4637-8e52-b7e2041c0a22","Type":"ContainerStarted","Data":"c9fce9008b2d4f78071235084c9e83ea23cf173034bd32c2f6924391c3008ccd"}
Sep 18 14:14:14.883214 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:14:14.883017    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-explicit-e11b4c-676cdf8446-67jbs"
Sep 18 14:14:14.885192 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:14:14.885167    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-explicit-e11b4c-6876b46f7c-xxvlx" event={"ID":"e2db5cb4-9648-441c-9808-dd89944fd867","Type":"ContainerStarted","Data":"8bd89752f8a2eed9f68a84cb374bc3b53f1c3f047d63c20bca9a556d46c92e85"}
Sep 18 14:14:14.885292 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:14:14.885215    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-explicit-e11b4c-6876b46f7c-xxvlx" event={"ID":"e2db5cb4-9648-441c-9808-dd89944fd867","Type":"ContainerStarted","Data":"a43ca13bb813db4a0fada037650cbee309acc38f5cbfea6ff0cf6d7adae0091b"}
Sep 18 14:14:14.885852 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:14:14.885677    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-explicit-e11b4c-6876b46f7c-xxvlx"
Sep 18 14:14:14.887151 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:14:14.887129    2565 generic.go:356] "Generic (PLEG): container finished" podID="6a27bdb8-3bfe-4f7e-afdb-ff20c1874d2d" containerID="bb744c520826a045a320f78f9d6f70b84ebe8617e50cdd9b45cf85b67331ec26" exitCode=0
Sep 18 14:14:14.887250 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:14:14.887172    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-derive-a721b5af-5bcf9c6877-7qmm9" event={"ID":"6a27bdb8-3bfe-4f7e-afdb-ff20c1874d2d","Type":"ContainerDied","Data":"bb744c520826a045a320f78f9d6f70b84ebe8617e50cdd9b45cf85b67331ec26"}
Sep 18 14:14:14.887250 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:14:14.887214    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-derive-a721b5af-5bcf9c6877-7qmm9" event={"ID":"6a27bdb8-3bfe-4f7e-afdb-ff20c1874d2d","Type":"ContainerDied","Data":"f4931274b067cf1d453e6daaa0c0b2eb55a2983442d7ee8f01669f59ffd47cda"}
Sep 18 14:14:14.887250 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:14:14.887173    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-derive-a721b5af-5bcf9c6877-7qmm9"
Sep 18 14:14:14.892441 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:14:14.892427    2565 scope.go:117] "RemoveContainer" containerID="33aa4bf200d80ec82f5b0dc5722ebe009a37d64da59c09a70aca5c7331fff4b9"
Sep 18 14:14:14.892740 ip-10-0-129-16 kubenswrapper[2565]: E0918 14:14:14.892715    2565 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"33aa4bf200d80ec82f5b0dc5722ebe009a37d64da59c09a70aca5c7331fff4b9\": container with ID starting with 33aa4bf200d80ec82f5b0dc5722ebe009a37d64da59c09a70aca5c7331fff4b9 not found: ID does not exist" containerID="33aa4bf200d80ec82f5b0dc5722ebe009a37d64da59c09a70aca5c7331fff4b9"
Sep 18 14:14:14.892847 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:14:14.892748    2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"33aa4bf200d80ec82f5b0dc5722ebe009a37d64da59c09a70aca5c7331fff4b9"} err="failed to get container status \"33aa4bf200d80ec82f5b0dc5722ebe009a37d64da59c09a70aca5c7331fff4b9\": rpc error: code = NotFound desc = could not find container \"33aa4bf200d80ec82f5b0dc5722ebe009a37d64da59c09a70aca5c7331fff4b9\": container with ID starting with 33aa4bf200d80ec82f5b0dc5722ebe009a37d64da59c09a70aca5c7331fff4b9 not found: ID does not exist"
Sep 18 14:14:14.892847 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:14:14.892766    2565 scope.go:117] "RemoveContainer" containerID="bb744c520826a045a320f78f9d6f70b84ebe8617e50cdd9b45cf85b67331ec26"
Sep 18 14:14:14.902890 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:14:14.902873    2565 scope.go:117] "RemoveContainer" containerID="bb744c520826a045a320f78f9d6f70b84ebe8617e50cdd9b45cf85b67331ec26"
Sep 18 14:14:14.903292 ip-10-0-129-16 kubenswrapper[2565]: E0918 14:14:14.903272    2565 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"bb744c520826a045a320f78f9d6f70b84ebe8617e50cdd9b45cf85b67331ec26\": container with ID starting with bb744c520826a045a320f78f9d6f70b84ebe8617e50cdd9b45cf85b67331ec26 not found: ID does not exist" containerID="bb744c520826a045a320f78f9d6f70b84ebe8617e50cdd9b45cf85b67331ec26"
Sep 18 14:14:14.903350 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:14:14.903300    2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"bb744c520826a045a320f78f9d6f70b84ebe8617e50cdd9b45cf85b67331ec26"} err="failed to get container status \"bb744c520826a045a320f78f9d6f70b84ebe8617e50cdd9b45cf85b67331ec26\": rpc error: code = NotFound desc = could not find container \"bb744c520826a045a320f78f9d6f70b84ebe8617e50cdd9b45cf85b67331ec26\": container with ID starting with bb744c520826a045a320f78f9d6f70b84ebe8617e50cdd9b45cf85b67331ec26 not found: ID does not exist"
Sep 18 14:14:14.907217 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:14:14.907177    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-explicit-e11b4c-676cdf8446-67jbs" podStartSLOduration=1.9071636029999999 podStartE2EDuration="1.907163603s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 14:14:13 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 14:14:14.905484361 +0000 UTC m=+923.079394020" watchObservedRunningTime="2026-09-18 14:14:14.907163603 +0000 UTC m=+923.081073263"
Sep 18 14:14:14.921485 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:14:14.921460    2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-derive-a721b5af-5bcf9c6877-7qmm9"]
Sep 18 14:14:14.926065 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:14:14.926041    2565 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-derive-a721b5af-5bcf9c6877-7qmm9"]
Sep 18 14:14:14.959462 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:14:14.959358    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-explicit-e11b4c-6876b46f7c-xxvlx" podStartSLOduration=1.9593420639999999 podStartE2EDuration="1.959342064s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 14:14:13 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 14:14:14.958183956 +0000 UTC m=+923.132093639" watchObservedRunningTime="2026-09-18 14:14:14.959342064 +0000 UTC m=+923.133251724"
Sep 18 14:14:14.979321 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:14:14.979141    2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-derive-a721b5af-858fff4c46-lhcg9"]
Sep 18 14:14:14.983717 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:14:14.983690    2565 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-derive-a721b5af-858fff4c46-lhcg9"]
Sep 18 14:14:15.502450 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:14:15.502411    2565 patch_prober.go:28] interesting pod/e2e-derive-a721b5af-openshift-default-7c7cdb57c6-lqd4l container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Sep 18 14:14:15.502710 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:14:15.502672    2565 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-derive-a721b5af-openshift-default-7c7cdb57c6-lqd4l" podUID="a9263603-24b9-44cf-8295-6e42afb3158b" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Sep 18 14:14:15.913411 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:14:15.913379    2565 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="38140408-473f-42d3-bf00-201cbb33c0e6" path="/var/lib/kubelet/pods/38140408-473f-42d3-bf00-201cbb33c0e6/volumes"
Sep 18 14:14:15.913761 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:14:15.913747    2565 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6a27bdb8-3bfe-4f7e-afdb-ff20c1874d2d" path="/var/lib/kubelet/pods/6a27bdb8-3bfe-4f7e-afdb-ff20c1874d2d/volumes"
Sep 18 14:14:19.882607 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:14:19.882545    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-a721b5af-openshift-default-7c7cdb57c6-lqd4l"
Sep 18 14:14:19.927833 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:14:19.927797    2565 generic.go:356] "Generic (PLEG): container finished" podID="a9263603-24b9-44cf-8295-6e42afb3158b" containerID="a2f6de37badbf23b15357c61ea80f60b1e3f645890d0ca33a41691e372ffde76" exitCode=0
Sep 18 14:14:19.928361 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:14:19.927851    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-a721b5af-openshift-default-7c7cdb57c6-lqd4l"
Sep 18 14:14:19.928361 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:14:19.927888    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-a721b5af-openshift-default-7c7cdb57c6-lqd4l" event={"ID":"a9263603-24b9-44cf-8295-6e42afb3158b","Type":"ContainerDied","Data":"a2f6de37badbf23b15357c61ea80f60b1e3f645890d0ca33a41691e372ffde76"}
Sep 18 14:14:19.928361 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:14:19.927959    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-a721b5af-openshift-default-7c7cdb57c6-lqd4l" event={"ID":"a9263603-24b9-44cf-8295-6e42afb3158b","Type":"ContainerDied","Data":"9573381140ccacf2e3fe7d7796b2d79e5416187c645fd34a76532c91927e32fc"}
Sep 18 14:14:19.928361 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:14:19.927992    2565 scope.go:117] "RemoveContainer" containerID="a2f6de37badbf23b15357c61ea80f60b1e3f645890d0ca33a41691e372ffde76"
Sep 18 14:14:19.940163 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:14:19.940140    2565 scope.go:117] "RemoveContainer" containerID="6596ac7e60eb1a0d9a30022246bae2b9e66967b38cf9f83d81c5b84d0f0761c6"
Sep 18 14:14:19.950473 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:14:19.950446    2565 scope.go:117] "RemoveContainer" containerID="a2f6de37badbf23b15357c61ea80f60b1e3f645890d0ca33a41691e372ffde76"
Sep 18 14:14:19.950830 ip-10-0-129-16 kubenswrapper[2565]: E0918 14:14:19.950809    2565 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a2f6de37badbf23b15357c61ea80f60b1e3f645890d0ca33a41691e372ffde76\": container with ID starting with a2f6de37badbf23b15357c61ea80f60b1e3f645890d0ca33a41691e372ffde76 not found: ID does not exist" containerID="a2f6de37badbf23b15357c61ea80f60b1e3f645890d0ca33a41691e372ffde76"
Sep 18 14:14:19.950901 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:14:19.950841    2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a2f6de37badbf23b15357c61ea80f60b1e3f645890d0ca33a41691e372ffde76"} err="failed to get container status \"a2f6de37badbf23b15357c61ea80f60b1e3f645890d0ca33a41691e372ffde76\": rpc error: code = NotFound desc = could not find container \"a2f6de37badbf23b15357c61ea80f60b1e3f645890d0ca33a41691e372ffde76\": container with ID starting with a2f6de37badbf23b15357c61ea80f60b1e3f645890d0ca33a41691e372ffde76 not found: ID does not exist"
Sep 18 14:14:19.950901 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:14:19.950859    2565 scope.go:117] "RemoveContainer" containerID="6596ac7e60eb1a0d9a30022246bae2b9e66967b38cf9f83d81c5b84d0f0761c6"
Sep 18 14:14:19.951186 ip-10-0-129-16 kubenswrapper[2565]: E0918 14:14:19.951162    2565 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6596ac7e60eb1a0d9a30022246bae2b9e66967b38cf9f83d81c5b84d0f0761c6\": container with ID starting with 6596ac7e60eb1a0d9a30022246bae2b9e66967b38cf9f83d81c5b84d0f0761c6 not found: ID does not exist" containerID="6596ac7e60eb1a0d9a30022246bae2b9e66967b38cf9f83d81c5b84d0f0761c6"
Sep 18 14:14:19.951261 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:14:19.951191    2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6596ac7e60eb1a0d9a30022246bae2b9e66967b38cf9f83d81c5b84d0f0761c6"} err="failed to get container status \"6596ac7e60eb1a0d9a30022246bae2b9e66967b38cf9f83d81c5b84d0f0761c6\": rpc error: code = NotFound desc = could not find container \"6596ac7e60eb1a0d9a30022246bae2b9e66967b38cf9f83d81c5b84d0f0761c6\": container with ID starting with 6596ac7e60eb1a0d9a30022246bae2b9e66967b38cf9f83d81c5b84d0f0761c6 not found: ID does not exist"
Sep 18 14:14:20.033629 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:14:20.033594    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a9263603-24b9-44cf-8295-6e42afb3158b-istio-token\") pod \"a9263603-24b9-44cf-8295-6e42afb3158b\" (UID: \"a9263603-24b9-44cf-8295-6e42afb3158b\") "
Sep 18 14:14:20.033819 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:14:20.033650    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a9263603-24b9-44cf-8295-6e42afb3158b-istio-data\") pod \"a9263603-24b9-44cf-8295-6e42afb3158b\" (UID: \"a9263603-24b9-44cf-8295-6e42afb3158b\") "
Sep 18 14:14:20.033819 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:14:20.033679    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-4tzv8\" (UniqueName: \"kubernetes.io/projected/a9263603-24b9-44cf-8295-6e42afb3158b-kube-api-access-4tzv8\") pod \"a9263603-24b9-44cf-8295-6e42afb3158b\" (UID: \"a9263603-24b9-44cf-8295-6e42afb3158b\") "
Sep 18 14:14:20.033819 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:14:20.033730    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a9263603-24b9-44cf-8295-6e42afb3158b-istio-podinfo\") pod \"a9263603-24b9-44cf-8295-6e42afb3158b\" (UID: \"a9263603-24b9-44cf-8295-6e42afb3158b\") "
Sep 18 14:14:20.033819 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:14:20.033777    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a9263603-24b9-44cf-8295-6e42afb3158b-credential-socket\") pod \"a9263603-24b9-44cf-8295-6e42afb3158b\" (UID: \"a9263603-24b9-44cf-8295-6e42afb3158b\") "
Sep 18 14:14:20.034065 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:14:20.033823    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a9263603-24b9-44cf-8295-6e42afb3158b-istio-envoy\") pod \"a9263603-24b9-44cf-8295-6e42afb3158b\" (UID: \"a9263603-24b9-44cf-8295-6e42afb3158b\") "
Sep 18 14:14:20.034065 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:14:20.033873    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a9263603-24b9-44cf-8295-6e42afb3158b-workload-certs\") pod \"a9263603-24b9-44cf-8295-6e42afb3158b\" (UID: \"a9263603-24b9-44cf-8295-6e42afb3158b\") "
Sep 18 14:14:20.034065 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:14:20.033904    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a9263603-24b9-44cf-8295-6e42afb3158b-workload-socket\") pod \"a9263603-24b9-44cf-8295-6e42afb3158b\" (UID: \"a9263603-24b9-44cf-8295-6e42afb3158b\") "
Sep 18 14:14:20.034065 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:14:20.034020    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a9263603-24b9-44cf-8295-6e42afb3158b-istio-data" (OuterVolumeSpecName: "istio-data") pod "a9263603-24b9-44cf-8295-6e42afb3158b" (UID: "a9263603-24b9-44cf-8295-6e42afb3158b"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 18 14:14:20.034275 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:14:20.034159    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a9263603-24b9-44cf-8295-6e42afb3158b-istiod-ca-cert\") pod \"a9263603-24b9-44cf-8295-6e42afb3158b\" (UID: \"a9263603-24b9-44cf-8295-6e42afb3158b\") "
Sep 18 14:14:20.034275 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:14:20.034239    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a9263603-24b9-44cf-8295-6e42afb3158b-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "a9263603-24b9-44cf-8295-6e42afb3158b" (UID: "a9263603-24b9-44cf-8295-6e42afb3158b"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 18 14:14:20.034375 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:14:20.034326    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a9263603-24b9-44cf-8295-6e42afb3158b-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "a9263603-24b9-44cf-8295-6e42afb3158b" (UID: "a9263603-24b9-44cf-8295-6e42afb3158b"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 18 14:14:20.034375 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:14:20.034338    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a9263603-24b9-44cf-8295-6e42afb3158b-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "a9263603-24b9-44cf-8295-6e42afb3158b" (UID: "a9263603-24b9-44cf-8295-6e42afb3158b"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 18 14:14:20.035126 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:14:20.034616    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a9263603-24b9-44cf-8295-6e42afb3158b-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "a9263603-24b9-44cf-8295-6e42afb3158b" (UID: "a9263603-24b9-44cf-8295-6e42afb3158b"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 18 14:14:20.035126 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:14:20.034746    2565 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a9263603-24b9-44cf-8295-6e42afb3158b-istiod-ca-cert\") on node \"ip-10-0-129-16.ec2.internal\" DevicePath \"\""
Sep 18 14:14:20.035126 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:14:20.034765    2565 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a9263603-24b9-44cf-8295-6e42afb3158b-istio-data\") on node \"ip-10-0-129-16.ec2.internal\" DevicePath \"\""
Sep 18 14:14:20.035126 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:14:20.034780    2565 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a9263603-24b9-44cf-8295-6e42afb3158b-credential-socket\") on node \"ip-10-0-129-16.ec2.internal\" DevicePath \"\""
Sep 18 14:14:20.035126 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:14:20.034795    2565 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a9263603-24b9-44cf-8295-6e42afb3158b-workload-certs\") on node \"ip-10-0-129-16.ec2.internal\" DevicePath \"\""
Sep 18 14:14:20.035126 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:14:20.034809    2565 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a9263603-24b9-44cf-8295-6e42afb3158b-workload-socket\") on node \"ip-10-0-129-16.ec2.internal\" DevicePath \"\""
Sep 18 14:14:20.036491 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:14:20.036456    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a9263603-24b9-44cf-8295-6e42afb3158b-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "a9263603-24b9-44cf-8295-6e42afb3158b" (UID: "a9263603-24b9-44cf-8295-6e42afb3158b"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 18 14:14:20.036593 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:14:20.036527    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/a9263603-24b9-44cf-8295-6e42afb3158b-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "a9263603-24b9-44cf-8295-6e42afb3158b" (UID: "a9263603-24b9-44cf-8295-6e42afb3158b"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 18 14:14:20.036593 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:14:20.036554    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a9263603-24b9-44cf-8295-6e42afb3158b-kube-api-access-4tzv8" (OuterVolumeSpecName: "kube-api-access-4tzv8") pod "a9263603-24b9-44cf-8295-6e42afb3158b" (UID: "a9263603-24b9-44cf-8295-6e42afb3158b"). InnerVolumeSpecName "kube-api-access-4tzv8". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 18 14:14:20.036674 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:14:20.036595    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a9263603-24b9-44cf-8295-6e42afb3158b-istio-token" (OuterVolumeSpecName: "istio-token") pod "a9263603-24b9-44cf-8295-6e42afb3158b" (UID: "a9263603-24b9-44cf-8295-6e42afb3158b"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 18 14:14:20.135860 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:14:20.135775    2565 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a9263603-24b9-44cf-8295-6e42afb3158b-istio-podinfo\") on node \"ip-10-0-129-16.ec2.internal\" DevicePath \"\""
Sep 18 14:14:20.135860 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:14:20.135803    2565 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a9263603-24b9-44cf-8295-6e42afb3158b-istio-envoy\") on node \"ip-10-0-129-16.ec2.internal\" DevicePath \"\""
Sep 18 14:14:20.135860 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:14:20.135811    2565 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a9263603-24b9-44cf-8295-6e42afb3158b-istio-token\") on node \"ip-10-0-129-16.ec2.internal\" DevicePath \"\""
Sep 18 14:14:20.135860 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:14:20.135819    2565 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4tzv8\" (UniqueName: \"kubernetes.io/projected/a9263603-24b9-44cf-8295-6e42afb3158b-kube-api-access-4tzv8\") on node \"ip-10-0-129-16.ec2.internal\" DevicePath \"\""
Sep 18 14:14:20.252219 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:14:20.252189    2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-derive-a721b5af-openshift-default-7c7cdb57c6-lqd4l"]
Sep 18 14:14:20.256381 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:14:20.256352    2565 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-derive-a721b5af-openshift-default-7c7cdb57c6-lqd4l"]
Sep 18 14:14:21.913840 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:14:21.913805    2565 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a9263603-24b9-44cf-8295-6e42afb3158b" path="/var/lib/kubelet/pods/a9263603-24b9-44cf-8295-6e42afb3158b/volumes"
Sep 18 14:14:25.897631 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:14:25.897595    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-explicit-e11b4c-676cdf8446-67jbs"
Sep 18 14:14:25.898101 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:14:25.897664    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-explicit-e11b4c-6876b46f7c-xxvlx"
Sep 18 14:14:45.837198 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:14:45.837164    2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["e2e-models-e2e-worker-w4-93827d/premium-w4-93827d-kserve-78c986f74c-hjr2v"]
Sep 18 14:14:45.837969 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:14:45.837947    2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6a27bdb8-3bfe-4f7e-afdb-ff20c1874d2d" containerName="payload-processing"
Sep 18 14:14:45.838018 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:14:45.837975    2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="6a27bdb8-3bfe-4f7e-afdb-ff20c1874d2d" containerName="payload-processing"
Sep 18 14:14:45.838018 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:14:45.837989    2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a9263603-24b9-44cf-8295-6e42afb3158b" containerName="istio-proxy"
Sep 18 14:14:45.838018 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:14:45.837998    2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="a9263603-24b9-44cf-8295-6e42afb3158b" containerName="istio-proxy"
Sep 18 14:14:45.838136 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:14:45.838061    2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="38140408-473f-42d3-bf00-201cbb33c0e6" containerName="payload-pre-processing"
Sep 18 14:14:45.838136 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:14:45.838071    2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="38140408-473f-42d3-bf00-201cbb33c0e6" containerName="payload-pre-processing"
Sep 18 14:14:45.838221 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:14:45.838205    2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="38140408-473f-42d3-bf00-201cbb33c0e6" containerName="payload-pre-processing"
Sep 18 14:14:45.838258 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:14:45.838222    2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="a9263603-24b9-44cf-8295-6e42afb3158b" containerName="istio-proxy"
Sep 18 14:14:45.838258 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:14:45.838230    2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="6a27bdb8-3bfe-4f7e-afdb-ff20c1874d2d" containerName="payload-processing"
Sep 18 14:14:45.838258 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:14:45.838241    2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="a9263603-24b9-44cf-8295-6e42afb3158b" containerName="istio-proxy"
Sep 18 14:14:45.841437 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:14:45.841421    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w4-93827d/premium-w4-93827d-kserve-78c986f74c-hjr2v"
Sep 18 14:14:45.844452 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:14:45.844429    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w4-93827d\"/\"kube-root-ca.crt\""
Sep 18 14:14:45.845536 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:14:45.845510    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w4-93827d\"/\"default-dockercfg-nlc42\""
Sep 18 14:14:45.845665 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:14:45.845537    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w4-93827d\"/\"openshift-service-ca.crt\""
Sep 18 14:14:45.845665 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:14:45.845581    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w4-93827d\"/\"premium-w4-93827d-kserve-self-signed-certs\""
Sep 18 14:14:45.854674 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:14:45.854649    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w4-93827d/premium-w4-93827d-kserve-78c986f74c-hjr2v"]
Sep 18 14:14:45.899078 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:14:45.899017    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6pqbc\" (UniqueName: \"kubernetes.io/projected/a1f4563b-6154-48d3-b22f-400041b7795e-kube-api-access-6pqbc\") pod \"premium-w4-93827d-kserve-78c986f74c-hjr2v\" (UID: \"a1f4563b-6154-48d3-b22f-400041b7795e\") " pod="e2e-models-e2e-worker-w4-93827d/premium-w4-93827d-kserve-78c986f74c-hjr2v"
Sep 18 14:14:45.899245 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:14:45.899102    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a1f4563b-6154-48d3-b22f-400041b7795e-tls-certs\") pod \"premium-w4-93827d-kserve-78c986f74c-hjr2v\" (UID: \"a1f4563b-6154-48d3-b22f-400041b7795e\") " pod="e2e-models-e2e-worker-w4-93827d/premium-w4-93827d-kserve-78c986f74c-hjr2v"
Sep 18 14:14:45.899245 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:14:45.899150    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a1f4563b-6154-48d3-b22f-400041b7795e-home\") pod \"premium-w4-93827d-kserve-78c986f74c-hjr2v\" (UID: \"a1f4563b-6154-48d3-b22f-400041b7795e\") " pod="e2e-models-e2e-worker-w4-93827d/premium-w4-93827d-kserve-78c986f74c-hjr2v"
Sep 18 14:14:45.899245 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:14:45.899183    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a1f4563b-6154-48d3-b22f-400041b7795e-dshm\") pod \"premium-w4-93827d-kserve-78c986f74c-hjr2v\" (UID: \"a1f4563b-6154-48d3-b22f-400041b7795e\") " pod="e2e-models-e2e-worker-w4-93827d/premium-w4-93827d-kserve-78c986f74c-hjr2v"
Sep 18 14:14:45.899245 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:14:45.899222    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a1f4563b-6154-48d3-b22f-400041b7795e-model-cache\") pod \"premium-w4-93827d-kserve-78c986f74c-hjr2v\" (UID: \"a1f4563b-6154-48d3-b22f-400041b7795e\") " pod="e2e-models-e2e-worker-w4-93827d/premium-w4-93827d-kserve-78c986f74c-hjr2v"
Sep 18 14:14:45.899390 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:14:45.899250    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a1f4563b-6154-48d3-b22f-400041b7795e-tmp-dir\") pod \"premium-w4-93827d-kserve-78c986f74c-hjr2v\" (UID: \"a1f4563b-6154-48d3-b22f-400041b7795e\") " pod="e2e-models-e2e-worker-w4-93827d/premium-w4-93827d-kserve-78c986f74c-hjr2v"
Sep 18 14:14:46.000261 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:14:46.000230    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6pqbc\" (UniqueName: \"kubernetes.io/projected/a1f4563b-6154-48d3-b22f-400041b7795e-kube-api-access-6pqbc\") pod \"premium-w4-93827d-kserve-78c986f74c-hjr2v\" (UID: \"a1f4563b-6154-48d3-b22f-400041b7795e\") " pod="e2e-models-e2e-worker-w4-93827d/premium-w4-93827d-kserve-78c986f74c-hjr2v"
Sep 18 14:14:46.000463 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:14:46.000271    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a1f4563b-6154-48d3-b22f-400041b7795e-tls-certs\") pod \"premium-w4-93827d-kserve-78c986f74c-hjr2v\" (UID: \"a1f4563b-6154-48d3-b22f-400041b7795e\") " pod="e2e-models-e2e-worker-w4-93827d/premium-w4-93827d-kserve-78c986f74c-hjr2v"
Sep 18 14:14:46.000582 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:14:46.000551    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a1f4563b-6154-48d3-b22f-400041b7795e-home\") pod \"premium-w4-93827d-kserve-78c986f74c-hjr2v\" (UID: \"a1f4563b-6154-48d3-b22f-400041b7795e\") " pod="e2e-models-e2e-worker-w4-93827d/premium-w4-93827d-kserve-78c986f74c-hjr2v"
Sep 18 14:14:46.000649 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:14:46.000629    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a1f4563b-6154-48d3-b22f-400041b7795e-dshm\") pod \"premium-w4-93827d-kserve-78c986f74c-hjr2v\" (UID: \"a1f4563b-6154-48d3-b22f-400041b7795e\") " pod="e2e-models-e2e-worker-w4-93827d/premium-w4-93827d-kserve-78c986f74c-hjr2v"
Sep 18 14:14:46.000756 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:14:46.000729    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a1f4563b-6154-48d3-b22f-400041b7795e-model-cache\") pod \"premium-w4-93827d-kserve-78c986f74c-hjr2v\" (UID: \"a1f4563b-6154-48d3-b22f-400041b7795e\") " pod="e2e-models-e2e-worker-w4-93827d/premium-w4-93827d-kserve-78c986f74c-hjr2v"
Sep 18 14:14:46.000819 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:14:46.000792    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a1f4563b-6154-48d3-b22f-400041b7795e-tmp-dir\") pod \"premium-w4-93827d-kserve-78c986f74c-hjr2v\" (UID: \"a1f4563b-6154-48d3-b22f-400041b7795e\") " pod="e2e-models-e2e-worker-w4-93827d/premium-w4-93827d-kserve-78c986f74c-hjr2v"
Sep 18 14:14:46.000948 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:14:46.000927    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a1f4563b-6154-48d3-b22f-400041b7795e-home\") pod \"premium-w4-93827d-kserve-78c986f74c-hjr2v\" (UID: \"a1f4563b-6154-48d3-b22f-400041b7795e\") " pod="e2e-models-e2e-worker-w4-93827d/premium-w4-93827d-kserve-78c986f74c-hjr2v"
Sep 18 14:14:46.001094 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:14:46.001048    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a1f4563b-6154-48d3-b22f-400041b7795e-model-cache\") pod \"premium-w4-93827d-kserve-78c986f74c-hjr2v\" (UID: \"a1f4563b-6154-48d3-b22f-400041b7795e\") " pod="e2e-models-e2e-worker-w4-93827d/premium-w4-93827d-kserve-78c986f74c-hjr2v"
Sep 18 14:14:46.001158 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:14:46.001142    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a1f4563b-6154-48d3-b22f-400041b7795e-tmp-dir\") pod \"premium-w4-93827d-kserve-78c986f74c-hjr2v\" (UID: \"a1f4563b-6154-48d3-b22f-400041b7795e\") " pod="e2e-models-e2e-worker-w4-93827d/premium-w4-93827d-kserve-78c986f74c-hjr2v"
Sep 18 14:14:46.002700 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:14:46.002676    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a1f4563b-6154-48d3-b22f-400041b7795e-dshm\") pod \"premium-w4-93827d-kserve-78c986f74c-hjr2v\" (UID: \"a1f4563b-6154-48d3-b22f-400041b7795e\") " pod="e2e-models-e2e-worker-w4-93827d/premium-w4-93827d-kserve-78c986f74c-hjr2v"
Sep 18 14:14:46.002969 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:14:46.002947    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a1f4563b-6154-48d3-b22f-400041b7795e-tls-certs\") pod \"premium-w4-93827d-kserve-78c986f74c-hjr2v\" (UID: \"a1f4563b-6154-48d3-b22f-400041b7795e\") " pod="e2e-models-e2e-worker-w4-93827d/premium-w4-93827d-kserve-78c986f74c-hjr2v"
Sep 18 14:14:46.011536 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:14:46.011510    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6pqbc\" (UniqueName: \"kubernetes.io/projected/a1f4563b-6154-48d3-b22f-400041b7795e-kube-api-access-6pqbc\") pod \"premium-w4-93827d-kserve-78c986f74c-hjr2v\" (UID: \"a1f4563b-6154-48d3-b22f-400041b7795e\") " pod="e2e-models-e2e-worker-w4-93827d/premium-w4-93827d-kserve-78c986f74c-hjr2v"
Sep 18 14:14:46.151168 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:14:46.151085    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w4-93827d/premium-w4-93827d-kserve-78c986f74c-hjr2v"
Sep 18 14:14:46.431216 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:14:46.430583    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w4-93827d/premium-w4-93827d-kserve-78c986f74c-hjr2v"]
Sep 18 14:14:46.432381 ip-10-0-129-16 kubenswrapper[2565]: W0918 14:14:46.432352    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-poda1f4563b_6154_48d3_b22f_400041b7795e.slice/crio-7b40e899307dc2f8a807d4797d4e5c7d0851011046d89983cff538452c056fde WatchSource:0}: Error finding container 7b40e899307dc2f8a807d4797d4e5c7d0851011046d89983cff538452c056fde: Status 404 returned error can't find the container with id 7b40e899307dc2f8a807d4797d4e5c7d0851011046d89983cff538452c056fde
Sep 18 14:14:47.090080 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:14:47.090048    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w4-93827d/premium-w4-93827d-kserve-78c986f74c-hjr2v" event={"ID":"a1f4563b-6154-48d3-b22f-400041b7795e","Type":"ContainerStarted","Data":"1c55fca471057ef537adfcdd5036d5fb69d1e357eadc227b24c0127f07404019"}
Sep 18 14:14:47.090471 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:14:47.090090    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w4-93827d/premium-w4-93827d-kserve-78c986f74c-hjr2v" event={"ID":"a1f4563b-6154-48d3-b22f-400041b7795e","Type":"ContainerStarted","Data":"7b40e899307dc2f8a807d4797d4e5c7d0851011046d89983cff538452c056fde"}
Sep 18 14:14:47.125139 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:14:47.125081    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="e2e-models-e2e-worker-w4-93827d/premium-w4-93827d-kserve-78c986f74c-hjr2v" podStartSLOduration=2.125062775 podStartE2EDuration="2.125062775s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 14:14:45 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 14:14:47.122317195 +0000 UTC m=+955.296226874" watchObservedRunningTime="2026-09-18 14:14:47.125062775 +0000 UTC m=+955.298972464"
Sep 18 14:14:56.151954 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:14:56.151921    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w4-93827d/premium-w4-93827d-kserve-78c986f74c-hjr2v"
Sep 18 14:14:56.152360 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:14:56.152005    2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-models-e2e-worker-w4-93827d/premium-w4-93827d-kserve-78c986f74c-hjr2v"
Sep 18 14:14:56.164342 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:14:56.164314    2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-models-e2e-worker-w4-93827d/premium-w4-93827d-kserve-78c986f74c-hjr2v"
Sep 18 14:14:57.165716 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:14:57.165684    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w4-93827d/premium-w4-93827d-kserve-78c986f74c-hjr2v"
Sep 18 14:15:00.159191 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:15:00.159152    2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-b-ee8a08-29829015-6jvrd"]
Sep 18 14:15:00.159865 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:15:00.159843    2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a9263603-24b9-44cf-8295-6e42afb3158b" containerName="istio-proxy"
Sep 18 14:15:00.159961 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:15:00.159867    2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="a9263603-24b9-44cf-8295-6e42afb3158b" containerName="istio-proxy"
Sep 18 14:15:00.164696 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:15:00.164673    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-b-ee8a08-29829015-6jvrd"
Sep 18 14:15:00.168456 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:15:00.168430    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-l4fn9\""
Sep 18 14:15:00.170675 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:15:00.170654    2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w4-93827d-29829015-hvlhq"]
Sep 18 14:15:00.175139 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:15:00.175115    2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-a-dc2cee-29829015-sl88h"]
Sep 18 14:15:00.175241 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:15:00.175218    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w4-93827d-29829015-hvlhq"
Sep 18 14:15:00.177476 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:15:00.177455    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-b-ee8a08-29829015-6jvrd"
Sep 18 14:15:00.179229 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:15:00.179206    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-a-dc2cee-29829015-sl88h"
Sep 18 14:15:00.179971 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:15:00.179946    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-b-ee8a08-29829015-6jvrd"]
Sep 18 14:15:00.196011 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:15:00.195764    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w4-93827d-29829015-hvlhq"]
Sep 18 14:15:00.209660 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:15:00.209612    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-a-dc2cee-29829015-sl88h"]
Sep 18 14:15:00.220678 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:15:00.220520    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w4-93827d-29829015-hvlhq"
Sep 18 14:15:00.241399 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:15:00.241367    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-a-dc2cee-29829015-sl88h"
Sep 18 14:15:00.482021 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:15:00.481996    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-b-ee8a08-29829015-6jvrd"]
Sep 18 14:15:00.483257 ip-10-0-129-16 kubenswrapper[2565]: W0918 14:15:00.483232    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd424cda9_e346_4c36_b2db_b2dc210bbabf.slice/crio-33177c844e9134d08c7e45961059912c030349e5814085564d1c94a80644008c WatchSource:0}: Error finding container 33177c844e9134d08c7e45961059912c030349e5814085564d1c94a80644008c: Status 404 returned error can't find the container with id 33177c844e9134d08c7e45961059912c030349e5814085564d1c94a80644008c
Sep 18 14:15:00.526498 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:15:00.526466    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w4-93827d-29829015-hvlhq"]
Sep 18 14:15:00.527167 ip-10-0-129-16 kubenswrapper[2565]: W0918 14:15:00.527140    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3019edca_2c9a_4eca_ba96_0530a5bc8213.slice/crio-82689b36dc3b1dc93c4c82984048622796a4ae47a7dda7e62be6105fdc4830ae WatchSource:0}: Error finding container 82689b36dc3b1dc93c4c82984048622796a4ae47a7dda7e62be6105fdc4830ae: Status 404 returned error can't find the container with id 82689b36dc3b1dc93c4c82984048622796a4ae47a7dda7e62be6105fdc4830ae
Sep 18 14:15:00.552840 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:15:00.552762    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-a-dc2cee-29829015-sl88h"]
Sep 18 14:15:00.556217 ip-10-0-129-16 kubenswrapper[2565]: W0918 14:15:00.556191    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd495891f_f94b_49e8_8391_1a9df17c1e82.slice/crio-54e6bf3c1137ae1c7cddf98ea653a2749ea53bf94b816caa0cdf5eeaa52c6970 WatchSource:0}: Error finding container 54e6bf3c1137ae1c7cddf98ea653a2749ea53bf94b816caa0cdf5eeaa52c6970: Status 404 returned error can't find the container with id 54e6bf3c1137ae1c7cddf98ea653a2749ea53bf94b816caa0cdf5eeaa52c6970
Sep 18 14:15:01.181380 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:15:01.181345    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w4-93827d-29829015-hvlhq" event={"ID":"3019edca-2c9a-4eca-ba96-0530a5bc8213","Type":"ContainerStarted","Data":"2603ebb591e13a58b6bd18d4d33958f45ebe30cfd64969bdba8f25972cd9356c"}
Sep 18 14:15:01.181867 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:15:01.181406    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w4-93827d-29829015-hvlhq" event={"ID":"3019edca-2c9a-4eca-ba96-0530a5bc8213","Type":"ContainerStarted","Data":"82689b36dc3b1dc93c4c82984048622796a4ae47a7dda7e62be6105fdc4830ae"}
Sep 18 14:15:01.182891 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:15:01.182862    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-a-dc2cee-29829015-sl88h" event={"ID":"d495891f-f94b-49e8-8391-1a9df17c1e82","Type":"ContainerStarted","Data":"6cc6684730a09fb270c3bf20681f6e71a1942f949df164ef9e0e585961042aa1"}
Sep 18 14:15:01.183015 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:15:01.182914    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-a-dc2cee-29829015-sl88h" event={"ID":"d495891f-f94b-49e8-8391-1a9df17c1e82","Type":"ContainerStarted","Data":"54e6bf3c1137ae1c7cddf98ea653a2749ea53bf94b816caa0cdf5eeaa52c6970"}
Sep 18 14:15:01.184401 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:15:01.184380    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-b-ee8a08-29829015-6jvrd" event={"ID":"d424cda9-e346-4c36-b2db-b2dc210bbabf","Type":"ContainerStarted","Data":"21cd300749400e8ba8c036160c8edce1732f502c195869023ada3f31398e8b69"}
Sep 18 14:15:01.184642 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:15:01.184421    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-b-ee8a08-29829015-6jvrd" event={"ID":"d424cda9-e346-4c36-b2db-b2dc210bbabf","Type":"ContainerStarted","Data":"33177c844e9134d08c7e45961059912c030349e5814085564d1c94a80644008c"}
Sep 18 14:15:01.227390 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:15:01.227331    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w4-93827d-29829015-hvlhq" podStartSLOduration=1.227311389 podStartE2EDuration="1.227311389s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 14:15:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 14:15:01.226212409 +0000 UTC m=+969.400122069" watchObservedRunningTime="2026-09-18 14:15:01.227311389 +0000 UTC m=+969.401221048"
Sep 18 14:15:01.263480 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:15:01.263421    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-a-dc2cee-29829015-sl88h" podStartSLOduration=1.263403861 podStartE2EDuration="1.263403861s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 14:15:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 14:15:01.257110594 +0000 UTC m=+969.431020286" watchObservedRunningTime="2026-09-18 14:15:01.263403861 +0000 UTC m=+969.437313518"
Sep 18 14:15:01.305716 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:15:01.305662    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-b-ee8a08-29829015-6jvrd" podStartSLOduration=1.305643905 podStartE2EDuration="1.305643905s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 14:15:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 14:15:01.298911547 +0000 UTC m=+969.472821208" watchObservedRunningTime="2026-09-18 14:15:01.305643905 +0000 UTC m=+969.479553609"
Sep 18 14:15:02.192505 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:15:02.192471    2565 generic.go:356] "Generic (PLEG): container finished" podID="d495891f-f94b-49e8-8391-1a9df17c1e82" containerID="6cc6684730a09fb270c3bf20681f6e71a1942f949df164ef9e0e585961042aa1" exitCode=0
Sep 18 14:15:02.193061 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:15:02.192559    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-a-dc2cee-29829015-sl88h" event={"ID":"d495891f-f94b-49e8-8391-1a9df17c1e82","Type":"ContainerDied","Data":"6cc6684730a09fb270c3bf20681f6e71a1942f949df164ef9e0e585961042aa1"}
Sep 18 14:15:02.194176 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:15:02.194154    2565 generic.go:356] "Generic (PLEG): container finished" podID="d424cda9-e346-4c36-b2db-b2dc210bbabf" containerID="21cd300749400e8ba8c036160c8edce1732f502c195869023ada3f31398e8b69" exitCode=0
Sep 18 14:15:02.194275 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:15:02.194184    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-b-ee8a08-29829015-6jvrd" event={"ID":"d424cda9-e346-4c36-b2db-b2dc210bbabf","Type":"ContainerDied","Data":"21cd300749400e8ba8c036160c8edce1732f502c195869023ada3f31398e8b69"}
Sep 18 14:15:02.195826 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:15:02.195801    2565 generic.go:356] "Generic (PLEG): container finished" podID="3019edca-2c9a-4eca-ba96-0530a5bc8213" containerID="2603ebb591e13a58b6bd18d4d33958f45ebe30cfd64969bdba8f25972cd9356c" exitCode=0
Sep 18 14:15:02.195928 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:15:02.195835    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w4-93827d-29829015-hvlhq" event={"ID":"3019edca-2c9a-4eca-ba96-0530a5bc8213","Type":"ContainerDied","Data":"2603ebb591e13a58b6bd18d4d33958f45ebe30cfd64969bdba8f25972cd9356c"}
Sep 18 14:15:03.367906 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:15:03.367882    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-b-ee8a08-29829015-6jvrd"
Sep 18 14:15:03.414642 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:15:03.414614    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w4-93827d-29829015-hvlhq"
Sep 18 14:15:03.418734 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:15:03.418708    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-a-dc2cee-29829015-sl88h"
Sep 18 14:15:04.210345 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:15:04.210253    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w4-93827d-29829015-hvlhq"
Sep 18 14:15:04.210345 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:15:04.210257    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w4-93827d-29829015-hvlhq" event={"ID":"3019edca-2c9a-4eca-ba96-0530a5bc8213","Type":"ContainerDied","Data":"82689b36dc3b1dc93c4c82984048622796a4ae47a7dda7e62be6105fdc4830ae"}
Sep 18 14:15:04.210585 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:15:04.210387    2565 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="82689b36dc3b1dc93c4c82984048622796a4ae47a7dda7e62be6105fdc4830ae"
Sep 18 14:15:04.211714 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:15:04.211694    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-a-dc2cee-29829015-sl88h"
Sep 18 14:15:04.211852 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:15:04.211736    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-a-dc2cee-29829015-sl88h" event={"ID":"d495891f-f94b-49e8-8391-1a9df17c1e82","Type":"ContainerDied","Data":"54e6bf3c1137ae1c7cddf98ea653a2749ea53bf94b816caa0cdf5eeaa52c6970"}
Sep 18 14:15:04.211852 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:15:04.211786    2565 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="54e6bf3c1137ae1c7cddf98ea653a2749ea53bf94b816caa0cdf5eeaa52c6970"
Sep 18 14:15:04.213096 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:15:04.213075    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-b-ee8a08-29829015-6jvrd" event={"ID":"d424cda9-e346-4c36-b2db-b2dc210bbabf","Type":"ContainerDied","Data":"33177c844e9134d08c7e45961059912c030349e5814085564d1c94a80644008c"}
Sep 18 14:15:04.213219 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:15:04.213105    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-b-ee8a08-29829015-6jvrd"
Sep 18 14:15:04.213219 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:15:04.213109    2565 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="33177c844e9134d08c7e45961059912c030349e5814085564d1c94a80644008c"
Sep 18 14:15:51.011823 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:15:51.011785    2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-explicit-e11b4c-vk8wq"]
Sep 18 14:15:51.012484 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:15:51.012301    2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3019edca-2c9a-4eca-ba96-0530a5bc8213" containerName="cleanup"
Sep 18 14:15:51.012484 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:15:51.012313    2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="3019edca-2c9a-4eca-ba96-0530a5bc8213" containerName="cleanup"
Sep 18 14:15:51.012484 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:15:51.012341    2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d424cda9-e346-4c36-b2db-b2dc210bbabf" containerName="cleanup"
Sep 18 14:15:51.012484 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:15:51.012347    2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="d424cda9-e346-4c36-b2db-b2dc210bbabf" containerName="cleanup"
Sep 18 14:15:51.012484 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:15:51.012356    2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d495891f-f94b-49e8-8391-1a9df17c1e82" containerName="cleanup"
Sep 18 14:15:51.012484 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:15:51.012361    2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="d495891f-f94b-49e8-8391-1a9df17c1e82" containerName="cleanup"
Sep 18 14:15:51.012484 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:15:51.012424    2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="d495891f-f94b-49e8-8391-1a9df17c1e82" containerName="cleanup"
Sep 18 14:15:51.012484 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:15:51.012433    2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="3019edca-2c9a-4eca-ba96-0530a5bc8213" containerName="cleanup"
Sep 18 14:15:51.012484 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:15:51.012443    2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="d424cda9-e346-4c36-b2db-b2dc210bbabf" containerName="cleanup"
Sep 18 14:15:51.015822 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:15:51.015800    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-explicit-e11b4c-vk8wq"
Sep 18 14:15:51.019105 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:15:51.019080    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-l4fn9\""
Sep 18 14:15:51.034081 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:15:51.034048    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-explicit-e11b4c-vk8wq"]
Sep 18 14:15:51.101110 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:15:51.101071    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/7b61942d-945c-453a-8958-9b6dd921fe69-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-explicit-e11b4c-vk8wq\" (UID: \"7b61942d-945c-453a-8958-9b6dd921fe69\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-explicit-e11b4c-vk8wq"
Sep 18 14:15:51.202709 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:15:51.202663    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/7b61942d-945c-453a-8958-9b6dd921fe69-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-explicit-e11b4c-vk8wq\" (UID: \"7b61942d-945c-453a-8958-9b6dd921fe69\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-explicit-e11b4c-vk8wq"
Sep 18 14:15:51.203445 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:15:51.203422    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/7b61942d-945c-453a-8958-9b6dd921fe69-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-explicit-e11b4c-vk8wq\" (UID: \"7b61942d-945c-453a-8958-9b6dd921fe69\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-explicit-e11b4c-vk8wq"
Sep 18 14:15:51.327101 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:15:51.327060    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-explicit-e11b4c-vk8wq"
Sep 18 14:15:51.630961 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:15:51.630935    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-explicit-e11b4c-vk8wq"]
Sep 18 14:15:51.632566 ip-10-0-129-16 kubenswrapper[2565]: W0918 14:15:51.632539    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7b61942d_945c_453a_8958_9b6dd921fe69.slice/crio-1dd46c8ab7ea7f51236368c4f5688e563e3cdbacec593b23c731e9eebfdeaf11 WatchSource:0}: Error finding container 1dd46c8ab7ea7f51236368c4f5688e563e3cdbacec593b23c731e9eebfdeaf11: Status 404 returned error can't find the container with id 1dd46c8ab7ea7f51236368c4f5688e563e3cdbacec593b23c731e9eebfdeaf11
Sep 18 14:15:52.505422 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:15:52.505383    2565 generic.go:356] "Generic (PLEG): container finished" podID="7b61942d-945c-453a-8958-9b6dd921fe69" containerID="7a40cac592febadff6a178266c369a249df8225900737755c947dc9c94854acd" exitCode=0
Sep 18 14:15:52.505913 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:15:52.505468    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-explicit-e11b4c-vk8wq" event={"ID":"7b61942d-945c-453a-8958-9b6dd921fe69","Type":"ContainerDied","Data":"7a40cac592febadff6a178266c369a249df8225900737755c947dc9c94854acd"}
Sep 18 14:15:52.505913 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:15:52.505527    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-explicit-e11b4c-vk8wq" event={"ID":"7b61942d-945c-453a-8958-9b6dd921fe69","Type":"ContainerStarted","Data":"1dd46c8ab7ea7f51236368c4f5688e563e3cdbacec593b23c731e9eebfdeaf11"}
Sep 18 14:15:53.654805 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:15:53.654776    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-explicit-e11b4c-vk8wq"
Sep 18 14:15:53.729814 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:15:53.729786    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/7b61942d-945c-453a-8958-9b6dd921fe69-maas-api-service-ca\") pod \"7b61942d-945c-453a-8958-9b6dd921fe69\" (UID: \"7b61942d-945c-453a-8958-9b6dd921fe69\") "
Sep 18 14:15:53.730140 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:15:53.730120    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7b61942d-945c-453a-8958-9b6dd921fe69-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "7b61942d-945c-453a-8958-9b6dd921fe69" (UID: "7b61942d-945c-453a-8958-9b6dd921fe69"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 18 14:15:53.831665 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:15:53.831632    2565 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/7b61942d-945c-453a-8958-9b6dd921fe69-maas-api-service-ca\") on node \"ip-10-0-129-16.ec2.internal\" DevicePath \"\""
Sep 18 14:15:54.520233 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:15:54.520200    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-explicit-e11b4c-vk8wq"
Sep 18 14:15:54.520417 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:15:54.520241    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-explicit-e11b4c-vk8wq" event={"ID":"7b61942d-945c-453a-8958-9b6dd921fe69","Type":"ContainerDied","Data":"1dd46c8ab7ea7f51236368c4f5688e563e3cdbacec593b23c731e9eebfdeaf11"}
Sep 18 14:15:54.520417 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:15:54.520298    2565 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="1dd46c8ab7ea7f51236368c4f5688e563e3cdbacec593b23c731e9eebfdeaf11"
Sep 18 14:16:06.952254 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:16:06.952213    2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-834b7e2a-596bb74754-pc98g"]
Sep 18 14:16:06.952911 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:16:06.952892    2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7b61942d-945c-453a-8958-9b6dd921fe69" containerName="revoke-keys"
Sep 18 14:16:06.952911 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:16:06.952912    2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="7b61942d-945c-453a-8958-9b6dd921fe69" containerName="revoke-keys"
Sep 18 14:16:06.953098 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:16:06.953027    2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="7b61942d-945c-453a-8958-9b6dd921fe69" containerName="revoke-keys"
Sep 18 14:16:06.956878 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:16:06.956860    2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-834b7e2a-5df45fd99c-4t8l6"]
Sep 18 14:16:06.957000 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:16:06.956982    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-834b7e2a-596bb74754-pc98g"
Sep 18 14:16:06.960418 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:16:06.960401    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-834b7e2a-5df45fd99c-4t8l6"
Sep 18 14:16:06.960610 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:16:06.960593    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-mt-834b7e2a\""
Sep 18 14:16:06.977061 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:16:06.977010    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-834b7e2a-596bb74754-pc98g"]
Sep 18 14:16:06.980366 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:16:06.980343    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-834b7e2a-5df45fd99c-4t8l6"]
Sep 18 14:16:06.981497 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:16:06.981471    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e6b5a27e-d202-4a8f-a7fe-1175a692d2cd-plugins-config-volume\") pod \"payload-pre-processing-e2e-mt-834b7e2a-596bb74754-pc98g\" (UID: \"e6b5a27e-d202-4a8f-a7fe-1175a692d2cd\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-834b7e2a-596bb74754-pc98g"
Sep 18 14:16:06.981596 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:16:06.981514    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fc7lf\" (UniqueName: \"kubernetes.io/projected/e6b5a27e-d202-4a8f-a7fe-1175a692d2cd-kube-api-access-fc7lf\") pod \"payload-pre-processing-e2e-mt-834b7e2a-596bb74754-pc98g\" (UID: \"e6b5a27e-d202-4a8f-a7fe-1175a692d2cd\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-834b7e2a-596bb74754-pc98g"
Sep 18 14:16:07.082451 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:16:07.082408    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e6b5a27e-d202-4a8f-a7fe-1175a692d2cd-plugins-config-volume\") pod \"payload-pre-processing-e2e-mt-834b7e2a-596bb74754-pc98g\" (UID: \"e6b5a27e-d202-4a8f-a7fe-1175a692d2cd\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-834b7e2a-596bb74754-pc98g"
Sep 18 14:16:07.082648 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:16:07.082468    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fc7lf\" (UniqueName: \"kubernetes.io/projected/e6b5a27e-d202-4a8f-a7fe-1175a692d2cd-kube-api-access-fc7lf\") pod \"payload-pre-processing-e2e-mt-834b7e2a-596bb74754-pc98g\" (UID: \"e6b5a27e-d202-4a8f-a7fe-1175a692d2cd\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-834b7e2a-596bb74754-pc98g"
Sep 18 14:16:07.082648 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:16:07.082543    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/4d019c34-dbcb-49b0-8758-8673c8d82b68-plugins-config-volume\") pod \"payload-processing-e2e-mt-834b7e2a-5df45fd99c-4t8l6\" (UID: \"4d019c34-dbcb-49b0-8758-8673c8d82b68\") " pod="openshift-ingress/payload-processing-e2e-mt-834b7e2a-5df45fd99c-4t8l6"
Sep 18 14:16:07.082783 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:16:07.082677    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-m6pq4\" (UniqueName: \"kubernetes.io/projected/4d019c34-dbcb-49b0-8758-8673c8d82b68-kube-api-access-m6pq4\") pod \"payload-processing-e2e-mt-834b7e2a-5df45fd99c-4t8l6\" (UID: \"4d019c34-dbcb-49b0-8758-8673c8d82b68\") " pod="openshift-ingress/payload-processing-e2e-mt-834b7e2a-5df45fd99c-4t8l6"
Sep 18 14:16:07.083775 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:16:07.083742    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e6b5a27e-d202-4a8f-a7fe-1175a692d2cd-plugins-config-volume\") pod \"payload-pre-processing-e2e-mt-834b7e2a-596bb74754-pc98g\" (UID: \"e6b5a27e-d202-4a8f-a7fe-1175a692d2cd\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-834b7e2a-596bb74754-pc98g"
Sep 18 14:16:07.101338 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:16:07.101306    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fc7lf\" (UniqueName: \"kubernetes.io/projected/e6b5a27e-d202-4a8f-a7fe-1175a692d2cd-kube-api-access-fc7lf\") pod \"payload-pre-processing-e2e-mt-834b7e2a-596bb74754-pc98g\" (UID: \"e6b5a27e-d202-4a8f-a7fe-1175a692d2cd\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-834b7e2a-596bb74754-pc98g"
Sep 18 14:16:07.183889 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:16:07.183850    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-m6pq4\" (UniqueName: \"kubernetes.io/projected/4d019c34-dbcb-49b0-8758-8673c8d82b68-kube-api-access-m6pq4\") pod \"payload-processing-e2e-mt-834b7e2a-5df45fd99c-4t8l6\" (UID: \"4d019c34-dbcb-49b0-8758-8673c8d82b68\") " pod="openshift-ingress/payload-processing-e2e-mt-834b7e2a-5df45fd99c-4t8l6"
Sep 18 14:16:07.184122 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:16:07.183941    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/4d019c34-dbcb-49b0-8758-8673c8d82b68-plugins-config-volume\") pod \"payload-processing-e2e-mt-834b7e2a-5df45fd99c-4t8l6\" (UID: \"4d019c34-dbcb-49b0-8758-8673c8d82b68\") " pod="openshift-ingress/payload-processing-e2e-mt-834b7e2a-5df45fd99c-4t8l6"
Sep 18 14:16:07.184650 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:16:07.184629    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/4d019c34-dbcb-49b0-8758-8673c8d82b68-plugins-config-volume\") pod \"payload-processing-e2e-mt-834b7e2a-5df45fd99c-4t8l6\" (UID: \"4d019c34-dbcb-49b0-8758-8673c8d82b68\") " pod="openshift-ingress/payload-processing-e2e-mt-834b7e2a-5df45fd99c-4t8l6"
Sep 18 14:16:07.195322 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:16:07.195298    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-m6pq4\" (UniqueName: \"kubernetes.io/projected/4d019c34-dbcb-49b0-8758-8673c8d82b68-kube-api-access-m6pq4\") pod \"payload-processing-e2e-mt-834b7e2a-5df45fd99c-4t8l6\" (UID: \"4d019c34-dbcb-49b0-8758-8673c8d82b68\") " pod="openshift-ingress/payload-processing-e2e-mt-834b7e2a-5df45fd99c-4t8l6"
Sep 18 14:16:07.216798 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:16:07.216698    2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-explicit-e11b4c-6876b46f7c-xxvlx"]
Sep 18 14:16:07.216975 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:16:07.216945    2565 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-explicit-e11b4c-6876b46f7c-xxvlx" podUID="e2db5cb4-9648-441c-9808-dd89944fd867" containerName="payload-processing" containerID="cri-o://8bd89752f8a2eed9f68a84cb374bc3b53f1c3f047d63c20bca9a556d46c92e85" gracePeriod=30
Sep 18 14:16:07.225848 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:16:07.225743    2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-explicit-e11b4c-676cdf8446-67jbs"]
Sep 18 14:16:07.226024 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:16:07.225990    2565 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-explicit-e11b4c-676cdf8446-67jbs" podUID="5abb7bbd-f229-4637-8e52-b7e2041c0a22" containerName="payload-pre-processing" containerID="cri-o://f8185aa32f74ac08cfa5e8376e3fc8f88ccf6a9c35619793b51df5812b2bc08f" gracePeriod=30
Sep 18 14:16:07.269146 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:16:07.269112    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-834b7e2a-596bb74754-pc98g"
Sep 18 14:16:07.276329 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:16:07.276188    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-834b7e2a-5df45fd99c-4t8l6"
Sep 18 14:16:07.481507 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:16:07.481379    2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-explicit-e11b4c-vk8wq"]
Sep 18 14:16:07.485552 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:16:07.485519    2565 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-explicit-e11b4c-vk8wq"]
Sep 18 14:16:07.569109 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:16:07.569082    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-explicit-e11b4c-6876b46f7c-xxvlx"
Sep 18 14:16:07.579224 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:16:07.579203    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-explicit-e11b4c-676cdf8446-67jbs"
Sep 18 14:16:07.585184 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:16:07.585047    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-834b7e2a-5df45fd99c-4t8l6"]
Sep 18 14:16:07.588012 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:16:07.587990    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-834b7e2a-596bb74754-pc98g"]
Sep 18 14:16:07.588349 ip-10-0-129-16 kubenswrapper[2565]: W0918 14:16:07.588331    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode6b5a27e_d202_4a8f_a7fe_1175a692d2cd.slice/crio-7a0538fdcd6a935195eccb0f27c4942ee654e032bcb9e76ec36ebda8bd0bf190 WatchSource:0}: Error finding container 7a0538fdcd6a935195eccb0f27c4942ee654e032bcb9e76ec36ebda8bd0bf190: Status 404 returned error can't find the container with id 7a0538fdcd6a935195eccb0f27c4942ee654e032bcb9e76ec36ebda8bd0bf190
Sep 18 14:16:07.589019 ip-10-0-129-16 kubenswrapper[2565]: W0918 14:16:07.588999    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4d019c34_dbcb_49b0_8758_8673c8d82b68.slice/crio-76c227430ba456be95629357ab4e131673627f4cca9ca6e36d877220b2a7fc2f WatchSource:0}: Error finding container 76c227430ba456be95629357ab4e131673627f4cca9ca6e36d877220b2a7fc2f: Status 404 returned error can't find the container with id 76c227430ba456be95629357ab4e131673627f4cca9ca6e36d877220b2a7fc2f
Sep 18 14:16:07.604081 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:16:07.604056    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-834b7e2a-596bb74754-pc98g" event={"ID":"e6b5a27e-d202-4a8f-a7fe-1175a692d2cd","Type":"ContainerStarted","Data":"7a0538fdcd6a935195eccb0f27c4942ee654e032bcb9e76ec36ebda8bd0bf190"}
Sep 18 14:16:07.605417 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:16:07.605391    2565 generic.go:356] "Generic (PLEG): container finished" podID="5abb7bbd-f229-4637-8e52-b7e2041c0a22" containerID="f8185aa32f74ac08cfa5e8376e3fc8f88ccf6a9c35619793b51df5812b2bc08f" exitCode=0
Sep 18 14:16:07.605506 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:16:07.605436    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-explicit-e11b4c-676cdf8446-67jbs"
Sep 18 14:16:07.605506 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:16:07.605471    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-explicit-e11b4c-676cdf8446-67jbs" event={"ID":"5abb7bbd-f229-4637-8e52-b7e2041c0a22","Type":"ContainerDied","Data":"f8185aa32f74ac08cfa5e8376e3fc8f88ccf6a9c35619793b51df5812b2bc08f"}
Sep 18 14:16:07.605641 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:16:07.605517    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-explicit-e11b4c-676cdf8446-67jbs" event={"ID":"5abb7bbd-f229-4637-8e52-b7e2041c0a22","Type":"ContainerDied","Data":"c9fce9008b2d4f78071235084c9e83ea23cf173034bd32c2f6924391c3008ccd"}
Sep 18 14:16:07.605641 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:16:07.605547    2565 scope.go:117] "RemoveContainer" containerID="f8185aa32f74ac08cfa5e8376e3fc8f88ccf6a9c35619793b51df5812b2bc08f"
Sep 18 14:16:07.607054 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:16:07.607008    2565 generic.go:356] "Generic (PLEG): container finished" podID="e2db5cb4-9648-441c-9808-dd89944fd867" containerID="8bd89752f8a2eed9f68a84cb374bc3b53f1c3f047d63c20bca9a556d46c92e85" exitCode=0
Sep 18 14:16:07.607156 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:16:07.607076    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-explicit-e11b4c-6876b46f7c-xxvlx" event={"ID":"e2db5cb4-9648-441c-9808-dd89944fd867","Type":"ContainerDied","Data":"8bd89752f8a2eed9f68a84cb374bc3b53f1c3f047d63c20bca9a556d46c92e85"}
Sep 18 14:16:07.607156 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:16:07.607129    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-explicit-e11b4c-6876b46f7c-xxvlx" event={"ID":"e2db5cb4-9648-441c-9808-dd89944fd867","Type":"ContainerDied","Data":"a43ca13bb813db4a0fada037650cbee309acc38f5cbfea6ff0cf6d7adae0091b"}
Sep 18 14:16:07.607156 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:16:07.607148    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-explicit-e11b4c-6876b46f7c-xxvlx"
Sep 18 14:16:07.608844 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:16:07.608824    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-834b7e2a-5df45fd99c-4t8l6" event={"ID":"4d019c34-dbcb-49b0-8758-8673c8d82b68","Type":"ContainerStarted","Data":"76c227430ba456be95629357ab4e131673627f4cca9ca6e36d877220b2a7fc2f"}
Sep 18 14:16:07.616221 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:16:07.616199    2565 scope.go:117] "RemoveContainer" containerID="f8185aa32f74ac08cfa5e8376e3fc8f88ccf6a9c35619793b51df5812b2bc08f"
Sep 18 14:16:07.616515 ip-10-0-129-16 kubenswrapper[2565]: E0918 14:16:07.616494    2565 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f8185aa32f74ac08cfa5e8376e3fc8f88ccf6a9c35619793b51df5812b2bc08f\": container with ID starting with f8185aa32f74ac08cfa5e8376e3fc8f88ccf6a9c35619793b51df5812b2bc08f not found: ID does not exist" containerID="f8185aa32f74ac08cfa5e8376e3fc8f88ccf6a9c35619793b51df5812b2bc08f"
Sep 18 14:16:07.616594 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:16:07.616521    2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f8185aa32f74ac08cfa5e8376e3fc8f88ccf6a9c35619793b51df5812b2bc08f"} err="failed to get container status \"f8185aa32f74ac08cfa5e8376e3fc8f88ccf6a9c35619793b51df5812b2bc08f\": rpc error: code = NotFound desc = could not find container \"f8185aa32f74ac08cfa5e8376e3fc8f88ccf6a9c35619793b51df5812b2bc08f\": container with ID starting with f8185aa32f74ac08cfa5e8376e3fc8f88ccf6a9c35619793b51df5812b2bc08f not found: ID does not exist"
Sep 18 14:16:07.616594 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:16:07.616536    2565 scope.go:117] "RemoveContainer" containerID="8bd89752f8a2eed9f68a84cb374bc3b53f1c3f047d63c20bca9a556d46c92e85"
Sep 18 14:16:07.644345 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:16:07.640924    2565 scope.go:117] "RemoveContainer" containerID="8bd89752f8a2eed9f68a84cb374bc3b53f1c3f047d63c20bca9a556d46c92e85"
Sep 18 14:16:07.644345 ip-10-0-129-16 kubenswrapper[2565]: E0918 14:16:07.641319    2565 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8bd89752f8a2eed9f68a84cb374bc3b53f1c3f047d63c20bca9a556d46c92e85\": container with ID starting with 8bd89752f8a2eed9f68a84cb374bc3b53f1c3f047d63c20bca9a556d46c92e85 not found: ID does not exist" containerID="8bd89752f8a2eed9f68a84cb374bc3b53f1c3f047d63c20bca9a556d46c92e85"
Sep 18 14:16:07.644345 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:16:07.641357    2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8bd89752f8a2eed9f68a84cb374bc3b53f1c3f047d63c20bca9a556d46c92e85"} err="failed to get container status \"8bd89752f8a2eed9f68a84cb374bc3b53f1c3f047d63c20bca9a556d46c92e85\": rpc error: code = NotFound desc = could not find container \"8bd89752f8a2eed9f68a84cb374bc3b53f1c3f047d63c20bca9a556d46c92e85\": container with ID starting with 8bd89752f8a2eed9f68a84cb374bc3b53f1c3f047d63c20bca9a556d46c92e85 not found: ID does not exist"
Sep 18 14:16:07.689474 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:16:07.689438    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e2db5cb4-9648-441c-9808-dd89944fd867-plugins-config-volume\") pod \"e2db5cb4-9648-441c-9808-dd89944fd867\" (UID: \"e2db5cb4-9648-441c-9808-dd89944fd867\") "
Sep 18 14:16:07.689753 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:16:07.689734    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-66jrg\" (UniqueName: \"kubernetes.io/projected/5abb7bbd-f229-4637-8e52-b7e2041c0a22-kube-api-access-66jrg\") pod \"5abb7bbd-f229-4637-8e52-b7e2041c0a22\" (UID: \"5abb7bbd-f229-4637-8e52-b7e2041c0a22\") "
Sep 18 14:16:07.689979 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:16:07.689961    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-cfh8g\" (UniqueName: \"kubernetes.io/projected/e2db5cb4-9648-441c-9808-dd89944fd867-kube-api-access-cfh8g\") pod \"e2db5cb4-9648-441c-9808-dd89944fd867\" (UID: \"e2db5cb4-9648-441c-9808-dd89944fd867\") "
Sep 18 14:16:07.691716 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:16:07.691214    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/5abb7bbd-f229-4637-8e52-b7e2041c0a22-plugins-config-volume\") pod \"5abb7bbd-f229-4637-8e52-b7e2041c0a22\" (UID: \"5abb7bbd-f229-4637-8e52-b7e2041c0a22\") "
Sep 18 14:16:07.693375 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:16:07.689994    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e2db5cb4-9648-441c-9808-dd89944fd867-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "e2db5cb4-9648-441c-9808-dd89944fd867" (UID: "e2db5cb4-9648-441c-9808-dd89944fd867"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 18 14:16:07.693375 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:16:07.691678    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5abb7bbd-f229-4637-8e52-b7e2041c0a22-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "5abb7bbd-f229-4637-8e52-b7e2041c0a22" (UID: "5abb7bbd-f229-4637-8e52-b7e2041c0a22"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 18 14:16:07.693728 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:16:07.693706    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e2db5cb4-9648-441c-9808-dd89944fd867-kube-api-access-cfh8g" (OuterVolumeSpecName: "kube-api-access-cfh8g") pod "e2db5cb4-9648-441c-9808-dd89944fd867" (UID: "e2db5cb4-9648-441c-9808-dd89944fd867"). InnerVolumeSpecName "kube-api-access-cfh8g". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 18 14:16:07.694344 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:16:07.694321    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5abb7bbd-f229-4637-8e52-b7e2041c0a22-kube-api-access-66jrg" (OuterVolumeSpecName: "kube-api-access-66jrg") pod "5abb7bbd-f229-4637-8e52-b7e2041c0a22" (UID: "5abb7bbd-f229-4637-8e52-b7e2041c0a22"). InnerVolumeSpecName "kube-api-access-66jrg". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 18 14:16:07.794824 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:16:07.794780    2565 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e2db5cb4-9648-441c-9808-dd89944fd867-plugins-config-volume\") on node \"ip-10-0-129-16.ec2.internal\" DevicePath \"\""
Sep 18 14:16:07.795072 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:16:07.794845    2565 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-66jrg\" (UniqueName: \"kubernetes.io/projected/5abb7bbd-f229-4637-8e52-b7e2041c0a22-kube-api-access-66jrg\") on node \"ip-10-0-129-16.ec2.internal\" DevicePath \"\""
Sep 18 14:16:07.795072 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:16:07.794859    2565 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-cfh8g\" (UniqueName: \"kubernetes.io/projected/e2db5cb4-9648-441c-9808-dd89944fd867-kube-api-access-cfh8g\") on node \"ip-10-0-129-16.ec2.internal\" DevicePath \"\""
Sep 18 14:16:07.795072 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:16:07.794872    2565 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/5abb7bbd-f229-4637-8e52-b7e2041c0a22-plugins-config-volume\") on node \"ip-10-0-129-16.ec2.internal\" DevicePath \"\""
Sep 18 14:16:07.916772 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:16:07.916743    2565 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7b61942d-945c-453a-8958-9b6dd921fe69" path="/var/lib/kubelet/pods/7b61942d-945c-453a-8958-9b6dd921fe69/volumes"
Sep 18 14:16:07.937962 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:16:07.937933    2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-explicit-e11b4c-676cdf8446-67jbs"]
Sep 18 14:16:07.944748 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:16:07.944720    2565 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-explicit-e11b4c-676cdf8446-67jbs"]
Sep 18 14:16:07.968318 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:16:07.968286    2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-explicit-e11b4c-6876b46f7c-xxvlx"]
Sep 18 14:16:07.982528 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:16:07.982502    2565 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-explicit-e11b4c-6876b46f7c-xxvlx"]
Sep 18 14:16:08.623347 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:16:08.623310    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-834b7e2a-5df45fd99c-4t8l6" event={"ID":"4d019c34-dbcb-49b0-8758-8673c8d82b68","Type":"ContainerStarted","Data":"500c7aa0f6837051a01405ca18738ed5e78ae18a9abe242eb4dfd911d14fbd22"}
Sep 18 14:16:08.623596 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:16:08.623576    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-mt-834b7e2a-5df45fd99c-4t8l6"
Sep 18 14:16:08.625071 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:16:08.625042    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-834b7e2a-596bb74754-pc98g" event={"ID":"e6b5a27e-d202-4a8f-a7fe-1175a692d2cd","Type":"ContainerStarted","Data":"d8019da3d65232c06506df6f2213e1a9e93fecfff0ed8db5bdb107d3fd15cd72"}
Sep 18 14:16:08.625198 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:16:08.625103    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-mt-834b7e2a-596bb74754-pc98g"
Sep 18 14:16:08.653536 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:16:08.653486    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-mt-834b7e2a-5df45fd99c-4t8l6" podStartSLOduration=2.653470036 podStartE2EDuration="2.653470036s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 14:16:06 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 14:16:08.651202762 +0000 UTC m=+1036.825112450" watchObservedRunningTime="2026-09-18 14:16:08.653470036 +0000 UTC m=+1036.827379748"
Sep 18 14:16:08.683237 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:16:08.683186    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-mt-834b7e2a-596bb74754-pc98g" podStartSLOduration=2.683169442 podStartE2EDuration="2.683169442s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 14:16:06 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 14:16:08.679680899 +0000 UTC m=+1036.853590577" watchObservedRunningTime="2026-09-18 14:16:08.683169442 +0000 UTC m=+1036.857079101"
Sep 18 14:16:09.914507 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:16:09.914479    2565 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5abb7bbd-f229-4637-8e52-b7e2041c0a22" path="/var/lib/kubelet/pods/5abb7bbd-f229-4637-8e52-b7e2041c0a22/volumes"
Sep 18 14:16:09.914915 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:16:09.914899    2565 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e2db5cb4-9648-441c-9808-dd89944fd867" path="/var/lib/kubelet/pods/e2db5cb4-9648-441c-9808-dd89944fd867/volumes"
Sep 18 14:16:19.634584 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:16:19.634547    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-mt-834b7e2a-596bb74754-pc98g"
Sep 18 14:16:19.635058 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:16:19.634634    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-mt-834b7e2a-5df45fd99c-4t8l6"
Sep 18 14:16:38.953170 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:16:38.953093    2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-834b7e2a-k6krg"]
Sep 18 14:16:38.953634 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:16:38.953620    2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5abb7bbd-f229-4637-8e52-b7e2041c0a22" containerName="payload-pre-processing"
Sep 18 14:16:38.953680 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:16:38.953636    2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="5abb7bbd-f229-4637-8e52-b7e2041c0a22" containerName="payload-pre-processing"
Sep 18 14:16:38.953680 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:16:38.953661    2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e2db5cb4-9648-441c-9808-dd89944fd867" containerName="payload-processing"
Sep 18 14:16:38.953680 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:16:38.953666    2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="e2db5cb4-9648-441c-9808-dd89944fd867" containerName="payload-processing"
Sep 18 14:16:38.953779 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:16:38.953744    2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="5abb7bbd-f229-4637-8e52-b7e2041c0a22" containerName="payload-pre-processing"
Sep 18 14:16:38.953779 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:16:38.953754    2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="e2db5cb4-9648-441c-9808-dd89944fd867" containerName="payload-processing"
Sep 18 14:16:38.957113 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:16:38.957090    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-834b7e2a-k6krg"
Sep 18 14:16:38.959851 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:16:38.959831    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-l4fn9\""
Sep 18 14:16:38.966458 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:16:38.966433    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-834b7e2a-k6krg"]
Sep 18 14:16:39.027501 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:16:39.027464    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/c64413e0-f710-4417-8495-2308fb7f2551-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-834b7e2a-k6krg\" (UID: \"c64413e0-f710-4417-8495-2308fb7f2551\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-834b7e2a-k6krg"
Sep 18 14:16:39.129023 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:16:39.128985    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/c64413e0-f710-4417-8495-2308fb7f2551-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-834b7e2a-k6krg\" (UID: \"c64413e0-f710-4417-8495-2308fb7f2551\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-834b7e2a-k6krg"
Sep 18 14:16:39.129581 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:16:39.129560    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/c64413e0-f710-4417-8495-2308fb7f2551-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-834b7e2a-k6krg\" (UID: \"c64413e0-f710-4417-8495-2308fb7f2551\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-834b7e2a-k6krg"
Sep 18 14:16:39.270072 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:16:39.269946    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-834b7e2a-k6krg"
Sep 18 14:16:39.561841 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:16:39.561816    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-834b7e2a-k6krg"]
Sep 18 14:16:39.563855 ip-10-0-129-16 kubenswrapper[2565]: W0918 14:16:39.563823    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc64413e0_f710_4417_8495_2308fb7f2551.slice/crio-85b15417bac27e5ad51a5182ab9c3baf17df822271ec4777ac089fc48b482684 WatchSource:0}: Error finding container 85b15417bac27e5ad51a5182ab9c3baf17df822271ec4777ac089fc48b482684: Status 404 returned error can't find the container with id 85b15417bac27e5ad51a5182ab9c3baf17df822271ec4777ac089fc48b482684
Sep 18 14:16:39.831691 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:16:39.831605    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-834b7e2a-k6krg" event={"ID":"c64413e0-f710-4417-8495-2308fb7f2551","Type":"ContainerStarted","Data":"85b15417bac27e5ad51a5182ab9c3baf17df822271ec4777ac089fc48b482684"}
Sep 18 14:16:40.838863 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:16:40.838827    2565 generic.go:356] "Generic (PLEG): container finished" podID="c64413e0-f710-4417-8495-2308fb7f2551" containerID="01f60abb94b3e30d29f7148984bcb6ea44b049c6cbb79972c5ce6c50015a5430" exitCode=0
Sep 18 14:16:40.839282 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:16:40.838916    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-834b7e2a-k6krg" event={"ID":"c64413e0-f710-4417-8495-2308fb7f2551","Type":"ContainerDied","Data":"01f60abb94b3e30d29f7148984bcb6ea44b049c6cbb79972c5ce6c50015a5430"}
Sep 18 14:16:42.007649 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:16:42.007619    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-834b7e2a-k6krg"
Sep 18 14:16:42.065978 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:16:42.065939    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/c64413e0-f710-4417-8495-2308fb7f2551-maas-api-service-ca\") pod \"c64413e0-f710-4417-8495-2308fb7f2551\" (UID: \"c64413e0-f710-4417-8495-2308fb7f2551\") "
Sep 18 14:16:42.066329 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:16:42.066305    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c64413e0-f710-4417-8495-2308fb7f2551-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "c64413e0-f710-4417-8495-2308fb7f2551" (UID: "c64413e0-f710-4417-8495-2308fb7f2551"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 18 14:16:42.066548 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:16:42.066527    2565 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/c64413e0-f710-4417-8495-2308fb7f2551-maas-api-service-ca\") on node \"ip-10-0-129-16.ec2.internal\" DevicePath \"\""
Sep 18 14:16:42.852377 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:16:42.852347    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-834b7e2a-k6krg"
Sep 18 14:16:42.852553 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:16:42.852435    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-834b7e2a-k6krg" event={"ID":"c64413e0-f710-4417-8495-2308fb7f2551","Type":"ContainerDied","Data":"85b15417bac27e5ad51a5182ab9c3baf17df822271ec4777ac089fc48b482684"}
Sep 18 14:16:42.852553 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:16:42.852479    2565 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="85b15417bac27e5ad51a5182ab9c3baf17df822271ec4777ac089fc48b482684"
Sep 18 14:16:51.056631 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:16:51.056592    2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-834b7e2a-5df45fd99c-4t8l6"]
Sep 18 14:16:51.057604 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:16:51.057568    2565 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-mt-834b7e2a-5df45fd99c-4t8l6" podUID="4d019c34-dbcb-49b0-8758-8673c8d82b68" containerName="payload-processing" containerID="cri-o://500c7aa0f6837051a01405ca18738ed5e78ae18a9abe242eb4dfd911d14fbd22" gracePeriod=30
Sep 18 14:16:51.090560 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:16:51.090533    2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-834b7e2a-596bb74754-pc98g"]
Sep 18 14:16:51.090832 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:16:51.090805    2565 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-mt-834b7e2a-596bb74754-pc98g" podUID="e6b5a27e-d202-4a8f-a7fe-1175a692d2cd" containerName="payload-pre-processing" containerID="cri-o://d8019da3d65232c06506df6f2213e1a9e93fecfff0ed8db5bdb107d3fd15cd72" gracePeriod=30
Sep 18 14:16:51.362027 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:16:51.361988    2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-834b7e2a-k6krg"]
Sep 18 14:16:51.366769 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:16:51.366746    2565 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-834b7e2a-k6krg"]
Sep 18 14:16:51.378283 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:16:51.378260    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-834b7e2a-5df45fd99c-4t8l6"
Sep 18 14:16:51.381570 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:16:51.381545    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-834b7e2a-596bb74754-pc98g"
Sep 18 14:16:51.469906 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:16:51.469877    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e6b5a27e-d202-4a8f-a7fe-1175a692d2cd-plugins-config-volume\") pod \"e6b5a27e-d202-4a8f-a7fe-1175a692d2cd\" (UID: \"e6b5a27e-d202-4a8f-a7fe-1175a692d2cd\") "
Sep 18 14:16:51.469906 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:16:51.469908    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-fc7lf\" (UniqueName: \"kubernetes.io/projected/e6b5a27e-d202-4a8f-a7fe-1175a692d2cd-kube-api-access-fc7lf\") pod \"e6b5a27e-d202-4a8f-a7fe-1175a692d2cd\" (UID: \"e6b5a27e-d202-4a8f-a7fe-1175a692d2cd\") "
Sep 18 14:16:51.470194 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:16:51.469949    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/4d019c34-dbcb-49b0-8758-8673c8d82b68-plugins-config-volume\") pod \"4d019c34-dbcb-49b0-8758-8673c8d82b68\" (UID: \"4d019c34-dbcb-49b0-8758-8673c8d82b68\") "
Sep 18 14:16:51.470194 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:16:51.470105    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-m6pq4\" (UniqueName: \"kubernetes.io/projected/4d019c34-dbcb-49b0-8758-8673c8d82b68-kube-api-access-m6pq4\") pod \"4d019c34-dbcb-49b0-8758-8673c8d82b68\" (UID: \"4d019c34-dbcb-49b0-8758-8673c8d82b68\") "
Sep 18 14:16:51.470369 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:16:51.470342    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4d019c34-dbcb-49b0-8758-8673c8d82b68-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "4d019c34-dbcb-49b0-8758-8673c8d82b68" (UID: "4d019c34-dbcb-49b0-8758-8673c8d82b68"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 18 14:16:51.470428 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:16:51.470366    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e6b5a27e-d202-4a8f-a7fe-1175a692d2cd-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "e6b5a27e-d202-4a8f-a7fe-1175a692d2cd" (UID: "e6b5a27e-d202-4a8f-a7fe-1175a692d2cd"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 18 14:16:51.470602 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:16:51.470583    2565 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e6b5a27e-d202-4a8f-a7fe-1175a692d2cd-plugins-config-volume\") on node \"ip-10-0-129-16.ec2.internal\" DevicePath \"\""
Sep 18 14:16:51.470602 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:16:51.470602    2565 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/4d019c34-dbcb-49b0-8758-8673c8d82b68-plugins-config-volume\") on node \"ip-10-0-129-16.ec2.internal\" DevicePath \"\""
Sep 18 14:16:51.472268 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:16:51.472237    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e6b5a27e-d202-4a8f-a7fe-1175a692d2cd-kube-api-access-fc7lf" (OuterVolumeSpecName: "kube-api-access-fc7lf") pod "e6b5a27e-d202-4a8f-a7fe-1175a692d2cd" (UID: "e6b5a27e-d202-4a8f-a7fe-1175a692d2cd"). InnerVolumeSpecName "kube-api-access-fc7lf". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 18 14:16:51.472346 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:16:51.472316    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4d019c34-dbcb-49b0-8758-8673c8d82b68-kube-api-access-m6pq4" (OuterVolumeSpecName: "kube-api-access-m6pq4") pod "4d019c34-dbcb-49b0-8758-8673c8d82b68" (UID: "4d019c34-dbcb-49b0-8758-8673c8d82b68"). InnerVolumeSpecName "kube-api-access-m6pq4". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 18 14:16:51.572306 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:16:51.572270    2565 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-fc7lf\" (UniqueName: \"kubernetes.io/projected/e6b5a27e-d202-4a8f-a7fe-1175a692d2cd-kube-api-access-fc7lf\") on node \"ip-10-0-129-16.ec2.internal\" DevicePath \"\""
Sep 18 14:16:51.572306 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:16:51.572299    2565 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-m6pq4\" (UniqueName: \"kubernetes.io/projected/4d019c34-dbcb-49b0-8758-8673c8d82b68-kube-api-access-m6pq4\") on node \"ip-10-0-129-16.ec2.internal\" DevicePath \"\""
Sep 18 14:16:51.911042 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:16:51.911008    2565 generic.go:356] "Generic (PLEG): container finished" podID="4d019c34-dbcb-49b0-8758-8673c8d82b68" containerID="500c7aa0f6837051a01405ca18738ed5e78ae18a9abe242eb4dfd911d14fbd22" exitCode=0
Sep 18 14:16:51.911252 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:16:51.911118    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-834b7e2a-5df45fd99c-4t8l6"
Sep 18 14:16:51.912666 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:16:51.912641    2565 generic.go:356] "Generic (PLEG): container finished" podID="e6b5a27e-d202-4a8f-a7fe-1175a692d2cd" containerID="d8019da3d65232c06506df6f2213e1a9e93fecfff0ed8db5bdb107d3fd15cd72" exitCode=0
Sep 18 14:16:51.912778 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:16:51.912707    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-834b7e2a-596bb74754-pc98g"
Sep 18 14:16:51.915305 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:16:51.915244    2565 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c64413e0-f710-4417-8495-2308fb7f2551" path="/var/lib/kubelet/pods/c64413e0-f710-4417-8495-2308fb7f2551/volumes"
Sep 18 14:16:51.922055 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:16:51.919581    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-834b7e2a-5df45fd99c-4t8l6" event={"ID":"4d019c34-dbcb-49b0-8758-8673c8d82b68","Type":"ContainerDied","Data":"500c7aa0f6837051a01405ca18738ed5e78ae18a9abe242eb4dfd911d14fbd22"}
Sep 18 14:16:51.922055 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:16:51.919719    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-834b7e2a-5df45fd99c-4t8l6" event={"ID":"4d019c34-dbcb-49b0-8758-8673c8d82b68","Type":"ContainerDied","Data":"76c227430ba456be95629357ab4e131673627f4cca9ca6e36d877220b2a7fc2f"}
Sep 18 14:16:51.922055 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:16:51.919754    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-834b7e2a-596bb74754-pc98g" event={"ID":"e6b5a27e-d202-4a8f-a7fe-1175a692d2cd","Type":"ContainerDied","Data":"d8019da3d65232c06506df6f2213e1a9e93fecfff0ed8db5bdb107d3fd15cd72"}
Sep 18 14:16:51.922055 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:16:51.919857    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-834b7e2a-596bb74754-pc98g" event={"ID":"e6b5a27e-d202-4a8f-a7fe-1175a692d2cd","Type":"ContainerDied","Data":"7a0538fdcd6a935195eccb0f27c4942ee654e032bcb9e76ec36ebda8bd0bf190"}
Sep 18 14:16:51.922055 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:16:51.919931    2565 scope.go:117] "RemoveContainer" containerID="500c7aa0f6837051a01405ca18738ed5e78ae18a9abe242eb4dfd911d14fbd22"
Sep 18 14:16:51.936381 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:16:51.936359    2565 scope.go:117] "RemoveContainer" containerID="500c7aa0f6837051a01405ca18738ed5e78ae18a9abe242eb4dfd911d14fbd22"
Sep 18 14:16:51.936730 ip-10-0-129-16 kubenswrapper[2565]: E0918 14:16:51.936709    2565 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"500c7aa0f6837051a01405ca18738ed5e78ae18a9abe242eb4dfd911d14fbd22\": container with ID starting with 500c7aa0f6837051a01405ca18738ed5e78ae18a9abe242eb4dfd911d14fbd22 not found: ID does not exist" containerID="500c7aa0f6837051a01405ca18738ed5e78ae18a9abe242eb4dfd911d14fbd22"
Sep 18 14:16:51.936801 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:16:51.936738    2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"500c7aa0f6837051a01405ca18738ed5e78ae18a9abe242eb4dfd911d14fbd22"} err="failed to get container status \"500c7aa0f6837051a01405ca18738ed5e78ae18a9abe242eb4dfd911d14fbd22\": rpc error: code = NotFound desc = could not find container \"500c7aa0f6837051a01405ca18738ed5e78ae18a9abe242eb4dfd911d14fbd22\": container with ID starting with 500c7aa0f6837051a01405ca18738ed5e78ae18a9abe242eb4dfd911d14fbd22 not found: ID does not exist"
Sep 18 14:16:51.936801 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:16:51.936756    2565 scope.go:117] "RemoveContainer" containerID="d8019da3d65232c06506df6f2213e1a9e93fecfff0ed8db5bdb107d3fd15cd72"
Sep 18 14:16:51.945658 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:16:51.945637    2565 scope.go:117] "RemoveContainer" containerID="d8019da3d65232c06506df6f2213e1a9e93fecfff0ed8db5bdb107d3fd15cd72"
Sep 18 14:16:51.945966 ip-10-0-129-16 kubenswrapper[2565]: E0918 14:16:51.945943    2565 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d8019da3d65232c06506df6f2213e1a9e93fecfff0ed8db5bdb107d3fd15cd72\": container with ID starting with d8019da3d65232c06506df6f2213e1a9e93fecfff0ed8db5bdb107d3fd15cd72 not found: ID does not exist" containerID="d8019da3d65232c06506df6f2213e1a9e93fecfff0ed8db5bdb107d3fd15cd72"
Sep 18 14:16:51.946013 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:16:51.945974    2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d8019da3d65232c06506df6f2213e1a9e93fecfff0ed8db5bdb107d3fd15cd72"} err="failed to get container status \"d8019da3d65232c06506df6f2213e1a9e93fecfff0ed8db5bdb107d3fd15cd72\": rpc error: code = NotFound desc = could not find container \"d8019da3d65232c06506df6f2213e1a9e93fecfff0ed8db5bdb107d3fd15cd72\": container with ID starting with d8019da3d65232c06506df6f2213e1a9e93fecfff0ed8db5bdb107d3fd15cd72 not found: ID does not exist"
Sep 18 14:16:51.954652 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:16:51.954632    2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-834b7e2a-5df45fd99c-4t8l6"]
Sep 18 14:16:51.958162 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:16:51.958140    2565 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-834b7e2a-5df45fd99c-4t8l6"]
Sep 18 14:16:51.974134 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:16:51.974099    2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-834b7e2a-596bb74754-pc98g"]
Sep 18 14:16:51.981403 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:16:51.981375    2565 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-834b7e2a-596bb74754-pc98g"]
Sep 18 14:16:53.914681 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:16:53.914635    2565 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4d019c34-dbcb-49b0-8758-8673c8d82b68" path="/var/lib/kubelet/pods/4d019c34-dbcb-49b0-8758-8673c8d82b68/volumes"
Sep 18 14:16:53.915249 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:16:53.915227    2565 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e6b5a27e-d202-4a8f-a7fe-1175a692d2cd" path="/var/lib/kubelet/pods/e6b5a27e-d202-4a8f-a7fe-1175a692d2cd/volumes"
Sep 18 14:17:22.356526 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:17:22.356488    2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w2-526b97-7486448b6b-hnk46"]
Sep 18 14:17:22.357005 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:17:22.356987    2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e6b5a27e-d202-4a8f-a7fe-1175a692d2cd" containerName="payload-pre-processing"
Sep 18 14:17:22.357005 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:17:22.357000    2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="e6b5a27e-d202-4a8f-a7fe-1175a692d2cd" containerName="payload-pre-processing"
Sep 18 14:17:22.357104 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:17:22.357014    2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4d019c34-dbcb-49b0-8758-8673c8d82b68" containerName="payload-processing"
Sep 18 14:17:22.357104 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:17:22.357020    2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="4d019c34-dbcb-49b0-8758-8673c8d82b68" containerName="payload-processing"
Sep 18 14:17:22.357104 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:17:22.357047    2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c64413e0-f710-4417-8495-2308fb7f2551" containerName="revoke-keys"
Sep 18 14:17:22.357104 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:17:22.357053    2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="c64413e0-f710-4417-8495-2308fb7f2551" containerName="revoke-keys"
Sep 18 14:17:22.357228 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:17:22.357116    2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="4d019c34-dbcb-49b0-8758-8673c8d82b68" containerName="payload-processing"
Sep 18 14:17:22.357228 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:17:22.357126    2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="c64413e0-f710-4417-8495-2308fb7f2551" containerName="revoke-keys"
Sep 18 14:17:22.357228 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:17:22.357133    2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="e6b5a27e-d202-4a8f-a7fe-1175a692d2cd" containerName="payload-pre-processing"
Sep 18 14:17:22.360236 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:17:22.360221    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-526b97-7486448b6b-hnk46"
Sep 18 14:17:22.363447 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:17:22.363427    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-shared-a-w2-526b97-dockercfg-t4xlt\""
Sep 18 14:17:22.363582 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:17:22.363460    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-shared-a-w2-526b97\""
Sep 18 14:17:22.371290 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:17:22.371250    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w2-526b97-7486448b6b-hnk46"]
Sep 18 14:17:22.423244 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:17:22.423199    2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w2-526b97-7bf5747d47-r2mpw"]
Sep 18 14:17:22.429192 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:17:22.429167    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-a-w2-526b97-7bf5747d47-r2mpw"
Sep 18 14:17:22.443233 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:17:22.443207    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w2-526b97-7bf5747d47-r2mpw"]
Sep 18 14:17:22.496716 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:17:22.496677    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-b5bkh\" (UniqueName: \"kubernetes.io/projected/b26d8b6f-f726-4238-b48a-f4d272cf27b7-kube-api-access-b5bkh\") pod \"payload-processing-e2e-shared-a-w2-526b97-7bf5747d47-r2mpw\" (UID: \"b26d8b6f-f726-4238-b48a-f4d272cf27b7\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w2-526b97-7bf5747d47-r2mpw"
Sep 18 14:17:22.496883 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:17:22.496830    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/8d339523-5739-49e1-aecf-93d3d545c43f-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-a-w2-526b97-7486448b6b-hnk46\" (UID: \"8d339523-5739-49e1-aecf-93d3d545c43f\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-526b97-7486448b6b-hnk46"
Sep 18 14:17:22.496959 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:17:22.496942    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b26d8b6f-f726-4238-b48a-f4d272cf27b7-plugins-config-volume\") pod \"payload-processing-e2e-shared-a-w2-526b97-7bf5747d47-r2mpw\" (UID: \"b26d8b6f-f726-4238-b48a-f4d272cf27b7\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w2-526b97-7bf5747d47-r2mpw"
Sep 18 14:17:22.497019 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:17:22.497007    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6kwzc\" (UniqueName: \"kubernetes.io/projected/8d339523-5739-49e1-aecf-93d3d545c43f-kube-api-access-6kwzc\") pod \"payload-pre-processing-e2e-shared-a-w2-526b97-7486448b6b-hnk46\" (UID: \"8d339523-5739-49e1-aecf-93d3d545c43f\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-526b97-7486448b6b-hnk46"
Sep 18 14:17:22.597701 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:17:22.597660    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b26d8b6f-f726-4238-b48a-f4d272cf27b7-plugins-config-volume\") pod \"payload-processing-e2e-shared-a-w2-526b97-7bf5747d47-r2mpw\" (UID: \"b26d8b6f-f726-4238-b48a-f4d272cf27b7\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w2-526b97-7bf5747d47-r2mpw"
Sep 18 14:17:22.597891 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:17:22.597710    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6kwzc\" (UniqueName: \"kubernetes.io/projected/8d339523-5739-49e1-aecf-93d3d545c43f-kube-api-access-6kwzc\") pod \"payload-pre-processing-e2e-shared-a-w2-526b97-7486448b6b-hnk46\" (UID: \"8d339523-5739-49e1-aecf-93d3d545c43f\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-526b97-7486448b6b-hnk46"
Sep 18 14:17:22.597891 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:17:22.597747    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-b5bkh\" (UniqueName: \"kubernetes.io/projected/b26d8b6f-f726-4238-b48a-f4d272cf27b7-kube-api-access-b5bkh\") pod \"payload-processing-e2e-shared-a-w2-526b97-7bf5747d47-r2mpw\" (UID: \"b26d8b6f-f726-4238-b48a-f4d272cf27b7\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w2-526b97-7bf5747d47-r2mpw"
Sep 18 14:17:22.597891 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:17:22.597821    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/8d339523-5739-49e1-aecf-93d3d545c43f-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-a-w2-526b97-7486448b6b-hnk46\" (UID: \"8d339523-5739-49e1-aecf-93d3d545c43f\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-526b97-7486448b6b-hnk46"
Sep 18 14:17:22.598441 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:17:22.598416    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b26d8b6f-f726-4238-b48a-f4d272cf27b7-plugins-config-volume\") pod \"payload-processing-e2e-shared-a-w2-526b97-7bf5747d47-r2mpw\" (UID: \"b26d8b6f-f726-4238-b48a-f4d272cf27b7\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w2-526b97-7bf5747d47-r2mpw"
Sep 18 14:17:22.598618 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:17:22.598596    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/8d339523-5739-49e1-aecf-93d3d545c43f-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-a-w2-526b97-7486448b6b-hnk46\" (UID: \"8d339523-5739-49e1-aecf-93d3d545c43f\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-526b97-7486448b6b-hnk46"
Sep 18 14:17:22.608799 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:17:22.608737    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-b5bkh\" (UniqueName: \"kubernetes.io/projected/b26d8b6f-f726-4238-b48a-f4d272cf27b7-kube-api-access-b5bkh\") pod \"payload-processing-e2e-shared-a-w2-526b97-7bf5747d47-r2mpw\" (UID: \"b26d8b6f-f726-4238-b48a-f4d272cf27b7\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w2-526b97-7bf5747d47-r2mpw"
Sep 18 14:17:22.608799 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:17:22.608748    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6kwzc\" (UniqueName: \"kubernetes.io/projected/8d339523-5739-49e1-aecf-93d3d545c43f-kube-api-access-6kwzc\") pod \"payload-pre-processing-e2e-shared-a-w2-526b97-7486448b6b-hnk46\" (UID: \"8d339523-5739-49e1-aecf-93d3d545c43f\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-526b97-7486448b6b-hnk46"
Sep 18 14:17:22.672705 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:17:22.672665    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-526b97-7486448b6b-hnk46"
Sep 18 14:17:22.741213 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:17:22.740983    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-a-w2-526b97-7bf5747d47-r2mpw"
Sep 18 14:17:22.951270 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:17:22.951243    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w2-526b97-7486448b6b-hnk46"]
Sep 18 14:17:22.951871 ip-10-0-129-16 kubenswrapper[2565]: W0918 14:17:22.951849    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8d339523_5739_49e1_aecf_93d3d545c43f.slice/crio-f1d784d3f3e068ba7cab108370ee59e527636f18c7fe13c3a4599eb7dcec56df WatchSource:0}: Error finding container f1d784d3f3e068ba7cab108370ee59e527636f18c7fe13c3a4599eb7dcec56df: Status 404 returned error can't find the container with id f1d784d3f3e068ba7cab108370ee59e527636f18c7fe13c3a4599eb7dcec56df
Sep 18 14:17:23.043564 ip-10-0-129-16 kubenswrapper[2565]: W0918 14:17:23.043534    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb26d8b6f_f726_4238_b48a_f4d272cf27b7.slice/crio-1b683c4819ec89997fac58a1b26bb4716521cc2ad1bb3e75a7f05cab35ae54f1 WatchSource:0}: Error finding container 1b683c4819ec89997fac58a1b26bb4716521cc2ad1bb3e75a7f05cab35ae54f1: Status 404 returned error can't find the container with id 1b683c4819ec89997fac58a1b26bb4716521cc2ad1bb3e75a7f05cab35ae54f1
Sep 18 14:17:23.044745 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:17:23.044725    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w2-526b97-7bf5747d47-r2mpw"]
Sep 18 14:17:23.119961 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:17:23.119926    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-a-w2-526b97-7bf5747d47-r2mpw" event={"ID":"b26d8b6f-f726-4238-b48a-f4d272cf27b7","Type":"ContainerStarted","Data":"1b683c4819ec89997fac58a1b26bb4716521cc2ad1bb3e75a7f05cab35ae54f1"}
Sep 18 14:17:23.121658 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:17:23.121636    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-526b97-7486448b6b-hnk46" event={"ID":"8d339523-5739-49e1-aecf-93d3d545c43f","Type":"ContainerStarted","Data":"6f8543f8bde4c233cef31da064ea695ac4a00bfe1092d8789d23e91f4cdc70bf"}
Sep 18 14:17:23.121779 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:17:23.121682    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-526b97-7486448b6b-hnk46" event={"ID":"8d339523-5739-49e1-aecf-93d3d545c43f","Type":"ContainerStarted","Data":"f1d784d3f3e068ba7cab108370ee59e527636f18c7fe13c3a4599eb7dcec56df"}
Sep 18 14:17:23.121779 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:17:23.121720    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-526b97-7486448b6b-hnk46"
Sep 18 14:17:23.163651 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:17:23.163600    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-526b97-7486448b6b-hnk46" podStartSLOduration=1.163582194 podStartE2EDuration="1.163582194s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 14:17:22 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 14:17:23.161794891 +0000 UTC m=+1111.335704552" watchObservedRunningTime="2026-09-18 14:17:23.163582194 +0000 UTC m=+1111.337491854"
Sep 18 14:17:24.129623 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:17:24.129581    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-a-w2-526b97-7bf5747d47-r2mpw" event={"ID":"b26d8b6f-f726-4238-b48a-f4d272cf27b7","Type":"ContainerStarted","Data":"d147a971b357f08a64c04396d98987d95900c19625ccf423f1cf2833987ecaac"}
Sep 18 14:17:24.129986 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:17:24.129652    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-shared-a-w2-526b97-7bf5747d47-r2mpw"
Sep 18 14:17:24.153766 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:17:24.153716    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-shared-a-w2-526b97-7bf5747d47-r2mpw" podStartSLOduration=2.153700096 podStartE2EDuration="2.153700096s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 14:17:22 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 14:17:24.151435328 +0000 UTC m=+1112.325345013" watchObservedRunningTime="2026-09-18 14:17:24.153700096 +0000 UTC m=+1112.327609757"
Sep 18 14:17:24.323349 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:17:24.323319    2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-dc2cee-rlw4j"]
Sep 18 14:17:24.327467 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:17:24.327450    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-dc2cee-rlw4j"
Sep 18 14:17:24.330289 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:17:24.330264    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-l4fn9\""
Sep 18 14:17:24.335329 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:17:24.335304    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-dc2cee-rlw4j"]
Sep 18 14:17:24.420952 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:17:24.420856    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/a0a3cf93-8d76-4622-ac1c-04d9c056aeda-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-a-dc2cee-rlw4j\" (UID: \"a0a3cf93-8d76-4622-ac1c-04d9c056aeda\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-dc2cee-rlw4j"
Sep 18 14:17:24.522469 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:17:24.522430    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/a0a3cf93-8d76-4622-ac1c-04d9c056aeda-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-a-dc2cee-rlw4j\" (UID: \"a0a3cf93-8d76-4622-ac1c-04d9c056aeda\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-dc2cee-rlw4j"
Sep 18 14:17:24.523103 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:17:24.523085    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/a0a3cf93-8d76-4622-ac1c-04d9c056aeda-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-a-dc2cee-rlw4j\" (UID: \"a0a3cf93-8d76-4622-ac1c-04d9c056aeda\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-dc2cee-rlw4j"
Sep 18 14:17:24.642084 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:17:24.642020    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-dc2cee-rlw4j"
Sep 18 14:17:24.919608 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:17:24.919584    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-dc2cee-rlw4j"]
Sep 18 14:17:24.921381 ip-10-0-129-16 kubenswrapper[2565]: W0918 14:17:24.921350    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda0a3cf93_8d76_4622_ac1c_04d9c056aeda.slice/crio-fb1f04bfe90c5914a63f223389fd95a937fd6714d47530cae2e70d23d5f69d6a WatchSource:0}: Error finding container fb1f04bfe90c5914a63f223389fd95a937fd6714d47530cae2e70d23d5f69d6a: Status 404 returned error can't find the container with id fb1f04bfe90c5914a63f223389fd95a937fd6714d47530cae2e70d23d5f69d6a
Sep 18 14:17:25.138507 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:17:25.138475    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-dc2cee-rlw4j" event={"ID":"a0a3cf93-8d76-4622-ac1c-04d9c056aeda","Type":"ContainerStarted","Data":"fb1f04bfe90c5914a63f223389fd95a937fd6714d47530cae2e70d23d5f69d6a"}
Sep 18 14:17:26.146079 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:17:26.146015    2565 generic.go:356] "Generic (PLEG): container finished" podID="a0a3cf93-8d76-4622-ac1c-04d9c056aeda" containerID="2fb1db2d09a1e077528a4bf4ec640c72e5df4d86d8972d0d5d1ec52720898899" exitCode=0
Sep 18 14:17:26.146471 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:17:26.146104    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-dc2cee-rlw4j" event={"ID":"a0a3cf93-8d76-4622-ac1c-04d9c056aeda","Type":"ContainerDied","Data":"2fb1db2d09a1e077528a4bf4ec640c72e5df4d86d8972d0d5d1ec52720898899"}
Sep 18 14:17:27.299624 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:17:27.299598    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-dc2cee-rlw4j"
Sep 18 14:17:27.464595 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:17:27.464487    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/a0a3cf93-8d76-4622-ac1c-04d9c056aeda-maas-api-service-ca\") pod \"a0a3cf93-8d76-4622-ac1c-04d9c056aeda\" (UID: \"a0a3cf93-8d76-4622-ac1c-04d9c056aeda\") "
Sep 18 14:17:27.464821 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:17:27.464798    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a0a3cf93-8d76-4622-ac1c-04d9c056aeda-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "a0a3cf93-8d76-4622-ac1c-04d9c056aeda" (UID: "a0a3cf93-8d76-4622-ac1c-04d9c056aeda"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 18 14:17:27.464922 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:17:27.464907    2565 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/a0a3cf93-8d76-4622-ac1c-04d9c056aeda-maas-api-service-ca\") on node \"ip-10-0-129-16.ec2.internal\" DevicePath \"\""
Sep 18 14:17:28.160495 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:17:28.160460    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-dc2cee-rlw4j" event={"ID":"a0a3cf93-8d76-4622-ac1c-04d9c056aeda","Type":"ContainerDied","Data":"fb1f04bfe90c5914a63f223389fd95a937fd6714d47530cae2e70d23d5f69d6a"}
Sep 18 14:17:28.160495 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:17:28.160494    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-dc2cee-rlw4j"
Sep 18 14:17:28.160709 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:17:28.160517    2565 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="fb1f04bfe90c5914a63f223389fd95a937fd6714d47530cae2e70d23d5f69d6a"
Sep 18 14:17:34.132270 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:17:34.132237    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-526b97-7486448b6b-hnk46"
Sep 18 14:17:34.871113 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:17:34.871083    2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-a-dc2cee-29829015-sl88h"]
Sep 18 14:17:34.873864 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:17:34.873837    2565 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-a-dc2cee-29829015-sl88h"]
Sep 18 14:17:34.908719 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:17:34.908687    2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-a-dc2cee-79685b4ffd-lqfbh"]
Sep 18 14:17:34.909657 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:17:34.909081    2565 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-dc2cee-79685b4ffd-lqfbh" podUID="10d1626f-264e-4036-be5d-4a6a6dbaaa69" containerName="payload-pre-processing" containerID="cri-o://d0214ca02a961afb125a10e2ab29ceabdc8cdaddefe6dcac1cdd2f199b4beb68" gracePeriod=30
Sep 18 14:17:34.913666 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:17:34.913636    2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-a-dc2cee-647f88fd5c-6qkz8"]
Sep 18 14:17:34.913846 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:17:34.913821    2565 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-inf-a-dc2cee-647f88fd5c-6qkz8" podUID="490e2772-09af-4a35-834b-fccc3f178137" containerName="payload-processing" containerID="cri-o://11eb7de5b5d6c335d94f98b18bf28663edeed6a55854b1a5c048b7db3388194d" gracePeriod=30
Sep 18 14:17:35.102685 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:17:35.102656    2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-dc2cee-rlw4j"]
Sep 18 14:17:35.108511 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:17:35.108473    2565 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-dc2cee-rlw4j"]
Sep 18 14:17:35.141693 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:17:35.141668    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-shared-a-w2-526b97-7bf5747d47-r2mpw"
Sep 18 14:17:35.212699 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:17:35.212665    2565 generic.go:356] "Generic (PLEG): container finished" podID="490e2772-09af-4a35-834b-fccc3f178137" containerID="11eb7de5b5d6c335d94f98b18bf28663edeed6a55854b1a5c048b7db3388194d" exitCode=0
Sep 18 14:17:35.212842 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:17:35.212804    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-a-dc2cee-647f88fd5c-6qkz8" event={"ID":"490e2772-09af-4a35-834b-fccc3f178137","Type":"ContainerDied","Data":"11eb7de5b5d6c335d94f98b18bf28663edeed6a55854b1a5c048b7db3388194d"}
Sep 18 14:17:35.214458 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:17:35.214433    2565 generic.go:356] "Generic (PLEG): container finished" podID="10d1626f-264e-4036-be5d-4a6a6dbaaa69" containerID="d0214ca02a961afb125a10e2ab29ceabdc8cdaddefe6dcac1cdd2f199b4beb68" exitCode=0
Sep 18 14:17:35.214583 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:17:35.214480    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-dc2cee-79685b4ffd-lqfbh" event={"ID":"10d1626f-264e-4036-be5d-4a6a6dbaaa69","Type":"ContainerDied","Data":"d0214ca02a961afb125a10e2ab29ceabdc8cdaddefe6dcac1cdd2f199b4beb68"}
Sep 18 14:17:35.214583 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:17:35.214517    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-dc2cee-79685b4ffd-lqfbh" event={"ID":"10d1626f-264e-4036-be5d-4a6a6dbaaa69","Type":"ContainerDied","Data":"e4f942e4e7dc0259f0f34e044256363bf7b1a463277006806a63d2a992b103f7"}
Sep 18 14:17:35.214583 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:17:35.214543    2565 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="e4f942e4e7dc0259f0f34e044256363bf7b1a463277006806a63d2a992b103f7"
Sep 18 14:17:35.215651 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:17:35.215630    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-dc2cee-79685b4ffd-lqfbh"
Sep 18 14:17:35.229904 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:17:35.229879    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-a-dc2cee-647f88fd5c-6qkz8"
Sep 18 14:17:35.242246 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:17:35.240746    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/10d1626f-264e-4036-be5d-4a6a6dbaaa69-plugins-config-volume\") pod \"10d1626f-264e-4036-be5d-4a6a6dbaaa69\" (UID: \"10d1626f-264e-4036-be5d-4a6a6dbaaa69\") "
Sep 18 14:17:35.242246 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:17:35.240810    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-6tmqs\" (UniqueName: \"kubernetes.io/projected/490e2772-09af-4a35-834b-fccc3f178137-kube-api-access-6tmqs\") pod \"490e2772-09af-4a35-834b-fccc3f178137\" (UID: \"490e2772-09af-4a35-834b-fccc3f178137\") "
Sep 18 14:17:35.242246 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:17:35.240861    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-vmfhz\" (UniqueName: \"kubernetes.io/projected/10d1626f-264e-4036-be5d-4a6a6dbaaa69-kube-api-access-vmfhz\") pod \"10d1626f-264e-4036-be5d-4a6a6dbaaa69\" (UID: \"10d1626f-264e-4036-be5d-4a6a6dbaaa69\") "
Sep 18 14:17:35.242246 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:17:35.241015    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/490e2772-09af-4a35-834b-fccc3f178137-plugins-config-volume\") pod \"490e2772-09af-4a35-834b-fccc3f178137\" (UID: \"490e2772-09af-4a35-834b-fccc3f178137\") "
Sep 18 14:17:35.242246 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:17:35.241920    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/10d1626f-264e-4036-be5d-4a6a6dbaaa69-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "10d1626f-264e-4036-be5d-4a6a6dbaaa69" (UID: "10d1626f-264e-4036-be5d-4a6a6dbaaa69"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 18 14:17:35.242246 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:17:35.241989    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/490e2772-09af-4a35-834b-fccc3f178137-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "490e2772-09af-4a35-834b-fccc3f178137" (UID: "490e2772-09af-4a35-834b-fccc3f178137"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 18 14:17:35.245226 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:17:35.245193    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/490e2772-09af-4a35-834b-fccc3f178137-kube-api-access-6tmqs" (OuterVolumeSpecName: "kube-api-access-6tmqs") pod "490e2772-09af-4a35-834b-fccc3f178137" (UID: "490e2772-09af-4a35-834b-fccc3f178137"). InnerVolumeSpecName "kube-api-access-6tmqs". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 18 14:17:35.245892 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:17:35.245866    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/10d1626f-264e-4036-be5d-4a6a6dbaaa69-kube-api-access-vmfhz" (OuterVolumeSpecName: "kube-api-access-vmfhz") pod "10d1626f-264e-4036-be5d-4a6a6dbaaa69" (UID: "10d1626f-264e-4036-be5d-4a6a6dbaaa69"). InnerVolumeSpecName "kube-api-access-vmfhz". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 18 14:17:35.342586 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:17:35.342554    2565 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6tmqs\" (UniqueName: \"kubernetes.io/projected/490e2772-09af-4a35-834b-fccc3f178137-kube-api-access-6tmqs\") on node \"ip-10-0-129-16.ec2.internal\" DevicePath \"\""
Sep 18 14:17:35.342586 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:17:35.342584    2565 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vmfhz\" (UniqueName: \"kubernetes.io/projected/10d1626f-264e-4036-be5d-4a6a6dbaaa69-kube-api-access-vmfhz\") on node \"ip-10-0-129-16.ec2.internal\" DevicePath \"\""
Sep 18 14:17:35.342586 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:17:35.342594    2565 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/490e2772-09af-4a35-834b-fccc3f178137-plugins-config-volume\") on node \"ip-10-0-129-16.ec2.internal\" DevicePath \"\""
Sep 18 14:17:35.342841 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:17:35.342603    2565 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/10d1626f-264e-4036-be5d-4a6a6dbaaa69-plugins-config-volume\") on node \"ip-10-0-129-16.ec2.internal\" DevicePath \"\""
Sep 18 14:17:35.914747 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:17:35.914709    2565 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a0a3cf93-8d76-4622-ac1c-04d9c056aeda" path="/var/lib/kubelet/pods/a0a3cf93-8d76-4622-ac1c-04d9c056aeda/volumes"
Sep 18 14:17:35.915136 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:17:35.915116    2565 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d495891f-f94b-49e8-8391-1a9df17c1e82" path="/var/lib/kubelet/pods/d495891f-f94b-49e8-8391-1a9df17c1e82/volumes"
Sep 18 14:17:36.223527 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:17:36.223441    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-a-dc2cee-647f88fd5c-6qkz8" event={"ID":"490e2772-09af-4a35-834b-fccc3f178137","Type":"ContainerDied","Data":"8640c697227cf46a7f3cc7ba38fbb7ee70ed618239b1b0567f9a424985f91a8a"}
Sep 18 14:17:36.223527 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:17:36.223465    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-dc2cee-79685b4ffd-lqfbh"
Sep 18 14:17:36.223527 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:17:36.223473    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-a-dc2cee-647f88fd5c-6qkz8"
Sep 18 14:17:36.223527 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:17:36.223510    2565 scope.go:117] "RemoveContainer" containerID="11eb7de5b5d6c335d94f98b18bf28663edeed6a55854b1a5c048b7db3388194d"
Sep 18 14:17:36.245089 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:17:36.245058    2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-a-dc2cee-647f88fd5c-6qkz8"]
Sep 18 14:17:36.253983 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:17:36.253950    2565 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-a-dc2cee-647f88fd5c-6qkz8"]
Sep 18 14:17:36.293970 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:17:36.293937    2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-a-dc2cee-79685b4ffd-lqfbh"]
Sep 18 14:17:36.312688 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:17:36.312654    2565 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-a-dc2cee-79685b4ffd-lqfbh"]
Sep 18 14:17:37.913116 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:17:37.913078    2565 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="10d1626f-264e-4036-be5d-4a6a6dbaaa69" path="/var/lib/kubelet/pods/10d1626f-264e-4036-be5d-4a6a6dbaaa69/volumes"
Sep 18 14:17:37.913488 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:17:37.913468    2565 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="490e2772-09af-4a35-834b-fccc3f178137" path="/var/lib/kubelet/pods/490e2772-09af-4a35-834b-fccc3f178137/volumes"
Sep 18 14:17:45.313353 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:17:45.313314    2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-ee8a08-qv5qv"]
Sep 18 14:17:45.313965 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:17:45.313947    2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="490e2772-09af-4a35-834b-fccc3f178137" containerName="payload-processing"
Sep 18 14:17:45.313965 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:17:45.313965    2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="490e2772-09af-4a35-834b-fccc3f178137" containerName="payload-processing"
Sep 18 14:17:45.314061 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:17:45.313973    2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a0a3cf93-8d76-4622-ac1c-04d9c056aeda" containerName="revoke-keys"
Sep 18 14:17:45.314061 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:17:45.313979    2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="a0a3cf93-8d76-4622-ac1c-04d9c056aeda" containerName="revoke-keys"
Sep 18 14:17:45.314061 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:17:45.313992    2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="10d1626f-264e-4036-be5d-4a6a6dbaaa69" containerName="payload-pre-processing"
Sep 18 14:17:45.314061 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:17:45.313998    2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="10d1626f-264e-4036-be5d-4a6a6dbaaa69" containerName="payload-pre-processing"
Sep 18 14:17:45.314195 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:17:45.314105    2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="10d1626f-264e-4036-be5d-4a6a6dbaaa69" containerName="payload-pre-processing"
Sep 18 14:17:45.314195 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:17:45.314115    2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="a0a3cf93-8d76-4622-ac1c-04d9c056aeda" containerName="revoke-keys"
Sep 18 14:17:45.314195 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:17:45.314123    2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="490e2772-09af-4a35-834b-fccc3f178137" containerName="payload-processing"
Sep 18 14:17:45.316888 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:17:45.316873    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-ee8a08-qv5qv"
Sep 18 14:17:45.320363 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:17:45.320341    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-l4fn9\""
Sep 18 14:17:45.329079 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:17:45.329054    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-ee8a08-qv5qv"]
Sep 18 14:17:45.454724 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:17:45.454684    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/a1b39111-972c-4dc8-b05d-8b1f59fca772-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-b-ee8a08-qv5qv\" (UID: \"a1b39111-972c-4dc8-b05d-8b1f59fca772\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-ee8a08-qv5qv"
Sep 18 14:17:45.555983 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:17:45.555950    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/a1b39111-972c-4dc8-b05d-8b1f59fca772-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-b-ee8a08-qv5qv\" (UID: \"a1b39111-972c-4dc8-b05d-8b1f59fca772\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-ee8a08-qv5qv"
Sep 18 14:17:45.556656 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:17:45.556628    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/a1b39111-972c-4dc8-b05d-8b1f59fca772-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-b-ee8a08-qv5qv\" (UID: \"a1b39111-972c-4dc8-b05d-8b1f59fca772\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-ee8a08-qv5qv"
Sep 18 14:17:45.628156 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:17:45.628068    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-ee8a08-qv5qv"
Sep 18 14:17:45.914754 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:17:45.914684    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-ee8a08-qv5qv"]
Sep 18 14:17:45.922970 ip-10-0-129-16 kubenswrapper[2565]: W0918 14:17:45.922945    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda1b39111_972c_4dc8_b05d_8b1f59fca772.slice/crio-3034f38adfe269226f4eb4a354c5f8d250fdd3d2ce23cd880c11cae2e9af5e8a WatchSource:0}: Error finding container 3034f38adfe269226f4eb4a354c5f8d250fdd3d2ce23cd880c11cae2e9af5e8a: Status 404 returned error can't find the container with id 3034f38adfe269226f4eb4a354c5f8d250fdd3d2ce23cd880c11cae2e9af5e8a
Sep 18 14:17:46.290499 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:17:46.290464    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-ee8a08-qv5qv" event={"ID":"a1b39111-972c-4dc8-b05d-8b1f59fca772","Type":"ContainerStarted","Data":"937c3b7413ee6a3e019d88537b1defe3e8f7e333b55815e505739a1b739afb5f"}
Sep 18 14:17:46.290701 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:17:46.290515    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-ee8a08-qv5qv" event={"ID":"a1b39111-972c-4dc8-b05d-8b1f59fca772","Type":"ContainerStarted","Data":"3034f38adfe269226f4eb4a354c5f8d250fdd3d2ce23cd880c11cae2e9af5e8a"}
Sep 18 14:17:46.311494 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:17:46.311448    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-ee8a08-qv5qv" podStartSLOduration=1.311433238 podStartE2EDuration="1.311433238s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 14:17:45 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 14:17:46.310625142 +0000 UTC m=+1134.484534793" watchObservedRunningTime="2026-09-18 14:17:46.311433238 +0000 UTC m=+1134.485342898"
Sep 18 14:17:47.298796 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:17:47.298760    2565 generic.go:356] "Generic (PLEG): container finished" podID="a1b39111-972c-4dc8-b05d-8b1f59fca772" containerID="937c3b7413ee6a3e019d88537b1defe3e8f7e333b55815e505739a1b739afb5f" exitCode=0
Sep 18 14:17:47.299263 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:17:47.298843    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-ee8a08-qv5qv" event={"ID":"a1b39111-972c-4dc8-b05d-8b1f59fca772","Type":"ContainerDied","Data":"937c3b7413ee6a3e019d88537b1defe3e8f7e333b55815e505739a1b739afb5f"}
Sep 18 14:17:48.448251 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:17:48.448224    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-ee8a08-qv5qv"
Sep 18 14:17:48.587797 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:17:48.587715    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/a1b39111-972c-4dc8-b05d-8b1f59fca772-maas-api-service-ca\") pod \"a1b39111-972c-4dc8-b05d-8b1f59fca772\" (UID: \"a1b39111-972c-4dc8-b05d-8b1f59fca772\") "
Sep 18 14:17:48.588073 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:17:48.588053    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a1b39111-972c-4dc8-b05d-8b1f59fca772-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "a1b39111-972c-4dc8-b05d-8b1f59fca772" (UID: "a1b39111-972c-4dc8-b05d-8b1f59fca772"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 18 14:17:48.688913 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:17:48.688880    2565 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/a1b39111-972c-4dc8-b05d-8b1f59fca772-maas-api-service-ca\") on node \"ip-10-0-129-16.ec2.internal\" DevicePath \"\""
Sep 18 14:17:49.313378 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:17:49.313337    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-ee8a08-qv5qv" event={"ID":"a1b39111-972c-4dc8-b05d-8b1f59fca772","Type":"ContainerDied","Data":"3034f38adfe269226f4eb4a354c5f8d250fdd3d2ce23cd880c11cae2e9af5e8a"}
Sep 18 14:17:49.313378 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:17:49.313370    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-ee8a08-qv5qv"
Sep 18 14:17:49.313378 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:17:49.313385    2565 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="3034f38adfe269226f4eb4a354c5f8d250fdd3d2ce23cd880c11cae2e9af5e8a"
Sep 18 14:17:51.175179 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:17:51.175141    2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w2-216d39-6d4d494cd-f622c"]
Sep 18 14:17:51.175630 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:17:51.175617    2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a1b39111-972c-4dc8-b05d-8b1f59fca772" containerName="revoke-keys"
Sep 18 14:17:51.175630 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:17:51.175630    2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="a1b39111-972c-4dc8-b05d-8b1f59fca772" containerName="revoke-keys"
Sep 18 14:17:51.175758 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:17:51.175748    2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="a1b39111-972c-4dc8-b05d-8b1f59fca772" containerName="revoke-keys"
Sep 18 14:17:51.179216 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:17:51.179194    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-216d39-6d4d494cd-f622c"
Sep 18 14:17:51.182064 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:17:51.182043    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-shared-b-w2-216d39\""
Sep 18 14:17:51.195324 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:17:51.195291    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w2-216d39-6d4d494cd-f622c"]
Sep 18 14:17:51.198822 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:17:51.198797    2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w2-216d39-59959cc765-wkjlv"]
Sep 18 14:17:51.203482 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:17:51.203461    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-b-w2-216d39-59959cc765-wkjlv"
Sep 18 14:17:51.206651 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:17:51.206622    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-shared-b-w2-216d39-dockercfg-kd2bv\""
Sep 18 14:17:51.213304 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:17:51.213273    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w2-216d39-59959cc765-wkjlv"]
Sep 18 14:17:51.319888 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:17:51.319856    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-h5j5d\" (UniqueName: \"kubernetes.io/projected/8eee79ba-0e22-4f42-aac9-0a35b93230f0-kube-api-access-h5j5d\") pod \"payload-pre-processing-e2e-shared-b-w2-216d39-6d4d494cd-f622c\" (UID: \"8eee79ba-0e22-4f42-aac9-0a35b93230f0\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-216d39-6d4d494cd-f622c"
Sep 18 14:17:51.320125 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:17:51.320026    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mb6tc\" (UniqueName: \"kubernetes.io/projected/b61ae7b1-71cd-45e0-b2a7-9f840216947e-kube-api-access-mb6tc\") pod \"payload-processing-e2e-shared-b-w2-216d39-59959cc765-wkjlv\" (UID: \"b61ae7b1-71cd-45e0-b2a7-9f840216947e\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w2-216d39-59959cc765-wkjlv"
Sep 18 14:17:51.320202 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:17:51.320175    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b61ae7b1-71cd-45e0-b2a7-9f840216947e-plugins-config-volume\") pod \"payload-processing-e2e-shared-b-w2-216d39-59959cc765-wkjlv\" (UID: \"b61ae7b1-71cd-45e0-b2a7-9f840216947e\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w2-216d39-59959cc765-wkjlv"
Sep 18 14:17:51.320261 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:17:51.320235    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/8eee79ba-0e22-4f42-aac9-0a35b93230f0-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-b-w2-216d39-6d4d494cd-f622c\" (UID: \"8eee79ba-0e22-4f42-aac9-0a35b93230f0\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-216d39-6d4d494cd-f622c"
Sep 18 14:17:51.421436 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:17:51.421391    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-h5j5d\" (UniqueName: \"kubernetes.io/projected/8eee79ba-0e22-4f42-aac9-0a35b93230f0-kube-api-access-h5j5d\") pod \"payload-pre-processing-e2e-shared-b-w2-216d39-6d4d494cd-f622c\" (UID: \"8eee79ba-0e22-4f42-aac9-0a35b93230f0\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-216d39-6d4d494cd-f622c"
Sep 18 14:17:51.421436 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:17:51.421436    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mb6tc\" (UniqueName: \"kubernetes.io/projected/b61ae7b1-71cd-45e0-b2a7-9f840216947e-kube-api-access-mb6tc\") pod \"payload-processing-e2e-shared-b-w2-216d39-59959cc765-wkjlv\" (UID: \"b61ae7b1-71cd-45e0-b2a7-9f840216947e\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w2-216d39-59959cc765-wkjlv"
Sep 18 14:17:51.421706 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:17:51.421504    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b61ae7b1-71cd-45e0-b2a7-9f840216947e-plugins-config-volume\") pod \"payload-processing-e2e-shared-b-w2-216d39-59959cc765-wkjlv\" (UID: \"b61ae7b1-71cd-45e0-b2a7-9f840216947e\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w2-216d39-59959cc765-wkjlv"
Sep 18 14:17:51.421706 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:17:51.421537    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/8eee79ba-0e22-4f42-aac9-0a35b93230f0-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-b-w2-216d39-6d4d494cd-f622c\" (UID: \"8eee79ba-0e22-4f42-aac9-0a35b93230f0\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-216d39-6d4d494cd-f622c"
Sep 18 14:17:51.422289 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:17:51.422269    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/8eee79ba-0e22-4f42-aac9-0a35b93230f0-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-b-w2-216d39-6d4d494cd-f622c\" (UID: \"8eee79ba-0e22-4f42-aac9-0a35b93230f0\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-216d39-6d4d494cd-f622c"
Sep 18 14:17:51.422351 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:17:51.422320    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b61ae7b1-71cd-45e0-b2a7-9f840216947e-plugins-config-volume\") pod \"payload-processing-e2e-shared-b-w2-216d39-59959cc765-wkjlv\" (UID: \"b61ae7b1-71cd-45e0-b2a7-9f840216947e\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w2-216d39-59959cc765-wkjlv"
Sep 18 14:17:51.434237 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:17:51.434172    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-h5j5d\" (UniqueName: \"kubernetes.io/projected/8eee79ba-0e22-4f42-aac9-0a35b93230f0-kube-api-access-h5j5d\") pod \"payload-pre-processing-e2e-shared-b-w2-216d39-6d4d494cd-f622c\" (UID: \"8eee79ba-0e22-4f42-aac9-0a35b93230f0\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-216d39-6d4d494cd-f622c"
Sep 18 14:17:51.434369 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:17:51.434350    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mb6tc\" (UniqueName: \"kubernetes.io/projected/b61ae7b1-71cd-45e0-b2a7-9f840216947e-kube-api-access-mb6tc\") pod \"payload-processing-e2e-shared-b-w2-216d39-59959cc765-wkjlv\" (UID: \"b61ae7b1-71cd-45e0-b2a7-9f840216947e\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w2-216d39-59959cc765-wkjlv"
Sep 18 14:17:51.490093 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:17:51.490020    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-216d39-6d4d494cd-f622c"
Sep 18 14:17:51.516412 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:17:51.516382    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-b-w2-216d39-59959cc765-wkjlv"
Sep 18 14:17:51.785415 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:17:51.785392    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w2-216d39-6d4d494cd-f622c"]
Sep 18 14:17:51.787291 ip-10-0-129-16 kubenswrapper[2565]: W0918 14:17:51.787264    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8eee79ba_0e22_4f42_aac9_0a35b93230f0.slice/crio-85705b38ee662e6657815802af334e01cdffe33af85cedc87a1796281bb862f6 WatchSource:0}: Error finding container 85705b38ee662e6657815802af334e01cdffe33af85cedc87a1796281bb862f6: Status 404 returned error can't find the container with id 85705b38ee662e6657815802af334e01cdffe33af85cedc87a1796281bb862f6
Sep 18 14:17:51.819243 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:17:51.819217    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w2-216d39-59959cc765-wkjlv"]
Sep 18 14:17:51.820618 ip-10-0-129-16 kubenswrapper[2565]: W0918 14:17:51.820597    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb61ae7b1_71cd_45e0_b2a7_9f840216947e.slice/crio-fcbeaef794172c1d306780f416809ac833da5a60d72437219de5a83141c138fe WatchSource:0}: Error finding container fcbeaef794172c1d306780f416809ac833da5a60d72437219de5a83141c138fe: Status 404 returned error can't find the container with id fcbeaef794172c1d306780f416809ac833da5a60d72437219de5a83141c138fe
Sep 18 14:17:52.341221 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:17:52.341178    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-216d39-6d4d494cd-f622c" event={"ID":"8eee79ba-0e22-4f42-aac9-0a35b93230f0","Type":"ContainerStarted","Data":"04e2342a3f4c88dd5a77447636974a180d243621658ab7f2f95c2a9a68be43ca"}
Sep 18 14:17:52.341677 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:17:52.341246    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-216d39-6d4d494cd-f622c" event={"ID":"8eee79ba-0e22-4f42-aac9-0a35b93230f0","Type":"ContainerStarted","Data":"85705b38ee662e6657815802af334e01cdffe33af85cedc87a1796281bb862f6"}
Sep 18 14:17:52.341677 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:17:52.341372    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-216d39-6d4d494cd-f622c"
Sep 18 14:17:52.342907 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:17:52.342880    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-b-w2-216d39-59959cc765-wkjlv" event={"ID":"b61ae7b1-71cd-45e0-b2a7-9f840216947e","Type":"ContainerStarted","Data":"d7adde985defa0b3ee4b32e4affc9f8e0f39a759a6a67a1ace340d38c55cb9e7"}
Sep 18 14:17:52.342988 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:17:52.342931    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-shared-b-w2-216d39-59959cc765-wkjlv"
Sep 18 14:17:52.342988 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:17:52.342947    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-b-w2-216d39-59959cc765-wkjlv" event={"ID":"b61ae7b1-71cd-45e0-b2a7-9f840216947e","Type":"ContainerStarted","Data":"fcbeaef794172c1d306780f416809ac833da5a60d72437219de5a83141c138fe"}
Sep 18 14:17:52.362942 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:17:52.362893    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-216d39-6d4d494cd-f622c" podStartSLOduration=1.362879628 podStartE2EDuration="1.362879628s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 14:17:51 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 14:17:52.359959652 +0000 UTC m=+1140.533869332" watchObservedRunningTime="2026-09-18 14:17:52.362879628 +0000 UTC m=+1140.536789288"
Sep 18 14:17:54.015094 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:17:54.015060    2565 scope.go:117] "RemoveContainer" containerID="d0214ca02a961afb125a10e2ab29ceabdc8cdaddefe6dcac1cdd2f199b4beb68"
Sep 18 14:17:54.027021 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:17:54.026993    2565 scope.go:117] "RemoveContainer" containerID="c0daf7e53f83b114c1765f7dfa2169ce33b44cf7a5f5e9542a43e3a13496272c"
Sep 18 14:17:56.101718 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:17:56.101652    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-shared-b-w2-216d39-59959cc765-wkjlv" podStartSLOduration=5.101631553 podStartE2EDuration="5.101631553s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 14:17:51 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 14:17:52.386345327 +0000 UTC m=+1140.560255014" watchObservedRunningTime="2026-09-18 14:17:56.101631553 +0000 UTC m=+1144.275541262"
Sep 18 14:17:56.102779 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:17:56.102759    2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-b-ee8a08-29829015-6jvrd"]
Sep 18 14:17:56.108360 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:17:56.108328    2565 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-b-ee8a08-29829015-6jvrd"]
Sep 18 14:17:56.377636 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:17:56.377543    2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-ee8a08-qv5qv"]
Sep 18 14:17:56.382654 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:17:56.382625    2565 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-ee8a08-qv5qv"]
Sep 18 14:17:57.912901 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:17:57.912870    2565 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a1b39111-972c-4dc8-b05d-8b1f59fca772" path="/var/lib/kubelet/pods/a1b39111-972c-4dc8-b05d-8b1f59fca772/volumes"
Sep 18 14:17:57.913297 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:17:57.913191    2565 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d424cda9-e346-4c36-b2db-b2dc210bbabf" path="/var/lib/kubelet/pods/d424cda9-e346-4c36-b2db-b2dc210bbabf/volumes"
Sep 18 14:18:03.351780 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:18:03.351746    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-shared-b-w2-216d39-59959cc765-wkjlv"
Sep 18 14:18:03.352166 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:18:03.351919    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-216d39-6d4d494cd-f622c"
Sep 18 14:18:13.817670 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:18:13.817589    2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w3-ae54af-5cc95698b7-w4fhd"]
Sep 18 14:18:13.822376 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:18:13.822357    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-a-w3-ae54af-5cc95698b7-w4fhd"
Sep 18 14:18:13.827258 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:18:13.827233    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-shared-a-w3-ae54af-dockercfg-dtpq9\""
Sep 18 14:18:13.827662 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:18:13.827646    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-shared-a-w3-ae54af\""
Sep 18 14:18:13.843298 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:18:13.843276    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w3-ae54af-5cc95698b7-w4fhd"]
Sep 18 14:18:13.868488 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:18:13.868457    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4qc9t\" (UniqueName: \"kubernetes.io/projected/6b1f9787-10c9-41a0-9244-a51cb19b61fc-kube-api-access-4qc9t\") pod \"payload-processing-e2e-shared-a-w3-ae54af-5cc95698b7-w4fhd\" (UID: \"6b1f9787-10c9-41a0-9244-a51cb19b61fc\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w3-ae54af-5cc95698b7-w4fhd"
Sep 18 14:18:13.868673 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:18:13.868522    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/6b1f9787-10c9-41a0-9244-a51cb19b61fc-plugins-config-volume\") pod \"payload-processing-e2e-shared-a-w3-ae54af-5cc95698b7-w4fhd\" (UID: \"6b1f9787-10c9-41a0-9244-a51cb19b61fc\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w3-ae54af-5cc95698b7-w4fhd"
Sep 18 14:18:13.969323 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:18:13.969283    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/6b1f9787-10c9-41a0-9244-a51cb19b61fc-plugins-config-volume\") pod \"payload-processing-e2e-shared-a-w3-ae54af-5cc95698b7-w4fhd\" (UID: \"6b1f9787-10c9-41a0-9244-a51cb19b61fc\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w3-ae54af-5cc95698b7-w4fhd"
Sep 18 14:18:13.969483 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:18:13.969432    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4qc9t\" (UniqueName: \"kubernetes.io/projected/6b1f9787-10c9-41a0-9244-a51cb19b61fc-kube-api-access-4qc9t\") pod \"payload-processing-e2e-shared-a-w3-ae54af-5cc95698b7-w4fhd\" (UID: \"6b1f9787-10c9-41a0-9244-a51cb19b61fc\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w3-ae54af-5cc95698b7-w4fhd"
Sep 18 14:18:13.970239 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:18:13.970215    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/6b1f9787-10c9-41a0-9244-a51cb19b61fc-plugins-config-volume\") pod \"payload-processing-e2e-shared-a-w3-ae54af-5cc95698b7-w4fhd\" (UID: \"6b1f9787-10c9-41a0-9244-a51cb19b61fc\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w3-ae54af-5cc95698b7-w4fhd"
Sep 18 14:18:13.978741 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:18:13.978710    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4qc9t\" (UniqueName: \"kubernetes.io/projected/6b1f9787-10c9-41a0-9244-a51cb19b61fc-kube-api-access-4qc9t\") pod \"payload-processing-e2e-shared-a-w3-ae54af-5cc95698b7-w4fhd\" (UID: \"6b1f9787-10c9-41a0-9244-a51cb19b61fc\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w3-ae54af-5cc95698b7-w4fhd"
Sep 18 14:18:14.133265 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:18:14.133176    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-a-w3-ae54af-5cc95698b7-w4fhd"
Sep 18 14:18:14.410157 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:18:14.410121    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w3-ae54af-5cc95698b7-w4fhd"]
Sep 18 14:18:14.410696 ip-10-0-129-16 kubenswrapper[2565]: W0918 14:18:14.410667    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6b1f9787_10c9_41a0_9244_a51cb19b61fc.slice/crio-b2c6b0273e6abf25adf29f335def06ae54b58775f7f7ec651663cb685a435cba WatchSource:0}: Error finding container b2c6b0273e6abf25adf29f335def06ae54b58775f7f7ec651663cb685a435cba: Status 404 returned error can't find the container with id b2c6b0273e6abf25adf29f335def06ae54b58775f7f7ec651663cb685a435cba
Sep 18 14:18:14.482904 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:18:14.482870    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-a-w3-ae54af-5cc95698b7-w4fhd" event={"ID":"6b1f9787-10c9-41a0-9244-a51cb19b61fc","Type":"ContainerStarted","Data":"b2c6b0273e6abf25adf29f335def06ae54b58775f7f7ec651663cb685a435cba"}
Sep 18 14:18:15.491168 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:18:15.491129    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-a-w3-ae54af-5cc95698b7-w4fhd" event={"ID":"6b1f9787-10c9-41a0-9244-a51cb19b61fc","Type":"ContainerStarted","Data":"4b700e7515dbdecf1ef9eb000646255dee79cd63203d3f7005363c92f715e278"}
Sep 18 14:18:15.491551 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:18:15.491195    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-shared-a-w3-ae54af-5cc95698b7-w4fhd"
Sep 18 14:18:15.542969 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:18:15.542915    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-shared-a-w3-ae54af-5cc95698b7-w4fhd" podStartSLOduration=2.542897903 podStartE2EDuration="2.542897903s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 14:18:13 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 14:18:15.541296702 +0000 UTC m=+1163.715206379" watchObservedRunningTime="2026-09-18 14:18:15.542897903 +0000 UTC m=+1163.716807602"
Sep 18 14:18:21.535660 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:18:21.535626    2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-988c9d38-7b49d89946-fxv7s"]
Sep 18 14:18:21.539735 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:18:21.539716    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-988c9d38-7b49d89946-fxv7s"
Sep 18 14:18:21.542452 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:18:21.542431    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-mt-988c9d38-dockercfg-bn9zd\""
Sep 18 14:18:21.542574 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:18:21.542459    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-mt-988c9d38\""
Sep 18 14:18:21.555481 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:18:21.555453    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-988c9d38-7b49d89946-fxv7s"]
Sep 18 14:18:21.620256 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:18:21.620223    2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-988c9d38-55cc7688d9-8p622"]
Sep 18 14:18:21.624166 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:18:21.624149    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-988c9d38-55cc7688d9-8p622"
Sep 18 14:18:21.636924 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:18:21.636894    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-988c9d38-55cc7688d9-8p622"]
Sep 18 14:18:21.651326 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:18:21.651285    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hh6mc\" (UniqueName: \"kubernetes.io/projected/5faa5aae-022a-4150-b351-bdf82de4379a-kube-api-access-hh6mc\") pod \"payload-pre-processing-e2e-mt-988c9d38-7b49d89946-fxv7s\" (UID: \"5faa5aae-022a-4150-b351-bdf82de4379a\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-988c9d38-7b49d89946-fxv7s"
Sep 18 14:18:21.651524 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:18:21.651355    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/5faa5aae-022a-4150-b351-bdf82de4379a-plugins-config-volume\") pod \"payload-pre-processing-e2e-mt-988c9d38-7b49d89946-fxv7s\" (UID: \"5faa5aae-022a-4150-b351-bdf82de4379a\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-988c9d38-7b49d89946-fxv7s"
Sep 18 14:18:21.752570 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:18:21.752531    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/72beae27-927d-4475-8923-c84bdb428f47-plugins-config-volume\") pod \"payload-processing-e2e-mt-988c9d38-55cc7688d9-8p622\" (UID: \"72beae27-927d-4475-8923-c84bdb428f47\") " pod="openshift-ingress/payload-processing-e2e-mt-988c9d38-55cc7688d9-8p622"
Sep 18 14:18:21.752570 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:18:21.752576    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-66fkk\" (UniqueName: \"kubernetes.io/projected/72beae27-927d-4475-8923-c84bdb428f47-kube-api-access-66fkk\") pod \"payload-processing-e2e-mt-988c9d38-55cc7688d9-8p622\" (UID: \"72beae27-927d-4475-8923-c84bdb428f47\") " pod="openshift-ingress/payload-processing-e2e-mt-988c9d38-55cc7688d9-8p622"
Sep 18 14:18:21.752855 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:18:21.752750    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hh6mc\" (UniqueName: \"kubernetes.io/projected/5faa5aae-022a-4150-b351-bdf82de4379a-kube-api-access-hh6mc\") pod \"payload-pre-processing-e2e-mt-988c9d38-7b49d89946-fxv7s\" (UID: \"5faa5aae-022a-4150-b351-bdf82de4379a\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-988c9d38-7b49d89946-fxv7s"
Sep 18 14:18:21.752855 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:18:21.752832    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/5faa5aae-022a-4150-b351-bdf82de4379a-plugins-config-volume\") pod \"payload-pre-processing-e2e-mt-988c9d38-7b49d89946-fxv7s\" (UID: \"5faa5aae-022a-4150-b351-bdf82de4379a\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-988c9d38-7b49d89946-fxv7s"
Sep 18 14:18:21.753562 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:18:21.753538    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/5faa5aae-022a-4150-b351-bdf82de4379a-plugins-config-volume\") pod \"payload-pre-processing-e2e-mt-988c9d38-7b49d89946-fxv7s\" (UID: \"5faa5aae-022a-4150-b351-bdf82de4379a\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-988c9d38-7b49d89946-fxv7s"
Sep 18 14:18:21.765358 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:18:21.765323    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hh6mc\" (UniqueName: \"kubernetes.io/projected/5faa5aae-022a-4150-b351-bdf82de4379a-kube-api-access-hh6mc\") pod \"payload-pre-processing-e2e-mt-988c9d38-7b49d89946-fxv7s\" (UID: \"5faa5aae-022a-4150-b351-bdf82de4379a\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-988c9d38-7b49d89946-fxv7s"
Sep 18 14:18:21.851153 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:18:21.851112    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-988c9d38-7b49d89946-fxv7s"
Sep 18 14:18:21.853622 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:18:21.853593    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/72beae27-927d-4475-8923-c84bdb428f47-plugins-config-volume\") pod \"payload-processing-e2e-mt-988c9d38-55cc7688d9-8p622\" (UID: \"72beae27-927d-4475-8923-c84bdb428f47\") " pod="openshift-ingress/payload-processing-e2e-mt-988c9d38-55cc7688d9-8p622"
Sep 18 14:18:21.853752 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:18:21.853634    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-66fkk\" (UniqueName: \"kubernetes.io/projected/72beae27-927d-4475-8923-c84bdb428f47-kube-api-access-66fkk\") pod \"payload-processing-e2e-mt-988c9d38-55cc7688d9-8p622\" (UID: \"72beae27-927d-4475-8923-c84bdb428f47\") " pod="openshift-ingress/payload-processing-e2e-mt-988c9d38-55cc7688d9-8p622"
Sep 18 14:18:21.854688 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:18:21.854503    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/72beae27-927d-4475-8923-c84bdb428f47-plugins-config-volume\") pod \"payload-processing-e2e-mt-988c9d38-55cc7688d9-8p622\" (UID: \"72beae27-927d-4475-8923-c84bdb428f47\") " pod="openshift-ingress/payload-processing-e2e-mt-988c9d38-55cc7688d9-8p622"
Sep 18 14:18:21.865446 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:18:21.865425    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-66fkk\" (UniqueName: \"kubernetes.io/projected/72beae27-927d-4475-8923-c84bdb428f47-kube-api-access-66fkk\") pod \"payload-processing-e2e-mt-988c9d38-55cc7688d9-8p622\" (UID: \"72beae27-927d-4475-8923-c84bdb428f47\") " pod="openshift-ingress/payload-processing-e2e-mt-988c9d38-55cc7688d9-8p622"
Sep 18 14:18:21.943468 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:18:21.943439    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-988c9d38-55cc7688d9-8p622"
Sep 18 14:18:22.127798 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:18:22.127773    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-988c9d38-7b49d89946-fxv7s"]
Sep 18 14:18:22.129004 ip-10-0-129-16 kubenswrapper[2565]: W0918 14:18:22.128981    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5faa5aae_022a_4150_b351_bdf82de4379a.slice/crio-bfc0fb406377c5c60a4fd27eac42709eb03ffbd237f7160c75494f37534163d2 WatchSource:0}: Error finding container bfc0fb406377c5c60a4fd27eac42709eb03ffbd237f7160c75494f37534163d2: Status 404 returned error can't find the container with id bfc0fb406377c5c60a4fd27eac42709eb03ffbd237f7160c75494f37534163d2
Sep 18 14:18:22.225366 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:18:22.225333    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-988c9d38-55cc7688d9-8p622"]
Sep 18 14:18:22.226093 ip-10-0-129-16 kubenswrapper[2565]: W0918 14:18:22.226071    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod72beae27_927d_4475_8923_c84bdb428f47.slice/crio-7a8b8ce779f224d1b5dfaa418c3bf1f10d549c22abacec0ee17a0c6aa8fd9474 WatchSource:0}: Error finding container 7a8b8ce779f224d1b5dfaa418c3bf1f10d549c22abacec0ee17a0c6aa8fd9474: Status 404 returned error can't find the container with id 7a8b8ce779f224d1b5dfaa418c3bf1f10d549c22abacec0ee17a0c6aa8fd9474
Sep 18 14:18:22.537145 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:18:22.537050    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-988c9d38-55cc7688d9-8p622" event={"ID":"72beae27-927d-4475-8923-c84bdb428f47","Type":"ContainerStarted","Data":"be8bf95b65e50a884c799985ac6e8380963bd0ab6fc36a52438e74c710cb57d0"}
Sep 18 14:18:22.537145 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:18:22.537114    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-988c9d38-55cc7688d9-8p622" event={"ID":"72beae27-927d-4475-8923-c84bdb428f47","Type":"ContainerStarted","Data":"7a8b8ce779f224d1b5dfaa418c3bf1f10d549c22abacec0ee17a0c6aa8fd9474"}
Sep 18 14:18:22.537715 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:18:22.537236    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-mt-988c9d38-55cc7688d9-8p622"
Sep 18 14:18:22.538988 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:18:22.538961    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-988c9d38-7b49d89946-fxv7s" event={"ID":"5faa5aae-022a-4150-b351-bdf82de4379a","Type":"ContainerStarted","Data":"7a1178d84ca8b75ed1cb76641d8b18ea7d39f95b756ac1b9448443cbff812b42"}
Sep 18 14:18:22.539145 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:18:22.539010    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-988c9d38-7b49d89946-fxv7s" event={"ID":"5faa5aae-022a-4150-b351-bdf82de4379a","Type":"ContainerStarted","Data":"bfc0fb406377c5c60a4fd27eac42709eb03ffbd237f7160c75494f37534163d2"}
Sep 18 14:18:22.539145 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:18:22.539081    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-mt-988c9d38-7b49d89946-fxv7s"
Sep 18 14:18:22.560411 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:18:22.560362    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-mt-988c9d38-55cc7688d9-8p622" podStartSLOduration=1.560347416 podStartE2EDuration="1.560347416s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 14:18:21 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 14:18:22.55759447 +0000 UTC m=+1170.731504154" watchObservedRunningTime="2026-09-18 14:18:22.560347416 +0000 UTC m=+1170.734257075"
Sep 18 14:18:22.578584 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:18:22.578517    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-mt-988c9d38-7b49d89946-fxv7s" podStartSLOduration=1.578497866 podStartE2EDuration="1.578497866s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 14:18:21 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 14:18:22.575325984 +0000 UTC m=+1170.749235670" watchObservedRunningTime="2026-09-18 14:18:22.578497866 +0000 UTC m=+1170.752407531"
Sep 18 14:18:26.500378 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:18:26.500345    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-shared-a-w3-ae54af-5cc95698b7-w4fhd"
Sep 18 14:18:33.546763 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:18:33.546732    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-mt-988c9d38-55cc7688d9-8p622"
Sep 18 14:18:33.547197 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:18:33.546954    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-mt-988c9d38-7b49d89946-fxv7s"
Sep 18 14:18:53.952610 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:18:53.952580    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-bvqls_0b0463f5-d647-4a38-b752-44afba77daff/ovn-acl-logging/0.log"
Sep 18 14:18:53.957224 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:18:53.957204    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-bvqls_0b0463f5-d647-4a38-b752-44afba77daff/ovn-acl-logging/0.log"
Sep 18 14:19:54.078295 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:19:54.078217    2565 scope.go:117] "RemoveContainer" containerID="1868f276911a84dec91efcc375edcc1042fd3852709c3000ef0aaf8e13d346cb"
Sep 18 14:19:54.087505 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:19:54.087480    2565 scope.go:117] "RemoveContainer" containerID="8025c40be09f8d47e082416544e182bb0031bca6be01b9e6e89fbf45c7765c8b"
Sep 18 14:20:47.150584 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:20:47.150544    2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-988c9d38-cs8kl"]
Sep 18 14:20:47.154504 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:20:47.154484    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-988c9d38-cs8kl"
Sep 18 14:20:47.160327 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:20:47.160306    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-l4fn9\""
Sep 18 14:20:47.169221 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:20:47.169199    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-988c9d38-cs8kl"]
Sep 18 14:20:47.234677 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:20:47.234646    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/97b4e07f-93d5-4c79-b13f-86f134e6eb64-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-988c9d38-cs8kl\" (UID: \"97b4e07f-93d5-4c79-b13f-86f134e6eb64\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-988c9d38-cs8kl"
Sep 18 14:20:47.336297 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:20:47.336235    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/97b4e07f-93d5-4c79-b13f-86f134e6eb64-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-988c9d38-cs8kl\" (UID: \"97b4e07f-93d5-4c79-b13f-86f134e6eb64\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-988c9d38-cs8kl"
Sep 18 14:20:47.336968 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:20:47.336944    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/97b4e07f-93d5-4c79-b13f-86f134e6eb64-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-988c9d38-cs8kl\" (UID: \"97b4e07f-93d5-4c79-b13f-86f134e6eb64\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-988c9d38-cs8kl"
Sep 18 14:20:47.467200 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:20:47.467108    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-988c9d38-cs8kl"
Sep 18 14:20:47.750635 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:20:47.750604    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-988c9d38-cs8kl"]
Sep 18 14:20:47.753195 ip-10-0-129-16 kubenswrapper[2565]: W0918 14:20:47.753163    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod97b4e07f_93d5_4c79_b13f_86f134e6eb64.slice/crio-330926988c106c6f11ea7f4883786103c72df038ba07d8f50f6ff8757e6f2389 WatchSource:0}: Error finding container 330926988c106c6f11ea7f4883786103c72df038ba07d8f50f6ff8757e6f2389: Status 404 returned error can't find the container with id 330926988c106c6f11ea7f4883786103c72df038ba07d8f50f6ff8757e6f2389
Sep 18 14:20:48.484121 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:20:48.484018    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-988c9d38-cs8kl" event={"ID":"97b4e07f-93d5-4c79-b13f-86f134e6eb64","Type":"ContainerStarted","Data":"efaf61e2c52990d8d882764d86a6498c2dacbf9ab77bbf1befe5b370dee1b9ef"}
Sep 18 14:20:48.484121 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:20:48.484081    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-988c9d38-cs8kl" event={"ID":"97b4e07f-93d5-4c79-b13f-86f134e6eb64","Type":"ContainerStarted","Data":"330926988c106c6f11ea7f4883786103c72df038ba07d8f50f6ff8757e6f2389"}
Sep 18 14:20:48.504047 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:20:48.503986    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-988c9d38-cs8kl" podStartSLOduration=1.503970143 podStartE2EDuration="1.503970143s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 14:20:47 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 14:20:48.503596432 +0000 UTC m=+1316.677506107" watchObservedRunningTime="2026-09-18 14:20:48.503970143 +0000 UTC m=+1316.677879803"
Sep 18 14:20:49.491451 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:20:49.491417    2565 generic.go:356] "Generic (PLEG): container finished" podID="97b4e07f-93d5-4c79-b13f-86f134e6eb64" containerID="efaf61e2c52990d8d882764d86a6498c2dacbf9ab77bbf1befe5b370dee1b9ef" exitCode=0
Sep 18 14:20:49.491829 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:20:49.491512    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-988c9d38-cs8kl" event={"ID":"97b4e07f-93d5-4c79-b13f-86f134e6eb64","Type":"ContainerDied","Data":"efaf61e2c52990d8d882764d86a6498c2dacbf9ab77bbf1befe5b370dee1b9ef"}
Sep 18 14:20:50.650214 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:20:50.650187    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-988c9d38-cs8kl"
Sep 18 14:20:50.774077 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:20:50.773956    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/97b4e07f-93d5-4c79-b13f-86f134e6eb64-maas-api-service-ca\") pod \"97b4e07f-93d5-4c79-b13f-86f134e6eb64\" (UID: \"97b4e07f-93d5-4c79-b13f-86f134e6eb64\") "
Sep 18 14:20:50.774330 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:20:50.774310    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/97b4e07f-93d5-4c79-b13f-86f134e6eb64-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "97b4e07f-93d5-4c79-b13f-86f134e6eb64" (UID: "97b4e07f-93d5-4c79-b13f-86f134e6eb64"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 18 14:20:50.875253 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:20:50.875219    2565 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/97b4e07f-93d5-4c79-b13f-86f134e6eb64-maas-api-service-ca\") on node \"ip-10-0-129-16.ec2.internal\" DevicePath \"\""
Sep 18 14:20:51.506836 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:20:51.506806    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-988c9d38-cs8kl"
Sep 18 14:20:51.507010 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:20:51.506807    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-988c9d38-cs8kl" event={"ID":"97b4e07f-93d5-4c79-b13f-86f134e6eb64","Type":"ContainerDied","Data":"330926988c106c6f11ea7f4883786103c72df038ba07d8f50f6ff8757e6f2389"}
Sep 18 14:20:51.507010 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:20:51.506956    2565 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="330926988c106c6f11ea7f4883786103c72df038ba07d8f50f6ff8757e6f2389"
Sep 18 14:21:03.042984 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:21:03.042947    2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-988c9d38-55cc7688d9-8p622"]
Sep 18 14:21:03.043641 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:21:03.043237    2565 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-mt-988c9d38-55cc7688d9-8p622" podUID="72beae27-927d-4475-8923-c84bdb428f47" containerName="payload-processing" containerID="cri-o://be8bf95b65e50a884c799985ac6e8380963bd0ab6fc36a52438e74c710cb57d0" gracePeriod=30
Sep 18 14:21:03.059992 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:21:03.059953    2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-988c9d38-7b49d89946-fxv7s"]
Sep 18 14:21:03.060550 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:21:03.060469    2565 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-mt-988c9d38-7b49d89946-fxv7s" podUID="5faa5aae-022a-4150-b351-bdf82de4379a" containerName="payload-pre-processing" containerID="cri-o://7a1178d84ca8b75ed1cb76641d8b18ea7d39f95b756ac1b9448443cbff812b42" gracePeriod=30
Sep 18 14:21:03.349617 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:21:03.349590    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-988c9d38-55cc7688d9-8p622"
Sep 18 14:21:03.370547 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:21:03.370524    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-988c9d38-7b49d89946-fxv7s"
Sep 18 14:21:03.511685 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:21:03.511648    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/72beae27-927d-4475-8923-c84bdb428f47-plugins-config-volume\") pod \"72beae27-927d-4475-8923-c84bdb428f47\" (UID: \"72beae27-927d-4475-8923-c84bdb428f47\") "
Sep 18 14:21:03.511855 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:21:03.511755    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-66fkk\" (UniqueName: \"kubernetes.io/projected/72beae27-927d-4475-8923-c84bdb428f47-kube-api-access-66fkk\") pod \"72beae27-927d-4475-8923-c84bdb428f47\" (UID: \"72beae27-927d-4475-8923-c84bdb428f47\") "
Sep 18 14:21:03.511855 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:21:03.511805    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-hh6mc\" (UniqueName: \"kubernetes.io/projected/5faa5aae-022a-4150-b351-bdf82de4379a-kube-api-access-hh6mc\") pod \"5faa5aae-022a-4150-b351-bdf82de4379a\" (UID: \"5faa5aae-022a-4150-b351-bdf82de4379a\") "
Sep 18 14:21:03.511968 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:21:03.511953    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/5faa5aae-022a-4150-b351-bdf82de4379a-plugins-config-volume\") pod \"5faa5aae-022a-4150-b351-bdf82de4379a\" (UID: \"5faa5aae-022a-4150-b351-bdf82de4379a\") "
Sep 18 14:21:03.512174 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:21:03.512139    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/72beae27-927d-4475-8923-c84bdb428f47-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "72beae27-927d-4475-8923-c84bdb428f47" (UID: "72beae27-927d-4475-8923-c84bdb428f47"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 18 14:21:03.512365 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:21:03.512342    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5faa5aae-022a-4150-b351-bdf82de4379a-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "5faa5aae-022a-4150-b351-bdf82de4379a" (UID: "5faa5aae-022a-4150-b351-bdf82de4379a"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 18 14:21:03.512446 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:21:03.512421    2565 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/72beae27-927d-4475-8923-c84bdb428f47-plugins-config-volume\") on node \"ip-10-0-129-16.ec2.internal\" DevicePath \"\""
Sep 18 14:21:03.514506 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:21:03.514486    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/72beae27-927d-4475-8923-c84bdb428f47-kube-api-access-66fkk" (OuterVolumeSpecName: "kube-api-access-66fkk") pod "72beae27-927d-4475-8923-c84bdb428f47" (UID: "72beae27-927d-4475-8923-c84bdb428f47"). InnerVolumeSpecName "kube-api-access-66fkk". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 18 14:21:03.514581 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:21:03.514510    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5faa5aae-022a-4150-b351-bdf82de4379a-kube-api-access-hh6mc" (OuterVolumeSpecName: "kube-api-access-hh6mc") pod "5faa5aae-022a-4150-b351-bdf82de4379a" (UID: "5faa5aae-022a-4150-b351-bdf82de4379a"). InnerVolumeSpecName "kube-api-access-hh6mc". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 18 14:21:03.582909 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:21:03.582813    2565 generic.go:356] "Generic (PLEG): container finished" podID="72beae27-927d-4475-8923-c84bdb428f47" containerID="be8bf95b65e50a884c799985ac6e8380963bd0ab6fc36a52438e74c710cb57d0" exitCode=0
Sep 18 14:21:03.582909 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:21:03.582848    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-988c9d38-55cc7688d9-8p622" event={"ID":"72beae27-927d-4475-8923-c84bdb428f47","Type":"ContainerDied","Data":"be8bf95b65e50a884c799985ac6e8380963bd0ab6fc36a52438e74c710cb57d0"}
Sep 18 14:21:03.582909 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:21:03.582906    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-988c9d38-55cc7688d9-8p622" event={"ID":"72beae27-927d-4475-8923-c84bdb428f47","Type":"ContainerDied","Data":"7a8b8ce779f224d1b5dfaa418c3bf1f10d549c22abacec0ee17a0c6aa8fd9474"}
Sep 18 14:21:03.583231 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:21:03.582863    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-988c9d38-55cc7688d9-8p622"
Sep 18 14:21:03.583231 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:21:03.582934    2565 scope.go:117] "RemoveContainer" containerID="be8bf95b65e50a884c799985ac6e8380963bd0ab6fc36a52438e74c710cb57d0"
Sep 18 14:21:03.584333 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:21:03.584312    2565 generic.go:356] "Generic (PLEG): container finished" podID="5faa5aae-022a-4150-b351-bdf82de4379a" containerID="7a1178d84ca8b75ed1cb76641d8b18ea7d39f95b756ac1b9448443cbff812b42" exitCode=0
Sep 18 14:21:03.584441 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:21:03.584353    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-988c9d38-7b49d89946-fxv7s"
Sep 18 14:21:03.584441 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:21:03.584366    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-988c9d38-7b49d89946-fxv7s" event={"ID":"5faa5aae-022a-4150-b351-bdf82de4379a","Type":"ContainerDied","Data":"7a1178d84ca8b75ed1cb76641d8b18ea7d39f95b756ac1b9448443cbff812b42"}
Sep 18 14:21:03.584441 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:21:03.584399    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-988c9d38-7b49d89946-fxv7s" event={"ID":"5faa5aae-022a-4150-b351-bdf82de4379a","Type":"ContainerDied","Data":"bfc0fb406377c5c60a4fd27eac42709eb03ffbd237f7160c75494f37534163d2"}
Sep 18 14:21:03.593179 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:21:03.593160    2565 scope.go:117] "RemoveContainer" containerID="be8bf95b65e50a884c799985ac6e8380963bd0ab6fc36a52438e74c710cb57d0"
Sep 18 14:21:03.593519 ip-10-0-129-16 kubenswrapper[2565]: E0918 14:21:03.593496    2565 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"be8bf95b65e50a884c799985ac6e8380963bd0ab6fc36a52438e74c710cb57d0\": container with ID starting with be8bf95b65e50a884c799985ac6e8380963bd0ab6fc36a52438e74c710cb57d0 not found: ID does not exist" containerID="be8bf95b65e50a884c799985ac6e8380963bd0ab6fc36a52438e74c710cb57d0"
Sep 18 14:21:03.593619 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:21:03.593531    2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"be8bf95b65e50a884c799985ac6e8380963bd0ab6fc36a52438e74c710cb57d0"} err="failed to get container status \"be8bf95b65e50a884c799985ac6e8380963bd0ab6fc36a52438e74c710cb57d0\": rpc error: code = NotFound desc = could not find container \"be8bf95b65e50a884c799985ac6e8380963bd0ab6fc36a52438e74c710cb57d0\": container with ID starting with be8bf95b65e50a884c799985ac6e8380963bd0ab6fc36a52438e74c710cb57d0 not found: ID does not exist"
Sep 18 14:21:03.593619 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:21:03.593555    2565 scope.go:117] "RemoveContainer" containerID="7a1178d84ca8b75ed1cb76641d8b18ea7d39f95b756ac1b9448443cbff812b42"
Sep 18 14:21:03.602456 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:21:03.602435    2565 scope.go:117] "RemoveContainer" containerID="7a1178d84ca8b75ed1cb76641d8b18ea7d39f95b756ac1b9448443cbff812b42"
Sep 18 14:21:03.602751 ip-10-0-129-16 kubenswrapper[2565]: E0918 14:21:03.602733    2565 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7a1178d84ca8b75ed1cb76641d8b18ea7d39f95b756ac1b9448443cbff812b42\": container with ID starting with 7a1178d84ca8b75ed1cb76641d8b18ea7d39f95b756ac1b9448443cbff812b42 not found: ID does not exist" containerID="7a1178d84ca8b75ed1cb76641d8b18ea7d39f95b756ac1b9448443cbff812b42"
Sep 18 14:21:03.602815 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:21:03.602758    2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7a1178d84ca8b75ed1cb76641d8b18ea7d39f95b756ac1b9448443cbff812b42"} err="failed to get container status \"7a1178d84ca8b75ed1cb76641d8b18ea7d39f95b756ac1b9448443cbff812b42\": rpc error: code = NotFound desc = could not find container \"7a1178d84ca8b75ed1cb76641d8b18ea7d39f95b756ac1b9448443cbff812b42\": container with ID starting with 7a1178d84ca8b75ed1cb76641d8b18ea7d39f95b756ac1b9448443cbff812b42 not found: ID does not exist"
Sep 18 14:21:03.613556 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:21:03.613526    2565 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-66fkk\" (UniqueName: \"kubernetes.io/projected/72beae27-927d-4475-8923-c84bdb428f47-kube-api-access-66fkk\") on node \"ip-10-0-129-16.ec2.internal\" DevicePath \"\""
Sep 18 14:21:03.613556 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:21:03.613558    2565 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hh6mc\" (UniqueName: \"kubernetes.io/projected/5faa5aae-022a-4150-b351-bdf82de4379a-kube-api-access-hh6mc\") on node \"ip-10-0-129-16.ec2.internal\" DevicePath \"\""
Sep 18 14:21:03.613776 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:21:03.613572    2565 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/5faa5aae-022a-4150-b351-bdf82de4379a-plugins-config-volume\") on node \"ip-10-0-129-16.ec2.internal\" DevicePath \"\""
Sep 18 14:21:03.616630 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:21:03.616603    2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-988c9d38-7b49d89946-fxv7s"]
Sep 18 14:21:03.629620 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:21:03.629593    2565 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-988c9d38-7b49d89946-fxv7s"]
Sep 18 14:21:03.649991 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:21:03.649956    2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-988c9d38-55cc7688d9-8p622"]
Sep 18 14:21:03.661962 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:21:03.661908    2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-988c9d38-cs8kl"]
Sep 18 14:21:03.665682 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:21:03.665653    2565 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-988c9d38-55cc7688d9-8p622"]
Sep 18 14:21:03.668320 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:21:03.668297    2565 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-988c9d38-cs8kl"]
Sep 18 14:21:03.914094 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:21:03.913984    2565 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5faa5aae-022a-4150-b351-bdf82de4379a" path="/var/lib/kubelet/pods/5faa5aae-022a-4150-b351-bdf82de4379a/volumes"
Sep 18 14:21:03.914408 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:21:03.914392    2565 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="72beae27-927d-4475-8923-c84bdb428f47" path="/var/lib/kubelet/pods/72beae27-927d-4475-8923-c84bdb428f47/volumes"
Sep 18 14:21:03.914719 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:21:03.914706    2565 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="97b4e07f-93d5-4c79-b13f-86f134e6eb64" path="/var/lib/kubelet/pods/97b4e07f-93d5-4c79-b13f-86f134e6eb64/volumes"
Sep 18 14:21:34.220473 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:21:34.220438    2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["e2e-models-e2e-worker-w0-4d9222/embedding-w0-4d9222-kserve-bb67c65-gnwmq"]
Sep 18 14:21:34.221098 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:21:34.220940    2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="72beae27-927d-4475-8923-c84bdb428f47" containerName="payload-processing"
Sep 18 14:21:34.221098 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:21:34.220954    2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="72beae27-927d-4475-8923-c84bdb428f47" containerName="payload-processing"
Sep 18 14:21:34.221098 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:21:34.220981    2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5faa5aae-022a-4150-b351-bdf82de4379a" containerName="payload-pre-processing"
Sep 18 14:21:34.221098 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:21:34.220986    2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="5faa5aae-022a-4150-b351-bdf82de4379a" containerName="payload-pre-processing"
Sep 18 14:21:34.221098 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:21:34.220999    2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="97b4e07f-93d5-4c79-b13f-86f134e6eb64" containerName="revoke-keys"
Sep 18 14:21:34.221098 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:21:34.221005    2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="97b4e07f-93d5-4c79-b13f-86f134e6eb64" containerName="revoke-keys"
Sep 18 14:21:34.221098 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:21:34.221091    2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="5faa5aae-022a-4150-b351-bdf82de4379a" containerName="payload-pre-processing"
Sep 18 14:21:34.221098 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:21:34.221101    2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="97b4e07f-93d5-4c79-b13f-86f134e6eb64" containerName="revoke-keys"
Sep 18 14:21:34.221547 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:21:34.221109    2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="72beae27-927d-4475-8923-c84bdb428f47" containerName="payload-processing"
Sep 18 14:21:34.224072 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:21:34.224026    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w0-4d9222/embedding-w0-4d9222-kserve-bb67c65-gnwmq"
Sep 18 14:21:34.226888 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:21:34.226864    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w0-4d9222\"/\"kube-root-ca.crt\""
Sep 18 14:21:34.227958 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:21:34.227936    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w0-4d9222\"/\"default-dockercfg-st5nr\""
Sep 18 14:21:34.228111 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:21:34.228062    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w0-4d9222\"/\"openshift-service-ca.crt\""
Sep 18 14:21:34.228176 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:21:34.228142    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w0-4d9222\"/\"embedding-w0-4d9222-kserve-self-signed-certs\""
Sep 18 14:21:34.234226 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:21:34.233935    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w0-4d9222/embedding-w0-4d9222-kserve-bb67c65-gnwmq"]
Sep 18 14:21:34.342580 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:21:34.342552    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/67d8cf34-7edd-4029-8038-4b4255e56c62-tmp-dir\") pod \"embedding-w0-4d9222-kserve-bb67c65-gnwmq\" (UID: \"67d8cf34-7edd-4029-8038-4b4255e56c62\") " pod="e2e-models-e2e-worker-w0-4d9222/embedding-w0-4d9222-kserve-bb67c65-gnwmq"
Sep 18 14:21:34.342772 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:21:34.342613    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-d8fqv\" (UniqueName: \"kubernetes.io/projected/67d8cf34-7edd-4029-8038-4b4255e56c62-kube-api-access-d8fqv\") pod \"embedding-w0-4d9222-kserve-bb67c65-gnwmq\" (UID: \"67d8cf34-7edd-4029-8038-4b4255e56c62\") " pod="e2e-models-e2e-worker-w0-4d9222/embedding-w0-4d9222-kserve-bb67c65-gnwmq"
Sep 18 14:21:34.342772 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:21:34.342680    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/67d8cf34-7edd-4029-8038-4b4255e56c62-dshm\") pod \"embedding-w0-4d9222-kserve-bb67c65-gnwmq\" (UID: \"67d8cf34-7edd-4029-8038-4b4255e56c62\") " pod="e2e-models-e2e-worker-w0-4d9222/embedding-w0-4d9222-kserve-bb67c65-gnwmq"
Sep 18 14:21:34.342772 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:21:34.342746    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/67d8cf34-7edd-4029-8038-4b4255e56c62-model-cache\") pod \"embedding-w0-4d9222-kserve-bb67c65-gnwmq\" (UID: \"67d8cf34-7edd-4029-8038-4b4255e56c62\") " pod="e2e-models-e2e-worker-w0-4d9222/embedding-w0-4d9222-kserve-bb67c65-gnwmq"
Sep 18 14:21:34.342910 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:21:34.342777    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/67d8cf34-7edd-4029-8038-4b4255e56c62-home\") pod \"embedding-w0-4d9222-kserve-bb67c65-gnwmq\" (UID: \"67d8cf34-7edd-4029-8038-4b4255e56c62\") " pod="e2e-models-e2e-worker-w0-4d9222/embedding-w0-4d9222-kserve-bb67c65-gnwmq"
Sep 18 14:21:34.342910 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:21:34.342820    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/67d8cf34-7edd-4029-8038-4b4255e56c62-tls-certs\") pod \"embedding-w0-4d9222-kserve-bb67c65-gnwmq\" (UID: \"67d8cf34-7edd-4029-8038-4b4255e56c62\") " pod="e2e-models-e2e-worker-w0-4d9222/embedding-w0-4d9222-kserve-bb67c65-gnwmq"
Sep 18 14:21:34.443687 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:21:34.443647    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/67d8cf34-7edd-4029-8038-4b4255e56c62-tls-certs\") pod \"embedding-w0-4d9222-kserve-bb67c65-gnwmq\" (UID: \"67d8cf34-7edd-4029-8038-4b4255e56c62\") " pod="e2e-models-e2e-worker-w0-4d9222/embedding-w0-4d9222-kserve-bb67c65-gnwmq"
Sep 18 14:21:34.443890 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:21:34.443714    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/67d8cf34-7edd-4029-8038-4b4255e56c62-tmp-dir\") pod \"embedding-w0-4d9222-kserve-bb67c65-gnwmq\" (UID: \"67d8cf34-7edd-4029-8038-4b4255e56c62\") " pod="e2e-models-e2e-worker-w0-4d9222/embedding-w0-4d9222-kserve-bb67c65-gnwmq"
Sep 18 14:21:34.443890 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:21:34.443763    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-d8fqv\" (UniqueName: \"kubernetes.io/projected/67d8cf34-7edd-4029-8038-4b4255e56c62-kube-api-access-d8fqv\") pod \"embedding-w0-4d9222-kserve-bb67c65-gnwmq\" (UID: \"67d8cf34-7edd-4029-8038-4b4255e56c62\") " pod="e2e-models-e2e-worker-w0-4d9222/embedding-w0-4d9222-kserve-bb67c65-gnwmq"
Sep 18 14:21:34.443890 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:21:34.443787    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/67d8cf34-7edd-4029-8038-4b4255e56c62-dshm\") pod \"embedding-w0-4d9222-kserve-bb67c65-gnwmq\" (UID: \"67d8cf34-7edd-4029-8038-4b4255e56c62\") " pod="e2e-models-e2e-worker-w0-4d9222/embedding-w0-4d9222-kserve-bb67c65-gnwmq"
Sep 18 14:21:34.443890 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:21:34.443829    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/67d8cf34-7edd-4029-8038-4b4255e56c62-model-cache\") pod \"embedding-w0-4d9222-kserve-bb67c65-gnwmq\" (UID: \"67d8cf34-7edd-4029-8038-4b4255e56c62\") " pod="e2e-models-e2e-worker-w0-4d9222/embedding-w0-4d9222-kserve-bb67c65-gnwmq"
Sep 18 14:21:34.443890 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:21:34.443851    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/67d8cf34-7edd-4029-8038-4b4255e56c62-home\") pod \"embedding-w0-4d9222-kserve-bb67c65-gnwmq\" (UID: \"67d8cf34-7edd-4029-8038-4b4255e56c62\") " pod="e2e-models-e2e-worker-w0-4d9222/embedding-w0-4d9222-kserve-bb67c65-gnwmq"
Sep 18 14:21:34.444252 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:21:34.444230    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/67d8cf34-7edd-4029-8038-4b4255e56c62-tmp-dir\") pod \"embedding-w0-4d9222-kserve-bb67c65-gnwmq\" (UID: \"67d8cf34-7edd-4029-8038-4b4255e56c62\") " pod="e2e-models-e2e-worker-w0-4d9222/embedding-w0-4d9222-kserve-bb67c65-gnwmq"
Sep 18 14:21:34.444312 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:21:34.444273    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/67d8cf34-7edd-4029-8038-4b4255e56c62-home\") pod \"embedding-w0-4d9222-kserve-bb67c65-gnwmq\" (UID: \"67d8cf34-7edd-4029-8038-4b4255e56c62\") " pod="e2e-models-e2e-worker-w0-4d9222/embedding-w0-4d9222-kserve-bb67c65-gnwmq"
Sep 18 14:21:34.444366 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:21:34.444322    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/67d8cf34-7edd-4029-8038-4b4255e56c62-model-cache\") pod \"embedding-w0-4d9222-kserve-bb67c65-gnwmq\" (UID: \"67d8cf34-7edd-4029-8038-4b4255e56c62\") " pod="e2e-models-e2e-worker-w0-4d9222/embedding-w0-4d9222-kserve-bb67c65-gnwmq"
Sep 18 14:21:34.446071 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:21:34.446049    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/67d8cf34-7edd-4029-8038-4b4255e56c62-dshm\") pod \"embedding-w0-4d9222-kserve-bb67c65-gnwmq\" (UID: \"67d8cf34-7edd-4029-8038-4b4255e56c62\") " pod="e2e-models-e2e-worker-w0-4d9222/embedding-w0-4d9222-kserve-bb67c65-gnwmq"
Sep 18 14:21:34.446346 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:21:34.446329    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/67d8cf34-7edd-4029-8038-4b4255e56c62-tls-certs\") pod \"embedding-w0-4d9222-kserve-bb67c65-gnwmq\" (UID: \"67d8cf34-7edd-4029-8038-4b4255e56c62\") " pod="e2e-models-e2e-worker-w0-4d9222/embedding-w0-4d9222-kserve-bb67c65-gnwmq"
Sep 18 14:21:34.452250 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:21:34.452226    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-d8fqv\" (UniqueName: \"kubernetes.io/projected/67d8cf34-7edd-4029-8038-4b4255e56c62-kube-api-access-d8fqv\") pod \"embedding-w0-4d9222-kserve-bb67c65-gnwmq\" (UID: \"67d8cf34-7edd-4029-8038-4b4255e56c62\") " pod="e2e-models-e2e-worker-w0-4d9222/embedding-w0-4d9222-kserve-bb67c65-gnwmq"
Sep 18 14:21:34.537551 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:21:34.537522    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w0-4d9222/embedding-w0-4d9222-kserve-bb67c65-gnwmq"
Sep 18 14:21:34.813088 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:21:34.813060    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w0-4d9222/embedding-w0-4d9222-kserve-bb67c65-gnwmq"]
Sep 18 14:21:34.814708 ip-10-0-129-16 kubenswrapper[2565]: W0918 14:21:34.814682    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod67d8cf34_7edd_4029_8038_4b4255e56c62.slice/crio-b53e829e9aabf66fe7d91c97573ec1e429a08de2f41e78b29621ba62bd53e985 WatchSource:0}: Error finding container b53e829e9aabf66fe7d91c97573ec1e429a08de2f41e78b29621ba62bd53e985: Status 404 returned error can't find the container with id b53e829e9aabf66fe7d91c97573ec1e429a08de2f41e78b29621ba62bd53e985
Sep 18 14:21:35.789602 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:21:35.789570    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w0-4d9222/embedding-w0-4d9222-kserve-bb67c65-gnwmq" event={"ID":"67d8cf34-7edd-4029-8038-4b4255e56c62","Type":"ContainerStarted","Data":"eac611d591e28a6b37125512508515a0c4c29775e13735e7f0e8b65258d2f5ac"}
Sep 18 14:21:35.790075 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:21:35.789614    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w0-4d9222/embedding-w0-4d9222-kserve-bb67c65-gnwmq" event={"ID":"67d8cf34-7edd-4029-8038-4b4255e56c62","Type":"ContainerStarted","Data":"b53e829e9aabf66fe7d91c97573ec1e429a08de2f41e78b29621ba62bd53e985"}
Sep 18 14:21:35.810002 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:21:35.809946    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="e2e-models-e2e-worker-w0-4d9222/embedding-w0-4d9222-kserve-bb67c65-gnwmq" podStartSLOduration=1.8099291819999999 podStartE2EDuration="1.809929182s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 14:21:34 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 14:21:35.807192956 +0000 UTC m=+1363.981102629" watchObservedRunningTime="2026-09-18 14:21:35.809929182 +0000 UTC m=+1363.983838841"
Sep 18 14:21:44.538488 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:21:44.538438    2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-models-e2e-worker-w0-4d9222/embedding-w0-4d9222-kserve-bb67c65-gnwmq"
Sep 18 14:21:44.538488 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:21:44.538494    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w0-4d9222/embedding-w0-4d9222-kserve-bb67c65-gnwmq"
Sep 18 14:21:44.551786 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:21:44.551760    2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-models-e2e-worker-w0-4d9222/embedding-w0-4d9222-kserve-bb67c65-gnwmq"
Sep 18 14:21:44.855594 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:21:44.855513    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w0-4d9222/embedding-w0-4d9222-kserve-bb67c65-gnwmq"
Sep 18 14:21:54.124813 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:21:54.124778    2565 scope.go:117] "RemoveContainer" containerID="7a40cac592febadff6a178266c369a249df8225900737755c947dc9c94854acd"
Sep 18 14:21:54.133961 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:21:54.133939    2565 scope.go:117] "RemoveContainer" containerID="6cc6684730a09fb270c3bf20681f6e71a1942f949df164ef9e0e585961042aa1"
Sep 18 14:21:54.142271 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:21:54.142247    2565 scope.go:117] "RemoveContainer" containerID="21cd300749400e8ba8c036160c8edce1732f502c195869023ada3f31398e8b69"
Sep 18 14:22:40.996634 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:22:40.996528    2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w0-4d9222-vxb2k"]
Sep 18 14:22:41.000810 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:22:41.000787    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w0-4d9222-vxb2k"
Sep 18 14:22:41.003567 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:22:41.003548    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-l4fn9\""
Sep 18 14:22:41.010160 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:22:41.010131    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w0-4d9222-vxb2k"]
Sep 18 14:22:41.075986 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:22:41.075956    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/bc133952-f64e-40bb-a76b-0b30fc979393-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-worker-w0-4d9222-vxb2k\" (UID: \"bc133952-f64e-40bb-a76b-0b30fc979393\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w0-4d9222-vxb2k"
Sep 18 14:22:41.177798 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:22:41.177761    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/bc133952-f64e-40bb-a76b-0b30fc979393-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-worker-w0-4d9222-vxb2k\" (UID: \"bc133952-f64e-40bb-a76b-0b30fc979393\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w0-4d9222-vxb2k"
Sep 18 14:22:41.178412 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:22:41.178393    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/bc133952-f64e-40bb-a76b-0b30fc979393-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-worker-w0-4d9222-vxb2k\" (UID: \"bc133952-f64e-40bb-a76b-0b30fc979393\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w0-4d9222-vxb2k"
Sep 18 14:22:41.312558 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:22:41.312526    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w0-4d9222-vxb2k"
Sep 18 14:22:41.588234 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:22:41.588204    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w0-4d9222-vxb2k"]
Sep 18 14:22:41.589808 ip-10-0-129-16 kubenswrapper[2565]: W0918 14:22:41.589774    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbc133952_f64e_40bb_a76b_0b30fc979393.slice/crio-182ebc0c08b212c49b342b53af804b3a964de8617bfc549eb105100b577eb7cb WatchSource:0}: Error finding container 182ebc0c08b212c49b342b53af804b3a964de8617bfc549eb105100b577eb7cb: Status 404 returned error can't find the container with id 182ebc0c08b212c49b342b53af804b3a964de8617bfc549eb105100b577eb7cb
Sep 18 14:22:42.193851 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:22:42.193813    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w0-4d9222-vxb2k" event={"ID":"bc133952-f64e-40bb-a76b-0b30fc979393","Type":"ContainerStarted","Data":"29eef1660d79ba3294b8daeabe7799040ce17a57ea6485de4026dedc10763bfb"}
Sep 18 14:22:42.194326 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:22:42.193865    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w0-4d9222-vxb2k" event={"ID":"bc133952-f64e-40bb-a76b-0b30fc979393","Type":"ContainerStarted","Data":"182ebc0c08b212c49b342b53af804b3a964de8617bfc549eb105100b577eb7cb"}
Sep 18 14:22:42.213202 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:22:42.213148    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w0-4d9222-vxb2k" podStartSLOduration=2.213131917 podStartE2EDuration="2.213131917s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 14:22:40 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 14:22:42.2112026 +0000 UTC m=+1430.385112261" watchObservedRunningTime="2026-09-18 14:22:42.213131917 +0000 UTC m=+1430.387041577"
Sep 18 14:22:43.201556 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:22:43.201521    2565 generic.go:356] "Generic (PLEG): container finished" podID="bc133952-f64e-40bb-a76b-0b30fc979393" containerID="29eef1660d79ba3294b8daeabe7799040ce17a57ea6485de4026dedc10763bfb" exitCode=0
Sep 18 14:22:43.201960 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:22:43.201608    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w0-4d9222-vxb2k" event={"ID":"bc133952-f64e-40bb-a76b-0b30fc979393","Type":"ContainerDied","Data":"29eef1660d79ba3294b8daeabe7799040ce17a57ea6485de4026dedc10763bfb"}
Sep 18 14:22:44.356647 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:22:44.356619    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w0-4d9222-vxb2k"
Sep 18 14:22:44.415177 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:22:44.415146    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/bc133952-f64e-40bb-a76b-0b30fc979393-maas-api-service-ca\") pod \"bc133952-f64e-40bb-a76b-0b30fc979393\" (UID: \"bc133952-f64e-40bb-a76b-0b30fc979393\") "
Sep 18 14:22:44.415542 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:22:44.415518    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/bc133952-f64e-40bb-a76b-0b30fc979393-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "bc133952-f64e-40bb-a76b-0b30fc979393" (UID: "bc133952-f64e-40bb-a76b-0b30fc979393"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 18 14:22:44.516547 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:22:44.516458    2565 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/bc133952-f64e-40bb-a76b-0b30fc979393-maas-api-service-ca\") on node \"ip-10-0-129-16.ec2.internal\" DevicePath \"\""
Sep 18 14:22:45.214437 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:22:45.214400    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w0-4d9222-vxb2k" event={"ID":"bc133952-f64e-40bb-a76b-0b30fc979393","Type":"ContainerDied","Data":"182ebc0c08b212c49b342b53af804b3a964de8617bfc549eb105100b577eb7cb"}
Sep 18 14:22:45.214614 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:22:45.214442    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w0-4d9222-vxb2k"
Sep 18 14:22:45.214614 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:22:45.214458    2565 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="182ebc0c08b212c49b342b53af804b3a964de8617bfc549eb105100b577eb7cb"
Sep 18 14:22:45.873503 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:22:45.873471    2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w0-4d9222/embedding-w0-4d9222-kserve-bb67c65-gnwmq"]
Sep 18 14:22:54.179858 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:22:54.179823    2565 scope.go:117] "RemoveContainer" containerID="01f60abb94b3e30d29f7148984bcb6ea44b049c6cbb79972c5ce6c50015a5430"
Sep 18 14:22:56.968244 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:22:56.968212    2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w0-4d9222-67d45969f8-gwf8b"]
Sep 18 14:22:56.968732 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:22:56.968413    2565 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-4d9222-67d45969f8-gwf8b" podUID="609ab3f8-150f-40fe-956c-139806aee3ce" containerName="payload-pre-processing" containerID="cri-o://e61330fad1730e830efaf350fd60aea2b2c56292d992579f4ea9ef889c7eaac5" gracePeriod=30
Sep 18 14:22:57.222806 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:22:57.222712    2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w0-4d9222-vxb2k"]
Sep 18 14:22:57.229163 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:22:57.229134    2565 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w0-4d9222-vxb2k"]
Sep 18 14:22:57.286522 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:22:57.286496    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-4d9222-67d45969f8-gwf8b"
Sep 18 14:22:57.292787 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:22:57.292760    2565 generic.go:356] "Generic (PLEG): container finished" podID="609ab3f8-150f-40fe-956c-139806aee3ce" containerID="e61330fad1730e830efaf350fd60aea2b2c56292d992579f4ea9ef889c7eaac5" exitCode=0
Sep 18 14:22:57.292907 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:22:57.292808    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-4d9222-67d45969f8-gwf8b"
Sep 18 14:22:57.292907 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:22:57.292803    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-4d9222-67d45969f8-gwf8b" event={"ID":"609ab3f8-150f-40fe-956c-139806aee3ce","Type":"ContainerDied","Data":"e61330fad1730e830efaf350fd60aea2b2c56292d992579f4ea9ef889c7eaac5"}
Sep 18 14:22:57.292999 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:22:57.292932    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-4d9222-67d45969f8-gwf8b" event={"ID":"609ab3f8-150f-40fe-956c-139806aee3ce","Type":"ContainerDied","Data":"80d9a6f343564abcd086f602d8623ee746c20bd8a487ff51bf7d0226c8a6df07"}
Sep 18 14:22:57.292999 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:22:57.292964    2565 scope.go:117] "RemoveContainer" containerID="e61330fad1730e830efaf350fd60aea2b2c56292d992579f4ea9ef889c7eaac5"
Sep 18 14:22:57.302824 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:22:57.302806    2565 scope.go:117] "RemoveContainer" containerID="e61330fad1730e830efaf350fd60aea2b2c56292d992579f4ea9ef889c7eaac5"
Sep 18 14:22:57.303133 ip-10-0-129-16 kubenswrapper[2565]: E0918 14:22:57.303113    2565 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e61330fad1730e830efaf350fd60aea2b2c56292d992579f4ea9ef889c7eaac5\": container with ID starting with e61330fad1730e830efaf350fd60aea2b2c56292d992579f4ea9ef889c7eaac5 not found: ID does not exist" containerID="e61330fad1730e830efaf350fd60aea2b2c56292d992579f4ea9ef889c7eaac5"
Sep 18 14:22:57.303203 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:22:57.303146    2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e61330fad1730e830efaf350fd60aea2b2c56292d992579f4ea9ef889c7eaac5"} err="failed to get container status \"e61330fad1730e830efaf350fd60aea2b2c56292d992579f4ea9ef889c7eaac5\": rpc error: code = NotFound desc = could not find container \"e61330fad1730e830efaf350fd60aea2b2c56292d992579f4ea9ef889c7eaac5\": container with ID starting with e61330fad1730e830efaf350fd60aea2b2c56292d992579f4ea9ef889c7eaac5 not found: ID does not exist"
Sep 18 14:22:57.369930 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:22:57.369899    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-m6znr\" (UniqueName: \"kubernetes.io/projected/609ab3f8-150f-40fe-956c-139806aee3ce-kube-api-access-m6znr\") pod \"609ab3f8-150f-40fe-956c-139806aee3ce\" (UID: \"609ab3f8-150f-40fe-956c-139806aee3ce\") "
Sep 18 14:22:57.370156 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:22:57.369953    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/609ab3f8-150f-40fe-956c-139806aee3ce-plugins-config-volume\") pod \"609ab3f8-150f-40fe-956c-139806aee3ce\" (UID: \"609ab3f8-150f-40fe-956c-139806aee3ce\") "
Sep 18 14:22:57.370388 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:22:57.370356    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/609ab3f8-150f-40fe-956c-139806aee3ce-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "609ab3f8-150f-40fe-956c-139806aee3ce" (UID: "609ab3f8-150f-40fe-956c-139806aee3ce"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 18 14:22:57.372083 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:22:57.372059    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/609ab3f8-150f-40fe-956c-139806aee3ce-kube-api-access-m6znr" (OuterVolumeSpecName: "kube-api-access-m6znr") pod "609ab3f8-150f-40fe-956c-139806aee3ce" (UID: "609ab3f8-150f-40fe-956c-139806aee3ce"). InnerVolumeSpecName "kube-api-access-m6znr". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 18 14:22:57.471706 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:22:57.471670    2565 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-m6znr\" (UniqueName: \"kubernetes.io/projected/609ab3f8-150f-40fe-956c-139806aee3ce-kube-api-access-m6znr\") on node \"ip-10-0-129-16.ec2.internal\" DevicePath \"\""
Sep 18 14:22:57.471706 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:22:57.471712    2565 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/609ab3f8-150f-40fe-956c-139806aee3ce-plugins-config-volume\") on node \"ip-10-0-129-16.ec2.internal\" DevicePath \"\""
Sep 18 14:22:57.616405 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:22:57.616357    2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w0-4d9222-67d45969f8-gwf8b"]
Sep 18 14:22:57.620459 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:22:57.620432    2565 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w0-4d9222-67d45969f8-gwf8b"]
Sep 18 14:22:57.914104 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:22:57.914002    2565 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="609ab3f8-150f-40fe-956c-139806aee3ce" path="/var/lib/kubelet/pods/609ab3f8-150f-40fe-956c-139806aee3ce/volumes"
Sep 18 14:22:57.914392 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:22:57.914377    2565 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="bc133952-f64e-40bb-a76b-0b30fc979393" path="/var/lib/kubelet/pods/bc133952-f64e-40bb-a76b-0b30fc979393/volumes"
Sep 18 14:23:00.894486 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:23:00.894426    2565 kuberuntime_container.go:873] "Killing container with a grace period" pod="e2e-models-e2e-worker-w0-4d9222/embedding-w0-4d9222-kserve-bb67c65-gnwmq" podUID="67d8cf34-7edd-4029-8038-4b4255e56c62" containerName="main" containerID="cri-o://eac611d591e28a6b37125512508515a0c4c29775e13735e7f0e8b65258d2f5ac" gracePeriod=45
Sep 18 14:23:01.173345 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:23:01.173319    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w0-4d9222/embedding-w0-4d9222-kserve-bb67c65-gnwmq"
Sep 18 14:23:01.314282 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:23:01.314242    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-d8fqv\" (UniqueName: \"kubernetes.io/projected/67d8cf34-7edd-4029-8038-4b4255e56c62-kube-api-access-d8fqv\") pod \"67d8cf34-7edd-4029-8038-4b4255e56c62\" (UID: \"67d8cf34-7edd-4029-8038-4b4255e56c62\") "
Sep 18 14:23:01.314484 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:23:01.314304    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/67d8cf34-7edd-4029-8038-4b4255e56c62-dshm\") pod \"67d8cf34-7edd-4029-8038-4b4255e56c62\" (UID: \"67d8cf34-7edd-4029-8038-4b4255e56c62\") "
Sep 18 14:23:01.314484 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:23:01.314359    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/67d8cf34-7edd-4029-8038-4b4255e56c62-tmp-dir\") pod \"67d8cf34-7edd-4029-8038-4b4255e56c62\" (UID: \"67d8cf34-7edd-4029-8038-4b4255e56c62\") "
Sep 18 14:23:01.314484 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:23:01.314452    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/67d8cf34-7edd-4029-8038-4b4255e56c62-tls-certs\") pod \"67d8cf34-7edd-4029-8038-4b4255e56c62\" (UID: \"67d8cf34-7edd-4029-8038-4b4255e56c62\") "
Sep 18 14:23:01.314699 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:23:01.314487    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/67d8cf34-7edd-4029-8038-4b4255e56c62-home\") pod \"67d8cf34-7edd-4029-8038-4b4255e56c62\" (UID: \"67d8cf34-7edd-4029-8038-4b4255e56c62\") "
Sep 18 14:23:01.314699 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:23:01.314516    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/67d8cf34-7edd-4029-8038-4b4255e56c62-model-cache\") pod \"67d8cf34-7edd-4029-8038-4b4255e56c62\" (UID: \"67d8cf34-7edd-4029-8038-4b4255e56c62\") "
Sep 18 14:23:01.314941 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:23:01.314899    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/67d8cf34-7edd-4029-8038-4b4255e56c62-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "67d8cf34-7edd-4029-8038-4b4255e56c62" (UID: "67d8cf34-7edd-4029-8038-4b4255e56c62"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 18 14:23:01.315188 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:23:01.315154    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/67d8cf34-7edd-4029-8038-4b4255e56c62-model-cache" (OuterVolumeSpecName: "model-cache") pod "67d8cf34-7edd-4029-8038-4b4255e56c62" (UID: "67d8cf34-7edd-4029-8038-4b4255e56c62"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 18 14:23:01.315329 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:23:01.315221    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/67d8cf34-7edd-4029-8038-4b4255e56c62-home" (OuterVolumeSpecName: "home") pod "67d8cf34-7edd-4029-8038-4b4255e56c62" (UID: "67d8cf34-7edd-4029-8038-4b4255e56c62"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 18 14:23:01.316820 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:23:01.316789    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/67d8cf34-7edd-4029-8038-4b4255e56c62-kube-api-access-d8fqv" (OuterVolumeSpecName: "kube-api-access-d8fqv") pod "67d8cf34-7edd-4029-8038-4b4255e56c62" (UID: "67d8cf34-7edd-4029-8038-4b4255e56c62"). InnerVolumeSpecName "kube-api-access-d8fqv". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 18 14:23:01.316918 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:23:01.316865    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/67d8cf34-7edd-4029-8038-4b4255e56c62-dshm" (OuterVolumeSpecName: "dshm") pod "67d8cf34-7edd-4029-8038-4b4255e56c62" (UID: "67d8cf34-7edd-4029-8038-4b4255e56c62"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 18 14:23:01.317138 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:23:01.317119    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/67d8cf34-7edd-4029-8038-4b4255e56c62-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "67d8cf34-7edd-4029-8038-4b4255e56c62" (UID: "67d8cf34-7edd-4029-8038-4b4255e56c62"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 18 14:23:01.322056 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:23:01.322018    2565 generic.go:356] "Generic (PLEG): container finished" podID="67d8cf34-7edd-4029-8038-4b4255e56c62" containerID="eac611d591e28a6b37125512508515a0c4c29775e13735e7f0e8b65258d2f5ac" exitCode=0
Sep 18 14:23:01.322164 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:23:01.322098    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w0-4d9222/embedding-w0-4d9222-kserve-bb67c65-gnwmq"
Sep 18 14:23:01.322164 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:23:01.322130    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w0-4d9222/embedding-w0-4d9222-kserve-bb67c65-gnwmq" event={"ID":"67d8cf34-7edd-4029-8038-4b4255e56c62","Type":"ContainerDied","Data":"eac611d591e28a6b37125512508515a0c4c29775e13735e7f0e8b65258d2f5ac"}
Sep 18 14:23:01.322290 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:23:01.322162    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w0-4d9222/embedding-w0-4d9222-kserve-bb67c65-gnwmq" event={"ID":"67d8cf34-7edd-4029-8038-4b4255e56c62","Type":"ContainerDied","Data":"b53e829e9aabf66fe7d91c97573ec1e429a08de2f41e78b29621ba62bd53e985"}
Sep 18 14:23:01.322290 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:23:01.322182    2565 scope.go:117] "RemoveContainer" containerID="eac611d591e28a6b37125512508515a0c4c29775e13735e7f0e8b65258d2f5ac"
Sep 18 14:23:01.341459 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:23:01.341439    2565 scope.go:117] "RemoveContainer" containerID="eac611d591e28a6b37125512508515a0c4c29775e13735e7f0e8b65258d2f5ac"
Sep 18 14:23:01.341814 ip-10-0-129-16 kubenswrapper[2565]: E0918 14:23:01.341795    2565 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"eac611d591e28a6b37125512508515a0c4c29775e13735e7f0e8b65258d2f5ac\": container with ID starting with eac611d591e28a6b37125512508515a0c4c29775e13735e7f0e8b65258d2f5ac not found: ID does not exist" containerID="eac611d591e28a6b37125512508515a0c4c29775e13735e7f0e8b65258d2f5ac"
Sep 18 14:23:01.341875 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:23:01.341826    2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"eac611d591e28a6b37125512508515a0c4c29775e13735e7f0e8b65258d2f5ac"} err="failed to get container status \"eac611d591e28a6b37125512508515a0c4c29775e13735e7f0e8b65258d2f5ac\": rpc error: code = NotFound desc = could not find container \"eac611d591e28a6b37125512508515a0c4c29775e13735e7f0e8b65258d2f5ac\": container with ID starting with eac611d591e28a6b37125512508515a0c4c29775e13735e7f0e8b65258d2f5ac not found: ID does not exist"
Sep 18 14:23:01.351441 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:23:01.351411    2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w0-4d9222/embedding-w0-4d9222-kserve-bb67c65-gnwmq"]
Sep 18 14:23:01.357006 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:23:01.356980    2565 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["e2e-models-e2e-worker-w0-4d9222/embedding-w0-4d9222-kserve-bb67c65-gnwmq"]
Sep 18 14:23:01.416587 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:23:01.416487    2565 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/67d8cf34-7edd-4029-8038-4b4255e56c62-tmp-dir\") on node \"ip-10-0-129-16.ec2.internal\" DevicePath \"\""
Sep 18 14:23:01.416587 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:23:01.416517    2565 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/67d8cf34-7edd-4029-8038-4b4255e56c62-tls-certs\") on node \"ip-10-0-129-16.ec2.internal\" DevicePath \"\""
Sep 18 14:23:01.416587 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:23:01.416528    2565 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/67d8cf34-7edd-4029-8038-4b4255e56c62-home\") on node \"ip-10-0-129-16.ec2.internal\" DevicePath \"\""
Sep 18 14:23:01.416587 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:23:01.416538    2565 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/67d8cf34-7edd-4029-8038-4b4255e56c62-model-cache\") on node \"ip-10-0-129-16.ec2.internal\" DevicePath \"\""
Sep 18 14:23:01.416587 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:23:01.416550    2565 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-d8fqv\" (UniqueName: \"kubernetes.io/projected/67d8cf34-7edd-4029-8038-4b4255e56c62-kube-api-access-d8fqv\") on node \"ip-10-0-129-16.ec2.internal\" DevicePath \"\""
Sep 18 14:23:01.416587 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:23:01.416561    2565 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/67d8cf34-7edd-4029-8038-4b4255e56c62-dshm\") on node \"ip-10-0-129-16.ec2.internal\" DevicePath \"\""
Sep 18 14:23:01.917247 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:23:01.915097    2565 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="67d8cf34-7edd-4029-8038-4b4255e56c62" path="/var/lib/kubelet/pods/67d8cf34-7edd-4029-8038-4b4255e56c62/volumes"
Sep 18 14:23:04.660951 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:23:04.660915    2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w0-4d9222-openshift-default-849bc99f86-v5mg4"]
Sep 18 14:23:04.661445 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:23:04.661160    2565 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w0-4d9222-openshift-default-849bc99f86-v5mg4" podUID="f8ff7559-3e54-44b4-a9ce-50f69e2b1908" containerName="istio-proxy" containerID="cri-o://85338d552b54e5d29fdab34775193237cbfae43d10bfd7511cbab811953a8a7f" gracePeriod=30
Sep 18 14:23:08.754526 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:23:08.754494    2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-e01050-6896bdcc57-5fd9d"]
Sep 18 14:23:08.755061 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:23:08.755045    2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="609ab3f8-150f-40fe-956c-139806aee3ce" containerName="payload-pre-processing"
Sep 18 14:23:08.755061 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:23:08.755062    2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="609ab3f8-150f-40fe-956c-139806aee3ce" containerName="payload-pre-processing"
Sep 18 14:23:08.755150 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:23:08.755087    2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="bc133952-f64e-40bb-a76b-0b30fc979393" containerName="revoke-keys"
Sep 18 14:23:08.755150 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:23:08.755092    2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="bc133952-f64e-40bb-a76b-0b30fc979393" containerName="revoke-keys"
Sep 18 14:23:08.755150 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:23:08.755112    2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="67d8cf34-7edd-4029-8038-4b4255e56c62" containerName="main"
Sep 18 14:23:08.755150 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:23:08.755118    2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="67d8cf34-7edd-4029-8038-4b4255e56c62" containerName="main"
Sep 18 14:23:08.755275 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:23:08.755203    2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="67d8cf34-7edd-4029-8038-4b4255e56c62" containerName="main"
Sep 18 14:23:08.755275 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:23:08.755213    2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="609ab3f8-150f-40fe-956c-139806aee3ce" containerName="payload-pre-processing"
Sep 18 14:23:08.755275 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:23:08.755220    2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="bc133952-f64e-40bb-a76b-0b30fc979393" containerName="revoke-keys"
Sep 18 14:23:08.765259 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:23:08.763387    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-a-e01050-6896bdcc57-5fd9d"
Sep 18 14:23:08.770076 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:23:08.770051    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-api-a-e01050-dockercfg-dwgkc\""
Sep 18 14:23:08.770336 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:23:08.770117    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-api-a-e01050\""
Sep 18 14:23:08.772979 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:23:08.772951    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-e01050-6896bdcc57-5fd9d"]
Sep 18 14:23:08.800518 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:23:08.800484    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/943c8a84-652b-4baa-aed8-d6faa3ca8dbc-plugins-config-volume\") pod \"payload-processing-e2e-api-a-e01050-6896bdcc57-5fd9d\" (UID: \"943c8a84-652b-4baa-aed8-d6faa3ca8dbc\") " pod="openshift-ingress/payload-processing-e2e-api-a-e01050-6896bdcc57-5fd9d"
Sep 18 14:23:08.800664 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:23:08.800603    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wcfx8\" (UniqueName: \"kubernetes.io/projected/943c8a84-652b-4baa-aed8-d6faa3ca8dbc-kube-api-access-wcfx8\") pod \"payload-processing-e2e-api-a-e01050-6896bdcc57-5fd9d\" (UID: \"943c8a84-652b-4baa-aed8-d6faa3ca8dbc\") " pod="openshift-ingress/payload-processing-e2e-api-a-e01050-6896bdcc57-5fd9d"
Sep 18 14:23:08.901717 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:23:08.901682    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/943c8a84-652b-4baa-aed8-d6faa3ca8dbc-plugins-config-volume\") pod \"payload-processing-e2e-api-a-e01050-6896bdcc57-5fd9d\" (UID: \"943c8a84-652b-4baa-aed8-d6faa3ca8dbc\") " pod="openshift-ingress/payload-processing-e2e-api-a-e01050-6896bdcc57-5fd9d"
Sep 18 14:23:08.901890 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:23:08.901783    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wcfx8\" (UniqueName: \"kubernetes.io/projected/943c8a84-652b-4baa-aed8-d6faa3ca8dbc-kube-api-access-wcfx8\") pod \"payload-processing-e2e-api-a-e01050-6896bdcc57-5fd9d\" (UID: \"943c8a84-652b-4baa-aed8-d6faa3ca8dbc\") " pod="openshift-ingress/payload-processing-e2e-api-a-e01050-6896bdcc57-5fd9d"
Sep 18 14:23:08.902491 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:23:08.902469    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/943c8a84-652b-4baa-aed8-d6faa3ca8dbc-plugins-config-volume\") pod \"payload-processing-e2e-api-a-e01050-6896bdcc57-5fd9d\" (UID: \"943c8a84-652b-4baa-aed8-d6faa3ca8dbc\") " pod="openshift-ingress/payload-processing-e2e-api-a-e01050-6896bdcc57-5fd9d"
Sep 18 14:23:08.910865 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:23:08.910831    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wcfx8\" (UniqueName: \"kubernetes.io/projected/943c8a84-652b-4baa-aed8-d6faa3ca8dbc-kube-api-access-wcfx8\") pod \"payload-processing-e2e-api-a-e01050-6896bdcc57-5fd9d\" (UID: \"943c8a84-652b-4baa-aed8-d6faa3ca8dbc\") " pod="openshift-ingress/payload-processing-e2e-api-a-e01050-6896bdcc57-5fd9d"
Sep 18 14:23:09.078389 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:23:09.078348    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-a-e01050-6896bdcc57-5fd9d"
Sep 18 14:23:09.357511 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:23:09.357482    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-e01050-6896bdcc57-5fd9d"]
Sep 18 14:23:09.358742 ip-10-0-129-16 kubenswrapper[2565]: W0918 14:23:09.358719    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod943c8a84_652b_4baa_aed8_d6faa3ca8dbc.slice/crio-612da483cf354a00fe979673cb97c7a68a87735f3def798dc16244da5512d39d WatchSource:0}: Error finding container 612da483cf354a00fe979673cb97c7a68a87735f3def798dc16244da5512d39d: Status 404 returned error can't find the container with id 612da483cf354a00fe979673cb97c7a68a87735f3def798dc16244da5512d39d
Sep 18 14:23:09.371250 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:23:09.371218    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-a-e01050-6896bdcc57-5fd9d" event={"ID":"943c8a84-652b-4baa-aed8-d6faa3ca8dbc","Type":"ContainerStarted","Data":"612da483cf354a00fe979673cb97c7a68a87735f3def798dc16244da5512d39d"}
Sep 18 14:23:09.912020 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:23:09.911992    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-4d9222-openshift-default-849bc99f86-v5mg4"
Sep 18 14:23:10.012638 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:23:10.012540    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f8ff7559-3e54-44b4-a9ce-50f69e2b1908-workload-certs\") pod \"f8ff7559-3e54-44b4-a9ce-50f69e2b1908\" (UID: \"f8ff7559-3e54-44b4-a9ce-50f69e2b1908\") "
Sep 18 14:23:10.012638 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:23:10.012603    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f8ff7559-3e54-44b4-a9ce-50f69e2b1908-workload-socket\") pod \"f8ff7559-3e54-44b4-a9ce-50f69e2b1908\" (UID: \"f8ff7559-3e54-44b4-a9ce-50f69e2b1908\") "
Sep 18 14:23:10.012638 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:23:10.012634    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-d6h64\" (UniqueName: \"kubernetes.io/projected/f8ff7559-3e54-44b4-a9ce-50f69e2b1908-kube-api-access-d6h64\") pod \"f8ff7559-3e54-44b4-a9ce-50f69e2b1908\" (UID: \"f8ff7559-3e54-44b4-a9ce-50f69e2b1908\") "
Sep 18 14:23:10.012922 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:23:10.012662    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f8ff7559-3e54-44b4-a9ce-50f69e2b1908-istio-envoy\") pod \"f8ff7559-3e54-44b4-a9ce-50f69e2b1908\" (UID: \"f8ff7559-3e54-44b4-a9ce-50f69e2b1908\") "
Sep 18 14:23:10.012922 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:23:10.012703    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f8ff7559-3e54-44b4-a9ce-50f69e2b1908-istio-podinfo\") pod \"f8ff7559-3e54-44b4-a9ce-50f69e2b1908\" (UID: \"f8ff7559-3e54-44b4-a9ce-50f69e2b1908\") "
Sep 18 14:23:10.012922 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:23:10.012740    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f8ff7559-3e54-44b4-a9ce-50f69e2b1908-istio-token\") pod \"f8ff7559-3e54-44b4-a9ce-50f69e2b1908\" (UID: \"f8ff7559-3e54-44b4-a9ce-50f69e2b1908\") "
Sep 18 14:23:10.012922 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:23:10.012791    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f8ff7559-3e54-44b4-a9ce-50f69e2b1908-credential-socket\") pod \"f8ff7559-3e54-44b4-a9ce-50f69e2b1908\" (UID: \"f8ff7559-3e54-44b4-a9ce-50f69e2b1908\") "
Sep 18 14:23:10.012922 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:23:10.012873    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f8ff7559-3e54-44b4-a9ce-50f69e2b1908-istiod-ca-cert\") pod \"f8ff7559-3e54-44b4-a9ce-50f69e2b1908\" (UID: \"f8ff7559-3e54-44b4-a9ce-50f69e2b1908\") "
Sep 18 14:23:10.012922 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:23:10.012876    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f8ff7559-3e54-44b4-a9ce-50f69e2b1908-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "f8ff7559-3e54-44b4-a9ce-50f69e2b1908" (UID: "f8ff7559-3e54-44b4-a9ce-50f69e2b1908"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 18 14:23:10.013250 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:23:10.012924    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f8ff7559-3e54-44b4-a9ce-50f69e2b1908-istio-data\") pod \"f8ff7559-3e54-44b4-a9ce-50f69e2b1908\" (UID: \"f8ff7559-3e54-44b4-a9ce-50f69e2b1908\") "
Sep 18 14:23:10.013250 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:23:10.012977    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f8ff7559-3e54-44b4-a9ce-50f69e2b1908-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "f8ff7559-3e54-44b4-a9ce-50f69e2b1908" (UID: "f8ff7559-3e54-44b4-a9ce-50f69e2b1908"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 18 14:23:10.013250 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:23:10.013125    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f8ff7559-3e54-44b4-a9ce-50f69e2b1908-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "f8ff7559-3e54-44b4-a9ce-50f69e2b1908" (UID: "f8ff7559-3e54-44b4-a9ce-50f69e2b1908"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 18 14:23:10.013388 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:23:10.013314    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f8ff7559-3e54-44b4-a9ce-50f69e2b1908-istio-data" (OuterVolumeSpecName: "istio-data") pod "f8ff7559-3e54-44b4-a9ce-50f69e2b1908" (UID: "f8ff7559-3e54-44b4-a9ce-50f69e2b1908"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 18 14:23:10.013451 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:23:10.013429    2565 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f8ff7559-3e54-44b4-a9ce-50f69e2b1908-workload-socket\") on node \"ip-10-0-129-16.ec2.internal\" DevicePath \"\""
Sep 18 14:23:10.013518 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:23:10.013449    2565 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f8ff7559-3e54-44b4-a9ce-50f69e2b1908-credential-socket\") on node \"ip-10-0-129-16.ec2.internal\" DevicePath \"\""
Sep 18 14:23:10.013518 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:23:10.013463    2565 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f8ff7559-3e54-44b4-a9ce-50f69e2b1908-istio-data\") on node \"ip-10-0-129-16.ec2.internal\" DevicePath \"\""
Sep 18 14:23:10.013518 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:23:10.013469    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f8ff7559-3e54-44b4-a9ce-50f69e2b1908-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "f8ff7559-3e54-44b4-a9ce-50f69e2b1908" (UID: "f8ff7559-3e54-44b4-a9ce-50f69e2b1908"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 18 14:23:10.013518 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:23:10.013476    2565 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f8ff7559-3e54-44b4-a9ce-50f69e2b1908-workload-certs\") on node \"ip-10-0-129-16.ec2.internal\" DevicePath \"\""
Sep 18 14:23:10.015134 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:23:10.015103    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/f8ff7559-3e54-44b4-a9ce-50f69e2b1908-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "f8ff7559-3e54-44b4-a9ce-50f69e2b1908" (UID: "f8ff7559-3e54-44b4-a9ce-50f69e2b1908"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 18 14:23:10.015340 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:23:10.015320    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f8ff7559-3e54-44b4-a9ce-50f69e2b1908-istio-token" (OuterVolumeSpecName: "istio-token") pod "f8ff7559-3e54-44b4-a9ce-50f69e2b1908" (UID: "f8ff7559-3e54-44b4-a9ce-50f69e2b1908"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 18 14:23:10.015457 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:23:10.015430    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f8ff7559-3e54-44b4-a9ce-50f69e2b1908-kube-api-access-d6h64" (OuterVolumeSpecName: "kube-api-access-d6h64") pod "f8ff7559-3e54-44b4-a9ce-50f69e2b1908" (UID: "f8ff7559-3e54-44b4-a9ce-50f69e2b1908"). InnerVolumeSpecName "kube-api-access-d6h64". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 18 14:23:10.015532 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:23:10.015512    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f8ff7559-3e54-44b4-a9ce-50f69e2b1908-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "f8ff7559-3e54-44b4-a9ce-50f69e2b1908" (UID: "f8ff7559-3e54-44b4-a9ce-50f69e2b1908"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 18 14:23:10.114688 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:23:10.114644    2565 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f8ff7559-3e54-44b4-a9ce-50f69e2b1908-istio-podinfo\") on node \"ip-10-0-129-16.ec2.internal\" DevicePath \"\""
Sep 18 14:23:10.114688 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:23:10.114688    2565 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f8ff7559-3e54-44b4-a9ce-50f69e2b1908-istio-token\") on node \"ip-10-0-129-16.ec2.internal\" DevicePath \"\""
Sep 18 14:23:10.114688 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:23:10.114700    2565 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f8ff7559-3e54-44b4-a9ce-50f69e2b1908-istiod-ca-cert\") on node \"ip-10-0-129-16.ec2.internal\" DevicePath \"\""
Sep 18 14:23:10.114931 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:23:10.114709    2565 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-d6h64\" (UniqueName: \"kubernetes.io/projected/f8ff7559-3e54-44b4-a9ce-50f69e2b1908-kube-api-access-d6h64\") on node \"ip-10-0-129-16.ec2.internal\" DevicePath \"\""
Sep 18 14:23:10.114931 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:23:10.114719    2565 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f8ff7559-3e54-44b4-a9ce-50f69e2b1908-istio-envoy\") on node \"ip-10-0-129-16.ec2.internal\" DevicePath \"\""
Sep 18 14:23:10.379562 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:23:10.379522    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-a-e01050-6896bdcc57-5fd9d" event={"ID":"943c8a84-652b-4baa-aed8-d6faa3ca8dbc","Type":"ContainerStarted","Data":"225dcbd6ff1fa786b59fcdf2a1a6fb0b58bc28a70b176537f71be18b8c7f43d4"}
Sep 18 14:23:10.379777 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:23:10.379593    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-api-a-e01050-6896bdcc57-5fd9d"
Sep 18 14:23:10.381304 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:23:10.381275    2565 generic.go:356] "Generic (PLEG): container finished" podID="f8ff7559-3e54-44b4-a9ce-50f69e2b1908" containerID="85338d552b54e5d29fdab34775193237cbfae43d10bfd7511cbab811953a8a7f" exitCode=0
Sep 18 14:23:10.381447 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:23:10.381321    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-4d9222-openshift-default-849bc99f86-v5mg4"
Sep 18 14:23:10.381447 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:23:10.381356    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-4d9222-openshift-default-849bc99f86-v5mg4" event={"ID":"f8ff7559-3e54-44b4-a9ce-50f69e2b1908","Type":"ContainerDied","Data":"85338d552b54e5d29fdab34775193237cbfae43d10bfd7511cbab811953a8a7f"}
Sep 18 14:23:10.381447 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:23:10.381408    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-4d9222-openshift-default-849bc99f86-v5mg4" event={"ID":"f8ff7559-3e54-44b4-a9ce-50f69e2b1908","Type":"ContainerDied","Data":"c6fb05d56dc5f22a17092f6bd8caf731e2f9d66755402e415b5dad3af8551927"}
Sep 18 14:23:10.381447 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:23:10.381431    2565 scope.go:117] "RemoveContainer" containerID="85338d552b54e5d29fdab34775193237cbfae43d10bfd7511cbab811953a8a7f"
Sep 18 14:23:10.392292 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:23:10.392272    2565 scope.go:117] "RemoveContainer" containerID="3470f50403473d7e198a40e9941623dc0840c6d203553afd39d44680ac8ed2cd"
Sep 18 14:23:10.398516 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:23:10.398456    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-api-a-e01050-6896bdcc57-5fd9d" podStartSLOduration=2.39844051 podStartE2EDuration="2.39844051s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 14:23:08 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 14:23:10.397372478 +0000 UTC m=+1458.571282138" watchObservedRunningTime="2026-09-18 14:23:10.39844051 +0000 UTC m=+1458.572350208"
Sep 18 14:23:10.401807 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:23:10.401787    2565 scope.go:117] "RemoveContainer" containerID="85338d552b54e5d29fdab34775193237cbfae43d10bfd7511cbab811953a8a7f"
Sep 18 14:23:10.402251 ip-10-0-129-16 kubenswrapper[2565]: E0918 14:23:10.402205    2565 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"85338d552b54e5d29fdab34775193237cbfae43d10bfd7511cbab811953a8a7f\": container with ID starting with 85338d552b54e5d29fdab34775193237cbfae43d10bfd7511cbab811953a8a7f not found: ID does not exist" containerID="85338d552b54e5d29fdab34775193237cbfae43d10bfd7511cbab811953a8a7f"
Sep 18 14:23:10.402251 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:23:10.402244    2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"85338d552b54e5d29fdab34775193237cbfae43d10bfd7511cbab811953a8a7f"} err="failed to get container status \"85338d552b54e5d29fdab34775193237cbfae43d10bfd7511cbab811953a8a7f\": rpc error: code = NotFound desc = could not find container \"85338d552b54e5d29fdab34775193237cbfae43d10bfd7511cbab811953a8a7f\": container with ID starting with 85338d552b54e5d29fdab34775193237cbfae43d10bfd7511cbab811953a8a7f not found: ID does not exist"
Sep 18 14:23:10.402468 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:23:10.402263    2565 scope.go:117] "RemoveContainer" containerID="3470f50403473d7e198a40e9941623dc0840c6d203553afd39d44680ac8ed2cd"
Sep 18 14:23:10.402628 ip-10-0-129-16 kubenswrapper[2565]: E0918 14:23:10.402607    2565 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3470f50403473d7e198a40e9941623dc0840c6d203553afd39d44680ac8ed2cd\": container with ID starting with 3470f50403473d7e198a40e9941623dc0840c6d203553afd39d44680ac8ed2cd not found: ID does not exist" containerID="3470f50403473d7e198a40e9941623dc0840c6d203553afd39d44680ac8ed2cd"
Sep 18 14:23:10.402705 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:23:10.402634    2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3470f50403473d7e198a40e9941623dc0840c6d203553afd39d44680ac8ed2cd"} err="failed to get container status \"3470f50403473d7e198a40e9941623dc0840c6d203553afd39d44680ac8ed2cd\": rpc error: code = NotFound desc = could not find container \"3470f50403473d7e198a40e9941623dc0840c6d203553afd39d44680ac8ed2cd\": container with ID starting with 3470f50403473d7e198a40e9941623dc0840c6d203553afd39d44680ac8ed2cd not found: ID does not exist"
Sep 18 14:23:10.414245 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:23:10.414212    2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w0-4d9222-openshift-default-849bc99f86-v5mg4"]
Sep 18 14:23:10.420713 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:23:10.420686    2565 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w0-4d9222-openshift-default-849bc99f86-v5mg4"]
Sep 18 14:23:11.913875 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:23:11.913843    2565 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f8ff7559-3e54-44b4-a9ce-50f69e2b1908" path="/var/lib/kubelet/pods/f8ff7559-3e54-44b4-a9ce-50f69e2b1908/volumes"
Sep 18 14:23:20.525783 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:23:20.525751    2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w4-93827d-4zxxk"]
Sep 18 14:23:20.526286 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:23:20.526272    2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f8ff7559-3e54-44b4-a9ce-50f69e2b1908" containerName="istio-proxy"
Sep 18 14:23:20.526286 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:23:20.526287    2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="f8ff7559-3e54-44b4-a9ce-50f69e2b1908" containerName="istio-proxy"
Sep 18 14:23:20.526379 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:23:20.526370    2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="f8ff7559-3e54-44b4-a9ce-50f69e2b1908" containerName="istio-proxy"
Sep 18 14:23:20.526420 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:23:20.526379    2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="f8ff7559-3e54-44b4-a9ce-50f69e2b1908" containerName="istio-proxy"
Sep 18 14:23:20.529536 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:23:20.529520    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w4-93827d-4zxxk"
Sep 18 14:23:20.532102 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:23:20.532084    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-l4fn9\""
Sep 18 14:23:20.536690 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:23:20.536667    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w4-93827d-4zxxk"]
Sep 18 14:23:20.624163 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:23:20.624126    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/457bbf5d-74af-4d4a-a2ca-9e7e3bd9d61a-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-worker-w4-93827d-4zxxk\" (UID: \"457bbf5d-74af-4d4a-a2ca-9e7e3bd9d61a\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w4-93827d-4zxxk"
Sep 18 14:23:20.724920 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:23:20.724884    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/457bbf5d-74af-4d4a-a2ca-9e7e3bd9d61a-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-worker-w4-93827d-4zxxk\" (UID: \"457bbf5d-74af-4d4a-a2ca-9e7e3bd9d61a\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w4-93827d-4zxxk"
Sep 18 14:23:20.725572 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:23:20.725551    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/457bbf5d-74af-4d4a-a2ca-9e7e3bd9d61a-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-worker-w4-93827d-4zxxk\" (UID: \"457bbf5d-74af-4d4a-a2ca-9e7e3bd9d61a\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w4-93827d-4zxxk"
Sep 18 14:23:20.843282 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:23:20.843244    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w4-93827d-4zxxk"
Sep 18 14:23:21.121458 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:23:21.121428    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w4-93827d-4zxxk"]
Sep 18 14:23:21.123016 ip-10-0-129-16 kubenswrapper[2565]: W0918 14:23:21.122987    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod457bbf5d_74af_4d4a_a2ca_9e7e3bd9d61a.slice/crio-5221e40a3cd39034627da1c2873dd22d7a63715344883e613b4e4c7ca9cb48d5 WatchSource:0}: Error finding container 5221e40a3cd39034627da1c2873dd22d7a63715344883e613b4e4c7ca9cb48d5: Status 404 returned error can't find the container with id 5221e40a3cd39034627da1c2873dd22d7a63715344883e613b4e4c7ca9cb48d5
Sep 18 14:23:21.392318 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:23:21.392235    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-api-a-e01050-6896bdcc57-5fd9d"
Sep 18 14:23:21.455633 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:23:21.455600    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w4-93827d-4zxxk" event={"ID":"457bbf5d-74af-4d4a-a2ca-9e7e3bd9d61a","Type":"ContainerStarted","Data":"5221e40a3cd39034627da1c2873dd22d7a63715344883e613b4e4c7ca9cb48d5"}
Sep 18 14:23:22.463852 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:23:22.463804    2565 generic.go:356] "Generic (PLEG): container finished" podID="457bbf5d-74af-4d4a-a2ca-9e7e3bd9d61a" containerID="88da6b1c63cf2af78861550f31569ed4f1bbb669e339f4c4041c36d21b4dde6e" exitCode=0
Sep 18 14:23:22.464393 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:23:22.463864    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w4-93827d-4zxxk" event={"ID":"457bbf5d-74af-4d4a-a2ca-9e7e3bd9d61a","Type":"ContainerDied","Data":"88da6b1c63cf2af78861550f31569ed4f1bbb669e339f4c4041c36d21b4dde6e"}
Sep 18 14:23:23.645507 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:23:23.645482    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w4-93827d-4zxxk"
Sep 18 14:23:23.768131 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:23:23.768020    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/457bbf5d-74af-4d4a-a2ca-9e7e3bd9d61a-maas-api-service-ca\") pod \"457bbf5d-74af-4d4a-a2ca-9e7e3bd9d61a\" (UID: \"457bbf5d-74af-4d4a-a2ca-9e7e3bd9d61a\") "
Sep 18 14:23:23.768394 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:23:23.768370    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/457bbf5d-74af-4d4a-a2ca-9e7e3bd9d61a-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "457bbf5d-74af-4d4a-a2ca-9e7e3bd9d61a" (UID: "457bbf5d-74af-4d4a-a2ca-9e7e3bd9d61a"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 18 14:23:23.768515 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:23:23.768501    2565 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/457bbf5d-74af-4d4a-a2ca-9e7e3bd9d61a-maas-api-service-ca\") on node \"ip-10-0-129-16.ec2.internal\" DevicePath \"\""
Sep 18 14:23:24.481419 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:23:24.481389    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w4-93827d-4zxxk"
Sep 18 14:23:24.481587 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:23:24.481437    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w4-93827d-4zxxk" event={"ID":"457bbf5d-74af-4d4a-a2ca-9e7e3bd9d61a","Type":"ContainerDied","Data":"5221e40a3cd39034627da1c2873dd22d7a63715344883e613b4e4c7ca9cb48d5"}
Sep 18 14:23:24.481587 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:23:24.481495    2565 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="5221e40a3cd39034627da1c2873dd22d7a63715344883e613b4e4c7ca9cb48d5"
Sep 18 14:23:25.400336 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:23:25.400305    2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w4-93827d/premium-w4-93827d-kserve-78c986f74c-hjr2v"]
Sep 18 14:23:34.864838 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:23:34.864802    2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-b-e9388b-7c99d75d46-jtfhl"]
Sep 18 14:23:34.865334 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:23:34.865318    2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="457bbf5d-74af-4d4a-a2ca-9e7e3bd9d61a" containerName="revoke-keys"
Sep 18 14:23:34.865392 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:23:34.865336    2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="457bbf5d-74af-4d4a-a2ca-9e7e3bd9d61a" containerName="revoke-keys"
Sep 18 14:23:34.865392 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:23:34.865352    2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f8ff7559-3e54-44b4-a9ce-50f69e2b1908" containerName="istio-proxy"
Sep 18 14:23:34.865392 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:23:34.865357    2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="f8ff7559-3e54-44b4-a9ce-50f69e2b1908" containerName="istio-proxy"
Sep 18 14:23:34.865490 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:23:34.865440    2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="457bbf5d-74af-4d4a-a2ca-9e7e3bd9d61a" containerName="revoke-keys"
Sep 18 14:23:34.868883 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:23:34.868866    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-b-e9388b-7c99d75d46-jtfhl"
Sep 18 14:23:34.872817 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:23:34.872793    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-api-b-e9388b\""
Sep 18 14:23:34.873187 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:23:34.873167    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-api-b-e9388b-dockercfg-glhnc\""
Sep 18 14:23:34.885785 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:23:34.885752    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pr4qg\" (UniqueName: \"kubernetes.io/projected/7c6320a2-acdb-41eb-a068-b879b515941e-kube-api-access-pr4qg\") pod \"payload-pre-processing-e2e-api-b-e9388b-7c99d75d46-jtfhl\" (UID: \"7c6320a2-acdb-41eb-a068-b879b515941e\") " pod="openshift-ingress/payload-pre-processing-e2e-api-b-e9388b-7c99d75d46-jtfhl"
Sep 18 14:23:34.885937 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:23:34.885862    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/7c6320a2-acdb-41eb-a068-b879b515941e-plugins-config-volume\") pod \"payload-pre-processing-e2e-api-b-e9388b-7c99d75d46-jtfhl\" (UID: \"7c6320a2-acdb-41eb-a068-b879b515941e\") " pod="openshift-ingress/payload-pre-processing-e2e-api-b-e9388b-7c99d75d46-jtfhl"
Sep 18 14:23:34.889397 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:23:34.889328    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-b-e9388b-7c99d75d46-jtfhl"]
Sep 18 14:23:34.891968 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:23:34.891944    2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-api-b-e9388b-b7c99946c-tgqfb"]
Sep 18 14:23:34.896669 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:23:34.896645    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-b-e9388b-b7c99946c-tgqfb"
Sep 18 14:23:34.908777 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:23:34.908746    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-b-e9388b-b7c99946c-tgqfb"]
Sep 18 14:23:34.987129 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:23:34.987089    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/7c6320a2-acdb-41eb-a068-b879b515941e-plugins-config-volume\") pod \"payload-pre-processing-e2e-api-b-e9388b-7c99d75d46-jtfhl\" (UID: \"7c6320a2-acdb-41eb-a068-b879b515941e\") " pod="openshift-ingress/payload-pre-processing-e2e-api-b-e9388b-7c99d75d46-jtfhl"
Sep 18 14:23:34.987334 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:23:34.987157    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/5bf97fdc-954a-4f6e-95ab-f97750de5462-plugins-config-volume\") pod \"payload-processing-e2e-api-b-e9388b-b7c99946c-tgqfb\" (UID: \"5bf97fdc-954a-4f6e-95ab-f97750de5462\") " pod="openshift-ingress/payload-processing-e2e-api-b-e9388b-b7c99946c-tgqfb"
Sep 18 14:23:34.987334 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:23:34.987258    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lwwgx\" (UniqueName: \"kubernetes.io/projected/5bf97fdc-954a-4f6e-95ab-f97750de5462-kube-api-access-lwwgx\") pod \"payload-processing-e2e-api-b-e9388b-b7c99946c-tgqfb\" (UID: \"5bf97fdc-954a-4f6e-95ab-f97750de5462\") " pod="openshift-ingress/payload-processing-e2e-api-b-e9388b-b7c99946c-tgqfb"
Sep 18 14:23:34.987334 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:23:34.987328    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pr4qg\" (UniqueName: \"kubernetes.io/projected/7c6320a2-acdb-41eb-a068-b879b515941e-kube-api-access-pr4qg\") pod \"payload-pre-processing-e2e-api-b-e9388b-7c99d75d46-jtfhl\" (UID: \"7c6320a2-acdb-41eb-a068-b879b515941e\") " pod="openshift-ingress/payload-pre-processing-e2e-api-b-e9388b-7c99d75d46-jtfhl"
Sep 18 14:23:34.987886 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:23:34.987861    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/7c6320a2-acdb-41eb-a068-b879b515941e-plugins-config-volume\") pod \"payload-pre-processing-e2e-api-b-e9388b-7c99d75d46-jtfhl\" (UID: \"7c6320a2-acdb-41eb-a068-b879b515941e\") " pod="openshift-ingress/payload-pre-processing-e2e-api-b-e9388b-7c99d75d46-jtfhl"
Sep 18 14:23:34.997558 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:23:34.997532    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pr4qg\" (UniqueName: \"kubernetes.io/projected/7c6320a2-acdb-41eb-a068-b879b515941e-kube-api-access-pr4qg\") pod \"payload-pre-processing-e2e-api-b-e9388b-7c99d75d46-jtfhl\" (UID: \"7c6320a2-acdb-41eb-a068-b879b515941e\") " pod="openshift-ingress/payload-pre-processing-e2e-api-b-e9388b-7c99d75d46-jtfhl"
Sep 18 14:23:35.088582 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:23:35.088543    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/5bf97fdc-954a-4f6e-95ab-f97750de5462-plugins-config-volume\") pod \"payload-processing-e2e-api-b-e9388b-b7c99946c-tgqfb\" (UID: \"5bf97fdc-954a-4f6e-95ab-f97750de5462\") " pod="openshift-ingress/payload-processing-e2e-api-b-e9388b-b7c99946c-tgqfb"
Sep 18 14:23:35.088582 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:23:35.088589    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lwwgx\" (UniqueName: \"kubernetes.io/projected/5bf97fdc-954a-4f6e-95ab-f97750de5462-kube-api-access-lwwgx\") pod \"payload-processing-e2e-api-b-e9388b-b7c99946c-tgqfb\" (UID: \"5bf97fdc-954a-4f6e-95ab-f97750de5462\") " pod="openshift-ingress/payload-processing-e2e-api-b-e9388b-b7c99946c-tgqfb"
Sep 18 14:23:35.089389 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:23:35.089368    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/5bf97fdc-954a-4f6e-95ab-f97750de5462-plugins-config-volume\") pod \"payload-processing-e2e-api-b-e9388b-b7c99946c-tgqfb\" (UID: \"5bf97fdc-954a-4f6e-95ab-f97750de5462\") " pod="openshift-ingress/payload-processing-e2e-api-b-e9388b-b7c99946c-tgqfb"
Sep 18 14:23:35.097119 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:23:35.097086    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lwwgx\" (UniqueName: \"kubernetes.io/projected/5bf97fdc-954a-4f6e-95ab-f97750de5462-kube-api-access-lwwgx\") pod \"payload-processing-e2e-api-b-e9388b-b7c99946c-tgqfb\" (UID: \"5bf97fdc-954a-4f6e-95ab-f97750de5462\") " pod="openshift-ingress/payload-processing-e2e-api-b-e9388b-b7c99946c-tgqfb"
Sep 18 14:23:35.180600 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:23:35.180499    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-b-e9388b-7c99d75d46-jtfhl"
Sep 18 14:23:35.211177 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:23:35.211145    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-b-e9388b-b7c99946c-tgqfb"
Sep 18 14:23:35.462929 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:23:35.462901    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-b-e9388b-7c99d75d46-jtfhl"]
Sep 18 14:23:35.464614 ip-10-0-129-16 kubenswrapper[2565]: W0918 14:23:35.464584    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7c6320a2_acdb_41eb_a068_b879b515941e.slice/crio-5cc332234c3fb873f3c11fa624203f464a04fb48217bd124eb7ecc6357f4ef95 WatchSource:0}: Error finding container 5cc332234c3fb873f3c11fa624203f464a04fb48217bd124eb7ecc6357f4ef95: Status 404 returned error can't find the container with id 5cc332234c3fb873f3c11fa624203f464a04fb48217bd124eb7ecc6357f4ef95
Sep 18 14:23:35.502859 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:23:35.502835    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-b-e9388b-b7c99946c-tgqfb"]
Sep 18 14:23:35.503971 ip-10-0-129-16 kubenswrapper[2565]: W0918 14:23:35.503942    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5bf97fdc_954a_4f6e_95ab_f97750de5462.slice/crio-ae5030d49434b65278449c12746fa2de16cd0f9464a3b7712365773e4511d2e5 WatchSource:0}: Error finding container ae5030d49434b65278449c12746fa2de16cd0f9464a3b7712365773e4511d2e5: Status 404 returned error can't find the container with id ae5030d49434b65278449c12746fa2de16cd0f9464a3b7712365773e4511d2e5
Sep 18 14:23:35.553537 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:23:35.553493    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-b-e9388b-7c99d75d46-jtfhl" event={"ID":"7c6320a2-acdb-41eb-a068-b879b515941e","Type":"ContainerStarted","Data":"aac7ca89afd595650c70b22917b6ebef0ad357f874c6f61697e820d793e308bb"}
Sep 18 14:23:35.553669 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:23:35.553582    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-b-e9388b-7c99d75d46-jtfhl" event={"ID":"7c6320a2-acdb-41eb-a068-b879b515941e","Type":"ContainerStarted","Data":"5cc332234c3fb873f3c11fa624203f464a04fb48217bd124eb7ecc6357f4ef95"}
Sep 18 14:23:35.553783 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:23:35.553767    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-api-b-e9388b-7c99d75d46-jtfhl"
Sep 18 14:23:35.555686 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:23:35.555656    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-b-e9388b-b7c99946c-tgqfb" event={"ID":"5bf97fdc-954a-4f6e-95ab-f97750de5462","Type":"ContainerStarted","Data":"ae5030d49434b65278449c12746fa2de16cd0f9464a3b7712365773e4511d2e5"}
Sep 18 14:23:35.575170 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:23:35.575111    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-api-b-e9388b-7c99d75d46-jtfhl" podStartSLOduration=1.575092124 podStartE2EDuration="1.575092124s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 14:23:34 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 14:23:35.5723827 +0000 UTC m=+1483.746292386" watchObservedRunningTime="2026-09-18 14:23:35.575092124 +0000 UTC m=+1483.749001785"
Sep 18 14:23:36.565395 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:23:36.565361    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-b-e9388b-b7c99946c-tgqfb" event={"ID":"5bf97fdc-954a-4f6e-95ab-f97750de5462","Type":"ContainerStarted","Data":"7f90a75e2246d4df7b7656a9c909769e6017d7632af247fddbeb0520a2199e0a"}
Sep 18 14:23:36.565893 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:23:36.565590    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-api-b-e9388b-b7c99946c-tgqfb"
Sep 18 14:23:36.582466 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:23:36.582406    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-api-b-e9388b-b7c99946c-tgqfb" podStartSLOduration=2.58238405 podStartE2EDuration="2.58238405s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 14:23:34 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 14:23:36.582185139 +0000 UTC m=+1484.756094801" watchObservedRunningTime="2026-09-18 14:23:36.58238405 +0000 UTC m=+1484.756293713"
Sep 18 14:23:39.959629 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:23:39.959598    2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w4-93827d-29829015-hvlhq"]
Sep 18 14:23:39.961927 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:23:39.961905    2565 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w4-93827d-29829015-hvlhq"]
Sep 18 14:23:39.987562 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:23:39.987527    2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w4-93827d-d6d7cb69c-m4ztv"]
Sep 18 14:23:39.987839 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:23:39.987804    2565 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-worker-w4-93827d-d6d7cb69c-m4ztv" podUID="e2ac808b-b832-4228-a2e3-ac84d0bfc436" containerName="payload-processing" containerID="cri-o://70f6cbc1ccb3e079a917b877f73fc483b59524cf03bcb6adcbfe5fc44793952b" gracePeriod=30
Sep 18 14:23:39.993500 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:23:39.993475    2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w4-93827d-7ff5557fbb-lbszk"]
Sep 18 14:23:39.993907 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:23:39.993858    2565 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-worker-w4-93827d-7ff5557fbb-lbszk" podUID="d3515ddb-013d-488f-a714-efbb75884469" containerName="payload-pre-processing" containerID="cri-o://f17a2ce76a9fcf82b9835bb5b8d0a3b751d8527a3fb9f88b4d2985de6988caba" gracePeriod=30
Sep 18 14:23:40.157011 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:23:40.156937    2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w4-93827d-4zxxk"]
Sep 18 14:23:40.165058 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:23:40.165000    2565 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w4-93827d-4zxxk"]
Sep 18 14:23:40.270220 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:23:40.270195    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-worker-w4-93827d-d6d7cb69c-m4ztv"
Sep 18 14:23:40.304348 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:23:40.304325    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-w4-93827d-7ff5557fbb-lbszk"
Sep 18 14:23:40.352056 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:23:40.350835    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d3515ddb-013d-488f-a714-efbb75884469-plugins-config-volume\") pod \"d3515ddb-013d-488f-a714-efbb75884469\" (UID: \"d3515ddb-013d-488f-a714-efbb75884469\") "
Sep 18 14:23:40.352056 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:23:40.350902    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-l4qn9\" (UniqueName: \"kubernetes.io/projected/e2ac808b-b832-4228-a2e3-ac84d0bfc436-kube-api-access-l4qn9\") pod \"e2ac808b-b832-4228-a2e3-ac84d0bfc436\" (UID: \"e2ac808b-b832-4228-a2e3-ac84d0bfc436\") "
Sep 18 14:23:40.352056 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:23:40.350950    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-5zgg8\" (UniqueName: \"kubernetes.io/projected/d3515ddb-013d-488f-a714-efbb75884469-kube-api-access-5zgg8\") pod \"d3515ddb-013d-488f-a714-efbb75884469\" (UID: \"d3515ddb-013d-488f-a714-efbb75884469\") "
Sep 18 14:23:40.352056 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:23:40.350994    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e2ac808b-b832-4228-a2e3-ac84d0bfc436-plugins-config-volume\") pod \"e2ac808b-b832-4228-a2e3-ac84d0bfc436\" (UID: \"e2ac808b-b832-4228-a2e3-ac84d0bfc436\") "
Sep 18 14:23:40.354630 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:23:40.354581    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d3515ddb-013d-488f-a714-efbb75884469-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "d3515ddb-013d-488f-a714-efbb75884469" (UID: "d3515ddb-013d-488f-a714-efbb75884469"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 18 14:23:40.355396 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:23:40.355370    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e2ac808b-b832-4228-a2e3-ac84d0bfc436-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "e2ac808b-b832-4228-a2e3-ac84d0bfc436" (UID: "e2ac808b-b832-4228-a2e3-ac84d0bfc436"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 18 14:23:40.356013 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:23:40.355978    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d3515ddb-013d-488f-a714-efbb75884469-kube-api-access-5zgg8" (OuterVolumeSpecName: "kube-api-access-5zgg8") pod "d3515ddb-013d-488f-a714-efbb75884469" (UID: "d3515ddb-013d-488f-a714-efbb75884469"). InnerVolumeSpecName "kube-api-access-5zgg8". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 18 14:23:40.360182 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:23:40.360156    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e2ac808b-b832-4228-a2e3-ac84d0bfc436-kube-api-access-l4qn9" (OuterVolumeSpecName: "kube-api-access-l4qn9") pod "e2ac808b-b832-4228-a2e3-ac84d0bfc436" (UID: "e2ac808b-b832-4228-a2e3-ac84d0bfc436"). InnerVolumeSpecName "kube-api-access-l4qn9". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 18 14:23:40.420861 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:23:40.420821    2565 kuberuntime_container.go:873] "Killing container with a grace period" pod="e2e-models-e2e-worker-w4-93827d/premium-w4-93827d-kserve-78c986f74c-hjr2v" podUID="a1f4563b-6154-48d3-b22f-400041b7795e" containerName="main" containerID="cri-o://1c55fca471057ef537adfcdd5036d5fb69d1e357eadc227b24c0127f07404019" gracePeriod=45
Sep 18 14:23:40.452991 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:23:40.452955    2565 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-l4qn9\" (UniqueName: \"kubernetes.io/projected/e2ac808b-b832-4228-a2e3-ac84d0bfc436-kube-api-access-l4qn9\") on node \"ip-10-0-129-16.ec2.internal\" DevicePath \"\""
Sep 18 14:23:40.452991 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:23:40.452989    2565 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5zgg8\" (UniqueName: \"kubernetes.io/projected/d3515ddb-013d-488f-a714-efbb75884469-kube-api-access-5zgg8\") on node \"ip-10-0-129-16.ec2.internal\" DevicePath \"\""
Sep 18 14:23:40.452991 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:23:40.452999    2565 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e2ac808b-b832-4228-a2e3-ac84d0bfc436-plugins-config-volume\") on node \"ip-10-0-129-16.ec2.internal\" DevicePath \"\""
Sep 18 14:23:40.453246 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:23:40.453009    2565 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d3515ddb-013d-488f-a714-efbb75884469-plugins-config-volume\") on node \"ip-10-0-129-16.ec2.internal\" DevicePath \"\""
Sep 18 14:23:40.595667 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:23:40.595626    2565 generic.go:356] "Generic (PLEG): container finished" podID="a1f4563b-6154-48d3-b22f-400041b7795e" containerID="1c55fca471057ef537adfcdd5036d5fb69d1e357eadc227b24c0127f07404019" exitCode=0
Sep 18 14:23:40.595889 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:23:40.595794    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w4-93827d/premium-w4-93827d-kserve-78c986f74c-hjr2v" event={"ID":"a1f4563b-6154-48d3-b22f-400041b7795e","Type":"ContainerDied","Data":"1c55fca471057ef537adfcdd5036d5fb69d1e357eadc227b24c0127f07404019"}
Sep 18 14:23:40.597872 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:23:40.597848    2565 generic.go:356] "Generic (PLEG): container finished" podID="d3515ddb-013d-488f-a714-efbb75884469" containerID="f17a2ce76a9fcf82b9835bb5b8d0a3b751d8527a3fb9f88b4d2985de6988caba" exitCode=0
Sep 18 14:23:40.598490 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:23:40.597913    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-w4-93827d-7ff5557fbb-lbszk" event={"ID":"d3515ddb-013d-488f-a714-efbb75884469","Type":"ContainerDied","Data":"f17a2ce76a9fcf82b9835bb5b8d0a3b751d8527a3fb9f88b4d2985de6988caba"}
Sep 18 14:23:40.598490 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:23:40.597919    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-w4-93827d-7ff5557fbb-lbszk"
Sep 18 14:23:40.598490 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:23:40.597956    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-w4-93827d-7ff5557fbb-lbszk" event={"ID":"d3515ddb-013d-488f-a714-efbb75884469","Type":"ContainerDied","Data":"381ae5e69f99e9e3e248e98730f5926e344ec3d307790952233eff6d7ef440a0"}
Sep 18 14:23:40.598490 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:23:40.597987    2565 scope.go:117] "RemoveContainer" containerID="f17a2ce76a9fcf82b9835bb5b8d0a3b751d8527a3fb9f88b4d2985de6988caba"
Sep 18 14:23:40.600544 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:23:40.600490    2565 generic.go:356] "Generic (PLEG): container finished" podID="e2ac808b-b832-4228-a2e3-ac84d0bfc436" containerID="70f6cbc1ccb3e079a917b877f73fc483b59524cf03bcb6adcbfe5fc44793952b" exitCode=0
Sep 18 14:23:40.601967 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:23:40.601939    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-worker-w4-93827d-d6d7cb69c-m4ztv" event={"ID":"e2ac808b-b832-4228-a2e3-ac84d0bfc436","Type":"ContainerDied","Data":"70f6cbc1ccb3e079a917b877f73fc483b59524cf03bcb6adcbfe5fc44793952b"}
Sep 18 14:23:40.602085 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:23:40.602001    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-worker-w4-93827d-d6d7cb69c-m4ztv" event={"ID":"e2ac808b-b832-4228-a2e3-ac84d0bfc436","Type":"ContainerDied","Data":"07fd8a6b46b66d31facdd440644d6b276d44cf1581d8e1bee40f11c9d9016f41"}
Sep 18 14:23:40.602149 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:23:40.602087    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-worker-w4-93827d-d6d7cb69c-m4ztv"
Sep 18 14:23:40.612176 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:23:40.612151    2565 scope.go:117] "RemoveContainer" containerID="f17a2ce76a9fcf82b9835bb5b8d0a3b751d8527a3fb9f88b4d2985de6988caba"
Sep 18 14:23:40.612846 ip-10-0-129-16 kubenswrapper[2565]: E0918 14:23:40.612819    2565 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f17a2ce76a9fcf82b9835bb5b8d0a3b751d8527a3fb9f88b4d2985de6988caba\": container with ID starting with f17a2ce76a9fcf82b9835bb5b8d0a3b751d8527a3fb9f88b4d2985de6988caba not found: ID does not exist" containerID="f17a2ce76a9fcf82b9835bb5b8d0a3b751d8527a3fb9f88b4d2985de6988caba"
Sep 18 14:23:40.612950 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:23:40.612854    2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f17a2ce76a9fcf82b9835bb5b8d0a3b751d8527a3fb9f88b4d2985de6988caba"} err="failed to get container status \"f17a2ce76a9fcf82b9835bb5b8d0a3b751d8527a3fb9f88b4d2985de6988caba\": rpc error: code = NotFound desc = could not find container \"f17a2ce76a9fcf82b9835bb5b8d0a3b751d8527a3fb9f88b4d2985de6988caba\": container with ID starting with f17a2ce76a9fcf82b9835bb5b8d0a3b751d8527a3fb9f88b4d2985de6988caba not found: ID does not exist"
Sep 18 14:23:40.612950 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:23:40.612881    2565 scope.go:117] "RemoveContainer" containerID="70f6cbc1ccb3e079a917b877f73fc483b59524cf03bcb6adcbfe5fc44793952b"
Sep 18 14:23:40.629797 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:23:40.625744    2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w4-93827d-7ff5557fbb-lbszk"]
Sep 18 14:23:40.629797 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:23:40.627754    2565 scope.go:117] "RemoveContainer" containerID="70f6cbc1ccb3e079a917b877f73fc483b59524cf03bcb6adcbfe5fc44793952b"
Sep 18 14:23:40.632392 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:23:40.632366    2565 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w4-93827d-7ff5557fbb-lbszk"]
Sep 18 14:23:40.632752 ip-10-0-129-16 kubenswrapper[2565]: E0918 14:23:40.632673    2565 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"70f6cbc1ccb3e079a917b877f73fc483b59524cf03bcb6adcbfe5fc44793952b\": container with ID starting with 70f6cbc1ccb3e079a917b877f73fc483b59524cf03bcb6adcbfe5fc44793952b not found: ID does not exist" containerID="70f6cbc1ccb3e079a917b877f73fc483b59524cf03bcb6adcbfe5fc44793952b"
Sep 18 14:23:40.632867 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:23:40.632760    2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"70f6cbc1ccb3e079a917b877f73fc483b59524cf03bcb6adcbfe5fc44793952b"} err="failed to get container status \"70f6cbc1ccb3e079a917b877f73fc483b59524cf03bcb6adcbfe5fc44793952b\": rpc error: code = NotFound desc = could not find container \"70f6cbc1ccb3e079a917b877f73fc483b59524cf03bcb6adcbfe5fc44793952b\": container with ID starting with 70f6cbc1ccb3e079a917b877f73fc483b59524cf03bcb6adcbfe5fc44793952b not found: ID does not exist"
Sep 18 14:23:40.646593 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:23:40.646567    2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w4-93827d-d6d7cb69c-m4ztv"]
Sep 18 14:23:40.650406 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:23:40.650378    2565 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w4-93827d-d6d7cb69c-m4ztv"]
Sep 18 14:23:40.675142 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:23:40.675116    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w4-93827d/premium-w4-93827d-kserve-78c986f74c-hjr2v"
Sep 18 14:23:40.756178 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:23:40.756147    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-6pqbc\" (UniqueName: \"kubernetes.io/projected/a1f4563b-6154-48d3-b22f-400041b7795e-kube-api-access-6pqbc\") pod \"a1f4563b-6154-48d3-b22f-400041b7795e\" (UID: \"a1f4563b-6154-48d3-b22f-400041b7795e\") "
Sep 18 14:23:40.756367 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:23:40.756198    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a1f4563b-6154-48d3-b22f-400041b7795e-tls-certs\") pod \"a1f4563b-6154-48d3-b22f-400041b7795e\" (UID: \"a1f4563b-6154-48d3-b22f-400041b7795e\") "
Sep 18 14:23:40.756367 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:23:40.756239    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a1f4563b-6154-48d3-b22f-400041b7795e-tmp-dir\") pod \"a1f4563b-6154-48d3-b22f-400041b7795e\" (UID: \"a1f4563b-6154-48d3-b22f-400041b7795e\") "
Sep 18 14:23:40.756367 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:23:40.756265    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a1f4563b-6154-48d3-b22f-400041b7795e-model-cache\") pod \"a1f4563b-6154-48d3-b22f-400041b7795e\" (UID: \"a1f4563b-6154-48d3-b22f-400041b7795e\") "
Sep 18 14:23:40.756367 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:23:40.756307    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a1f4563b-6154-48d3-b22f-400041b7795e-dshm\") pod \"a1f4563b-6154-48d3-b22f-400041b7795e\" (UID: \"a1f4563b-6154-48d3-b22f-400041b7795e\") "
Sep 18 14:23:40.756611 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:23:40.756393    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a1f4563b-6154-48d3-b22f-400041b7795e-home\") pod \"a1f4563b-6154-48d3-b22f-400041b7795e\" (UID: \"a1f4563b-6154-48d3-b22f-400041b7795e\") "
Sep 18 14:23:40.756671 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:23:40.756625    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a1f4563b-6154-48d3-b22f-400041b7795e-home" (OuterVolumeSpecName: "home") pod "a1f4563b-6154-48d3-b22f-400041b7795e" (UID: "a1f4563b-6154-48d3-b22f-400041b7795e"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 18 14:23:40.756721 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:23:40.756662    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a1f4563b-6154-48d3-b22f-400041b7795e-model-cache" (OuterVolumeSpecName: "model-cache") pod "a1f4563b-6154-48d3-b22f-400041b7795e" (UID: "a1f4563b-6154-48d3-b22f-400041b7795e"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 18 14:23:40.756842 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:23:40.756814    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a1f4563b-6154-48d3-b22f-400041b7795e-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "a1f4563b-6154-48d3-b22f-400041b7795e" (UID: "a1f4563b-6154-48d3-b22f-400041b7795e"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 18 14:23:40.756988 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:23:40.756858    2565 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a1f4563b-6154-48d3-b22f-400041b7795e-model-cache\") on node \"ip-10-0-129-16.ec2.internal\" DevicePath \"\""
Sep 18 14:23:40.756988 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:23:40.756873    2565 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a1f4563b-6154-48d3-b22f-400041b7795e-home\") on node \"ip-10-0-129-16.ec2.internal\" DevicePath \"\""
Sep 18 14:23:40.758506 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:23:40.758483    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a1f4563b-6154-48d3-b22f-400041b7795e-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "a1f4563b-6154-48d3-b22f-400041b7795e" (UID: "a1f4563b-6154-48d3-b22f-400041b7795e"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 18 14:23:40.758675 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:23:40.758657    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a1f4563b-6154-48d3-b22f-400041b7795e-dshm" (OuterVolumeSpecName: "dshm") pod "a1f4563b-6154-48d3-b22f-400041b7795e" (UID: "a1f4563b-6154-48d3-b22f-400041b7795e"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 18 14:23:40.758753 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:23:40.758734    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a1f4563b-6154-48d3-b22f-400041b7795e-kube-api-access-6pqbc" (OuterVolumeSpecName: "kube-api-access-6pqbc") pod "a1f4563b-6154-48d3-b22f-400041b7795e" (UID: "a1f4563b-6154-48d3-b22f-400041b7795e"). InnerVolumeSpecName "kube-api-access-6pqbc". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 18 14:23:40.857776 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:23:40.857742    2565 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6pqbc\" (UniqueName: \"kubernetes.io/projected/a1f4563b-6154-48d3-b22f-400041b7795e-kube-api-access-6pqbc\") on node \"ip-10-0-129-16.ec2.internal\" DevicePath \"\""
Sep 18 14:23:40.857776 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:23:40.857772    2565 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a1f4563b-6154-48d3-b22f-400041b7795e-tls-certs\") on node \"ip-10-0-129-16.ec2.internal\" DevicePath \"\""
Sep 18 14:23:40.857776 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:23:40.857782    2565 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a1f4563b-6154-48d3-b22f-400041b7795e-tmp-dir\") on node \"ip-10-0-129-16.ec2.internal\" DevicePath \"\""
Sep 18 14:23:40.858020 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:23:40.857789    2565 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a1f4563b-6154-48d3-b22f-400041b7795e-dshm\") on node \"ip-10-0-129-16.ec2.internal\" DevicePath \"\""
Sep 18 14:23:41.609077 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:23:41.609028    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w4-93827d/premium-w4-93827d-kserve-78c986f74c-hjr2v"
Sep 18 14:23:41.609490 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:23:41.609069    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w4-93827d/premium-w4-93827d-kserve-78c986f74c-hjr2v" event={"ID":"a1f4563b-6154-48d3-b22f-400041b7795e","Type":"ContainerDied","Data":"7b40e899307dc2f8a807d4797d4e5c7d0851011046d89983cff538452c056fde"}
Sep 18 14:23:41.609490 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:23:41.609134    2565 scope.go:117] "RemoveContainer" containerID="1c55fca471057ef537adfcdd5036d5fb69d1e357eadc227b24c0127f07404019"
Sep 18 14:23:41.633627 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:23:41.633595    2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w4-93827d/premium-w4-93827d-kserve-78c986f74c-hjr2v"]
Sep 18 14:23:41.635826 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:23:41.635802    2565 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["e2e-models-e2e-worker-w4-93827d/premium-w4-93827d-kserve-78c986f74c-hjr2v"]
Sep 18 14:23:41.914511 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:23:41.914427    2565 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3019edca-2c9a-4eca-ba96-0530a5bc8213" path="/var/lib/kubelet/pods/3019edca-2c9a-4eca-ba96-0530a5bc8213/volumes"
Sep 18 14:23:41.914928 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:23:41.914910    2565 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="457bbf5d-74af-4d4a-a2ca-9e7e3bd9d61a" path="/var/lib/kubelet/pods/457bbf5d-74af-4d4a-a2ca-9e7e3bd9d61a/volumes"
Sep 18 14:23:41.915363 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:23:41.915342    2565 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a1f4563b-6154-48d3-b22f-400041b7795e" path="/var/lib/kubelet/pods/a1f4563b-6154-48d3-b22f-400041b7795e/volumes"
Sep 18 14:23:41.915697 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:23:41.915682    2565 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d3515ddb-013d-488f-a714-efbb75884469" path="/var/lib/kubelet/pods/d3515ddb-013d-488f-a714-efbb75884469/volumes"
Sep 18 14:23:41.916235 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:23:41.916215    2565 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e2ac808b-b832-4228-a2e3-ac84d0bfc436" path="/var/lib/kubelet/pods/e2ac808b-b832-4228-a2e3-ac84d0bfc436/volumes"
Sep 18 14:23:46.567522 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:23:46.567489    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-api-b-e9388b-7c99d75d46-jtfhl"
Sep 18 14:23:47.575260 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:23:47.575226    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-api-b-e9388b-b7c99946c-tgqfb"
Sep 18 14:23:47.999546 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:23:47.999463    2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w4-93827d-openshift-default-6c5bbfcf98-7nrxb"]
Sep 18 14:23:47.999790 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:23:47.999761    2565 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w4-93827d-openshift-default-6c5bbfcf98-7nrxb" podUID="97394e77-67c5-435f-9457-f9af6436ad5c" containerName="istio-proxy" containerID="cri-o://3afd768339cfe21115a68520d406c6e364c9d56db530769f00873cd1b1eb0561" gracePeriod=30
Sep 18 14:23:48.725988 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:23:48.725952    2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-e01050-7gpm4"]
Sep 18 14:23:48.726515 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:23:48.726499    2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e2ac808b-b832-4228-a2e3-ac84d0bfc436" containerName="payload-processing"
Sep 18 14:23:48.726515 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:23:48.726516    2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="e2ac808b-b832-4228-a2e3-ac84d0bfc436" containerName="payload-processing"
Sep 18 14:23:48.726594 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:23:48.726534    2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d3515ddb-013d-488f-a714-efbb75884469" containerName="payload-pre-processing"
Sep 18 14:23:48.726594 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:23:48.726543    2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="d3515ddb-013d-488f-a714-efbb75884469" containerName="payload-pre-processing"
Sep 18 14:23:48.726594 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:23:48.726560    2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a1f4563b-6154-48d3-b22f-400041b7795e" containerName="main"
Sep 18 14:23:48.726594 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:23:48.726566    2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="a1f4563b-6154-48d3-b22f-400041b7795e" containerName="main"
Sep 18 14:23:48.726768 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:23:48.726655    2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="e2ac808b-b832-4228-a2e3-ac84d0bfc436" containerName="payload-processing"
Sep 18 14:23:48.726768 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:23:48.726668    2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="d3515ddb-013d-488f-a714-efbb75884469" containerName="payload-pre-processing"
Sep 18 14:23:48.726768 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:23:48.726675    2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="a1f4563b-6154-48d3-b22f-400041b7795e" containerName="main"
Sep 18 14:23:48.730145 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:23:48.730125    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-e01050-7gpm4"
Sep 18 14:23:48.732667 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:23:48.732647    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-l4fn9\""
Sep 18 14:23:48.737550 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:23:48.737524    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-e01050-7gpm4"]
Sep 18 14:23:48.848713 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:23:48.848410    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/638a4823-d35e-4ac4-80e5-8574000d9a80-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-a-e01050-7gpm4\" (UID: \"638a4823-d35e-4ac4-80e5-8574000d9a80\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-e01050-7gpm4"
Sep 18 14:23:48.950018 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:23:48.949972    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/638a4823-d35e-4ac4-80e5-8574000d9a80-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-a-e01050-7gpm4\" (UID: \"638a4823-d35e-4ac4-80e5-8574000d9a80\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-e01050-7gpm4"
Sep 18 14:23:48.950669 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:23:48.950641    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/638a4823-d35e-4ac4-80e5-8574000d9a80-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-a-e01050-7gpm4\" (UID: \"638a4823-d35e-4ac4-80e5-8574000d9a80\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-e01050-7gpm4"
Sep 18 14:23:49.043179 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:23:49.043150    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-e01050-7gpm4"
Sep 18 14:23:49.315917 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:23:49.315888    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-e01050-7gpm4"]
Sep 18 14:23:49.317549 ip-10-0-129-16 kubenswrapper[2565]: W0918 14:23:49.317522    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod638a4823_d35e_4ac4_80e5_8574000d9a80.slice/crio-344f76681e1924af70f75e92f0c96cdbca0d0ea285538ff044ece6e414b08331 WatchSource:0}: Error finding container 344f76681e1924af70f75e92f0c96cdbca0d0ea285538ff044ece6e414b08331: Status 404 returned error can't find the container with id 344f76681e1924af70f75e92f0c96cdbca0d0ea285538ff044ece6e414b08331
Sep 18 14:23:49.661774 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:23:49.661747    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-e01050-7gpm4" event={"ID":"638a4823-d35e-4ac4-80e5-8574000d9a80","Type":"ContainerStarted","Data":"d372f34077fe7108208f9dcf88f82dab05c1aca54e24cfe946c05541e7b0a2d5"}
Sep 18 14:23:49.661895 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:23:49.661796    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-e01050-7gpm4" event={"ID":"638a4823-d35e-4ac4-80e5-8574000d9a80","Type":"ContainerStarted","Data":"344f76681e1924af70f75e92f0c96cdbca0d0ea285538ff044ece6e414b08331"}
Sep 18 14:23:49.676646 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:23:49.676604    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-e01050-7gpm4" podStartSLOduration=1.6765890909999999 podStartE2EDuration="1.676589091s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 14:23:48 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 14:23:49.675765163 +0000 UTC m=+1497.849674824" watchObservedRunningTime="2026-09-18 14:23:49.676589091 +0000 UTC m=+1497.850498751"
Sep 18 14:23:50.669342 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:23:50.669300    2565 generic.go:356] "Generic (PLEG): container finished" podID="638a4823-d35e-4ac4-80e5-8574000d9a80" containerID="d372f34077fe7108208f9dcf88f82dab05c1aca54e24cfe946c05541e7b0a2d5" exitCode=0
Sep 18 14:23:50.669723 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:23:50.669393    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-e01050-7gpm4" event={"ID":"638a4823-d35e-4ac4-80e5-8574000d9a80","Type":"ContainerDied","Data":"d372f34077fe7108208f9dcf88f82dab05c1aca54e24cfe946c05541e7b0a2d5"}
Sep 18 14:23:51.814919 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:23:51.814896    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-e01050-7gpm4"
Sep 18 14:23:51.882893 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:23:51.882861    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/638a4823-d35e-4ac4-80e5-8574000d9a80-maas-api-service-ca\") pod \"638a4823-d35e-4ac4-80e5-8574000d9a80\" (UID: \"638a4823-d35e-4ac4-80e5-8574000d9a80\") "
Sep 18 14:23:51.883274 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:23:51.883249    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/638a4823-d35e-4ac4-80e5-8574000d9a80-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "638a4823-d35e-4ac4-80e5-8574000d9a80" (UID: "638a4823-d35e-4ac4-80e5-8574000d9a80"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 18 14:23:51.984276 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:23:51.984191    2565 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/638a4823-d35e-4ac4-80e5-8574000d9a80-maas-api-service-ca\") on node \"ip-10-0-129-16.ec2.internal\" DevicePath \"\""
Sep 18 14:23:52.683400 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:23:52.683369    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-e01050-7gpm4"
Sep 18 14:23:52.683400 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:23:52.683401    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-e01050-7gpm4" event={"ID":"638a4823-d35e-4ac4-80e5-8574000d9a80","Type":"ContainerDied","Data":"344f76681e1924af70f75e92f0c96cdbca0d0ea285538ff044ece6e414b08331"}
Sep 18 14:23:52.683634 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:23:52.683451    2565 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="344f76681e1924af70f75e92f0c96cdbca0d0ea285538ff044ece6e414b08331"
Sep 18 14:23:53.253265 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:23:53.253238    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-93827d-openshift-default-6c5bbfcf98-7nrxb"
Sep 18 14:23:53.398898 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:23:53.398859    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/97394e77-67c5-435f-9457-f9af6436ad5c-credential-socket\") pod \"97394e77-67c5-435f-9457-f9af6436ad5c\" (UID: \"97394e77-67c5-435f-9457-f9af6436ad5c\") "
Sep 18 14:23:53.399107 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:23:53.398909    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/97394e77-67c5-435f-9457-f9af6436ad5c-workload-socket\") pod \"97394e77-67c5-435f-9457-f9af6436ad5c\" (UID: \"97394e77-67c5-435f-9457-f9af6436ad5c\") "
Sep 18 14:23:53.399107 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:23:53.398931    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/97394e77-67c5-435f-9457-f9af6436ad5c-workload-certs\") pod \"97394e77-67c5-435f-9457-f9af6436ad5c\" (UID: \"97394e77-67c5-435f-9457-f9af6436ad5c\") "
Sep 18 14:23:53.399107 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:23:53.398990    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/97394e77-67c5-435f-9457-f9af6436ad5c-istio-podinfo\") pod \"97394e77-67c5-435f-9457-f9af6436ad5c\" (UID: \"97394e77-67c5-435f-9457-f9af6436ad5c\") "
Sep 18 14:23:53.399107 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:23:53.399100    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/97394e77-67c5-435f-9457-f9af6436ad5c-istio-token\") pod \"97394e77-67c5-435f-9457-f9af6436ad5c\" (UID: \"97394e77-67c5-435f-9457-f9af6436ad5c\") "
Sep 18 14:23:53.399372 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:23:53.399152    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-2ghg8\" (UniqueName: \"kubernetes.io/projected/97394e77-67c5-435f-9457-f9af6436ad5c-kube-api-access-2ghg8\") pod \"97394e77-67c5-435f-9457-f9af6436ad5c\" (UID: \"97394e77-67c5-435f-9457-f9af6436ad5c\") "
Sep 18 14:23:53.399372 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:23:53.399230    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/97394e77-67c5-435f-9457-f9af6436ad5c-istio-envoy\") pod \"97394e77-67c5-435f-9457-f9af6436ad5c\" (UID: \"97394e77-67c5-435f-9457-f9af6436ad5c\") "
Sep 18 14:23:53.399372 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:23:53.399256    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/97394e77-67c5-435f-9457-f9af6436ad5c-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "97394e77-67c5-435f-9457-f9af6436ad5c" (UID: "97394e77-67c5-435f-9457-f9af6436ad5c"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 18 14:23:53.399372 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:23:53.399325    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/97394e77-67c5-435f-9457-f9af6436ad5c-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "97394e77-67c5-435f-9457-f9af6436ad5c" (UID: "97394e77-67c5-435f-9457-f9af6436ad5c"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 18 14:23:53.399372 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:23:53.399341    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/97394e77-67c5-435f-9457-f9af6436ad5c-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "97394e77-67c5-435f-9457-f9af6436ad5c" (UID: "97394e77-67c5-435f-9457-f9af6436ad5c"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 18 14:23:53.400230 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:23:53.399809    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/97394e77-67c5-435f-9457-f9af6436ad5c-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "97394e77-67c5-435f-9457-f9af6436ad5c" (UID: "97394e77-67c5-435f-9457-f9af6436ad5c"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 18 14:23:53.400230 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:23:53.399279    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/97394e77-67c5-435f-9457-f9af6436ad5c-istiod-ca-cert\") pod \"97394e77-67c5-435f-9457-f9af6436ad5c\" (UID: \"97394e77-67c5-435f-9457-f9af6436ad5c\") "
Sep 18 14:23:53.400230 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:23:53.399905    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/97394e77-67c5-435f-9457-f9af6436ad5c-istio-data\") pod \"97394e77-67c5-435f-9457-f9af6436ad5c\" (UID: \"97394e77-67c5-435f-9457-f9af6436ad5c\") "
Sep 18 14:23:53.400230 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:23:53.400165    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/97394e77-67c5-435f-9457-f9af6436ad5c-istio-data" (OuterVolumeSpecName: "istio-data") pod "97394e77-67c5-435f-9457-f9af6436ad5c" (UID: "97394e77-67c5-435f-9457-f9af6436ad5c"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 18 14:23:53.400799 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:23:53.400773    2565 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/97394e77-67c5-435f-9457-f9af6436ad5c-istiod-ca-cert\") on node \"ip-10-0-129-16.ec2.internal\" DevicePath \"\""
Sep 18 14:23:53.400799 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:23:53.400800    2565 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/97394e77-67c5-435f-9457-f9af6436ad5c-istio-data\") on node \"ip-10-0-129-16.ec2.internal\" DevicePath \"\""
Sep 18 14:23:53.400964 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:23:53.400811    2565 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/97394e77-67c5-435f-9457-f9af6436ad5c-credential-socket\") on node \"ip-10-0-129-16.ec2.internal\" DevicePath \"\""
Sep 18 14:23:53.400964 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:23:53.400825    2565 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/97394e77-67c5-435f-9457-f9af6436ad5c-workload-socket\") on node \"ip-10-0-129-16.ec2.internal\" DevicePath \"\""
Sep 18 14:23:53.400964 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:23:53.400840    2565 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/97394e77-67c5-435f-9457-f9af6436ad5c-workload-certs\") on node \"ip-10-0-129-16.ec2.internal\" DevicePath \"\""
Sep 18 14:23:53.402012 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:23:53.401984    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/97394e77-67c5-435f-9457-f9af6436ad5c-istio-token" (OuterVolumeSpecName: "istio-token") pod "97394e77-67c5-435f-9457-f9af6436ad5c" (UID: "97394e77-67c5-435f-9457-f9af6436ad5c"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 18 14:23:53.402138 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:23:53.402018    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/97394e77-67c5-435f-9457-f9af6436ad5c-kube-api-access-2ghg8" (OuterVolumeSpecName: "kube-api-access-2ghg8") pod "97394e77-67c5-435f-9457-f9af6436ad5c" (UID: "97394e77-67c5-435f-9457-f9af6436ad5c"). InnerVolumeSpecName "kube-api-access-2ghg8". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 18 14:23:53.402314 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:23:53.402280    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/97394e77-67c5-435f-9457-f9af6436ad5c-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "97394e77-67c5-435f-9457-f9af6436ad5c" (UID: "97394e77-67c5-435f-9457-f9af6436ad5c"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 18 14:23:53.403129 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:23:53.403093    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/97394e77-67c5-435f-9457-f9af6436ad5c-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "97394e77-67c5-435f-9457-f9af6436ad5c" (UID: "97394e77-67c5-435f-9457-f9af6436ad5c"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 18 14:23:53.501871 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:23:53.501837    2565 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/97394e77-67c5-435f-9457-f9af6436ad5c-istio-podinfo\") on node \"ip-10-0-129-16.ec2.internal\" DevicePath \"\""
Sep 18 14:23:53.501871 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:23:53.501864    2565 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/97394e77-67c5-435f-9457-f9af6436ad5c-istio-token\") on node \"ip-10-0-129-16.ec2.internal\" DevicePath \"\""
Sep 18 14:23:53.501871 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:23:53.501873    2565 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2ghg8\" (UniqueName: \"kubernetes.io/projected/97394e77-67c5-435f-9457-f9af6436ad5c-kube-api-access-2ghg8\") on node \"ip-10-0-129-16.ec2.internal\" DevicePath \"\""
Sep 18 14:23:53.501871 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:23:53.501881    2565 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/97394e77-67c5-435f-9457-f9af6436ad5c-istio-envoy\") on node \"ip-10-0-129-16.ec2.internal\" DevicePath \"\""
Sep 18 14:23:53.692101 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:23:53.691997    2565 generic.go:356] "Generic (PLEG): container finished" podID="97394e77-67c5-435f-9457-f9af6436ad5c" containerID="3afd768339cfe21115a68520d406c6e364c9d56db530769f00873cd1b1eb0561" exitCode=0
Sep 18 14:23:53.692101 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:23:53.692060    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-93827d-openshift-default-6c5bbfcf98-7nrxb" event={"ID":"97394e77-67c5-435f-9457-f9af6436ad5c","Type":"ContainerDied","Data":"3afd768339cfe21115a68520d406c6e364c9d56db530769f00873cd1b1eb0561"}
Sep 18 14:23:53.692101 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:23:53.692080    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-93827d-openshift-default-6c5bbfcf98-7nrxb"
Sep 18 14:23:53.692386 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:23:53.692115    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-93827d-openshift-default-6c5bbfcf98-7nrxb" event={"ID":"97394e77-67c5-435f-9457-f9af6436ad5c","Type":"ContainerDied","Data":"bf8f522046d319b170f150aadbbeb99575d9bbceb8042cce4bbb6e4275107327"}
Sep 18 14:23:53.692386 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:23:53.692143    2565 scope.go:117] "RemoveContainer" containerID="3afd768339cfe21115a68520d406c6e364c9d56db530769f00873cd1b1eb0561"
Sep 18 14:23:53.702013 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:23:53.701994    2565 scope.go:117] "RemoveContainer" containerID="cf019763542150ecaef6b5767da1064e3fe353cd4c2e357badc0cb9b908aaad0"
Sep 18 14:23:53.711043 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:23:53.711015    2565 scope.go:117] "RemoveContainer" containerID="3afd768339cfe21115a68520d406c6e364c9d56db530769f00873cd1b1eb0561"
Sep 18 14:23:53.711438 ip-10-0-129-16 kubenswrapper[2565]: E0918 14:23:53.711414    2565 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3afd768339cfe21115a68520d406c6e364c9d56db530769f00873cd1b1eb0561\": container with ID starting with 3afd768339cfe21115a68520d406c6e364c9d56db530769f00873cd1b1eb0561 not found: ID does not exist" containerID="3afd768339cfe21115a68520d406c6e364c9d56db530769f00873cd1b1eb0561"
Sep 18 14:23:53.711572 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:23:53.711448    2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3afd768339cfe21115a68520d406c6e364c9d56db530769f00873cd1b1eb0561"} err="failed to get container status \"3afd768339cfe21115a68520d406c6e364c9d56db530769f00873cd1b1eb0561\": rpc error: code = NotFound desc = could not find container \"3afd768339cfe21115a68520d406c6e364c9d56db530769f00873cd1b1eb0561\": container with ID starting with 3afd768339cfe21115a68520d406c6e364c9d56db530769f00873cd1b1eb0561 not found: ID does not exist"
Sep 18 14:23:53.711572 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:23:53.711471    2565 scope.go:117] "RemoveContainer" containerID="cf019763542150ecaef6b5767da1064e3fe353cd4c2e357badc0cb9b908aaad0"
Sep 18 14:23:53.711872 ip-10-0-129-16 kubenswrapper[2565]: E0918 14:23:53.711846    2565 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"cf019763542150ecaef6b5767da1064e3fe353cd4c2e357badc0cb9b908aaad0\": container with ID starting with cf019763542150ecaef6b5767da1064e3fe353cd4c2e357badc0cb9b908aaad0 not found: ID does not exist" containerID="cf019763542150ecaef6b5767da1064e3fe353cd4c2e357badc0cb9b908aaad0"
Sep 18 14:23:53.711960 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:23:53.711880    2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"cf019763542150ecaef6b5767da1064e3fe353cd4c2e357badc0cb9b908aaad0"} err="failed to get container status \"cf019763542150ecaef6b5767da1064e3fe353cd4c2e357badc0cb9b908aaad0\": rpc error: code = NotFound desc = could not find container \"cf019763542150ecaef6b5767da1064e3fe353cd4c2e357badc0cb9b908aaad0\": container with ID starting with cf019763542150ecaef6b5767da1064e3fe353cd4c2e357badc0cb9b908aaad0 not found: ID does not exist"
Sep 18 14:23:53.715199 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:23:53.715176    2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w4-93827d-openshift-default-6c5bbfcf98-7nrxb"]
Sep 18 14:23:53.718824 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:23:53.718802    2565 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w4-93827d-openshift-default-6c5bbfcf98-7nrxb"]
Sep 18 14:23:53.915501 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:23:53.915462    2565 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="97394e77-67c5-435f-9457-f9af6436ad5c" path="/var/lib/kubelet/pods/97394e77-67c5-435f-9457-f9af6436ad5c/volumes"
Sep 18 14:23:54.005575 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:23:54.005404    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-bvqls_0b0463f5-d647-4a38-b752-44afba77daff/ovn-acl-logging/0.log"
Sep 18 14:23:54.030446 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:23:54.012761    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-bvqls_0b0463f5-d647-4a38-b752-44afba77daff/ovn-acl-logging/0.log"
Sep 18 14:23:54.204134 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:23:54.204100    2565 scope.go:117] "RemoveContainer" containerID="937c3b7413ee6a3e019d88537b1defe3e8f7e333b55815e505739a1b739afb5f"
Sep 18 14:23:54.213113 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:23:54.213095    2565 scope.go:117] "RemoveContainer" containerID="2fb1db2d09a1e077528a4bf4ec640c72e5df4d86d8972d0d5d1ec52720898899"
Sep 18 14:23:54.221632 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:23:54.221611    2565 scope.go:117] "RemoveContainer" containerID="2603ebb591e13a58b6bd18d4d33958f45ebe30cfd64969bdba8f25972cd9356c"
Sep 18 14:24:04.485127 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:24:04.485082    2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-e01050-6896bdcc57-5fd9d"]
Sep 18 14:24:04.485607 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:24:04.485302    2565 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-api-a-e01050-6896bdcc57-5fd9d" podUID="943c8a84-652b-4baa-aed8-d6faa3ca8dbc" containerName="payload-processing" containerID="cri-o://225dcbd6ff1fa786b59fcdf2a1a6fb0b58bc28a70b176537f71be18b8c7f43d4" gracePeriod=30
Sep 18 14:24:04.658355 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:24:04.658287    2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-e01050-7gpm4"]
Sep 18 14:24:04.661181 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:24:04.661153    2565 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-e01050-7gpm4"]
Sep 18 14:24:04.761385 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:24:04.761325    2565 generic.go:356] "Generic (PLEG): container finished" podID="943c8a84-652b-4baa-aed8-d6faa3ca8dbc" containerID="225dcbd6ff1fa786b59fcdf2a1a6fb0b58bc28a70b176537f71be18b8c7f43d4" exitCode=0
Sep 18 14:24:04.761385 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:24:04.761358    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-a-e01050-6896bdcc57-5fd9d" event={"ID":"943c8a84-652b-4baa-aed8-d6faa3ca8dbc","Type":"ContainerDied","Data":"225dcbd6ff1fa786b59fcdf2a1a6fb0b58bc28a70b176537f71be18b8c7f43d4"}
Sep 18 14:24:04.761652 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:24:04.761424    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-a-e01050-6896bdcc57-5fd9d" event={"ID":"943c8a84-652b-4baa-aed8-d6faa3ca8dbc","Type":"ContainerDied","Data":"612da483cf354a00fe979673cb97c7a68a87735f3def798dc16244da5512d39d"}
Sep 18 14:24:04.761652 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:24:04.761448    2565 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="612da483cf354a00fe979673cb97c7a68a87735f3def798dc16244da5512d39d"
Sep 18 14:24:04.764211 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:24:04.764191    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-a-e01050-6896bdcc57-5fd9d"
Sep 18 14:24:04.916826 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:24:04.916797    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/943c8a84-652b-4baa-aed8-d6faa3ca8dbc-plugins-config-volume\") pod \"943c8a84-652b-4baa-aed8-d6faa3ca8dbc\" (UID: \"943c8a84-652b-4baa-aed8-d6faa3ca8dbc\") "
Sep 18 14:24:04.917041 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:24:04.916893    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-wcfx8\" (UniqueName: \"kubernetes.io/projected/943c8a84-652b-4baa-aed8-d6faa3ca8dbc-kube-api-access-wcfx8\") pod \"943c8a84-652b-4baa-aed8-d6faa3ca8dbc\" (UID: \"943c8a84-652b-4baa-aed8-d6faa3ca8dbc\") "
Sep 18 14:24:04.917184 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:24:04.917163    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/943c8a84-652b-4baa-aed8-d6faa3ca8dbc-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "943c8a84-652b-4baa-aed8-d6faa3ca8dbc" (UID: "943c8a84-652b-4baa-aed8-d6faa3ca8dbc"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 18 14:24:04.918975 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:24:04.918938    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/943c8a84-652b-4baa-aed8-d6faa3ca8dbc-kube-api-access-wcfx8" (OuterVolumeSpecName: "kube-api-access-wcfx8") pod "943c8a84-652b-4baa-aed8-d6faa3ca8dbc" (UID: "943c8a84-652b-4baa-aed8-d6faa3ca8dbc"). InnerVolumeSpecName "kube-api-access-wcfx8". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 18 14:24:05.018815 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:24:05.018718    2565 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/943c8a84-652b-4baa-aed8-d6faa3ca8dbc-plugins-config-volume\") on node \"ip-10-0-129-16.ec2.internal\" DevicePath \"\""
Sep 18 14:24:05.018815 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:24:05.018752    2565 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wcfx8\" (UniqueName: \"kubernetes.io/projected/943c8a84-652b-4baa-aed8-d6faa3ca8dbc-kube-api-access-wcfx8\") on node \"ip-10-0-129-16.ec2.internal\" DevicePath \"\""
Sep 18 14:24:05.767069 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:24:05.766987    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-a-e01050-6896bdcc57-5fd9d"
Sep 18 14:24:05.795658 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:24:05.795619    2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-e01050-6896bdcc57-5fd9d"]
Sep 18 14:24:05.806471 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:24:05.806440    2565 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-e01050-6896bdcc57-5fd9d"]
Sep 18 14:24:05.913603 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:24:05.913566    2565 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="638a4823-d35e-4ac4-80e5-8574000d9a80" path="/var/lib/kubelet/pods/638a4823-d35e-4ac4-80e5-8574000d9a80/volumes"
Sep 18 14:24:05.913952 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:24:05.913938    2565 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="943c8a84-652b-4baa-aed8-d6faa3ca8dbc" path="/var/lib/kubelet/pods/943c8a84-652b-4baa-aed8-d6faa3ca8dbc/volumes"
Sep 18 14:24:11.353615 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:24:11.353579    2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-e9388b-fxb7z"]
Sep 18 14:24:11.354219 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:24:11.354201    2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="943c8a84-652b-4baa-aed8-d6faa3ca8dbc" containerName="payload-processing"
Sep 18 14:24:11.354283 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:24:11.354220    2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="943c8a84-652b-4baa-aed8-d6faa3ca8dbc" containerName="payload-processing"
Sep 18 14:24:11.354283 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:24:11.354240    2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="97394e77-67c5-435f-9457-f9af6436ad5c" containerName="istio-proxy"
Sep 18 14:24:11.354283 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:24:11.354246    2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="97394e77-67c5-435f-9457-f9af6436ad5c" containerName="istio-proxy"
Sep 18 14:24:11.354283 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:24:11.354257    2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="638a4823-d35e-4ac4-80e5-8574000d9a80" containerName="revoke-keys"
Sep 18 14:24:11.354283 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:24:11.354266    2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="638a4823-d35e-4ac4-80e5-8574000d9a80" containerName="revoke-keys"
Sep 18 14:24:11.354283 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:24:11.354280    2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="97394e77-67c5-435f-9457-f9af6436ad5c" containerName="istio-proxy"
Sep 18 14:24:11.354283 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:24:11.354286    2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="97394e77-67c5-435f-9457-f9af6436ad5c" containerName="istio-proxy"
Sep 18 14:24:11.354499 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:24:11.354354    2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="943c8a84-652b-4baa-aed8-d6faa3ca8dbc" containerName="payload-processing"
Sep 18 14:24:11.354499 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:24:11.354366    2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="638a4823-d35e-4ac4-80e5-8574000d9a80" containerName="revoke-keys"
Sep 18 14:24:11.354499 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:24:11.354375    2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="97394e77-67c5-435f-9457-f9af6436ad5c" containerName="istio-proxy"
Sep 18 14:24:11.354499 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:24:11.354383    2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="97394e77-67c5-435f-9457-f9af6436ad5c" containerName="istio-proxy"
Sep 18 14:24:11.358847 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:24:11.358828    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-e9388b-fxb7z"
Sep 18 14:24:11.362738 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:24:11.362542    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-l4fn9\""
Sep 18 14:24:11.365541 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:24:11.365513    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-e9388b-fxb7z"]
Sep 18 14:24:11.380810 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:24:11.380783    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/7df69525-1651-4080-97fc-175505bf40fa-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-b-e9388b-fxb7z\" (UID: \"7df69525-1651-4080-97fc-175505bf40fa\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-e9388b-fxb7z"
Sep 18 14:24:11.482091 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:24:11.482025    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/7df69525-1651-4080-97fc-175505bf40fa-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-b-e9388b-fxb7z\" (UID: \"7df69525-1651-4080-97fc-175505bf40fa\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-e9388b-fxb7z"
Sep 18 14:24:11.482709 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:24:11.482684    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/7df69525-1651-4080-97fc-175505bf40fa-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-b-e9388b-fxb7z\" (UID: \"7df69525-1651-4080-97fc-175505bf40fa\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-e9388b-fxb7z"
Sep 18 14:24:11.672356 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:24:11.672270    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-e9388b-fxb7z"
Sep 18 14:24:11.945496 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:24:11.945469    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-e9388b-fxb7z"]
Sep 18 14:24:11.947404 ip-10-0-129-16 kubenswrapper[2565]: W0918 14:24:11.947378    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7df69525_1651_4080_97fc_175505bf40fa.slice/crio-6b62e5dd5e13ce96c873ab767ece593e229e34b6acbd6f08eba1dc00e9d09b11 WatchSource:0}: Error finding container 6b62e5dd5e13ce96c873ab767ece593e229e34b6acbd6f08eba1dc00e9d09b11: Status 404 returned error can't find the container with id 6b62e5dd5e13ce96c873ab767ece593e229e34b6acbd6f08eba1dc00e9d09b11
Sep 18 14:24:12.810070 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:24:12.810021    2565 generic.go:356] "Generic (PLEG): container finished" podID="7df69525-1651-4080-97fc-175505bf40fa" containerID="e07bacc00c18f34cece8b6310c0d9ed194fd03809c8c8c924585543359e69bfb" exitCode=0
Sep 18 14:24:12.810481 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:24:12.810138    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-e9388b-fxb7z" event={"ID":"7df69525-1651-4080-97fc-175505bf40fa","Type":"ContainerDied","Data":"e07bacc00c18f34cece8b6310c0d9ed194fd03809c8c8c924585543359e69bfb"}
Sep 18 14:24:12.810481 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:24:12.810194    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-e9388b-fxb7z" event={"ID":"7df69525-1651-4080-97fc-175505bf40fa","Type":"ContainerStarted","Data":"6b62e5dd5e13ce96c873ab767ece593e229e34b6acbd6f08eba1dc00e9d09b11"}
Sep 18 14:24:13.955716 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:24:13.955689    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-e9388b-fxb7z"
Sep 18 14:24:14.008703 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:24:14.008664    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/7df69525-1651-4080-97fc-175505bf40fa-maas-api-service-ca\") pod \"7df69525-1651-4080-97fc-175505bf40fa\" (UID: \"7df69525-1651-4080-97fc-175505bf40fa\") "
Sep 18 14:24:14.009071 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:24:14.009014    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7df69525-1651-4080-97fc-175505bf40fa-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "7df69525-1651-4080-97fc-175505bf40fa" (UID: "7df69525-1651-4080-97fc-175505bf40fa"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 18 14:24:14.110357 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:24:14.110267    2565 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/7df69525-1651-4080-97fc-175505bf40fa-maas-api-service-ca\") on node \"ip-10-0-129-16.ec2.internal\" DevicePath \"\""
Sep 18 14:24:14.822439 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:24:14.822408    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-e9388b-fxb7z"
Sep 18 14:24:14.822619 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:24:14.822450    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-e9388b-fxb7z" event={"ID":"7df69525-1651-4080-97fc-175505bf40fa","Type":"ContainerDied","Data":"6b62e5dd5e13ce96c873ab767ece593e229e34b6acbd6f08eba1dc00e9d09b11"}
Sep 18 14:24:14.822619 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:24:14.822498    2565 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="6b62e5dd5e13ce96c873ab767ece593e229e34b6acbd6f08eba1dc00e9d09b11"
Sep 18 14:24:27.045217 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:24:27.045174    2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-b-e9388b-b7c99946c-tgqfb"]
Sep 18 14:24:27.045696 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:24:27.045421    2565 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-api-b-e9388b-b7c99946c-tgqfb" podUID="5bf97fdc-954a-4f6e-95ab-f97750de5462" containerName="payload-processing" containerID="cri-o://7f90a75e2246d4df7b7656a9c909769e6017d7632af247fddbeb0520a2199e0a" gracePeriod=30
Sep 18 14:24:27.051443 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:24:27.051401    2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-b-e9388b-7c99d75d46-jtfhl"]
Sep 18 14:24:27.051669 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:24:27.051626    2565 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-api-b-e9388b-7c99d75d46-jtfhl" podUID="7c6320a2-acdb-41eb-a068-b879b515941e" containerName="payload-pre-processing" containerID="cri-o://aac7ca89afd595650c70b22917b6ebef0ad357f874c6f61697e820d793e308bb" gracePeriod=30
Sep 18 14:24:27.272410 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:24:27.272361    2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-e9388b-fxb7z"]
Sep 18 14:24:27.276054 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:24:27.276003    2565 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-e9388b-fxb7z"]
Sep 18 14:24:27.344050 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:24:27.344002    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-b-e9388b-b7c99946c-tgqfb"
Sep 18 14:24:27.348086 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:24:27.348063    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-b-e9388b-7c99d75d46-jtfhl"
Sep 18 14:24:27.446814 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:24:27.446777    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-lwwgx\" (UniqueName: \"kubernetes.io/projected/5bf97fdc-954a-4f6e-95ab-f97750de5462-kube-api-access-lwwgx\") pod \"5bf97fdc-954a-4f6e-95ab-f97750de5462\" (UID: \"5bf97fdc-954a-4f6e-95ab-f97750de5462\") "
Sep 18 14:24:27.447016 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:24:27.446879    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-pr4qg\" (UniqueName: \"kubernetes.io/projected/7c6320a2-acdb-41eb-a068-b879b515941e-kube-api-access-pr4qg\") pod \"7c6320a2-acdb-41eb-a068-b879b515941e\" (UID: \"7c6320a2-acdb-41eb-a068-b879b515941e\") "
Sep 18 14:24:27.447016 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:24:27.446913    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/7c6320a2-acdb-41eb-a068-b879b515941e-plugins-config-volume\") pod \"7c6320a2-acdb-41eb-a068-b879b515941e\" (UID: \"7c6320a2-acdb-41eb-a068-b879b515941e\") "
Sep 18 14:24:27.447016 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:24:27.446956    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/5bf97fdc-954a-4f6e-95ab-f97750de5462-plugins-config-volume\") pod \"5bf97fdc-954a-4f6e-95ab-f97750de5462\" (UID: \"5bf97fdc-954a-4f6e-95ab-f97750de5462\") "
Sep 18 14:24:27.447407 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:24:27.447378    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7c6320a2-acdb-41eb-a068-b879b515941e-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "7c6320a2-acdb-41eb-a068-b879b515941e" (UID: "7c6320a2-acdb-41eb-a068-b879b515941e"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 18 14:24:27.447407 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:24:27.447391    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5bf97fdc-954a-4f6e-95ab-f97750de5462-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "5bf97fdc-954a-4f6e-95ab-f97750de5462" (UID: "5bf97fdc-954a-4f6e-95ab-f97750de5462"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 18 14:24:27.449213 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:24:27.449187    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7c6320a2-acdb-41eb-a068-b879b515941e-kube-api-access-pr4qg" (OuterVolumeSpecName: "kube-api-access-pr4qg") pod "7c6320a2-acdb-41eb-a068-b879b515941e" (UID: "7c6320a2-acdb-41eb-a068-b879b515941e"). InnerVolumeSpecName "kube-api-access-pr4qg". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 18 14:24:27.449320 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:24:27.449242    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5bf97fdc-954a-4f6e-95ab-f97750de5462-kube-api-access-lwwgx" (OuterVolumeSpecName: "kube-api-access-lwwgx") pod "5bf97fdc-954a-4f6e-95ab-f97750de5462" (UID: "5bf97fdc-954a-4f6e-95ab-f97750de5462"). InnerVolumeSpecName "kube-api-access-lwwgx". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 18 14:24:27.548730 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:24:27.548686    2565 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-lwwgx\" (UniqueName: \"kubernetes.io/projected/5bf97fdc-954a-4f6e-95ab-f97750de5462-kube-api-access-lwwgx\") on node \"ip-10-0-129-16.ec2.internal\" DevicePath \"\""
Sep 18 14:24:27.548730 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:24:27.548727    2565 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-pr4qg\" (UniqueName: \"kubernetes.io/projected/7c6320a2-acdb-41eb-a068-b879b515941e-kube-api-access-pr4qg\") on node \"ip-10-0-129-16.ec2.internal\" DevicePath \"\""
Sep 18 14:24:27.548943 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:24:27.548741    2565 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/7c6320a2-acdb-41eb-a068-b879b515941e-plugins-config-volume\") on node \"ip-10-0-129-16.ec2.internal\" DevicePath \"\""
Sep 18 14:24:27.548943 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:24:27.548753    2565 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/5bf97fdc-954a-4f6e-95ab-f97750de5462-plugins-config-volume\") on node \"ip-10-0-129-16.ec2.internal\" DevicePath \"\""
Sep 18 14:24:27.900495 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:24:27.900459    2565 generic.go:356] "Generic (PLEG): container finished" podID="7c6320a2-acdb-41eb-a068-b879b515941e" containerID="aac7ca89afd595650c70b22917b6ebef0ad357f874c6f61697e820d793e308bb" exitCode=0
Sep 18 14:24:27.900734 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:24:27.900509    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-b-e9388b-7c99d75d46-jtfhl"
Sep 18 14:24:27.900734 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:24:27.900560    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-b-e9388b-7c99d75d46-jtfhl" event={"ID":"7c6320a2-acdb-41eb-a068-b879b515941e","Type":"ContainerDied","Data":"aac7ca89afd595650c70b22917b6ebef0ad357f874c6f61697e820d793e308bb"}
Sep 18 14:24:27.900734 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:24:27.900626    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-b-e9388b-7c99d75d46-jtfhl" event={"ID":"7c6320a2-acdb-41eb-a068-b879b515941e","Type":"ContainerDied","Data":"5cc332234c3fb873f3c11fa624203f464a04fb48217bd124eb7ecc6357f4ef95"}
Sep 18 14:24:27.900734 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:24:27.900655    2565 scope.go:117] "RemoveContainer" containerID="aac7ca89afd595650c70b22917b6ebef0ad357f874c6f61697e820d793e308bb"
Sep 18 14:24:27.902152 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:24:27.902131    2565 generic.go:356] "Generic (PLEG): container finished" podID="5bf97fdc-954a-4f6e-95ab-f97750de5462" containerID="7f90a75e2246d4df7b7656a9c909769e6017d7632af247fddbeb0520a2199e0a" exitCode=0
Sep 18 14:24:27.902260 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:24:27.902184    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-b-e9388b-b7c99946c-tgqfb"
Sep 18 14:24:27.902260 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:24:27.902216    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-b-e9388b-b7c99946c-tgqfb" event={"ID":"5bf97fdc-954a-4f6e-95ab-f97750de5462","Type":"ContainerDied","Data":"7f90a75e2246d4df7b7656a9c909769e6017d7632af247fddbeb0520a2199e0a"}
Sep 18 14:24:27.902388 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:24:27.902257    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-b-e9388b-b7c99946c-tgqfb" event={"ID":"5bf97fdc-954a-4f6e-95ab-f97750de5462","Type":"ContainerDied","Data":"ae5030d49434b65278449c12746fa2de16cd0f9464a3b7712365773e4511d2e5"}
Sep 18 14:24:27.912422 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:24:27.912402    2565 scope.go:117] "RemoveContainer" containerID="aac7ca89afd595650c70b22917b6ebef0ad357f874c6f61697e820d793e308bb"
Sep 18 14:24:27.912892 ip-10-0-129-16 kubenswrapper[2565]: E0918 14:24:27.912695    2565 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"aac7ca89afd595650c70b22917b6ebef0ad357f874c6f61697e820d793e308bb\": container with ID starting with aac7ca89afd595650c70b22917b6ebef0ad357f874c6f61697e820d793e308bb not found: ID does not exist" containerID="aac7ca89afd595650c70b22917b6ebef0ad357f874c6f61697e820d793e308bb"
Sep 18 14:24:27.912892 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:24:27.912718    2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"aac7ca89afd595650c70b22917b6ebef0ad357f874c6f61697e820d793e308bb"} err="failed to get container status \"aac7ca89afd595650c70b22917b6ebef0ad357f874c6f61697e820d793e308bb\": rpc error: code = NotFound desc = could not find container \"aac7ca89afd595650c70b22917b6ebef0ad357f874c6f61697e820d793e308bb\": container with ID starting with aac7ca89afd595650c70b22917b6ebef0ad357f874c6f61697e820d793e308bb not found: ID does not exist"
Sep 18 14:24:27.912892 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:24:27.912738    2565 scope.go:117] "RemoveContainer" containerID="7f90a75e2246d4df7b7656a9c909769e6017d7632af247fddbeb0520a2199e0a"
Sep 18 14:24:27.915382 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:24:27.915359    2565 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7df69525-1651-4080-97fc-175505bf40fa" path="/var/lib/kubelet/pods/7df69525-1651-4080-97fc-175505bf40fa/volumes"
Sep 18 14:24:27.921680 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:24:27.921661    2565 scope.go:117] "RemoveContainer" containerID="7f90a75e2246d4df7b7656a9c909769e6017d7632af247fddbeb0520a2199e0a"
Sep 18 14:24:27.922057 ip-10-0-129-16 kubenswrapper[2565]: E0918 14:24:27.922009    2565 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7f90a75e2246d4df7b7656a9c909769e6017d7632af247fddbeb0520a2199e0a\": container with ID starting with 7f90a75e2246d4df7b7656a9c909769e6017d7632af247fddbeb0520a2199e0a not found: ID does not exist" containerID="7f90a75e2246d4df7b7656a9c909769e6017d7632af247fddbeb0520a2199e0a"
Sep 18 14:24:27.922219 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:24:27.922187    2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7f90a75e2246d4df7b7656a9c909769e6017d7632af247fddbeb0520a2199e0a"} err="failed to get container status \"7f90a75e2246d4df7b7656a9c909769e6017d7632af247fddbeb0520a2199e0a\": rpc error: code = NotFound desc = could not find container \"7f90a75e2246d4df7b7656a9c909769e6017d7632af247fddbeb0520a2199e0a\": container with ID starting with 7f90a75e2246d4df7b7656a9c909769e6017d7632af247fddbeb0520a2199e0a not found: ID does not exist"
Sep 18 14:24:27.924724 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:24:27.924704    2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-b-e9388b-7c99d75d46-jtfhl"]
Sep 18 14:24:27.928926 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:24:27.928903    2565 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-b-e9388b-7c99d75d46-jtfhl"]
Sep 18 14:24:27.948285 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:24:27.948249    2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-b-e9388b-b7c99946c-tgqfb"]
Sep 18 14:24:27.948453 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:24:27.948305    2565 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-b-e9388b-b7c99946c-tgqfb"]
Sep 18 14:24:29.913418 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:24:29.913374    2565 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5bf97fdc-954a-4f6e-95ab-f97750de5462" path="/var/lib/kubelet/pods/5bf97fdc-954a-4f6e-95ab-f97750de5462/volumes"
Sep 18 14:24:29.913918 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:24:29.913824    2565 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7c6320a2-acdb-41eb-a068-b879b515941e" path="/var/lib/kubelet/pods/7c6320a2-acdb-41eb-a068-b879b515941e/volumes"
Sep 18 14:24:34.895009 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:24:34.894974    2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-526b97-s2b6q"]
Sep 18 14:24:34.895491 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:24:34.895476    2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7c6320a2-acdb-41eb-a068-b879b515941e" containerName="payload-pre-processing"
Sep 18 14:24:34.895491 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:24:34.895491    2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="7c6320a2-acdb-41eb-a068-b879b515941e" containerName="payload-pre-processing"
Sep 18 14:24:34.895576 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:24:34.895508    2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5bf97fdc-954a-4f6e-95ab-f97750de5462" containerName="payload-processing"
Sep 18 14:24:34.895576 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:24:34.895515    2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="5bf97fdc-954a-4f6e-95ab-f97750de5462" containerName="payload-processing"
Sep 18 14:24:34.895576 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:24:34.895521    2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7df69525-1651-4080-97fc-175505bf40fa" containerName="revoke-keys"
Sep 18 14:24:34.895576 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:24:34.895526    2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="7df69525-1651-4080-97fc-175505bf40fa" containerName="revoke-keys"
Sep 18 14:24:34.895715 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:24:34.895595    2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="7df69525-1651-4080-97fc-175505bf40fa" containerName="revoke-keys"
Sep 18 14:24:34.895715 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:24:34.895602    2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="5bf97fdc-954a-4f6e-95ab-f97750de5462" containerName="payload-processing"
Sep 18 14:24:34.895715 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:24:34.895609    2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="7c6320a2-acdb-41eb-a068-b879b515941e" containerName="payload-pre-processing"
Sep 18 14:24:34.900394 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:24:34.900378    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-526b97-s2b6q"
Sep 18 14:24:34.903327 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:24:34.903293    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-l4fn9\""
Sep 18 14:24:34.908006 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:24:34.907985    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-526b97-s2b6q"]
Sep 18 14:24:35.037377 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:24:35.037341    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/0231bae0-4ae7-4dd5-90fc-4c283f65d553-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-a-w2-526b97-s2b6q\" (UID: \"0231bae0-4ae7-4dd5-90fc-4c283f65d553\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-526b97-s2b6q"
Sep 18 14:24:35.138172 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:24:35.138138    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/0231bae0-4ae7-4dd5-90fc-4c283f65d553-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-a-w2-526b97-s2b6q\" (UID: \"0231bae0-4ae7-4dd5-90fc-4c283f65d553\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-526b97-s2b6q"
Sep 18 14:24:35.138796 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:24:35.138776    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/0231bae0-4ae7-4dd5-90fc-4c283f65d553-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-a-w2-526b97-s2b6q\" (UID: \"0231bae0-4ae7-4dd5-90fc-4c283f65d553\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-526b97-s2b6q"
Sep 18 14:24:35.213243 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:24:35.213153    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-526b97-s2b6q"
Sep 18 14:24:35.489580 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:24:35.489556    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-526b97-s2b6q"]
Sep 18 14:24:35.949949 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:24:35.949911    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-526b97-s2b6q" event={"ID":"0231bae0-4ae7-4dd5-90fc-4c283f65d553","Type":"ContainerStarted","Data":"6df5bd6777d02b1ec6d83c3f224b9eba5f4475a3f84fd488a8e70d950fe2cc2c"}
Sep 18 14:24:35.950375 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:24:35.949968    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-526b97-s2b6q" event={"ID":"0231bae0-4ae7-4dd5-90fc-4c283f65d553","Type":"ContainerStarted","Data":"09ea44e0e29ea337c31fb2b1ada15910a7a917d5025fa56d2d8e1e03399d5f35"}
Sep 18 14:24:35.968540 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:24:35.968488    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-526b97-s2b6q" podStartSLOduration=1.9684717470000002 podStartE2EDuration="1.968471747s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 14:24:34 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 14:24:35.966456459 +0000 UTC m=+1544.140366143" watchObservedRunningTime="2026-09-18 14:24:35.968471747 +0000 UTC m=+1544.142381406"
Sep 18 14:24:36.956906 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:24:36.956852    2565 generic.go:356] "Generic (PLEG): container finished" podID="0231bae0-4ae7-4dd5-90fc-4c283f65d553" containerID="6df5bd6777d02b1ec6d83c3f224b9eba5f4475a3f84fd488a8e70d950fe2cc2c" exitCode=0
Sep 18 14:24:36.957356 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:24:36.956947    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-526b97-s2b6q" event={"ID":"0231bae0-4ae7-4dd5-90fc-4c283f65d553","Type":"ContainerDied","Data":"6df5bd6777d02b1ec6d83c3f224b9eba5f4475a3f84fd488a8e70d950fe2cc2c"}
Sep 18 14:24:38.107116 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:24:38.107091    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-526b97-s2b6q"
Sep 18 14:24:38.271123 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:24:38.270996    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/0231bae0-4ae7-4dd5-90fc-4c283f65d553-maas-api-service-ca\") pod \"0231bae0-4ae7-4dd5-90fc-4c283f65d553\" (UID: \"0231bae0-4ae7-4dd5-90fc-4c283f65d553\") "
Sep 18 14:24:38.271355 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:24:38.271333    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0231bae0-4ae7-4dd5-90fc-4c283f65d553-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "0231bae0-4ae7-4dd5-90fc-4c283f65d553" (UID: "0231bae0-4ae7-4dd5-90fc-4c283f65d553"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 18 14:24:38.372090 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:24:38.372056    2565 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/0231bae0-4ae7-4dd5-90fc-4c283f65d553-maas-api-service-ca\") on node \"ip-10-0-129-16.ec2.internal\" DevicePath \"\""
Sep 18 14:24:38.970384 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:24:38.970357    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-526b97-s2b6q"
Sep 18 14:24:38.970384 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:24:38.970388    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-526b97-s2b6q" event={"ID":"0231bae0-4ae7-4dd5-90fc-4c283f65d553","Type":"ContainerDied","Data":"09ea44e0e29ea337c31fb2b1ada15910a7a917d5025fa56d2d8e1e03399d5f35"}
Sep 18 14:24:38.970600 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:24:38.970434    2565 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="09ea44e0e29ea337c31fb2b1ada15910a7a917d5025fa56d2d8e1e03399d5f35"
Sep 18 14:24:45.454513 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:24:45.454476    2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w2-526b97-7bf5747d47-r2mpw"]
Sep 18 14:24:45.454943 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:24:45.454700    2565 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-shared-a-w2-526b97-7bf5747d47-r2mpw" podUID="b26d8b6f-f726-4238-b48a-f4d272cf27b7" containerName="payload-processing" containerID="cri-o://d147a971b357f08a64c04396d98987d95900c19625ccf423f1cf2833987ecaac" gracePeriod=30
Sep 18 14:24:45.466147 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:24:45.466122    2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w2-526b97-7486448b6b-hnk46"]
Sep 18 14:24:45.466344 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:24:45.466326    2565 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-526b97-7486448b6b-hnk46" podUID="8d339523-5739-49e1-aecf-93d3d545c43f" containerName="payload-pre-processing" containerID="cri-o://6f8543f8bde4c233cef31da064ea695ac4a00bfe1092d8789d23e91f4cdc70bf" gracePeriod=30
Sep 18 14:24:45.702902 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:24:45.702868    2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-526b97-s2b6q"]
Sep 18 14:24:45.709379 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:24:45.709308    2565 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-526b97-s2b6q"]
Sep 18 14:24:45.746796 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:24:45.746775    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-a-w2-526b97-7bf5747d47-r2mpw"
Sep 18 14:24:45.749818 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:24:45.749801    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-526b97-7486448b6b-hnk46"
Sep 18 14:24:45.842902 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:24:45.842863    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/8d339523-5739-49e1-aecf-93d3d545c43f-plugins-config-volume\") pod \"8d339523-5739-49e1-aecf-93d3d545c43f\" (UID: \"8d339523-5739-49e1-aecf-93d3d545c43f\") "
Sep 18 14:24:45.843119 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:24:45.842932    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-b5bkh\" (UniqueName: \"kubernetes.io/projected/b26d8b6f-f726-4238-b48a-f4d272cf27b7-kube-api-access-b5bkh\") pod \"b26d8b6f-f726-4238-b48a-f4d272cf27b7\" (UID: \"b26d8b6f-f726-4238-b48a-f4d272cf27b7\") "
Sep 18 14:24:45.843119 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:24:45.842956    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-6kwzc\" (UniqueName: \"kubernetes.io/projected/8d339523-5739-49e1-aecf-93d3d545c43f-kube-api-access-6kwzc\") pod \"8d339523-5739-49e1-aecf-93d3d545c43f\" (UID: \"8d339523-5739-49e1-aecf-93d3d545c43f\") "
Sep 18 14:24:45.843119 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:24:45.842995    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b26d8b6f-f726-4238-b48a-f4d272cf27b7-plugins-config-volume\") pod \"b26d8b6f-f726-4238-b48a-f4d272cf27b7\" (UID: \"b26d8b6f-f726-4238-b48a-f4d272cf27b7\") "
Sep 18 14:24:45.843430 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:24:45.843399    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8d339523-5739-49e1-aecf-93d3d545c43f-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "8d339523-5739-49e1-aecf-93d3d545c43f" (UID: "8d339523-5739-49e1-aecf-93d3d545c43f"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 18 14:24:45.843566 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:24:45.843425    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b26d8b6f-f726-4238-b48a-f4d272cf27b7-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "b26d8b6f-f726-4238-b48a-f4d272cf27b7" (UID: "b26d8b6f-f726-4238-b48a-f4d272cf27b7"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 18 14:24:45.845289 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:24:45.845265    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8d339523-5739-49e1-aecf-93d3d545c43f-kube-api-access-6kwzc" (OuterVolumeSpecName: "kube-api-access-6kwzc") pod "8d339523-5739-49e1-aecf-93d3d545c43f" (UID: "8d339523-5739-49e1-aecf-93d3d545c43f"). InnerVolumeSpecName "kube-api-access-6kwzc". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 18 14:24:45.845379 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:24:45.845287    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b26d8b6f-f726-4238-b48a-f4d272cf27b7-kube-api-access-b5bkh" (OuterVolumeSpecName: "kube-api-access-b5bkh") pod "b26d8b6f-f726-4238-b48a-f4d272cf27b7" (UID: "b26d8b6f-f726-4238-b48a-f4d272cf27b7"). InnerVolumeSpecName "kube-api-access-b5bkh". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 18 14:24:45.920454 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:24:45.920419    2565 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0231bae0-4ae7-4dd5-90fc-4c283f65d553" path="/var/lib/kubelet/pods/0231bae0-4ae7-4dd5-90fc-4c283f65d553/volumes"
Sep 18 14:24:45.944443 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:24:45.944414    2565 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b26d8b6f-f726-4238-b48a-f4d272cf27b7-plugins-config-volume\") on node \"ip-10-0-129-16.ec2.internal\" DevicePath \"\""
Sep 18 14:24:45.944595 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:24:45.944447    2565 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/8d339523-5739-49e1-aecf-93d3d545c43f-plugins-config-volume\") on node \"ip-10-0-129-16.ec2.internal\" DevicePath \"\""
Sep 18 14:24:45.944595 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:24:45.944465    2565 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-b5bkh\" (UniqueName: \"kubernetes.io/projected/b26d8b6f-f726-4238-b48a-f4d272cf27b7-kube-api-access-b5bkh\") on node \"ip-10-0-129-16.ec2.internal\" DevicePath \"\""
Sep 18 14:24:45.944595 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:24:45.944479    2565 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6kwzc\" (UniqueName: \"kubernetes.io/projected/8d339523-5739-49e1-aecf-93d3d545c43f-kube-api-access-6kwzc\") on node \"ip-10-0-129-16.ec2.internal\" DevicePath \"\""
Sep 18 14:24:46.014006 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:24:46.013909    2565 generic.go:356] "Generic (PLEG): container finished" podID="b26d8b6f-f726-4238-b48a-f4d272cf27b7" containerID="d147a971b357f08a64c04396d98987d95900c19625ccf423f1cf2833987ecaac" exitCode=0
Sep 18 14:24:46.014006 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:24:46.013969    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-a-w2-526b97-7bf5747d47-r2mpw"
Sep 18 14:24:46.014006 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:24:46.013990    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-a-w2-526b97-7bf5747d47-r2mpw" event={"ID":"b26d8b6f-f726-4238-b48a-f4d272cf27b7","Type":"ContainerDied","Data":"d147a971b357f08a64c04396d98987d95900c19625ccf423f1cf2833987ecaac"}
Sep 18 14:24:46.014312 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:24:46.014077    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-a-w2-526b97-7bf5747d47-r2mpw" event={"ID":"b26d8b6f-f726-4238-b48a-f4d272cf27b7","Type":"ContainerDied","Data":"1b683c4819ec89997fac58a1b26bb4716521cc2ad1bb3e75a7f05cab35ae54f1"}
Sep 18 14:24:46.014312 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:24:46.014093    2565 scope.go:117] "RemoveContainer" containerID="d147a971b357f08a64c04396d98987d95900c19625ccf423f1cf2833987ecaac"
Sep 18 14:24:46.016987 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:24:46.016461    2565 generic.go:356] "Generic (PLEG): container finished" podID="8d339523-5739-49e1-aecf-93d3d545c43f" containerID="6f8543f8bde4c233cef31da064ea695ac4a00bfe1092d8789d23e91f4cdc70bf" exitCode=0
Sep 18 14:24:46.016987 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:24:46.016523    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-526b97-7486448b6b-hnk46" event={"ID":"8d339523-5739-49e1-aecf-93d3d545c43f","Type":"ContainerDied","Data":"6f8543f8bde4c233cef31da064ea695ac4a00bfe1092d8789d23e91f4cdc70bf"}
Sep 18 14:24:46.016987 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:24:46.016542    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-526b97-7486448b6b-hnk46"
Sep 18 14:24:46.016987 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:24:46.016564    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-526b97-7486448b6b-hnk46" event={"ID":"8d339523-5739-49e1-aecf-93d3d545c43f","Type":"ContainerDied","Data":"f1d784d3f3e068ba7cab108370ee59e527636f18c7fe13c3a4599eb7dcec56df"}
Sep 18 14:24:46.025469 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:24:46.025449    2565 scope.go:117] "RemoveContainer" containerID="d147a971b357f08a64c04396d98987d95900c19625ccf423f1cf2833987ecaac"
Sep 18 14:24:46.025803 ip-10-0-129-16 kubenswrapper[2565]: E0918 14:24:46.025786    2565 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d147a971b357f08a64c04396d98987d95900c19625ccf423f1cf2833987ecaac\": container with ID starting with d147a971b357f08a64c04396d98987d95900c19625ccf423f1cf2833987ecaac not found: ID does not exist" containerID="d147a971b357f08a64c04396d98987d95900c19625ccf423f1cf2833987ecaac"
Sep 18 14:24:46.025857 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:24:46.025811    2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d147a971b357f08a64c04396d98987d95900c19625ccf423f1cf2833987ecaac"} err="failed to get container status \"d147a971b357f08a64c04396d98987d95900c19625ccf423f1cf2833987ecaac\": rpc error: code = NotFound desc = could not find container \"d147a971b357f08a64c04396d98987d95900c19625ccf423f1cf2833987ecaac\": container with ID starting with d147a971b357f08a64c04396d98987d95900c19625ccf423f1cf2833987ecaac not found: ID does not exist"
Sep 18 14:24:46.025857 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:24:46.025829    2565 scope.go:117] "RemoveContainer" containerID="6f8543f8bde4c233cef31da064ea695ac4a00bfe1092d8789d23e91f4cdc70bf"
Sep 18 14:24:46.034429 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:24:46.034414    2565 scope.go:117] "RemoveContainer" containerID="6f8543f8bde4c233cef31da064ea695ac4a00bfe1092d8789d23e91f4cdc70bf"
Sep 18 14:24:46.034696 ip-10-0-129-16 kubenswrapper[2565]: E0918 14:24:46.034677    2565 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6f8543f8bde4c233cef31da064ea695ac4a00bfe1092d8789d23e91f4cdc70bf\": container with ID starting with 6f8543f8bde4c233cef31da064ea695ac4a00bfe1092d8789d23e91f4cdc70bf not found: ID does not exist" containerID="6f8543f8bde4c233cef31da064ea695ac4a00bfe1092d8789d23e91f4cdc70bf"
Sep 18 14:24:46.034767 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:24:46.034701    2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6f8543f8bde4c233cef31da064ea695ac4a00bfe1092d8789d23e91f4cdc70bf"} err="failed to get container status \"6f8543f8bde4c233cef31da064ea695ac4a00bfe1092d8789d23e91f4cdc70bf\": rpc error: code = NotFound desc = could not find container \"6f8543f8bde4c233cef31da064ea695ac4a00bfe1092d8789d23e91f4cdc70bf\": container with ID starting with 6f8543f8bde4c233cef31da064ea695ac4a00bfe1092d8789d23e91f4cdc70bf not found: ID does not exist"
Sep 18 14:24:46.056723 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:24:46.056694    2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w2-526b97-7bf5747d47-r2mpw"]
Sep 18 14:24:46.074239 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:24:46.074211    2565 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w2-526b97-7bf5747d47-r2mpw"]
Sep 18 14:24:46.103282 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:24:46.103254    2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w2-526b97-7486448b6b-hnk46"]
Sep 18 14:24:46.123750 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:24:46.123725    2565 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w2-526b97-7486448b6b-hnk46"]
Sep 18 14:24:47.913877 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:24:47.913844    2565 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8d339523-5739-49e1-aecf-93d3d545c43f" path="/var/lib/kubelet/pods/8d339523-5739-49e1-aecf-93d3d545c43f/volumes"
Sep 18 14:24:47.914373 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:24:47.914240    2565 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b26d8b6f-f726-4238-b48a-f4d272cf27b7" path="/var/lib/kubelet/pods/b26d8b6f-f726-4238-b48a-f4d272cf27b7/volumes"
Sep 18 14:24:52.689926 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:24:52.689887    2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-216d39-8fnh5"]
Sep 18 14:24:52.690354 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:24:52.690340    2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b26d8b6f-f726-4238-b48a-f4d272cf27b7" containerName="payload-processing"
Sep 18 14:24:52.690395 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:24:52.690355    2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="b26d8b6f-f726-4238-b48a-f4d272cf27b7" containerName="payload-processing"
Sep 18 14:24:52.690395 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:24:52.690366    2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0231bae0-4ae7-4dd5-90fc-4c283f65d553" containerName="revoke-keys"
Sep 18 14:24:52.690395 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:24:52.690372    2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="0231bae0-4ae7-4dd5-90fc-4c283f65d553" containerName="revoke-keys"
Sep 18 14:24:52.690395 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:24:52.690392    2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8d339523-5739-49e1-aecf-93d3d545c43f" containerName="payload-pre-processing"
Sep 18 14:24:52.690533 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:24:52.690399    2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="8d339523-5739-49e1-aecf-93d3d545c43f" containerName="payload-pre-processing"
Sep 18 14:24:52.690533 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:24:52.690464    2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="0231bae0-4ae7-4dd5-90fc-4c283f65d553" containerName="revoke-keys"
Sep 18 14:24:52.690533 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:24:52.690472    2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="b26d8b6f-f726-4238-b48a-f4d272cf27b7" containerName="payload-processing"
Sep 18 14:24:52.690533 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:24:52.690479    2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="8d339523-5739-49e1-aecf-93d3d545c43f" containerName="payload-pre-processing"
Sep 18 14:24:52.695301 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:24:52.695287    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-216d39-8fnh5"
Sep 18 14:24:52.698445 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:24:52.698423    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-l4fn9\""
Sep 18 14:24:52.702629 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:24:52.702602    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-216d39-8fnh5"]
Sep 18 14:24:52.803907 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:24:52.803871    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/475cae79-81ba-45a6-af46-048f588e2a29-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-b-w2-216d39-8fnh5\" (UID: \"475cae79-81ba-45a6-af46-048f588e2a29\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-216d39-8fnh5"
Sep 18 14:24:52.905356 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:24:52.905317    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/475cae79-81ba-45a6-af46-048f588e2a29-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-b-w2-216d39-8fnh5\" (UID: \"475cae79-81ba-45a6-af46-048f588e2a29\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-216d39-8fnh5"
Sep 18 14:24:52.905950 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:24:52.905928    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/475cae79-81ba-45a6-af46-048f588e2a29-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-b-w2-216d39-8fnh5\" (UID: \"475cae79-81ba-45a6-af46-048f588e2a29\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-216d39-8fnh5"
Sep 18 14:24:53.007461 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:24:53.007378    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-216d39-8fnh5"
Sep 18 14:24:53.288277 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:24:53.288250    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-216d39-8fnh5"]
Sep 18 14:24:53.290340 ip-10-0-129-16 kubenswrapper[2565]: W0918 14:24:53.290307    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod475cae79_81ba_45a6_af46_048f588e2a29.slice/crio-034408b0e33e576e20c5c34f408edc1a42342bab8daf85b809a0b69734246f26 WatchSource:0}: Error finding container 034408b0e33e576e20c5c34f408edc1a42342bab8daf85b809a0b69734246f26: Status 404 returned error can't find the container with id 034408b0e33e576e20c5c34f408edc1a42342bab8daf85b809a0b69734246f26
Sep 18 14:24:54.070962 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:24:54.070934    2565 generic.go:356] "Generic (PLEG): container finished" podID="475cae79-81ba-45a6-af46-048f588e2a29" containerID="9d090b2ee3c36b33747eac2c7d93a3911a09870c1a9135968955f53f81925097" exitCode=0
Sep 18 14:24:54.071305 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:24:54.071050    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-216d39-8fnh5" event={"ID":"475cae79-81ba-45a6-af46-048f588e2a29","Type":"ContainerDied","Data":"9d090b2ee3c36b33747eac2c7d93a3911a09870c1a9135968955f53f81925097"}
Sep 18 14:24:54.071305 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:24:54.071101    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-216d39-8fnh5" event={"ID":"475cae79-81ba-45a6-af46-048f588e2a29","Type":"ContainerStarted","Data":"034408b0e33e576e20c5c34f408edc1a42342bab8daf85b809a0b69734246f26"}
Sep 18 14:24:55.219834 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:24:55.219803    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-216d39-8fnh5"
Sep 18 14:24:55.331471 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:24:55.331372    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/475cae79-81ba-45a6-af46-048f588e2a29-maas-api-service-ca\") pod \"475cae79-81ba-45a6-af46-048f588e2a29\" (UID: \"475cae79-81ba-45a6-af46-048f588e2a29\") "
Sep 18 14:24:55.331744 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:24:55.331720    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/475cae79-81ba-45a6-af46-048f588e2a29-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "475cae79-81ba-45a6-af46-048f588e2a29" (UID: "475cae79-81ba-45a6-af46-048f588e2a29"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 18 14:24:55.432416 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:24:55.432383    2565 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/475cae79-81ba-45a6-af46-048f588e2a29-maas-api-service-ca\") on node \"ip-10-0-129-16.ec2.internal\" DevicePath \"\""
Sep 18 14:24:56.084262 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:24:56.084233    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-216d39-8fnh5"
Sep 18 14:24:56.084428 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:24:56.084233    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-216d39-8fnh5" event={"ID":"475cae79-81ba-45a6-af46-048f588e2a29","Type":"ContainerDied","Data":"034408b0e33e576e20c5c34f408edc1a42342bab8daf85b809a0b69734246f26"}
Sep 18 14:24:56.084428 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:24:56.084365    2565 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="034408b0e33e576e20c5c34f408edc1a42342bab8daf85b809a0b69734246f26"
Sep 18 14:25:03.299430 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:25:03.299385    2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w2-216d39-59959cc765-wkjlv"]
Sep 18 14:25:03.300145 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:25:03.299671    2565 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-shared-b-w2-216d39-59959cc765-wkjlv" podUID="b61ae7b1-71cd-45e0-b2a7-9f840216947e" containerName="payload-processing" containerID="cri-o://d7adde985defa0b3ee4b32e4affc9f8e0f39a759a6a67a1ace340d38c55cb9e7" gracePeriod=30
Sep 18 14:25:03.308633 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:25:03.308601    2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w2-216d39-6d4d494cd-f622c"]
Sep 18 14:25:03.308883 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:25:03.308840    2565 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-216d39-6d4d494cd-f622c" podUID="8eee79ba-0e22-4f42-aac9-0a35b93230f0" containerName="payload-pre-processing" containerID="cri-o://04e2342a3f4c88dd5a77447636974a180d243621658ab7f2f95c2a9a68be43ca" gracePeriod=30
Sep 18 14:25:03.349746 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:25:03.349703    2565 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-216d39-6d4d494cd-f622c" podUID="8eee79ba-0e22-4f42-aac9-0a35b93230f0" containerName="payload-pre-processing" probeResult="failure" output="dial tcp 10.134.0.74:9004: connect: connection refused"
Sep 18 14:25:03.349910 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:25:03.349827    2565 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-processing-e2e-shared-b-w2-216d39-59959cc765-wkjlv" podUID="b61ae7b1-71cd-45e0-b2a7-9f840216947e" containerName="payload-processing" probeResult="failure" output="dial tcp 10.134.0.75:9004: connect: connection refused"
Sep 18 14:25:03.488948 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:25:03.488886    2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-216d39-8fnh5"]
Sep 18 14:25:03.492261 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:25:03.492224    2565 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-216d39-8fnh5"]
Sep 18 14:25:03.567923 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:25:03.567901    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-b-w2-216d39-59959cc765-wkjlv"
Sep 18 14:25:03.596394 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:25:03.596362    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-216d39-6d4d494cd-f622c"
Sep 18 14:25:03.617312 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:25:03.617278    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-mb6tc\" (UniqueName: \"kubernetes.io/projected/b61ae7b1-71cd-45e0-b2a7-9f840216947e-kube-api-access-mb6tc\") pod \"b61ae7b1-71cd-45e0-b2a7-9f840216947e\" (UID: \"b61ae7b1-71cd-45e0-b2a7-9f840216947e\") "
Sep 18 14:25:03.617515 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:25:03.617377    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b61ae7b1-71cd-45e0-b2a7-9f840216947e-plugins-config-volume\") pod \"b61ae7b1-71cd-45e0-b2a7-9f840216947e\" (UID: \"b61ae7b1-71cd-45e0-b2a7-9f840216947e\") "
Sep 18 14:25:03.617774 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:25:03.617738    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b61ae7b1-71cd-45e0-b2a7-9f840216947e-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "b61ae7b1-71cd-45e0-b2a7-9f840216947e" (UID: "b61ae7b1-71cd-45e0-b2a7-9f840216947e"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 18 14:25:03.619490 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:25:03.619462    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b61ae7b1-71cd-45e0-b2a7-9f840216947e-kube-api-access-mb6tc" (OuterVolumeSpecName: "kube-api-access-mb6tc") pod "b61ae7b1-71cd-45e0-b2a7-9f840216947e" (UID: "b61ae7b1-71cd-45e0-b2a7-9f840216947e"). InnerVolumeSpecName "kube-api-access-mb6tc". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 18 14:25:03.718086 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:25:03.718012    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-h5j5d\" (UniqueName: \"kubernetes.io/projected/8eee79ba-0e22-4f42-aac9-0a35b93230f0-kube-api-access-h5j5d\") pod \"8eee79ba-0e22-4f42-aac9-0a35b93230f0\" (UID: \"8eee79ba-0e22-4f42-aac9-0a35b93230f0\") "
Sep 18 14:25:03.718086 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:25:03.718096    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/8eee79ba-0e22-4f42-aac9-0a35b93230f0-plugins-config-volume\") pod \"8eee79ba-0e22-4f42-aac9-0a35b93230f0\" (UID: \"8eee79ba-0e22-4f42-aac9-0a35b93230f0\") "
Sep 18 14:25:03.718467 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:25:03.718448    2565 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b61ae7b1-71cd-45e0-b2a7-9f840216947e-plugins-config-volume\") on node \"ip-10-0-129-16.ec2.internal\" DevicePath \"\""
Sep 18 14:25:03.718542 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:25:03.718478    2565 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mb6tc\" (UniqueName: \"kubernetes.io/projected/b61ae7b1-71cd-45e0-b2a7-9f840216947e-kube-api-access-mb6tc\") on node \"ip-10-0-129-16.ec2.internal\" DevicePath \"\""
Sep 18 14:25:03.718542 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:25:03.718445    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8eee79ba-0e22-4f42-aac9-0a35b93230f0-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "8eee79ba-0e22-4f42-aac9-0a35b93230f0" (UID: "8eee79ba-0e22-4f42-aac9-0a35b93230f0"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 18 14:25:03.720118 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:25:03.720096    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8eee79ba-0e22-4f42-aac9-0a35b93230f0-kube-api-access-h5j5d" (OuterVolumeSpecName: "kube-api-access-h5j5d") pod "8eee79ba-0e22-4f42-aac9-0a35b93230f0" (UID: "8eee79ba-0e22-4f42-aac9-0a35b93230f0"). InnerVolumeSpecName "kube-api-access-h5j5d". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 18 14:25:03.819589 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:25:03.819494    2565 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-h5j5d\" (UniqueName: \"kubernetes.io/projected/8eee79ba-0e22-4f42-aac9-0a35b93230f0-kube-api-access-h5j5d\") on node \"ip-10-0-129-16.ec2.internal\" DevicePath \"\""
Sep 18 14:25:03.819589 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:25:03.819524    2565 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/8eee79ba-0e22-4f42-aac9-0a35b93230f0-plugins-config-volume\") on node \"ip-10-0-129-16.ec2.internal\" DevicePath \"\""
Sep 18 14:25:03.913146 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:25:03.913111    2565 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="475cae79-81ba-45a6-af46-048f588e2a29" path="/var/lib/kubelet/pods/475cae79-81ba-45a6-af46-048f588e2a29/volumes"
Sep 18 14:25:04.129354 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:25:04.129258    2565 generic.go:356] "Generic (PLEG): container finished" podID="8eee79ba-0e22-4f42-aac9-0a35b93230f0" containerID="04e2342a3f4c88dd5a77447636974a180d243621658ab7f2f95c2a9a68be43ca" exitCode=0
Sep 18 14:25:04.129354 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:25:04.129314    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-216d39-6d4d494cd-f622c"
Sep 18 14:25:04.129354 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:25:04.129340    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-216d39-6d4d494cd-f622c" event={"ID":"8eee79ba-0e22-4f42-aac9-0a35b93230f0","Type":"ContainerDied","Data":"04e2342a3f4c88dd5a77447636974a180d243621658ab7f2f95c2a9a68be43ca"}
Sep 18 14:25:04.129648 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:25:04.129394    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-216d39-6d4d494cd-f622c" event={"ID":"8eee79ba-0e22-4f42-aac9-0a35b93230f0","Type":"ContainerDied","Data":"85705b38ee662e6657815802af334e01cdffe33af85cedc87a1796281bb862f6"}
Sep 18 14:25:04.129648 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:25:04.129424    2565 scope.go:117] "RemoveContainer" containerID="04e2342a3f4c88dd5a77447636974a180d243621658ab7f2f95c2a9a68be43ca"
Sep 18 14:25:04.130722 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:25:04.130700    2565 generic.go:356] "Generic (PLEG): container finished" podID="b61ae7b1-71cd-45e0-b2a7-9f840216947e" containerID="d7adde985defa0b3ee4b32e4affc9f8e0f39a759a6a67a1ace340d38c55cb9e7" exitCode=0
Sep 18 14:25:04.130817 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:25:04.130740    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-b-w2-216d39-59959cc765-wkjlv" event={"ID":"b61ae7b1-71cd-45e0-b2a7-9f840216947e","Type":"ContainerDied","Data":"d7adde985defa0b3ee4b32e4affc9f8e0f39a759a6a67a1ace340d38c55cb9e7"}
Sep 18 14:25:04.130817 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:25:04.130758    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-b-w2-216d39-59959cc765-wkjlv"
Sep 18 14:25:04.130817 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:25:04.130797    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-b-w2-216d39-59959cc765-wkjlv" event={"ID":"b61ae7b1-71cd-45e0-b2a7-9f840216947e","Type":"ContainerDied","Data":"fcbeaef794172c1d306780f416809ac833da5a60d72437219de5a83141c138fe"}
Sep 18 14:25:04.141677 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:25:04.141651    2565 scope.go:117] "RemoveContainer" containerID="04e2342a3f4c88dd5a77447636974a180d243621658ab7f2f95c2a9a68be43ca"
Sep 18 14:25:04.141982 ip-10-0-129-16 kubenswrapper[2565]: E0918 14:25:04.141962    2565 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"04e2342a3f4c88dd5a77447636974a180d243621658ab7f2f95c2a9a68be43ca\": container with ID starting with 04e2342a3f4c88dd5a77447636974a180d243621658ab7f2f95c2a9a68be43ca not found: ID does not exist" containerID="04e2342a3f4c88dd5a77447636974a180d243621658ab7f2f95c2a9a68be43ca"
Sep 18 14:25:04.142056 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:25:04.141990    2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"04e2342a3f4c88dd5a77447636974a180d243621658ab7f2f95c2a9a68be43ca"} err="failed to get container status \"04e2342a3f4c88dd5a77447636974a180d243621658ab7f2f95c2a9a68be43ca\": rpc error: code = NotFound desc = could not find container \"04e2342a3f4c88dd5a77447636974a180d243621658ab7f2f95c2a9a68be43ca\": container with ID starting with 04e2342a3f4c88dd5a77447636974a180d243621658ab7f2f95c2a9a68be43ca not found: ID does not exist"
Sep 18 14:25:04.142056 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:25:04.142006    2565 scope.go:117] "RemoveContainer" containerID="d7adde985defa0b3ee4b32e4affc9f8e0f39a759a6a67a1ace340d38c55cb9e7"
Sep 18 14:25:04.150768 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:25:04.150721    2565 scope.go:117] "RemoveContainer" containerID="d7adde985defa0b3ee4b32e4affc9f8e0f39a759a6a67a1ace340d38c55cb9e7"
Sep 18 14:25:04.151113 ip-10-0-129-16 kubenswrapper[2565]: E0918 14:25:04.151087    2565 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d7adde985defa0b3ee4b32e4affc9f8e0f39a759a6a67a1ace340d38c55cb9e7\": container with ID starting with d7adde985defa0b3ee4b32e4affc9f8e0f39a759a6a67a1ace340d38c55cb9e7 not found: ID does not exist" containerID="d7adde985defa0b3ee4b32e4affc9f8e0f39a759a6a67a1ace340d38c55cb9e7"
Sep 18 14:25:04.151205 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:25:04.151121    2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d7adde985defa0b3ee4b32e4affc9f8e0f39a759a6a67a1ace340d38c55cb9e7"} err="failed to get container status \"d7adde985defa0b3ee4b32e4affc9f8e0f39a759a6a67a1ace340d38c55cb9e7\": rpc error: code = NotFound desc = could not find container \"d7adde985defa0b3ee4b32e4affc9f8e0f39a759a6a67a1ace340d38c55cb9e7\": container with ID starting with d7adde985defa0b3ee4b32e4affc9f8e0f39a759a6a67a1ace340d38c55cb9e7 not found: ID does not exist"
Sep 18 14:25:04.153191 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:25:04.153170    2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w2-216d39-6d4d494cd-f622c"]
Sep 18 14:25:04.163102 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:25:04.163078    2565 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w2-216d39-6d4d494cd-f622c"]
Sep 18 14:25:04.182509 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:25:04.182484    2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w2-216d39-59959cc765-wkjlv"]
Sep 18 14:25:04.186630 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:25:04.186605    2565 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w2-216d39-59959cc765-wkjlv"]
Sep 18 14:25:05.913271 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:25:05.913236    2565 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8eee79ba-0e22-4f42-aac9-0a35b93230f0" path="/var/lib/kubelet/pods/8eee79ba-0e22-4f42-aac9-0a35b93230f0/volumes"
Sep 18 14:25:05.913722 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:25:05.913638    2565 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b61ae7b1-71cd-45e0-b2a7-9f840216947e" path="/var/lib/kubelet/pods/b61ae7b1-71cd-45e0-b2a7-9f840216947e/volumes"
Sep 18 14:25:50.227341 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:25:50.227269    2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-b-w3-980661/sub-test-model-980661-kserve-b6bc57dc5-wjrlk"]
Sep 18 14:25:50.227804 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:25:50.227712    2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="475cae79-81ba-45a6-af46-048f588e2a29" containerName="revoke-keys"
Sep 18 14:25:50.227804 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:25:50.227723    2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="475cae79-81ba-45a6-af46-048f588e2a29" containerName="revoke-keys"
Sep 18 14:25:50.227804 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:25:50.227746    2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b61ae7b1-71cd-45e0-b2a7-9f840216947e" containerName="payload-processing"
Sep 18 14:25:50.227804 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:25:50.227752    2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="b61ae7b1-71cd-45e0-b2a7-9f840216947e" containerName="payload-processing"
Sep 18 14:25:50.227804 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:25:50.227766    2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8eee79ba-0e22-4f42-aac9-0a35b93230f0" containerName="payload-pre-processing"
Sep 18 14:25:50.227804 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:25:50.227772    2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="8eee79ba-0e22-4f42-aac9-0a35b93230f0" containerName="payload-pre-processing"
Sep 18 14:25:50.228022 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:25:50.227832    2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="475cae79-81ba-45a6-af46-048f588e2a29" containerName="revoke-keys"
Sep 18 14:25:50.228022 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:25:50.227841    2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="b61ae7b1-71cd-45e0-b2a7-9f840216947e" containerName="payload-processing"
Sep 18 14:25:50.228022 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:25:50.227849    2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="8eee79ba-0e22-4f42-aac9-0a35b93230f0" containerName="payload-pre-processing"
Sep 18 14:25:50.231000 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:25:50.230984    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-w3-980661/sub-test-model-980661-kserve-b6bc57dc5-wjrlk"
Sep 18 14:25:50.233864 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:25:50.233835    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-w3-980661\"/\"kube-root-ca.crt\""
Sep 18 14:25:50.234002 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:25:50.233915    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-w3-980661\"/\"default-dockercfg-qvxhp\""
Sep 18 14:25:50.234002 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:25:50.233940    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-w3-980661\"/\"openshift-service-ca.crt\""
Sep 18 14:25:50.235228 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:25:50.235214    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-w3-980661\"/\"sub-test-model-980661-kserve-self-signed-certs\""
Sep 18 14:25:50.240569 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:25:50.240538    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-w3-980661/sub-test-model-980661-kserve-b6bc57dc5-wjrlk"]
Sep 18 14:25:50.385476 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:25:50.385439    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/66aee6e1-b470-4071-ae28-f35168294bbe-model-cache\") pod \"sub-test-model-980661-kserve-b6bc57dc5-wjrlk\" (UID: \"66aee6e1-b470-4071-ae28-f35168294bbe\") " pod="ai-tenant-e2e-shared-b-w3-980661/sub-test-model-980661-kserve-b6bc57dc5-wjrlk"
Sep 18 14:25:50.385651 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:25:50.385481    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/66aee6e1-b470-4071-ae28-f35168294bbe-tls-certs\") pod \"sub-test-model-980661-kserve-b6bc57dc5-wjrlk\" (UID: \"66aee6e1-b470-4071-ae28-f35168294bbe\") " pod="ai-tenant-e2e-shared-b-w3-980661/sub-test-model-980661-kserve-b6bc57dc5-wjrlk"
Sep 18 14:25:50.385651 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:25:50.385527    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/66aee6e1-b470-4071-ae28-f35168294bbe-tmp-dir\") pod \"sub-test-model-980661-kserve-b6bc57dc5-wjrlk\" (UID: \"66aee6e1-b470-4071-ae28-f35168294bbe\") " pod="ai-tenant-e2e-shared-b-w3-980661/sub-test-model-980661-kserve-b6bc57dc5-wjrlk"
Sep 18 14:25:50.385651 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:25:50.385597    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dgzgv\" (UniqueName: \"kubernetes.io/projected/66aee6e1-b470-4071-ae28-f35168294bbe-kube-api-access-dgzgv\") pod \"sub-test-model-980661-kserve-b6bc57dc5-wjrlk\" (UID: \"66aee6e1-b470-4071-ae28-f35168294bbe\") " pod="ai-tenant-e2e-shared-b-w3-980661/sub-test-model-980661-kserve-b6bc57dc5-wjrlk"
Sep 18 14:25:50.385792 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:25:50.385779    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/66aee6e1-b470-4071-ae28-f35168294bbe-home\") pod \"sub-test-model-980661-kserve-b6bc57dc5-wjrlk\" (UID: \"66aee6e1-b470-4071-ae28-f35168294bbe\") " pod="ai-tenant-e2e-shared-b-w3-980661/sub-test-model-980661-kserve-b6bc57dc5-wjrlk"
Sep 18 14:25:50.385845 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:25:50.385827    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/66aee6e1-b470-4071-ae28-f35168294bbe-dshm\") pod \"sub-test-model-980661-kserve-b6bc57dc5-wjrlk\" (UID: \"66aee6e1-b470-4071-ae28-f35168294bbe\") " pod="ai-tenant-e2e-shared-b-w3-980661/sub-test-model-980661-kserve-b6bc57dc5-wjrlk"
Sep 18 14:25:50.487139 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:25:50.487006    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/66aee6e1-b470-4071-ae28-f35168294bbe-model-cache\") pod \"sub-test-model-980661-kserve-b6bc57dc5-wjrlk\" (UID: \"66aee6e1-b470-4071-ae28-f35168294bbe\") " pod="ai-tenant-e2e-shared-b-w3-980661/sub-test-model-980661-kserve-b6bc57dc5-wjrlk"
Sep 18 14:25:50.487314 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:25:50.487132    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/66aee6e1-b470-4071-ae28-f35168294bbe-tls-certs\") pod \"sub-test-model-980661-kserve-b6bc57dc5-wjrlk\" (UID: \"66aee6e1-b470-4071-ae28-f35168294bbe\") " pod="ai-tenant-e2e-shared-b-w3-980661/sub-test-model-980661-kserve-b6bc57dc5-wjrlk"
Sep 18 14:25:50.487314 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:25:50.487195    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/66aee6e1-b470-4071-ae28-f35168294bbe-tmp-dir\") pod \"sub-test-model-980661-kserve-b6bc57dc5-wjrlk\" (UID: \"66aee6e1-b470-4071-ae28-f35168294bbe\") " pod="ai-tenant-e2e-shared-b-w3-980661/sub-test-model-980661-kserve-b6bc57dc5-wjrlk"
Sep 18 14:25:50.487314 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:25:50.487235    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dgzgv\" (UniqueName: \"kubernetes.io/projected/66aee6e1-b470-4071-ae28-f35168294bbe-kube-api-access-dgzgv\") pod \"sub-test-model-980661-kserve-b6bc57dc5-wjrlk\" (UID: \"66aee6e1-b470-4071-ae28-f35168294bbe\") " pod="ai-tenant-e2e-shared-b-w3-980661/sub-test-model-980661-kserve-b6bc57dc5-wjrlk"
Sep 18 14:25:50.487488 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:25:50.487327    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/66aee6e1-b470-4071-ae28-f35168294bbe-home\") pod \"sub-test-model-980661-kserve-b6bc57dc5-wjrlk\" (UID: \"66aee6e1-b470-4071-ae28-f35168294bbe\") " pod="ai-tenant-e2e-shared-b-w3-980661/sub-test-model-980661-kserve-b6bc57dc5-wjrlk"
Sep 18 14:25:50.487488 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:25:50.487372    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/66aee6e1-b470-4071-ae28-f35168294bbe-dshm\") pod \"sub-test-model-980661-kserve-b6bc57dc5-wjrlk\" (UID: \"66aee6e1-b470-4071-ae28-f35168294bbe\") " pod="ai-tenant-e2e-shared-b-w3-980661/sub-test-model-980661-kserve-b6bc57dc5-wjrlk"
Sep 18 14:25:50.487488 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:25:50.487384    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/66aee6e1-b470-4071-ae28-f35168294bbe-model-cache\") pod \"sub-test-model-980661-kserve-b6bc57dc5-wjrlk\" (UID: \"66aee6e1-b470-4071-ae28-f35168294bbe\") " pod="ai-tenant-e2e-shared-b-w3-980661/sub-test-model-980661-kserve-b6bc57dc5-wjrlk"
Sep 18 14:25:50.487651 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:25:50.487579    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/66aee6e1-b470-4071-ae28-f35168294bbe-tmp-dir\") pod \"sub-test-model-980661-kserve-b6bc57dc5-wjrlk\" (UID: \"66aee6e1-b470-4071-ae28-f35168294bbe\") " pod="ai-tenant-e2e-shared-b-w3-980661/sub-test-model-980661-kserve-b6bc57dc5-wjrlk"
Sep 18 14:25:50.487708 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:25:50.487680    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/66aee6e1-b470-4071-ae28-f35168294bbe-home\") pod \"sub-test-model-980661-kserve-b6bc57dc5-wjrlk\" (UID: \"66aee6e1-b470-4071-ae28-f35168294bbe\") " pod="ai-tenant-e2e-shared-b-w3-980661/sub-test-model-980661-kserve-b6bc57dc5-wjrlk"
Sep 18 14:25:50.489528 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:25:50.489496    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/66aee6e1-b470-4071-ae28-f35168294bbe-dshm\") pod \"sub-test-model-980661-kserve-b6bc57dc5-wjrlk\" (UID: \"66aee6e1-b470-4071-ae28-f35168294bbe\") " pod="ai-tenant-e2e-shared-b-w3-980661/sub-test-model-980661-kserve-b6bc57dc5-wjrlk"
Sep 18 14:25:50.489642 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:25:50.489559    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/66aee6e1-b470-4071-ae28-f35168294bbe-tls-certs\") pod \"sub-test-model-980661-kserve-b6bc57dc5-wjrlk\" (UID: \"66aee6e1-b470-4071-ae28-f35168294bbe\") " pod="ai-tenant-e2e-shared-b-w3-980661/sub-test-model-980661-kserve-b6bc57dc5-wjrlk"
Sep 18 14:25:50.495845 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:25:50.495827    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dgzgv\" (UniqueName: \"kubernetes.io/projected/66aee6e1-b470-4071-ae28-f35168294bbe-kube-api-access-dgzgv\") pod \"sub-test-model-980661-kserve-b6bc57dc5-wjrlk\" (UID: \"66aee6e1-b470-4071-ae28-f35168294bbe\") " pod="ai-tenant-e2e-shared-b-w3-980661/sub-test-model-980661-kserve-b6bc57dc5-wjrlk"
Sep 18 14:25:50.541971 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:25:50.541936    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-w3-980661/sub-test-model-980661-kserve-b6bc57dc5-wjrlk"
Sep 18 14:25:50.815668 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:25:50.815639    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-w3-980661/sub-test-model-980661-kserve-b6bc57dc5-wjrlk"]
Sep 18 14:25:50.817252 ip-10-0-129-16 kubenswrapper[2565]: W0918 14:25:50.817225    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod66aee6e1_b470_4071_ae28_f35168294bbe.slice/crio-a623dd80d0050ff60e4ca5a1dc03aa8106bb940c06f22a9c64871a18cb4e99cd WatchSource:0}: Error finding container a623dd80d0050ff60e4ca5a1dc03aa8106bb940c06f22a9c64871a18cb4e99cd: Status 404 returned error can't find the container with id a623dd80d0050ff60e4ca5a1dc03aa8106bb940c06f22a9c64871a18cb4e99cd
Sep 18 14:25:51.396770 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:25:51.396734    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-w3-980661/sub-test-model-980661-kserve-b6bc57dc5-wjrlk" event={"ID":"66aee6e1-b470-4071-ae28-f35168294bbe","Type":"ContainerStarted","Data":"ef2e9a07cae1122e154c2e0badbca6481dd591675aef71e8234572b3d4d3d942"}
Sep 18 14:25:51.396770 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:25:51.396774    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-w3-980661/sub-test-model-980661-kserve-b6bc57dc5-wjrlk" event={"ID":"66aee6e1-b470-4071-ae28-f35168294bbe","Type":"ContainerStarted","Data":"a623dd80d0050ff60e4ca5a1dc03aa8106bb940c06f22a9c64871a18cb4e99cd"}
Sep 18 14:25:51.413368 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:25:51.413304    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-b-w3-980661/sub-test-model-980661-kserve-b6bc57dc5-wjrlk" podStartSLOduration=1.413287247 podStartE2EDuration="1.413287247s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 14:25:50 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 14:25:51.413279237 +0000 UTC m=+1619.587189113" watchObservedRunningTime="2026-09-18 14:25:51.413287247 +0000 UTC m=+1619.587196910"
Sep 18 14:26:00.542551 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:26:00.542501    2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-b-w3-980661/sub-test-model-980661-kserve-b6bc57dc5-wjrlk"
Sep 18 14:26:00.542551 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:26:00.542550    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-b-w3-980661/sub-test-model-980661-kserve-b6bc57dc5-wjrlk"
Sep 18 14:26:00.555144 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:26:00.555117    2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-b-w3-980661/sub-test-model-980661-kserve-b6bc57dc5-wjrlk"
Sep 18 14:26:01.463439 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:26:01.463409    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-b-w3-980661/sub-test-model-980661-kserve-b6bc57dc5-wjrlk"
Sep 18 14:26:32.804869 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:26:32.804835    2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-w3-980661/sub-test-model-980661-kserve-b6bc57dc5-wjrlk"]
Sep 18 14:26:47.824703 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:26:47.824658    2565 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-b-w3-980661/sub-test-model-980661-kserve-b6bc57dc5-wjrlk" podUID="66aee6e1-b470-4071-ae28-f35168294bbe" containerName="main" containerID="cri-o://ef2e9a07cae1122e154c2e0badbca6481dd591675aef71e8234572b3d4d3d942" gracePeriod=45
Sep 18 14:26:48.068423 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:26:48.068395    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-w3-980661/sub-test-model-980661-kserve-b6bc57dc5-wjrlk"
Sep 18 14:26:48.176471 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:26:48.176389    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/66aee6e1-b470-4071-ae28-f35168294bbe-tmp-dir\") pod \"66aee6e1-b470-4071-ae28-f35168294bbe\" (UID: \"66aee6e1-b470-4071-ae28-f35168294bbe\") "
Sep 18 14:26:48.176471 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:26:48.176435    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/66aee6e1-b470-4071-ae28-f35168294bbe-tls-certs\") pod \"66aee6e1-b470-4071-ae28-f35168294bbe\" (UID: \"66aee6e1-b470-4071-ae28-f35168294bbe\") "
Sep 18 14:26:48.176724 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:26:48.176493    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/66aee6e1-b470-4071-ae28-f35168294bbe-model-cache\") pod \"66aee6e1-b470-4071-ae28-f35168294bbe\" (UID: \"66aee6e1-b470-4071-ae28-f35168294bbe\") "
Sep 18 14:26:48.176724 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:26:48.176520    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/66aee6e1-b470-4071-ae28-f35168294bbe-dshm\") pod \"66aee6e1-b470-4071-ae28-f35168294bbe\" (UID: \"66aee6e1-b470-4071-ae28-f35168294bbe\") "
Sep 18 14:26:48.176724 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:26:48.176574    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-dgzgv\" (UniqueName: \"kubernetes.io/projected/66aee6e1-b470-4071-ae28-f35168294bbe-kube-api-access-dgzgv\") pod \"66aee6e1-b470-4071-ae28-f35168294bbe\" (UID: \"66aee6e1-b470-4071-ae28-f35168294bbe\") "
Sep 18 14:26:48.176724 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:26:48.176644    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/66aee6e1-b470-4071-ae28-f35168294bbe-home\") pod \"66aee6e1-b470-4071-ae28-f35168294bbe\" (UID: \"66aee6e1-b470-4071-ae28-f35168294bbe\") "
Sep 18 14:26:48.176934 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:26:48.176724    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/66aee6e1-b470-4071-ae28-f35168294bbe-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "66aee6e1-b470-4071-ae28-f35168294bbe" (UID: "66aee6e1-b470-4071-ae28-f35168294bbe"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 18 14:26:48.176934 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:26:48.176852    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/66aee6e1-b470-4071-ae28-f35168294bbe-model-cache" (OuterVolumeSpecName: "model-cache") pod "66aee6e1-b470-4071-ae28-f35168294bbe" (UID: "66aee6e1-b470-4071-ae28-f35168294bbe"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 18 14:26:48.177048 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:26:48.177000    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/66aee6e1-b470-4071-ae28-f35168294bbe-home" (OuterVolumeSpecName: "home") pod "66aee6e1-b470-4071-ae28-f35168294bbe" (UID: "66aee6e1-b470-4071-ae28-f35168294bbe"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 18 14:26:48.177188 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:26:48.177169    2565 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/66aee6e1-b470-4071-ae28-f35168294bbe-tmp-dir\") on node \"ip-10-0-129-16.ec2.internal\" DevicePath \"\""
Sep 18 14:26:48.177260 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:26:48.177192    2565 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/66aee6e1-b470-4071-ae28-f35168294bbe-model-cache\") on node \"ip-10-0-129-16.ec2.internal\" DevicePath \"\""
Sep 18 14:26:48.177260 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:26:48.177206    2565 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/66aee6e1-b470-4071-ae28-f35168294bbe-home\") on node \"ip-10-0-129-16.ec2.internal\" DevicePath \"\""
Sep 18 14:26:48.179062 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:26:48.179015    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/66aee6e1-b470-4071-ae28-f35168294bbe-dshm" (OuterVolumeSpecName: "dshm") pod "66aee6e1-b470-4071-ae28-f35168294bbe" (UID: "66aee6e1-b470-4071-ae28-f35168294bbe"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 18 14:26:48.179174 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:26:48.179077    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/66aee6e1-b470-4071-ae28-f35168294bbe-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "66aee6e1-b470-4071-ae28-f35168294bbe" (UID: "66aee6e1-b470-4071-ae28-f35168294bbe"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 18 14:26:48.179174 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:26:48.179115    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/66aee6e1-b470-4071-ae28-f35168294bbe-kube-api-access-dgzgv" (OuterVolumeSpecName: "kube-api-access-dgzgv") pod "66aee6e1-b470-4071-ae28-f35168294bbe" (UID: "66aee6e1-b470-4071-ae28-f35168294bbe"). InnerVolumeSpecName "kube-api-access-dgzgv". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 18 14:26:48.278395 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:26:48.278360    2565 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/66aee6e1-b470-4071-ae28-f35168294bbe-tls-certs\") on node \"ip-10-0-129-16.ec2.internal\" DevicePath \"\""
Sep 18 14:26:48.278395 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:26:48.278390    2565 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/66aee6e1-b470-4071-ae28-f35168294bbe-dshm\") on node \"ip-10-0-129-16.ec2.internal\" DevicePath \"\""
Sep 18 14:26:48.278395 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:26:48.278399    2565 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-dgzgv\" (UniqueName: \"kubernetes.io/projected/66aee6e1-b470-4071-ae28-f35168294bbe-kube-api-access-dgzgv\") on node \"ip-10-0-129-16.ec2.internal\" DevicePath \"\""
Sep 18 14:26:48.722010 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:26:48.721973    2565 generic.go:356] "Generic (PLEG): container finished" podID="66aee6e1-b470-4071-ae28-f35168294bbe" containerID="ef2e9a07cae1122e154c2e0badbca6481dd591675aef71e8234572b3d4d3d942" exitCode=0
Sep 18 14:26:48.722247 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:26:48.722019    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-w3-980661/sub-test-model-980661-kserve-b6bc57dc5-wjrlk"
Sep 18 14:26:48.722247 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:26:48.722068    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-w3-980661/sub-test-model-980661-kserve-b6bc57dc5-wjrlk" event={"ID":"66aee6e1-b470-4071-ae28-f35168294bbe","Type":"ContainerDied","Data":"ef2e9a07cae1122e154c2e0badbca6481dd591675aef71e8234572b3d4d3d942"}
Sep 18 14:26:48.722247 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:26:48.722121    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-w3-980661/sub-test-model-980661-kserve-b6bc57dc5-wjrlk" event={"ID":"66aee6e1-b470-4071-ae28-f35168294bbe","Type":"ContainerDied","Data":"a623dd80d0050ff60e4ca5a1dc03aa8106bb940c06f22a9c64871a18cb4e99cd"}
Sep 18 14:26:48.722247 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:26:48.722145    2565 scope.go:117] "RemoveContainer" containerID="ef2e9a07cae1122e154c2e0badbca6481dd591675aef71e8234572b3d4d3d942"
Sep 18 14:26:48.733547 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:26:48.733529    2565 scope.go:117] "RemoveContainer" containerID="ef2e9a07cae1122e154c2e0badbca6481dd591675aef71e8234572b3d4d3d942"
Sep 18 14:26:48.733875 ip-10-0-129-16 kubenswrapper[2565]: E0918 14:26:48.733854    2565 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ef2e9a07cae1122e154c2e0badbca6481dd591675aef71e8234572b3d4d3d942\": container with ID starting with ef2e9a07cae1122e154c2e0badbca6481dd591675aef71e8234572b3d4d3d942 not found: ID does not exist" containerID="ef2e9a07cae1122e154c2e0badbca6481dd591675aef71e8234572b3d4d3d942"
Sep 18 14:26:48.733924 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:26:48.733885    2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ef2e9a07cae1122e154c2e0badbca6481dd591675aef71e8234572b3d4d3d942"} err="failed to get container status \"ef2e9a07cae1122e154c2e0badbca6481dd591675aef71e8234572b3d4d3d942\": rpc error: code = NotFound desc = could not find container \"ef2e9a07cae1122e154c2e0badbca6481dd591675aef71e8234572b3d4d3d942\": container with ID starting with ef2e9a07cae1122e154c2e0badbca6481dd591675aef71e8234572b3d4d3d942 not found: ID does not exist"
Sep 18 14:26:48.748962 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:26:48.748926    2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-w3-980661/sub-test-model-980661-kserve-b6bc57dc5-wjrlk"]
Sep 18 14:26:48.751703 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:26:48.751677    2565 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-b-w3-980661/sub-test-model-980661-kserve-b6bc57dc5-wjrlk"]
Sep 18 14:26:49.918843 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:26:49.918805    2565 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="66aee6e1-b470-4071-ae28-f35168294bbe" path="/var/lib/kubelet/pods/66aee6e1-b470-4071-ae28-f35168294bbe/volumes"
Sep 18 14:26:54.324179 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:26:54.324141    2565 scope.go:117] "RemoveContainer" containerID="efaf61e2c52990d8d882764d86a6498c2dacbf9ab77bbf1befe5b370dee1b9ef"
Sep 18 14:27:48.048129 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:27:48.048090    2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ipp-a-3ab8a5-openshift-default-6d6b5486cd-vbv4x"]
Sep 18 14:27:48.048981 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:27:48.048550    2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="66aee6e1-b470-4071-ae28-f35168294bbe" containerName="main"
Sep 18 14:27:48.048981 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:27:48.048562    2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="66aee6e1-b470-4071-ae28-f35168294bbe" containerName="main"
Sep 18 14:27:48.048981 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:27:48.048647    2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="66aee6e1-b470-4071-ae28-f35168294bbe" containerName="main"
Sep 18 14:27:48.051966 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:27:48.051934    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-a-3ab8a5-openshift-default-6d6b5486cd-vbv4x"
Sep 18 14:27:48.055160 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:27:48.055132    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ipp-a-3ab8a5-openshift-default-dockercfg-9pdss\""
Sep 18 14:27:48.070557 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:27:48.070526    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ipp-a-3ab8a5-openshift-default-6d6b5486cd-vbv4x"]
Sep 18 14:27:48.085779 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:27:48.085743    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/fc7144e0-5c5f-4457-97ac-7dbc3c2ae54d-istio-token\") pod \"e2e-ipp-a-3ab8a5-openshift-default-6d6b5486cd-vbv4x\" (UID: \"fc7144e0-5c5f-4457-97ac-7dbc3c2ae54d\") " pod="openshift-ingress/e2e-ipp-a-3ab8a5-openshift-default-6d6b5486cd-vbv4x"
Sep 18 14:27:48.085948 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:27:48.085811    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/fc7144e0-5c5f-4457-97ac-7dbc3c2ae54d-istiod-ca-cert\") pod \"e2e-ipp-a-3ab8a5-openshift-default-6d6b5486cd-vbv4x\" (UID: \"fc7144e0-5c5f-4457-97ac-7dbc3c2ae54d\") " pod="openshift-ingress/e2e-ipp-a-3ab8a5-openshift-default-6d6b5486cd-vbv4x"
Sep 18 14:27:48.085948 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:27:48.085846    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/fc7144e0-5c5f-4457-97ac-7dbc3c2ae54d-credential-socket\") pod \"e2e-ipp-a-3ab8a5-openshift-default-6d6b5486cd-vbv4x\" (UID: \"fc7144e0-5c5f-4457-97ac-7dbc3c2ae54d\") " pod="openshift-ingress/e2e-ipp-a-3ab8a5-openshift-default-6d6b5486cd-vbv4x"
Sep 18 14:27:48.086115 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:27:48.085947    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/fc7144e0-5c5f-4457-97ac-7dbc3c2ae54d-workload-socket\") pod \"e2e-ipp-a-3ab8a5-openshift-default-6d6b5486cd-vbv4x\" (UID: \"fc7144e0-5c5f-4457-97ac-7dbc3c2ae54d\") " pod="openshift-ingress/e2e-ipp-a-3ab8a5-openshift-default-6d6b5486cd-vbv4x"
Sep 18 14:27:48.086115 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:27:48.085973    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gcrh6\" (UniqueName: \"kubernetes.io/projected/fc7144e0-5c5f-4457-97ac-7dbc3c2ae54d-kube-api-access-gcrh6\") pod \"e2e-ipp-a-3ab8a5-openshift-default-6d6b5486cd-vbv4x\" (UID: \"fc7144e0-5c5f-4457-97ac-7dbc3c2ae54d\") " pod="openshift-ingress/e2e-ipp-a-3ab8a5-openshift-default-6d6b5486cd-vbv4x"
Sep 18 14:27:48.086115 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:27:48.086054    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/fc7144e0-5c5f-4457-97ac-7dbc3c2ae54d-workload-certs\") pod \"e2e-ipp-a-3ab8a5-openshift-default-6d6b5486cd-vbv4x\" (UID: \"fc7144e0-5c5f-4457-97ac-7dbc3c2ae54d\") " pod="openshift-ingress/e2e-ipp-a-3ab8a5-openshift-default-6d6b5486cd-vbv4x"
Sep 18 14:27:48.086115 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:27:48.086091    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/fc7144e0-5c5f-4457-97ac-7dbc3c2ae54d-istio-data\") pod \"e2e-ipp-a-3ab8a5-openshift-default-6d6b5486cd-vbv4x\" (UID: \"fc7144e0-5c5f-4457-97ac-7dbc3c2ae54d\") " pod="openshift-ingress/e2e-ipp-a-3ab8a5-openshift-default-6d6b5486cd-vbv4x"
Sep 18 14:27:48.086387 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:27:48.086116    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/fc7144e0-5c5f-4457-97ac-7dbc3c2ae54d-istio-podinfo\") pod \"e2e-ipp-a-3ab8a5-openshift-default-6d6b5486cd-vbv4x\" (UID: \"fc7144e0-5c5f-4457-97ac-7dbc3c2ae54d\") " pod="openshift-ingress/e2e-ipp-a-3ab8a5-openshift-default-6d6b5486cd-vbv4x"
Sep 18 14:27:48.086387 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:27:48.086161    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/fc7144e0-5c5f-4457-97ac-7dbc3c2ae54d-istio-envoy\") pod \"e2e-ipp-a-3ab8a5-openshift-default-6d6b5486cd-vbv4x\" (UID: \"fc7144e0-5c5f-4457-97ac-7dbc3c2ae54d\") " pod="openshift-ingress/e2e-ipp-a-3ab8a5-openshift-default-6d6b5486cd-vbv4x"
Sep 18 14:27:48.187003 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:27:48.186972    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/fc7144e0-5c5f-4457-97ac-7dbc3c2ae54d-workload-socket\") pod \"e2e-ipp-a-3ab8a5-openshift-default-6d6b5486cd-vbv4x\" (UID: \"fc7144e0-5c5f-4457-97ac-7dbc3c2ae54d\") " pod="openshift-ingress/e2e-ipp-a-3ab8a5-openshift-default-6d6b5486cd-vbv4x"
Sep 18 14:27:48.187003 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:27:48.187008    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gcrh6\" (UniqueName: \"kubernetes.io/projected/fc7144e0-5c5f-4457-97ac-7dbc3c2ae54d-kube-api-access-gcrh6\") pod \"e2e-ipp-a-3ab8a5-openshift-default-6d6b5486cd-vbv4x\" (UID: \"fc7144e0-5c5f-4457-97ac-7dbc3c2ae54d\") " pod="openshift-ingress/e2e-ipp-a-3ab8a5-openshift-default-6d6b5486cd-vbv4x"
Sep 18 14:27:48.187268 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:27:48.187048    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/fc7144e0-5c5f-4457-97ac-7dbc3c2ae54d-workload-certs\") pod \"e2e-ipp-a-3ab8a5-openshift-default-6d6b5486cd-vbv4x\" (UID: \"fc7144e0-5c5f-4457-97ac-7dbc3c2ae54d\") " pod="openshift-ingress/e2e-ipp-a-3ab8a5-openshift-default-6d6b5486cd-vbv4x"
Sep 18 14:27:48.187268 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:27:48.187075    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/fc7144e0-5c5f-4457-97ac-7dbc3c2ae54d-istio-data\") pod \"e2e-ipp-a-3ab8a5-openshift-default-6d6b5486cd-vbv4x\" (UID: \"fc7144e0-5c5f-4457-97ac-7dbc3c2ae54d\") " pod="openshift-ingress/e2e-ipp-a-3ab8a5-openshift-default-6d6b5486cd-vbv4x"
Sep 18 14:27:48.187373 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:27:48.187344    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/fc7144e0-5c5f-4457-97ac-7dbc3c2ae54d-istio-podinfo\") pod \"e2e-ipp-a-3ab8a5-openshift-default-6d6b5486cd-vbv4x\" (UID: \"fc7144e0-5c5f-4457-97ac-7dbc3c2ae54d\") " pod="openshift-ingress/e2e-ipp-a-3ab8a5-openshift-default-6d6b5486cd-vbv4x"
Sep 18 14:27:48.187435 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:27:48.187417    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/fc7144e0-5c5f-4457-97ac-7dbc3c2ae54d-istio-envoy\") pod \"e2e-ipp-a-3ab8a5-openshift-default-6d6b5486cd-vbv4x\" (UID: \"fc7144e0-5c5f-4457-97ac-7dbc3c2ae54d\") " pod="openshift-ingress/e2e-ipp-a-3ab8a5-openshift-default-6d6b5486cd-vbv4x"
Sep 18 14:27:48.187490 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:27:48.187451    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/fc7144e0-5c5f-4457-97ac-7dbc3c2ae54d-workload-certs\") pod \"e2e-ipp-a-3ab8a5-openshift-default-6d6b5486cd-vbv4x\" (UID: \"fc7144e0-5c5f-4457-97ac-7dbc3c2ae54d\") " pod="openshift-ingress/e2e-ipp-a-3ab8a5-openshift-default-6d6b5486cd-vbv4x"
Sep 18 14:27:48.187490 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:27:48.187425    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/fc7144e0-5c5f-4457-97ac-7dbc3c2ae54d-workload-socket\") pod \"e2e-ipp-a-3ab8a5-openshift-default-6d6b5486cd-vbv4x\" (UID: \"fc7144e0-5c5f-4457-97ac-7dbc3c2ae54d\") " pod="openshift-ingress/e2e-ipp-a-3ab8a5-openshift-default-6d6b5486cd-vbv4x"
Sep 18 14:27:48.187577 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:27:48.187484    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/fc7144e0-5c5f-4457-97ac-7dbc3c2ae54d-istio-data\") pod \"e2e-ipp-a-3ab8a5-openshift-default-6d6b5486cd-vbv4x\" (UID: \"fc7144e0-5c5f-4457-97ac-7dbc3c2ae54d\") " pod="openshift-ingress/e2e-ipp-a-3ab8a5-openshift-default-6d6b5486cd-vbv4x"
Sep 18 14:27:48.187577 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:27:48.187511    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/fc7144e0-5c5f-4457-97ac-7dbc3c2ae54d-istio-token\") pod \"e2e-ipp-a-3ab8a5-openshift-default-6d6b5486cd-vbv4x\" (UID: \"fc7144e0-5c5f-4457-97ac-7dbc3c2ae54d\") " pod="openshift-ingress/e2e-ipp-a-3ab8a5-openshift-default-6d6b5486cd-vbv4x"
Sep 18 14:27:48.187577 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:27:48.187559    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/fc7144e0-5c5f-4457-97ac-7dbc3c2ae54d-istiod-ca-cert\") pod \"e2e-ipp-a-3ab8a5-openshift-default-6d6b5486cd-vbv4x\" (UID: \"fc7144e0-5c5f-4457-97ac-7dbc3c2ae54d\") " pod="openshift-ingress/e2e-ipp-a-3ab8a5-openshift-default-6d6b5486cd-vbv4x"
Sep 18 14:27:48.187698 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:27:48.187681    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/fc7144e0-5c5f-4457-97ac-7dbc3c2ae54d-credential-socket\") pod \"e2e-ipp-a-3ab8a5-openshift-default-6d6b5486cd-vbv4x\" (UID: \"fc7144e0-5c5f-4457-97ac-7dbc3c2ae54d\") " pod="openshift-ingress/e2e-ipp-a-3ab8a5-openshift-default-6d6b5486cd-vbv4x"
Sep 18 14:27:48.188054 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:27:48.188004    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/fc7144e0-5c5f-4457-97ac-7dbc3c2ae54d-credential-socket\") pod \"e2e-ipp-a-3ab8a5-openshift-default-6d6b5486cd-vbv4x\" (UID: \"fc7144e0-5c5f-4457-97ac-7dbc3c2ae54d\") " pod="openshift-ingress/e2e-ipp-a-3ab8a5-openshift-default-6d6b5486cd-vbv4x"
Sep 18 14:27:48.188188 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:27:48.188082    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/fc7144e0-5c5f-4457-97ac-7dbc3c2ae54d-istiod-ca-cert\") pod \"e2e-ipp-a-3ab8a5-openshift-default-6d6b5486cd-vbv4x\" (UID: \"fc7144e0-5c5f-4457-97ac-7dbc3c2ae54d\") " pod="openshift-ingress/e2e-ipp-a-3ab8a5-openshift-default-6d6b5486cd-vbv4x"
Sep 18 14:27:48.189721 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:27:48.189699    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/fc7144e0-5c5f-4457-97ac-7dbc3c2ae54d-istio-envoy\") pod \"e2e-ipp-a-3ab8a5-openshift-default-6d6b5486cd-vbv4x\" (UID: \"fc7144e0-5c5f-4457-97ac-7dbc3c2ae54d\") " pod="openshift-ingress/e2e-ipp-a-3ab8a5-openshift-default-6d6b5486cd-vbv4x"
Sep 18 14:27:48.189836 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:27:48.189776    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/fc7144e0-5c5f-4457-97ac-7dbc3c2ae54d-istio-podinfo\") pod \"e2e-ipp-a-3ab8a5-openshift-default-6d6b5486cd-vbv4x\" (UID: \"fc7144e0-5c5f-4457-97ac-7dbc3c2ae54d\") " pod="openshift-ingress/e2e-ipp-a-3ab8a5-openshift-default-6d6b5486cd-vbv4x"
Sep 18 14:27:48.196195 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:27:48.196163    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gcrh6\" (UniqueName: \"kubernetes.io/projected/fc7144e0-5c5f-4457-97ac-7dbc3c2ae54d-kube-api-access-gcrh6\") pod \"e2e-ipp-a-3ab8a5-openshift-default-6d6b5486cd-vbv4x\" (UID: \"fc7144e0-5c5f-4457-97ac-7dbc3c2ae54d\") " pod="openshift-ingress/e2e-ipp-a-3ab8a5-openshift-default-6d6b5486cd-vbv4x"
Sep 18 14:27:48.196744 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:27:48.196727    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/fc7144e0-5c5f-4457-97ac-7dbc3c2ae54d-istio-token\") pod \"e2e-ipp-a-3ab8a5-openshift-default-6d6b5486cd-vbv4x\" (UID: \"fc7144e0-5c5f-4457-97ac-7dbc3c2ae54d\") " pod="openshift-ingress/e2e-ipp-a-3ab8a5-openshift-default-6d6b5486cd-vbv4x"
Sep 18 14:27:48.362700 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:27:48.362614    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-a-3ab8a5-openshift-default-6d6b5486cd-vbv4x"
Sep 18 14:27:48.641844 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:27:48.641816    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ipp-a-3ab8a5-openshift-default-6d6b5486cd-vbv4x"]
Sep 18 14:27:48.643815 ip-10-0-129-16 kubenswrapper[2565]: W0918 14:27:48.643787    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfc7144e0_5c5f_4457_97ac_7dbc3c2ae54d.slice/crio-8859f67407976c5f5c9285d6844bc6c3f064b7fa93683fd76f878769d7c2f53d WatchSource:0}: Error finding container 8859f67407976c5f5c9285d6844bc6c3f064b7fa93683fd76f878769d7c2f53d: Status 404 returned error can't find the container with id 8859f67407976c5f5c9285d6844bc6c3f064b7fa93683fd76f878769d7c2f53d
Sep 18 14:27:48.646105 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:27:48.646075    2565 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 18 14:27:48.646191 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:27:48.646140    2565 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 18 14:27:48.646191 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:27:48.646166    2565 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 18 14:27:49.064676 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:27:49.064642    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-a-3ab8a5-openshift-default-6d6b5486cd-vbv4x" event={"ID":"fc7144e0-5c5f-4457-97ac-7dbc3c2ae54d","Type":"ContainerStarted","Data":"d2583c5faa5be01fe42560fb39aa54676f29bad44f7c98018189f2766284688b"}
Sep 18 14:27:49.065099 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:27:49.064703    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-a-3ab8a5-openshift-default-6d6b5486cd-vbv4x" event={"ID":"fc7144e0-5c5f-4457-97ac-7dbc3c2ae54d","Type":"ContainerStarted","Data":"8859f67407976c5f5c9285d6844bc6c3f064b7fa93683fd76f878769d7c2f53d"}
Sep 18 14:27:49.105114 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:27:49.105068    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-ipp-a-3ab8a5-openshift-default-6d6b5486cd-vbv4x" podStartSLOduration=1.105050276 podStartE2EDuration="1.105050276s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 14:27:48 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 14:27:49.103999455 +0000 UTC m=+1737.277909136" watchObservedRunningTime="2026-09-18 14:27:49.105050276 +0000 UTC m=+1737.278959936"
Sep 18 14:27:49.363200 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:27:49.363102    2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ipp-a-3ab8a5-openshift-default-6d6b5486cd-vbv4x"
Sep 18 14:27:49.368355 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:27:49.368327    2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ipp-a-3ab8a5-openshift-default-6d6b5486cd-vbv4x"
Sep 18 14:27:50.070310 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:27:50.070277    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ipp-a-3ab8a5-openshift-default-6d6b5486cd-vbv4x"
Sep 18 14:27:50.071211 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:27:50.071192    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ipp-a-3ab8a5-openshift-default-6d6b5486cd-vbv4x"
Sep 18 14:27:53.616437 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:27:53.616405    2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-a-3ab8a5-656fccc874-gq8hx"]
Sep 18 14:27:53.620204 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:27:53.620181    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-3ab8a5-656fccc874-gq8hx"
Sep 18 14:27:53.622717 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:27:53.622698    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-ipp-a-3ab8a5-dockercfg-npq2j\""
Sep 18 14:27:53.622818 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:27:53.622765    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-ipp-a-3ab8a5\""
Sep 18 14:27:53.627790 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:27:53.627765    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-a-3ab8a5-656fccc874-gq8hx"]
Sep 18 14:27:53.657574 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:27:53.657545    2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-a-3ab8a5-88fccfcf7-96nsp"]
Sep 18 14:27:53.661250 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:27:53.661233    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-a-3ab8a5-88fccfcf7-96nsp"
Sep 18 14:27:53.670988 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:27:53.670951    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-a-3ab8a5-88fccfcf7-96nsp"]
Sep 18 14:27:53.745690 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:27:53.745657    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-szgh6\" (UniqueName: \"kubernetes.io/projected/84a68a1d-c393-4b06-8125-a9b70370eb55-kube-api-access-szgh6\") pod \"payload-pre-processing-e2e-ipp-a-3ab8a5-656fccc874-gq8hx\" (UID: \"84a68a1d-c393-4b06-8125-a9b70370eb55\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-3ab8a5-656fccc874-gq8hx"
Sep 18 14:27:53.745851 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:27:53.745703    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/84a68a1d-c393-4b06-8125-a9b70370eb55-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-a-3ab8a5-656fccc874-gq8hx\" (UID: \"84a68a1d-c393-4b06-8125-a9b70370eb55\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-3ab8a5-656fccc874-gq8hx"
Sep 18 14:27:53.745851 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:27:53.745783    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/68e15645-97e4-48fd-913a-7d060eb1d7f1-plugins-config-volume\") pod \"payload-processing-e2e-ipp-a-3ab8a5-88fccfcf7-96nsp\" (UID: \"68e15645-97e4-48fd-913a-7d060eb1d7f1\") " pod="openshift-ingress/payload-processing-e2e-ipp-a-3ab8a5-88fccfcf7-96nsp"
Sep 18 14:27:53.745851 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:27:53.745833    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-v5l4z\" (UniqueName: \"kubernetes.io/projected/68e15645-97e4-48fd-913a-7d060eb1d7f1-kube-api-access-v5l4z\") pod \"payload-processing-e2e-ipp-a-3ab8a5-88fccfcf7-96nsp\" (UID: \"68e15645-97e4-48fd-913a-7d060eb1d7f1\") " pod="openshift-ingress/payload-processing-e2e-ipp-a-3ab8a5-88fccfcf7-96nsp"
Sep 18 14:27:53.846511 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:27:53.846472    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/84a68a1d-c393-4b06-8125-a9b70370eb55-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-a-3ab8a5-656fccc874-gq8hx\" (UID: \"84a68a1d-c393-4b06-8125-a9b70370eb55\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-3ab8a5-656fccc874-gq8hx"
Sep 18 14:27:53.846792 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:27:53.846773    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/68e15645-97e4-48fd-913a-7d060eb1d7f1-plugins-config-volume\") pod \"payload-processing-e2e-ipp-a-3ab8a5-88fccfcf7-96nsp\" (UID: \"68e15645-97e4-48fd-913a-7d060eb1d7f1\") " pod="openshift-ingress/payload-processing-e2e-ipp-a-3ab8a5-88fccfcf7-96nsp"
Sep 18 14:27:53.846986 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:27:53.846971    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-v5l4z\" (UniqueName: \"kubernetes.io/projected/68e15645-97e4-48fd-913a-7d060eb1d7f1-kube-api-access-v5l4z\") pod \"payload-processing-e2e-ipp-a-3ab8a5-88fccfcf7-96nsp\" (UID: \"68e15645-97e4-48fd-913a-7d060eb1d7f1\") " pod="openshift-ingress/payload-processing-e2e-ipp-a-3ab8a5-88fccfcf7-96nsp"
Sep 18 14:27:53.847213 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:27:53.847195    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-szgh6\" (UniqueName: \"kubernetes.io/projected/84a68a1d-c393-4b06-8125-a9b70370eb55-kube-api-access-szgh6\") pod \"payload-pre-processing-e2e-ipp-a-3ab8a5-656fccc874-gq8hx\" (UID: \"84a68a1d-c393-4b06-8125-a9b70370eb55\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-3ab8a5-656fccc874-gq8hx"
Sep 18 14:27:53.849805 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:27:53.849780    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-ipp-a-3ab8a5\""
Sep 18 14:27:53.856439 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:27:53.856410    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-v5l4z\" (UniqueName: \"kubernetes.io/projected/68e15645-97e4-48fd-913a-7d060eb1d7f1-kube-api-access-v5l4z\") pod \"payload-processing-e2e-ipp-a-3ab8a5-88fccfcf7-96nsp\" (UID: \"68e15645-97e4-48fd-913a-7d060eb1d7f1\") " pod="openshift-ingress/payload-processing-e2e-ipp-a-3ab8a5-88fccfcf7-96nsp"
Sep 18 14:27:53.856803 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:27:53.856782    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-szgh6\" (UniqueName: \"kubernetes.io/projected/84a68a1d-c393-4b06-8125-a9b70370eb55-kube-api-access-szgh6\") pod \"payload-pre-processing-e2e-ipp-a-3ab8a5-656fccc874-gq8hx\" (UID: \"84a68a1d-c393-4b06-8125-a9b70370eb55\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-3ab8a5-656fccc874-gq8hx"
Sep 18 14:27:53.857817 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:27:53.857800    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/84a68a1d-c393-4b06-8125-a9b70370eb55-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-a-3ab8a5-656fccc874-gq8hx\" (UID: \"84a68a1d-c393-4b06-8125-a9b70370eb55\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-3ab8a5-656fccc874-gq8hx"
Sep 18 14:27:53.858351 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:27:53.858335    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/68e15645-97e4-48fd-913a-7d060eb1d7f1-plugins-config-volume\") pod \"payload-processing-e2e-ipp-a-3ab8a5-88fccfcf7-96nsp\" (UID: \"68e15645-97e4-48fd-913a-7d060eb1d7f1\") " pod="openshift-ingress/payload-processing-e2e-ipp-a-3ab8a5-88fccfcf7-96nsp"
Sep 18 14:27:53.935587 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:27:53.935508    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-ipp-a-3ab8a5-dockercfg-npq2j\""
Sep 18 14:27:53.943450 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:27:53.943422    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-3ab8a5-656fccc874-gq8hx"
Sep 18 14:27:53.972455 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:27:53.972418    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-a-3ab8a5-88fccfcf7-96nsp"
Sep 18 14:27:54.220989 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:27:54.220959    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-a-3ab8a5-656fccc874-gq8hx"]
Sep 18 14:27:54.223725 ip-10-0-129-16 kubenswrapper[2565]: W0918 14:27:54.223695    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod84a68a1d_c393_4b06_8125_a9b70370eb55.slice/crio-12c10d3d450bc36934330fb9b34089b9392add8d0833a45ba5d8091b49df7b00 WatchSource:0}: Error finding container 12c10d3d450bc36934330fb9b34089b9392add8d0833a45ba5d8091b49df7b00: Status 404 returned error can't find the container with id 12c10d3d450bc36934330fb9b34089b9392add8d0833a45ba5d8091b49df7b00
Sep 18 14:27:54.259511 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:27:54.259487    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-a-3ab8a5-88fccfcf7-96nsp"]
Sep 18 14:27:54.261022 ip-10-0-129-16 kubenswrapper[2565]: W0918 14:27:54.260999    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod68e15645_97e4_48fd_913a_7d060eb1d7f1.slice/crio-7ea0c8476128d67b5587c4f70c3de3dd61e985ead0c4587d43b365d7c7a9a520 WatchSource:0}: Error finding container 7ea0c8476128d67b5587c4f70c3de3dd61e985ead0c4587d43b365d7c7a9a520: Status 404 returned error can't find the container with id 7ea0c8476128d67b5587c4f70c3de3dd61e985ead0c4587d43b365d7c7a9a520
Sep 18 14:27:55.104751 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:27:55.104702    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-a-3ab8a5-88fccfcf7-96nsp" event={"ID":"68e15645-97e4-48fd-913a-7d060eb1d7f1","Type":"ContainerStarted","Data":"76461c6b9be65e45fc5ca572972a4c3e8e7a330b9ae9ca2704ded95870e9acc2"}
Sep 18 14:27:55.105223 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:27:55.104778    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-a-3ab8a5-88fccfcf7-96nsp" event={"ID":"68e15645-97e4-48fd-913a-7d060eb1d7f1","Type":"ContainerStarted","Data":"7ea0c8476128d67b5587c4f70c3de3dd61e985ead0c4587d43b365d7c7a9a520"}
Sep 18 14:27:55.105223 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:27:55.104812    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-ipp-a-3ab8a5-88fccfcf7-96nsp"
Sep 18 14:27:55.106360 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:27:55.106339    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-3ab8a5-656fccc874-gq8hx" event={"ID":"84a68a1d-c393-4b06-8125-a9b70370eb55","Type":"ContainerStarted","Data":"141a030225da6a74367f4a174d8e3b024112f5057322266360a67518ca67a039"}
Sep 18 14:27:55.106444 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:27:55.106375    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-3ab8a5-656fccc874-gq8hx" event={"ID":"84a68a1d-c393-4b06-8125-a9b70370eb55","Type":"ContainerStarted","Data":"12c10d3d450bc36934330fb9b34089b9392add8d0833a45ba5d8091b49df7b00"}
Sep 18 14:27:55.106444 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:27:55.106395    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-3ab8a5-656fccc874-gq8hx"
Sep 18 14:27:55.133672 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:27:55.133620    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-ipp-a-3ab8a5-88fccfcf7-96nsp" podStartSLOduration=2.133602873 podStartE2EDuration="2.133602873s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 14:27:53 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 14:27:55.131519634 +0000 UTC m=+1743.305429294" watchObservedRunningTime="2026-09-18 14:27:55.133602873 +0000 UTC m=+1743.307512533"
Sep 18 14:27:55.147011 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:27:55.146955    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-3ab8a5-656fccc874-gq8hx" podStartSLOduration=2.146934756 podStartE2EDuration="2.146934756s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 14:27:53 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 14:27:55.146508104 +0000 UTC m=+1743.320417779" watchObservedRunningTime="2026-09-18 14:27:55.146934756 +0000 UTC m=+1743.320844420"
Sep 18 14:28:06.112163 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:28:06.112126    2565 patch_prober.go:28] interesting pod/e2e-ipp-a-3ab8a5-openshift-default-6d6b5486cd-vbv4x container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.91:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Sep 18 14:28:06.112620 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:28:06.112221    2565 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-ipp-a-3ab8a5-openshift-default-6d6b5486cd-vbv4x" podUID="fc7144e0-5c5f-4457-97ac-7dbc3c2ae54d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.91:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Sep 18 14:28:06.113996 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:28:06.113971    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-3ab8a5-656fccc874-gq8hx"
Sep 18 14:28:06.114272 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:28:06.114249    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-ipp-a-3ab8a5-88fccfcf7-96nsp"
Sep 18 14:28:07.199831 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:28:07.199736    2565 generic.go:356] "Generic (PLEG): container finished" podID="fc7144e0-5c5f-4457-97ac-7dbc3c2ae54d" containerID="d2583c5faa5be01fe42560fb39aa54676f29bad44f7c98018189f2766284688b" exitCode=0
Sep 18 14:28:07.199831 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:28:07.199810    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-a-3ab8a5-openshift-default-6d6b5486cd-vbv4x" event={"ID":"fc7144e0-5c5f-4457-97ac-7dbc3c2ae54d","Type":"ContainerDied","Data":"d2583c5faa5be01fe42560fb39aa54676f29bad44f7c98018189f2766284688b"}
Sep 18 14:28:07.200285 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:28:07.200245    2565 scope.go:117] "RemoveContainer" containerID="d2583c5faa5be01fe42560fb39aa54676f29bad44f7c98018189f2766284688b"
Sep 18 14:28:07.201381 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:28:07.201353    2565 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 18 14:28:07.201502 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:28:07.201431    2565 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 18 14:28:07.201502 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:28:07.201477    2565 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 18 14:28:08.206666 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:28:08.206636    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-a-3ab8a5-openshift-default-6d6b5486cd-vbv4x" event={"ID":"fc7144e0-5c5f-4457-97ac-7dbc3c2ae54d","Type":"ContainerStarted","Data":"ba674a9297a558984f8d2c00425bfccf759dd26b2a55d73c63600d420b7a047d"}
Sep 18 14:28:08.363460 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:28:08.363418    2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ipp-a-3ab8a5-openshift-default-6d6b5486cd-vbv4x"
Sep 18 14:28:08.363746 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:28:08.363722    2565 patch_prober.go:28] interesting pod/e2e-ipp-a-3ab8a5-openshift-default-6d6b5486cd-vbv4x container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.91:15021/healthz/ready\": dial tcp 10.134.0.91:15021: connect: connection refused" start-of-body=
Sep 18 14:28:08.363809 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:28:08.363770    2565 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-a-3ab8a5-openshift-default-6d6b5486cd-vbv4x" podUID="fc7144e0-5c5f-4457-97ac-7dbc3c2ae54d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.91:15021/healthz/ready\": dial tcp 10.134.0.91:15021: connect: connection refused"
Sep 18 14:28:09.363610 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:28:09.363573    2565 patch_prober.go:28] interesting pod/e2e-ipp-a-3ab8a5-openshift-default-6d6b5486cd-vbv4x container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.91:15021/healthz/ready\": dial tcp 10.134.0.91:15021: connect: connection refused" start-of-body=
Sep 18 14:28:09.364003 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:28:09.363638    2565 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-a-3ab8a5-openshift-default-6d6b5486cd-vbv4x" podUID="fc7144e0-5c5f-4457-97ac-7dbc3c2ae54d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.91:15021/healthz/ready\": dial tcp 10.134.0.91:15021: connect: connection refused"
Sep 18 14:28:10.363107 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:28:10.363070    2565 patch_prober.go:28] interesting pod/e2e-ipp-a-3ab8a5-openshift-default-6d6b5486cd-vbv4x container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.91:15021/healthz/ready\": dial tcp 10.134.0.91:15021: connect: connection refused" start-of-body=
Sep 18 14:28:10.363314 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:28:10.363132    2565 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-a-3ab8a5-openshift-default-6d6b5486cd-vbv4x" podUID="fc7144e0-5c5f-4457-97ac-7dbc3c2ae54d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.91:15021/healthz/ready\": dial tcp 10.134.0.91:15021: connect: connection refused"
Sep 18 14:28:11.363368 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:28:11.363327    2565 patch_prober.go:28] interesting pod/e2e-ipp-a-3ab8a5-openshift-default-6d6b5486cd-vbv4x container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.91:15021/healthz/ready\": dial tcp 10.134.0.91:15021: connect: connection refused" start-of-body=
Sep 18 14:28:11.363763 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:28:11.363403    2565 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-a-3ab8a5-openshift-default-6d6b5486cd-vbv4x" podUID="fc7144e0-5c5f-4457-97ac-7dbc3c2ae54d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.91:15021/healthz/ready\": dial tcp 10.134.0.91:15021: connect: connection refused"
Sep 18 14:28:12.363334 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:28:12.363298    2565 patch_prober.go:28] interesting pod/e2e-ipp-a-3ab8a5-openshift-default-6d6b5486cd-vbv4x container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.91:15021/healthz/ready\": dial tcp 10.134.0.91:15021: connect: connection refused" start-of-body=
Sep 18 14:28:12.363572 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:28:12.363371    2565 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-a-3ab8a5-openshift-default-6d6b5486cd-vbv4x" podUID="fc7144e0-5c5f-4457-97ac-7dbc3c2ae54d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.91:15021/healthz/ready\": dial tcp 10.134.0.91:15021: connect: connection refused"
Sep 18 14:28:13.367128 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:28:13.367095    2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ipp-a-3ab8a5-openshift-default-6d6b5486cd-vbv4x"
Sep 18 14:28:13.367563 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:28:13.367451    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ipp-a-3ab8a5-openshift-default-6d6b5486cd-vbv4x"
Sep 18 14:28:13.368272 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:28:13.368251    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ipp-a-3ab8a5-openshift-default-6d6b5486cd-vbv4x"
Sep 18 14:28:23.640460 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:28:23.640424    2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-b-7ed186-765f59c65-wp8v9"]
Sep 18 14:28:23.647494 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:28:23.647457    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-b-7ed186-765f59c65-wp8v9"
Sep 18 14:28:23.651664 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:28:23.651628    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-ipp-b-7ed186-dockercfg-gb4rv\""
Sep 18 14:28:23.651924 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:28:23.651654    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-ipp-b-7ed186\""
Sep 18 14:28:23.654144 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:28:23.654118    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-b-7ed186-765f59c65-wp8v9"]
Sep 18 14:28:23.659840 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:28:23.659820    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-klcsr\" (UniqueName: \"kubernetes.io/projected/00cd7d42-6964-4908-bcfb-4f42895892de-kube-api-access-klcsr\") pod \"payload-processing-e2e-ipp-b-7ed186-765f59c65-wp8v9\" (UID: \"00cd7d42-6964-4908-bcfb-4f42895892de\") " pod="openshift-ingress/payload-processing-e2e-ipp-b-7ed186-765f59c65-wp8v9"
Sep 18 14:28:23.659952 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:28:23.659882    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/00cd7d42-6964-4908-bcfb-4f42895892de-plugins-config-volume\") pod \"payload-processing-e2e-ipp-b-7ed186-765f59c65-wp8v9\" (UID: \"00cd7d42-6964-4908-bcfb-4f42895892de\") " pod="openshift-ingress/payload-processing-e2e-ipp-b-7ed186-765f59c65-wp8v9"
Sep 18 14:28:23.761440 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:28:23.761406    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-klcsr\" (UniqueName: \"kubernetes.io/projected/00cd7d42-6964-4908-bcfb-4f42895892de-kube-api-access-klcsr\") pod \"payload-processing-e2e-ipp-b-7ed186-765f59c65-wp8v9\" (UID: \"00cd7d42-6964-4908-bcfb-4f42895892de\") " pod="openshift-ingress/payload-processing-e2e-ipp-b-7ed186-765f59c65-wp8v9"
Sep 18 14:28:23.761632 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:28:23.761458    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/00cd7d42-6964-4908-bcfb-4f42895892de-plugins-config-volume\") pod \"payload-processing-e2e-ipp-b-7ed186-765f59c65-wp8v9\" (UID: \"00cd7d42-6964-4908-bcfb-4f42895892de\") " pod="openshift-ingress/payload-processing-e2e-ipp-b-7ed186-765f59c65-wp8v9"
Sep 18 14:28:23.762156 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:28:23.762133    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/00cd7d42-6964-4908-bcfb-4f42895892de-plugins-config-volume\") pod \"payload-processing-e2e-ipp-b-7ed186-765f59c65-wp8v9\" (UID: \"00cd7d42-6964-4908-bcfb-4f42895892de\") " pod="openshift-ingress/payload-processing-e2e-ipp-b-7ed186-765f59c65-wp8v9"
Sep 18 14:28:23.770924 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:28:23.770900    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-klcsr\" (UniqueName: \"kubernetes.io/projected/00cd7d42-6964-4908-bcfb-4f42895892de-kube-api-access-klcsr\") pod \"payload-processing-e2e-ipp-b-7ed186-765f59c65-wp8v9\" (UID: \"00cd7d42-6964-4908-bcfb-4f42895892de\") " pod="openshift-ingress/payload-processing-e2e-ipp-b-7ed186-765f59c65-wp8v9"
Sep 18 14:28:23.961643 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:28:23.961568    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-b-7ed186-765f59c65-wp8v9"
Sep 18 14:28:24.239538 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:28:24.239510    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-b-7ed186-765f59c65-wp8v9"]
Sep 18 14:28:24.241619 ip-10-0-129-16 kubenswrapper[2565]: W0918 14:28:24.241594    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod00cd7d42_6964_4908_bcfb_4f42895892de.slice/crio-bf05291d17d5e36107827f048ecb7d470e3aa5a6458d43d58ec200d3f097f521 WatchSource:0}: Error finding container bf05291d17d5e36107827f048ecb7d470e3aa5a6458d43d58ec200d3f097f521: Status 404 returned error can't find the container with id bf05291d17d5e36107827f048ecb7d470e3aa5a6458d43d58ec200d3f097f521
Sep 18 14:28:24.310047 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:28:24.310001    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-b-7ed186-765f59c65-wp8v9" event={"ID":"00cd7d42-6964-4908-bcfb-4f42895892de","Type":"ContainerStarted","Data":"bf05291d17d5e36107827f048ecb7d470e3aa5a6458d43d58ec200d3f097f521"}
Sep 18 14:28:25.318244 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:28:25.318205    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-b-7ed186-765f59c65-wp8v9" event={"ID":"00cd7d42-6964-4908-bcfb-4f42895892de","Type":"ContainerStarted","Data":"8bcdcf7338cb3222c40cfd1282fd3aef5aa321426e762ec9f22e581eef850997"}
Sep 18 14:28:25.318644 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:28:25.318273    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-ipp-b-7ed186-765f59c65-wp8v9"
Sep 18 14:28:25.335826 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:28:25.335776    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-ipp-b-7ed186-765f59c65-wp8v9" podStartSLOduration=2.335760483 podStartE2EDuration="2.335760483s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 14:28:23 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 14:28:25.334288606 +0000 UTC m=+1773.508198291" watchObservedRunningTime="2026-09-18 14:28:25.335760483 +0000 UTC m=+1773.509670192"
Sep 18 14:28:36.326286 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:28:36.326197    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-ipp-b-7ed186-765f59c65-wp8v9"
Sep 18 14:28:50.060057 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:28:50.060009    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5b54b7fbb7-9v8fj_5b843100-3abb-415e-80d7-fd7cff369593/payload-processing/0.log"
Sep 18 14:28:50.169977 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:28:50.169948    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-e2e-ipp-b-7ed186-765f59c65-wp8v9_00cd7d42-6964-4908-bcfb-4f42895892de/payload-processing/0.log"
Sep 18 14:28:54.045603 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:28:54.045577    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-bvqls_0b0463f5-d647-4a38-b752-44afba77daff/ovn-acl-logging/0.log"
Sep 18 14:28:54.053700 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:28:54.053672    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-bvqls_0b0463f5-d647-4a38-b752-44afba77daff/ovn-acl-logging/0.log"
Sep 18 14:28:54.362324 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:28:54.362243    2565 scope.go:117] "RemoveContainer" containerID="29eef1660d79ba3294b8daeabe7799040ce17a57ea6485de4026dedc10763bfb"
Sep 18 14:29:20.942001 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:29:20.941971    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-e2e-ipp-a-3ab8a5-88fccfcf7-96nsp_68e15645-97e4-48fd-913a-7d060eb1d7f1/payload-processing/0.log"
Sep 18 14:29:21.066203 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:29:21.066174    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5b54b7fbb7-9v8fj_5b843100-3abb-415e-80d7-fd7cff369593/payload-processing/0.log"
Sep 18 14:29:21.182625 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:29:21.182595    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-e2e-ipp-b-7ed186-765f59c65-wp8v9_00cd7d42-6964-4908-bcfb-4f42895892de/payload-processing/0.log"
Sep 18 14:29:28.141454 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:29:28.141410    2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-cleanup-b58948-58947dfd79-z7nbl"]
Sep 18 14:29:28.145276 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:29:28.145259    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-b58948-58947dfd79-z7nbl"
Sep 18 14:29:28.148205 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:29:28.148179    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-ipp-cleanup-b58948\""
Sep 18 14:29:28.148324 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:29:28.148261    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-ipp-cleanup-b58948-dockercfg-4x6c9\""
Sep 18 14:29:28.156217 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:29:28.156193    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-cleanup-b58948-58947dfd79-z7nbl"]
Sep 18 14:29:28.301510 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:29:28.301473    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-r7rvt\" (UniqueName: \"kubernetes.io/projected/b96037bf-d70f-48a9-aabb-388f3f529266-kube-api-access-r7rvt\") pod \"payload-processing-e2e-ipp-cleanup-b58948-58947dfd79-z7nbl\" (UID: \"b96037bf-d70f-48a9-aabb-388f3f529266\") " pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-b58948-58947dfd79-z7nbl"
Sep 18 14:29:28.301698 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:29:28.301532    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b96037bf-d70f-48a9-aabb-388f3f529266-plugins-config-volume\") pod \"payload-processing-e2e-ipp-cleanup-b58948-58947dfd79-z7nbl\" (UID: \"b96037bf-d70f-48a9-aabb-388f3f529266\") " pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-b58948-58947dfd79-z7nbl"
Sep 18 14:29:28.402359 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:29:28.402262    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-r7rvt\" (UniqueName: \"kubernetes.io/projected/b96037bf-d70f-48a9-aabb-388f3f529266-kube-api-access-r7rvt\") pod \"payload-processing-e2e-ipp-cleanup-b58948-58947dfd79-z7nbl\" (UID: \"b96037bf-d70f-48a9-aabb-388f3f529266\") " pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-b58948-58947dfd79-z7nbl"
Sep 18 14:29:28.402359 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:29:28.402341    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b96037bf-d70f-48a9-aabb-388f3f529266-plugins-config-volume\") pod \"payload-processing-e2e-ipp-cleanup-b58948-58947dfd79-z7nbl\" (UID: \"b96037bf-d70f-48a9-aabb-388f3f529266\") " pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-b58948-58947dfd79-z7nbl"
Sep 18 14:29:28.403105 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:29:28.403085    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b96037bf-d70f-48a9-aabb-388f3f529266-plugins-config-volume\") pod \"payload-processing-e2e-ipp-cleanup-b58948-58947dfd79-z7nbl\" (UID: \"b96037bf-d70f-48a9-aabb-388f3f529266\") " pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-b58948-58947dfd79-z7nbl"
Sep 18 14:29:28.410135 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:29:28.410107    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-r7rvt\" (UniqueName: \"kubernetes.io/projected/b96037bf-d70f-48a9-aabb-388f3f529266-kube-api-access-r7rvt\") pod \"payload-processing-e2e-ipp-cleanup-b58948-58947dfd79-z7nbl\" (UID: \"b96037bf-d70f-48a9-aabb-388f3f529266\") " pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-b58948-58947dfd79-z7nbl"
Sep 18 14:29:28.456962 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:29:28.456928    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-b58948-58947dfd79-z7nbl"
Sep 18 14:29:28.729644 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:29:28.729615    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-cleanup-b58948-58947dfd79-z7nbl"]
Sep 18 14:29:28.731831 ip-10-0-129-16 kubenswrapper[2565]: W0918 14:29:28.731806    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb96037bf_d70f_48a9_aabb_388f3f529266.slice/crio-bf5bd6e67278cc4b0416b34ebc0a02bfcb8b03ad662ba5d2710c0ec653980b24 WatchSource:0}: Error finding container bf5bd6e67278cc4b0416b34ebc0a02bfcb8b03ad662ba5d2710c0ec653980b24: Status 404 returned error can't find the container with id bf5bd6e67278cc4b0416b34ebc0a02bfcb8b03ad662ba5d2710c0ec653980b24
Sep 18 14:29:29.692754 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:29:29.692707    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-b58948-58947dfd79-z7nbl" event={"ID":"b96037bf-d70f-48a9-aabb-388f3f529266","Type":"ContainerStarted","Data":"3a9dc9299f2f3c64ecfa5fef5d1d22a7ef2b6a4890c859ed621755ef9d8abfe5"}
Sep 18 14:29:29.693315 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:29:29.692780    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-b58948-58947dfd79-z7nbl" event={"ID":"b96037bf-d70f-48a9-aabb-388f3f529266","Type":"ContainerStarted","Data":"bf5bd6e67278cc4b0416b34ebc0a02bfcb8b03ad662ba5d2710c0ec653980b24"}
Sep 18 14:29:29.693315 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:29:29.692822    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-b58948-58947dfd79-z7nbl"
Sep 18 14:29:29.712601 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:29:29.712541    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-b58948-58947dfd79-z7nbl" podStartSLOduration=1.712526509 podStartE2EDuration="1.712526509s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 14:29:28 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 14:29:29.710951188 +0000 UTC m=+1837.884860873" watchObservedRunningTime="2026-09-18 14:29:29.712526509 +0000 UTC m=+1837.886436187"
Sep 18 14:29:40.705018 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:29:40.704988    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-b58948-58947dfd79-z7nbl"
Sep 18 14:29:50.005371 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:29:50.005330    2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-b58948-x5znh"]
Sep 18 14:29:50.009529 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:29:50.009509    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-b58948-x5znh"
Sep 18 14:29:50.012302 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:29:50.012278    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-l4fn9\""
Sep 18 14:29:50.017004 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:29:50.016968    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-b58948-x5znh"]
Sep 18 14:29:50.124233 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:29:50.124199    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/d0c87dd8-5614-48b8-9968-c4381aaec9f0-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-cleanup-b58948-x5znh\" (UID: \"d0c87dd8-5614-48b8-9968-c4381aaec9f0\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-b58948-x5znh"
Sep 18 14:29:50.225360 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:29:50.225322    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/d0c87dd8-5614-48b8-9968-c4381aaec9f0-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-cleanup-b58948-x5znh\" (UID: \"d0c87dd8-5614-48b8-9968-c4381aaec9f0\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-b58948-x5znh"
Sep 18 14:29:50.226111 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:29:50.226087    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/d0c87dd8-5614-48b8-9968-c4381aaec9f0-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-cleanup-b58948-x5znh\" (UID: \"d0c87dd8-5614-48b8-9968-c4381aaec9f0\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-b58948-x5znh"
Sep 18 14:29:50.322389 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:29:50.322361    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-b58948-x5znh"
Sep 18 14:29:50.597149 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:29:50.596948    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-b58948-x5znh"]
Sep 18 14:29:50.599971 ip-10-0-129-16 kubenswrapper[2565]: W0918 14:29:50.599936    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd0c87dd8_5614_48b8_9968_c4381aaec9f0.slice/crio-ff219259ae4eae276b0f0bc225e207d6e4ed881462749df49aac113341b9b443 WatchSource:0}: Error finding container ff219259ae4eae276b0f0bc225e207d6e4ed881462749df49aac113341b9b443: Status 404 returned error can't find the container with id ff219259ae4eae276b0f0bc225e207d6e4ed881462749df49aac113341b9b443
Sep 18 14:29:50.820504 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:29:50.820473    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-b58948-x5znh" event={"ID":"d0c87dd8-5614-48b8-9968-c4381aaec9f0","Type":"ContainerStarted","Data":"ff219259ae4eae276b0f0bc225e207d6e4ed881462749df49aac113341b9b443"}
Sep 18 14:29:51.827850 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:29:51.827810    2565 generic.go:356] "Generic (PLEG): container finished" podID="d0c87dd8-5614-48b8-9968-c4381aaec9f0" containerID="22ad5a54ea01898c836aab2461e24866d3db5ea429c369bbd44bb8167cd6d72e" exitCode=0
Sep 18 14:29:51.828313 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:29:51.827873    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-b58948-x5znh" event={"ID":"d0c87dd8-5614-48b8-9968-c4381aaec9f0","Type":"ContainerDied","Data":"22ad5a54ea01898c836aab2461e24866d3db5ea429c369bbd44bb8167cd6d72e"}
Sep 18 14:29:52.971412 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:29:52.971384    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-b58948-x5znh"
Sep 18 14:29:53.052723 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:29:53.052687    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/d0c87dd8-5614-48b8-9968-c4381aaec9f0-maas-api-service-ca\") pod \"d0c87dd8-5614-48b8-9968-c4381aaec9f0\" (UID: \"d0c87dd8-5614-48b8-9968-c4381aaec9f0\") "
Sep 18 14:29:53.053081 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:29:53.053058    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d0c87dd8-5614-48b8-9968-c4381aaec9f0-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "d0c87dd8-5614-48b8-9968-c4381aaec9f0" (UID: "d0c87dd8-5614-48b8-9968-c4381aaec9f0"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 18 14:29:53.153771 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:29:53.153683    2565 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/d0c87dd8-5614-48b8-9968-c4381aaec9f0-maas-api-service-ca\") on node \"ip-10-0-129-16.ec2.internal\" DevicePath \"\""
Sep 18 14:29:53.840764 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:29:53.840738    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-b58948-x5znh"
Sep 18 14:29:53.840928 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:29:53.840784    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-b58948-x5znh" event={"ID":"d0c87dd8-5614-48b8-9968-c4381aaec9f0","Type":"ContainerDied","Data":"ff219259ae4eae276b0f0bc225e207d6e4ed881462749df49aac113341b9b443"}
Sep 18 14:29:53.840928 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:29:53.840831    2565 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="ff219259ae4eae276b0f0bc225e207d6e4ed881462749df49aac113341b9b443"
Sep 18 14:29:54.383558 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:29:54.383528    2565 scope.go:117] "RemoveContainer" containerID="d372f34077fe7108208f9dcf88f82dab05c1aca54e24cfe946c05541e7b0a2d5"
Sep 18 14:29:54.392800 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:29:54.392780    2565 scope.go:117] "RemoveContainer" containerID="88da6b1c63cf2af78861550f31569ed4f1bbb669e339f4c4041c36d21b4dde6e"
Sep 18 14:29:54.401468 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:29:54.401443    2565 scope.go:117] "RemoveContainer" containerID="225dcbd6ff1fa786b59fcdf2a1a6fb0b58bc28a70b176537f71be18b8c7f43d4"
Sep 18 14:29:56.316936 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:29:56.316896    2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-cleanup-b58948-58947dfd79-z7nbl"]
Sep 18 14:29:56.317735 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:29:56.317443    2565 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-b58948-58947dfd79-z7nbl" podUID="b96037bf-d70f-48a9-aabb-388f3f529266" containerName="payload-processing" containerID="cri-o://3a9dc9299f2f3c64ecfa5fef5d1d22a7ef2b6a4890c859ed621755ef9d8abfe5" gracePeriod=30
Sep 18 14:29:56.498047 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:29:56.497062    2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-b58948-x5znh"]
Sep 18 14:29:56.506256 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:29:56.503918    2565 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-b58948-x5znh"]
Sep 18 14:29:56.586754 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:29:56.586673    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-b58948-58947dfd79-z7nbl"
Sep 18 14:29:56.687110 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:29:56.687069    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-r7rvt\" (UniqueName: \"kubernetes.io/projected/b96037bf-d70f-48a9-aabb-388f3f529266-kube-api-access-r7rvt\") pod \"b96037bf-d70f-48a9-aabb-388f3f529266\" (UID: \"b96037bf-d70f-48a9-aabb-388f3f529266\") "
Sep 18 14:29:56.687310 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:29:56.687208    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b96037bf-d70f-48a9-aabb-388f3f529266-plugins-config-volume\") pod \"b96037bf-d70f-48a9-aabb-388f3f529266\" (UID: \"b96037bf-d70f-48a9-aabb-388f3f529266\") "
Sep 18 14:29:56.687571 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:29:56.687545    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b96037bf-d70f-48a9-aabb-388f3f529266-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "b96037bf-d70f-48a9-aabb-388f3f529266" (UID: "b96037bf-d70f-48a9-aabb-388f3f529266"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 18 14:29:56.689332 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:29:56.689310    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b96037bf-d70f-48a9-aabb-388f3f529266-kube-api-access-r7rvt" (OuterVolumeSpecName: "kube-api-access-r7rvt") pod "b96037bf-d70f-48a9-aabb-388f3f529266" (UID: "b96037bf-d70f-48a9-aabb-388f3f529266"). InnerVolumeSpecName "kube-api-access-r7rvt". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 18 14:29:56.788095 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:29:56.788055    2565 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b96037bf-d70f-48a9-aabb-388f3f529266-plugins-config-volume\") on node \"ip-10-0-129-16.ec2.internal\" DevicePath \"\""
Sep 18 14:29:56.788095 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:29:56.788089    2565 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-r7rvt\" (UniqueName: \"kubernetes.io/projected/b96037bf-d70f-48a9-aabb-388f3f529266-kube-api-access-r7rvt\") on node \"ip-10-0-129-16.ec2.internal\" DevicePath \"\""
Sep 18 14:29:56.859579 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:29:56.859492    2565 generic.go:356] "Generic (PLEG): container finished" podID="b96037bf-d70f-48a9-aabb-388f3f529266" containerID="3a9dc9299f2f3c64ecfa5fef5d1d22a7ef2b6a4890c859ed621755ef9d8abfe5" exitCode=0
Sep 18 14:29:56.859579 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:29:56.859542    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-b58948-58947dfd79-z7nbl"
Sep 18 14:29:56.859773 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:29:56.859581    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-b58948-58947dfd79-z7nbl" event={"ID":"b96037bf-d70f-48a9-aabb-388f3f529266","Type":"ContainerDied","Data":"3a9dc9299f2f3c64ecfa5fef5d1d22a7ef2b6a4890c859ed621755ef9d8abfe5"}
Sep 18 14:29:56.859773 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:29:56.859674    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-b58948-58947dfd79-z7nbl" event={"ID":"b96037bf-d70f-48a9-aabb-388f3f529266","Type":"ContainerDied","Data":"bf5bd6e67278cc4b0416b34ebc0a02bfcb8b03ad662ba5d2710c0ec653980b24"}
Sep 18 14:29:56.859773 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:29:56.859705    2565 scope.go:117] "RemoveContainer" containerID="3a9dc9299f2f3c64ecfa5fef5d1d22a7ef2b6a4890c859ed621755ef9d8abfe5"
Sep 18 14:29:56.869622 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:29:56.869605    2565 scope.go:117] "RemoveContainer" containerID="3a9dc9299f2f3c64ecfa5fef5d1d22a7ef2b6a4890c859ed621755ef9d8abfe5"
Sep 18 14:29:56.869920 ip-10-0-129-16 kubenswrapper[2565]: E0918 14:29:56.869900    2565 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3a9dc9299f2f3c64ecfa5fef5d1d22a7ef2b6a4890c859ed621755ef9d8abfe5\": container with ID starting with 3a9dc9299f2f3c64ecfa5fef5d1d22a7ef2b6a4890c859ed621755ef9d8abfe5 not found: ID does not exist" containerID="3a9dc9299f2f3c64ecfa5fef5d1d22a7ef2b6a4890c859ed621755ef9d8abfe5"
Sep 18 14:29:56.869990 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:29:56.869931    2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3a9dc9299f2f3c64ecfa5fef5d1d22a7ef2b6a4890c859ed621755ef9d8abfe5"} err="failed to get container status \"3a9dc9299f2f3c64ecfa5fef5d1d22a7ef2b6a4890c859ed621755ef9d8abfe5\": rpc error: code = NotFound desc = could not find container \"3a9dc9299f2f3c64ecfa5fef5d1d22a7ef2b6a4890c859ed621755ef9d8abfe5\": container with ID starting with 3a9dc9299f2f3c64ecfa5fef5d1d22a7ef2b6a4890c859ed621755ef9d8abfe5 not found: ID does not exist"
Sep 18 14:29:56.882619 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:29:56.882591    2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-cleanup-b58948-58947dfd79-z7nbl"]
Sep 18 14:29:56.885262 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:29:56.885227    2565 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-cleanup-b58948-58947dfd79-z7nbl"]
Sep 18 14:29:57.913300 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:29:57.913264    2565 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b96037bf-d70f-48a9-aabb-388f3f529266" path="/var/lib/kubelet/pods/b96037bf-d70f-48a9-aabb-388f3f529266/volumes"
Sep 18 14:29:57.913683 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:29:57.913626    2565 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d0c87dd8-5614-48b8-9968-c4381aaec9f0" path="/var/lib/kubelet/pods/d0c87dd8-5614-48b8-9968-c4381aaec9f0/volumes"
Sep 18 14:30:00.211187 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:30:00.211152    2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ipp-a-3ab8a5-29829030-vmhpf"]
Sep 18 14:30:00.211665 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:30:00.211628    2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d0c87dd8-5614-48b8-9968-c4381aaec9f0" containerName="revoke-keys"
Sep 18 14:30:00.211665 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:30:00.211641    2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="d0c87dd8-5614-48b8-9968-c4381aaec9f0" containerName="revoke-keys"
Sep 18 14:30:00.211665 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:30:00.211661    2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b96037bf-d70f-48a9-aabb-388f3f529266" containerName="payload-processing"
Sep 18 14:30:00.211665 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:30:00.211667    2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="b96037bf-d70f-48a9-aabb-388f3f529266" containerName="payload-processing"
Sep 18 14:30:00.211805 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:30:00.211731    2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="d0c87dd8-5614-48b8-9968-c4381aaec9f0" containerName="revoke-keys"
Sep 18 14:30:00.211805 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:30:00.211740    2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="b96037bf-d70f-48a9-aabb-388f3f529266" containerName="payload-processing"
Sep 18 14:30:00.216277 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:30:00.216257    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ipp-a-3ab8a5-29829030-vmhpf"
Sep 18 14:30:00.219802 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:30:00.219774    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-l4fn9\""
Sep 18 14:30:00.222725 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:30:00.222700    2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29829030-mwdkm"]
Sep 18 14:30:00.226545 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:30:00.226522    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29829030-mwdkm"
Sep 18 14:30:00.226817 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:30:00.226800    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ipp-a-3ab8a5-29829030-vmhpf"
Sep 18 14:30:00.237868 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:30:00.237838    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ipp-a-3ab8a5-29829030-vmhpf"]
Sep 18 14:30:00.242632 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:30:00.239743    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29829030-mwdkm"
Sep 18 14:30:00.248993 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:30:00.248965    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29829030-mwdkm"]
Sep 18 14:30:00.261426 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:30:00.261392    2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ipp-b-7ed186-29829030-jphs4"]
Sep 18 14:30:00.267272 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:30:00.267241    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ipp-b-7ed186-29829030-jphs4"
Sep 18 14:30:00.300334 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:30:00.295141    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ipp-b-7ed186-29829030-jphs4"
Sep 18 14:30:00.302322 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:30:00.301718    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ipp-b-7ed186-29829030-jphs4"]
Sep 18 14:30:00.528553 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:30:00.528523    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ipp-a-3ab8a5-29829030-vmhpf"]
Sep 18 14:30:00.531425 ip-10-0-129-16 kubenswrapper[2565]: W0918 14:30:00.531400    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8996ebcc_f175_4fc7_ad26_19e704430dfb.slice/crio-1276ea4167578236976560c154dbe56b15996f13151e733f8b502152aa68bd3b WatchSource:0}: Error finding container 1276ea4167578236976560c154dbe56b15996f13151e733f8b502152aa68bd3b: Status 404 returned error can't find the container with id 1276ea4167578236976560c154dbe56b15996f13151e733f8b502152aa68bd3b
Sep 18 14:30:00.555781 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:30:00.555755    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29829030-mwdkm"]
Sep 18 14:30:00.561393 ip-10-0-129-16 kubenswrapper[2565]: W0918 14:30:00.561368    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9f94e766_bf61_4894_a29a_a85418b1432f.slice/crio-d15b654df34f7901f2527bf9ac32d5e94840a311d83509cef7b7282b2fcbe74b WatchSource:0}: Error finding container d15b654df34f7901f2527bf9ac32d5e94840a311d83509cef7b7282b2fcbe74b: Status 404 returned error can't find the container with id d15b654df34f7901f2527bf9ac32d5e94840a311d83509cef7b7282b2fcbe74b
Sep 18 14:30:00.604271 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:30:00.604246    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ipp-b-7ed186-29829030-jphs4"]
Sep 18 14:30:00.606745 ip-10-0-129-16 kubenswrapper[2565]: W0918 14:30:00.606718    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0302137b_5d71_44b0_9ff5_e0ada9d2a8cb.slice/crio-953c48157e467714af40237e6555345f3c43627f0e455a44292abea26dbe9478 WatchSource:0}: Error finding container 953c48157e467714af40237e6555345f3c43627f0e455a44292abea26dbe9478: Status 404 returned error can't find the container with id 953c48157e467714af40237e6555345f3c43627f0e455a44292abea26dbe9478
Sep 18 14:30:00.889487 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:30:00.889444    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ipp-b-7ed186-29829030-jphs4" event={"ID":"0302137b-5d71-44b0-9ff5-e0ada9d2a8cb","Type":"ContainerStarted","Data":"953c48157e467714af40237e6555345f3c43627f0e455a44292abea26dbe9478"}
Sep 18 14:30:00.892007 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:30:00.891967    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ipp-a-3ab8a5-29829030-vmhpf" event={"ID":"8996ebcc-f175-4fc7-ad26-19e704430dfb","Type":"ContainerStarted","Data":"6bccfb174f68e14178ee52e704ce03a2a686a02bb9081a367b39752758618d1e"}
Sep 18 14:30:00.892184 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:30:00.892058    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ipp-a-3ab8a5-29829030-vmhpf" event={"ID":"8996ebcc-f175-4fc7-ad26-19e704430dfb","Type":"ContainerStarted","Data":"1276ea4167578236976560c154dbe56b15996f13151e733f8b502152aa68bd3b"}
Sep 18 14:30:00.894279 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:30:00.894249    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29829030-mwdkm" event={"ID":"9f94e766-bf61-4894-a29a-a85418b1432f","Type":"ContainerStarted","Data":"aa9fb1904b9d1ae2507dc7f98b6963aac632d49728cd279177c242838c06fd5d"}
Sep 18 14:30:00.894400 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:30:00.894296    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29829030-mwdkm" event={"ID":"9f94e766-bf61-4894-a29a-a85418b1432f","Type":"ContainerStarted","Data":"d15b654df34f7901f2527bf9ac32d5e94840a311d83509cef7b7282b2fcbe74b"}
Sep 18 14:30:00.910047 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:30:00.909991    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ipp-a-3ab8a5-29829030-vmhpf" podStartSLOduration=0.909975188 podStartE2EDuration="909.975188ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 14:30:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 14:30:00.908705185 +0000 UTC m=+1869.082614846" watchObservedRunningTime="2026-09-18 14:30:00.909975188 +0000 UTC m=+1869.083884848"
Sep 18 14:30:00.932660 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:30:00.932569    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29829030-mwdkm" podStartSLOduration=0.932551345 podStartE2EDuration="932.551345ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 14:30:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 14:30:00.93213889 +0000 UTC m=+1869.106048580" watchObservedRunningTime="2026-09-18 14:30:00.932551345 +0000 UTC m=+1869.106460985"
Sep 18 14:30:01.901968 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:30:01.901929    2565 generic.go:356] "Generic (PLEG): container finished" podID="9f94e766-bf61-4894-a29a-a85418b1432f" containerID="aa9fb1904b9d1ae2507dc7f98b6963aac632d49728cd279177c242838c06fd5d" exitCode=0
Sep 18 14:30:01.902385 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:30:01.902017    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29829030-mwdkm" event={"ID":"9f94e766-bf61-4894-a29a-a85418b1432f","Type":"ContainerDied","Data":"aa9fb1904b9d1ae2507dc7f98b6963aac632d49728cd279177c242838c06fd5d"}
Sep 18 14:30:01.903648 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:30:01.903625    2565 generic.go:356] "Generic (PLEG): container finished" podID="0302137b-5d71-44b0-9ff5-e0ada9d2a8cb" containerID="97bff83ae8db185b47f7dbe2535e9599a57afaf0072a869a8c38886bd35517b7" exitCode=0
Sep 18 14:30:01.903767 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:30:01.903680    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ipp-b-7ed186-29829030-jphs4" event={"ID":"0302137b-5d71-44b0-9ff5-e0ada9d2a8cb","Type":"ContainerDied","Data":"97bff83ae8db185b47f7dbe2535e9599a57afaf0072a869a8c38886bd35517b7"}
Sep 18 14:30:01.905213 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:30:01.905192    2565 generic.go:356] "Generic (PLEG): container finished" podID="8996ebcc-f175-4fc7-ad26-19e704430dfb" containerID="6bccfb174f68e14178ee52e704ce03a2a686a02bb9081a367b39752758618d1e" exitCode=0
Sep 18 14:30:01.905281 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:30:01.905246    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ipp-a-3ab8a5-29829030-vmhpf" event={"ID":"8996ebcc-f175-4fc7-ad26-19e704430dfb","Type":"ContainerDied","Data":"6bccfb174f68e14178ee52e704ce03a2a686a02bb9081a367b39752758618d1e"}
Sep 18 14:30:03.065777 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:30:03.065749    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ipp-a-3ab8a5-29829030-vmhpf"
Sep 18 14:30:03.120629 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:30:03.120594    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29829030-mwdkm"
Sep 18 14:30:03.123916 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:30:03.123894    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ipp-b-7ed186-29829030-jphs4"
Sep 18 14:30:03.918901 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:30:03.918872    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ipp-a-3ab8a5-29829030-vmhpf"
Sep 18 14:30:03.919142 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:30:03.918911    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ipp-a-3ab8a5-29829030-vmhpf" event={"ID":"8996ebcc-f175-4fc7-ad26-19e704430dfb","Type":"ContainerDied","Data":"1276ea4167578236976560c154dbe56b15996f13151e733f8b502152aa68bd3b"}
Sep 18 14:30:03.919142 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:30:03.918959    2565 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="1276ea4167578236976560c154dbe56b15996f13151e733f8b502152aa68bd3b"
Sep 18 14:30:03.920543 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:30:03.920514    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29829030-mwdkm"
Sep 18 14:30:03.920730 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:30:03.920698    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29829030-mwdkm" event={"ID":"9f94e766-bf61-4894-a29a-a85418b1432f","Type":"ContainerDied","Data":"d15b654df34f7901f2527bf9ac32d5e94840a311d83509cef7b7282b2fcbe74b"}
Sep 18 14:30:03.920869 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:30:03.920754    2565 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="d15b654df34f7901f2527bf9ac32d5e94840a311d83509cef7b7282b2fcbe74b"
Sep 18 14:30:03.922571 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:30:03.922205    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ipp-b-7ed186-29829030-jphs4" event={"ID":"0302137b-5d71-44b0-9ff5-e0ada9d2a8cb","Type":"ContainerDied","Data":"953c48157e467714af40237e6555345f3c43627f0e455a44292abea26dbe9478"}
Sep 18 14:30:03.922571 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:30:03.922227    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ipp-b-7ed186-29829030-jphs4"
Sep 18 14:30:03.922571 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:30:03.922243    2565 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="953c48157e467714af40237e6555345f3c43627f0e455a44292abea26dbe9478"
Sep 18 14:30:07.038293 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:30:07.038197    2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-3ab8a5-6qzxn"]
Sep 18 14:30:07.038910 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:30:07.038888    2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8996ebcc-f175-4fc7-ad26-19e704430dfb" containerName="cleanup"
Sep 18 14:30:07.038994 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:30:07.038913    2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="8996ebcc-f175-4fc7-ad26-19e704430dfb" containerName="cleanup"
Sep 18 14:30:07.038994 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:30:07.038929    2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9f94e766-bf61-4894-a29a-a85418b1432f" containerName="cleanup"
Sep 18 14:30:07.038994 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:30:07.038937    2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="9f94e766-bf61-4894-a29a-a85418b1432f" containerName="cleanup"
Sep 18 14:30:07.038994 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:30:07.038951    2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0302137b-5d71-44b0-9ff5-e0ada9d2a8cb" containerName="cleanup"
Sep 18 14:30:07.038994 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:30:07.038962    2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="0302137b-5d71-44b0-9ff5-e0ada9d2a8cb" containerName="cleanup"
Sep 18 14:30:07.039275 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:30:07.039088    2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="8996ebcc-f175-4fc7-ad26-19e704430dfb" containerName="cleanup"
Sep 18 14:30:07.039275 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:30:07.039107    2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="9f94e766-bf61-4894-a29a-a85418b1432f" containerName="cleanup"
Sep 18 14:30:07.039275 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:30:07.039117    2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="0302137b-5d71-44b0-9ff5-e0ada9d2a8cb" containerName="cleanup"
Sep 18 14:30:07.042444 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:30:07.042420    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-3ab8a5-6qzxn"
Sep 18 14:30:07.045210 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:30:07.045191    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-l4fn9\""
Sep 18 14:30:07.051593 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:30:07.051564    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-3ab8a5-6qzxn"]
Sep 18 14:30:07.197907 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:30:07.197876    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/aee91c3b-b7de-4f3f-929f-1b965f56ff68-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-a-3ab8a5-6qzxn\" (UID: \"aee91c3b-b7de-4f3f-929f-1b965f56ff68\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-3ab8a5-6qzxn"
Sep 18 14:30:07.299054 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:30:07.298937    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/aee91c3b-b7de-4f3f-929f-1b965f56ff68-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-a-3ab8a5-6qzxn\" (UID: \"aee91c3b-b7de-4f3f-929f-1b965f56ff68\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-3ab8a5-6qzxn"
Sep 18 14:30:07.299601 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:30:07.299578    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/aee91c3b-b7de-4f3f-929f-1b965f56ff68-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-a-3ab8a5-6qzxn\" (UID: \"aee91c3b-b7de-4f3f-929f-1b965f56ff68\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-3ab8a5-6qzxn"
Sep 18 14:30:07.355440 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:30:07.355411    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-3ab8a5-6qzxn"
Sep 18 14:30:07.633569 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:30:07.633535    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-3ab8a5-6qzxn"]
Sep 18 14:30:07.635129 ip-10-0-129-16 kubenswrapper[2565]: W0918 14:30:07.635094    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podaee91c3b_b7de_4f3f_929f_1b965f56ff68.slice/crio-2720c9662e6f6c4a3fa8eed8e029c6bd6cd5fecdf5a8fb4652a748c9f3ca37e4 WatchSource:0}: Error finding container 2720c9662e6f6c4a3fa8eed8e029c6bd6cd5fecdf5a8fb4652a748c9f3ca37e4: Status 404 returned error can't find the container with id 2720c9662e6f6c4a3fa8eed8e029c6bd6cd5fecdf5a8fb4652a748c9f3ca37e4
Sep 18 14:30:07.947005 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:30:07.946981    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-3ab8a5-6qzxn" event={"ID":"aee91c3b-b7de-4f3f-929f-1b965f56ff68","Type":"ContainerStarted","Data":"2720c9662e6f6c4a3fa8eed8e029c6bd6cd5fecdf5a8fb4652a748c9f3ca37e4"}
Sep 18 14:30:08.955214 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:30:08.955177    2565 generic.go:356] "Generic (PLEG): container finished" podID="aee91c3b-b7de-4f3f-929f-1b965f56ff68" containerID="eb49410c73c0fb7506a97d1409489eb8dd703be1c6a5bdeff22c185c9334b7be" exitCode=0
Sep 18 14:30:08.955575 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:30:08.955254    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-3ab8a5-6qzxn" event={"ID":"aee91c3b-b7de-4f3f-929f-1b965f56ff68","Type":"ContainerDied","Data":"eb49410c73c0fb7506a97d1409489eb8dd703be1c6a5bdeff22c185c9334b7be"}
Sep 18 14:30:10.099832 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:30:10.099803    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-3ab8a5-6qzxn"
Sep 18 14:30:10.231441 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:30:10.231342    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/aee91c3b-b7de-4f3f-929f-1b965f56ff68-maas-api-service-ca\") pod \"aee91c3b-b7de-4f3f-929f-1b965f56ff68\" (UID: \"aee91c3b-b7de-4f3f-929f-1b965f56ff68\") "
Sep 18 14:30:10.231724 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:30:10.231698    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/aee91c3b-b7de-4f3f-929f-1b965f56ff68-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "aee91c3b-b7de-4f3f-929f-1b965f56ff68" (UID: "aee91c3b-b7de-4f3f-929f-1b965f56ff68"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 18 14:30:10.332471 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:30:10.332434    2565 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/aee91c3b-b7de-4f3f-929f-1b965f56ff68-maas-api-service-ca\") on node \"ip-10-0-129-16.ec2.internal\" DevicePath \"\""
Sep 18 14:30:10.967183 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:30:10.967146    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-3ab8a5-6qzxn" event={"ID":"aee91c3b-b7de-4f3f-929f-1b965f56ff68","Type":"ContainerDied","Data":"2720c9662e6f6c4a3fa8eed8e029c6bd6cd5fecdf5a8fb4652a748c9f3ca37e4"}
Sep 18 14:30:10.967389 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:30:10.967207    2565 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="2720c9662e6f6c4a3fa8eed8e029c6bd6cd5fecdf5a8fb4652a748c9f3ca37e4"
Sep 18 14:30:10.967389 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:30:10.967163    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-3ab8a5-6qzxn"
Sep 18 14:30:22.640733 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:30:22.640695    2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ipp-a-3ab8a5-29829030-vmhpf"]
Sep 18 14:30:22.643177 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:30:22.643149    2565 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ipp-a-3ab8a5-29829030-vmhpf"]
Sep 18 14:30:22.670833 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:30:22.670803    2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-a-3ab8a5-88fccfcf7-96nsp"]
Sep 18 14:30:22.671083 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:30:22.671020    2565 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-ipp-a-3ab8a5-88fccfcf7-96nsp" podUID="68e15645-97e4-48fd-913a-7d060eb1d7f1" containerName="payload-processing" containerID="cri-o://76461c6b9be65e45fc5ca572972a4c3e8e7a330b9ae9ca2704ded95870e9acc2" gracePeriod=30
Sep 18 14:30:22.678950 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:30:22.678925    2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-a-3ab8a5-656fccc874-gq8hx"]
Sep 18 14:30:22.679203 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:30:22.679158    2565 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-3ab8a5-656fccc874-gq8hx" podUID="84a68a1d-c393-4b06-8125-a9b70370eb55" containerName="payload-pre-processing" containerID="cri-o://141a030225da6a74367f4a174d8e3b024112f5057322266360a67518ca67a039" gracePeriod=30
Sep 18 14:30:22.849702 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:30:22.849649    2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-3ab8a5-6qzxn"]
Sep 18 14:30:22.850489 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:30:22.849710    2565 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-3ab8a5-6qzxn"]
Sep 18 14:30:22.937463 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:30:22.937436    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-a-3ab8a5-88fccfcf7-96nsp"
Sep 18 14:30:22.961048 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:30:22.960993    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-v5l4z\" (UniqueName: \"kubernetes.io/projected/68e15645-97e4-48fd-913a-7d060eb1d7f1-kube-api-access-v5l4z\") pod \"68e15645-97e4-48fd-913a-7d060eb1d7f1\" (UID: \"68e15645-97e4-48fd-913a-7d060eb1d7f1\") "
Sep 18 14:30:22.961189 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:30:22.961130    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/68e15645-97e4-48fd-913a-7d060eb1d7f1-plugins-config-volume\") pod \"68e15645-97e4-48fd-913a-7d060eb1d7f1\" (UID: \"68e15645-97e4-48fd-913a-7d060eb1d7f1\") "
Sep 18 14:30:22.962761 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:30:22.962721    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/68e15645-97e4-48fd-913a-7d060eb1d7f1-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "68e15645-97e4-48fd-913a-7d060eb1d7f1" (UID: "68e15645-97e4-48fd-913a-7d060eb1d7f1"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 18 14:30:22.965147 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:30:22.965117    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/68e15645-97e4-48fd-913a-7d060eb1d7f1-kube-api-access-v5l4z" (OuterVolumeSpecName: "kube-api-access-v5l4z") pod "68e15645-97e4-48fd-913a-7d060eb1d7f1" (UID: "68e15645-97e4-48fd-913a-7d060eb1d7f1"). InnerVolumeSpecName "kube-api-access-v5l4z". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 18 14:30:22.974755 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:30:22.974729    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-3ab8a5-656fccc874-gq8hx"
Sep 18 14:30:23.044002 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:30:23.043970    2565 generic.go:356] "Generic (PLEG): container finished" podID="84a68a1d-c393-4b06-8125-a9b70370eb55" containerID="141a030225da6a74367f4a174d8e3b024112f5057322266360a67518ca67a039" exitCode=0
Sep 18 14:30:23.044216 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:30:23.044022    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-3ab8a5-656fccc874-gq8hx"
Sep 18 14:30:23.044216 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:30:23.044105    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-3ab8a5-656fccc874-gq8hx" event={"ID":"84a68a1d-c393-4b06-8125-a9b70370eb55","Type":"ContainerDied","Data":"141a030225da6a74367f4a174d8e3b024112f5057322266360a67518ca67a039"}
Sep 18 14:30:23.044216 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:30:23.044163    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-3ab8a5-656fccc874-gq8hx" event={"ID":"84a68a1d-c393-4b06-8125-a9b70370eb55","Type":"ContainerDied","Data":"12c10d3d450bc36934330fb9b34089b9392add8d0833a45ba5d8091b49df7b00"}
Sep 18 14:30:23.044216 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:30:23.044188    2565 scope.go:117] "RemoveContainer" containerID="141a030225da6a74367f4a174d8e3b024112f5057322266360a67518ca67a039"
Sep 18 14:30:23.045714 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:30:23.045694    2565 generic.go:356] "Generic (PLEG): container finished" podID="68e15645-97e4-48fd-913a-7d060eb1d7f1" containerID="76461c6b9be65e45fc5ca572972a4c3e8e7a330b9ae9ca2704ded95870e9acc2" exitCode=0
Sep 18 14:30:23.045831 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:30:23.045724    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-a-3ab8a5-88fccfcf7-96nsp" event={"ID":"68e15645-97e4-48fd-913a-7d060eb1d7f1","Type":"ContainerDied","Data":"76461c6b9be65e45fc5ca572972a4c3e8e7a330b9ae9ca2704ded95870e9acc2"}
Sep 18 14:30:23.045831 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:30:23.045753    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-a-3ab8a5-88fccfcf7-96nsp"
Sep 18 14:30:23.045831 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:30:23.045755    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-a-3ab8a5-88fccfcf7-96nsp" event={"ID":"68e15645-97e4-48fd-913a-7d060eb1d7f1","Type":"ContainerDied","Data":"7ea0c8476128d67b5587c4f70c3de3dd61e985ead0c4587d43b365d7c7a9a520"}
Sep 18 14:30:23.054396 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:30:23.054359    2565 scope.go:117] "RemoveContainer" containerID="141a030225da6a74367f4a174d8e3b024112f5057322266360a67518ca67a039"
Sep 18 14:30:23.054794 ip-10-0-129-16 kubenswrapper[2565]: E0918 14:30:23.054770    2565 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"141a030225da6a74367f4a174d8e3b024112f5057322266360a67518ca67a039\": container with ID starting with 141a030225da6a74367f4a174d8e3b024112f5057322266360a67518ca67a039 not found: ID does not exist" containerID="141a030225da6a74367f4a174d8e3b024112f5057322266360a67518ca67a039"
Sep 18 14:30:23.054905 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:30:23.054805    2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"141a030225da6a74367f4a174d8e3b024112f5057322266360a67518ca67a039"} err="failed to get container status \"141a030225da6a74367f4a174d8e3b024112f5057322266360a67518ca67a039\": rpc error: code = NotFound desc = could not find container \"141a030225da6a74367f4a174d8e3b024112f5057322266360a67518ca67a039\": container with ID starting with 141a030225da6a74367f4a174d8e3b024112f5057322266360a67518ca67a039 not found: ID does not exist"
Sep 18 14:30:23.054982 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:30:23.054827    2565 scope.go:117] "RemoveContainer" containerID="76461c6b9be65e45fc5ca572972a4c3e8e7a330b9ae9ca2704ded95870e9acc2"
Sep 18 14:30:23.063194 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:30:23.063171    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-szgh6\" (UniqueName: \"kubernetes.io/projected/84a68a1d-c393-4b06-8125-a9b70370eb55-kube-api-access-szgh6\") pod \"84a68a1d-c393-4b06-8125-a9b70370eb55\" (UID: \"84a68a1d-c393-4b06-8125-a9b70370eb55\") "
Sep 18 14:30:23.063297 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:30:23.063276    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/84a68a1d-c393-4b06-8125-a9b70370eb55-plugins-config-volume\") pod \"84a68a1d-c393-4b06-8125-a9b70370eb55\" (UID: \"84a68a1d-c393-4b06-8125-a9b70370eb55\") "
Sep 18 14:30:23.063681 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:30:23.063650    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/84a68a1d-c393-4b06-8125-a9b70370eb55-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "84a68a1d-c393-4b06-8125-a9b70370eb55" (UID: "84a68a1d-c393-4b06-8125-a9b70370eb55"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 18 14:30:23.063785 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:30:23.063710    2565 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/68e15645-97e4-48fd-913a-7d060eb1d7f1-plugins-config-volume\") on node \"ip-10-0-129-16.ec2.internal\" DevicePath \"\""
Sep 18 14:30:23.063785 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:30:23.063726    2565 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-v5l4z\" (UniqueName: \"kubernetes.io/projected/68e15645-97e4-48fd-913a-7d060eb1d7f1-kube-api-access-v5l4z\") on node \"ip-10-0-129-16.ec2.internal\" DevicePath \"\""
Sep 18 14:30:23.065252 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:30:23.065184    2565 scope.go:117] "RemoveContainer" containerID="76461c6b9be65e45fc5ca572972a4c3e8e7a330b9ae9ca2704ded95870e9acc2"
Sep 18 14:30:23.065399 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:30:23.065375    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/84a68a1d-c393-4b06-8125-a9b70370eb55-kube-api-access-szgh6" (OuterVolumeSpecName: "kube-api-access-szgh6") pod "84a68a1d-c393-4b06-8125-a9b70370eb55" (UID: "84a68a1d-c393-4b06-8125-a9b70370eb55"). InnerVolumeSpecName "kube-api-access-szgh6". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 18 14:30:23.065563 ip-10-0-129-16 kubenswrapper[2565]: E0918 14:30:23.065541    2565 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"76461c6b9be65e45fc5ca572972a4c3e8e7a330b9ae9ca2704ded95870e9acc2\": container with ID starting with 76461c6b9be65e45fc5ca572972a4c3e8e7a330b9ae9ca2704ded95870e9acc2 not found: ID does not exist" containerID="76461c6b9be65e45fc5ca572972a4c3e8e7a330b9ae9ca2704ded95870e9acc2"
Sep 18 14:30:23.065605 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:30:23.065572    2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"76461c6b9be65e45fc5ca572972a4c3e8e7a330b9ae9ca2704ded95870e9acc2"} err="failed to get container status \"76461c6b9be65e45fc5ca572972a4c3e8e7a330b9ae9ca2704ded95870e9acc2\": rpc error: code = NotFound desc = could not find container \"76461c6b9be65e45fc5ca572972a4c3e8e7a330b9ae9ca2704ded95870e9acc2\": container with ID starting with 76461c6b9be65e45fc5ca572972a4c3e8e7a330b9ae9ca2704ded95870e9acc2 not found: ID does not exist"
Sep 18 14:30:23.075111 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:30:23.075086    2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-a-3ab8a5-88fccfcf7-96nsp"]
Sep 18 14:30:23.087670 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:30:23.087643    2565 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-a-3ab8a5-88fccfcf7-96nsp"]
Sep 18 14:30:23.164587 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:30:23.164501    2565 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-szgh6\" (UniqueName: \"kubernetes.io/projected/84a68a1d-c393-4b06-8125-a9b70370eb55-kube-api-access-szgh6\") on node \"ip-10-0-129-16.ec2.internal\" DevicePath \"\""
Sep 18 14:30:23.164587 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:30:23.164528    2565 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/84a68a1d-c393-4b06-8125-a9b70370eb55-plugins-config-volume\") on node \"ip-10-0-129-16.ec2.internal\" DevicePath \"\""
Sep 18 14:30:23.365509 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:30:23.365478    2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-a-3ab8a5-656fccc874-gq8hx"]
Sep 18 14:30:23.368902 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:30:23.368877    2565 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-a-3ab8a5-656fccc874-gq8hx"]
Sep 18 14:30:23.914425 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:30:23.914387    2565 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="68e15645-97e4-48fd-913a-7d060eb1d7f1" path="/var/lib/kubelet/pods/68e15645-97e4-48fd-913a-7d060eb1d7f1/volumes"
Sep 18 14:30:23.914777 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:30:23.914743    2565 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="84a68a1d-c393-4b06-8125-a9b70370eb55" path="/var/lib/kubelet/pods/84a68a1d-c393-4b06-8125-a9b70370eb55/volumes"
Sep 18 14:30:23.914991 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:30:23.914978    2565 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8996ebcc-f175-4fc7-ad26-19e704430dfb" path="/var/lib/kubelet/pods/8996ebcc-f175-4fc7-ad26-19e704430dfb/volumes"
Sep 18 14:30:23.915272 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:30:23.915260    2565 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="aee91c3b-b7de-4f3f-929f-1b965f56ff68" path="/var/lib/kubelet/pods/aee91c3b-b7de-4f3f-929f-1b965f56ff68/volumes"
Sep 18 14:30:54.431313 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:30:54.431280    2565 scope.go:117] "RemoveContainer" containerID="6df5bd6777d02b1ec6d83c3f224b9eba5f4475a3f84fd488a8e70d950fe2cc2c"
Sep 18 14:30:54.439922 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:30:54.439894    2565 scope.go:117] "RemoveContainer" containerID="9d090b2ee3c36b33747eac2c7d93a3911a09870c1a9135968955f53f81925097"
Sep 18 14:30:54.447852 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:30:54.447832    2565 scope.go:117] "RemoveContainer" containerID="e07bacc00c18f34cece8b6310c0d9ed194fd03809c8c8c924585543359e69bfb"
Sep 18 14:31:26.653162 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:31:26.653116    2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ipp-a-3ab8a5-openshift-default-6d6b5486cd-vbv4x"]
Sep 18 14:31:26.653810 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:31:26.653420    2565 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-ipp-a-3ab8a5-openshift-default-6d6b5486cd-vbv4x" podUID="fc7144e0-5c5f-4457-97ac-7dbc3c2ae54d" containerName="istio-proxy" containerID="cri-o://ba674a9297a558984f8d2c00425bfccf759dd26b2a55d73c63600d420b7a047d" gracePeriod=30
Sep 18 14:31:27.058204 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:31:27.058165    2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-7ed186-gfgtq"]
Sep 18 14:31:27.058915 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:31:27.058896    2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="aee91c3b-b7de-4f3f-929f-1b965f56ff68" containerName="revoke-keys"
Sep 18 14:31:27.058967 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:31:27.058920    2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="aee91c3b-b7de-4f3f-929f-1b965f56ff68" containerName="revoke-keys"
Sep 18 14:31:27.058967 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:31:27.058943    2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="84a68a1d-c393-4b06-8125-a9b70370eb55" containerName="payload-pre-processing"
Sep 18 14:31:27.058967 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:31:27.058953    2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="84a68a1d-c393-4b06-8125-a9b70370eb55" containerName="payload-pre-processing"
Sep 18 14:31:27.059093 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:31:27.058980    2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="68e15645-97e4-48fd-913a-7d060eb1d7f1" containerName="payload-processing"
Sep 18 14:31:27.059093 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:31:27.058990    2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="68e15645-97e4-48fd-913a-7d060eb1d7f1" containerName="payload-processing"
Sep 18 14:31:27.059158 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:31:27.059110    2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="84a68a1d-c393-4b06-8125-a9b70370eb55" containerName="payload-pre-processing"
Sep 18 14:31:27.059158 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:31:27.059122    2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="68e15645-97e4-48fd-913a-7d060eb1d7f1" containerName="payload-processing"
Sep 18 14:31:27.059158 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:31:27.059132    2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="aee91c3b-b7de-4f3f-929f-1b965f56ff68" containerName="revoke-keys"
Sep 18 14:31:27.062748 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:31:27.062731    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-7ed186-gfgtq"
Sep 18 14:31:27.066156 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:31:27.066135    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-l4fn9\""
Sep 18 14:31:27.075903 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:31:27.075880    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-7ed186-gfgtq"]
Sep 18 14:31:27.121962 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:31:27.121926    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/4b3af8b5-8002-4e15-b2ef-e9f45fb80d28-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-b-7ed186-gfgtq\" (UID: \"4b3af8b5-8002-4e15-b2ef-e9f45fb80d28\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-7ed186-gfgtq"
Sep 18 14:31:27.222817 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:31:27.222774    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/4b3af8b5-8002-4e15-b2ef-e9f45fb80d28-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-b-7ed186-gfgtq\" (UID: \"4b3af8b5-8002-4e15-b2ef-e9f45fb80d28\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-7ed186-gfgtq"
Sep 18 14:31:27.223548 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:31:27.223526    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/4b3af8b5-8002-4e15-b2ef-e9f45fb80d28-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-b-7ed186-gfgtq\" (UID: \"4b3af8b5-8002-4e15-b2ef-e9f45fb80d28\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-7ed186-gfgtq"
Sep 18 14:31:27.373088 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:31:27.372981    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-7ed186-gfgtq"
Sep 18 14:31:27.662308 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:31:27.662278    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-7ed186-gfgtq"]
Sep 18 14:31:27.664437 ip-10-0-129-16 kubenswrapper[2565]: W0918 14:31:27.664409    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4b3af8b5_8002_4e15_b2ef_e9f45fb80d28.slice/crio-aea94aa0f6d59cb10ec7681a832e16cd7566affd6471638ab4e6fa1fed6a9a63 WatchSource:0}: Error finding container aea94aa0f6d59cb10ec7681a832e16cd7566affd6471638ab4e6fa1fed6a9a63: Status 404 returned error can't find the container with id aea94aa0f6d59cb10ec7681a832e16cd7566affd6471638ab4e6fa1fed6a9a63
Sep 18 14:31:28.370242 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:31:28.370203    2565 patch_prober.go:28] interesting pod/e2e-ipp-a-3ab8a5-openshift-default-6d6b5486cd-vbv4x container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Sep 18 14:31:28.370402 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:31:28.370263    2565 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-ipp-a-3ab8a5-openshift-default-6d6b5486cd-vbv4x" podUID="fc7144e0-5c5f-4457-97ac-7dbc3c2ae54d" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Sep 18 14:31:28.414787 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:31:28.414754    2565 generic.go:356] "Generic (PLEG): container finished" podID="4b3af8b5-8002-4e15-b2ef-e9f45fb80d28" containerID="b95c30801f5691adb0dca8bc84dd002ac961e5bc039200c4c8df2597b43db732" exitCode=0
Sep 18 14:31:28.414958 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:31:28.414842    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-7ed186-gfgtq" event={"ID":"4b3af8b5-8002-4e15-b2ef-e9f45fb80d28","Type":"ContainerDied","Data":"b95c30801f5691adb0dca8bc84dd002ac961e5bc039200c4c8df2597b43db732"}
Sep 18 14:31:28.414958 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:31:28.414897    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-7ed186-gfgtq" event={"ID":"4b3af8b5-8002-4e15-b2ef-e9f45fb80d28","Type":"ContainerStarted","Data":"aea94aa0f6d59cb10ec7681a832e16cd7566affd6471638ab4e6fa1fed6a9a63"}
Sep 18 14:31:29.559827 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:31:29.559801    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-7ed186-gfgtq"
Sep 18 14:31:29.648965 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:31:29.648929    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/4b3af8b5-8002-4e15-b2ef-e9f45fb80d28-maas-api-service-ca\") pod \"4b3af8b5-8002-4e15-b2ef-e9f45fb80d28\" (UID: \"4b3af8b5-8002-4e15-b2ef-e9f45fb80d28\") "
Sep 18 14:31:29.649334 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:31:29.649309    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4b3af8b5-8002-4e15-b2ef-e9f45fb80d28-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "4b3af8b5-8002-4e15-b2ef-e9f45fb80d28" (UID: "4b3af8b5-8002-4e15-b2ef-e9f45fb80d28"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 18 14:31:29.750890 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:31:29.750803    2565 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/4b3af8b5-8002-4e15-b2ef-e9f45fb80d28-maas-api-service-ca\") on node \"ip-10-0-129-16.ec2.internal\" DevicePath \"\""
Sep 18 14:31:30.427470 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:31:30.427434    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-7ed186-gfgtq" event={"ID":"4b3af8b5-8002-4e15-b2ef-e9f45fb80d28","Type":"ContainerDied","Data":"aea94aa0f6d59cb10ec7681a832e16cd7566affd6471638ab4e6fa1fed6a9a63"}
Sep 18 14:31:30.427470 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:31:30.427474    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-7ed186-gfgtq"
Sep 18 14:31:30.427687 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:31:30.427515    2565 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="aea94aa0f6d59cb10ec7681a832e16cd7566affd6471638ab4e6fa1fed6a9a63"
Sep 18 14:31:31.915168 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:31:31.915135    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-a-3ab8a5-openshift-default-6d6b5486cd-vbv4x"
Sep 18 14:31:31.971747 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:31:31.971668    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/fc7144e0-5c5f-4457-97ac-7dbc3c2ae54d-istio-podinfo\") pod \"fc7144e0-5c5f-4457-97ac-7dbc3c2ae54d\" (UID: \"fc7144e0-5c5f-4457-97ac-7dbc3c2ae54d\") "
Sep 18 14:31:31.971747 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:31:31.971741    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/fc7144e0-5c5f-4457-97ac-7dbc3c2ae54d-credential-socket\") pod \"fc7144e0-5c5f-4457-97ac-7dbc3c2ae54d\" (UID: \"fc7144e0-5c5f-4457-97ac-7dbc3c2ae54d\") "
Sep 18 14:31:31.971960 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:31:31.971764    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/fc7144e0-5c5f-4457-97ac-7dbc3c2ae54d-workload-socket\") pod \"fc7144e0-5c5f-4457-97ac-7dbc3c2ae54d\" (UID: \"fc7144e0-5c5f-4457-97ac-7dbc3c2ae54d\") "
Sep 18 14:31:31.971960 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:31:31.971815    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/fc7144e0-5c5f-4457-97ac-7dbc3c2ae54d-istio-data\") pod \"fc7144e0-5c5f-4457-97ac-7dbc3c2ae54d\" (UID: \"fc7144e0-5c5f-4457-97ac-7dbc3c2ae54d\") "
Sep 18 14:31:31.971960 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:31:31.971839    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/fc7144e0-5c5f-4457-97ac-7dbc3c2ae54d-istiod-ca-cert\") pod \"fc7144e0-5c5f-4457-97ac-7dbc3c2ae54d\" (UID: \"fc7144e0-5c5f-4457-97ac-7dbc3c2ae54d\") "
Sep 18 14:31:31.971960 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:31:31.971873    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/fc7144e0-5c5f-4457-97ac-7dbc3c2ae54d-istio-envoy\") pod \"fc7144e0-5c5f-4457-97ac-7dbc3c2ae54d\" (UID: \"fc7144e0-5c5f-4457-97ac-7dbc3c2ae54d\") "
Sep 18 14:31:31.971960 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:31:31.971919    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/fc7144e0-5c5f-4457-97ac-7dbc3c2ae54d-istio-token\") pod \"fc7144e0-5c5f-4457-97ac-7dbc3c2ae54d\" (UID: \"fc7144e0-5c5f-4457-97ac-7dbc3c2ae54d\") "
Sep 18 14:31:31.971960 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:31:31.971947    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/fc7144e0-5c5f-4457-97ac-7dbc3c2ae54d-workload-certs\") pod \"fc7144e0-5c5f-4457-97ac-7dbc3c2ae54d\" (UID: \"fc7144e0-5c5f-4457-97ac-7dbc3c2ae54d\") "
Sep 18 14:31:31.972267 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:31:31.971971    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-gcrh6\" (UniqueName: \"kubernetes.io/projected/fc7144e0-5c5f-4457-97ac-7dbc3c2ae54d-kube-api-access-gcrh6\") pod \"fc7144e0-5c5f-4457-97ac-7dbc3c2ae54d\" (UID: \"fc7144e0-5c5f-4457-97ac-7dbc3c2ae54d\") "
Sep 18 14:31:31.972267 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:31:31.972019    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fc7144e0-5c5f-4457-97ac-7dbc3c2ae54d-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "fc7144e0-5c5f-4457-97ac-7dbc3c2ae54d" (UID: "fc7144e0-5c5f-4457-97ac-7dbc3c2ae54d"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 18 14:31:31.972267 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:31:31.972210    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fc7144e0-5c5f-4457-97ac-7dbc3c2ae54d-istio-data" (OuterVolumeSpecName: "istio-data") pod "fc7144e0-5c5f-4457-97ac-7dbc3c2ae54d" (UID: "fc7144e0-5c5f-4457-97ac-7dbc3c2ae54d"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 18 14:31:31.972434 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:31:31.972360    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fc7144e0-5c5f-4457-97ac-7dbc3c2ae54d-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "fc7144e0-5c5f-4457-97ac-7dbc3c2ae54d" (UID: "fc7144e0-5c5f-4457-97ac-7dbc3c2ae54d"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 18 14:31:31.972434 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:31:31.972373    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/fc7144e0-5c5f-4457-97ac-7dbc3c2ae54d-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "fc7144e0-5c5f-4457-97ac-7dbc3c2ae54d" (UID: "fc7144e0-5c5f-4457-97ac-7dbc3c2ae54d"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 18 14:31:31.973215 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:31:31.972607    2565 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/fc7144e0-5c5f-4457-97ac-7dbc3c2ae54d-credential-socket\") on node \"ip-10-0-129-16.ec2.internal\" DevicePath \"\""
Sep 18 14:31:31.973215 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:31:31.972632    2565 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/fc7144e0-5c5f-4457-97ac-7dbc3c2ae54d-istio-data\") on node \"ip-10-0-129-16.ec2.internal\" DevicePath \"\""
Sep 18 14:31:31.973215 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:31:31.972646    2565 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/fc7144e0-5c5f-4457-97ac-7dbc3c2ae54d-istiod-ca-cert\") on node \"ip-10-0-129-16.ec2.internal\" DevicePath \"\""
Sep 18 14:31:31.973215 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:31:31.972660    2565 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/fc7144e0-5c5f-4457-97ac-7dbc3c2ae54d-workload-certs\") on node \"ip-10-0-129-16.ec2.internal\" DevicePath \"\""
Sep 18 14:31:31.973215 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:31:31.973178    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fc7144e0-5c5f-4457-97ac-7dbc3c2ae54d-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "fc7144e0-5c5f-4457-97ac-7dbc3c2ae54d" (UID: "fc7144e0-5c5f-4457-97ac-7dbc3c2ae54d"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 18 14:31:31.974735 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:31:31.974709    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fc7144e0-5c5f-4457-97ac-7dbc3c2ae54d-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "fc7144e0-5c5f-4457-97ac-7dbc3c2ae54d" (UID: "fc7144e0-5c5f-4457-97ac-7dbc3c2ae54d"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 18 14:31:31.974869 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:31:31.974759    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fc7144e0-5c5f-4457-97ac-7dbc3c2ae54d-kube-api-access-gcrh6" (OuterVolumeSpecName: "kube-api-access-gcrh6") pod "fc7144e0-5c5f-4457-97ac-7dbc3c2ae54d" (UID: "fc7144e0-5c5f-4457-97ac-7dbc3c2ae54d"). InnerVolumeSpecName "kube-api-access-gcrh6". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 18 14:31:31.975023 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:31:31.974989    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fc7144e0-5c5f-4457-97ac-7dbc3c2ae54d-istio-token" (OuterVolumeSpecName: "istio-token") pod "fc7144e0-5c5f-4457-97ac-7dbc3c2ae54d" (UID: "fc7144e0-5c5f-4457-97ac-7dbc3c2ae54d"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 18 14:31:31.975419 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:31:31.975397    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/fc7144e0-5c5f-4457-97ac-7dbc3c2ae54d-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "fc7144e0-5c5f-4457-97ac-7dbc3c2ae54d" (UID: "fc7144e0-5c5f-4457-97ac-7dbc3c2ae54d"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 18 14:31:32.074129 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:31:32.074092    2565 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/fc7144e0-5c5f-4457-97ac-7dbc3c2ae54d-istio-podinfo\") on node \"ip-10-0-129-16.ec2.internal\" DevicePath \"\""
Sep 18 14:31:32.074129 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:31:32.074121    2565 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/fc7144e0-5c5f-4457-97ac-7dbc3c2ae54d-workload-socket\") on node \"ip-10-0-129-16.ec2.internal\" DevicePath \"\""
Sep 18 14:31:32.074129 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:31:32.074133    2565 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/fc7144e0-5c5f-4457-97ac-7dbc3c2ae54d-istio-envoy\") on node \"ip-10-0-129-16.ec2.internal\" DevicePath \"\""
Sep 18 14:31:32.074129 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:31:32.074141    2565 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/fc7144e0-5c5f-4457-97ac-7dbc3c2ae54d-istio-token\") on node \"ip-10-0-129-16.ec2.internal\" DevicePath \"\""
Sep 18 14:31:32.074405 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:31:32.074150    2565 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-gcrh6\" (UniqueName: \"kubernetes.io/projected/fc7144e0-5c5f-4457-97ac-7dbc3c2ae54d-kube-api-access-gcrh6\") on node \"ip-10-0-129-16.ec2.internal\" DevicePath \"\""
Sep 18 14:31:32.441631 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:31:32.441588    2565 generic.go:356] "Generic (PLEG): container finished" podID="fc7144e0-5c5f-4457-97ac-7dbc3c2ae54d" containerID="ba674a9297a558984f8d2c00425bfccf759dd26b2a55d73c63600d420b7a047d" exitCode=0
Sep 18 14:31:32.441805 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:31:32.441643    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-a-3ab8a5-openshift-default-6d6b5486cd-vbv4x"
Sep 18 14:31:32.441805 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:31:32.441674    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-a-3ab8a5-openshift-default-6d6b5486cd-vbv4x" event={"ID":"fc7144e0-5c5f-4457-97ac-7dbc3c2ae54d","Type":"ContainerDied","Data":"ba674a9297a558984f8d2c00425bfccf759dd26b2a55d73c63600d420b7a047d"}
Sep 18 14:31:32.441805 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:31:32.441730    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-a-3ab8a5-openshift-default-6d6b5486cd-vbv4x" event={"ID":"fc7144e0-5c5f-4457-97ac-7dbc3c2ae54d","Type":"ContainerDied","Data":"8859f67407976c5f5c9285d6844bc6c3f064b7fa93683fd76f878769d7c2f53d"}
Sep 18 14:31:32.441805 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:31:32.441754    2565 scope.go:117] "RemoveContainer" containerID="ba674a9297a558984f8d2c00425bfccf759dd26b2a55d73c63600d420b7a047d"
Sep 18 14:31:32.451239 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:31:32.451218    2565 scope.go:117] "RemoveContainer" containerID="d2583c5faa5be01fe42560fb39aa54676f29bad44f7c98018189f2766284688b"
Sep 18 14:31:32.459825 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:31:32.459807    2565 scope.go:117] "RemoveContainer" containerID="ba674a9297a558984f8d2c00425bfccf759dd26b2a55d73c63600d420b7a047d"
Sep 18 14:31:32.460126 ip-10-0-129-16 kubenswrapper[2565]: E0918 14:31:32.460096    2565 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ba674a9297a558984f8d2c00425bfccf759dd26b2a55d73c63600d420b7a047d\": container with ID starting with ba674a9297a558984f8d2c00425bfccf759dd26b2a55d73c63600d420b7a047d not found: ID does not exist" containerID="ba674a9297a558984f8d2c00425bfccf759dd26b2a55d73c63600d420b7a047d"
Sep 18 14:31:32.460189 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:31:32.460136    2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ba674a9297a558984f8d2c00425bfccf759dd26b2a55d73c63600d420b7a047d"} err="failed to get container status \"ba674a9297a558984f8d2c00425bfccf759dd26b2a55d73c63600d420b7a047d\": rpc error: code = NotFound desc = could not find container \"ba674a9297a558984f8d2c00425bfccf759dd26b2a55d73c63600d420b7a047d\": container with ID starting with ba674a9297a558984f8d2c00425bfccf759dd26b2a55d73c63600d420b7a047d not found: ID does not exist"
Sep 18 14:31:32.460189 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:31:32.460152    2565 scope.go:117] "RemoveContainer" containerID="d2583c5faa5be01fe42560fb39aa54676f29bad44f7c98018189f2766284688b"
Sep 18 14:31:32.460429 ip-10-0-129-16 kubenswrapper[2565]: E0918 14:31:32.460413    2565 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d2583c5faa5be01fe42560fb39aa54676f29bad44f7c98018189f2766284688b\": container with ID starting with d2583c5faa5be01fe42560fb39aa54676f29bad44f7c98018189f2766284688b not found: ID does not exist" containerID="d2583c5faa5be01fe42560fb39aa54676f29bad44f7c98018189f2766284688b"
Sep 18 14:31:32.460480 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:31:32.460433    2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d2583c5faa5be01fe42560fb39aa54676f29bad44f7c98018189f2766284688b"} err="failed to get container status \"d2583c5faa5be01fe42560fb39aa54676f29bad44f7c98018189f2766284688b\": rpc error: code = NotFound desc = could not find container \"d2583c5faa5be01fe42560fb39aa54676f29bad44f7c98018189f2766284688b\": container with ID starting with d2583c5faa5be01fe42560fb39aa54676f29bad44f7c98018189f2766284688b not found: ID does not exist"
Sep 18 14:31:32.471103 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:31:32.471080    2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ipp-a-3ab8a5-openshift-default-6d6b5486cd-vbv4x"]
Sep 18 14:31:32.482349 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:31:32.482323    2565 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ipp-a-3ab8a5-openshift-default-6d6b5486cd-vbv4x"]
Sep 18 14:31:33.913318 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:31:33.913283    2565 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="fc7144e0-5c5f-4457-97ac-7dbc3c2ae54d" path="/var/lib/kubelet/pods/fc7144e0-5c5f-4457-97ac-7dbc3c2ae54d/volumes"
Sep 18 14:31:37.771934 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:31:37.771855    2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ipp-b-7ed186-29829030-jphs4"]
Sep 18 14:31:37.781292 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:31:37.781267    2565 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ipp-b-7ed186-29829030-jphs4"]
Sep 18 14:31:37.825997 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:31:37.825965    2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-b-7ed186-765f59c65-wp8v9"]
Sep 18 14:31:37.826235 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:31:37.826204    2565 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-ipp-b-7ed186-765f59c65-wp8v9" podUID="00cd7d42-6964-4908-bcfb-4f42895892de" containerName="payload-processing" containerID="cri-o://8bcdcf7338cb3222c40cfd1282fd3aef5aa321426e762ec9f22e581eef850997" gracePeriod=30
Sep 18 14:31:37.912542 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:31:37.912509    2565 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0302137b-5d71-44b0-9ff5-e0ada9d2a8cb" path="/var/lib/kubelet/pods/0302137b-5d71-44b0-9ff5-e0ada9d2a8cb/volumes"
Sep 18 14:31:38.082696 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:31:38.082669    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-b-7ed186-765f59c65-wp8v9"
Sep 18 14:31:38.135354 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:31:38.135325    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/00cd7d42-6964-4908-bcfb-4f42895892de-plugins-config-volume\") pod \"00cd7d42-6964-4908-bcfb-4f42895892de\" (UID: \"00cd7d42-6964-4908-bcfb-4f42895892de\") "
Sep 18 14:31:38.135354 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:31:38.135361    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-klcsr\" (UniqueName: \"kubernetes.io/projected/00cd7d42-6964-4908-bcfb-4f42895892de-kube-api-access-klcsr\") pod \"00cd7d42-6964-4908-bcfb-4f42895892de\" (UID: \"00cd7d42-6964-4908-bcfb-4f42895892de\") "
Sep 18 14:31:38.135790 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:31:38.135760    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/00cd7d42-6964-4908-bcfb-4f42895892de-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "00cd7d42-6964-4908-bcfb-4f42895892de" (UID: "00cd7d42-6964-4908-bcfb-4f42895892de"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 18 14:31:38.137590 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:31:38.137556    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/00cd7d42-6964-4908-bcfb-4f42895892de-kube-api-access-klcsr" (OuterVolumeSpecName: "kube-api-access-klcsr") pod "00cd7d42-6964-4908-bcfb-4f42895892de" (UID: "00cd7d42-6964-4908-bcfb-4f42895892de"). InnerVolumeSpecName "kube-api-access-klcsr". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 18 14:31:38.202129 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:31:38.202096    2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-7ed186-gfgtq"]
Sep 18 14:31:38.209897 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:31:38.209865    2565 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-7ed186-gfgtq"]
Sep 18 14:31:38.237147 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:31:38.237109    2565 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/00cd7d42-6964-4908-bcfb-4f42895892de-plugins-config-volume\") on node \"ip-10-0-129-16.ec2.internal\" DevicePath \"\""
Sep 18 14:31:38.237147 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:31:38.237138    2565 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-klcsr\" (UniqueName: \"kubernetes.io/projected/00cd7d42-6964-4908-bcfb-4f42895892de-kube-api-access-klcsr\") on node \"ip-10-0-129-16.ec2.internal\" DevicePath \"\""
Sep 18 14:31:38.485361 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:31:38.485263    2565 generic.go:356] "Generic (PLEG): container finished" podID="00cd7d42-6964-4908-bcfb-4f42895892de" containerID="8bcdcf7338cb3222c40cfd1282fd3aef5aa321426e762ec9f22e581eef850997" exitCode=0
Sep 18 14:31:38.485361 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:31:38.485311    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-b-7ed186-765f59c65-wp8v9"
Sep 18 14:31:38.485361 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:31:38.485347    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-b-7ed186-765f59c65-wp8v9" event={"ID":"00cd7d42-6964-4908-bcfb-4f42895892de","Type":"ContainerDied","Data":"8bcdcf7338cb3222c40cfd1282fd3aef5aa321426e762ec9f22e581eef850997"}
Sep 18 14:31:38.485648 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:31:38.485419    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-b-7ed186-765f59c65-wp8v9" event={"ID":"00cd7d42-6964-4908-bcfb-4f42895892de","Type":"ContainerDied","Data":"bf05291d17d5e36107827f048ecb7d470e3aa5a6458d43d58ec200d3f097f521"}
Sep 18 14:31:38.485648 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:31:38.485449    2565 scope.go:117] "RemoveContainer" containerID="8bcdcf7338cb3222c40cfd1282fd3aef5aa321426e762ec9f22e581eef850997"
Sep 18 14:31:38.494798 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:31:38.494778    2565 scope.go:117] "RemoveContainer" containerID="8bcdcf7338cb3222c40cfd1282fd3aef5aa321426e762ec9f22e581eef850997"
Sep 18 14:31:38.495093 ip-10-0-129-16 kubenswrapper[2565]: E0918 14:31:38.495076    2565 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8bcdcf7338cb3222c40cfd1282fd3aef5aa321426e762ec9f22e581eef850997\": container with ID starting with 8bcdcf7338cb3222c40cfd1282fd3aef5aa321426e762ec9f22e581eef850997 not found: ID does not exist" containerID="8bcdcf7338cb3222c40cfd1282fd3aef5aa321426e762ec9f22e581eef850997"
Sep 18 14:31:38.495158 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:31:38.495101    2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8bcdcf7338cb3222c40cfd1282fd3aef5aa321426e762ec9f22e581eef850997"} err="failed to get container status \"8bcdcf7338cb3222c40cfd1282fd3aef5aa321426e762ec9f22e581eef850997\": rpc error: code = NotFound desc = could not find container \"8bcdcf7338cb3222c40cfd1282fd3aef5aa321426e762ec9f22e581eef850997\": container with ID starting with 8bcdcf7338cb3222c40cfd1282fd3aef5aa321426e762ec9f22e581eef850997 not found: ID does not exist"
Sep 18 14:31:38.523797 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:31:38.523766    2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-b-7ed186-765f59c65-wp8v9"]
Sep 18 14:31:38.537254 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:31:38.537222    2565 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-b-7ed186-765f59c65-wp8v9"]
Sep 18 14:31:39.913503 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:31:39.913468    2565 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="00cd7d42-6964-4908-bcfb-4f42895892de" path="/var/lib/kubelet/pods/00cd7d42-6964-4908-bcfb-4f42895892de/volumes"
Sep 18 14:31:39.913867 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:31:39.913815    2565 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4b3af8b5-8002-4e15-b2ef-e9f45fb80d28" path="/var/lib/kubelet/pods/4b3af8b5-8002-4e15-b2ef-e9f45fb80d28/volumes"
Sep 18 14:31:45.973844 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:31:45.973809    2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w3-ae54af-xfpxw"]
Sep 18 14:31:45.974487 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:31:45.974469    2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="fc7144e0-5c5f-4457-97ac-7dbc3c2ae54d" containerName="istio-proxy"
Sep 18 14:31:45.974535 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:31:45.974490    2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="fc7144e0-5c5f-4457-97ac-7dbc3c2ae54d" containerName="istio-proxy"
Sep 18 14:31:45.974535 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:31:45.974503    2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="00cd7d42-6964-4908-bcfb-4f42895892de" containerName="payload-processing"
Sep 18 14:31:45.974535 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:31:45.974511    2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="00cd7d42-6964-4908-bcfb-4f42895892de" containerName="payload-processing"
Sep 18 14:31:45.974629 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:31:45.974552    2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4b3af8b5-8002-4e15-b2ef-e9f45fb80d28" containerName="revoke-keys"
Sep 18 14:31:45.974629 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:31:45.974561    2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="4b3af8b5-8002-4e15-b2ef-e9f45fb80d28" containerName="revoke-keys"
Sep 18 14:31:45.974707 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:31:45.974665    2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="fc7144e0-5c5f-4457-97ac-7dbc3c2ae54d" containerName="istio-proxy"
Sep 18 14:31:45.974707 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:31:45.974677    2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="fc7144e0-5c5f-4457-97ac-7dbc3c2ae54d" containerName="istio-proxy"
Sep 18 14:31:45.974707 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:31:45.974687    2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="4b3af8b5-8002-4e15-b2ef-e9f45fb80d28" containerName="revoke-keys"
Sep 18 14:31:45.974707 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:31:45.974697    2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="00cd7d42-6964-4908-bcfb-4f42895892de" containerName="payload-processing"
Sep 18 14:31:45.979570 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:31:45.979552    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w3-ae54af-xfpxw"
Sep 18 14:31:45.982311 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:31:45.982289    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-l4fn9\""
Sep 18 14:31:45.988531 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:31:45.988506    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w3-ae54af-xfpxw"]
Sep 18 14:31:46.120131 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:31:46.120085    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/dfce5faa-4ac8-4d14-9468-d856773584b0-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-a-w3-ae54af-xfpxw\" (UID: \"dfce5faa-4ac8-4d14-9468-d856773584b0\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w3-ae54af-xfpxw"
Sep 18 14:31:46.220953 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:31:46.220915    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/dfce5faa-4ac8-4d14-9468-d856773584b0-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-a-w3-ae54af-xfpxw\" (UID: \"dfce5faa-4ac8-4d14-9468-d856773584b0\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w3-ae54af-xfpxw"
Sep 18 14:31:46.221590 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:31:46.221565    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/dfce5faa-4ac8-4d14-9468-d856773584b0-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-a-w3-ae54af-xfpxw\" (UID: \"dfce5faa-4ac8-4d14-9468-d856773584b0\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w3-ae54af-xfpxw"
Sep 18 14:31:46.291413 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:31:46.291376    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w3-ae54af-xfpxw"
Sep 18 14:31:46.576434 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:31:46.576405    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w3-ae54af-xfpxw"]
Sep 18 14:31:46.578790 ip-10-0-129-16 kubenswrapper[2565]: W0918 14:31:46.578762    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddfce5faa_4ac8_4d14_9468_d856773584b0.slice/crio-bc85c2341b87851938b37c7a998c563cc207a7802d127c3138dc858294863b0f WatchSource:0}: Error finding container bc85c2341b87851938b37c7a998c563cc207a7802d127c3138dc858294863b0f: Status 404 returned error can't find the container with id bc85c2341b87851938b37c7a998c563cc207a7802d127c3138dc858294863b0f
Sep 18 14:31:47.539658 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:31:47.539618    2565 generic.go:356] "Generic (PLEG): container finished" podID="dfce5faa-4ac8-4d14-9468-d856773584b0" containerID="c344e27ecd6f9dea1bbf11cb9f236e18e9914b2f82bc0618165057ee4a70eae3" exitCode=0
Sep 18 14:31:47.540143 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:31:47.539718    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w3-ae54af-xfpxw" event={"ID":"dfce5faa-4ac8-4d14-9468-d856773584b0","Type":"ContainerDied","Data":"c344e27ecd6f9dea1bbf11cb9f236e18e9914b2f82bc0618165057ee4a70eae3"}
Sep 18 14:31:47.540143 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:31:47.539770    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w3-ae54af-xfpxw" event={"ID":"dfce5faa-4ac8-4d14-9468-d856773584b0","Type":"ContainerStarted","Data":"bc85c2341b87851938b37c7a998c563cc207a7802d127c3138dc858294863b0f"}
Sep 18 14:31:48.681297 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:31:48.681274    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w3-ae54af-xfpxw"
Sep 18 14:31:48.747503 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:31:48.747468    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/dfce5faa-4ac8-4d14-9468-d856773584b0-maas-api-service-ca\") pod \"dfce5faa-4ac8-4d14-9468-d856773584b0\" (UID: \"dfce5faa-4ac8-4d14-9468-d856773584b0\") "
Sep 18 14:31:48.747863 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:31:48.747841    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/dfce5faa-4ac8-4d14-9468-d856773584b0-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "dfce5faa-4ac8-4d14-9468-d856773584b0" (UID: "dfce5faa-4ac8-4d14-9468-d856773584b0"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 18 14:31:48.849574 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:31:48.849483    2565 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/dfce5faa-4ac8-4d14-9468-d856773584b0-maas-api-service-ca\") on node \"ip-10-0-129-16.ec2.internal\" DevicePath \"\""
Sep 18 14:31:49.552763 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:31:49.552735    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w3-ae54af-xfpxw"
Sep 18 14:31:49.552944 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:31:49.552771    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w3-ae54af-xfpxw" event={"ID":"dfce5faa-4ac8-4d14-9468-d856773584b0","Type":"ContainerDied","Data":"bc85c2341b87851938b37c7a998c563cc207a7802d127c3138dc858294863b0f"}
Sep 18 14:31:49.552944 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:31:49.552820    2565 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="bc85c2341b87851938b37c7a998c563cc207a7802d127c3138dc858294863b0f"
Sep 18 14:31:56.547678 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:31:56.547642    2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w3-ae54af-5cc95698b7-w4fhd"]
Sep 18 14:31:56.548240 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:31:56.547873    2565 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-shared-a-w3-ae54af-5cc95698b7-w4fhd" podUID="6b1f9787-10c9-41a0-9244-a51cb19b61fc" containerName="payload-processing" containerID="cri-o://4b700e7515dbdecf1ef9eb000646255dee79cd63203d3f7005363c92f715e278" gracePeriod=30
Sep 18 14:31:56.802137 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:31:56.802072    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-a-w3-ae54af-5cc95698b7-w4fhd"
Sep 18 14:31:56.820927 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:31:56.820887    2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w3-ae54af-xfpxw"]
Sep 18 14:31:56.826678 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:31:56.826650    2565 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w3-ae54af-xfpxw"]
Sep 18 14:31:56.935186 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:31:56.935157    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-4qc9t\" (UniqueName: \"kubernetes.io/projected/6b1f9787-10c9-41a0-9244-a51cb19b61fc-kube-api-access-4qc9t\") pod \"6b1f9787-10c9-41a0-9244-a51cb19b61fc\" (UID: \"6b1f9787-10c9-41a0-9244-a51cb19b61fc\") "
Sep 18 14:31:56.935391 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:31:56.935306    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/6b1f9787-10c9-41a0-9244-a51cb19b61fc-plugins-config-volume\") pod \"6b1f9787-10c9-41a0-9244-a51cb19b61fc\" (UID: \"6b1f9787-10c9-41a0-9244-a51cb19b61fc\") "
Sep 18 14:31:56.935740 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:31:56.935705    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6b1f9787-10c9-41a0-9244-a51cb19b61fc-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "6b1f9787-10c9-41a0-9244-a51cb19b61fc" (UID: "6b1f9787-10c9-41a0-9244-a51cb19b61fc"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 18 14:31:56.937533 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:31:56.937500    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6b1f9787-10c9-41a0-9244-a51cb19b61fc-kube-api-access-4qc9t" (OuterVolumeSpecName: "kube-api-access-4qc9t") pod "6b1f9787-10c9-41a0-9244-a51cb19b61fc" (UID: "6b1f9787-10c9-41a0-9244-a51cb19b61fc"). InnerVolumeSpecName "kube-api-access-4qc9t". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 18 14:31:57.036795 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:31:57.036751    2565 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4qc9t\" (UniqueName: \"kubernetes.io/projected/6b1f9787-10c9-41a0-9244-a51cb19b61fc-kube-api-access-4qc9t\") on node \"ip-10-0-129-16.ec2.internal\" DevicePath \"\""
Sep 18 14:31:57.036795 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:31:57.036782    2565 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/6b1f9787-10c9-41a0-9244-a51cb19b61fc-plugins-config-volume\") on node \"ip-10-0-129-16.ec2.internal\" DevicePath \"\""
Sep 18 14:31:57.599206 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:31:57.599156    2565 generic.go:356] "Generic (PLEG): container finished" podID="6b1f9787-10c9-41a0-9244-a51cb19b61fc" containerID="4b700e7515dbdecf1ef9eb000646255dee79cd63203d3f7005363c92f715e278" exitCode=0
Sep 18 14:31:57.599206 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:31:57.599208    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-a-w3-ae54af-5cc95698b7-w4fhd"
Sep 18 14:31:57.599689 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:31:57.599238    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-a-w3-ae54af-5cc95698b7-w4fhd" event={"ID":"6b1f9787-10c9-41a0-9244-a51cb19b61fc","Type":"ContainerDied","Data":"4b700e7515dbdecf1ef9eb000646255dee79cd63203d3f7005363c92f715e278"}
Sep 18 14:31:57.599689 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:31:57.599298    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-a-w3-ae54af-5cc95698b7-w4fhd" event={"ID":"6b1f9787-10c9-41a0-9244-a51cb19b61fc","Type":"ContainerDied","Data":"b2c6b0273e6abf25adf29f335def06ae54b58775f7f7ec651663cb685a435cba"}
Sep 18 14:31:57.599689 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:31:57.599322    2565 scope.go:117] "RemoveContainer" containerID="4b700e7515dbdecf1ef9eb000646255dee79cd63203d3f7005363c92f715e278"
Sep 18 14:31:57.609191 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:31:57.609165    2565 scope.go:117] "RemoveContainer" containerID="4b700e7515dbdecf1ef9eb000646255dee79cd63203d3f7005363c92f715e278"
Sep 18 14:31:57.609451 ip-10-0-129-16 kubenswrapper[2565]: E0918 14:31:57.609433    2565 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4b700e7515dbdecf1ef9eb000646255dee79cd63203d3f7005363c92f715e278\": container with ID starting with 4b700e7515dbdecf1ef9eb000646255dee79cd63203d3f7005363c92f715e278 not found: ID does not exist" containerID="4b700e7515dbdecf1ef9eb000646255dee79cd63203d3f7005363c92f715e278"
Sep 18 14:31:57.609504 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:31:57.609459    2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4b700e7515dbdecf1ef9eb000646255dee79cd63203d3f7005363c92f715e278"} err="failed to get container status \"4b700e7515dbdecf1ef9eb000646255dee79cd63203d3f7005363c92f715e278\": rpc error: code = NotFound desc = could not find container \"4b700e7515dbdecf1ef9eb000646255dee79cd63203d3f7005363c92f715e278\": container with ID starting with 4b700e7515dbdecf1ef9eb000646255dee79cd63203d3f7005363c92f715e278 not found: ID does not exist"
Sep 18 14:31:57.623341 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:31:57.623304    2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w3-ae54af-5cc95698b7-w4fhd"]
Sep 18 14:31:57.627455 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:31:57.627433    2565 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w3-ae54af-5cc95698b7-w4fhd"]
Sep 18 14:31:57.913917 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:31:57.913834    2565 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6b1f9787-10c9-41a0-9244-a51cb19b61fc" path="/var/lib/kubelet/pods/6b1f9787-10c9-41a0-9244-a51cb19b61fc/volumes"
Sep 18 14:31:57.914204 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:31:57.914191    2565 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="dfce5faa-4ac8-4d14-9468-d856773584b0" path="/var/lib/kubelet/pods/dfce5faa-4ac8-4d14-9468-d856773584b0/volumes"
Sep 18 14:32:04.424407 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:32:04.424347    2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w3-980661-8v9cj"]
Sep 18 14:32:04.424842 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:32:04.424828    2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6b1f9787-10c9-41a0-9244-a51cb19b61fc" containerName="payload-processing"
Sep 18 14:32:04.424842 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:32:04.424843    2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="6b1f9787-10c9-41a0-9244-a51cb19b61fc" containerName="payload-processing"
Sep 18 14:32:04.424924 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:32:04.424860    2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="dfce5faa-4ac8-4d14-9468-d856773584b0" containerName="revoke-keys"
Sep 18 14:32:04.424924 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:32:04.424867    2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="dfce5faa-4ac8-4d14-9468-d856773584b0" containerName="revoke-keys"
Sep 18 14:32:04.424924 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:32:04.424886    2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="fc7144e0-5c5f-4457-97ac-7dbc3c2ae54d" containerName="istio-proxy"
Sep 18 14:32:04.424924 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:32:04.424892    2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="fc7144e0-5c5f-4457-97ac-7dbc3c2ae54d" containerName="istio-proxy"
Sep 18 14:32:04.425077 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:32:04.424959    2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="6b1f9787-10c9-41a0-9244-a51cb19b61fc" containerName="payload-processing"
Sep 18 14:32:04.425077 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:32:04.424968    2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="dfce5faa-4ac8-4d14-9468-d856773584b0" containerName="revoke-keys"
Sep 18 14:32:04.429314 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:32:04.429294    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w3-980661-8v9cj"
Sep 18 14:32:04.431945 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:32:04.431922    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-l4fn9\""
Sep 18 14:32:04.440414 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:32:04.440390    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w3-980661-8v9cj"]
Sep 18 14:32:04.511487 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:32:04.511449    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/80e9da7f-1054-4347-a3bb-4d410e230c50-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-b-w3-980661-8v9cj\" (UID: \"80e9da7f-1054-4347-a3bb-4d410e230c50\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w3-980661-8v9cj"
Sep 18 14:32:04.613058 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:32:04.612982    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/80e9da7f-1054-4347-a3bb-4d410e230c50-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-b-w3-980661-8v9cj\" (UID: \"80e9da7f-1054-4347-a3bb-4d410e230c50\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w3-980661-8v9cj"
Sep 18 14:32:04.613631 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:32:04.613600    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/80e9da7f-1054-4347-a3bb-4d410e230c50-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-b-w3-980661-8v9cj\" (UID: \"80e9da7f-1054-4347-a3bb-4d410e230c50\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w3-980661-8v9cj"
Sep 18 14:32:04.740916 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:32:04.740820    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w3-980661-8v9cj"
Sep 18 14:32:05.015609 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:32:05.015584    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w3-980661-8v9cj"]
Sep 18 14:32:05.017673 ip-10-0-129-16 kubenswrapper[2565]: W0918 14:32:05.017640    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod80e9da7f_1054_4347_a3bb_4d410e230c50.slice/crio-08e9691d907e5d43058a47eff766f972b22b32a8de16437d940e124207e5d27b WatchSource:0}: Error finding container 08e9691d907e5d43058a47eff766f972b22b32a8de16437d940e124207e5d27b: Status 404 returned error can't find the container with id 08e9691d907e5d43058a47eff766f972b22b32a8de16437d940e124207e5d27b
Sep 18 14:32:05.649408 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:32:05.649372    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w3-980661-8v9cj" event={"ID":"80e9da7f-1054-4347-a3bb-4d410e230c50","Type":"ContainerStarted","Data":"0c2f53c557a45208e804f5e40a2ee4d5eebc32c56eca7698850b8bd3604389d3"}
Sep 18 14:32:05.649749 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:32:05.649440    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w3-980661-8v9cj" event={"ID":"80e9da7f-1054-4347-a3bb-4d410e230c50","Type":"ContainerStarted","Data":"08e9691d907e5d43058a47eff766f972b22b32a8de16437d940e124207e5d27b"}
Sep 18 14:32:05.675502 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:32:05.675454    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w3-980661-8v9cj" podStartSLOduration=1.6754405060000002 podStartE2EDuration="1.675440506s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 14:32:04 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 14:32:05.672510639 +0000 UTC m=+1993.846420298" watchObservedRunningTime="2026-09-18 14:32:05.675440506 +0000 UTC m=+1993.849350165"
Sep 18 14:32:06.656228 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:32:06.656193    2565 generic.go:356] "Generic (PLEG): container finished" podID="80e9da7f-1054-4347-a3bb-4d410e230c50" containerID="0c2f53c557a45208e804f5e40a2ee4d5eebc32c56eca7698850b8bd3604389d3" exitCode=0
Sep 18 14:32:06.656612 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:32:06.656286    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w3-980661-8v9cj" event={"ID":"80e9da7f-1054-4347-a3bb-4d410e230c50","Type":"ContainerDied","Data":"0c2f53c557a45208e804f5e40a2ee4d5eebc32c56eca7698850b8bd3604389d3"}
Sep 18 14:32:07.799403 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:32:07.799374    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w3-980661-8v9cj"
Sep 18 14:32:07.848245 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:32:07.848215    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/80e9da7f-1054-4347-a3bb-4d410e230c50-maas-api-service-ca\") pod \"80e9da7f-1054-4347-a3bb-4d410e230c50\" (UID: \"80e9da7f-1054-4347-a3bb-4d410e230c50\") "
Sep 18 14:32:07.848605 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:32:07.848582    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/80e9da7f-1054-4347-a3bb-4d410e230c50-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "80e9da7f-1054-4347-a3bb-4d410e230c50" (UID: "80e9da7f-1054-4347-a3bb-4d410e230c50"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 18 14:32:07.949779 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:32:07.949709    2565 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/80e9da7f-1054-4347-a3bb-4d410e230c50-maas-api-service-ca\") on node \"ip-10-0-129-16.ec2.internal\" DevicePath \"\""
Sep 18 14:32:08.668745 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:32:08.668707    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w3-980661-8v9cj" event={"ID":"80e9da7f-1054-4347-a3bb-4d410e230c50","Type":"ContainerDied","Data":"08e9691d907e5d43058a47eff766f972b22b32a8de16437d940e124207e5d27b"}
Sep 18 14:32:08.668745 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:32:08.668735    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w3-980661-8v9cj"
Sep 18 14:32:08.668745 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:32:08.668754    2565 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="08e9691d907e5d43058a47eff766f972b22b32a8de16437d940e124207e5d27b"
Sep 18 14:32:15.288177 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:32:15.288134    2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w3-980661-8v9cj"]
Sep 18 14:32:15.290313 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:32:15.290289    2565 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w3-980661-8v9cj"]
Sep 18 14:32:15.912857 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:32:15.912820    2565 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="80e9da7f-1054-4347-a3bb-4d410e230c50" path="/var/lib/kubelet/pods/80e9da7f-1054-4347-a3bb-4d410e230c50/volumes"
Sep 18 14:32:31.297957 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:32:31.297917    2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-765845fd55-t69bn"]
Sep 18 14:32:31.298476 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:32:31.298213    2565 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/maas-controller-765845fd55-t69bn" podUID="d24cf07c-4ff3-4dd9-b06f-8cc8ac4c12f8" containerName="manager" containerID="cri-o://c93f47c21dde70d9a522d79f94ff2fd0c4476e0175408f91c659eed8d70b14a7" gracePeriod=30
Sep 18 14:32:31.560090 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:32:31.560007    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-765845fd55-t69bn"
Sep 18 14:32:31.685891 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:32:31.685853    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-k988t\" (UniqueName: \"kubernetes.io/projected/d24cf07c-4ff3-4dd9-b06f-8cc8ac4c12f8-kube-api-access-k988t\") pod \"d24cf07c-4ff3-4dd9-b06f-8cc8ac4c12f8\" (UID: \"d24cf07c-4ff3-4dd9-b06f-8cc8ac4c12f8\") "
Sep 18 14:32:31.686086 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:32:31.686009    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/d24cf07c-4ff3-4dd9-b06f-8cc8ac4c12f8-webhook-cert\") pod \"d24cf07c-4ff3-4dd9-b06f-8cc8ac4c12f8\" (UID: \"d24cf07c-4ff3-4dd9-b06f-8cc8ac4c12f8\") "
Sep 18 14:32:31.686086 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:32:31.686057    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d24cf07c-4ff3-4dd9-b06f-8cc8ac4c12f8-metrics-certs\") pod \"d24cf07c-4ff3-4dd9-b06f-8cc8ac4c12f8\" (UID: \"d24cf07c-4ff3-4dd9-b06f-8cc8ac4c12f8\") "
Sep 18 14:32:31.688195 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:32:31.688163    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d24cf07c-4ff3-4dd9-b06f-8cc8ac4c12f8-kube-api-access-k988t" (OuterVolumeSpecName: "kube-api-access-k988t") pod "d24cf07c-4ff3-4dd9-b06f-8cc8ac4c12f8" (UID: "d24cf07c-4ff3-4dd9-b06f-8cc8ac4c12f8"). InnerVolumeSpecName "kube-api-access-k988t". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 18 14:32:31.688195 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:32:31.688184    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d24cf07c-4ff3-4dd9-b06f-8cc8ac4c12f8-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "d24cf07c-4ff3-4dd9-b06f-8cc8ac4c12f8" (UID: "d24cf07c-4ff3-4dd9-b06f-8cc8ac4c12f8"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 18 14:32:31.688387 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:32:31.688220    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d24cf07c-4ff3-4dd9-b06f-8cc8ac4c12f8-webhook-cert" (OuterVolumeSpecName: "webhook-cert") pod "d24cf07c-4ff3-4dd9-b06f-8cc8ac4c12f8" (UID: "d24cf07c-4ff3-4dd9-b06f-8cc8ac4c12f8"). InnerVolumeSpecName "webhook-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 18 14:32:31.787229 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:32:31.787191    2565 reconciler_common.go:299] "Volume detached for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/d24cf07c-4ff3-4dd9-b06f-8cc8ac4c12f8-webhook-cert\") on node \"ip-10-0-129-16.ec2.internal\" DevicePath \"\""
Sep 18 14:32:31.787229 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:32:31.787222    2565 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d24cf07c-4ff3-4dd9-b06f-8cc8ac4c12f8-metrics-certs\") on node \"ip-10-0-129-16.ec2.internal\" DevicePath \"\""
Sep 18 14:32:31.787229 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:32:31.787231    2565 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-k988t\" (UniqueName: \"kubernetes.io/projected/d24cf07c-4ff3-4dd9-b06f-8cc8ac4c12f8-kube-api-access-k988t\") on node \"ip-10-0-129-16.ec2.internal\" DevicePath \"\""
Sep 18 14:32:31.802734 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:32:31.802703    2565 generic.go:356] "Generic (PLEG): container finished" podID="d24cf07c-4ff3-4dd9-b06f-8cc8ac4c12f8" containerID="c93f47c21dde70d9a522d79f94ff2fd0c4476e0175408f91c659eed8d70b14a7" exitCode=0
Sep 18 14:32:31.802862 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:32:31.802744    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-765845fd55-t69bn"
Sep 18 14:32:31.802862 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:32:31.802782    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-765845fd55-t69bn" event={"ID":"d24cf07c-4ff3-4dd9-b06f-8cc8ac4c12f8","Type":"ContainerDied","Data":"c93f47c21dde70d9a522d79f94ff2fd0c4476e0175408f91c659eed8d70b14a7"}
Sep 18 14:32:31.802862 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:32:31.802835    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-765845fd55-t69bn" event={"ID":"d24cf07c-4ff3-4dd9-b06f-8cc8ac4c12f8","Type":"ContainerDied","Data":"36db87eb71458d563cd0065679f6d6e5abce4dbc32cc4d0a066ef2c2ef565a0e"}
Sep 18 14:32:31.802862 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:32:31.802856    2565 scope.go:117] "RemoveContainer" containerID="c93f47c21dde70d9a522d79f94ff2fd0c4476e0175408f91c659eed8d70b14a7"
Sep 18 14:32:31.813619 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:32:31.813602    2565 scope.go:117] "RemoveContainer" containerID="c93f47c21dde70d9a522d79f94ff2fd0c4476e0175408f91c659eed8d70b14a7"
Sep 18 14:32:31.813918 ip-10-0-129-16 kubenswrapper[2565]: E0918 14:32:31.813896    2565 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c93f47c21dde70d9a522d79f94ff2fd0c4476e0175408f91c659eed8d70b14a7\": container with ID starting with c93f47c21dde70d9a522d79f94ff2fd0c4476e0175408f91c659eed8d70b14a7 not found: ID does not exist" containerID="c93f47c21dde70d9a522d79f94ff2fd0c4476e0175408f91c659eed8d70b14a7"
Sep 18 14:32:31.813967 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:32:31.813927    2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c93f47c21dde70d9a522d79f94ff2fd0c4476e0175408f91c659eed8d70b14a7"} err="failed to get container status \"c93f47c21dde70d9a522d79f94ff2fd0c4476e0175408f91c659eed8d70b14a7\": rpc error: code = NotFound desc = could not find container \"c93f47c21dde70d9a522d79f94ff2fd0c4476e0175408f91c659eed8d70b14a7\": container with ID starting with c93f47c21dde70d9a522d79f94ff2fd0c4476e0175408f91c659eed8d70b14a7 not found: ID does not exist"
Sep 18 14:32:31.830547 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:32:31.830518    2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-765845fd55-t69bn"]
Sep 18 14:32:31.834189 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:32:31.834166    2565 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-765845fd55-t69bn"]
Sep 18 14:32:31.913465 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:32:31.913432    2565 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d24cf07c-4ff3-4dd9-b06f-8cc8ac4c12f8" path="/var/lib/kubelet/pods/d24cf07c-4ff3-4dd9-b06f-8cc8ac4c12f8/volumes"
Sep 18 14:33:54.082234 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:33:54.082204    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-bvqls_0b0463f5-d647-4a38-b752-44afba77daff/ovn-acl-logging/0.log"
Sep 18 14:33:54.091857 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:33:54.091832    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-bvqls_0b0463f5-d647-4a38-b752-44afba77daff/ovn-acl-logging/0.log"
Sep 18 14:35:54.541420 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:35:54.541385    2565 scope.go:117] "RemoveContainer" containerID="22ad5a54ea01898c836aab2461e24866d3db5ea429c369bbd44bb8167cd6d72e"
Sep 18 14:36:54.564879 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:36:54.564842    2565 scope.go:117] "RemoveContainer" containerID="eb49410c73c0fb7506a97d1409489eb8dd703be1c6a5bdeff22c185c9334b7be"
Sep 18 14:36:54.574972 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:36:54.574949    2565 scope.go:117] "RemoveContainer" containerID="6bccfb174f68e14178ee52e704ce03a2a686a02bb9081a367b39752758618d1e"
Sep 18 14:36:54.584344 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:36:54.584313    2565 scope.go:117] "RemoveContainer" containerID="97bff83ae8db185b47f7dbe2535e9599a57afaf0072a869a8c38886bd35517b7"
Sep 18 14:37:54.617302 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:37:54.617213    2565 scope.go:117] "RemoveContainer" containerID="b95c30801f5691adb0dca8bc84dd002ac961e5bc039200c4c8df2597b43db732"
Sep 18 14:37:54.626954 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:37:54.626930    2565 scope.go:117] "RemoveContainer" containerID="c344e27ecd6f9dea1bbf11cb9f236e18e9914b2f82bc0618165057ee4a70eae3"
Sep 18 14:38:45.075202 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:38:45.075171    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_kserve-controller-manager-d6f4b65b8-8rpmc_38a34857-f87d-4673-b963-15634897fc08/manager/0.log"
Sep 18 14:38:45.456669 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:38:45.456574    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_model-serving-api-54d89bd5fc-rgjv6_e4fbc7b3-7e66-404b-a98b-231f84048b51/server/0.log"
Sep 18 14:38:45.583163 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:38:45.583137    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_odh-model-controller-7998c67dc4-jjhv5_2c6f3c8f-9a24-4688-9ce5-8f6d2469cc37/manager/0.log"
Sep 18 14:38:45.842380 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:38:45.842353    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-74d96fdcf9-hvj2l_7b2fb2cd-9edd-4856-aeaf-342eec275da1/copy-manifests/0.log"
Sep 18 14:38:45.849158 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:38:45.849132    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-74d96fdcf9-hvj2l_7b2fb2cd-9edd-4856-aeaf-342eec275da1/manager/0.log"
Sep 18 14:38:46.666048 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:38:46.666003    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-key-cleanup-29829030-mwdkm_9f94e766-bf61-4894-a29a-a85418b1432f/cleanup/0.log"
Sep 18 14:38:46.780904 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:38:46.780878    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_postgres-69fb4594cf-mx9q7_8107a555-0c1e-453d-b2ca-2dcb8aa64914/postgres/0.log"
Sep 18 14:38:48.278425 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:38:48.278386    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-operator-controller-manager-74495c69f8-llhlr_1f5c710a-07d4-4029-b265-6d73b6fb1c53/manager/0.log"
Sep 18 14:38:48.652441 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:38:48.652404    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-d66cd8c7c-kl5k8_17bd7b6a-c1c7-4c39-972d-78c6b3f9d996/istio-proxy/0.log"
Sep 18 14:38:49.179876 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:38:49.179844    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-78bb78ff78-phnkt_77ad5822-b925-43b7-8337-db3d1dd680cb/istio-proxy/1.log"
Sep 18 14:38:49.310494 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:38:49.310462    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-59fbc47bf8-8fw8g_c9eadd96-985e-4259-b048-3adac3d7c5c1/payload-pre-processing/0.log"
Sep 18 14:38:49.441968 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:38:49.441888    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5b54b7fbb7-9v8fj_5b843100-3abb-415e-80d7-fd7cff369593/payload-processing/0.log"
Sep 18 14:38:50.051400 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:38:50.051373    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-distinct-simulated-kserve-59975d69f9-r5l5k_d7732a87-9d53-48f9-aced-33692ad8a442/main/0.log"
Sep 18 14:38:50.297778 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:38:50.297743    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-trlp-test-simulated-kserve-598466548d-s69nq_87bb6eb6-8c3f-4e67-8e6e-39dd729b30a8/main/0.log"
Sep 18 14:38:50.568891 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:38:50.568861    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_facebook-opt-125m-simulated-kserve-74b68447df-jnzsl_5c48326f-2106-4e18-840d-119ec89289de/main/0.log"
Sep 18 14:38:54.117531 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:38:54.117504    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-bvqls_0b0463f5-d647-4a38-b752-44afba77daff/ovn-acl-logging/0.log"
Sep 18 14:38:54.127589 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:38:54.127569    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-bvqls_0b0463f5-d647-4a38-b752-44afba77daff/ovn-acl-logging/0.log"
Sep 18 14:38:54.650733 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:38:54.650703    2565 scope.go:117] "RemoveContainer" containerID="0c2f53c557a45208e804f5e40a2ee4d5eebc32c56eca7698850b8bd3604389d3"
Sep 18 14:39:00.324281 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:39:00.324245    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-xqrkr_d83ca837-1f57-4ff5-a513-a35f36198a99/global-pull-secret-syncer/0.log"
Sep 18 14:39:00.443302 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:39:00.443265    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-rzgjk_bff1008a-4c27-4278-969d-8974600e696f/konnectivity-agent/0.log"
Sep 18 14:39:00.570261 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:39:00.570230    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-129-16.ec2.internal_0092863d56373d9006f35b4f163e59f7/haproxy/0.log"
Sep 18 14:39:05.089854 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:39:05.089819    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-operator-controller-manager-74495c69f8-llhlr_1f5c710a-07d4-4029-b265-6d73b6fb1c53/manager/0.log"
Sep 18 14:39:06.428215 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:39:06.428185    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_400c6df4-8db3-4ad7-86fc-0948cee05b23/alertmanager/0.log"
Sep 18 14:39:06.479215 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:39:06.479190    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_400c6df4-8db3-4ad7-86fc-0948cee05b23/config-reloader/0.log"
Sep 18 14:39:06.529285 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:39:06.529255    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_400c6df4-8db3-4ad7-86fc-0948cee05b23/kube-rbac-proxy-web/0.log"
Sep 18 14:39:06.576628 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:39:06.576604    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_400c6df4-8db3-4ad7-86fc-0948cee05b23/kube-rbac-proxy/0.log"
Sep 18 14:39:06.632168 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:39:06.632139    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_400c6df4-8db3-4ad7-86fc-0948cee05b23/kube-rbac-proxy-metric/0.log"
Sep 18 14:39:06.690983 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:39:06.690904    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_400c6df4-8db3-4ad7-86fc-0948cee05b23/prom-label-proxy/0.log"
Sep 18 14:39:06.762130 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:39:06.762102    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_400c6df4-8db3-4ad7-86fc-0948cee05b23/init-config-reloader/0.log"
Sep 18 14:39:06.997712 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:39:06.997632    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-84c885658c-hwgkx_98c35b3b-f121-4202-bf45-0e9c31a2d848/kube-state-metrics/0.log"
Sep 18 14:39:07.163871 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:39:07.163842    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-84c885658c-hwgkx_98c35b3b-f121-4202-bf45-0e9c31a2d848/kube-rbac-proxy-main/0.log"
Sep 18 14:39:07.237396 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:39:07.237365    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-84c885658c-hwgkx_98c35b3b-f121-4202-bf45-0e9c31a2d848/kube-rbac-proxy-self/0.log"
Sep 18 14:39:07.378915 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:39:07.378885    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_metrics-server-99bb6cbd5-4mdgm_9533560d-e6fc-4d8b-984e-a9cf6f0f0315/metrics-server/0.log"
Sep 18 14:39:07.442513 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:39:07.442484    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_monitoring-plugin-5984459ff8-p9gl7_9c851437-b7ed-4359-8335-7af07f726404/monitoring-plugin/0.log"
Sep 18 14:39:07.710007 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:39:07.709923    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-psstj_5ac47b4f-a039-4016-a1df-9195a2124c08/node-exporter/0.log"
Sep 18 14:39:07.742107 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:39:07.742081    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-psstj_5ac47b4f-a039-4016-a1df-9195a2124c08/kube-rbac-proxy/0.log"
Sep 18 14:39:07.772066 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:39:07.772009    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-psstj_5ac47b4f-a039-4016-a1df-9195a2124c08/init-textfile/0.log"
Sep 18 14:39:08.006336 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:39:08.006240    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-86c5bc79d6-tgk6f_27863c15-1a0a-4577-a57e-39db30081e27/kube-rbac-proxy-main/0.log"
Sep 18 14:39:08.041085 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:39:08.041060    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-86c5bc79d6-tgk6f_27863c15-1a0a-4577-a57e-39db30081e27/kube-rbac-proxy-self/0.log"
Sep 18 14:39:08.086655 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:39:08.086630    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-86c5bc79d6-tgk6f_27863c15-1a0a-4577-a57e-39db30081e27/openshift-state-metrics/0.log"
Sep 18 14:39:08.179736 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:39:08.179691    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_bbc5c0d4-5395-4791-a3e5-65ba18efef52/prometheus/0.log"
Sep 18 14:39:08.247122 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:39:08.247097    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_bbc5c0d4-5395-4791-a3e5-65ba18efef52/config-reloader/0.log"
Sep 18 14:39:08.300455 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:39:08.300432    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_bbc5c0d4-5395-4791-a3e5-65ba18efef52/thanos-sidecar/0.log"
Sep 18 14:39:08.398335 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:39:08.398312    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_bbc5c0d4-5395-4791-a3e5-65ba18efef52/kube-rbac-proxy-web/0.log"
Sep 18 14:39:08.490159 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:39:08.490134    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_bbc5c0d4-5395-4791-a3e5-65ba18efef52/kube-rbac-proxy/0.log"
Sep 18 14:39:08.596694 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:39:08.596619    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_bbc5c0d4-5395-4791-a3e5-65ba18efef52/kube-rbac-proxy-thanos/0.log"
Sep 18 14:39:08.656412 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:39:08.656367    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_bbc5c0d4-5395-4791-a3e5-65ba18efef52/init-config-reloader/0.log"
Sep 18 14:39:08.730271 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:39:08.730232    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-7bd5d56d6-dbqgx_8e0b8c6d-ac6c-4621-bc2b-3cc23dbee621/prometheus-operator/0.log"
Sep 18 14:39:08.812623 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:39:08.812576    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-7bd5d56d6-dbqgx_8e0b8c6d-ac6c-4621-bc2b-3cc23dbee621/kube-rbac-proxy/0.log"
Sep 18 14:39:08.894631 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:39:08.894555    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-5c6545f8b5-c8mmm_20618239-3f82-4756-a0ad-d892a1bf13e8/telemeter-client/0.log"
Sep 18 14:39:08.934768 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:39:08.934739    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-5c6545f8b5-c8mmm_20618239-3f82-4756-a0ad-d892a1bf13e8/reload/0.log"
Sep 18 14:39:08.970629 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:39:08.970608    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-5c6545f8b5-c8mmm_20618239-3f82-4756-a0ad-d892a1bf13e8/kube-rbac-proxy/0.log"
Sep 18 14:39:09.024515 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:39:09.024493    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-5f599748b7-k2vqh_476315e9-c60b-485e-aa57-9ec551db2c80/thanos-query/0.log"
Sep 18 14:39:09.055995 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:39:09.055972    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-5f599748b7-k2vqh_476315e9-c60b-485e-aa57-9ec551db2c80/kube-rbac-proxy-web/0.log"
Sep 18 14:39:09.116404 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:39:09.116382    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-5f599748b7-k2vqh_476315e9-c60b-485e-aa57-9ec551db2c80/kube-rbac-proxy/0.log"
Sep 18 14:39:09.152480 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:39:09.152410    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-5f599748b7-k2vqh_476315e9-c60b-485e-aa57-9ec551db2c80/prom-label-proxy/0.log"
Sep 18 14:39:09.207946 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:39:09.207918    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-5f599748b7-k2vqh_476315e9-c60b-485e-aa57-9ec551db2c80/kube-rbac-proxy-rules/0.log"
Sep 18 14:39:09.248630 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:39:09.248603    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-5f599748b7-k2vqh_476315e9-c60b-485e-aa57-9ec551db2c80/kube-rbac-proxy-metrics/0.log"
Sep 18 14:39:09.836986 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:39:09.836952    2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-9xtsh/perf-node-gather-daemonset-jkcwp"]
Sep 18 14:39:09.837465 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:39:09.837415    2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d24cf07c-4ff3-4dd9-b06f-8cc8ac4c12f8" containerName="manager"
Sep 18 14:39:09.837465 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:39:09.837428    2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="d24cf07c-4ff3-4dd9-b06f-8cc8ac4c12f8" containerName="manager"
Sep 18 14:39:09.837465 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:39:09.837447    2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="80e9da7f-1054-4347-a3bb-4d410e230c50" containerName="revoke-keys"
Sep 18 14:39:09.837465 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:39:09.837453    2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="80e9da7f-1054-4347-a3bb-4d410e230c50" containerName="revoke-keys"
Sep 18 14:39:09.837599 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:39:09.837524    2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="d24cf07c-4ff3-4dd9-b06f-8cc8ac4c12f8" containerName="manager"
Sep 18 14:39:09.837599 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:39:09.837538    2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="80e9da7f-1054-4347-a3bb-4d410e230c50" containerName="revoke-keys"
Sep 18 14:39:09.842046 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:39:09.842016    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-9xtsh/perf-node-gather-daemonset-jkcwp"
Sep 18 14:39:09.845135 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:39:09.845110    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-9xtsh\"/\"default-dockercfg-r9lr6\""
Sep 18 14:39:09.845991 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:39:09.845970    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-9xtsh\"/\"openshift-service-ca.crt\""
Sep 18 14:39:09.845991 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:39:09.845983    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-9xtsh\"/\"kube-root-ca.crt\""
Sep 18 14:39:09.859027 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:39:09.859002    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-9xtsh/perf-node-gather-daemonset-jkcwp"]
Sep 18 14:39:09.955853 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:39:09.955820    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/8888f9c7-d5bf-4c7b-ab50-353d272efafe-sys\") pod \"perf-node-gather-daemonset-jkcwp\" (UID: \"8888f9c7-d5bf-4c7b-ab50-353d272efafe\") " pod="openshift-must-gather-9xtsh/perf-node-gather-daemonset-jkcwp"
Sep 18 14:39:09.956062 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:39:09.955914    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/8888f9c7-d5bf-4c7b-ab50-353d272efafe-proc\") pod \"perf-node-gather-daemonset-jkcwp\" (UID: \"8888f9c7-d5bf-4c7b-ab50-353d272efafe\") " pod="openshift-must-gather-9xtsh/perf-node-gather-daemonset-jkcwp"
Sep 18 14:39:09.956062 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:39:09.955967    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/8888f9c7-d5bf-4c7b-ab50-353d272efafe-lib-modules\") pod \"perf-node-gather-daemonset-jkcwp\" (UID: \"8888f9c7-d5bf-4c7b-ab50-353d272efafe\") " pod="openshift-must-gather-9xtsh/perf-node-gather-daemonset-jkcwp"
Sep 18 14:39:09.956062 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:39:09.956041    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/8888f9c7-d5bf-4c7b-ab50-353d272efafe-podres\") pod \"perf-node-gather-daemonset-jkcwp\" (UID: \"8888f9c7-d5bf-4c7b-ab50-353d272efafe\") " pod="openshift-must-gather-9xtsh/perf-node-gather-daemonset-jkcwp"
Sep 18 14:39:09.956187 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:39:09.956107    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-66zwk\" (UniqueName: \"kubernetes.io/projected/8888f9c7-d5bf-4c7b-ab50-353d272efafe-kube-api-access-66zwk\") pod \"perf-node-gather-daemonset-jkcwp\" (UID: \"8888f9c7-d5bf-4c7b-ab50-353d272efafe\") " pod="openshift-must-gather-9xtsh/perf-node-gather-daemonset-jkcwp"
Sep 18 14:39:10.056642 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:39:10.056603    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/8888f9c7-d5bf-4c7b-ab50-353d272efafe-proc\") pod \"perf-node-gather-daemonset-jkcwp\" (UID: \"8888f9c7-d5bf-4c7b-ab50-353d272efafe\") " pod="openshift-must-gather-9xtsh/perf-node-gather-daemonset-jkcwp"
Sep 18 14:39:10.056799 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:39:10.056661    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/8888f9c7-d5bf-4c7b-ab50-353d272efafe-lib-modules\") pod \"perf-node-gather-daemonset-jkcwp\" (UID: \"8888f9c7-d5bf-4c7b-ab50-353d272efafe\") " pod="openshift-must-gather-9xtsh/perf-node-gather-daemonset-jkcwp"
Sep 18 14:39:10.056799 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:39:10.056679    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/8888f9c7-d5bf-4c7b-ab50-353d272efafe-proc\") pod \"perf-node-gather-daemonset-jkcwp\" (UID: \"8888f9c7-d5bf-4c7b-ab50-353d272efafe\") " pod="openshift-must-gather-9xtsh/perf-node-gather-daemonset-jkcwp"
Sep 18 14:39:10.056799 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:39:10.056692    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/8888f9c7-d5bf-4c7b-ab50-353d272efafe-podres\") pod \"perf-node-gather-daemonset-jkcwp\" (UID: \"8888f9c7-d5bf-4c7b-ab50-353d272efafe\") " pod="openshift-must-gather-9xtsh/perf-node-gather-daemonset-jkcwp"
Sep 18 14:39:10.056799 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:39:10.056744    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-66zwk\" (UniqueName: \"kubernetes.io/projected/8888f9c7-d5bf-4c7b-ab50-353d272efafe-kube-api-access-66zwk\") pod \"perf-node-gather-daemonset-jkcwp\" (UID: \"8888f9c7-d5bf-4c7b-ab50-353d272efafe\") " pod="openshift-must-gather-9xtsh/perf-node-gather-daemonset-jkcwp"
Sep 18 14:39:10.056799 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:39:10.056785    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/8888f9c7-d5bf-4c7b-ab50-353d272efafe-podres\") pod \"perf-node-gather-daemonset-jkcwp\" (UID: \"8888f9c7-d5bf-4c7b-ab50-353d272efafe\") " pod="openshift-must-gather-9xtsh/perf-node-gather-daemonset-jkcwp"
Sep 18 14:39:10.057081 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:39:10.056809    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/8888f9c7-d5bf-4c7b-ab50-353d272efafe-lib-modules\") pod \"perf-node-gather-daemonset-jkcwp\" (UID: \"8888f9c7-d5bf-4c7b-ab50-353d272efafe\") " pod="openshift-must-gather-9xtsh/perf-node-gather-daemonset-jkcwp"
Sep 18 14:39:10.057081 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:39:10.056815    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/8888f9c7-d5bf-4c7b-ab50-353d272efafe-sys\") pod \"perf-node-gather-daemonset-jkcwp\" (UID: \"8888f9c7-d5bf-4c7b-ab50-353d272efafe\") " pod="openshift-must-gather-9xtsh/perf-node-gather-daemonset-jkcwp"
Sep 18 14:39:10.057081 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:39:10.056843    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/8888f9c7-d5bf-4c7b-ab50-353d272efafe-sys\") pod \"perf-node-gather-daemonset-jkcwp\" (UID: \"8888f9c7-d5bf-4c7b-ab50-353d272efafe\") " pod="openshift-must-gather-9xtsh/perf-node-gather-daemonset-jkcwp"
Sep 18 14:39:10.067216 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:39:10.067185    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-66zwk\" (UniqueName: \"kubernetes.io/projected/8888f9c7-d5bf-4c7b-ab50-353d272efafe-kube-api-access-66zwk\") pod \"perf-node-gather-daemonset-jkcwp\" (UID: \"8888f9c7-d5bf-4c7b-ab50-353d272efafe\") " pod="openshift-must-gather-9xtsh/perf-node-gather-daemonset-jkcwp"
Sep 18 14:39:10.152111 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:39:10.151984    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-9xtsh/perf-node-gather-daemonset-jkcwp"
Sep 18 14:39:10.430145 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:39:10.430112    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-9xtsh/perf-node-gather-daemonset-jkcwp"]
Sep 18 14:39:10.431292 ip-10-0-129-16 kubenswrapper[2565]: W0918 14:39:10.431262    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pod8888f9c7_d5bf_4c7b_ab50_353d272efafe.slice/crio-99cb088c5234cd75b8869a36c73b6d0eac56c4e8289e3a882ffad07e344180c7 WatchSource:0}: Error finding container 99cb088c5234cd75b8869a36c73b6d0eac56c4e8289e3a882ffad07e344180c7: Status 404 returned error can't find the container with id 99cb088c5234cd75b8869a36c73b6d0eac56c4e8289e3a882ffad07e344180c7
Sep 18 14:39:10.954446 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:39:10.954410    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-9xtsh/perf-node-gather-daemonset-jkcwp" event={"ID":"8888f9c7-d5bf-4c7b-ab50-353d272efafe","Type":"ContainerStarted","Data":"dff0913f262eaf8b4ab896a254f954d010473f6cce54fa4405566246288e08fb"}
Sep 18 14:39:10.954847 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:39:10.954472    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-9xtsh/perf-node-gather-daemonset-jkcwp" event={"ID":"8888f9c7-d5bf-4c7b-ab50-353d272efafe","Type":"ContainerStarted","Data":"99cb088c5234cd75b8869a36c73b6d0eac56c4e8289e3a882ffad07e344180c7"}
Sep 18 14:39:10.954847 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:39:10.954564    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-9xtsh/perf-node-gather-daemonset-jkcwp"
Sep 18 14:39:10.993727 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:39:10.993678    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-9xtsh/perf-node-gather-daemonset-jkcwp" podStartSLOduration=1.993663775 podStartE2EDuration="1.993663775s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-18 14:39:09 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 14:39:10.988128307 +0000 UTC m=+2419.162037981" watchObservedRunningTime="2026-09-18 14:39:10.993663775 +0000 UTC m=+2419.167573473"
Sep 18 14:39:13.860432 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:39:13.860403    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-54mm2_95c207c5-e264-4f7d-b4b4-9b48bee32aa6/dns/0.log"
Sep 18 14:39:13.957331 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:39:13.957302    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-54mm2_95c207c5-e264-4f7d-b4b4-9b48bee32aa6/kube-rbac-proxy/0.log"
Sep 18 14:39:14.234448 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:39:14.234372    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-lqtdm_2044301a-3777-480b-9c7e-cdca3c1eb61a/dns-node-resolver/0.log"
Sep 18 14:39:15.171195 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:39:15.171168    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-hvhcs_75c7d00d-fb1a-4716-9963-b7ef1c0b35bd/node-ca/0.log"
Sep 18 14:39:16.505093 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:39:16.505058    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-d66cd8c7c-kl5k8_17bd7b6a-c1c7-4c39-972d-78c6b3f9d996/istio-proxy/0.log"
Sep 18 14:39:16.969836 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:39:16.969808    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-9xtsh/perf-node-gather-daemonset-jkcwp"
Sep 18 14:39:17.178765 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:39:17.178731    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-78bb78ff78-phnkt_77ad5822-b925-43b7-8337-db3d1dd680cb/istio-proxy/0.log"
Sep 18 14:39:17.181667 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:39:17.181646    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-78bb78ff78-phnkt_77ad5822-b925-43b7-8337-db3d1dd680cb/istio-proxy/1.log"
Sep 18 14:39:17.262631 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:39:17.262545    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-59fbc47bf8-8fw8g_c9eadd96-985e-4259-b048-3adac3d7c5c1/payload-pre-processing/0.log"
Sep 18 14:39:17.406398 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:39:17.406372    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5b54b7fbb7-9v8fj_5b843100-3abb-415e-80d7-fd7cff369593/payload-processing/0.log"
Sep 18 14:39:18.732885 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:39:18.732860    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-bzgmq_fc0c22c0-58aa-4c0e-9d86-ef684058c2ca/serve-healthcheck-canary/0.log"
Sep 18 14:39:19.552928 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:39:19.552903    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-d66cd8c7c-kl5k8_17bd7b6a-c1c7-4c39-972d-78c6b3f9d996/istio-proxy/0.log"
Sep 18 14:39:19.673303 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:39:19.673274    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-msrpt_cbf59139-0ecb-4f9c-bc28-0045adf52ecf/kube-rbac-proxy/0.log"
Sep 18 14:39:19.709670 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:39:19.709644    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-msrpt_cbf59139-0ecb-4f9c-bc28-0045adf52ecf/exporter/0.log"
Sep 18 14:39:19.744168 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:39:19.744145    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-msrpt_cbf59139-0ecb-4f9c-bc28-0045adf52ecf/extractor/0.log"
Sep 18 14:39:20.013466 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:39:20.013442    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-78bb78ff78-phnkt_77ad5822-b925-43b7-8337-db3d1dd680cb/istio-proxy/0.log"
Sep 18 14:39:20.015994 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:39:20.015974    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-78bb78ff78-phnkt_77ad5822-b925-43b7-8337-db3d1dd680cb/istio-proxy/1.log"
Sep 18 14:39:20.041993 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:39:20.041973    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-59fbc47bf8-8fw8g_c9eadd96-985e-4259-b048-3adac3d7c5c1/payload-pre-processing/0.log"
Sep 18 14:39:20.071516 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:39:20.071491    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5b54b7fbb7-9v8fj_5b843100-3abb-415e-80d7-fd7cff369593/payload-processing/0.log"
Sep 18 14:39:21.589913 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:39:21.589883    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-d66cd8c7c-kl5k8_17bd7b6a-c1c7-4c39-972d-78c6b3f9d996/istio-proxy/0.log"
Sep 18 14:39:22.040984 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:39:22.040949    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-78bb78ff78-phnkt_77ad5822-b925-43b7-8337-db3d1dd680cb/istio-proxy/0.log"
Sep 18 14:39:22.043494 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:39:22.043477    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-78bb78ff78-phnkt_77ad5822-b925-43b7-8337-db3d1dd680cb/istio-proxy/1.log"
Sep 18 14:39:22.067617 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:39:22.067594    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-59fbc47bf8-8fw8g_c9eadd96-985e-4259-b048-3adac3d7c5c1/payload-pre-processing/0.log"
Sep 18 14:39:22.094475 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:39:22.094451    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5b54b7fbb7-9v8fj_5b843100-3abb-415e-80d7-fd7cff369593/payload-processing/0.log"
Sep 18 14:39:22.278225 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:39:22.278197    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_kserve-controller-manager-d6f4b65b8-8rpmc_38a34857-f87d-4673-b963-15634897fc08/manager/0.log"
Sep 18 14:39:22.612493 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:39:22.612467    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_model-serving-api-54d89bd5fc-rgjv6_e4fbc7b3-7e66-404b-a98b-231f84048b51/server/0.log"
Sep 18 14:39:22.704194 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:39:22.704162    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_odh-model-controller-7998c67dc4-jjhv5_2c6f3c8f-9a24-4688-9ce5-8f6d2469cc37/manager/0.log"
Sep 18 14:39:22.822206 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:39:22.822176    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-74d96fdcf9-hvj2l_7b2fb2cd-9edd-4856-aeaf-342eec275da1/manager/0.log"
Sep 18 14:39:22.850619 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:39:22.850594    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-74d96fdcf9-hvj2l_7b2fb2cd-9edd-4856-aeaf-342eec275da1/copy-manifests/0.log"
Sep 18 14:39:24.154269 ip-10-0-129-16 kubenswrapper[2565]: I0918 14:39:24.154239    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_lws-controller-manager-ff9cccccb-ntpsv_5ce17b25-ef8c-4736-b9d4-319e627c1d9d/manager/0.log"