Sep 10 17:34:11.550939 ip-10-0-141-180 systemd[1]: kubelet.service: Failed to load environment files: No such file or directory
Sep 10 17:34:11.550950 ip-10-0-141-180 systemd[1]: kubelet.service: Failed to run 'start-pre' task: No such file or directory
Sep 10 17:34:11.550957 ip-10-0-141-180 systemd[1]: kubelet.service: Failed with result 'resources'.
Sep 10 17:34:11.551120 ip-10-0-141-180 systemd[1]: Failed to start Kubernetes Kubelet.
Sep 10 17:34:21.708945 ip-10-0-141-180 systemd[1]: kubelet.service: Failed to schedule restart job: Unit crio.service not found.
Sep 10 17:34:21.708962 ip-10-0-141-180 systemd[1]: kubelet.service: Failed with result 'resources'.
-- Boot 94d457ad74fe4cbab15fa085513c934c --
Sep 10 17:36:19.210484 ip-10-0-141-180 systemd[1]: Starting Kubernetes Kubelet...
Sep 10 17:36:19.609497 ip-10-0-141-180 kubenswrapper[2581]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Sep 10 17:36:19.609497 ip-10-0-141-180 kubenswrapper[2581]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version.
Sep 10 17:36:19.609497 ip-10-0-141-180 kubenswrapper[2581]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Sep 10 17:36:19.609497 ip-10-0-141-180 kubenswrapper[2581]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI.
Sep 10 17:36:19.609497 ip-10-0-141-180 kubenswrapper[2581]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Sep 10 17:36:19.610574 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:36:19.610436    2581 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime"
Sep 10 17:36:19.614285 ip-10-0-141-180 kubenswrapper[2581]: W0910 17:36:19.614270    2581 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 10 17:36:19.614285 ip-10-0-141-180 kubenswrapper[2581]: W0910 17:36:19.614284    2581 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 10 17:36:19.614347 ip-10-0-141-180 kubenswrapper[2581]: W0910 17:36:19.614288    2581 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 10 17:36:19.614347 ip-10-0-141-180 kubenswrapper[2581]: W0910 17:36:19.614292    2581 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 10 17:36:19.614347 ip-10-0-141-180 kubenswrapper[2581]: W0910 17:36:19.614295    2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 10 17:36:19.614347 ip-10-0-141-180 kubenswrapper[2581]: W0910 17:36:19.614298    2581 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 10 17:36:19.614347 ip-10-0-141-180 kubenswrapper[2581]: W0910 17:36:19.614302    2581 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 10 17:36:19.614347 ip-10-0-141-180 kubenswrapper[2581]: W0910 17:36:19.614305    2581 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 10 17:36:19.614347 ip-10-0-141-180 kubenswrapper[2581]: W0910 17:36:19.614308    2581 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 10 17:36:19.614347 ip-10-0-141-180 kubenswrapper[2581]: W0910 17:36:19.614310    2581 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 10 17:36:19.614347 ip-10-0-141-180 kubenswrapper[2581]: W0910 17:36:19.614313    2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 10 17:36:19.614347 ip-10-0-141-180 kubenswrapper[2581]: W0910 17:36:19.614316    2581 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 10 17:36:19.614347 ip-10-0-141-180 kubenswrapper[2581]: W0910 17:36:19.614318    2581 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 10 17:36:19.614347 ip-10-0-141-180 kubenswrapper[2581]: W0910 17:36:19.614321    2581 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 10 17:36:19.614347 ip-10-0-141-180 kubenswrapper[2581]: W0910 17:36:19.614329    2581 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 10 17:36:19.614347 ip-10-0-141-180 kubenswrapper[2581]: W0910 17:36:19.614331    2581 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 10 17:36:19.614347 ip-10-0-141-180 kubenswrapper[2581]: W0910 17:36:19.614334    2581 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 10 17:36:19.614347 ip-10-0-141-180 kubenswrapper[2581]: W0910 17:36:19.614337    2581 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 10 17:36:19.614347 ip-10-0-141-180 kubenswrapper[2581]: W0910 17:36:19.614339    2581 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 10 17:36:19.614347 ip-10-0-141-180 kubenswrapper[2581]: W0910 17:36:19.614342    2581 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 10 17:36:19.614347 ip-10-0-141-180 kubenswrapper[2581]: W0910 17:36:19.614344    2581 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 10 17:36:19.614347 ip-10-0-141-180 kubenswrapper[2581]: W0910 17:36:19.614347    2581 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 10 17:36:19.614824 ip-10-0-141-180 kubenswrapper[2581]: W0910 17:36:19.614349    2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 10 17:36:19.614824 ip-10-0-141-180 kubenswrapper[2581]: W0910 17:36:19.614352    2581 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 10 17:36:19.614824 ip-10-0-141-180 kubenswrapper[2581]: W0910 17:36:19.614355    2581 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 10 17:36:19.614824 ip-10-0-141-180 kubenswrapper[2581]: W0910 17:36:19.614358    2581 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 10 17:36:19.614824 ip-10-0-141-180 kubenswrapper[2581]: W0910 17:36:19.614361    2581 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 10 17:36:19.614824 ip-10-0-141-180 kubenswrapper[2581]: W0910 17:36:19.614363    2581 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 10 17:36:19.614824 ip-10-0-141-180 kubenswrapper[2581]: W0910 17:36:19.614367    2581 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 10 17:36:19.614824 ip-10-0-141-180 kubenswrapper[2581]: W0910 17:36:19.614371    2581 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 10 17:36:19.614824 ip-10-0-141-180 kubenswrapper[2581]: W0910 17:36:19.614374    2581 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 10 17:36:19.614824 ip-10-0-141-180 kubenswrapper[2581]: W0910 17:36:19.614376    2581 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 10 17:36:19.614824 ip-10-0-141-180 kubenswrapper[2581]: W0910 17:36:19.614379    2581 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 10 17:36:19.614824 ip-10-0-141-180 kubenswrapper[2581]: W0910 17:36:19.614382    2581 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 10 17:36:19.614824 ip-10-0-141-180 kubenswrapper[2581]: W0910 17:36:19.614384    2581 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 10 17:36:19.614824 ip-10-0-141-180 kubenswrapper[2581]: W0910 17:36:19.614387    2581 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 10 17:36:19.614824 ip-10-0-141-180 kubenswrapper[2581]: W0910 17:36:19.614389    2581 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 10 17:36:19.614824 ip-10-0-141-180 kubenswrapper[2581]: W0910 17:36:19.614392    2581 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 10 17:36:19.614824 ip-10-0-141-180 kubenswrapper[2581]: W0910 17:36:19.614394    2581 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 10 17:36:19.614824 ip-10-0-141-180 kubenswrapper[2581]: W0910 17:36:19.614398    2581 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 10 17:36:19.614824 ip-10-0-141-180 kubenswrapper[2581]: W0910 17:36:19.614401    2581 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 10 17:36:19.615308 ip-10-0-141-180 kubenswrapper[2581]: W0910 17:36:19.614404    2581 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 10 17:36:19.615308 ip-10-0-141-180 kubenswrapper[2581]: W0910 17:36:19.614407    2581 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 10 17:36:19.615308 ip-10-0-141-180 kubenswrapper[2581]: W0910 17:36:19.614409    2581 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 10 17:36:19.615308 ip-10-0-141-180 kubenswrapper[2581]: W0910 17:36:19.614412    2581 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 10 17:36:19.615308 ip-10-0-141-180 kubenswrapper[2581]: W0910 17:36:19.614415    2581 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 10 17:36:19.615308 ip-10-0-141-180 kubenswrapper[2581]: W0910 17:36:19.614418    2581 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 10 17:36:19.615308 ip-10-0-141-180 kubenswrapper[2581]: W0910 17:36:19.614420    2581 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 10 17:36:19.615308 ip-10-0-141-180 kubenswrapper[2581]: W0910 17:36:19.614423    2581 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 10 17:36:19.615308 ip-10-0-141-180 kubenswrapper[2581]: W0910 17:36:19.614425    2581 feature_gate.go:328] unrecognized feature gate: Example
Sep 10 17:36:19.615308 ip-10-0-141-180 kubenswrapper[2581]: W0910 17:36:19.614429    2581 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 10 17:36:19.615308 ip-10-0-141-180 kubenswrapper[2581]: W0910 17:36:19.614432    2581 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 10 17:36:19.615308 ip-10-0-141-180 kubenswrapper[2581]: W0910 17:36:19.614434    2581 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 10 17:36:19.615308 ip-10-0-141-180 kubenswrapper[2581]: W0910 17:36:19.614437    2581 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 10 17:36:19.615308 ip-10-0-141-180 kubenswrapper[2581]: W0910 17:36:19.614440    2581 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 10 17:36:19.615308 ip-10-0-141-180 kubenswrapper[2581]: W0910 17:36:19.614443    2581 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 10 17:36:19.615308 ip-10-0-141-180 kubenswrapper[2581]: W0910 17:36:19.614445    2581 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 10 17:36:19.615308 ip-10-0-141-180 kubenswrapper[2581]: W0910 17:36:19.614448    2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 10 17:36:19.615308 ip-10-0-141-180 kubenswrapper[2581]: W0910 17:36:19.614451    2581 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 10 17:36:19.615308 ip-10-0-141-180 kubenswrapper[2581]: W0910 17:36:19.614453    2581 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 10 17:36:19.615308 ip-10-0-141-180 kubenswrapper[2581]: W0910 17:36:19.614456    2581 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 10 17:36:19.615794 ip-10-0-141-180 kubenswrapper[2581]: W0910 17:36:19.614458    2581 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 10 17:36:19.615794 ip-10-0-141-180 kubenswrapper[2581]: W0910 17:36:19.614461    2581 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 10 17:36:19.615794 ip-10-0-141-180 kubenswrapper[2581]: W0910 17:36:19.614464    2581 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 10 17:36:19.615794 ip-10-0-141-180 kubenswrapper[2581]: W0910 17:36:19.614466    2581 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 10 17:36:19.615794 ip-10-0-141-180 kubenswrapper[2581]: W0910 17:36:19.614469    2581 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 10 17:36:19.615794 ip-10-0-141-180 kubenswrapper[2581]: W0910 17:36:19.614471    2581 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 10 17:36:19.615794 ip-10-0-141-180 kubenswrapper[2581]: W0910 17:36:19.614474    2581 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 10 17:36:19.615794 ip-10-0-141-180 kubenswrapper[2581]: W0910 17:36:19.614476    2581 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 10 17:36:19.615794 ip-10-0-141-180 kubenswrapper[2581]: W0910 17:36:19.614479    2581 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 10 17:36:19.615794 ip-10-0-141-180 kubenswrapper[2581]: W0910 17:36:19.614482    2581 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 10 17:36:19.615794 ip-10-0-141-180 kubenswrapper[2581]: W0910 17:36:19.614484    2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 10 17:36:19.615794 ip-10-0-141-180 kubenswrapper[2581]: W0910 17:36:19.614487    2581 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 10 17:36:19.615794 ip-10-0-141-180 kubenswrapper[2581]: W0910 17:36:19.614489    2581 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 10 17:36:19.615794 ip-10-0-141-180 kubenswrapper[2581]: W0910 17:36:19.614492    2581 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 10 17:36:19.615794 ip-10-0-141-180 kubenswrapper[2581]: W0910 17:36:19.614494    2581 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 10 17:36:19.615794 ip-10-0-141-180 kubenswrapper[2581]: W0910 17:36:19.614497    2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 10 17:36:19.615794 ip-10-0-141-180 kubenswrapper[2581]: W0910 17:36:19.614499    2581 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 10 17:36:19.615794 ip-10-0-141-180 kubenswrapper[2581]: W0910 17:36:19.614501    2581 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 10 17:36:19.615794 ip-10-0-141-180 kubenswrapper[2581]: W0910 17:36:19.614504    2581 feature_gate.go:328] unrecognized feature gate: Example2
Sep 10 17:36:19.615794 ip-10-0-141-180 kubenswrapper[2581]: W0910 17:36:19.614510    2581 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 10 17:36:19.616302 ip-10-0-141-180 kubenswrapper[2581]: W0910 17:36:19.614513    2581 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 10 17:36:19.616302 ip-10-0-141-180 kubenswrapper[2581]: W0910 17:36:19.614516    2581 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 10 17:36:19.616302 ip-10-0-141-180 kubenswrapper[2581]: W0910 17:36:19.614519    2581 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 10 17:36:19.616302 ip-10-0-141-180 kubenswrapper[2581]: W0910 17:36:19.614522    2581 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 10 17:36:19.616302 ip-10-0-141-180 kubenswrapper[2581]: W0910 17:36:19.614526    2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 10 17:36:19.616302 ip-10-0-141-180 kubenswrapper[2581]: W0910 17:36:19.614528    2581 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 10 17:36:19.616302 ip-10-0-141-180 kubenswrapper[2581]: W0910 17:36:19.614531    2581 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 10 17:36:19.616302 ip-10-0-141-180 kubenswrapper[2581]: W0910 17:36:19.614534    2581 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 10 17:36:19.616302 ip-10-0-141-180 kubenswrapper[2581]: W0910 17:36:19.614536    2581 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 10 17:36:19.616302 ip-10-0-141-180 kubenswrapper[2581]: W0910 17:36:19.614539    2581 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 10 17:36:19.616302 ip-10-0-141-180 kubenswrapper[2581]: W0910 17:36:19.614541    2581 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 10 17:36:19.616302 ip-10-0-141-180 kubenswrapper[2581]: W0910 17:36:19.614544    2581 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 10 17:36:19.616302 ip-10-0-141-180 kubenswrapper[2581]: W0910 17:36:19.614546    2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 10 17:36:19.616302 ip-10-0-141-180 kubenswrapper[2581]: W0910 17:36:19.614549    2581 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 10 17:36:19.616302 ip-10-0-141-180 kubenswrapper[2581]: W0910 17:36:19.614551    2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 10 17:36:19.616302 ip-10-0-141-180 kubenswrapper[2581]: W0910 17:36:19.614555    2581 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 10 17:36:19.616302 ip-10-0-141-180 kubenswrapper[2581]: W0910 17:36:19.614558    2581 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 10 17:36:19.616302 ip-10-0-141-180 kubenswrapper[2581]: W0910 17:36:19.614561    2581 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 10 17:36:19.616302 ip-10-0-141-180 kubenswrapper[2581]: W0910 17:36:19.615250    2581 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 10 17:36:19.616756 ip-10-0-141-180 kubenswrapper[2581]: W0910 17:36:19.615258    2581 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 10 17:36:19.616756 ip-10-0-141-180 kubenswrapper[2581]: W0910 17:36:19.615262    2581 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 10 17:36:19.616756 ip-10-0-141-180 kubenswrapper[2581]: W0910 17:36:19.615265    2581 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 10 17:36:19.616756 ip-10-0-141-180 kubenswrapper[2581]: W0910 17:36:19.615268    2581 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 10 17:36:19.616756 ip-10-0-141-180 kubenswrapper[2581]: W0910 17:36:19.615271    2581 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 10 17:36:19.616756 ip-10-0-141-180 kubenswrapper[2581]: W0910 17:36:19.615274    2581 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 10 17:36:19.616756 ip-10-0-141-180 kubenswrapper[2581]: W0910 17:36:19.615277    2581 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 10 17:36:19.616756 ip-10-0-141-180 kubenswrapper[2581]: W0910 17:36:19.615279    2581 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 10 17:36:19.616756 ip-10-0-141-180 kubenswrapper[2581]: W0910 17:36:19.615282    2581 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 10 17:36:19.616756 ip-10-0-141-180 kubenswrapper[2581]: W0910 17:36:19.615285    2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 10 17:36:19.616756 ip-10-0-141-180 kubenswrapper[2581]: W0910 17:36:19.615289    2581 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 10 17:36:19.616756 ip-10-0-141-180 kubenswrapper[2581]: W0910 17:36:19.615292    2581 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 10 17:36:19.616756 ip-10-0-141-180 kubenswrapper[2581]: W0910 17:36:19.615295    2581 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 10 17:36:19.616756 ip-10-0-141-180 kubenswrapper[2581]: W0910 17:36:19.615299    2581 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 10 17:36:19.616756 ip-10-0-141-180 kubenswrapper[2581]: W0910 17:36:19.615302    2581 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 10 17:36:19.616756 ip-10-0-141-180 kubenswrapper[2581]: W0910 17:36:19.615305    2581 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 10 17:36:19.616756 ip-10-0-141-180 kubenswrapper[2581]: W0910 17:36:19.615307    2581 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 10 17:36:19.616756 ip-10-0-141-180 kubenswrapper[2581]: W0910 17:36:19.615310    2581 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 10 17:36:19.616756 ip-10-0-141-180 kubenswrapper[2581]: W0910 17:36:19.615313    2581 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 10 17:36:19.617245 ip-10-0-141-180 kubenswrapper[2581]: W0910 17:36:19.615316    2581 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 10 17:36:19.617245 ip-10-0-141-180 kubenswrapper[2581]: W0910 17:36:19.615318    2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 10 17:36:19.617245 ip-10-0-141-180 kubenswrapper[2581]: W0910 17:36:19.615321    2581 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 10 17:36:19.617245 ip-10-0-141-180 kubenswrapper[2581]: W0910 17:36:19.615324    2581 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 10 17:36:19.617245 ip-10-0-141-180 kubenswrapper[2581]: W0910 17:36:19.615327    2581 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 10 17:36:19.617245 ip-10-0-141-180 kubenswrapper[2581]: W0910 17:36:19.615329    2581 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 10 17:36:19.617245 ip-10-0-141-180 kubenswrapper[2581]: W0910 17:36:19.615332    2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 10 17:36:19.617245 ip-10-0-141-180 kubenswrapper[2581]: W0910 17:36:19.615334    2581 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 10 17:36:19.617245 ip-10-0-141-180 kubenswrapper[2581]: W0910 17:36:19.615337    2581 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 10 17:36:19.617245 ip-10-0-141-180 kubenswrapper[2581]: W0910 17:36:19.615339    2581 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 10 17:36:19.617245 ip-10-0-141-180 kubenswrapper[2581]: W0910 17:36:19.615342    2581 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 10 17:36:19.617245 ip-10-0-141-180 kubenswrapper[2581]: W0910 17:36:19.615344    2581 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 10 17:36:19.617245 ip-10-0-141-180 kubenswrapper[2581]: W0910 17:36:19.615347    2581 feature_gate.go:328] unrecognized feature gate: Example2
Sep 10 17:36:19.617245 ip-10-0-141-180 kubenswrapper[2581]: W0910 17:36:19.615349    2581 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 10 17:36:19.617245 ip-10-0-141-180 kubenswrapper[2581]: W0910 17:36:19.615352    2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 10 17:36:19.617245 ip-10-0-141-180 kubenswrapper[2581]: W0910 17:36:19.615355    2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 10 17:36:19.617245 ip-10-0-141-180 kubenswrapper[2581]: W0910 17:36:19.615357    2581 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 10 17:36:19.617245 ip-10-0-141-180 kubenswrapper[2581]: W0910 17:36:19.615359    2581 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 10 17:36:19.617245 ip-10-0-141-180 kubenswrapper[2581]: W0910 17:36:19.615362    2581 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 10 17:36:19.617245 ip-10-0-141-180 kubenswrapper[2581]: W0910 17:36:19.615365    2581 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 10 17:36:19.617735 ip-10-0-141-180 kubenswrapper[2581]: W0910 17:36:19.615368    2581 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 10 17:36:19.617735 ip-10-0-141-180 kubenswrapper[2581]: W0910 17:36:19.615370    2581 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 10 17:36:19.617735 ip-10-0-141-180 kubenswrapper[2581]: W0910 17:36:19.615373    2581 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 10 17:36:19.617735 ip-10-0-141-180 kubenswrapper[2581]: W0910 17:36:19.615376    2581 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 10 17:36:19.617735 ip-10-0-141-180 kubenswrapper[2581]: W0910 17:36:19.615378    2581 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 10 17:36:19.617735 ip-10-0-141-180 kubenswrapper[2581]: W0910 17:36:19.615381    2581 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 10 17:36:19.617735 ip-10-0-141-180 kubenswrapper[2581]: W0910 17:36:19.615383    2581 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 10 17:36:19.617735 ip-10-0-141-180 kubenswrapper[2581]: W0910 17:36:19.615387    2581 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 10 17:36:19.617735 ip-10-0-141-180 kubenswrapper[2581]: W0910 17:36:19.615389    2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 10 17:36:19.617735 ip-10-0-141-180 kubenswrapper[2581]: W0910 17:36:19.615392    2581 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 10 17:36:19.617735 ip-10-0-141-180 kubenswrapper[2581]: W0910 17:36:19.615395    2581 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 10 17:36:19.617735 ip-10-0-141-180 kubenswrapper[2581]: W0910 17:36:19.615398    2581 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 10 17:36:19.617735 ip-10-0-141-180 kubenswrapper[2581]: W0910 17:36:19.615402    2581 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 10 17:36:19.617735 ip-10-0-141-180 kubenswrapper[2581]: W0910 17:36:19.615405    2581 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 10 17:36:19.617735 ip-10-0-141-180 kubenswrapper[2581]: W0910 17:36:19.615408    2581 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 10 17:36:19.617735 ip-10-0-141-180 kubenswrapper[2581]: W0910 17:36:19.615411    2581 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 10 17:36:19.617735 ip-10-0-141-180 kubenswrapper[2581]: W0910 17:36:19.615414    2581 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 10 17:36:19.617735 ip-10-0-141-180 kubenswrapper[2581]: W0910 17:36:19.615417    2581 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 10 17:36:19.617735 ip-10-0-141-180 kubenswrapper[2581]: W0910 17:36:19.615420    2581 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 10 17:36:19.618318 ip-10-0-141-180 kubenswrapper[2581]: W0910 17:36:19.615422    2581 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 10 17:36:19.618318 ip-10-0-141-180 kubenswrapper[2581]: W0910 17:36:19.615424    2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 10 17:36:19.618318 ip-10-0-141-180 kubenswrapper[2581]: W0910 17:36:19.615427    2581 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 10 17:36:19.618318 ip-10-0-141-180 kubenswrapper[2581]: W0910 17:36:19.615430    2581 feature_gate.go:328] unrecognized feature gate: Example
Sep 10 17:36:19.618318 ip-10-0-141-180 kubenswrapper[2581]: W0910 17:36:19.615433    2581 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 10 17:36:19.618318 ip-10-0-141-180 kubenswrapper[2581]: W0910 17:36:19.615436    2581 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 10 17:36:19.618318 ip-10-0-141-180 kubenswrapper[2581]: W0910 17:36:19.615438    2581 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 10 17:36:19.618318 ip-10-0-141-180 kubenswrapper[2581]: W0910 17:36:19.615441    2581 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 10 17:36:19.618318 ip-10-0-141-180 kubenswrapper[2581]: W0910 17:36:19.615444    2581 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 10 17:36:19.618318 ip-10-0-141-180 kubenswrapper[2581]: W0910 17:36:19.615446    2581 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 10 17:36:19.618318 ip-10-0-141-180 kubenswrapper[2581]: W0910 17:36:19.615449    2581 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 10 17:36:19.618318 ip-10-0-141-180 kubenswrapper[2581]: W0910 17:36:19.615451    2581 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 10 17:36:19.618318 ip-10-0-141-180 kubenswrapper[2581]: W0910 17:36:19.615454    2581 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 10 17:36:19.618318 ip-10-0-141-180 kubenswrapper[2581]: W0910 17:36:19.615457    2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 10 17:36:19.618318 ip-10-0-141-180 kubenswrapper[2581]: W0910 17:36:19.615459    2581 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 10 17:36:19.618318 ip-10-0-141-180 kubenswrapper[2581]: W0910 17:36:19.615462    2581 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 10 17:36:19.618318 ip-10-0-141-180 kubenswrapper[2581]: W0910 17:36:19.615464    2581 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 10 17:36:19.618318 ip-10-0-141-180 kubenswrapper[2581]: W0910 17:36:19.615467    2581 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 10 17:36:19.618318 ip-10-0-141-180 kubenswrapper[2581]: W0910 17:36:19.615469    2581 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 10 17:36:19.618318 ip-10-0-141-180 kubenswrapper[2581]: W0910 17:36:19.615471    2581 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 10 17:36:19.618811 ip-10-0-141-180 kubenswrapper[2581]: W0910 17:36:19.615475    2581 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 10 17:36:19.618811 ip-10-0-141-180 kubenswrapper[2581]: W0910 17:36:19.615478    2581 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 10 17:36:19.618811 ip-10-0-141-180 kubenswrapper[2581]: W0910 17:36:19.615480    2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 10 17:36:19.618811 ip-10-0-141-180 kubenswrapper[2581]: W0910 17:36:19.615484    2581 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 10 17:36:19.618811 ip-10-0-141-180 kubenswrapper[2581]: W0910 17:36:19.615487    2581 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 10 17:36:19.618811 ip-10-0-141-180 kubenswrapper[2581]: W0910 17:36:19.615490    2581 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 10 17:36:19.618811 ip-10-0-141-180 kubenswrapper[2581]: W0910 17:36:19.615493    2581 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 10 17:36:19.618811 ip-10-0-141-180 kubenswrapper[2581]: W0910 17:36:19.615495    2581 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 10 17:36:19.618811 ip-10-0-141-180 kubenswrapper[2581]: W0910 17:36:19.615498    2581 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 10 17:36:19.618811 ip-10-0-141-180 kubenswrapper[2581]: W0910 17:36:19.615500    2581 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 10 17:36:19.618811 ip-10-0-141-180 kubenswrapper[2581]: W0910 17:36:19.615503    2581 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 10 17:36:19.618811 ip-10-0-141-180 kubenswrapper[2581]: W0910 17:36:19.615505    2581 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 10 17:36:19.618811 ip-10-0-141-180 kubenswrapper[2581]: W0910 17:36:19.615508    2581 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 10 17:36:19.618811 ip-10-0-141-180 kubenswrapper[2581]: W0910 17:36:19.615510    2581 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 10 17:36:19.618811 ip-10-0-141-180 kubenswrapper[2581]: W0910 17:36:19.615513    2581 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 10 17:36:19.618811 ip-10-0-141-180 kubenswrapper[2581]: W0910 17:36:19.615515    2581 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 10 17:36:19.618811 ip-10-0-141-180 kubenswrapper[2581]: W0910 17:36:19.615518    2581 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 10 17:36:19.618811 ip-10-0-141-180 kubenswrapper[2581]: W0910 17:36:19.615520    2581 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 10 17:36:19.618811 ip-10-0-141-180 kubenswrapper[2581]: W0910 17:36:19.615523    2581 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 10 17:36:19.618811 ip-10-0-141-180 kubenswrapper[2581]: W0910 17:36:19.615526    2581 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 10 17:36:19.619343 ip-10-0-141-180 kubenswrapper[2581]: W0910 17:36:19.615879    2581 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 10 17:36:19.619343 ip-10-0-141-180 kubenswrapper[2581]: W0910 17:36:19.615884    2581 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 10 17:36:19.619343 ip-10-0-141-180 kubenswrapper[2581]: W0910 17:36:19.615887    2581 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 10 17:36:19.619343 ip-10-0-141-180 kubenswrapper[2581]: W0910 17:36:19.615889    2581 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 10 17:36:19.619343 ip-10-0-141-180 kubenswrapper[2581]: W0910 17:36:19.615892    2581 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 10 17:36:19.619343 ip-10-0-141-180 kubenswrapper[2581]: W0910 17:36:19.615895    2581 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 10 17:36:19.619343 ip-10-0-141-180 kubenswrapper[2581]: W0910 17:36:19.615898    2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 10 17:36:19.619343 ip-10-0-141-180 kubenswrapper[2581]: W0910 17:36:19.615901    2581 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 10 17:36:19.619343 ip-10-0-141-180 kubenswrapper[2581]: W0910 17:36:19.615903    2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 10 17:36:19.619343 ip-10-0-141-180 kubenswrapper[2581]: W0910 17:36:19.615906    2581 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 10 17:36:19.619343 ip-10-0-141-180 kubenswrapper[2581]: W0910 17:36:19.615908    2581 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 10 17:36:19.619343 ip-10-0-141-180 kubenswrapper[2581]: W0910 17:36:19.615911    2581 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 10 17:36:19.619343 ip-10-0-141-180 kubenswrapper[2581]: W0910 17:36:19.615914    2581 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 10 17:36:19.619343 ip-10-0-141-180 kubenswrapper[2581]: W0910 17:36:19.615917    2581 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 10 17:36:19.619343 ip-10-0-141-180 kubenswrapper[2581]: W0910 17:36:19.615920    2581 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 10 17:36:19.619343 ip-10-0-141-180 kubenswrapper[2581]: W0910 17:36:19.615923    2581 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 10 17:36:19.619343 ip-10-0-141-180 kubenswrapper[2581]: W0910 17:36:19.615926    2581 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 10 17:36:19.619343 ip-10-0-141-180 kubenswrapper[2581]: W0910 17:36:19.615929    2581 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 10 17:36:19.619343 ip-10-0-141-180 kubenswrapper[2581]: W0910 17:36:19.615932    2581 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 10 17:36:19.619343 ip-10-0-141-180 kubenswrapper[2581]: W0910 17:36:19.615934    2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 10 17:36:19.619825 ip-10-0-141-180 kubenswrapper[2581]: W0910 17:36:19.615937    2581 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 10 17:36:19.619825 ip-10-0-141-180 kubenswrapper[2581]: W0910 17:36:19.615940    2581 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 10 17:36:19.619825 ip-10-0-141-180 kubenswrapper[2581]: W0910 17:36:19.615942    2581 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 10 17:36:19.619825 ip-10-0-141-180 kubenswrapper[2581]: W0910 17:36:19.615945    2581 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 10 17:36:19.619825 ip-10-0-141-180 kubenswrapper[2581]: W0910 17:36:19.615949    2581 feature_gate.go:328] unrecognized feature gate: Example2
Sep 10 17:36:19.619825 ip-10-0-141-180 kubenswrapper[2581]: W0910 17:36:19.615952    2581 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 10 17:36:19.619825 ip-10-0-141-180 kubenswrapper[2581]: W0910 17:36:19.615954    2581 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 10 17:36:19.619825 ip-10-0-141-180 kubenswrapper[2581]: W0910 17:36:19.615957    2581 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 10 17:36:19.619825 ip-10-0-141-180 kubenswrapper[2581]: W0910 17:36:19.615959    2581 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 10 17:36:19.619825 ip-10-0-141-180 kubenswrapper[2581]: W0910 17:36:19.615962    2581 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 10 17:36:19.619825 ip-10-0-141-180 kubenswrapper[2581]: W0910 17:36:19.615965    2581 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 10 17:36:19.619825 ip-10-0-141-180 kubenswrapper[2581]: W0910 17:36:19.615968    2581 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 10 17:36:19.619825 ip-10-0-141-180 kubenswrapper[2581]: W0910 17:36:19.615970    2581 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 10 17:36:19.619825 ip-10-0-141-180 kubenswrapper[2581]: W0910 17:36:19.615972    2581 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 10 17:36:19.619825 ip-10-0-141-180 kubenswrapper[2581]: W0910 17:36:19.615975    2581 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 10 17:36:19.619825 ip-10-0-141-180 kubenswrapper[2581]: W0910 17:36:19.615977    2581 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 10 17:36:19.619825 ip-10-0-141-180 kubenswrapper[2581]: W0910 17:36:19.615980    2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 10 17:36:19.619825 ip-10-0-141-180 kubenswrapper[2581]: W0910 17:36:19.615982    2581 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 10 17:36:19.619825 ip-10-0-141-180 kubenswrapper[2581]: W0910 17:36:19.615985    2581 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 10 17:36:19.619825 ip-10-0-141-180 kubenswrapper[2581]: W0910 17:36:19.615987    2581 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 10 17:36:19.620363 ip-10-0-141-180 kubenswrapper[2581]: W0910 17:36:19.615990    2581 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 10 17:36:19.620363 ip-10-0-141-180 kubenswrapper[2581]: W0910 17:36:19.615992    2581 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 10 17:36:19.620363 ip-10-0-141-180 kubenswrapper[2581]: W0910 17:36:19.615994    2581 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 10 17:36:19.620363 ip-10-0-141-180 kubenswrapper[2581]: W0910 17:36:19.615997    2581 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 10 17:36:19.620363 ip-10-0-141-180 kubenswrapper[2581]: W0910 17:36:19.615999    2581 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 10 17:36:19.620363 ip-10-0-141-180 kubenswrapper[2581]: W0910 17:36:19.616002    2581 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 10 17:36:19.620363 ip-10-0-141-180 kubenswrapper[2581]: W0910 17:36:19.616005    2581 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 10 17:36:19.620363 ip-10-0-141-180 kubenswrapper[2581]: W0910 17:36:19.616008    2581 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 10 17:36:19.620363 ip-10-0-141-180 kubenswrapper[2581]: W0910 17:36:19.616011    2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 10 17:36:19.620363 ip-10-0-141-180 kubenswrapper[2581]: W0910 17:36:19.616013    2581 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 10 17:36:19.620363 ip-10-0-141-180 kubenswrapper[2581]: W0910 17:36:19.616016    2581 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 10 17:36:19.620363 ip-10-0-141-180 kubenswrapper[2581]: W0910 17:36:19.616019    2581 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 10 17:36:19.620363 ip-10-0-141-180 kubenswrapper[2581]: W0910 17:36:19.616021    2581 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 10 17:36:19.620363 ip-10-0-141-180 kubenswrapper[2581]: W0910 17:36:19.616023    2581 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 10 17:36:19.620363 ip-10-0-141-180 kubenswrapper[2581]: W0910 17:36:19.616026    2581 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 10 17:36:19.620363 ip-10-0-141-180 kubenswrapper[2581]: W0910 17:36:19.616028    2581 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 10 17:36:19.620363 ip-10-0-141-180 kubenswrapper[2581]: W0910 17:36:19.616030    2581 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 10 17:36:19.620363 ip-10-0-141-180 kubenswrapper[2581]: W0910 17:36:19.616033    2581 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 10 17:36:19.620363 ip-10-0-141-180 kubenswrapper[2581]: W0910 17:36:19.616036    2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 10 17:36:19.620363 ip-10-0-141-180 kubenswrapper[2581]: W0910 17:36:19.616038    2581 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 10 17:36:19.620884 ip-10-0-141-180 kubenswrapper[2581]: W0910 17:36:19.616040    2581 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 10 17:36:19.620884 ip-10-0-141-180 kubenswrapper[2581]: W0910 17:36:19.616043    2581 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 10 17:36:19.620884 ip-10-0-141-180 kubenswrapper[2581]: W0910 17:36:19.616045    2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 10 17:36:19.620884 ip-10-0-141-180 kubenswrapper[2581]: W0910 17:36:19.616048    2581 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 10 17:36:19.620884 ip-10-0-141-180 kubenswrapper[2581]: W0910 17:36:19.616050    2581 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 10 17:36:19.620884 ip-10-0-141-180 kubenswrapper[2581]: W0910 17:36:19.616054    2581 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 10 17:36:19.620884 ip-10-0-141-180 kubenswrapper[2581]: W0910 17:36:19.616058    2581 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 10 17:36:19.620884 ip-10-0-141-180 kubenswrapper[2581]: W0910 17:36:19.616061    2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 10 17:36:19.620884 ip-10-0-141-180 kubenswrapper[2581]: W0910 17:36:19.616064    2581 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 10 17:36:19.620884 ip-10-0-141-180 kubenswrapper[2581]: W0910 17:36:19.616067    2581 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 10 17:36:19.620884 ip-10-0-141-180 kubenswrapper[2581]: W0910 17:36:19.616070    2581 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 10 17:36:19.620884 ip-10-0-141-180 kubenswrapper[2581]: W0910 17:36:19.616073    2581 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 10 17:36:19.620884 ip-10-0-141-180 kubenswrapper[2581]: W0910 17:36:19.616075    2581 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 10 17:36:19.620884 ip-10-0-141-180 kubenswrapper[2581]: W0910 17:36:19.616078    2581 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 10 17:36:19.620884 ip-10-0-141-180 kubenswrapper[2581]: W0910 17:36:19.616081    2581 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 10 17:36:19.620884 ip-10-0-141-180 kubenswrapper[2581]: W0910 17:36:19.616083    2581 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 10 17:36:19.620884 ip-10-0-141-180 kubenswrapper[2581]: W0910 17:36:19.616086    2581 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 10 17:36:19.620884 ip-10-0-141-180 kubenswrapper[2581]: W0910 17:36:19.616088    2581 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 10 17:36:19.620884 ip-10-0-141-180 kubenswrapper[2581]: W0910 17:36:19.616091    2581 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 10 17:36:19.621387 ip-10-0-141-180 kubenswrapper[2581]: W0910 17:36:19.616100    2581 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 10 17:36:19.621387 ip-10-0-141-180 kubenswrapper[2581]: W0910 17:36:19.616103    2581 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 10 17:36:19.621387 ip-10-0-141-180 kubenswrapper[2581]: W0910 17:36:19.616106    2581 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 10 17:36:19.621387 ip-10-0-141-180 kubenswrapper[2581]: W0910 17:36:19.616109    2581 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 10 17:36:19.621387 ip-10-0-141-180 kubenswrapper[2581]: W0910 17:36:19.616112    2581 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 10 17:36:19.621387 ip-10-0-141-180 kubenswrapper[2581]: W0910 17:36:19.616114    2581 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 10 17:36:19.621387 ip-10-0-141-180 kubenswrapper[2581]: W0910 17:36:19.616117    2581 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 10 17:36:19.621387 ip-10-0-141-180 kubenswrapper[2581]: W0910 17:36:19.616119    2581 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 10 17:36:19.621387 ip-10-0-141-180 kubenswrapper[2581]: W0910 17:36:19.616122    2581 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 10 17:36:19.621387 ip-10-0-141-180 kubenswrapper[2581]: W0910 17:36:19.616124    2581 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 10 17:36:19.621387 ip-10-0-141-180 kubenswrapper[2581]: W0910 17:36:19.616127    2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 10 17:36:19.621387 ip-10-0-141-180 kubenswrapper[2581]: W0910 17:36:19.616131    2581 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 10 17:36:19.621387 ip-10-0-141-180 kubenswrapper[2581]: W0910 17:36:19.616134    2581 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 10 17:36:19.621387 ip-10-0-141-180 kubenswrapper[2581]: W0910 17:36:19.616138    2581 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 10 17:36:19.621387 ip-10-0-141-180 kubenswrapper[2581]: W0910 17:36:19.616140    2581 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 10 17:36:19.621387 ip-10-0-141-180 kubenswrapper[2581]: W0910 17:36:19.616143    2581 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 10 17:36:19.621387 ip-10-0-141-180 kubenswrapper[2581]: W0910 17:36:19.616146    2581 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 10 17:36:19.621387 ip-10-0-141-180 kubenswrapper[2581]: W0910 17:36:19.616149    2581 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 10 17:36:19.621387 ip-10-0-141-180 kubenswrapper[2581]: W0910 17:36:19.616151    2581 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 10 17:36:19.621387 ip-10-0-141-180 kubenswrapper[2581]: W0910 17:36:19.616154    2581 feature_gate.go:328] unrecognized feature gate: Example
Sep 10 17:36:19.621906 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:36:19.616248    2581 flags.go:64] FLAG: --address="0.0.0.0"
Sep 10 17:36:19.621906 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:36:19.616287    2581 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]"
Sep 10 17:36:19.621906 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:36:19.616293    2581 flags.go:64] FLAG: --anonymous-auth="true"
Sep 10 17:36:19.621906 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:36:19.616298    2581 flags.go:64] FLAG: --application-metrics-count-limit="100"
Sep 10 17:36:19.621906 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:36:19.616303    2581 flags.go:64] FLAG: --authentication-token-webhook="false"
Sep 10 17:36:19.621906 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:36:19.616306    2581 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s"
Sep 10 17:36:19.621906 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:36:19.616310    2581 flags.go:64] FLAG: --authorization-mode="AlwaysAllow"
Sep 10 17:36:19.621906 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:36:19.616314    2581 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s"
Sep 10 17:36:19.621906 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:36:19.616318    2581 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s"
Sep 10 17:36:19.621906 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:36:19.616321    2581 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id"
Sep 10 17:36:19.621906 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:36:19.616325    2581 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig"
Sep 10 17:36:19.621906 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:36:19.616328    2581 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki"
Sep 10 17:36:19.621906 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:36:19.616332    2581 flags.go:64] FLAG: --cgroup-driver="cgroupfs"
Sep 10 17:36:19.621906 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:36:19.616336    2581 flags.go:64] FLAG: --cgroup-root=""
Sep 10 17:36:19.621906 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:36:19.616339    2581 flags.go:64] FLAG: --cgroups-per-qos="true"
Sep 10 17:36:19.621906 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:36:19.616342    2581 flags.go:64] FLAG: --client-ca-file=""
Sep 10 17:36:19.621906 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:36:19.616345    2581 flags.go:64] FLAG: --cloud-provider="external"
Sep 10 17:36:19.621906 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:36:19.616348    2581 flags.go:64] FLAG: --cluster-dns="[]"
Sep 10 17:36:19.621906 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:36:19.616352    2581 flags.go:64] FLAG: --cluster-domain=""
Sep 10 17:36:19.621906 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:36:19.616355    2581 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf"
Sep 10 17:36:19.621906 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:36:19.616358    2581 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d"
Sep 10 17:36:19.621906 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:36:19.616362    2581 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json"
Sep 10 17:36:19.621906 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:36:19.616365    2581 flags.go:64] FLAG: --container-log-max-files="5"
Sep 10 17:36:19.621906 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:36:19.616369    2581 flags.go:64] FLAG: --container-log-max-size="10Mi"
Sep 10 17:36:19.622513 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:36:19.616372    2581 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock"
Sep 10 17:36:19.622513 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:36:19.616375    2581 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock"
Sep 10 17:36:19.622513 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:36:19.616379    2581 flags.go:64] FLAG: --containerd-namespace="k8s.io"
Sep 10 17:36:19.622513 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:36:19.616382    2581 flags.go:64] FLAG: --contention-profiling="false"
Sep 10 17:36:19.622513 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:36:19.616385    2581 flags.go:64] FLAG: --cpu-cfs-quota="true"
Sep 10 17:36:19.622513 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:36:19.616388    2581 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms"
Sep 10 17:36:19.622513 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:36:19.616391    2581 flags.go:64] FLAG: --cpu-manager-policy="none"
Sep 10 17:36:19.622513 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:36:19.616394    2581 flags.go:64] FLAG: --cpu-manager-policy-options=""
Sep 10 17:36:19.622513 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:36:19.616398    2581 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s"
Sep 10 17:36:19.622513 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:36:19.616401    2581 flags.go:64] FLAG: --enable-controller-attach-detach="true"
Sep 10 17:36:19.622513 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:36:19.616404    2581 flags.go:64] FLAG: --enable-debugging-handlers="true"
Sep 10 17:36:19.622513 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:36:19.616407    2581 flags.go:64] FLAG: --enable-load-reader="false"
Sep 10 17:36:19.622513 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:36:19.616410    2581 flags.go:64] FLAG: --enable-server="true"
Sep 10 17:36:19.622513 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:36:19.616415    2581 flags.go:64] FLAG: --enforce-node-allocatable="[pods]"
Sep 10 17:36:19.622513 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:36:19.616420    2581 flags.go:64] FLAG: --event-burst="100"
Sep 10 17:36:19.622513 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:36:19.616423    2581 flags.go:64] FLAG: --event-qps="50"
Sep 10 17:36:19.622513 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:36:19.616426    2581 flags.go:64] FLAG: --event-storage-age-limit="default=0"
Sep 10 17:36:19.622513 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:36:19.616429    2581 flags.go:64] FLAG: --event-storage-event-limit="default=0"
Sep 10 17:36:19.622513 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:36:19.616432    2581 flags.go:64] FLAG: --eviction-hard=""
Sep 10 17:36:19.622513 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:36:19.616436    2581 flags.go:64] FLAG: --eviction-max-pod-grace-period="0"
Sep 10 17:36:19.622513 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:36:19.616438    2581 flags.go:64] FLAG: --eviction-minimum-reclaim=""
Sep 10 17:36:19.622513 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:36:19.616441    2581 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s"
Sep 10 17:36:19.622513 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:36:19.616445    2581 flags.go:64] FLAG: --eviction-soft=""
Sep 10 17:36:19.622513 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:36:19.616448    2581 flags.go:64] FLAG: --eviction-soft-grace-period=""
Sep 10 17:36:19.622513 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:36:19.616451    2581 flags.go:64] FLAG: --exit-on-lock-contention="false"
Sep 10 17:36:19.623146 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:36:19.616454    2581 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false"
Sep 10 17:36:19.623146 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:36:19.616457    2581 flags.go:64] FLAG: --experimental-mounter-path=""
Sep 10 17:36:19.623146 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:36:19.616460    2581 flags.go:64] FLAG: --fail-cgroupv1="false"
Sep 10 17:36:19.623146 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:36:19.616462    2581 flags.go:64] FLAG: --fail-swap-on="true"
Sep 10 17:36:19.623146 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:36:19.616465    2581 flags.go:64] FLAG: --feature-gates=""
Sep 10 17:36:19.623146 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:36:19.616469    2581 flags.go:64] FLAG: --file-check-frequency="20s"
Sep 10 17:36:19.623146 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:36:19.616472    2581 flags.go:64] FLAG: --global-housekeeping-interval="1m0s"
Sep 10 17:36:19.623146 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:36:19.616475    2581 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge"
Sep 10 17:36:19.623146 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:36:19.616478    2581 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1"
Sep 10 17:36:19.623146 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:36:19.616481    2581 flags.go:64] FLAG: --healthz-port="10248"
Sep 10 17:36:19.623146 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:36:19.616484    2581 flags.go:64] FLAG: --help="false"
Sep 10 17:36:19.623146 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:36:19.616487    2581 flags.go:64] FLAG: --hostname-override="ip-10-0-141-180.ec2.internal"
Sep 10 17:36:19.623146 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:36:19.616491    2581 flags.go:64] FLAG: --housekeeping-interval="10s"
Sep 10 17:36:19.623146 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:36:19.616494    2581 flags.go:64] FLAG: --http-check-frequency="20s"
Sep 10 17:36:19.623146 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:36:19.616496    2581 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins"
Sep 10 17:36:19.623146 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:36:19.616500    2581 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml"
Sep 10 17:36:19.623146 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:36:19.616503    2581 flags.go:64] FLAG: --image-gc-high-threshold="85"
Sep 10 17:36:19.623146 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:36:19.616506    2581 flags.go:64] FLAG: --image-gc-low-threshold="80"
Sep 10 17:36:19.623146 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:36:19.616509    2581 flags.go:64] FLAG: --image-service-endpoint=""
Sep 10 17:36:19.623146 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:36:19.616512    2581 flags.go:64] FLAG: --kernel-memcg-notification="false"
Sep 10 17:36:19.623146 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:36:19.616516    2581 flags.go:64] FLAG: --kube-api-burst="100"
Sep 10 17:36:19.623146 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:36:19.616519    2581 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf"
Sep 10 17:36:19.623146 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:36:19.616522    2581 flags.go:64] FLAG: --kube-api-qps="50"
Sep 10 17:36:19.623146 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:36:19.616525    2581 flags.go:64] FLAG: --kube-reserved=""
Sep 10 17:36:19.623756 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:36:19.616528    2581 flags.go:64] FLAG: --kube-reserved-cgroup=""
Sep 10 17:36:19.623756 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:36:19.616530    2581 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig"
Sep 10 17:36:19.623756 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:36:19.616534    2581 flags.go:64] FLAG: --kubelet-cgroups=""
Sep 10 17:36:19.623756 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:36:19.616536    2581 flags.go:64] FLAG: --local-storage-capacity-isolation="true"
Sep 10 17:36:19.623756 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:36:19.616539    2581 flags.go:64] FLAG: --lock-file=""
Sep 10 17:36:19.623756 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:36:19.616542    2581 flags.go:64] FLAG: --log-cadvisor-usage="false"
Sep 10 17:36:19.623756 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:36:19.616545    2581 flags.go:64] FLAG: --log-flush-frequency="5s"
Sep 10 17:36:19.623756 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:36:19.616549    2581 flags.go:64] FLAG: --log-json-info-buffer-size="0"
Sep 10 17:36:19.623756 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:36:19.616554    2581 flags.go:64] FLAG: --log-json-split-stream="false"
Sep 10 17:36:19.623756 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:36:19.616557    2581 flags.go:64] FLAG: --log-text-info-buffer-size="0"
Sep 10 17:36:19.623756 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:36:19.616560    2581 flags.go:64] FLAG: --log-text-split-stream="false"
Sep 10 17:36:19.623756 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:36:19.616563    2581 flags.go:64] FLAG: --logging-format="text"
Sep 10 17:36:19.623756 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:36:19.616565    2581 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id"
Sep 10 17:36:19.623756 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:36:19.616569    2581 flags.go:64] FLAG: --make-iptables-util-chains="true"
Sep 10 17:36:19.623756 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:36:19.616571    2581 flags.go:64] FLAG: --manifest-url=""
Sep 10 17:36:19.623756 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:36:19.616574    2581 flags.go:64] FLAG: --manifest-url-header=""
Sep 10 17:36:19.623756 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:36:19.616578    2581 flags.go:64] FLAG: --max-housekeeping-interval="15s"
Sep 10 17:36:19.623756 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:36:19.616581    2581 flags.go:64] FLAG: --max-open-files="1000000"
Sep 10 17:36:19.623756 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:36:19.616585    2581 flags.go:64] FLAG: --max-pods="110"
Sep 10 17:36:19.623756 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:36:19.616588    2581 flags.go:64] FLAG: --maximum-dead-containers="-1"
Sep 10 17:36:19.623756 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:36:19.616591    2581 flags.go:64] FLAG: --maximum-dead-containers-per-container="1"
Sep 10 17:36:19.623756 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:36:19.616594    2581 flags.go:64] FLAG: --memory-manager-policy="None"
Sep 10 17:36:19.623756 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:36:19.616597    2581 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s"
Sep 10 17:36:19.623756 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:36:19.616600    2581 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s"
Sep 10 17:36:19.623756 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:36:19.616602    2581 flags.go:64] FLAG: --node-ip="0.0.0.0"
Sep 10 17:36:19.624671 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:36:19.616605    2581 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel"
Sep 10 17:36:19.624671 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:36:19.616612    2581 flags.go:64] FLAG: --node-status-max-images="50"
Sep 10 17:36:19.624671 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:36:19.616615    2581 flags.go:64] FLAG: --node-status-update-frequency="10s"
Sep 10 17:36:19.624671 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:36:19.616619    2581 flags.go:64] FLAG: --oom-score-adj="-999"
Sep 10 17:36:19.624671 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:36:19.616622    2581 flags.go:64] FLAG: --pod-cidr=""
Sep 10 17:36:19.624671 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:36:19.616625    2581 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:8ccbacc999dbf9abad8f4009240188312b683f36ced4763eed166c1500c164b1"
Sep 10 17:36:19.624671 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:36:19.616630    2581 flags.go:64] FLAG: --pod-manifest-path=""
Sep 10 17:36:19.624671 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:36:19.616633    2581 flags.go:64] FLAG: --pod-max-pids="-1"
Sep 10 17:36:19.624671 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:36:19.616636    2581 flags.go:64] FLAG: --pods-per-core="0"
Sep 10 17:36:19.624671 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:36:19.616638    2581 flags.go:64] FLAG: --port="10250"
Sep 10 17:36:19.624671 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:36:19.616641    2581 flags.go:64] FLAG: --protect-kernel-defaults="false"
Sep 10 17:36:19.624671 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:36:19.616644    2581 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-0029b128e4469f309"
Sep 10 17:36:19.624671 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:36:19.616647    2581 flags.go:64] FLAG: --qos-reserved=""
Sep 10 17:36:19.624671 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:36:19.616650    2581 flags.go:64] FLAG: --read-only-port="10255"
Sep 10 17:36:19.624671 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:36:19.616654    2581 flags.go:64] FLAG: --register-node="true"
Sep 10 17:36:19.624671 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:36:19.616657    2581 flags.go:64] FLAG: --register-with-taints=""
Sep 10 17:36:19.624671 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:36:19.616660    2581 flags.go:64] FLAG: --registry-burst="10"
Sep 10 17:36:19.624671 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:36:19.616663    2581 flags.go:64] FLAG: --registry-qps="5"
Sep 10 17:36:19.624671 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:36:19.616666    2581 flags.go:64] FLAG: --reserved-cpus=""
Sep 10 17:36:19.624671 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:36:19.616669    2581 flags.go:64] FLAG: --reserved-memory=""
Sep 10 17:36:19.624671 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:36:19.616672    2581 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf"
Sep 10 17:36:19.624671 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:36:19.616675    2581 flags.go:64] FLAG: --root-dir="/var/lib/kubelet"
Sep 10 17:36:19.624671 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:36:19.616678    2581 flags.go:64] FLAG: --rotate-certificates="false"
Sep 10 17:36:19.624671 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:36:19.616681    2581 flags.go:64] FLAG: --rotate-server-certificates="false"
Sep 10 17:36:19.624671 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:36:19.616684    2581 flags.go:64] FLAG: --runonce="false"
Sep 10 17:36:19.625506 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:36:19.616687    2581 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service"
Sep 10 17:36:19.625506 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:36:19.616690    2581 flags.go:64] FLAG: --runtime-request-timeout="2m0s"
Sep 10 17:36:19.625506 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:36:19.616693    2581 flags.go:64] FLAG: --seccomp-default="false"
Sep 10 17:36:19.625506 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:36:19.616696    2581 flags.go:64] FLAG: --serialize-image-pulls="true"
Sep 10 17:36:19.625506 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:36:19.616699    2581 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s"
Sep 10 17:36:19.625506 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:36:19.616702    2581 flags.go:64] FLAG: --storage-driver-db="cadvisor"
Sep 10 17:36:19.625506 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:36:19.616705    2581 flags.go:64] FLAG: --storage-driver-host="localhost:8086"
Sep 10 17:36:19.625506 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:36:19.616708    2581 flags.go:64] FLAG: --storage-driver-password="root"
Sep 10 17:36:19.625506 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:36:19.616710    2581 flags.go:64] FLAG: --storage-driver-secure="false"
Sep 10 17:36:19.625506 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:36:19.616714    2581 flags.go:64] FLAG: --storage-driver-table="stats"
Sep 10 17:36:19.625506 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:36:19.616716    2581 flags.go:64] FLAG: --storage-driver-user="root"
Sep 10 17:36:19.625506 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:36:19.616721    2581 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s"
Sep 10 17:36:19.625506 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:36:19.616724    2581 flags.go:64] FLAG: --sync-frequency="1m0s"
Sep 10 17:36:19.625506 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:36:19.616727    2581 flags.go:64] FLAG: --system-cgroups=""
Sep 10 17:36:19.625506 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:36:19.616729    2581 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi"
Sep 10 17:36:19.625506 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:36:19.616735    2581 flags.go:64] FLAG: --system-reserved-cgroup=""
Sep 10 17:36:19.625506 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:36:19.616737    2581 flags.go:64] FLAG: --tls-cert-file=""
Sep 10 17:36:19.625506 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:36:19.616740    2581 flags.go:64] FLAG: --tls-cipher-suites="[]"
Sep 10 17:36:19.625506 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:36:19.616744    2581 flags.go:64] FLAG: --tls-min-version=""
Sep 10 17:36:19.625506 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:36:19.616747    2581 flags.go:64] FLAG: --tls-private-key-file=""
Sep 10 17:36:19.625506 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:36:19.616749    2581 flags.go:64] FLAG: --topology-manager-policy="none"
Sep 10 17:36:19.625506 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:36:19.616752    2581 flags.go:64] FLAG: --topology-manager-policy-options=""
Sep 10 17:36:19.625506 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:36:19.616756    2581 flags.go:64] FLAG: --topology-manager-scope="container"
Sep 10 17:36:19.625506 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:36:19.616760    2581 flags.go:64] FLAG: --v="2"
Sep 10 17:36:19.625506 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:36:19.616766    2581 flags.go:64] FLAG: --version="false"
Sep 10 17:36:19.626165 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:36:19.616775    2581 flags.go:64] FLAG: --vmodule=""
Sep 10 17:36:19.626165 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:36:19.616780    2581 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec"
Sep 10 17:36:19.626165 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:36:19.616785    2581 flags.go:64] FLAG: --volume-stats-agg-period="1m0s"
Sep 10 17:36:19.626165 ip-10-0-141-180 kubenswrapper[2581]: W0910 17:36:19.616907    2581 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 10 17:36:19.626165 ip-10-0-141-180 kubenswrapper[2581]: W0910 17:36:19.616912    2581 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 10 17:36:19.626165 ip-10-0-141-180 kubenswrapper[2581]: W0910 17:36:19.616917    2581 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 10 17:36:19.626165 ip-10-0-141-180 kubenswrapper[2581]: W0910 17:36:19.616920    2581 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 10 17:36:19.626165 ip-10-0-141-180 kubenswrapper[2581]: W0910 17:36:19.616923    2581 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 10 17:36:19.626165 ip-10-0-141-180 kubenswrapper[2581]: W0910 17:36:19.616926    2581 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 10 17:36:19.626165 ip-10-0-141-180 kubenswrapper[2581]: W0910 17:36:19.616929    2581 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 10 17:36:19.626165 ip-10-0-141-180 kubenswrapper[2581]: W0910 17:36:19.616931    2581 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 10 17:36:19.626165 ip-10-0-141-180 kubenswrapper[2581]: W0910 17:36:19.616934    2581 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 10 17:36:19.626165 ip-10-0-141-180 kubenswrapper[2581]: W0910 17:36:19.616937    2581 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 10 17:36:19.626165 ip-10-0-141-180 kubenswrapper[2581]: W0910 17:36:19.616939    2581 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 10 17:36:19.626165 ip-10-0-141-180 kubenswrapper[2581]: W0910 17:36:19.616942    2581 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 10 17:36:19.626165 ip-10-0-141-180 kubenswrapper[2581]: W0910 17:36:19.616944    2581 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 10 17:36:19.626165 ip-10-0-141-180 kubenswrapper[2581]: W0910 17:36:19.616947    2581 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 10 17:36:19.626165 ip-10-0-141-180 kubenswrapper[2581]: W0910 17:36:19.616949    2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 10 17:36:19.626165 ip-10-0-141-180 kubenswrapper[2581]: W0910 17:36:19.616953    2581 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 10 17:36:19.626165 ip-10-0-141-180 kubenswrapper[2581]: W0910 17:36:19.616955    2581 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 10 17:36:19.626718 ip-10-0-141-180 kubenswrapper[2581]: W0910 17:36:19.616958    2581 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 10 17:36:19.626718 ip-10-0-141-180 kubenswrapper[2581]: W0910 17:36:19.616960    2581 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 10 17:36:19.626718 ip-10-0-141-180 kubenswrapper[2581]: W0910 17:36:19.616963    2581 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 10 17:36:19.626718 ip-10-0-141-180 kubenswrapper[2581]: W0910 17:36:19.616966    2581 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 10 17:36:19.626718 ip-10-0-141-180 kubenswrapper[2581]: W0910 17:36:19.616968    2581 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 10 17:36:19.626718 ip-10-0-141-180 kubenswrapper[2581]: W0910 17:36:19.616971    2581 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 10 17:36:19.626718 ip-10-0-141-180 kubenswrapper[2581]: W0910 17:36:19.616973    2581 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 10 17:36:19.626718 ip-10-0-141-180 kubenswrapper[2581]: W0910 17:36:19.616976    2581 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 10 17:36:19.626718 ip-10-0-141-180 kubenswrapper[2581]: W0910 17:36:19.616979    2581 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 10 17:36:19.626718 ip-10-0-141-180 kubenswrapper[2581]: W0910 17:36:19.616982    2581 feature_gate.go:328] unrecognized feature gate: Example
Sep 10 17:36:19.626718 ip-10-0-141-180 kubenswrapper[2581]: W0910 17:36:19.616985    2581 feature_gate.go:328] unrecognized feature gate: Example2
Sep 10 17:36:19.626718 ip-10-0-141-180 kubenswrapper[2581]: W0910 17:36:19.616988    2581 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 10 17:36:19.626718 ip-10-0-141-180 kubenswrapper[2581]: W0910 17:36:19.616990    2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 10 17:36:19.626718 ip-10-0-141-180 kubenswrapper[2581]: W0910 17:36:19.616993    2581 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 10 17:36:19.626718 ip-10-0-141-180 kubenswrapper[2581]: W0910 17:36:19.616995    2581 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 10 17:36:19.626718 ip-10-0-141-180 kubenswrapper[2581]: W0910 17:36:19.616998    2581 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 10 17:36:19.626718 ip-10-0-141-180 kubenswrapper[2581]: W0910 17:36:19.617000    2581 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 10 17:36:19.626718 ip-10-0-141-180 kubenswrapper[2581]: W0910 17:36:19.617004    2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 10 17:36:19.626718 ip-10-0-141-180 kubenswrapper[2581]: W0910 17:36:19.617007    2581 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 10 17:36:19.627167 ip-10-0-141-180 kubenswrapper[2581]: W0910 17:36:19.617010    2581 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 10 17:36:19.627167 ip-10-0-141-180 kubenswrapper[2581]: W0910 17:36:19.617012    2581 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 10 17:36:19.627167 ip-10-0-141-180 kubenswrapper[2581]: W0910 17:36:19.617015    2581 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 10 17:36:19.627167 ip-10-0-141-180 kubenswrapper[2581]: W0910 17:36:19.617018    2581 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 10 17:36:19.627167 ip-10-0-141-180 kubenswrapper[2581]: W0910 17:36:19.617020    2581 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 10 17:36:19.627167 ip-10-0-141-180 kubenswrapper[2581]: W0910 17:36:19.617023    2581 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 10 17:36:19.627167 ip-10-0-141-180 kubenswrapper[2581]: W0910 17:36:19.617025    2581 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 10 17:36:19.627167 ip-10-0-141-180 kubenswrapper[2581]: W0910 17:36:19.617028    2581 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 10 17:36:19.627167 ip-10-0-141-180 kubenswrapper[2581]: W0910 17:36:19.617030    2581 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 10 17:36:19.627167 ip-10-0-141-180 kubenswrapper[2581]: W0910 17:36:19.617033    2581 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 10 17:36:19.627167 ip-10-0-141-180 kubenswrapper[2581]: W0910 17:36:19.617036    2581 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 10 17:36:19.627167 ip-10-0-141-180 kubenswrapper[2581]: W0910 17:36:19.617039    2581 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 10 17:36:19.627167 ip-10-0-141-180 kubenswrapper[2581]: W0910 17:36:19.617041    2581 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 10 17:36:19.627167 ip-10-0-141-180 kubenswrapper[2581]: W0910 17:36:19.617044    2581 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 10 17:36:19.627167 ip-10-0-141-180 kubenswrapper[2581]: W0910 17:36:19.617046    2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 10 17:36:19.627167 ip-10-0-141-180 kubenswrapper[2581]: W0910 17:36:19.617049    2581 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 10 17:36:19.627167 ip-10-0-141-180 kubenswrapper[2581]: W0910 17:36:19.617052    2581 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 10 17:36:19.627167 ip-10-0-141-180 kubenswrapper[2581]: W0910 17:36:19.617055    2581 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 10 17:36:19.627167 ip-10-0-141-180 kubenswrapper[2581]: W0910 17:36:19.617057    2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 10 17:36:19.627167 ip-10-0-141-180 kubenswrapper[2581]: W0910 17:36:19.617060    2581 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 10 17:36:19.627680 ip-10-0-141-180 kubenswrapper[2581]: W0910 17:36:19.617062    2581 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 10 17:36:19.627680 ip-10-0-141-180 kubenswrapper[2581]: W0910 17:36:19.617065    2581 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 10 17:36:19.627680 ip-10-0-141-180 kubenswrapper[2581]: W0910 17:36:19.617067    2581 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 10 17:36:19.627680 ip-10-0-141-180 kubenswrapper[2581]: W0910 17:36:19.617070    2581 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 10 17:36:19.627680 ip-10-0-141-180 kubenswrapper[2581]: W0910 17:36:19.617073    2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 10 17:36:19.627680 ip-10-0-141-180 kubenswrapper[2581]: W0910 17:36:19.617077    2581 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 10 17:36:19.627680 ip-10-0-141-180 kubenswrapper[2581]: W0910 17:36:19.617081    2581 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 10 17:36:19.627680 ip-10-0-141-180 kubenswrapper[2581]: W0910 17:36:19.617084    2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 10 17:36:19.627680 ip-10-0-141-180 kubenswrapper[2581]: W0910 17:36:19.617087    2581 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 10 17:36:19.627680 ip-10-0-141-180 kubenswrapper[2581]: W0910 17:36:19.617090    2581 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 10 17:36:19.627680 ip-10-0-141-180 kubenswrapper[2581]: W0910 17:36:19.617094    2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 10 17:36:19.627680 ip-10-0-141-180 kubenswrapper[2581]: W0910 17:36:19.617097    2581 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 10 17:36:19.627680 ip-10-0-141-180 kubenswrapper[2581]: W0910 17:36:19.617099    2581 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 10 17:36:19.627680 ip-10-0-141-180 kubenswrapper[2581]: W0910 17:36:19.617102    2581 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 10 17:36:19.627680 ip-10-0-141-180 kubenswrapper[2581]: W0910 17:36:19.617105    2581 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 10 17:36:19.627680 ip-10-0-141-180 kubenswrapper[2581]: W0910 17:36:19.617107    2581 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 10 17:36:19.627680 ip-10-0-141-180 kubenswrapper[2581]: W0910 17:36:19.617110    2581 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 10 17:36:19.627680 ip-10-0-141-180 kubenswrapper[2581]: W0910 17:36:19.617112    2581 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 10 17:36:19.627680 ip-10-0-141-180 kubenswrapper[2581]: W0910 17:36:19.617115    2581 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 10 17:36:19.627680 ip-10-0-141-180 kubenswrapper[2581]: W0910 17:36:19.617118    2581 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 10 17:36:19.628175 ip-10-0-141-180 kubenswrapper[2581]: W0910 17:36:19.617120    2581 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 10 17:36:19.628175 ip-10-0-141-180 kubenswrapper[2581]: W0910 17:36:19.617123    2581 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 10 17:36:19.628175 ip-10-0-141-180 kubenswrapper[2581]: W0910 17:36:19.617125    2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 10 17:36:19.628175 ip-10-0-141-180 kubenswrapper[2581]: W0910 17:36:19.617128    2581 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 10 17:36:19.628175 ip-10-0-141-180 kubenswrapper[2581]: W0910 17:36:19.617131    2581 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 10 17:36:19.628175 ip-10-0-141-180 kubenswrapper[2581]: W0910 17:36:19.617133    2581 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 10 17:36:19.628175 ip-10-0-141-180 kubenswrapper[2581]: W0910 17:36:19.617136    2581 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 10 17:36:19.628175 ip-10-0-141-180 kubenswrapper[2581]: W0910 17:36:19.617139    2581 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 10 17:36:19.628175 ip-10-0-141-180 kubenswrapper[2581]: W0910 17:36:19.617141    2581 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 10 17:36:19.628175 ip-10-0-141-180 kubenswrapper[2581]: W0910 17:36:19.617144    2581 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 10 17:36:19.628175 ip-10-0-141-180 kubenswrapper[2581]: W0910 17:36:19.617146    2581 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 10 17:36:19.628175 ip-10-0-141-180 kubenswrapper[2581]: W0910 17:36:19.617149    2581 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 10 17:36:19.628175 ip-10-0-141-180 kubenswrapper[2581]: W0910 17:36:19.617151    2581 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 10 17:36:19.628175 ip-10-0-141-180 kubenswrapper[2581]: W0910 17:36:19.617154    2581 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 10 17:36:19.628175 ip-10-0-141-180 kubenswrapper[2581]: W0910 17:36:19.617156    2581 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 10 17:36:19.628175 ip-10-0-141-180 kubenswrapper[2581]: W0910 17:36:19.617160    2581 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 10 17:36:19.628175 ip-10-0-141-180 kubenswrapper[2581]: W0910 17:36:19.617163    2581 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 10 17:36:19.628175 ip-10-0-141-180 kubenswrapper[2581]: W0910 17:36:19.617167    2581 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 10 17:36:19.628175 ip-10-0-141-180 kubenswrapper[2581]: W0910 17:36:19.617170    2581 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 10 17:36:19.628652 ip-10-0-141-180 kubenswrapper[2581]: W0910 17:36:19.617172    2581 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 10 17:36:19.628652 ip-10-0-141-180 kubenswrapper[2581]: W0910 17:36:19.617175    2581 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 10 17:36:19.628652 ip-10-0-141-180 kubenswrapper[2581]: W0910 17:36:19.617178    2581 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 10 17:36:19.628652 ip-10-0-141-180 kubenswrapper[2581]: W0910 17:36:19.617182    2581 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 10 17:36:19.628652 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:36:19.617203    2581 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Sep 10 17:36:19.628652 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:36:19.625713    2581 server.go:529] "Kubelet version" kubeletVersion="v1.34.9"
Sep 10 17:36:19.628652 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:36:19.625729    2581 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK=""
Sep 10 17:36:19.628652 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:36:19.627268    2581 watchdog_linux.go:95] "Systemd watchdog is not enabled"
Sep 10 17:36:19.628652 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:36:19.627280    2581 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started."
Sep 10 17:36:19.628652 ip-10-0-141-180 kubenswrapper[2581]: W0910 17:36:19.627338    2581 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 10 17:36:19.628652 ip-10-0-141-180 kubenswrapper[2581]: W0910 17:36:19.627344    2581 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 10 17:36:19.628652 ip-10-0-141-180 kubenswrapper[2581]: W0910 17:36:19.627348    2581 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 10 17:36:19.628652 ip-10-0-141-180 kubenswrapper[2581]: W0910 17:36:19.627351    2581 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 10 17:36:19.628652 ip-10-0-141-180 kubenswrapper[2581]: W0910 17:36:19.627354    2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 10 17:36:19.628652 ip-10-0-141-180 kubenswrapper[2581]: W0910 17:36:19.627357    2581 feature_gate.go:328] unrecognized feature gate: Example2
Sep 10 17:36:19.629029 ip-10-0-141-180 kubenswrapper[2581]: W0910 17:36:19.627360    2581 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 10 17:36:19.629029 ip-10-0-141-180 kubenswrapper[2581]: W0910 17:36:19.627362    2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 10 17:36:19.629029 ip-10-0-141-180 kubenswrapper[2581]: W0910 17:36:19.627365    2581 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 10 17:36:19.629029 ip-10-0-141-180 kubenswrapper[2581]: W0910 17:36:19.627368    2581 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 10 17:36:19.629029 ip-10-0-141-180 kubenswrapper[2581]: W0910 17:36:19.627370    2581 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 10 17:36:19.629029 ip-10-0-141-180 kubenswrapper[2581]: W0910 17:36:19.627375    2581 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 10 17:36:19.629029 ip-10-0-141-180 kubenswrapper[2581]: W0910 17:36:19.627379    2581 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 10 17:36:19.629029 ip-10-0-141-180 kubenswrapper[2581]: W0910 17:36:19.627382    2581 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 10 17:36:19.629029 ip-10-0-141-180 kubenswrapper[2581]: W0910 17:36:19.627385    2581 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 10 17:36:19.629029 ip-10-0-141-180 kubenswrapper[2581]: W0910 17:36:19.627388    2581 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 10 17:36:19.629029 ip-10-0-141-180 kubenswrapper[2581]: W0910 17:36:19.627391    2581 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 10 17:36:19.629029 ip-10-0-141-180 kubenswrapper[2581]: W0910 17:36:19.627394    2581 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 10 17:36:19.629029 ip-10-0-141-180 kubenswrapper[2581]: W0910 17:36:19.627397    2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 10 17:36:19.629029 ip-10-0-141-180 kubenswrapper[2581]: W0910 17:36:19.627399    2581 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 10 17:36:19.629029 ip-10-0-141-180 kubenswrapper[2581]: W0910 17:36:19.627402    2581 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 10 17:36:19.629029 ip-10-0-141-180 kubenswrapper[2581]: W0910 17:36:19.627404    2581 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 10 17:36:19.629029 ip-10-0-141-180 kubenswrapper[2581]: W0910 17:36:19.627407    2581 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 10 17:36:19.629029 ip-10-0-141-180 kubenswrapper[2581]: W0910 17:36:19.627410    2581 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 10 17:36:19.629029 ip-10-0-141-180 kubenswrapper[2581]: W0910 17:36:19.627412    2581 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 10 17:36:19.629029 ip-10-0-141-180 kubenswrapper[2581]: W0910 17:36:19.627415    2581 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 10 17:36:19.629552 ip-10-0-141-180 kubenswrapper[2581]: W0910 17:36:19.627417    2581 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 10 17:36:19.629552 ip-10-0-141-180 kubenswrapper[2581]: W0910 17:36:19.627420    2581 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 10 17:36:19.629552 ip-10-0-141-180 kubenswrapper[2581]: W0910 17:36:19.627428    2581 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 10 17:36:19.629552 ip-10-0-141-180 kubenswrapper[2581]: W0910 17:36:19.627431    2581 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 10 17:36:19.629552 ip-10-0-141-180 kubenswrapper[2581]: W0910 17:36:19.627434    2581 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 10 17:36:19.629552 ip-10-0-141-180 kubenswrapper[2581]: W0910 17:36:19.627437    2581 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 10 17:36:19.629552 ip-10-0-141-180 kubenswrapper[2581]: W0910 17:36:19.627440    2581 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 10 17:36:19.629552 ip-10-0-141-180 kubenswrapper[2581]: W0910 17:36:19.627442    2581 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 10 17:36:19.629552 ip-10-0-141-180 kubenswrapper[2581]: W0910 17:36:19.627445    2581 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 10 17:36:19.629552 ip-10-0-141-180 kubenswrapper[2581]: W0910 17:36:19.627447    2581 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 10 17:36:19.629552 ip-10-0-141-180 kubenswrapper[2581]: W0910 17:36:19.627450    2581 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 10 17:36:19.629552 ip-10-0-141-180 kubenswrapper[2581]: W0910 17:36:19.627453    2581 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 10 17:36:19.629552 ip-10-0-141-180 kubenswrapper[2581]: W0910 17:36:19.627456    2581 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 10 17:36:19.629552 ip-10-0-141-180 kubenswrapper[2581]: W0910 17:36:19.627458    2581 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 10 17:36:19.629552 ip-10-0-141-180 kubenswrapper[2581]: W0910 17:36:19.627461    2581 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 10 17:36:19.629552 ip-10-0-141-180 kubenswrapper[2581]: W0910 17:36:19.627463    2581 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 10 17:36:19.629552 ip-10-0-141-180 kubenswrapper[2581]: W0910 17:36:19.627466    2581 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 10 17:36:19.629552 ip-10-0-141-180 kubenswrapper[2581]: W0910 17:36:19.627468    2581 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 10 17:36:19.629552 ip-10-0-141-180 kubenswrapper[2581]: W0910 17:36:19.627471    2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 10 17:36:19.629552 ip-10-0-141-180 kubenswrapper[2581]: W0910 17:36:19.627473    2581 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 10 17:36:19.630035 ip-10-0-141-180 kubenswrapper[2581]: W0910 17:36:19.627475    2581 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 10 17:36:19.630035 ip-10-0-141-180 kubenswrapper[2581]: W0910 17:36:19.627478    2581 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 10 17:36:19.630035 ip-10-0-141-180 kubenswrapper[2581]: W0910 17:36:19.627480    2581 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 10 17:36:19.630035 ip-10-0-141-180 kubenswrapper[2581]: W0910 17:36:19.627483    2581 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 10 17:36:19.630035 ip-10-0-141-180 kubenswrapper[2581]: W0910 17:36:19.627485    2581 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 10 17:36:19.630035 ip-10-0-141-180 kubenswrapper[2581]: W0910 17:36:19.627488    2581 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 10 17:36:19.630035 ip-10-0-141-180 kubenswrapper[2581]: W0910 17:36:19.627490    2581 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 10 17:36:19.630035 ip-10-0-141-180 kubenswrapper[2581]: W0910 17:36:19.627493    2581 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 10 17:36:19.630035 ip-10-0-141-180 kubenswrapper[2581]: W0910 17:36:19.627496    2581 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 10 17:36:19.630035 ip-10-0-141-180 kubenswrapper[2581]: W0910 17:36:19.627498    2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 10 17:36:19.630035 ip-10-0-141-180 kubenswrapper[2581]: W0910 17:36:19.627501    2581 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 10 17:36:19.630035 ip-10-0-141-180 kubenswrapper[2581]: W0910 17:36:19.627503    2581 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 10 17:36:19.630035 ip-10-0-141-180 kubenswrapper[2581]: W0910 17:36:19.627506    2581 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 10 17:36:19.630035 ip-10-0-141-180 kubenswrapper[2581]: W0910 17:36:19.627509    2581 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 10 17:36:19.630035 ip-10-0-141-180 kubenswrapper[2581]: W0910 17:36:19.627511    2581 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 10 17:36:19.630035 ip-10-0-141-180 kubenswrapper[2581]: W0910 17:36:19.627514    2581 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 10 17:36:19.630035 ip-10-0-141-180 kubenswrapper[2581]: W0910 17:36:19.627517    2581 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 10 17:36:19.630035 ip-10-0-141-180 kubenswrapper[2581]: W0910 17:36:19.627520    2581 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 10 17:36:19.630035 ip-10-0-141-180 kubenswrapper[2581]: W0910 17:36:19.627523    2581 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 10 17:36:19.630035 ip-10-0-141-180 kubenswrapper[2581]: W0910 17:36:19.627525    2581 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 10 17:36:19.630602 ip-10-0-141-180 kubenswrapper[2581]: W0910 17:36:19.627528    2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 10 17:36:19.630602 ip-10-0-141-180 kubenswrapper[2581]: W0910 17:36:19.627530    2581 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 10 17:36:19.630602 ip-10-0-141-180 kubenswrapper[2581]: W0910 17:36:19.627534    2581 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 10 17:36:19.630602 ip-10-0-141-180 kubenswrapper[2581]: W0910 17:36:19.627538    2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 10 17:36:19.630602 ip-10-0-141-180 kubenswrapper[2581]: W0910 17:36:19.627541    2581 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 10 17:36:19.630602 ip-10-0-141-180 kubenswrapper[2581]: W0910 17:36:19.627544    2581 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 10 17:36:19.630602 ip-10-0-141-180 kubenswrapper[2581]: W0910 17:36:19.627547    2581 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 10 17:36:19.630602 ip-10-0-141-180 kubenswrapper[2581]: W0910 17:36:19.627549    2581 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 10 17:36:19.630602 ip-10-0-141-180 kubenswrapper[2581]: W0910 17:36:19.627552    2581 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 10 17:36:19.630602 ip-10-0-141-180 kubenswrapper[2581]: W0910 17:36:19.627554    2581 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 10 17:36:19.630602 ip-10-0-141-180 kubenswrapper[2581]: W0910 17:36:19.627557    2581 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 10 17:36:19.630602 ip-10-0-141-180 kubenswrapper[2581]: W0910 17:36:19.627559    2581 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 10 17:36:19.630602 ip-10-0-141-180 kubenswrapper[2581]: W0910 17:36:19.627562    2581 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 10 17:36:19.630602 ip-10-0-141-180 kubenswrapper[2581]: W0910 17:36:19.627566    2581 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 10 17:36:19.630602 ip-10-0-141-180 kubenswrapper[2581]: W0910 17:36:19.627570    2581 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 10 17:36:19.630602 ip-10-0-141-180 kubenswrapper[2581]: W0910 17:36:19.627572    2581 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 10 17:36:19.630602 ip-10-0-141-180 kubenswrapper[2581]: W0910 17:36:19.627575    2581 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 10 17:36:19.630602 ip-10-0-141-180 kubenswrapper[2581]: W0910 17:36:19.627577    2581 feature_gate.go:328] unrecognized feature gate: Example
Sep 10 17:36:19.630602 ip-10-0-141-180 kubenswrapper[2581]: W0910 17:36:19.627580    2581 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 10 17:36:19.630602 ip-10-0-141-180 kubenswrapper[2581]: W0910 17:36:19.627582    2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 10 17:36:19.631090 ip-10-0-141-180 kubenswrapper[2581]: W0910 17:36:19.627585    2581 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 10 17:36:19.631090 ip-10-0-141-180 kubenswrapper[2581]: W0910 17:36:19.627587    2581 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 10 17:36:19.631090 ip-10-0-141-180 kubenswrapper[2581]: W0910 17:36:19.627590    2581 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 10 17:36:19.631090 ip-10-0-141-180 kubenswrapper[2581]: W0910 17:36:19.627593    2581 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 10 17:36:19.631090 ip-10-0-141-180 kubenswrapper[2581]: W0910 17:36:19.627595    2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 10 17:36:19.631090 ip-10-0-141-180 kubenswrapper[2581]: W0910 17:36:19.627598    2581 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 10 17:36:19.631090 ip-10-0-141-180 kubenswrapper[2581]: W0910 17:36:19.627601    2581 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 10 17:36:19.631090 ip-10-0-141-180 kubenswrapper[2581]: W0910 17:36:19.627605    2581 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 10 17:36:19.631090 ip-10-0-141-180 kubenswrapper[2581]: W0910 17:36:19.627609    2581 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 10 17:36:19.631090 ip-10-0-141-180 kubenswrapper[2581]: W0910 17:36:19.627612    2581 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 10 17:36:19.631090 ip-10-0-141-180 kubenswrapper[2581]: W0910 17:36:19.627614    2581 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 10 17:36:19.631090 ip-10-0-141-180 kubenswrapper[2581]: W0910 17:36:19.627617    2581 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 10 17:36:19.631090 ip-10-0-141-180 kubenswrapper[2581]: W0910 17:36:19.627619    2581 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 10 17:36:19.631090 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:36:19.627624    2581 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Sep 10 17:36:19.631440 ip-10-0-141-180 kubenswrapper[2581]: W0910 17:36:19.627748    2581 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 10 17:36:19.631440 ip-10-0-141-180 kubenswrapper[2581]: W0910 17:36:19.627754    2581 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 10 17:36:19.631440 ip-10-0-141-180 kubenswrapper[2581]: W0910 17:36:19.627757    2581 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 10 17:36:19.631440 ip-10-0-141-180 kubenswrapper[2581]: W0910 17:36:19.627760    2581 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 10 17:36:19.631440 ip-10-0-141-180 kubenswrapper[2581]: W0910 17:36:19.627763    2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 10 17:36:19.631440 ip-10-0-141-180 kubenswrapper[2581]: W0910 17:36:19.627765    2581 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 10 17:36:19.631440 ip-10-0-141-180 kubenswrapper[2581]: W0910 17:36:19.627768    2581 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 10 17:36:19.631440 ip-10-0-141-180 kubenswrapper[2581]: W0910 17:36:19.627770    2581 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 10 17:36:19.631440 ip-10-0-141-180 kubenswrapper[2581]: W0910 17:36:19.627773    2581 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 10 17:36:19.631440 ip-10-0-141-180 kubenswrapper[2581]: W0910 17:36:19.627776    2581 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 10 17:36:19.631440 ip-10-0-141-180 kubenswrapper[2581]: W0910 17:36:19.627779    2581 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 10 17:36:19.631440 ip-10-0-141-180 kubenswrapper[2581]: W0910 17:36:19.627781    2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 10 17:36:19.631440 ip-10-0-141-180 kubenswrapper[2581]: W0910 17:36:19.627785    2581 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 10 17:36:19.631440 ip-10-0-141-180 kubenswrapper[2581]: W0910 17:36:19.627788    2581 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 10 17:36:19.631440 ip-10-0-141-180 kubenswrapper[2581]: W0910 17:36:19.627790    2581 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 10 17:36:19.631440 ip-10-0-141-180 kubenswrapper[2581]: W0910 17:36:19.627793    2581 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 10 17:36:19.631440 ip-10-0-141-180 kubenswrapper[2581]: W0910 17:36:19.627796    2581 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 10 17:36:19.631440 ip-10-0-141-180 kubenswrapper[2581]: W0910 17:36:19.627798    2581 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 10 17:36:19.631440 ip-10-0-141-180 kubenswrapper[2581]: W0910 17:36:19.627801    2581 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 10 17:36:19.631882 ip-10-0-141-180 kubenswrapper[2581]: W0910 17:36:19.627803    2581 feature_gate.go:328] unrecognized feature gate: Example
Sep 10 17:36:19.631882 ip-10-0-141-180 kubenswrapper[2581]: W0910 17:36:19.627806    2581 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 10 17:36:19.631882 ip-10-0-141-180 kubenswrapper[2581]: W0910 17:36:19.627808    2581 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 10 17:36:19.631882 ip-10-0-141-180 kubenswrapper[2581]: W0910 17:36:19.627811    2581 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 10 17:36:19.631882 ip-10-0-141-180 kubenswrapper[2581]: W0910 17:36:19.627813    2581 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 10 17:36:19.631882 ip-10-0-141-180 kubenswrapper[2581]: W0910 17:36:19.627817    2581 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 10 17:36:19.631882 ip-10-0-141-180 kubenswrapper[2581]: W0910 17:36:19.627821    2581 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 10 17:36:19.631882 ip-10-0-141-180 kubenswrapper[2581]: W0910 17:36:19.627824    2581 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 10 17:36:19.631882 ip-10-0-141-180 kubenswrapper[2581]: W0910 17:36:19.627827    2581 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 10 17:36:19.631882 ip-10-0-141-180 kubenswrapper[2581]: W0910 17:36:19.627830    2581 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 10 17:36:19.631882 ip-10-0-141-180 kubenswrapper[2581]: W0910 17:36:19.627832    2581 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 10 17:36:19.631882 ip-10-0-141-180 kubenswrapper[2581]: W0910 17:36:19.627835    2581 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 10 17:36:19.631882 ip-10-0-141-180 kubenswrapper[2581]: W0910 17:36:19.627838    2581 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 10 17:36:19.631882 ip-10-0-141-180 kubenswrapper[2581]: W0910 17:36:19.627840    2581 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 10 17:36:19.631882 ip-10-0-141-180 kubenswrapper[2581]: W0910 17:36:19.627843    2581 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 10 17:36:19.631882 ip-10-0-141-180 kubenswrapper[2581]: W0910 17:36:19.627845    2581 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 10 17:36:19.631882 ip-10-0-141-180 kubenswrapper[2581]: W0910 17:36:19.627848    2581 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 10 17:36:19.631882 ip-10-0-141-180 kubenswrapper[2581]: W0910 17:36:19.627850    2581 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 10 17:36:19.631882 ip-10-0-141-180 kubenswrapper[2581]: W0910 17:36:19.627853    2581 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 10 17:36:19.631882 ip-10-0-141-180 kubenswrapper[2581]: W0910 17:36:19.627855    2581 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 10 17:36:19.632404 ip-10-0-141-180 kubenswrapper[2581]: W0910 17:36:19.627858    2581 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 10 17:36:19.632404 ip-10-0-141-180 kubenswrapper[2581]: W0910 17:36:19.627860    2581 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 10 17:36:19.632404 ip-10-0-141-180 kubenswrapper[2581]: W0910 17:36:19.627863    2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 10 17:36:19.632404 ip-10-0-141-180 kubenswrapper[2581]: W0910 17:36:19.627865    2581 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 10 17:36:19.632404 ip-10-0-141-180 kubenswrapper[2581]: W0910 17:36:19.627867    2581 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 10 17:36:19.632404 ip-10-0-141-180 kubenswrapper[2581]: W0910 17:36:19.627870    2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 10 17:36:19.632404 ip-10-0-141-180 kubenswrapper[2581]: W0910 17:36:19.627873    2581 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 10 17:36:19.632404 ip-10-0-141-180 kubenswrapper[2581]: W0910 17:36:19.627875    2581 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 10 17:36:19.632404 ip-10-0-141-180 kubenswrapper[2581]: W0910 17:36:19.627878    2581 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 10 17:36:19.632404 ip-10-0-141-180 kubenswrapper[2581]: W0910 17:36:19.627880    2581 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 10 17:36:19.632404 ip-10-0-141-180 kubenswrapper[2581]: W0910 17:36:19.627883    2581 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 10 17:36:19.632404 ip-10-0-141-180 kubenswrapper[2581]: W0910 17:36:19.627885    2581 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 10 17:36:19.632404 ip-10-0-141-180 kubenswrapper[2581]: W0910 17:36:19.627887    2581 feature_gate.go:328] unrecognized feature gate: Example2
Sep 10 17:36:19.632404 ip-10-0-141-180 kubenswrapper[2581]: W0910 17:36:19.627890    2581 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 10 17:36:19.632404 ip-10-0-141-180 kubenswrapper[2581]: W0910 17:36:19.627892    2581 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 10 17:36:19.632404 ip-10-0-141-180 kubenswrapper[2581]: W0910 17:36:19.627895    2581 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 10 17:36:19.632404 ip-10-0-141-180 kubenswrapper[2581]: W0910 17:36:19.627897    2581 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 10 17:36:19.632404 ip-10-0-141-180 kubenswrapper[2581]: W0910 17:36:19.627900    2581 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 10 17:36:19.632404 ip-10-0-141-180 kubenswrapper[2581]: W0910 17:36:19.627903    2581 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 10 17:36:19.632404 ip-10-0-141-180 kubenswrapper[2581]: W0910 17:36:19.627906    2581 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 10 17:36:19.632892 ip-10-0-141-180 kubenswrapper[2581]: W0910 17:36:19.627909    2581 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 10 17:36:19.632892 ip-10-0-141-180 kubenswrapper[2581]: W0910 17:36:19.627912    2581 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 10 17:36:19.632892 ip-10-0-141-180 kubenswrapper[2581]: W0910 17:36:19.627916    2581 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 10 17:36:19.632892 ip-10-0-141-180 kubenswrapper[2581]: W0910 17:36:19.627919    2581 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 10 17:36:19.632892 ip-10-0-141-180 kubenswrapper[2581]: W0910 17:36:19.627922    2581 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 10 17:36:19.632892 ip-10-0-141-180 kubenswrapper[2581]: W0910 17:36:19.627925    2581 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 10 17:36:19.632892 ip-10-0-141-180 kubenswrapper[2581]: W0910 17:36:19.627927    2581 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 10 17:36:19.632892 ip-10-0-141-180 kubenswrapper[2581]: W0910 17:36:19.627930    2581 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 10 17:36:19.632892 ip-10-0-141-180 kubenswrapper[2581]: W0910 17:36:19.627932    2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 10 17:36:19.632892 ip-10-0-141-180 kubenswrapper[2581]: W0910 17:36:19.627935    2581 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 10 17:36:19.632892 ip-10-0-141-180 kubenswrapper[2581]: W0910 17:36:19.627937    2581 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 10 17:36:19.632892 ip-10-0-141-180 kubenswrapper[2581]: W0910 17:36:19.627940    2581 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 10 17:36:19.632892 ip-10-0-141-180 kubenswrapper[2581]: W0910 17:36:19.627942    2581 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 10 17:36:19.632892 ip-10-0-141-180 kubenswrapper[2581]: W0910 17:36:19.627945    2581 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 10 17:36:19.632892 ip-10-0-141-180 kubenswrapper[2581]: W0910 17:36:19.627947    2581 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 10 17:36:19.632892 ip-10-0-141-180 kubenswrapper[2581]: W0910 17:36:19.627950    2581 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 10 17:36:19.632892 ip-10-0-141-180 kubenswrapper[2581]: W0910 17:36:19.627952    2581 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 10 17:36:19.632892 ip-10-0-141-180 kubenswrapper[2581]: W0910 17:36:19.627955    2581 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 10 17:36:19.632892 ip-10-0-141-180 kubenswrapper[2581]: W0910 17:36:19.627957    2581 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 10 17:36:19.632892 ip-10-0-141-180 kubenswrapper[2581]: W0910 17:36:19.627960    2581 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 10 17:36:19.633386 ip-10-0-141-180 kubenswrapper[2581]: W0910 17:36:19.627962    2581 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 10 17:36:19.633386 ip-10-0-141-180 kubenswrapper[2581]: W0910 17:36:19.627965    2581 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 10 17:36:19.633386 ip-10-0-141-180 kubenswrapper[2581]: W0910 17:36:19.627968    2581 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 10 17:36:19.633386 ip-10-0-141-180 kubenswrapper[2581]: W0910 17:36:19.627970    2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 10 17:36:19.633386 ip-10-0-141-180 kubenswrapper[2581]: W0910 17:36:19.627973    2581 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 10 17:36:19.633386 ip-10-0-141-180 kubenswrapper[2581]: W0910 17:36:19.627975    2581 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 10 17:36:19.633386 ip-10-0-141-180 kubenswrapper[2581]: W0910 17:36:19.627978    2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 10 17:36:19.633386 ip-10-0-141-180 kubenswrapper[2581]: W0910 17:36:19.627980    2581 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 10 17:36:19.633386 ip-10-0-141-180 kubenswrapper[2581]: W0910 17:36:19.627983    2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 10 17:36:19.633386 ip-10-0-141-180 kubenswrapper[2581]: W0910 17:36:19.627985    2581 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 10 17:36:19.633386 ip-10-0-141-180 kubenswrapper[2581]: W0910 17:36:19.627988    2581 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 10 17:36:19.633386 ip-10-0-141-180 kubenswrapper[2581]: W0910 17:36:19.627991    2581 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 10 17:36:19.633386 ip-10-0-141-180 kubenswrapper[2581]: W0910 17:36:19.627994    2581 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 10 17:36:19.633386 ip-10-0-141-180 kubenswrapper[2581]: W0910 17:36:19.627996    2581 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 10 17:36:19.633386 ip-10-0-141-180 kubenswrapper[2581]: W0910 17:36:19.627999    2581 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 10 17:36:19.633386 ip-10-0-141-180 kubenswrapper[2581]: W0910 17:36:19.628001    2581 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 10 17:36:19.633386 ip-10-0-141-180 kubenswrapper[2581]: W0910 17:36:19.628005    2581 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 10 17:36:19.633386 ip-10-0-141-180 kubenswrapper[2581]: W0910 17:36:19.628009    2581 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 10 17:36:19.633386 ip-10-0-141-180 kubenswrapper[2581]: W0910 17:36:19.628012    2581 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 10 17:36:19.633845 ip-10-0-141-180 kubenswrapper[2581]: W0910 17:36:19.628015    2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 10 17:36:19.633845 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:36:19.628020    2581 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Sep 10 17:36:19.633845 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:36:19.628119    2581 server.go:962] "Client rotation is on, will bootstrap in background"
Sep 10 17:36:19.633845 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:36:19.631016    2581 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir"
Sep 10 17:36:19.633845 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:36:19.631978    2581 server.go:1019] "Starting client certificate rotation"
Sep 10 17:36:19.633845 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:36:19.632077    2581 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet"
Sep 10 17:36:19.633845 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:36:19.632104    2581 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet"
Sep 10 17:36:19.634661 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:36:19.634645    2581 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Sep 10 17:36:19.635975 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:36:19.635958    2581 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Sep 10 17:36:19.648774 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:36:19.648758    2581 log.go:25] "Validated CRI v1 runtime API"
Sep 10 17:36:19.651513 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:36:19.651498    2581 log.go:25] "Validated CRI v1 image API"
Sep 10 17:36:19.652401 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:36:19.652381    2581 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd"
Sep 10 17:36:19.654752 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:36:19.654732    2581 fs.go:135] Filesystem UUIDs: map[5d9fc753-67eb-47fb-9258-020edfe36f0e:/dev/nvme0n1p4 7B77-95E7:/dev/nvme0n1p2 9fcf83e4-0fc0-4411-9810-88970b8196b5:/dev/nvme0n1p3]
Sep 10 17:36:19.654815 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:36:19.654751    2581 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}]
Sep 10 17:36:19.657444 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:36:19.657429    2581 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Sep 10 17:36:19.661140 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:36:19.661039    2581 manager.go:216] Machine: {Timestamp:2026-09-10 17:36:19.659498508 +0000 UTC m=+0.338022249 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3100415 MemoryCapacity:33164480512 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec27262687a35b0684ba4c3cec1a2d39 SystemUUID:ec272626-87a3-5b06-84ba-4c3cec1a2d39 BootID:94d457ad-74fe-4cba-b15f-a085513c934c Filesystems:[{Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16582238208 Type:vfs Inodes:4048398 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6632898560 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6111232 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16582242304 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:fb:1e:3a:b2:77 Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:fb:1e:3a:b2:77 Speed:0 Mtu:9001} {Name:ovs-system MacAddress:1e:50:e9:6d:f2:9a Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:33164480512 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:37486592 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None}
Sep 10 17:36:19.661140 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:36:19.661140    2581 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available.
Sep 10 17:36:19.661247 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:36:19.661229    2581 manager.go:232] Version: {KernelVersion:5.14.0-570.138.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260901-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:}
Sep 10 17:36:19.664248 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:36:19.664223    2581 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[]
Sep 10 17:36:19.664383 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:36:19.664250    2581 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-141-180.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2}
Sep 10 17:36:19.665070 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:36:19.665060    2581 topology_manager.go:138] "Creating topology manager with none policy"
Sep 10 17:36:19.665099 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:36:19.665072    2581 container_manager_linux.go:309] "Creating device plugin manager"
Sep 10 17:36:19.665099 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:36:19.665080    2581 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock"
Sep 10 17:36:19.665822 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:36:19.665802    2581 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock"
Sep 10 17:36:19.667009 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:36:19.667000    2581 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager"
Sep 10 17:36:19.669708 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:36:19.669697    2581 state_mem.go:36] "Initialized new in-memory state store"
Sep 10 17:36:19.669838 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:36:19.669829    2581 server.go:1240] "Using root directory" path="/var/lib/kubelet"
Sep 10 17:36:19.669874 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:36:19.669861    2581 kubelet.go:486] "Attempting to sync node with API server"
Sep 10 17:36:19.669901 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:36:19.669876    2581 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests"
Sep 10 17:36:19.669901 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:36:19.669893    2581 file.go:69] "Watching path" path="/etc/kubernetes/manifests"
Sep 10 17:36:19.669954 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:36:19.669911    2581 kubelet.go:398] "Adding apiserver pod source"
Sep 10 17:36:19.670024 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:36:19.669960    2581 apiserver.go:42] "Waiting for node sync before watching apiserver pods"
Sep 10 17:36:19.670977 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:36:19.670964    2581 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Sep 10 17:36:19.671044 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:36:19.670984    2581 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Sep 10 17:36:19.674446 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:36:19.674420    2581 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.12-5.rhaos4.21.gitd1ec63c.el9" apiVersion="v1"
Sep 10 17:36:19.676965 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:36:19.676950    2581 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled"
Sep 10 17:36:19.677044 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:36:19.676980    2581 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled"
Sep 10 17:36:19.677133 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:36:19.677124    2581 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume"
Sep 10 17:36:19.677163 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:36:19.677139    2581 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir"
Sep 10 17:36:19.677163 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:36:19.677145    2581 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo"
Sep 10 17:36:19.677163 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:36:19.677151    2581 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path"
Sep 10 17:36:19.677163 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:36:19.677158    2581 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs"
Sep 10 17:36:19.677163 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:36:19.677163    2581 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret"
Sep 10 17:36:19.677636 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:36:19.677627    2581 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi"
Sep 10 17:36:19.677636 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:36:19.677637    2581 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api"
Sep 10 17:36:19.677690 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:36:19.677644    2581 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc"
Sep 10 17:36:19.677690 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:36:19.677649    2581 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap"
Sep 10 17:36:19.677690 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:36:19.677657    2581 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected"
Sep 10 17:36:19.677690 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:36:19.677665    2581 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume"
Sep 10 17:36:19.678464 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:36:19.678451    2581 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi"
Sep 10 17:36:19.678464 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:36:19.678461    2581 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image"
Sep 10 17:36:19.680334 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:36:19.680316    2581 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-141-180.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Sep 10 17:36:19.680927 ip-10-0-141-180 kubenswrapper[2581]: E0910 17:36:19.680899    2581 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service"
Sep 10 17:36:19.680962 ip-10-0-141-180 kubenswrapper[2581]: E0910 17:36:19.680899    2581 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-141-180.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node"
Sep 10 17:36:19.682484 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:36:19.682472    2581 server.go:1268] "Started kubelet"
Sep 10 17:36:19.682593 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:36:19.682559    2581 server.go:180] "Starting to listen" address="0.0.0.0" port=10250
Sep 10 17:36:19.682630 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:36:19.682547    2581 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10
Sep 10 17:36:19.682665 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:36:19.682641    2581 server_v1.go:49] "podresources" method="list" useActivePods=true
Sep 10 17:36:19.683251 ip-10-0-141-180 systemd[1]: Started Kubernetes Kubelet.
Sep 10 17:36:19.684253 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:36:19.684235    2581 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock"
Sep 10 17:36:19.685292 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:36:19.685276    2581 server.go:310] "Adding debug handlers to kubelet server"
Sep 10 17:36:19.686158 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:36:19.686138    2581 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-6qsjb"
Sep 10 17:36:19.688009 ip-10-0-141-180 kubenswrapper[2581]: E0910 17:36:19.687991    2581 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache"
Sep 10 17:36:19.688098 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:36:19.688084    2581 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer"
Sep 10 17:36:19.688577 ip-10-0-141-180 kubenswrapper[2581]: E0910 17:36:19.688212    2581 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-141-180.ec2.internal\" not found"
Sep 10 17:36:19.688577 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:36:19.688220    2581 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving"
Sep 10 17:36:19.688731 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:36:19.688652    2581 volume_manager.go:311] "The desired_state_of_world populator starts"
Sep 10 17:36:19.688731 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:36:19.688665    2581 desired_state_of_world_populator.go:146] "Desired state populator starts to run"
Sep 10 17:36:19.688731 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:36:19.688667    2581 volume_manager.go:313] "Starting Kubelet Volume Manager"
Sep 10 17:36:19.688848 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:36:19.688786    2581 reconstruct.go:97] "Volume reconstruction finished"
Sep 10 17:36:19.688848 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:36:19.688795    2581 reconciler.go:29] "Reconciler: start to sync state"
Sep 10 17:36:19.689121 ip-10-0-141-180 kubenswrapper[2581]: E0910 17:36:19.688039    2581 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-141-180.ec2.internal.18d40635b3334721  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-141-180.ec2.internal,UID:ip-10-0-141-180.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-141-180.ec2.internal,},FirstTimestamp:2026-09-10 17:36:19.682445089 +0000 UTC m=+0.360968810,LastTimestamp:2026-09-10 17:36:19.682445089 +0000 UTC m=+0.360968810,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-141-180.ec2.internal,}"
Sep 10 17:36:19.690376 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:36:19.690338    2581 factory.go:55] Registering systemd factory
Sep 10 17:36:19.690376 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:36:19.690352    2581 factory.go:223] Registration of the systemd container factory successfully
Sep 10 17:36:19.690570 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:36:19.690556    2581 factory.go:153] Registering CRI-O factory
Sep 10 17:36:19.690616 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:36:19.690574    2581 factory.go:223] Registration of the crio container factory successfully
Sep 10 17:36:19.690655 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:36:19.690621    2581 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory
Sep 10 17:36:19.690655 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:36:19.690644    2581 factory.go:103] Registering Raw factory
Sep 10 17:36:19.690728 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:36:19.690658    2581 manager.go:1196] Started watching for new ooms in manager
Sep 10 17:36:19.691047 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:36:19.691037    2581 manager.go:349] Starting recovery of all containers
Sep 10 17:36:19.694069 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:36:19.694039    2581 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-6qsjb"
Sep 10 17:36:19.697862 ip-10-0-141-180 kubenswrapper[2581]: E0910 17:36:19.697837    2581 reflector.go:205] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver"
Sep 10 17:36:19.698022 ip-10-0-141-180 kubenswrapper[2581]: E0910 17:36:19.697959    2581 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-141-180.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms"
Sep 10 17:36:19.701500 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:36:19.701262    2581 manager.go:354] Recovery completed
Sep 10 17:36:19.706472 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:36:19.706457    2581 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 10 17:36:19.708233 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:36:19.708217    2581 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-180.ec2.internal" event="NodeHasSufficientMemory"
Sep 10 17:36:19.708316 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:36:19.708246    2581 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-180.ec2.internal" event="NodeHasNoDiskPressure"
Sep 10 17:36:19.708316 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:36:19.708259    2581 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-180.ec2.internal" event="NodeHasSufficientPID"
Sep 10 17:36:19.708845 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:36:19.708831    2581 cpu_manager.go:222] "Starting CPU manager" policy="none"
Sep 10 17:36:19.708845 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:36:19.708845    2581 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s"
Sep 10 17:36:19.708965 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:36:19.708862    2581 state_mem.go:36] "Initialized new in-memory state store"
Sep 10 17:36:19.708965 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:36:19.708833    2581 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 10 17:36:19.709590 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:36:19.709575    2581 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-180.ec2.internal" event="NodeHasSufficientMemory"
Sep 10 17:36:19.709651 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:36:19.709601    2581 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-180.ec2.internal" event="NodeHasNoDiskPressure"
Sep 10 17:36:19.709651 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:36:19.709611    2581 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-180.ec2.internal" event="NodeHasSufficientPID"
Sep 10 17:36:19.709952 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:36:19.709934    2581 policy_none.go:49] "None policy: Start"
Sep 10 17:36:19.709952 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:36:19.709952    2581 memory_manager.go:187] "Starting memorymanager" policy="None"
Sep 10 17:36:19.710083 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:36:19.709962    2581 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint"
Sep 10 17:36:19.711846 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:36:19.711832    2581 policy_none.go:47] "Start"
Sep 10 17:36:19.739783 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:36:19.739769    2581 manager.go:337] "Starting Device Plugin manager"
Sep 10 17:36:19.739859 ip-10-0-141-180 kubenswrapper[2581]: E0910 17:36:19.739806    2581 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint"
Sep 10 17:36:19.739859 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:36:19.739816    2581 server.go:87] "Starting device plugin registration server"
Sep 10 17:36:19.740063 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:36:19.740045    2581 eviction_manager.go:189] "Eviction manager: starting control loop"
Sep 10 17:36:19.740144 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:36:19.740062    2581 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s"
Sep 10 17:36:19.740271 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:36:19.740253    2581 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry"
Sep 10 17:36:19.740363 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:36:19.740343    2581 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts"
Sep 10 17:36:19.740363 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:36:19.740352    2581 plugin_manager.go:118] "Starting Kubelet Plugin Manager"
Sep 10 17:36:19.741042 ip-10-0-141-180 kubenswrapper[2581]: E0910 17:36:19.741020    2581 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\""
Sep 10 17:36:19.741133 ip-10-0-141-180 kubenswrapper[2581]: E0910 17:36:19.741087    2581 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-141-180.ec2.internal\" not found"
Sep 10 17:36:19.829317 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:36:19.829289    2581 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4"
Sep 10 17:36:19.830529 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:36:19.830514    2581 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6"
Sep 10 17:36:19.830619 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:36:19.830570    2581 status_manager.go:244] "Starting to sync pod status with apiserver"
Sep 10 17:36:19.830619 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:36:19.830597    2581 kubelet.go:2448] "Starting kubelet main sync loop"
Sep 10 17:36:19.830718 ip-10-0-141-180 kubenswrapper[2581]: E0910 17:36:19.830694    2581 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful"
Sep 10 17:36:19.833100 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:36:19.833078    2581 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 10 17:36:19.840813 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:36:19.840798    2581 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 10 17:36:19.841666 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:36:19.841651    2581 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-180.ec2.internal" event="NodeHasSufficientMemory"
Sep 10 17:36:19.841752 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:36:19.841687    2581 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-180.ec2.internal" event="NodeHasNoDiskPressure"
Sep 10 17:36:19.841752 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:36:19.841702    2581 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-180.ec2.internal" event="NodeHasSufficientPID"
Sep 10 17:36:19.841752 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:36:19.841729    2581 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-141-180.ec2.internal"
Sep 10 17:36:19.848037 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:36:19.848022    2581 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-141-180.ec2.internal"
Sep 10 17:36:19.848094 ip-10-0-141-180 kubenswrapper[2581]: E0910 17:36:19.848043    2581 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-141-180.ec2.internal\": node \"ip-10-0-141-180.ec2.internal\" not found"
Sep 10 17:36:19.868043 ip-10-0-141-180 kubenswrapper[2581]: E0910 17:36:19.867990    2581 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-141-180.ec2.internal\" not found"
Sep 10 17:36:19.931430 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:36:19.931385    2581 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["kube-system/kube-apiserver-proxy-ip-10-0-141-180.ec2.internal","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-180.ec2.internal"]
Sep 10 17:36:19.931572 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:36:19.931465    2581 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 10 17:36:19.932919 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:36:19.932901    2581 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-180.ec2.internal" event="NodeHasSufficientMemory"
Sep 10 17:36:19.933012 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:36:19.932930    2581 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-180.ec2.internal" event="NodeHasNoDiskPressure"
Sep 10 17:36:19.933012 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:36:19.932940    2581 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-180.ec2.internal" event="NodeHasSufficientPID"
Sep 10 17:36:19.934294 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:36:19.934282    2581 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 10 17:36:19.934416 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:36:19.934392    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-141-180.ec2.internal"
Sep 10 17:36:19.934451 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:36:19.934428    2581 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 10 17:36:19.935034 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:36:19.935020    2581 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-180.ec2.internal" event="NodeHasSufficientMemory"
Sep 10 17:36:19.935107 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:36:19.935044    2581 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-180.ec2.internal" event="NodeHasNoDiskPressure"
Sep 10 17:36:19.935107 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:36:19.935053    2581 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-180.ec2.internal" event="NodeHasSufficientPID"
Sep 10 17:36:19.935792 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:36:19.935778    2581 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-180.ec2.internal" event="NodeHasSufficientMemory"
Sep 10 17:36:19.935869 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:36:19.935803    2581 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-180.ec2.internal" event="NodeHasNoDiskPressure"
Sep 10 17:36:19.935869 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:36:19.935817    2581 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-180.ec2.internal" event="NodeHasSufficientPID"
Sep 10 17:36:19.936907 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:36:19.936892    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-180.ec2.internal"
Sep 10 17:36:19.937038 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:36:19.936922    2581 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 10 17:36:19.937691 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:36:19.937677    2581 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-180.ec2.internal" event="NodeHasSufficientMemory"
Sep 10 17:36:19.937758 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:36:19.937708    2581 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-180.ec2.internal" event="NodeHasNoDiskPressure"
Sep 10 17:36:19.937758 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:36:19.937723    2581 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-180.ec2.internal" event="NodeHasSufficientPID"
Sep 10 17:36:19.965782 ip-10-0-141-180 kubenswrapper[2581]: E0910 17:36:19.965763    2581 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-141-180.ec2.internal\" not found" node="ip-10-0-141-180.ec2.internal"
Sep 10 17:36:19.968113 ip-10-0-141-180 kubenswrapper[2581]: E0910 17:36:19.968099    2581 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-141-180.ec2.internal\" not found"
Sep 10 17:36:19.970021 ip-10-0-141-180 kubenswrapper[2581]: E0910 17:36:19.970008    2581 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-141-180.ec2.internal\" not found" node="ip-10-0-141-180.ec2.internal"
Sep 10 17:36:19.990728 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:36:19.990707    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/a4eaab8f22f33ba54d00328e9bf6c226-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-141-180.ec2.internal\" (UID: \"a4eaab8f22f33ba54d00328e9bf6c226\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-180.ec2.internal"
Sep 10 17:36:19.990728 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:36:19.990729    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/a4eaab8f22f33ba54d00328e9bf6c226-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-141-180.ec2.internal\" (UID: \"a4eaab8f22f33ba54d00328e9bf6c226\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-180.ec2.internal"
Sep 10 17:36:19.990862 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:36:19.990746    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/414c1afd223c55080845d6fd4dfe2409-config\") pod \"kube-apiserver-proxy-ip-10-0-141-180.ec2.internal\" (UID: \"414c1afd223c55080845d6fd4dfe2409\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-141-180.ec2.internal"
Sep 10 17:36:20.068833 ip-10-0-141-180 kubenswrapper[2581]: E0910 17:36:20.068805    2581 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-141-180.ec2.internal\" not found"
Sep 10 17:36:20.091266 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:36:20.091235    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/414c1afd223c55080845d6fd4dfe2409-config\") pod \"kube-apiserver-proxy-ip-10-0-141-180.ec2.internal\" (UID: \"414c1afd223c55080845d6fd4dfe2409\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-141-180.ec2.internal"
Sep 10 17:36:20.091332 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:36:20.091183    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/414c1afd223c55080845d6fd4dfe2409-config\") pod \"kube-apiserver-proxy-ip-10-0-141-180.ec2.internal\" (UID: \"414c1afd223c55080845d6fd4dfe2409\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-141-180.ec2.internal"
Sep 10 17:36:20.091332 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:36:20.091303    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/a4eaab8f22f33ba54d00328e9bf6c226-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-141-180.ec2.internal\" (UID: \"a4eaab8f22f33ba54d00328e9bf6c226\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-180.ec2.internal"
Sep 10 17:36:20.091332 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:36:20.091320    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/a4eaab8f22f33ba54d00328e9bf6c226-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-141-180.ec2.internal\" (UID: \"a4eaab8f22f33ba54d00328e9bf6c226\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-180.ec2.internal"
Sep 10 17:36:20.091431 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:36:20.091337    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/a4eaab8f22f33ba54d00328e9bf6c226-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-141-180.ec2.internal\" (UID: \"a4eaab8f22f33ba54d00328e9bf6c226\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-180.ec2.internal"
Sep 10 17:36:20.091431 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:36:20.091358    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/a4eaab8f22f33ba54d00328e9bf6c226-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-141-180.ec2.internal\" (UID: \"a4eaab8f22f33ba54d00328e9bf6c226\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-180.ec2.internal"
Sep 10 17:36:20.169509 ip-10-0-141-180 kubenswrapper[2581]: E0910 17:36:20.169447    2581 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-141-180.ec2.internal\" not found"
Sep 10 17:36:20.268043 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:36:20.268008    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-141-180.ec2.internal"
Sep 10 17:36:20.270112 ip-10-0-141-180 kubenswrapper[2581]: E0910 17:36:20.270099    2581 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-141-180.ec2.internal\" not found"
Sep 10 17:36:20.272794 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:36:20.272475    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-180.ec2.internal"
Sep 10 17:36:21.395469 ip-10-0-141-180 kubenswrapper[2581]: E0910 17:36:21.395443    2581 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-141-180.ec2.internal\" not found"
Sep 10 17:36:21.495988 ip-10-0-141-180 kubenswrapper[2581]: E0910 17:36:21.495896    2581 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-141-180.ec2.internal\" not found"
Sep 10 17:36:21.596482 ip-10-0-141-180 kubenswrapper[2581]: E0910 17:36:21.596456    2581 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-141-180.ec2.internal\" not found"
Sep 10 17:36:21.655973 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:36:21.655947    2581 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials"
Sep 10 17:36:21.656603 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:36:21.656091    2581 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received"
Sep 10 17:36:21.697391 ip-10-0-141-180 kubenswrapper[2581]: E0910 17:36:21.697369    2581 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-141-180.ec2.internal\" not found"
Sep 10 17:36:21.721436 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:36:21.721413    2581 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-09-09 17:31:20 +0000 UTC" deadline="2028-05-24 06:20:19.336683864 +0000 UTC"
Sep 10 17:36:21.721530 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:36:21.721436    2581 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="14916h43m57.615250743s"
Sep 10 17:36:21.798109 ip-10-0-141-180 kubenswrapper[2581]: E0910 17:36:21.798039    2581 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-141-180.ec2.internal\" not found"
Sep 10 17:36:21.813949 ip-10-0-141-180 kubenswrapper[2581]: W0910 17:36:21.813915    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod414c1afd223c55080845d6fd4dfe2409.slice/crio-85bfcfd1eb306880fc334addc5b98fe9a260df860bb47b65bba148288b65d1bf WatchSource:0}: Error finding container 85bfcfd1eb306880fc334addc5b98fe9a260df860bb47b65bba148288b65d1bf: Status 404 returned error can't find the container with id 85bfcfd1eb306880fc334addc5b98fe9a260df860bb47b65bba148288b65d1bf
Sep 10 17:36:21.817909 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:36:21.817893    2581 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 10 17:36:21.858039 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:36:21.858000    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-141-180.ec2.internal" event={"ID":"414c1afd223c55080845d6fd4dfe2409","Type":"ContainerStarted","Data":"85bfcfd1eb306880fc334addc5b98fe9a260df860bb47b65bba148288b65d1bf"}
Sep 10 17:36:21.899157 ip-10-0-141-180 kubenswrapper[2581]: E0910 17:36:21.899136    2581 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-141-180.ec2.internal\" not found"
Sep 10 17:36:21.967345 ip-10-0-141-180 kubenswrapper[2581]: W0910 17:36:21.967323    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda4eaab8f22f33ba54d00328e9bf6c226.slice/crio-5afff424eda1f744bfaeceb24636455a94febef98454492a73fdb59814e433af WatchSource:0}: Error finding container 5afff424eda1f744bfaeceb24636455a94febef98454492a73fdb59814e433af: Status 404 returned error can't find the container with id 5afff424eda1f744bfaeceb24636455a94febef98454492a73fdb59814e433af
Sep 10 17:36:21.999798 ip-10-0-141-180 kubenswrapper[2581]: E0910 17:36:21.999776    2581 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-141-180.ec2.internal\" not found"
Sep 10 17:36:22.100402 ip-10-0-141-180 kubenswrapper[2581]: E0910 17:36:22.100336    2581 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-141-180.ec2.internal\" not found"
Sep 10 17:36:22.140765 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:36:22.140739    2581 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 10 17:36:22.161404 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:36:22.161385    2581 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 10 17:36:22.198181 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:36:22.198157    2581 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 10 17:36:22.213201 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:36:22.213179    2581 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-141-180.ec2.internal"
Sep 10 17:36:22.223383 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:36:22.223366    2581 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Sep 10 17:36:22.224521 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:36:22.224500    2581 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-180.ec2.internal"
Sep 10 17:36:22.234967 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:36:22.234951    2581 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Sep 10 17:36:22.696231 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:36:22.696196    2581 apiserver.go:52] "Watching apiserver"
Sep 10 17:36:22.706821 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:36:22.706796    2581 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66"
Sep 10 17:36:22.707246 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:36:22.707221    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-cluster-node-tuning-operator/tuned-s8k6d","openshift-image-registry/node-ca-h6rpt","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-180.ec2.internal","openshift-multus/multus-additional-cni-plugins-bqj9g","openshift-multus/network-metrics-daemon-b88bx","openshift-network-operator/iptables-alerter-tlnqf","kube-system/global-pull-secret-syncer-2t2rd","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-w9ldt","openshift-dns/node-resolver-jkplw","openshift-multus/multus-z5bpf","openshift-network-diagnostics/network-check-target-7ptnb","openshift-ovn-kubernetes/ovnkube-node-7nn7k","kube-system/konnectivity-agent-7p25j","kube-system/kube-apiserver-proxy-ip-10-0-141-180.ec2.internal"]
Sep 10 17:36:22.708881 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:36:22.708815    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-7ptnb"
Sep 10 17:36:22.708991 ip-10-0-141-180 kubenswrapper[2581]: E0910 17:36:22.708923    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-7ptnb" podUID="766766d2-fa56-49c2-9928-b50cbbf9173a"
Sep 10 17:36:22.709995 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:36:22.709972    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-h6rpt"
Sep 10 17:36:22.711109 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:36:22.711089    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-7p25j"
Sep 10 17:36:22.712521 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:36:22.712259    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-bqj9g"
Sep 10 17:36:22.713739 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:36:22.713326    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\""
Sep 10 17:36:22.713739 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:36:22.713449    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\""
Sep 10 17:36:22.713739 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:36:22.713551    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-r6cnt\""
Sep 10 17:36:22.713739 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:36:22.713685    2581 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving"
Sep 10 17:36:22.713968 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:36:22.713817    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\""
Sep 10 17:36:22.713968 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:36:22.713856    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-glkpd\""
Sep 10 17:36:22.714065 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:36:22.714036    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\""
Sep 10 17:36:22.714114 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:36:22.714101    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-b88bx"
Sep 10 17:36:22.714202 ip-10-0-141-180 kubenswrapper[2581]: E0910 17:36:22.714159    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-b88bx" podUID="88d343e2-9756-438b-a19c-973b53948e68"
Sep 10 17:36:22.714202 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:36:22.714185    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\""
Sep 10 17:36:22.715005 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:36:22.714985    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\""
Sep 10 17:36:22.715096 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:36:22.714988    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\""
Sep 10 17:36:22.715096 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:36:22.715053    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\""
Sep 10 17:36:22.715338 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:36:22.715253    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-vhqr9\""
Sep 10 17:36:22.715338 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:36:22.715270    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\""
Sep 10 17:36:22.715338 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:36:22.715335    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\""
Sep 10 17:36:22.715552 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:36:22.715534    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-tlnqf"
Sep 10 17:36:22.716743 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:36:22.716723    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-s8k6d"
Sep 10 17:36:22.717759 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:36:22.717740    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-m9rq7\""
Sep 10 17:36:22.717882 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:36:22.717864    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\""
Sep 10 17:36:22.717995 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:36:22.717976    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-w9ldt"
Sep 10 17:36:22.718099 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:36:22.718018    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\""
Sep 10 17:36:22.718326 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:36:22.718225    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\""
Sep 10 17:36:22.719261 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:36:22.719239    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\""
Sep 10 17:36:22.719460 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:36:22.719442    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\""
Sep 10 17:36:22.720801 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:36:22.720468    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\""
Sep 10 17:36:22.720801 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:36:22.720229    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\""
Sep 10 17:36:22.720801 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:36:22.720287    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\""
Sep 10 17:36:22.720801 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:36:22.719493    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-fhwhr\""
Sep 10 17:36:22.721038 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:36:22.720903    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-mcxr9\""
Sep 10 17:36:22.721038 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:36:22.720965    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-7nn7k"
Sep 10 17:36:22.721265 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:36:22.721139    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-jkplw"
Sep 10 17:36:22.722487 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:36:22.722467    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-2t2rd"
Sep 10 17:36:22.722579 ip-10-0-141-180 kubenswrapper[2581]: E0910 17:36:22.722540    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-2t2rd" podUID="dee7aebb-2ba6-4bfe-8fb4-371bcc4e56b1"
Sep 10 17:36:22.723163 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:36:22.723145    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\""
Sep 10 17:36:22.723163 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:36:22.723161    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\""
Sep 10 17:36:22.723509 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:36:22.723488    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\""
Sep 10 17:36:22.723579 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:36:22.723537    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\""
Sep 10 17:36:22.723832 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:36:22.723777    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\""
Sep 10 17:36:22.723832 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:36:22.723811    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-2t96x\""
Sep 10 17:36:22.723960 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:36:22.723897    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-z5bpf"
Sep 10 17:36:22.723997 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:36:22.723969    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-w28mp\" (UniqueName: \"kubernetes.io/projected/766766d2-fa56-49c2-9928-b50cbbf9173a-kube-api-access-w28mp\") pod \"network-check-target-7ptnb\" (UID: \"766766d2-fa56-49c2-9928-b50cbbf9173a\") " pod="openshift-network-diagnostics/network-check-target-7ptnb"
Sep 10 17:36:22.724055 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:36:22.724003    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/22cfca39-1a39-4266-9ee7-25dfc8723196-host\") pod \"node-ca-h6rpt\" (UID: \"22cfca39-1a39-4266-9ee7-25dfc8723196\") " pod="openshift-image-registry/node-ca-h6rpt"
Sep 10 17:36:22.724055 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:36:22.724029    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/22cfca39-1a39-4266-9ee7-25dfc8723196-serviceca\") pod \"node-ca-h6rpt\" (UID: \"22cfca39-1a39-4266-9ee7-25dfc8723196\") " pod="openshift-image-registry/node-ca-h6rpt"
Sep 10 17:36:22.724055 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:36:22.724051    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/459a65ca-0e07-47e2-aba9-6af10bb5387f-konnectivity-ca\") pod \"konnectivity-agent-7p25j\" (UID: \"459a65ca-0e07-47e2-aba9-6af10bb5387f\") " pod="kube-system/konnectivity-agent-7p25j"
Sep 10 17:36:22.724199 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:36:22.724078    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/099b7fa9-6908-43e3-b444-73b7f327290d-cni-binary-copy\") pod \"multus-additional-cni-plugins-bqj9g\" (UID: \"099b7fa9-6908-43e3-b444-73b7f327290d\") " pod="openshift-multus/multus-additional-cni-plugins-bqj9g"
Sep 10 17:36:22.724199 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:36:22.724101    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/099b7fa9-6908-43e3-b444-73b7f327290d-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-bqj9g\" (UID: \"099b7fa9-6908-43e3-b444-73b7f327290d\") " pod="openshift-multus/multus-additional-cni-plugins-bqj9g"
Sep 10 17:36:22.724199 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:36:22.724160    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lcsqq\" (UniqueName: \"kubernetes.io/projected/22cfca39-1a39-4266-9ee7-25dfc8723196-kube-api-access-lcsqq\") pod \"node-ca-h6rpt\" (UID: \"22cfca39-1a39-4266-9ee7-25dfc8723196\") " pod="openshift-image-registry/node-ca-h6rpt"
Sep 10 17:36:22.724346 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:36:22.724201    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/099b7fa9-6908-43e3-b444-73b7f327290d-system-cni-dir\") pod \"multus-additional-cni-plugins-bqj9g\" (UID: \"099b7fa9-6908-43e3-b444-73b7f327290d\") " pod="openshift-multus/multus-additional-cni-plugins-bqj9g"
Sep 10 17:36:22.724346 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:36:22.724225    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/099b7fa9-6908-43e3-b444-73b7f327290d-cnibin\") pod \"multus-additional-cni-plugins-bqj9g\" (UID: \"099b7fa9-6908-43e3-b444-73b7f327290d\") " pod="openshift-multus/multus-additional-cni-plugins-bqj9g"
Sep 10 17:36:22.724346 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:36:22.724255    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jph88\" (UniqueName: \"kubernetes.io/projected/88d343e2-9756-438b-a19c-973b53948e68-kube-api-access-jph88\") pod \"network-metrics-daemon-b88bx\" (UID: \"88d343e2-9756-438b-a19c-973b53948e68\") " pod="openshift-multus/network-metrics-daemon-b88bx"
Sep 10 17:36:22.724346 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:36:22.724281    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/099b7fa9-6908-43e3-b444-73b7f327290d-tuning-conf-dir\") pod \"multus-additional-cni-plugins-bqj9g\" (UID: \"099b7fa9-6908-43e3-b444-73b7f327290d\") " pod="openshift-multus/multus-additional-cni-plugins-bqj9g"
Sep 10 17:36:22.724346 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:36:22.724304    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/099b7fa9-6908-43e3-b444-73b7f327290d-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-bqj9g\" (UID: \"099b7fa9-6908-43e3-b444-73b7f327290d\") " pod="openshift-multus/multus-additional-cni-plugins-bqj9g"
Sep 10 17:36:22.724346 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:36:22.724334    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hlrdr\" (UniqueName: \"kubernetes.io/projected/099b7fa9-6908-43e3-b444-73b7f327290d-kube-api-access-hlrdr\") pod \"multus-additional-cni-plugins-bqj9g\" (UID: \"099b7fa9-6908-43e3-b444-73b7f327290d\") " pod="openshift-multus/multus-additional-cni-plugins-bqj9g"
Sep 10 17:36:22.724621 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:36:22.724359    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/459a65ca-0e07-47e2-aba9-6af10bb5387f-agent-certs\") pod \"konnectivity-agent-7p25j\" (UID: \"459a65ca-0e07-47e2-aba9-6af10bb5387f\") " pod="kube-system/konnectivity-agent-7p25j"
Sep 10 17:36:22.724621 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:36:22.724399    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\""
Sep 10 17:36:22.724621 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:36:22.724411    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/88d343e2-9756-438b-a19c-973b53948e68-metrics-certs\") pod \"network-metrics-daemon-b88bx\" (UID: \"88d343e2-9756-438b-a19c-973b53948e68\") " pod="openshift-multus/network-metrics-daemon-b88bx"
Sep 10 17:36:22.724621 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:36:22.724436    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\""
Sep 10 17:36:22.724621 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:36:22.724521    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-j2qvw\""
Sep 10 17:36:22.724621 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:36:22.724555    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\""
Sep 10 17:36:22.726229 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:36:22.726211    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\""
Sep 10 17:36:22.726855 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:36:22.726837    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-fh95l\""
Sep 10 17:36:22.727064 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:36:22.727042    2581 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Sep 10 17:36:22.746178 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:36:22.746157    2581 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-69wmp"
Sep 10 17:36:22.754903 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:36:22.754885    2581 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-69wmp"
Sep 10 17:36:22.814226 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:36:22.814202    2581 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world"
Sep 10 17:36:22.824732 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:36:22.824671    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/099b7fa9-6908-43e3-b444-73b7f327290d-system-cni-dir\") pod \"multus-additional-cni-plugins-bqj9g\" (UID: \"099b7fa9-6908-43e3-b444-73b7f327290d\") " pod="openshift-multus/multus-additional-cni-plugins-bqj9g"
Sep 10 17:36:22.824732 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:36:22.824709    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/81e433e1-460f-4d44-88dc-4ab3666e140e-systemd-units\") pod \"ovnkube-node-7nn7k\" (UID: \"81e433e1-460f-4d44-88dc-4ab3666e140e\") " pod="openshift-ovn-kubernetes/ovnkube-node-7nn7k"
Sep 10 17:36:22.824732 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:36:22.824735    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/81e433e1-460f-4d44-88dc-4ab3666e140e-var-lib-openvswitch\") pod \"ovnkube-node-7nn7k\" (UID: \"81e433e1-460f-4d44-88dc-4ab3666e140e\") " pod="openshift-ovn-kubernetes/ovnkube-node-7nn7k"
Sep 10 17:36:22.824972 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:36:22.824758    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/b4e9eaae-6a8f-459d-9953-f9b6a529bef4-device-dir\") pod \"aws-ebs-csi-driver-node-w9ldt\" (UID: \"b4e9eaae-6a8f-459d-9953-f9b6a529bef4\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-w9ldt"
Sep 10 17:36:22.824972 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:36:22.824781    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/b4e9eaae-6a8f-459d-9953-f9b6a529bef4-sys-fs\") pod \"aws-ebs-csi-driver-node-w9ldt\" (UID: \"b4e9eaae-6a8f-459d-9953-f9b6a529bef4\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-w9ldt"
Sep 10 17:36:22.824972 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:36:22.824803    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/099b7fa9-6908-43e3-b444-73b7f327290d-system-cni-dir\") pod \"multus-additional-cni-plugins-bqj9g\" (UID: \"099b7fa9-6908-43e3-b444-73b7f327290d\") " pod="openshift-multus/multus-additional-cni-plugins-bqj9g"
Sep 10 17:36:22.824972 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:36:22.824820    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/b1aa3439-0cd7-4496-b1e8-7b535c305002-multus-cni-dir\") pod \"multus-z5bpf\" (UID: \"b1aa3439-0cd7-4496-b1e8-7b535c305002\") " pod="openshift-multus/multus-z5bpf"
Sep 10 17:36:22.824972 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:36:22.824850    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/b1aa3439-0cd7-4496-b1e8-7b535c305002-host-run-k8s-cni-cncf-io\") pod \"multus-z5bpf\" (UID: \"b1aa3439-0cd7-4496-b1e8-7b535c305002\") " pod="openshift-multus/multus-z5bpf"
Sep 10 17:36:22.824972 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:36:22.824896    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hlrdr\" (UniqueName: \"kubernetes.io/projected/099b7fa9-6908-43e3-b444-73b7f327290d-kube-api-access-hlrdr\") pod \"multus-additional-cni-plugins-bqj9g\" (UID: \"099b7fa9-6908-43e3-b444-73b7f327290d\") " pod="openshift-multus/multus-additional-cni-plugins-bqj9g"
Sep 10 17:36:22.824972 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:36:22.824924    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/356f6437-710e-4991-b8db-e96a35f1152d-etc-sysctl-d\") pod \"tuned-s8k6d\" (UID: \"356f6437-710e-4991-b8db-e96a35f1152d\") " pod="openshift-cluster-node-tuning-operator/tuned-s8k6d"
Sep 10 17:36:22.824972 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:36:22.824948    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/356f6437-710e-4991-b8db-e96a35f1152d-etc-systemd\") pod \"tuned-s8k6d\" (UID: \"356f6437-710e-4991-b8db-e96a35f1152d\") " pod="openshift-cluster-node-tuning-operator/tuned-s8k6d"
Sep 10 17:36:22.824972 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:36:22.824973    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/b1aa3439-0cd7-4496-b1e8-7b535c305002-system-cni-dir\") pod \"multus-z5bpf\" (UID: \"b1aa3439-0cd7-4496-b1e8-7b535c305002\") " pod="openshift-multus/multus-z5bpf"
Sep 10 17:36:22.825373 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:36:22.824997    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pl8ps\" (UniqueName: \"kubernetes.io/projected/b1aa3439-0cd7-4496-b1e8-7b535c305002-kube-api-access-pl8ps\") pod \"multus-z5bpf\" (UID: \"b1aa3439-0cd7-4496-b1e8-7b535c305002\") " pod="openshift-multus/multus-z5bpf"
Sep 10 17:36:22.825373 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:36:22.825022    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/22cfca39-1a39-4266-9ee7-25dfc8723196-host\") pod \"node-ca-h6rpt\" (UID: \"22cfca39-1a39-4266-9ee7-25dfc8723196\") " pod="openshift-image-registry/node-ca-h6rpt"
Sep 10 17:36:22.825373 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:36:22.825046    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/459a65ca-0e07-47e2-aba9-6af10bb5387f-konnectivity-ca\") pod \"konnectivity-agent-7p25j\" (UID: \"459a65ca-0e07-47e2-aba9-6af10bb5387f\") " pod="kube-system/konnectivity-agent-7p25j"
Sep 10 17:36:22.825373 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:36:22.825084    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/22cfca39-1a39-4266-9ee7-25dfc8723196-host\") pod \"node-ca-h6rpt\" (UID: \"22cfca39-1a39-4266-9ee7-25dfc8723196\") " pod="openshift-image-registry/node-ca-h6rpt"
Sep 10 17:36:22.825373 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:36:22.825116    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/dee7aebb-2ba6-4bfe-8fb4-371bcc4e56b1-original-pull-secret\") pod \"global-pull-secret-syncer-2t2rd\" (UID: \"dee7aebb-2ba6-4bfe-8fb4-371bcc4e56b1\") " pod="kube-system/global-pull-secret-syncer-2t2rd"
Sep 10 17:36:22.825373 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:36:22.825147    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/b4e9eaae-6a8f-459d-9953-f9b6a529bef4-etc-selinux\") pod \"aws-ebs-csi-driver-node-w9ldt\" (UID: \"b4e9eaae-6a8f-459d-9953-f9b6a529bef4\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-w9ldt"
Sep 10 17:36:22.825373 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:36:22.825175    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/b1aa3439-0cd7-4496-b1e8-7b535c305002-multus-conf-dir\") pod \"multus-z5bpf\" (UID: \"b1aa3439-0cd7-4496-b1e8-7b535c305002\") " pod="openshift-multus/multus-z5bpf"
Sep 10 17:36:22.825373 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:36:22.825199    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/81e433e1-460f-4d44-88dc-4ab3666e140e-log-socket\") pod \"ovnkube-node-7nn7k\" (UID: \"81e433e1-460f-4d44-88dc-4ab3666e140e\") " pod="openshift-ovn-kubernetes/ovnkube-node-7nn7k"
Sep 10 17:36:22.825373 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:36:22.825225    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jph88\" (UniqueName: \"kubernetes.io/projected/88d343e2-9756-438b-a19c-973b53948e68-kube-api-access-jph88\") pod \"network-metrics-daemon-b88bx\" (UID: \"88d343e2-9756-438b-a19c-973b53948e68\") " pod="openshift-multus/network-metrics-daemon-b88bx"
Sep 10 17:36:22.825373 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:36:22.825247    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/91b2593e-0a42-4735-bbd6-616678613892-host-slash\") pod \"iptables-alerter-tlnqf\" (UID: \"91b2593e-0a42-4735-bbd6-616678613892\") " pod="openshift-network-operator/iptables-alerter-tlnqf"
Sep 10 17:36:22.825373 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:36:22.825271    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/81e433e1-460f-4d44-88dc-4ab3666e140e-node-log\") pod \"ovnkube-node-7nn7k\" (UID: \"81e433e1-460f-4d44-88dc-4ab3666e140e\") " pod="openshift-ovn-kubernetes/ovnkube-node-7nn7k"
Sep 10 17:36:22.825373 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:36:22.825294    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/81e433e1-460f-4d44-88dc-4ab3666e140e-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-7nn7k\" (UID: \"81e433e1-460f-4d44-88dc-4ab3666e140e\") " pod="openshift-ovn-kubernetes/ovnkube-node-7nn7k"
Sep 10 17:36:22.825373 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:36:22.825359    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/b4e9eaae-6a8f-459d-9953-f9b6a529bef4-registration-dir\") pod \"aws-ebs-csi-driver-node-w9ldt\" (UID: \"b4e9eaae-6a8f-459d-9953-f9b6a529bef4\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-w9ldt"
Sep 10 17:36:22.826029 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:36:22.825394    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/099b7fa9-6908-43e3-b444-73b7f327290d-tuning-conf-dir\") pod \"multus-additional-cni-plugins-bqj9g\" (UID: \"099b7fa9-6908-43e3-b444-73b7f327290d\") " pod="openshift-multus/multus-additional-cni-plugins-bqj9g"
Sep 10 17:36:22.826029 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:36:22.825428    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/459a65ca-0e07-47e2-aba9-6af10bb5387f-agent-certs\") pod \"konnectivity-agent-7p25j\" (UID: \"459a65ca-0e07-47e2-aba9-6af10bb5387f\") " pod="kube-system/konnectivity-agent-7p25j"
Sep 10 17:36:22.826029 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:36:22.825465    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/356f6437-710e-4991-b8db-e96a35f1152d-etc-modprobe-d\") pod \"tuned-s8k6d\" (UID: \"356f6437-710e-4991-b8db-e96a35f1152d\") " pod="openshift-cluster-node-tuning-operator/tuned-s8k6d"
Sep 10 17:36:22.826029 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:36:22.825500    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/81e433e1-460f-4d44-88dc-4ab3666e140e-run-systemd\") pod \"ovnkube-node-7nn7k\" (UID: \"81e433e1-460f-4d44-88dc-4ab3666e140e\") " pod="openshift-ovn-kubernetes/ovnkube-node-7nn7k"
Sep 10 17:36:22.826029 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:36:22.825524    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/81e433e1-460f-4d44-88dc-4ab3666e140e-run-openvswitch\") pod \"ovnkube-node-7nn7k\" (UID: \"81e433e1-460f-4d44-88dc-4ab3666e140e\") " pod="openshift-ovn-kubernetes/ovnkube-node-7nn7k"
Sep 10 17:36:22.826029 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:36:22.825552    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/b1aa3439-0cd7-4496-b1e8-7b535c305002-cni-binary-copy\") pod \"multus-z5bpf\" (UID: \"b1aa3439-0cd7-4496-b1e8-7b535c305002\") " pod="openshift-multus/multus-z5bpf"
Sep 10 17:36:22.826029 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:36:22.825575    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/b1aa3439-0cd7-4496-b1e8-7b535c305002-host-run-multus-certs\") pod \"multus-z5bpf\" (UID: \"b1aa3439-0cd7-4496-b1e8-7b535c305002\") " pod="openshift-multus/multus-z5bpf"
Sep 10 17:36:22.826029 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:36:22.825602    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/22cfca39-1a39-4266-9ee7-25dfc8723196-serviceca\") pod \"node-ca-h6rpt\" (UID: \"22cfca39-1a39-4266-9ee7-25dfc8723196\") " pod="openshift-image-registry/node-ca-h6rpt"
Sep 10 17:36:22.826029 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:36:22.825626    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/81e433e1-460f-4d44-88dc-4ab3666e140e-host-cni-netd\") pod \"ovnkube-node-7nn7k\" (UID: \"81e433e1-460f-4d44-88dc-4ab3666e140e\") " pod="openshift-ovn-kubernetes/ovnkube-node-7nn7k"
Sep 10 17:36:22.826029 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:36:22.825655    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/099b7fa9-6908-43e3-b444-73b7f327290d-tuning-conf-dir\") pod \"multus-additional-cni-plugins-bqj9g\" (UID: \"099b7fa9-6908-43e3-b444-73b7f327290d\") " pod="openshift-multus/multus-additional-cni-plugins-bqj9g"
Sep 10 17:36:22.826029 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:36:22.825688    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/b1aa3439-0cd7-4496-b1e8-7b535c305002-hostroot\") pod \"multus-z5bpf\" (UID: \"b1aa3439-0cd7-4496-b1e8-7b535c305002\") " pod="openshift-multus/multus-z5bpf"
Sep 10 17:36:22.826029 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:36:22.825718    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/356f6437-710e-4991-b8db-e96a35f1152d-etc-tuned\") pod \"tuned-s8k6d\" (UID: \"356f6437-710e-4991-b8db-e96a35f1152d\") " pod="openshift-cluster-node-tuning-operator/tuned-s8k6d"
Sep 10 17:36:22.826029 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:36:22.825742    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/81e433e1-460f-4d44-88dc-4ab3666e140e-etc-openvswitch\") pod \"ovnkube-node-7nn7k\" (UID: \"81e433e1-460f-4d44-88dc-4ab3666e140e\") " pod="openshift-ovn-kubernetes/ovnkube-node-7nn7k"
Sep 10 17:36:22.826029 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:36:22.825767    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/81e433e1-460f-4d44-88dc-4ab3666e140e-host-run-ovn-kubernetes\") pod \"ovnkube-node-7nn7k\" (UID: \"81e433e1-460f-4d44-88dc-4ab3666e140e\") " pod="openshift-ovn-kubernetes/ovnkube-node-7nn7k"
Sep 10 17:36:22.826029 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:36:22.825794    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jrbvs\" (UniqueName: \"kubernetes.io/projected/b4e9eaae-6a8f-459d-9953-f9b6a529bef4-kube-api-access-jrbvs\") pod \"aws-ebs-csi-driver-node-w9ldt\" (UID: \"b4e9eaae-6a8f-459d-9953-f9b6a529bef4\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-w9ldt"
Sep 10 17:36:22.826029 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:36:22.825808    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/459a65ca-0e07-47e2-aba9-6af10bb5387f-konnectivity-ca\") pod \"konnectivity-agent-7p25j\" (UID: \"459a65ca-0e07-47e2-aba9-6af10bb5387f\") " pod="kube-system/konnectivity-agent-7p25j"
Sep 10 17:36:22.826029 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:36:22.825817    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/b1aa3439-0cd7-4496-b1e8-7b535c305002-multus-daemon-config\") pod \"multus-z5bpf\" (UID: \"b1aa3439-0cd7-4496-b1e8-7b535c305002\") " pod="openshift-multus/multus-z5bpf"
Sep 10 17:36:22.826835 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:36:22.825858    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/91b2593e-0a42-4735-bbd6-616678613892-iptables-alerter-script\") pod \"iptables-alerter-tlnqf\" (UID: \"91b2593e-0a42-4735-bbd6-616678613892\") " pod="openshift-network-operator/iptables-alerter-tlnqf"
Sep 10 17:36:22.826835 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:36:22.825883    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/356f6437-710e-4991-b8db-e96a35f1152d-lib-modules\") pod \"tuned-s8k6d\" (UID: \"356f6437-710e-4991-b8db-e96a35f1152d\") " pod="openshift-cluster-node-tuning-operator/tuned-s8k6d"
Sep 10 17:36:22.826835 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:36:22.825900    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/356f6437-710e-4991-b8db-e96a35f1152d-host\") pod \"tuned-s8k6d\" (UID: \"356f6437-710e-4991-b8db-e96a35f1152d\") " pod="openshift-cluster-node-tuning-operator/tuned-s8k6d"
Sep 10 17:36:22.826835 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:36:22.825922    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/099b7fa9-6908-43e3-b444-73b7f327290d-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-bqj9g\" (UID: \"099b7fa9-6908-43e3-b444-73b7f327290d\") " pod="openshift-multus/multus-additional-cni-plugins-bqj9g"
Sep 10 17:36:22.826835 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:36:22.825945    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/84e91010-05f8-4b38-801c-8e462cd4e38d-tmp-dir\") pod \"node-resolver-jkplw\" (UID: \"84e91010-05f8-4b38-801c-8e462cd4e38d\") " pod="openshift-dns/node-resolver-jkplw"
Sep 10 17:36:22.826835 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:36:22.825989    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/81e433e1-460f-4d44-88dc-4ab3666e140e-host-cni-bin\") pod \"ovnkube-node-7nn7k\" (UID: \"81e433e1-460f-4d44-88dc-4ab3666e140e\") " pod="openshift-ovn-kubernetes/ovnkube-node-7nn7k"
Sep 10 17:36:22.826835 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:36:22.825993    2581 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory"
Sep 10 17:36:22.826835 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:36:22.826027    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/81e433e1-460f-4d44-88dc-4ab3666e140e-ovnkube-config\") pod \"ovnkube-node-7nn7k\" (UID: \"81e433e1-460f-4d44-88dc-4ab3666e140e\") " pod="openshift-ovn-kubernetes/ovnkube-node-7nn7k"
Sep 10 17:36:22.826835 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:36:22.826029    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/22cfca39-1a39-4266-9ee7-25dfc8723196-serviceca\") pod \"node-ca-h6rpt\" (UID: \"22cfca39-1a39-4266-9ee7-25dfc8723196\") " pod="openshift-image-registry/node-ca-h6rpt"
Sep 10 17:36:22.826835 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:36:22.826055    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/81e433e1-460f-4d44-88dc-4ab3666e140e-env-overrides\") pod \"ovnkube-node-7nn7k\" (UID: \"81e433e1-460f-4d44-88dc-4ab3666e140e\") " pod="openshift-ovn-kubernetes/ovnkube-node-7nn7k"
Sep 10 17:36:22.826835 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:36:22.826071    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/81e433e1-460f-4d44-88dc-4ab3666e140e-ovn-node-metrics-cert\") pod \"ovnkube-node-7nn7k\" (UID: \"81e433e1-460f-4d44-88dc-4ab3666e140e\") " pod="openshift-ovn-kubernetes/ovnkube-node-7nn7k"
Sep 10 17:36:22.826835 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:36:22.826093    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/b4e9eaae-6a8f-459d-9953-f9b6a529bef4-kubelet-dir\") pod \"aws-ebs-csi-driver-node-w9ldt\" (UID: \"b4e9eaae-6a8f-459d-9953-f9b6a529bef4\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-w9ldt"
Sep 10 17:36:22.826835 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:36:22.826112    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/b1aa3439-0cd7-4496-b1e8-7b535c305002-multus-socket-dir-parent\") pod \"multus-z5bpf\" (UID: \"b1aa3439-0cd7-4496-b1e8-7b535c305002\") " pod="openshift-multus/multus-z5bpf"
Sep 10 17:36:22.826835 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:36:22.826130    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-w28mp\" (UniqueName: \"kubernetes.io/projected/766766d2-fa56-49c2-9928-b50cbbf9173a-kube-api-access-w28mp\") pod \"network-check-target-7ptnb\" (UID: \"766766d2-fa56-49c2-9928-b50cbbf9173a\") " pod="openshift-network-diagnostics/network-check-target-7ptnb"
Sep 10 17:36:22.826835 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:36:22.826146    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qqwzw\" (UniqueName: \"kubernetes.io/projected/91b2593e-0a42-4735-bbd6-616678613892-kube-api-access-qqwzw\") pod \"iptables-alerter-tlnqf\" (UID: \"91b2593e-0a42-4735-bbd6-616678613892\") " pod="openshift-network-operator/iptables-alerter-tlnqf"
Sep 10 17:36:22.826835 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:36:22.826169    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/356f6437-710e-4991-b8db-e96a35f1152d-etc-sysctl-conf\") pod \"tuned-s8k6d\" (UID: \"356f6437-710e-4991-b8db-e96a35f1152d\") " pod="openshift-cluster-node-tuning-operator/tuned-s8k6d"
Sep 10 17:36:22.826835 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:36:22.826224    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/b1aa3439-0cd7-4496-b1e8-7b535c305002-host-var-lib-cni-multus\") pod \"multus-z5bpf\" (UID: \"b1aa3439-0cd7-4496-b1e8-7b535c305002\") " pod="openshift-multus/multus-z5bpf"
Sep 10 17:36:22.827592 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:36:22.826271    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/b1aa3439-0cd7-4496-b1e8-7b535c305002-host-var-lib-kubelet\") pod \"multus-z5bpf\" (UID: \"b1aa3439-0cd7-4496-b1e8-7b535c305002\") " pod="openshift-multus/multus-z5bpf"
Sep 10 17:36:22.827592 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:36:22.826333    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/b1aa3439-0cd7-4496-b1e8-7b535c305002-etc-kubernetes\") pod \"multus-z5bpf\" (UID: \"b1aa3439-0cd7-4496-b1e8-7b535c305002\") " pod="openshift-multus/multus-z5bpf"
Sep 10 17:36:22.827592 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:36:22.826365    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/099b7fa9-6908-43e3-b444-73b7f327290d-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-bqj9g\" (UID: \"099b7fa9-6908-43e3-b444-73b7f327290d\") " pod="openshift-multus/multus-additional-cni-plugins-bqj9g"
Sep 10 17:36:22.827592 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:36:22.826416    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/356f6437-710e-4991-b8db-e96a35f1152d-sys\") pod \"tuned-s8k6d\" (UID: \"356f6437-710e-4991-b8db-e96a35f1152d\") " pod="openshift-cluster-node-tuning-operator/tuned-s8k6d"
Sep 10 17:36:22.827592 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:36:22.826444    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/356f6437-710e-4991-b8db-e96a35f1152d-tmp\") pod \"tuned-s8k6d\" (UID: \"356f6437-710e-4991-b8db-e96a35f1152d\") " pod="openshift-cluster-node-tuning-operator/tuned-s8k6d"
Sep 10 17:36:22.827592 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:36:22.826463    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/099b7fa9-6908-43e3-b444-73b7f327290d-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-bqj9g\" (UID: \"099b7fa9-6908-43e3-b444-73b7f327290d\") " pod="openshift-multus/multus-additional-cni-plugins-bqj9g"
Sep 10 17:36:22.827592 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:36:22.826468    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/84e91010-05f8-4b38-801c-8e462cd4e38d-hosts-file\") pod \"node-resolver-jkplw\" (UID: \"84e91010-05f8-4b38-801c-8e462cd4e38d\") " pod="openshift-dns/node-resolver-jkplw"
Sep 10 17:36:22.827592 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:36:22.826510    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/81e433e1-460f-4d44-88dc-4ab3666e140e-run-ovn\") pod \"ovnkube-node-7nn7k\" (UID: \"81e433e1-460f-4d44-88dc-4ab3666e140e\") " pod="openshift-ovn-kubernetes/ovnkube-node-7nn7k"
Sep 10 17:36:22.827592 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:36:22.826535    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/dee7aebb-2ba6-4bfe-8fb4-371bcc4e56b1-dbus\") pod \"global-pull-secret-syncer-2t2rd\" (UID: \"dee7aebb-2ba6-4bfe-8fb4-371bcc4e56b1\") " pod="kube-system/global-pull-secret-syncer-2t2rd"
Sep 10 17:36:22.827592 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:36:22.826578    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/b1aa3439-0cd7-4496-b1e8-7b535c305002-cnibin\") pod \"multus-z5bpf\" (UID: \"b1aa3439-0cd7-4496-b1e8-7b535c305002\") " pod="openshift-multus/multus-z5bpf"
Sep 10 17:36:22.827592 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:36:22.826605    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/099b7fa9-6908-43e3-b444-73b7f327290d-cnibin\") pod \"multus-additional-cni-plugins-bqj9g\" (UID: \"099b7fa9-6908-43e3-b444-73b7f327290d\") " pod="openshift-multus/multus-additional-cni-plugins-bqj9g"
Sep 10 17:36:22.827592 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:36:22.826669    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/356f6437-710e-4991-b8db-e96a35f1152d-etc-sysconfig\") pod \"tuned-s8k6d\" (UID: \"356f6437-710e-4991-b8db-e96a35f1152d\") " pod="openshift-cluster-node-tuning-operator/tuned-s8k6d"
Sep 10 17:36:22.827592 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:36:22.826688    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/099b7fa9-6908-43e3-b444-73b7f327290d-cnibin\") pod \"multus-additional-cni-plugins-bqj9g\" (UID: \"099b7fa9-6908-43e3-b444-73b7f327290d\") " pod="openshift-multus/multus-additional-cni-plugins-bqj9g"
Sep 10 17:36:22.827592 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:36:22.826698    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/356f6437-710e-4991-b8db-e96a35f1152d-var-lib-kubelet\") pod \"tuned-s8k6d\" (UID: \"356f6437-710e-4991-b8db-e96a35f1152d\") " pod="openshift-cluster-node-tuning-operator/tuned-s8k6d"
Sep 10 17:36:22.827592 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:36:22.826750    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/81e433e1-460f-4d44-88dc-4ab3666e140e-ovnkube-script-lib\") pod \"ovnkube-node-7nn7k\" (UID: \"81e433e1-460f-4d44-88dc-4ab3666e140e\") " pod="openshift-ovn-kubernetes/ovnkube-node-7nn7k"
Sep 10 17:36:22.827592 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:36:22.826779    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/dee7aebb-2ba6-4bfe-8fb4-371bcc4e56b1-kubelet-config\") pod \"global-pull-secret-syncer-2t2rd\" (UID: \"dee7aebb-2ba6-4bfe-8fb4-371bcc4e56b1\") " pod="kube-system/global-pull-secret-syncer-2t2rd"
Sep 10 17:36:22.827592 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:36:22.826816    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/b1aa3439-0cd7-4496-b1e8-7b535c305002-host-run-netns\") pod \"multus-z5bpf\" (UID: \"b1aa3439-0cd7-4496-b1e8-7b535c305002\") " pod="openshift-multus/multus-z5bpf"
Sep 10 17:36:22.828374 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:36:22.826841    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hgdlq\" (UniqueName: \"kubernetes.io/projected/356f6437-710e-4991-b8db-e96a35f1152d-kube-api-access-hgdlq\") pod \"tuned-s8k6d\" (UID: \"356f6437-710e-4991-b8db-e96a35f1152d\") " pod="openshift-cluster-node-tuning-operator/tuned-s8k6d"
Sep 10 17:36:22.828374 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:36:22.826880    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vf9k6\" (UniqueName: \"kubernetes.io/projected/84e91010-05f8-4b38-801c-8e462cd4e38d-kube-api-access-vf9k6\") pod \"node-resolver-jkplw\" (UID: \"84e91010-05f8-4b38-801c-8e462cd4e38d\") " pod="openshift-dns/node-resolver-jkplw"
Sep 10 17:36:22.828374 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:36:22.826910    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/81e433e1-460f-4d44-88dc-4ab3666e140e-host-run-netns\") pod \"ovnkube-node-7nn7k\" (UID: \"81e433e1-460f-4d44-88dc-4ab3666e140e\") " pod="openshift-ovn-kubernetes/ovnkube-node-7nn7k"
Sep 10 17:36:22.828374 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:36:22.826931    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/356f6437-710e-4991-b8db-e96a35f1152d-run\") pod \"tuned-s8k6d\" (UID: \"356f6437-710e-4991-b8db-e96a35f1152d\") " pod="openshift-cluster-node-tuning-operator/tuned-s8k6d"
Sep 10 17:36:22.828374 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:36:22.826952    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/88d343e2-9756-438b-a19c-973b53948e68-metrics-certs\") pod \"network-metrics-daemon-b88bx\" (UID: \"88d343e2-9756-438b-a19c-973b53948e68\") " pod="openshift-multus/network-metrics-daemon-b88bx"
Sep 10 17:36:22.828374 ip-10-0-141-180 kubenswrapper[2581]: E0910 17:36:22.827025    2581 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 10 17:36:22.828374 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:36:22.827046    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/81e433e1-460f-4d44-88dc-4ab3666e140e-host-slash\") pod \"ovnkube-node-7nn7k\" (UID: \"81e433e1-460f-4d44-88dc-4ab3666e140e\") " pod="openshift-ovn-kubernetes/ovnkube-node-7nn7k"
Sep 10 17:36:22.828374 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:36:22.827073    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sntfq\" (UniqueName: \"kubernetes.io/projected/81e433e1-460f-4d44-88dc-4ab3666e140e-kube-api-access-sntfq\") pod \"ovnkube-node-7nn7k\" (UID: \"81e433e1-460f-4d44-88dc-4ab3666e140e\") " pod="openshift-ovn-kubernetes/ovnkube-node-7nn7k"
Sep 10 17:36:22.828374 ip-10-0-141-180 kubenswrapper[2581]: E0910 17:36:22.827097    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/88d343e2-9756-438b-a19c-973b53948e68-metrics-certs podName:88d343e2-9756-438b-a19c-973b53948e68 nodeName:}" failed. No retries permitted until 2026-09-10 17:36:23.327068285 +0000 UTC m=+2.981142183 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/88d343e2-9756-438b-a19c-973b53948e68-metrics-certs") pod "network-metrics-daemon-b88bx" (UID: "88d343e2-9756-438b-a19c-973b53948e68") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 10 17:36:22.828374 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:36:22.827094    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/099b7fa9-6908-43e3-b444-73b7f327290d-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-bqj9g\" (UID: \"099b7fa9-6908-43e3-b444-73b7f327290d\") " pod="openshift-multus/multus-additional-cni-plugins-bqj9g"
Sep 10 17:36:22.828374 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:36:22.827176    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/b4e9eaae-6a8f-459d-9953-f9b6a529bef4-socket-dir\") pod \"aws-ebs-csi-driver-node-w9ldt\" (UID: \"b4e9eaae-6a8f-459d-9953-f9b6a529bef4\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-w9ldt"
Sep 10 17:36:22.828374 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:36:22.827236    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/099b7fa9-6908-43e3-b444-73b7f327290d-cni-binary-copy\") pod \"multus-additional-cni-plugins-bqj9g\" (UID: \"099b7fa9-6908-43e3-b444-73b7f327290d\") " pod="openshift-multus/multus-additional-cni-plugins-bqj9g"
Sep 10 17:36:22.828374 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:36:22.827264    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lcsqq\" (UniqueName: \"kubernetes.io/projected/22cfca39-1a39-4266-9ee7-25dfc8723196-kube-api-access-lcsqq\") pod \"node-ca-h6rpt\" (UID: \"22cfca39-1a39-4266-9ee7-25dfc8723196\") " pod="openshift-image-registry/node-ca-h6rpt"
Sep 10 17:36:22.828374 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:36:22.827296    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/356f6437-710e-4991-b8db-e96a35f1152d-etc-kubernetes\") pod \"tuned-s8k6d\" (UID: \"356f6437-710e-4991-b8db-e96a35f1152d\") " pod="openshift-cluster-node-tuning-operator/tuned-s8k6d"
Sep 10 17:36:22.828374 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:36:22.827319    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/81e433e1-460f-4d44-88dc-4ab3666e140e-host-kubelet\") pod \"ovnkube-node-7nn7k\" (UID: \"81e433e1-460f-4d44-88dc-4ab3666e140e\") " pod="openshift-ovn-kubernetes/ovnkube-node-7nn7k"
Sep 10 17:36:22.828374 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:36:22.827344    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/b1aa3439-0cd7-4496-b1e8-7b535c305002-host-var-lib-cni-bin\") pod \"multus-z5bpf\" (UID: \"b1aa3439-0cd7-4496-b1e8-7b535c305002\") " pod="openshift-multus/multus-z5bpf"
Sep 10 17:36:22.829030 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:36:22.827786    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/099b7fa9-6908-43e3-b444-73b7f327290d-cni-binary-copy\") pod \"multus-additional-cni-plugins-bqj9g\" (UID: \"099b7fa9-6908-43e3-b444-73b7f327290d\") " pod="openshift-multus/multus-additional-cni-plugins-bqj9g"
Sep 10 17:36:22.829972 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:36:22.829945    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/459a65ca-0e07-47e2-aba9-6af10bb5387f-agent-certs\") pod \"konnectivity-agent-7p25j\" (UID: \"459a65ca-0e07-47e2-aba9-6af10bb5387f\") " pod="kube-system/konnectivity-agent-7p25j"
Sep 10 17:36:22.831046 ip-10-0-141-180 kubenswrapper[2581]: E0910 17:36:22.831026    2581 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 10 17:36:22.831141 ip-10-0-141-180 kubenswrapper[2581]: E0910 17:36:22.831051    2581 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 10 17:36:22.831141 ip-10-0-141-180 kubenswrapper[2581]: E0910 17:36:22.831063    2581 projected.go:196] Error preparing data for projected volume kube-api-access-w28mp for pod openshift-network-diagnostics/network-check-target-7ptnb: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 10 17:36:22.831141 ip-10-0-141-180 kubenswrapper[2581]: E0910 17:36:22.831133    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/766766d2-fa56-49c2-9928-b50cbbf9173a-kube-api-access-w28mp podName:766766d2-fa56-49c2-9928-b50cbbf9173a nodeName:}" failed. No retries permitted until 2026-09-10 17:36:23.331115356 +0000 UTC m=+2.985189243 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-w28mp" (UniqueName: "kubernetes.io/projected/766766d2-fa56-49c2-9928-b50cbbf9173a-kube-api-access-w28mp") pod "network-check-target-7ptnb" (UID: "766766d2-fa56-49c2-9928-b50cbbf9173a") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 10 17:36:22.833250 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:36:22.833225    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hlrdr\" (UniqueName: \"kubernetes.io/projected/099b7fa9-6908-43e3-b444-73b7f327290d-kube-api-access-hlrdr\") pod \"multus-additional-cni-plugins-bqj9g\" (UID: \"099b7fa9-6908-43e3-b444-73b7f327290d\") " pod="openshift-multus/multus-additional-cni-plugins-bqj9g"
Sep 10 17:36:22.833337 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:36:22.833254    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jph88\" (UniqueName: \"kubernetes.io/projected/88d343e2-9756-438b-a19c-973b53948e68-kube-api-access-jph88\") pod \"network-metrics-daemon-b88bx\" (UID: \"88d343e2-9756-438b-a19c-973b53948e68\") " pod="openshift-multus/network-metrics-daemon-b88bx"
Sep 10 17:36:22.834871 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:36:22.834850    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lcsqq\" (UniqueName: \"kubernetes.io/projected/22cfca39-1a39-4266-9ee7-25dfc8723196-kube-api-access-lcsqq\") pod \"node-ca-h6rpt\" (UID: \"22cfca39-1a39-4266-9ee7-25dfc8723196\") " pod="openshift-image-registry/node-ca-h6rpt"
Sep 10 17:36:22.860387 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:36:22.860360    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-180.ec2.internal" event={"ID":"a4eaab8f22f33ba54d00328e9bf6c226","Type":"ContainerStarted","Data":"5afff424eda1f744bfaeceb24636455a94febef98454492a73fdb59814e433af"}
Sep 10 17:36:22.915855 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:36:22.915828    2581 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 10 17:36:22.928144 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:36:22.928116    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vf9k6\" (UniqueName: \"kubernetes.io/projected/84e91010-05f8-4b38-801c-8e462cd4e38d-kube-api-access-vf9k6\") pod \"node-resolver-jkplw\" (UID: \"84e91010-05f8-4b38-801c-8e462cd4e38d\") " pod="openshift-dns/node-resolver-jkplw"
Sep 10 17:36:22.928256 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:36:22.928161    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/81e433e1-460f-4d44-88dc-4ab3666e140e-host-run-netns\") pod \"ovnkube-node-7nn7k\" (UID: \"81e433e1-460f-4d44-88dc-4ab3666e140e\") " pod="openshift-ovn-kubernetes/ovnkube-node-7nn7k"
Sep 10 17:36:22.928256 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:36:22.928183    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/356f6437-710e-4991-b8db-e96a35f1152d-run\") pod \"tuned-s8k6d\" (UID: \"356f6437-710e-4991-b8db-e96a35f1152d\") " pod="openshift-cluster-node-tuning-operator/tuned-s8k6d"
Sep 10 17:36:22.928256 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:36:22.928208    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/81e433e1-460f-4d44-88dc-4ab3666e140e-host-slash\") pod \"ovnkube-node-7nn7k\" (UID: \"81e433e1-460f-4d44-88dc-4ab3666e140e\") " pod="openshift-ovn-kubernetes/ovnkube-node-7nn7k"
Sep 10 17:36:22.928256 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:36:22.928225    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sntfq\" (UniqueName: \"kubernetes.io/projected/81e433e1-460f-4d44-88dc-4ab3666e140e-kube-api-access-sntfq\") pod \"ovnkube-node-7nn7k\" (UID: \"81e433e1-460f-4d44-88dc-4ab3666e140e\") " pod="openshift-ovn-kubernetes/ovnkube-node-7nn7k"
Sep 10 17:36:22.928592 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:36:22.928249    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/b4e9eaae-6a8f-459d-9953-f9b6a529bef4-socket-dir\") pod \"aws-ebs-csi-driver-node-w9ldt\" (UID: \"b4e9eaae-6a8f-459d-9953-f9b6a529bef4\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-w9ldt"
Sep 10 17:36:22.928592 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:36:22.928282    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/356f6437-710e-4991-b8db-e96a35f1152d-etc-kubernetes\") pod \"tuned-s8k6d\" (UID: \"356f6437-710e-4991-b8db-e96a35f1152d\") " pod="openshift-cluster-node-tuning-operator/tuned-s8k6d"
Sep 10 17:36:22.928592 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:36:22.928287    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/356f6437-710e-4991-b8db-e96a35f1152d-run\") pod \"tuned-s8k6d\" (UID: \"356f6437-710e-4991-b8db-e96a35f1152d\") " pod="openshift-cluster-node-tuning-operator/tuned-s8k6d"
Sep 10 17:36:22.928592 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:36:22.928304    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/81e433e1-460f-4d44-88dc-4ab3666e140e-host-kubelet\") pod \"ovnkube-node-7nn7k\" (UID: \"81e433e1-460f-4d44-88dc-4ab3666e140e\") " pod="openshift-ovn-kubernetes/ovnkube-node-7nn7k"
Sep 10 17:36:22.928592 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:36:22.928289    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/81e433e1-460f-4d44-88dc-4ab3666e140e-host-slash\") pod \"ovnkube-node-7nn7k\" (UID: \"81e433e1-460f-4d44-88dc-4ab3666e140e\") " pod="openshift-ovn-kubernetes/ovnkube-node-7nn7k"
Sep 10 17:36:22.928592 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:36:22.928329    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/b1aa3439-0cd7-4496-b1e8-7b535c305002-host-var-lib-cni-bin\") pod \"multus-z5bpf\" (UID: \"b1aa3439-0cd7-4496-b1e8-7b535c305002\") " pod="openshift-multus/multus-z5bpf"
Sep 10 17:36:22.928592 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:36:22.928333    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/81e433e1-460f-4d44-88dc-4ab3666e140e-host-run-netns\") pod \"ovnkube-node-7nn7k\" (UID: \"81e433e1-460f-4d44-88dc-4ab3666e140e\") " pod="openshift-ovn-kubernetes/ovnkube-node-7nn7k"
Sep 10 17:36:22.928592 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:36:22.928352    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/81e433e1-460f-4d44-88dc-4ab3666e140e-systemd-units\") pod \"ovnkube-node-7nn7k\" (UID: \"81e433e1-460f-4d44-88dc-4ab3666e140e\") " pod="openshift-ovn-kubernetes/ovnkube-node-7nn7k"
Sep 10 17:36:22.928592 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:36:22.928372    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/81e433e1-460f-4d44-88dc-4ab3666e140e-host-kubelet\") pod \"ovnkube-node-7nn7k\" (UID: \"81e433e1-460f-4d44-88dc-4ab3666e140e\") " pod="openshift-ovn-kubernetes/ovnkube-node-7nn7k"
Sep 10 17:36:22.928592 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:36:22.928383    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/81e433e1-460f-4d44-88dc-4ab3666e140e-systemd-units\") pod \"ovnkube-node-7nn7k\" (UID: \"81e433e1-460f-4d44-88dc-4ab3666e140e\") " pod="openshift-ovn-kubernetes/ovnkube-node-7nn7k"
Sep 10 17:36:22.928592 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:36:22.928391    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/81e433e1-460f-4d44-88dc-4ab3666e140e-var-lib-openvswitch\") pod \"ovnkube-node-7nn7k\" (UID: \"81e433e1-460f-4d44-88dc-4ab3666e140e\") " pod="openshift-ovn-kubernetes/ovnkube-node-7nn7k"
Sep 10 17:36:22.928592 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:36:22.928411    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/b1aa3439-0cd7-4496-b1e8-7b535c305002-host-var-lib-cni-bin\") pod \"multus-z5bpf\" (UID: \"b1aa3439-0cd7-4496-b1e8-7b535c305002\") " pod="openshift-multus/multus-z5bpf"
Sep 10 17:36:22.928592 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:36:22.928422    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/81e433e1-460f-4d44-88dc-4ab3666e140e-var-lib-openvswitch\") pod \"ovnkube-node-7nn7k\" (UID: \"81e433e1-460f-4d44-88dc-4ab3666e140e\") " pod="openshift-ovn-kubernetes/ovnkube-node-7nn7k"
Sep 10 17:36:22.928592 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:36:22.928421    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/b4e9eaae-6a8f-459d-9953-f9b6a529bef4-device-dir\") pod \"aws-ebs-csi-driver-node-w9ldt\" (UID: \"b4e9eaae-6a8f-459d-9953-f9b6a529bef4\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-w9ldt"
Sep 10 17:36:22.928592 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:36:22.928431    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/b4e9eaae-6a8f-459d-9953-f9b6a529bef4-socket-dir\") pod \"aws-ebs-csi-driver-node-w9ldt\" (UID: \"b4e9eaae-6a8f-459d-9953-f9b6a529bef4\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-w9ldt"
Sep 10 17:36:22.928592 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:36:22.928436    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/356f6437-710e-4991-b8db-e96a35f1152d-etc-kubernetes\") pod \"tuned-s8k6d\" (UID: \"356f6437-710e-4991-b8db-e96a35f1152d\") " pod="openshift-cluster-node-tuning-operator/tuned-s8k6d"
Sep 10 17:36:22.928592 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:36:22.928451    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/b4e9eaae-6a8f-459d-9953-f9b6a529bef4-sys-fs\") pod \"aws-ebs-csi-driver-node-w9ldt\" (UID: \"b4e9eaae-6a8f-459d-9953-f9b6a529bef4\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-w9ldt"
Sep 10 17:36:22.928592 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:36:22.928478    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/b1aa3439-0cd7-4496-b1e8-7b535c305002-multus-cni-dir\") pod \"multus-z5bpf\" (UID: \"b1aa3439-0cd7-4496-b1e8-7b535c305002\") " pod="openshift-multus/multus-z5bpf"
Sep 10 17:36:22.929522 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:36:22.928495    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/b4e9eaae-6a8f-459d-9953-f9b6a529bef4-sys-fs\") pod \"aws-ebs-csi-driver-node-w9ldt\" (UID: \"b4e9eaae-6a8f-459d-9953-f9b6a529bef4\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-w9ldt"
Sep 10 17:36:22.929522 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:36:22.928503    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/b1aa3439-0cd7-4496-b1e8-7b535c305002-host-run-k8s-cni-cncf-io\") pod \"multus-z5bpf\" (UID: \"b1aa3439-0cd7-4496-b1e8-7b535c305002\") " pod="openshift-multus/multus-z5bpf"
Sep 10 17:36:22.929522 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:36:22.928527    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/356f6437-710e-4991-b8db-e96a35f1152d-etc-sysctl-d\") pod \"tuned-s8k6d\" (UID: \"356f6437-710e-4991-b8db-e96a35f1152d\") " pod="openshift-cluster-node-tuning-operator/tuned-s8k6d"
Sep 10 17:36:22.929522 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:36:22.928532    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/b4e9eaae-6a8f-459d-9953-f9b6a529bef4-device-dir\") pod \"aws-ebs-csi-driver-node-w9ldt\" (UID: \"b4e9eaae-6a8f-459d-9953-f9b6a529bef4\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-w9ldt"
Sep 10 17:36:22.929522 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:36:22.928549    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/356f6437-710e-4991-b8db-e96a35f1152d-etc-systemd\") pod \"tuned-s8k6d\" (UID: \"356f6437-710e-4991-b8db-e96a35f1152d\") " pod="openshift-cluster-node-tuning-operator/tuned-s8k6d"
Sep 10 17:36:22.929522 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:36:22.928560    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/b1aa3439-0cd7-4496-b1e8-7b535c305002-host-run-k8s-cni-cncf-io\") pod \"multus-z5bpf\" (UID: \"b1aa3439-0cd7-4496-b1e8-7b535c305002\") " pod="openshift-multus/multus-z5bpf"
Sep 10 17:36:22.929522 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:36:22.928566    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/b1aa3439-0cd7-4496-b1e8-7b535c305002-multus-cni-dir\") pod \"multus-z5bpf\" (UID: \"b1aa3439-0cd7-4496-b1e8-7b535c305002\") " pod="openshift-multus/multus-z5bpf"
Sep 10 17:36:22.929522 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:36:22.928572    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/b1aa3439-0cd7-4496-b1e8-7b535c305002-system-cni-dir\") pod \"multus-z5bpf\" (UID: \"b1aa3439-0cd7-4496-b1e8-7b535c305002\") " pod="openshift-multus/multus-z5bpf"
Sep 10 17:36:22.929522 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:36:22.928665    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/356f6437-710e-4991-b8db-e96a35f1152d-etc-sysctl-d\") pod \"tuned-s8k6d\" (UID: \"356f6437-710e-4991-b8db-e96a35f1152d\") " pod="openshift-cluster-node-tuning-operator/tuned-s8k6d"
Sep 10 17:36:22.929522 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:36:22.928672    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pl8ps\" (UniqueName: \"kubernetes.io/projected/b1aa3439-0cd7-4496-b1e8-7b535c305002-kube-api-access-pl8ps\") pod \"multus-z5bpf\" (UID: \"b1aa3439-0cd7-4496-b1e8-7b535c305002\") " pod="openshift-multus/multus-z5bpf"
Sep 10 17:36:22.929522 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:36:22.928698    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/b1aa3439-0cd7-4496-b1e8-7b535c305002-system-cni-dir\") pod \"multus-z5bpf\" (UID: \"b1aa3439-0cd7-4496-b1e8-7b535c305002\") " pod="openshift-multus/multus-z5bpf"
Sep 10 17:36:22.929522 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:36:22.928705    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/dee7aebb-2ba6-4bfe-8fb4-371bcc4e56b1-original-pull-secret\") pod \"global-pull-secret-syncer-2t2rd\" (UID: \"dee7aebb-2ba6-4bfe-8fb4-371bcc4e56b1\") " pod="kube-system/global-pull-secret-syncer-2t2rd"
Sep 10 17:36:22.929522 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:36:22.928711    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/356f6437-710e-4991-b8db-e96a35f1152d-etc-systemd\") pod \"tuned-s8k6d\" (UID: \"356f6437-710e-4991-b8db-e96a35f1152d\") " pod="openshift-cluster-node-tuning-operator/tuned-s8k6d"
Sep 10 17:36:22.929522 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:36:22.928737    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/b4e9eaae-6a8f-459d-9953-f9b6a529bef4-etc-selinux\") pod \"aws-ebs-csi-driver-node-w9ldt\" (UID: \"b4e9eaae-6a8f-459d-9953-f9b6a529bef4\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-w9ldt"
Sep 10 17:36:22.929522 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:36:22.928765    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/b1aa3439-0cd7-4496-b1e8-7b535c305002-multus-conf-dir\") pod \"multus-z5bpf\" (UID: \"b1aa3439-0cd7-4496-b1e8-7b535c305002\") " pod="openshift-multus/multus-z5bpf"
Sep 10 17:36:22.929522 ip-10-0-141-180 kubenswrapper[2581]: E0910 17:36:22.928776    2581 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 10 17:36:22.929522 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:36:22.928787    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/81e433e1-460f-4d44-88dc-4ab3666e140e-log-socket\") pod \"ovnkube-node-7nn7k\" (UID: \"81e433e1-460f-4d44-88dc-4ab3666e140e\") " pod="openshift-ovn-kubernetes/ovnkube-node-7nn7k"
Sep 10 17:36:22.929522 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:36:22.928814    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/91b2593e-0a42-4735-bbd6-616678613892-host-slash\") pod \"iptables-alerter-tlnqf\" (UID: \"91b2593e-0a42-4735-bbd6-616678613892\") " pod="openshift-network-operator/iptables-alerter-tlnqf"
Sep 10 17:36:22.930449 ip-10-0-141-180 kubenswrapper[2581]: E0910 17:36:22.928838    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/dee7aebb-2ba6-4bfe-8fb4-371bcc4e56b1-original-pull-secret podName:dee7aebb-2ba6-4bfe-8fb4-371bcc4e56b1 nodeName:}" failed. No retries permitted until 2026-09-10 17:36:23.428818053 +0000 UTC m=+3.082891939 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/dee7aebb-2ba6-4bfe-8fb4-371bcc4e56b1-original-pull-secret") pod "global-pull-secret-syncer-2t2rd" (UID: "dee7aebb-2ba6-4bfe-8fb4-371bcc4e56b1") : object "kube-system"/"original-pull-secret" not registered
Sep 10 17:36:22.930449 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:36:22.928843    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/b1aa3439-0cd7-4496-b1e8-7b535c305002-multus-conf-dir\") pod \"multus-z5bpf\" (UID: \"b1aa3439-0cd7-4496-b1e8-7b535c305002\") " pod="openshift-multus/multus-z5bpf"
Sep 10 17:36:22.930449 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:36:22.928848    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/b4e9eaae-6a8f-459d-9953-f9b6a529bef4-etc-selinux\") pod \"aws-ebs-csi-driver-node-w9ldt\" (UID: \"b4e9eaae-6a8f-459d-9953-f9b6a529bef4\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-w9ldt"
Sep 10 17:36:22.930449 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:36:22.928863    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/81e433e1-460f-4d44-88dc-4ab3666e140e-node-log\") pod \"ovnkube-node-7nn7k\" (UID: \"81e433e1-460f-4d44-88dc-4ab3666e140e\") " pod="openshift-ovn-kubernetes/ovnkube-node-7nn7k"
Sep 10 17:36:22.930449 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:36:22.928851    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/91b2593e-0a42-4735-bbd6-616678613892-host-slash\") pod \"iptables-alerter-tlnqf\" (UID: \"91b2593e-0a42-4735-bbd6-616678613892\") " pod="openshift-network-operator/iptables-alerter-tlnqf"
Sep 10 17:36:22.930449 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:36:22.928891    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/81e433e1-460f-4d44-88dc-4ab3666e140e-log-socket\") pod \"ovnkube-node-7nn7k\" (UID: \"81e433e1-460f-4d44-88dc-4ab3666e140e\") " pod="openshift-ovn-kubernetes/ovnkube-node-7nn7k"
Sep 10 17:36:22.930449 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:36:22.928898    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/81e433e1-460f-4d44-88dc-4ab3666e140e-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-7nn7k\" (UID: \"81e433e1-460f-4d44-88dc-4ab3666e140e\") " pod="openshift-ovn-kubernetes/ovnkube-node-7nn7k"
Sep 10 17:36:22.930449 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:36:22.928906    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/81e433e1-460f-4d44-88dc-4ab3666e140e-node-log\") pod \"ovnkube-node-7nn7k\" (UID: \"81e433e1-460f-4d44-88dc-4ab3666e140e\") " pod="openshift-ovn-kubernetes/ovnkube-node-7nn7k"
Sep 10 17:36:22.930449 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:36:22.928924    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/b4e9eaae-6a8f-459d-9953-f9b6a529bef4-registration-dir\") pod \"aws-ebs-csi-driver-node-w9ldt\" (UID: \"b4e9eaae-6a8f-459d-9953-f9b6a529bef4\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-w9ldt"
Sep 10 17:36:22.930449 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:36:22.928951    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/356f6437-710e-4991-b8db-e96a35f1152d-etc-modprobe-d\") pod \"tuned-s8k6d\" (UID: \"356f6437-710e-4991-b8db-e96a35f1152d\") " pod="openshift-cluster-node-tuning-operator/tuned-s8k6d"
Sep 10 17:36:22.930449 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:36:22.928974    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/81e433e1-460f-4d44-88dc-4ab3666e140e-run-systemd\") pod \"ovnkube-node-7nn7k\" (UID: \"81e433e1-460f-4d44-88dc-4ab3666e140e\") " pod="openshift-ovn-kubernetes/ovnkube-node-7nn7k"
Sep 10 17:36:22.930449 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:36:22.928988    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/81e433e1-460f-4d44-88dc-4ab3666e140e-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-7nn7k\" (UID: \"81e433e1-460f-4d44-88dc-4ab3666e140e\") " pod="openshift-ovn-kubernetes/ovnkube-node-7nn7k"
Sep 10 17:36:22.930449 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:36:22.928995    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/81e433e1-460f-4d44-88dc-4ab3666e140e-run-openvswitch\") pod \"ovnkube-node-7nn7k\" (UID: \"81e433e1-460f-4d44-88dc-4ab3666e140e\") " pod="openshift-ovn-kubernetes/ovnkube-node-7nn7k"
Sep 10 17:36:22.930449 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:36:22.929038    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/b1aa3439-0cd7-4496-b1e8-7b535c305002-cni-binary-copy\") pod \"multus-z5bpf\" (UID: \"b1aa3439-0cd7-4496-b1e8-7b535c305002\") " pod="openshift-multus/multus-z5bpf"
Sep 10 17:36:22.930449 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:36:22.929037    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/b4e9eaae-6a8f-459d-9953-f9b6a529bef4-registration-dir\") pod \"aws-ebs-csi-driver-node-w9ldt\" (UID: \"b4e9eaae-6a8f-459d-9953-f9b6a529bef4\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-w9ldt"
Sep 10 17:36:22.930449 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:36:22.929052    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/356f6437-710e-4991-b8db-e96a35f1152d-etc-modprobe-d\") pod \"tuned-s8k6d\" (UID: \"356f6437-710e-4991-b8db-e96a35f1152d\") " pod="openshift-cluster-node-tuning-operator/tuned-s8k6d"
Sep 10 17:36:22.931184 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:36:22.929040    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/81e433e1-460f-4d44-88dc-4ab3666e140e-run-systemd\") pod \"ovnkube-node-7nn7k\" (UID: \"81e433e1-460f-4d44-88dc-4ab3666e140e\") " pod="openshift-ovn-kubernetes/ovnkube-node-7nn7k"
Sep 10 17:36:22.931184 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:36:22.929076    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/81e433e1-460f-4d44-88dc-4ab3666e140e-run-openvswitch\") pod \"ovnkube-node-7nn7k\" (UID: \"81e433e1-460f-4d44-88dc-4ab3666e140e\") " pod="openshift-ovn-kubernetes/ovnkube-node-7nn7k"
Sep 10 17:36:22.931184 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:36:22.929063    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/b1aa3439-0cd7-4496-b1e8-7b535c305002-host-run-multus-certs\") pod \"multus-z5bpf\" (UID: \"b1aa3439-0cd7-4496-b1e8-7b535c305002\") " pod="openshift-multus/multus-z5bpf"
Sep 10 17:36:22.931184 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:36:22.929111    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/b1aa3439-0cd7-4496-b1e8-7b535c305002-host-run-multus-certs\") pod \"multus-z5bpf\" (UID: \"b1aa3439-0cd7-4496-b1e8-7b535c305002\") " pod="openshift-multus/multus-z5bpf"
Sep 10 17:36:22.931184 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:36:22.929118    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/81e433e1-460f-4d44-88dc-4ab3666e140e-host-cni-netd\") pod \"ovnkube-node-7nn7k\" (UID: \"81e433e1-460f-4d44-88dc-4ab3666e140e\") " pod="openshift-ovn-kubernetes/ovnkube-node-7nn7k"
Sep 10 17:36:22.931184 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:36:22.929143    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/b1aa3439-0cd7-4496-b1e8-7b535c305002-hostroot\") pod \"multus-z5bpf\" (UID: \"b1aa3439-0cd7-4496-b1e8-7b535c305002\") " pod="openshift-multus/multus-z5bpf"
Sep 10 17:36:22.931184 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:36:22.929152    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/81e433e1-460f-4d44-88dc-4ab3666e140e-host-cni-netd\") pod \"ovnkube-node-7nn7k\" (UID: \"81e433e1-460f-4d44-88dc-4ab3666e140e\") " pod="openshift-ovn-kubernetes/ovnkube-node-7nn7k"
Sep 10 17:36:22.931184 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:36:22.929169    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/356f6437-710e-4991-b8db-e96a35f1152d-etc-tuned\") pod \"tuned-s8k6d\" (UID: \"356f6437-710e-4991-b8db-e96a35f1152d\") " pod="openshift-cluster-node-tuning-operator/tuned-s8k6d"
Sep 10 17:36:22.931184 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:36:22.929205    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/81e433e1-460f-4d44-88dc-4ab3666e140e-etc-openvswitch\") pod \"ovnkube-node-7nn7k\" (UID: \"81e433e1-460f-4d44-88dc-4ab3666e140e\") " pod="openshift-ovn-kubernetes/ovnkube-node-7nn7k"
Sep 10 17:36:22.931184 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:36:22.929229    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/81e433e1-460f-4d44-88dc-4ab3666e140e-host-run-ovn-kubernetes\") pod \"ovnkube-node-7nn7k\" (UID: \"81e433e1-460f-4d44-88dc-4ab3666e140e\") " pod="openshift-ovn-kubernetes/ovnkube-node-7nn7k"
Sep 10 17:36:22.931184 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:36:22.929253    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jrbvs\" (UniqueName: \"kubernetes.io/projected/b4e9eaae-6a8f-459d-9953-f9b6a529bef4-kube-api-access-jrbvs\") pod \"aws-ebs-csi-driver-node-w9ldt\" (UID: \"b4e9eaae-6a8f-459d-9953-f9b6a529bef4\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-w9ldt"
Sep 10 17:36:22.931184 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:36:22.929258    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/b1aa3439-0cd7-4496-b1e8-7b535c305002-hostroot\") pod \"multus-z5bpf\" (UID: \"b1aa3439-0cd7-4496-b1e8-7b535c305002\") " pod="openshift-multus/multus-z5bpf"
Sep 10 17:36:22.931184 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:36:22.929277    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/b1aa3439-0cd7-4496-b1e8-7b535c305002-multus-daemon-config\") pod \"multus-z5bpf\" (UID: \"b1aa3439-0cd7-4496-b1e8-7b535c305002\") " pod="openshift-multus/multus-z5bpf"
Sep 10 17:36:22.931184 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:36:22.929302    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/91b2593e-0a42-4735-bbd6-616678613892-iptables-alerter-script\") pod \"iptables-alerter-tlnqf\" (UID: \"91b2593e-0a42-4735-bbd6-616678613892\") " pod="openshift-network-operator/iptables-alerter-tlnqf"
Sep 10 17:36:22.931184 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:36:22.929315    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/81e433e1-460f-4d44-88dc-4ab3666e140e-etc-openvswitch\") pod \"ovnkube-node-7nn7k\" (UID: \"81e433e1-460f-4d44-88dc-4ab3666e140e\") " pod="openshift-ovn-kubernetes/ovnkube-node-7nn7k"
Sep 10 17:36:22.931184 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:36:22.929325    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/356f6437-710e-4991-b8db-e96a35f1152d-lib-modules\") pod \"tuned-s8k6d\" (UID: \"356f6437-710e-4991-b8db-e96a35f1152d\") " pod="openshift-cluster-node-tuning-operator/tuned-s8k6d"
Sep 10 17:36:22.931184 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:36:22.929346    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/356f6437-710e-4991-b8db-e96a35f1152d-host\") pod \"tuned-s8k6d\" (UID: \"356f6437-710e-4991-b8db-e96a35f1152d\") " pod="openshift-cluster-node-tuning-operator/tuned-s8k6d"
Sep 10 17:36:22.931184 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:36:22.929370    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/84e91010-05f8-4b38-801c-8e462cd4e38d-tmp-dir\") pod \"node-resolver-jkplw\" (UID: \"84e91010-05f8-4b38-801c-8e462cd4e38d\") " pod="openshift-dns/node-resolver-jkplw"
Sep 10 17:36:22.931826 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:36:22.929391    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/81e433e1-460f-4d44-88dc-4ab3666e140e-host-cni-bin\") pod \"ovnkube-node-7nn7k\" (UID: \"81e433e1-460f-4d44-88dc-4ab3666e140e\") " pod="openshift-ovn-kubernetes/ovnkube-node-7nn7k"
Sep 10 17:36:22.931826 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:36:22.929412    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/81e433e1-460f-4d44-88dc-4ab3666e140e-ovnkube-config\") pod \"ovnkube-node-7nn7k\" (UID: \"81e433e1-460f-4d44-88dc-4ab3666e140e\") " pod="openshift-ovn-kubernetes/ovnkube-node-7nn7k"
Sep 10 17:36:22.931826 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:36:22.929454    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/81e433e1-460f-4d44-88dc-4ab3666e140e-env-overrides\") pod \"ovnkube-node-7nn7k\" (UID: \"81e433e1-460f-4d44-88dc-4ab3666e140e\") " pod="openshift-ovn-kubernetes/ovnkube-node-7nn7k"
Sep 10 17:36:22.931826 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:36:22.929494    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/81e433e1-460f-4d44-88dc-4ab3666e140e-ovn-node-metrics-cert\") pod \"ovnkube-node-7nn7k\" (UID: \"81e433e1-460f-4d44-88dc-4ab3666e140e\") " pod="openshift-ovn-kubernetes/ovnkube-node-7nn7k"
Sep 10 17:36:22.931826 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:36:22.929496    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/356f6437-710e-4991-b8db-e96a35f1152d-host\") pod \"tuned-s8k6d\" (UID: \"356f6437-710e-4991-b8db-e96a35f1152d\") " pod="openshift-cluster-node-tuning-operator/tuned-s8k6d"
Sep 10 17:36:22.931826 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:36:22.929518    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/b4e9eaae-6a8f-459d-9953-f9b6a529bef4-kubelet-dir\") pod \"aws-ebs-csi-driver-node-w9ldt\" (UID: \"b4e9eaae-6a8f-459d-9953-f9b6a529bef4\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-w9ldt"
Sep 10 17:36:22.931826 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:36:22.929529    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/81e433e1-460f-4d44-88dc-4ab3666e140e-host-cni-bin\") pod \"ovnkube-node-7nn7k\" (UID: \"81e433e1-460f-4d44-88dc-4ab3666e140e\") " pod="openshift-ovn-kubernetes/ovnkube-node-7nn7k"
Sep 10 17:36:22.931826 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:36:22.929634    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/b1aa3439-0cd7-4496-b1e8-7b535c305002-cni-binary-copy\") pod \"multus-z5bpf\" (UID: \"b1aa3439-0cd7-4496-b1e8-7b535c305002\") " pod="openshift-multus/multus-z5bpf"
Sep 10 17:36:22.931826 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:36:22.929303    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/81e433e1-460f-4d44-88dc-4ab3666e140e-host-run-ovn-kubernetes\") pod \"ovnkube-node-7nn7k\" (UID: \"81e433e1-460f-4d44-88dc-4ab3666e140e\") " pod="openshift-ovn-kubernetes/ovnkube-node-7nn7k"
Sep 10 17:36:22.931826 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:36:22.929825    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/356f6437-710e-4991-b8db-e96a35f1152d-lib-modules\") pod \"tuned-s8k6d\" (UID: \"356f6437-710e-4991-b8db-e96a35f1152d\") " pod="openshift-cluster-node-tuning-operator/tuned-s8k6d"
Sep 10 17:36:22.931826 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:36:22.929852    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/b1aa3439-0cd7-4496-b1e8-7b535c305002-multus-daemon-config\") pod \"multus-z5bpf\" (UID: \"b1aa3439-0cd7-4496-b1e8-7b535c305002\") " pod="openshift-multus/multus-z5bpf"
Sep 10 17:36:22.931826 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:36:22.929886    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/84e91010-05f8-4b38-801c-8e462cd4e38d-tmp-dir\") pod \"node-resolver-jkplw\" (UID: \"84e91010-05f8-4b38-801c-8e462cd4e38d\") " pod="openshift-dns/node-resolver-jkplw"
Sep 10 17:36:22.931826 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:36:22.929901    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/b1aa3439-0cd7-4496-b1e8-7b535c305002-multus-socket-dir-parent\") pod \"multus-z5bpf\" (UID: \"b1aa3439-0cd7-4496-b1e8-7b535c305002\") " pod="openshift-multus/multus-z5bpf"
Sep 10 17:36:22.931826 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:36:22.929892    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/b4e9eaae-6a8f-459d-9953-f9b6a529bef4-kubelet-dir\") pod \"aws-ebs-csi-driver-node-w9ldt\" (UID: \"b4e9eaae-6a8f-459d-9953-f9b6a529bef4\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-w9ldt"
Sep 10 17:36:22.931826 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:36:22.929951    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qqwzw\" (UniqueName: \"kubernetes.io/projected/91b2593e-0a42-4735-bbd6-616678613892-kube-api-access-qqwzw\") pod \"iptables-alerter-tlnqf\" (UID: \"91b2593e-0a42-4735-bbd6-616678613892\") " pod="openshift-network-operator/iptables-alerter-tlnqf"
Sep 10 17:36:22.931826 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:36:22.929976    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/356f6437-710e-4991-b8db-e96a35f1152d-etc-sysctl-conf\") pod \"tuned-s8k6d\" (UID: \"356f6437-710e-4991-b8db-e96a35f1152d\") " pod="openshift-cluster-node-tuning-operator/tuned-s8k6d"
Sep 10 17:36:22.931826 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:36:22.929995    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/b1aa3439-0cd7-4496-b1e8-7b535c305002-multus-socket-dir-parent\") pod \"multus-z5bpf\" (UID: \"b1aa3439-0cd7-4496-b1e8-7b535c305002\") " pod="openshift-multus/multus-z5bpf"
Sep 10 17:36:22.931826 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:36:22.930004    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/b1aa3439-0cd7-4496-b1e8-7b535c305002-host-var-lib-cni-multus\") pod \"multus-z5bpf\" (UID: \"b1aa3439-0cd7-4496-b1e8-7b535c305002\") " pod="openshift-multus/multus-z5bpf"
Sep 10 17:36:22.932707 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:36:22.930037    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/b1aa3439-0cd7-4496-b1e8-7b535c305002-host-var-lib-kubelet\") pod \"multus-z5bpf\" (UID: \"b1aa3439-0cd7-4496-b1e8-7b535c305002\") " pod="openshift-multus/multus-z5bpf"
Sep 10 17:36:22.932707 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:36:22.930064    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/b1aa3439-0cd7-4496-b1e8-7b535c305002-etc-kubernetes\") pod \"multus-z5bpf\" (UID: \"b1aa3439-0cd7-4496-b1e8-7b535c305002\") " pod="openshift-multus/multus-z5bpf"
Sep 10 17:36:22.932707 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:36:22.930082    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/81e433e1-460f-4d44-88dc-4ab3666e140e-env-overrides\") pod \"ovnkube-node-7nn7k\" (UID: \"81e433e1-460f-4d44-88dc-4ab3666e140e\") " pod="openshift-ovn-kubernetes/ovnkube-node-7nn7k"
Sep 10 17:36:22.932707 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:36:22.930103    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/356f6437-710e-4991-b8db-e96a35f1152d-sys\") pod \"tuned-s8k6d\" (UID: \"356f6437-710e-4991-b8db-e96a35f1152d\") " pod="openshift-cluster-node-tuning-operator/tuned-s8k6d"
Sep 10 17:36:22.932707 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:36:22.930131    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/356f6437-710e-4991-b8db-e96a35f1152d-tmp\") pod \"tuned-s8k6d\" (UID: \"356f6437-710e-4991-b8db-e96a35f1152d\") " pod="openshift-cluster-node-tuning-operator/tuned-s8k6d"
Sep 10 17:36:22.932707 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:36:22.930137    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/b1aa3439-0cd7-4496-b1e8-7b535c305002-etc-kubernetes\") pod \"multus-z5bpf\" (UID: \"b1aa3439-0cd7-4496-b1e8-7b535c305002\") " pod="openshift-multus/multus-z5bpf"
Sep 10 17:36:22.932707 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:36:22.930142    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/b1aa3439-0cd7-4496-b1e8-7b535c305002-host-var-lib-kubelet\") pod \"multus-z5bpf\" (UID: \"b1aa3439-0cd7-4496-b1e8-7b535c305002\") " pod="openshift-multus/multus-z5bpf"
Sep 10 17:36:22.932707 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:36:22.930152    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/356f6437-710e-4991-b8db-e96a35f1152d-etc-sysctl-conf\") pod \"tuned-s8k6d\" (UID: \"356f6437-710e-4991-b8db-e96a35f1152d\") " pod="openshift-cluster-node-tuning-operator/tuned-s8k6d"
Sep 10 17:36:22.932707 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:36:22.930060    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/91b2593e-0a42-4735-bbd6-616678613892-iptables-alerter-script\") pod \"iptables-alerter-tlnqf\" (UID: \"91b2593e-0a42-4735-bbd6-616678613892\") " pod="openshift-network-operator/iptables-alerter-tlnqf"
Sep 10 17:36:22.932707 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:36:22.930107    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/b1aa3439-0cd7-4496-b1e8-7b535c305002-host-var-lib-cni-multus\") pod \"multus-z5bpf\" (UID: \"b1aa3439-0cd7-4496-b1e8-7b535c305002\") " pod="openshift-multus/multus-z5bpf"
Sep 10 17:36:22.932707 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:36:22.930154    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/84e91010-05f8-4b38-801c-8e462cd4e38d-hosts-file\") pod \"node-resolver-jkplw\" (UID: \"84e91010-05f8-4b38-801c-8e462cd4e38d\") " pod="openshift-dns/node-resolver-jkplw"
Sep 10 17:36:22.932707 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:36:22.930188    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/356f6437-710e-4991-b8db-e96a35f1152d-sys\") pod \"tuned-s8k6d\" (UID: \"356f6437-710e-4991-b8db-e96a35f1152d\") " pod="openshift-cluster-node-tuning-operator/tuned-s8k6d"
Sep 10 17:36:22.932707 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:36:22.930207    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/84e91010-05f8-4b38-801c-8e462cd4e38d-hosts-file\") pod \"node-resolver-jkplw\" (UID: \"84e91010-05f8-4b38-801c-8e462cd4e38d\") " pod="openshift-dns/node-resolver-jkplw"
Sep 10 17:36:22.932707 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:36:22.930246    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/81e433e1-460f-4d44-88dc-4ab3666e140e-run-ovn\") pod \"ovnkube-node-7nn7k\" (UID: \"81e433e1-460f-4d44-88dc-4ab3666e140e\") " pod="openshift-ovn-kubernetes/ovnkube-node-7nn7k"
Sep 10 17:36:22.932707 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:36:22.930291    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/81e433e1-460f-4d44-88dc-4ab3666e140e-run-ovn\") pod \"ovnkube-node-7nn7k\" (UID: \"81e433e1-460f-4d44-88dc-4ab3666e140e\") " pod="openshift-ovn-kubernetes/ovnkube-node-7nn7k"
Sep 10 17:36:22.932707 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:36:22.930322    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/dee7aebb-2ba6-4bfe-8fb4-371bcc4e56b1-dbus\") pod \"global-pull-secret-syncer-2t2rd\" (UID: \"dee7aebb-2ba6-4bfe-8fb4-371bcc4e56b1\") " pod="kube-system/global-pull-secret-syncer-2t2rd"
Sep 10 17:36:22.932707 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:36:22.930347    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/b1aa3439-0cd7-4496-b1e8-7b535c305002-cnibin\") pod \"multus-z5bpf\" (UID: \"b1aa3439-0cd7-4496-b1e8-7b535c305002\") " pod="openshift-multus/multus-z5bpf"
Sep 10 17:36:22.932707 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:36:22.930377    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/356f6437-710e-4991-b8db-e96a35f1152d-etc-sysconfig\") pod \"tuned-s8k6d\" (UID: \"356f6437-710e-4991-b8db-e96a35f1152d\") " pod="openshift-cluster-node-tuning-operator/tuned-s8k6d"
Sep 10 17:36:22.933576 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:36:22.930399    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/356f6437-710e-4991-b8db-e96a35f1152d-var-lib-kubelet\") pod \"tuned-s8k6d\" (UID: \"356f6437-710e-4991-b8db-e96a35f1152d\") " pod="openshift-cluster-node-tuning-operator/tuned-s8k6d"
Sep 10 17:36:22.933576 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:36:22.930377    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/dee7aebb-2ba6-4bfe-8fb4-371bcc4e56b1-dbus\") pod \"global-pull-secret-syncer-2t2rd\" (UID: \"dee7aebb-2ba6-4bfe-8fb4-371bcc4e56b1\") " pod="kube-system/global-pull-secret-syncer-2t2rd"
Sep 10 17:36:22.933576 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:36:22.930422    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/81e433e1-460f-4d44-88dc-4ab3666e140e-ovnkube-script-lib\") pod \"ovnkube-node-7nn7k\" (UID: \"81e433e1-460f-4d44-88dc-4ab3666e140e\") " pod="openshift-ovn-kubernetes/ovnkube-node-7nn7k"
Sep 10 17:36:22.933576 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:36:22.930423    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/b1aa3439-0cd7-4496-b1e8-7b535c305002-cnibin\") pod \"multus-z5bpf\" (UID: \"b1aa3439-0cd7-4496-b1e8-7b535c305002\") " pod="openshift-multus/multus-z5bpf"
Sep 10 17:36:22.933576 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:36:22.930481    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/356f6437-710e-4991-b8db-e96a35f1152d-var-lib-kubelet\") pod \"tuned-s8k6d\" (UID: \"356f6437-710e-4991-b8db-e96a35f1152d\") " pod="openshift-cluster-node-tuning-operator/tuned-s8k6d"
Sep 10 17:36:22.933576 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:36:22.930505    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/356f6437-710e-4991-b8db-e96a35f1152d-etc-sysconfig\") pod \"tuned-s8k6d\" (UID: \"356f6437-710e-4991-b8db-e96a35f1152d\") " pod="openshift-cluster-node-tuning-operator/tuned-s8k6d"
Sep 10 17:36:22.933576 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:36:22.930534    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/dee7aebb-2ba6-4bfe-8fb4-371bcc4e56b1-kubelet-config\") pod \"global-pull-secret-syncer-2t2rd\" (UID: \"dee7aebb-2ba6-4bfe-8fb4-371bcc4e56b1\") " pod="kube-system/global-pull-secret-syncer-2t2rd"
Sep 10 17:36:22.933576 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:36:22.930558    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/b1aa3439-0cd7-4496-b1e8-7b535c305002-host-run-netns\") pod \"multus-z5bpf\" (UID: \"b1aa3439-0cd7-4496-b1e8-7b535c305002\") " pod="openshift-multus/multus-z5bpf"
Sep 10 17:36:22.933576 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:36:22.930602    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/b1aa3439-0cd7-4496-b1e8-7b535c305002-host-run-netns\") pod \"multus-z5bpf\" (UID: \"b1aa3439-0cd7-4496-b1e8-7b535c305002\") " pod="openshift-multus/multus-z5bpf"
Sep 10 17:36:22.933576 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:36:22.930609    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/dee7aebb-2ba6-4bfe-8fb4-371bcc4e56b1-kubelet-config\") pod \"global-pull-secret-syncer-2t2rd\" (UID: \"dee7aebb-2ba6-4bfe-8fb4-371bcc4e56b1\") " pod="kube-system/global-pull-secret-syncer-2t2rd"
Sep 10 17:36:22.933576 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:36:22.930630    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hgdlq\" (UniqueName: \"kubernetes.io/projected/356f6437-710e-4991-b8db-e96a35f1152d-kube-api-access-hgdlq\") pod \"tuned-s8k6d\" (UID: \"356f6437-710e-4991-b8db-e96a35f1152d\") " pod="openshift-cluster-node-tuning-operator/tuned-s8k6d"
Sep 10 17:36:22.933576 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:36:22.930653    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/81e433e1-460f-4d44-88dc-4ab3666e140e-ovnkube-config\") pod \"ovnkube-node-7nn7k\" (UID: \"81e433e1-460f-4d44-88dc-4ab3666e140e\") " pod="openshift-ovn-kubernetes/ovnkube-node-7nn7k"
Sep 10 17:36:22.933576 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:36:22.930950    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/81e433e1-460f-4d44-88dc-4ab3666e140e-ovnkube-script-lib\") pod \"ovnkube-node-7nn7k\" (UID: \"81e433e1-460f-4d44-88dc-4ab3666e140e\") " pod="openshift-ovn-kubernetes/ovnkube-node-7nn7k"
Sep 10 17:36:22.933576 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:36:22.932082    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/356f6437-710e-4991-b8db-e96a35f1152d-etc-tuned\") pod \"tuned-s8k6d\" (UID: \"356f6437-710e-4991-b8db-e96a35f1152d\") " pod="openshift-cluster-node-tuning-operator/tuned-s8k6d"
Sep 10 17:36:22.933576 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:36:22.932114    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/356f6437-710e-4991-b8db-e96a35f1152d-tmp\") pod \"tuned-s8k6d\" (UID: \"356f6437-710e-4991-b8db-e96a35f1152d\") " pod="openshift-cluster-node-tuning-operator/tuned-s8k6d"
Sep 10 17:36:22.933576 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:36:22.932351    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/81e433e1-460f-4d44-88dc-4ab3666e140e-ovn-node-metrics-cert\") pod \"ovnkube-node-7nn7k\" (UID: \"81e433e1-460f-4d44-88dc-4ab3666e140e\") " pod="openshift-ovn-kubernetes/ovnkube-node-7nn7k"
Sep 10 17:36:22.938300 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:36:22.938250    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-sntfq\" (UniqueName: \"kubernetes.io/projected/81e433e1-460f-4d44-88dc-4ab3666e140e-kube-api-access-sntfq\") pod \"ovnkube-node-7nn7k\" (UID: \"81e433e1-460f-4d44-88dc-4ab3666e140e\") " pod="openshift-ovn-kubernetes/ovnkube-node-7nn7k"
Sep 10 17:36:22.939709 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:36:22.939633    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vf9k6\" (UniqueName: \"kubernetes.io/projected/84e91010-05f8-4b38-801c-8e462cd4e38d-kube-api-access-vf9k6\") pod \"node-resolver-jkplw\" (UID: \"84e91010-05f8-4b38-801c-8e462cd4e38d\") " pod="openshift-dns/node-resolver-jkplw"
Sep 10 17:36:22.939986 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:36:22.939970    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pl8ps\" (UniqueName: \"kubernetes.io/projected/b1aa3439-0cd7-4496-b1e8-7b535c305002-kube-api-access-pl8ps\") pod \"multus-z5bpf\" (UID: \"b1aa3439-0cd7-4496-b1e8-7b535c305002\") " pod="openshift-multus/multus-z5bpf"
Sep 10 17:36:22.940549 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:36:22.940531    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hgdlq\" (UniqueName: \"kubernetes.io/projected/356f6437-710e-4991-b8db-e96a35f1152d-kube-api-access-hgdlq\") pod \"tuned-s8k6d\" (UID: \"356f6437-710e-4991-b8db-e96a35f1152d\") " pod="openshift-cluster-node-tuning-operator/tuned-s8k6d"
Sep 10 17:36:22.940720 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:36:22.940705    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jrbvs\" (UniqueName: \"kubernetes.io/projected/b4e9eaae-6a8f-459d-9953-f9b6a529bef4-kube-api-access-jrbvs\") pod \"aws-ebs-csi-driver-node-w9ldt\" (UID: \"b4e9eaae-6a8f-459d-9953-f9b6a529bef4\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-w9ldt"
Sep 10 17:36:22.940856 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:36:22.940837    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qqwzw\" (UniqueName: \"kubernetes.io/projected/91b2593e-0a42-4735-bbd6-616678613892-kube-api-access-qqwzw\") pod \"iptables-alerter-tlnqf\" (UID: \"91b2593e-0a42-4735-bbd6-616678613892\") " pod="openshift-network-operator/iptables-alerter-tlnqf"
Sep 10 17:36:23.025088 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:36:23.025018    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-h6rpt"
Sep 10 17:36:23.033625 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:36:23.033603    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-7p25j"
Sep 10 17:36:23.042143 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:36:23.042123    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-bqj9g"
Sep 10 17:36:23.049110 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:36:23.049089    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-tlnqf"
Sep 10 17:36:23.057327 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:36:23.057312    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-s8k6d"
Sep 10 17:36:23.066100 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:36:23.066083    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-w9ldt"
Sep 10 17:36:23.072531 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:36:23.072512    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-7nn7k"
Sep 10 17:36:23.079612 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:36:23.079597    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-jkplw"
Sep 10 17:36:23.085349 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:36:23.085327    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-z5bpf"
Sep 10 17:36:23.333814 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:36:23.333743    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-w28mp\" (UniqueName: \"kubernetes.io/projected/766766d2-fa56-49c2-9928-b50cbbf9173a-kube-api-access-w28mp\") pod \"network-check-target-7ptnb\" (UID: \"766766d2-fa56-49c2-9928-b50cbbf9173a\") " pod="openshift-network-diagnostics/network-check-target-7ptnb"
Sep 10 17:36:23.333814 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:36:23.333791    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/88d343e2-9756-438b-a19c-973b53948e68-metrics-certs\") pod \"network-metrics-daemon-b88bx\" (UID: \"88d343e2-9756-438b-a19c-973b53948e68\") " pod="openshift-multus/network-metrics-daemon-b88bx"
Sep 10 17:36:23.334078 ip-10-0-141-180 kubenswrapper[2581]: E0910 17:36:23.333932    2581 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 10 17:36:23.334078 ip-10-0-141-180 kubenswrapper[2581]: E0910 17:36:23.333944    2581 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 10 17:36:23.334078 ip-10-0-141-180 kubenswrapper[2581]: E0910 17:36:23.333959    2581 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 10 17:36:23.334078 ip-10-0-141-180 kubenswrapper[2581]: E0910 17:36:23.333972    2581 projected.go:196] Error preparing data for projected volume kube-api-access-w28mp for pod openshift-network-diagnostics/network-check-target-7ptnb: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 10 17:36:23.334078 ip-10-0-141-180 kubenswrapper[2581]: E0910 17:36:23.334008    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/88d343e2-9756-438b-a19c-973b53948e68-metrics-certs podName:88d343e2-9756-438b-a19c-973b53948e68 nodeName:}" failed. No retries permitted until 2026-09-10 17:36:24.333989177 +0000 UTC m=+3.988063067 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/88d343e2-9756-438b-a19c-973b53948e68-metrics-certs") pod "network-metrics-daemon-b88bx" (UID: "88d343e2-9756-438b-a19c-973b53948e68") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 10 17:36:23.334078 ip-10-0-141-180 kubenswrapper[2581]: E0910 17:36:23.334035    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/766766d2-fa56-49c2-9928-b50cbbf9173a-kube-api-access-w28mp podName:766766d2-fa56-49c2-9928-b50cbbf9173a nodeName:}" failed. No retries permitted until 2026-09-10 17:36:24.334025953 +0000 UTC m=+3.988099837 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-w28mp" (UniqueName: "kubernetes.io/projected/766766d2-fa56-49c2-9928-b50cbbf9173a-kube-api-access-w28mp") pod "network-check-target-7ptnb" (UID: "766766d2-fa56-49c2-9928-b50cbbf9173a") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 10 17:36:23.435057 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:36:23.435025    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/dee7aebb-2ba6-4bfe-8fb4-371bcc4e56b1-original-pull-secret\") pod \"global-pull-secret-syncer-2t2rd\" (UID: \"dee7aebb-2ba6-4bfe-8fb4-371bcc4e56b1\") " pod="kube-system/global-pull-secret-syncer-2t2rd"
Sep 10 17:36:23.435205 ip-10-0-141-180 kubenswrapper[2581]: E0910 17:36:23.435170    2581 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 10 17:36:23.435266 ip-10-0-141-180 kubenswrapper[2581]: E0910 17:36:23.435229    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/dee7aebb-2ba6-4bfe-8fb4-371bcc4e56b1-original-pull-secret podName:dee7aebb-2ba6-4bfe-8fb4-371bcc4e56b1 nodeName:}" failed. No retries permitted until 2026-09-10 17:36:24.435214545 +0000 UTC m=+4.089288432 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/dee7aebb-2ba6-4bfe-8fb4-371bcc4e56b1-original-pull-secret") pod "global-pull-secret-syncer-2t2rd" (UID: "dee7aebb-2ba6-4bfe-8fb4-371bcc4e56b1") : object "kube-system"/"original-pull-secret" not registered
Sep 10 17:36:23.654827 ip-10-0-141-180 kubenswrapper[2581]: W0910 17:36:23.654803    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod356f6437_710e_4991_b8db_e96a35f1152d.slice/crio-0729c23bd332609205ca0dda0880ddf1d9b1186480f9b72f65e91a922e05e1e0 WatchSource:0}: Error finding container 0729c23bd332609205ca0dda0880ddf1d9b1186480f9b72f65e91a922e05e1e0: Status 404 returned error can't find the container with id 0729c23bd332609205ca0dda0880ddf1d9b1186480f9b72f65e91a922e05e1e0
Sep 10 17:36:23.656201 ip-10-0-141-180 kubenswrapper[2581]: W0910 17:36:23.656175    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod91b2593e_0a42_4735_bbd6_616678613892.slice/crio-18aaa0a0966688333093b064b1e5cef74f3060c995fe4e2635fb20be626df046 WatchSource:0}: Error finding container 18aaa0a0966688333093b064b1e5cef74f3060c995fe4e2635fb20be626df046: Status 404 returned error can't find the container with id 18aaa0a0966688333093b064b1e5cef74f3060c995fe4e2635fb20be626df046
Sep 10 17:36:23.659216 ip-10-0-141-180 kubenswrapper[2581]: W0910 17:36:23.659191    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb1aa3439_0cd7_4496_b1e8_7b535c305002.slice/crio-f4b43e94e7c8d757786859377774748803547c76d532587e39c6a2b9b0d3c5fd WatchSource:0}: Error finding container f4b43e94e7c8d757786859377774748803547c76d532587e39c6a2b9b0d3c5fd: Status 404 returned error can't find the container with id f4b43e94e7c8d757786859377774748803547c76d532587e39c6a2b9b0d3c5fd
Sep 10 17:36:23.660018 ip-10-0-141-180 kubenswrapper[2581]: W0910 17:36:23.659993    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb4e9eaae_6a8f_459d_9953_f9b6a529bef4.slice/crio-4ebd4853ea3cd03eb25dfce8efd1b705184655129be469da40aaf18e57f38f99 WatchSource:0}: Error finding container 4ebd4853ea3cd03eb25dfce8efd1b705184655129be469da40aaf18e57f38f99: Status 404 returned error can't find the container with id 4ebd4853ea3cd03eb25dfce8efd1b705184655129be469da40aaf18e57f38f99
Sep 10 17:36:23.661184 ip-10-0-141-180 kubenswrapper[2581]: W0910 17:36:23.661161    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod84e91010_05f8_4b38_801c_8e462cd4e38d.slice/crio-f4cbe07e8f27ffad5bf13890ad2d09a9971f92f86f27386f321b74641ce8bd3f WatchSource:0}: Error finding container f4cbe07e8f27ffad5bf13890ad2d09a9971f92f86f27386f321b74641ce8bd3f: Status 404 returned error can't find the container with id f4cbe07e8f27ffad5bf13890ad2d09a9971f92f86f27386f321b74641ce8bd3f
Sep 10 17:36:23.662036 ip-10-0-141-180 kubenswrapper[2581]: W0910 17:36:23.662018    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod22cfca39_1a39_4266_9ee7_25dfc8723196.slice/crio-92a87456c1455e42764ac811482247fcbef6dfbabb721ac3df077b3ad08b2385 WatchSource:0}: Error finding container 92a87456c1455e42764ac811482247fcbef6dfbabb721ac3df077b3ad08b2385: Status 404 returned error can't find the container with id 92a87456c1455e42764ac811482247fcbef6dfbabb721ac3df077b3ad08b2385
Sep 10 17:36:23.662989 ip-10-0-141-180 kubenswrapper[2581]: W0910 17:36:23.662955    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod81e433e1_460f_4d44_88dc_4ab3666e140e.slice/crio-aec13a55b467b97a9f763c12ce7bd8db871d832e807bdee0fddf0143bb25b508 WatchSource:0}: Error finding container aec13a55b467b97a9f763c12ce7bd8db871d832e807bdee0fddf0143bb25b508: Status 404 returned error can't find the container with id aec13a55b467b97a9f763c12ce7bd8db871d832e807bdee0fddf0143bb25b508
Sep 10 17:36:23.663794 ip-10-0-141-180 kubenswrapper[2581]: W0910 17:36:23.663744    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod459a65ca_0e07_47e2_aba9_6af10bb5387f.slice/crio-0d515b2f8f13e0a15ff23f5dbd23909f07ec6838c173eb342b9a2dbc9206d412 WatchSource:0}: Error finding container 0d515b2f8f13e0a15ff23f5dbd23909f07ec6838c173eb342b9a2dbc9206d412: Status 404 returned error can't find the container with id 0d515b2f8f13e0a15ff23f5dbd23909f07ec6838c173eb342b9a2dbc9206d412
Sep 10 17:36:23.665835 ip-10-0-141-180 kubenswrapper[2581]: W0910 17:36:23.665532    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod099b7fa9_6908_43e3_b444_73b7f327290d.slice/crio-b06721ae6ce310ada55f5630aac75338f04096d0c0f60ad1960abfa7acb28bf1 WatchSource:0}: Error finding container b06721ae6ce310ada55f5630aac75338f04096d0c0f60ad1960abfa7acb28bf1: Status 404 returned error can't find the container with id b06721ae6ce310ada55f5630aac75338f04096d0c0f60ad1960abfa7acb28bf1
Sep 10 17:36:23.758124 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:36:23.757987    2581 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-09-09 17:31:22 +0000 UTC" deadline="2028-05-09 15:29:36.43448611 +0000 UTC"
Sep 10 17:36:23.758449 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:36:23.758130    2581 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="14565h53m12.676360983s"
Sep 10 17:36:23.863093 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:36:23.863063    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-bqj9g" event={"ID":"099b7fa9-6908-43e3-b444-73b7f327290d","Type":"ContainerStarted","Data":"b06721ae6ce310ada55f5630aac75338f04096d0c0f60ad1960abfa7acb28bf1"}
Sep 10 17:36:23.864257 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:36:23.864228    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-w9ldt" event={"ID":"b4e9eaae-6a8f-459d-9953-f9b6a529bef4","Type":"ContainerStarted","Data":"4ebd4853ea3cd03eb25dfce8efd1b705184655129be469da40aaf18e57f38f99"}
Sep 10 17:36:23.865184 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:36:23.865164    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-7nn7k" event={"ID":"81e433e1-460f-4d44-88dc-4ab3666e140e","Type":"ContainerStarted","Data":"aec13a55b467b97a9f763c12ce7bd8db871d832e807bdee0fddf0143bb25b508"}
Sep 10 17:36:23.866148 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:36:23.866130    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-h6rpt" event={"ID":"22cfca39-1a39-4266-9ee7-25dfc8723196","Type":"ContainerStarted","Data":"92a87456c1455e42764ac811482247fcbef6dfbabb721ac3df077b3ad08b2385"}
Sep 10 17:36:23.867153 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:36:23.867136    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-7p25j" event={"ID":"459a65ca-0e07-47e2-aba9-6af10bb5387f","Type":"ContainerStarted","Data":"0d515b2f8f13e0a15ff23f5dbd23909f07ec6838c173eb342b9a2dbc9206d412"}
Sep 10 17:36:23.868152 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:36:23.868135    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-z5bpf" event={"ID":"b1aa3439-0cd7-4496-b1e8-7b535c305002","Type":"ContainerStarted","Data":"f4b43e94e7c8d757786859377774748803547c76d532587e39c6a2b9b0d3c5fd"}
Sep 10 17:36:23.869245 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:36:23.869225    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-tlnqf" event={"ID":"91b2593e-0a42-4735-bbd6-616678613892","Type":"ContainerStarted","Data":"18aaa0a0966688333093b064b1e5cef74f3060c995fe4e2635fb20be626df046"}
Sep 10 17:36:23.871034 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:36:23.870999    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-141-180.ec2.internal" event={"ID":"414c1afd223c55080845d6fd4dfe2409","Type":"ContainerStarted","Data":"c5fa945cfe267c304972e2f91ec5d82b7251687b4f862eb20749d8cb9bbab14f"}
Sep 10 17:36:23.872185 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:36:23.872158    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-jkplw" event={"ID":"84e91010-05f8-4b38-801c-8e462cd4e38d","Type":"ContainerStarted","Data":"f4cbe07e8f27ffad5bf13890ad2d09a9971f92f86f27386f321b74641ce8bd3f"}
Sep 10 17:36:23.873520 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:36:23.873497    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-s8k6d" event={"ID":"356f6437-710e-4991-b8db-e96a35f1152d","Type":"ContainerStarted","Data":"0729c23bd332609205ca0dda0880ddf1d9b1186480f9b72f65e91a922e05e1e0"}
Sep 10 17:36:24.343397 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:36:24.341614    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/88d343e2-9756-438b-a19c-973b53948e68-metrics-certs\") pod \"network-metrics-daemon-b88bx\" (UID: \"88d343e2-9756-438b-a19c-973b53948e68\") " pod="openshift-multus/network-metrics-daemon-b88bx"
Sep 10 17:36:24.343397 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:36:24.341734    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-w28mp\" (UniqueName: \"kubernetes.io/projected/766766d2-fa56-49c2-9928-b50cbbf9173a-kube-api-access-w28mp\") pod \"network-check-target-7ptnb\" (UID: \"766766d2-fa56-49c2-9928-b50cbbf9173a\") " pod="openshift-network-diagnostics/network-check-target-7ptnb"
Sep 10 17:36:24.352271 ip-10-0-141-180 kubenswrapper[2581]: E0910 17:36:24.352245    2581 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 10 17:36:24.352408 ip-10-0-141-180 kubenswrapper[2581]: E0910 17:36:24.352324    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/88d343e2-9756-438b-a19c-973b53948e68-metrics-certs podName:88d343e2-9756-438b-a19c-973b53948e68 nodeName:}" failed. No retries permitted until 2026-09-10 17:36:26.352303128 +0000 UTC m=+6.006377025 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/88d343e2-9756-438b-a19c-973b53948e68-metrics-certs") pod "network-metrics-daemon-b88bx" (UID: "88d343e2-9756-438b-a19c-973b53948e68") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 10 17:36:24.352854 ip-10-0-141-180 kubenswrapper[2581]: E0910 17:36:24.352837    2581 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 10 17:36:24.352931 ip-10-0-141-180 kubenswrapper[2581]: E0910 17:36:24.352862    2581 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 10 17:36:24.352931 ip-10-0-141-180 kubenswrapper[2581]: E0910 17:36:24.352884    2581 projected.go:196] Error preparing data for projected volume kube-api-access-w28mp for pod openshift-network-diagnostics/network-check-target-7ptnb: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 10 17:36:24.353075 ip-10-0-141-180 kubenswrapper[2581]: E0910 17:36:24.352943    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/766766d2-fa56-49c2-9928-b50cbbf9173a-kube-api-access-w28mp podName:766766d2-fa56-49c2-9928-b50cbbf9173a nodeName:}" failed. No retries permitted until 2026-09-10 17:36:26.352917736 +0000 UTC m=+6.006991621 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-w28mp" (UniqueName: "kubernetes.io/projected/766766d2-fa56-49c2-9928-b50cbbf9173a-kube-api-access-w28mp") pod "network-check-target-7ptnb" (UID: "766766d2-fa56-49c2-9928-b50cbbf9173a") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 10 17:36:24.443055 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:36:24.443015    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/dee7aebb-2ba6-4bfe-8fb4-371bcc4e56b1-original-pull-secret\") pod \"global-pull-secret-syncer-2t2rd\" (UID: \"dee7aebb-2ba6-4bfe-8fb4-371bcc4e56b1\") " pod="kube-system/global-pull-secret-syncer-2t2rd"
Sep 10 17:36:24.443242 ip-10-0-141-180 kubenswrapper[2581]: E0910 17:36:24.443212    2581 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 10 17:36:24.443316 ip-10-0-141-180 kubenswrapper[2581]: E0910 17:36:24.443279    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/dee7aebb-2ba6-4bfe-8fb4-371bcc4e56b1-original-pull-secret podName:dee7aebb-2ba6-4bfe-8fb4-371bcc4e56b1 nodeName:}" failed. No retries permitted until 2026-09-10 17:36:26.443260768 +0000 UTC m=+6.097334665 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/dee7aebb-2ba6-4bfe-8fb4-371bcc4e56b1-original-pull-secret") pod "global-pull-secret-syncer-2t2rd" (UID: "dee7aebb-2ba6-4bfe-8fb4-371bcc4e56b1") : object "kube-system"/"original-pull-secret" not registered
Sep 10 17:36:24.759179 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:36:24.759036    2581 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-09-09 17:31:22 +0000 UTC" deadline="2028-03-19 07:57:59.706548421 +0000 UTC"
Sep 10 17:36:24.759179 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:36:24.759097    2581 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="13334h21m34.947455886s"
Sep 10 17:36:24.857683 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:36:24.857652    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-2t2rd"
Sep 10 17:36:24.857860 ip-10-0-141-180 kubenswrapper[2581]: E0910 17:36:24.857784    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-2t2rd" podUID="dee7aebb-2ba6-4bfe-8fb4-371bcc4e56b1"
Sep 10 17:36:24.858175 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:36:24.858156    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-7ptnb"
Sep 10 17:36:24.858271 ip-10-0-141-180 kubenswrapper[2581]: E0910 17:36:24.858250    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-7ptnb" podUID="766766d2-fa56-49c2-9928-b50cbbf9173a"
Sep 10 17:36:24.858328 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:36:24.858316    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-b88bx"
Sep 10 17:36:24.858409 ip-10-0-141-180 kubenswrapper[2581]: E0910 17:36:24.858393    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-b88bx" podUID="88d343e2-9756-438b-a19c-973b53948e68"
Sep 10 17:36:24.908462 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:36:24.908429    2581 generic.go:356] "Generic (PLEG): container finished" podID="a4eaab8f22f33ba54d00328e9bf6c226" containerID="d14629acc1e7ccb36e27f6ea3a36920f2a820cab36ba8e011bb6a39c1798dc43" exitCode=0
Sep 10 17:36:24.909385 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:36:24.909356    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-180.ec2.internal" event={"ID":"a4eaab8f22f33ba54d00328e9bf6c226","Type":"ContainerDied","Data":"d14629acc1e7ccb36e27f6ea3a36920f2a820cab36ba8e011bb6a39c1798dc43"}
Sep 10 17:36:24.924979 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:36:24.924552    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-141-180.ec2.internal" podStartSLOduration=2.924534321 podStartE2EDuration="2.924534321s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-10 17:36:22 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 17:36:23.890495097 +0000 UTC m=+3.544569004" watchObservedRunningTime="2026-09-10 17:36:24.924534321 +0000 UTC m=+4.578608226"
Sep 10 17:36:25.926748 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:36:25.925885    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-180.ec2.internal" event={"ID":"a4eaab8f22f33ba54d00328e9bf6c226","Type":"ContainerStarted","Data":"1d42e874992f92981efef482c8d99aa78ac3db132ae95052aae5605084dab8b5"}
Sep 10 17:36:26.358662 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:36:26.358555    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-w28mp\" (UniqueName: \"kubernetes.io/projected/766766d2-fa56-49c2-9928-b50cbbf9173a-kube-api-access-w28mp\") pod \"network-check-target-7ptnb\" (UID: \"766766d2-fa56-49c2-9928-b50cbbf9173a\") " pod="openshift-network-diagnostics/network-check-target-7ptnb"
Sep 10 17:36:26.358662 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:36:26.358620    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/88d343e2-9756-438b-a19c-973b53948e68-metrics-certs\") pod \"network-metrics-daemon-b88bx\" (UID: \"88d343e2-9756-438b-a19c-973b53948e68\") " pod="openshift-multus/network-metrics-daemon-b88bx"
Sep 10 17:36:26.358879 ip-10-0-141-180 kubenswrapper[2581]: E0910 17:36:26.358759    2581 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 10 17:36:26.358879 ip-10-0-141-180 kubenswrapper[2581]: E0910 17:36:26.358823    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/88d343e2-9756-438b-a19c-973b53948e68-metrics-certs podName:88d343e2-9756-438b-a19c-973b53948e68 nodeName:}" failed. No retries permitted until 2026-09-10 17:36:30.35880483 +0000 UTC m=+10.012878725 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/88d343e2-9756-438b-a19c-973b53948e68-metrics-certs") pod "network-metrics-daemon-b88bx" (UID: "88d343e2-9756-438b-a19c-973b53948e68") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 10 17:36:26.359254 ip-10-0-141-180 kubenswrapper[2581]: E0910 17:36:26.359230    2581 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 10 17:36:26.359254 ip-10-0-141-180 kubenswrapper[2581]: E0910 17:36:26.359258    2581 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 10 17:36:26.359419 ip-10-0-141-180 kubenswrapper[2581]: E0910 17:36:26.359270    2581 projected.go:196] Error preparing data for projected volume kube-api-access-w28mp for pod openshift-network-diagnostics/network-check-target-7ptnb: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 10 17:36:26.359419 ip-10-0-141-180 kubenswrapper[2581]: E0910 17:36:26.359318    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/766766d2-fa56-49c2-9928-b50cbbf9173a-kube-api-access-w28mp podName:766766d2-fa56-49c2-9928-b50cbbf9173a nodeName:}" failed. No retries permitted until 2026-09-10 17:36:30.35930154 +0000 UTC m=+10.013375422 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-w28mp" (UniqueName: "kubernetes.io/projected/766766d2-fa56-49c2-9928-b50cbbf9173a-kube-api-access-w28mp") pod "network-check-target-7ptnb" (UID: "766766d2-fa56-49c2-9928-b50cbbf9173a") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 10 17:36:26.459245 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:36:26.459204    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/dee7aebb-2ba6-4bfe-8fb4-371bcc4e56b1-original-pull-secret\") pod \"global-pull-secret-syncer-2t2rd\" (UID: \"dee7aebb-2ba6-4bfe-8fb4-371bcc4e56b1\") " pod="kube-system/global-pull-secret-syncer-2t2rd"
Sep 10 17:36:26.459424 ip-10-0-141-180 kubenswrapper[2581]: E0910 17:36:26.459382    2581 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 10 17:36:26.459484 ip-10-0-141-180 kubenswrapper[2581]: E0910 17:36:26.459447    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/dee7aebb-2ba6-4bfe-8fb4-371bcc4e56b1-original-pull-secret podName:dee7aebb-2ba6-4bfe-8fb4-371bcc4e56b1 nodeName:}" failed. No retries permitted until 2026-09-10 17:36:30.459428372 +0000 UTC m=+10.113502260 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/dee7aebb-2ba6-4bfe-8fb4-371bcc4e56b1-original-pull-secret") pod "global-pull-secret-syncer-2t2rd" (UID: "dee7aebb-2ba6-4bfe-8fb4-371bcc4e56b1") : object "kube-system"/"original-pull-secret" not registered
Sep 10 17:36:26.859424 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:36:26.859362    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-7ptnb"
Sep 10 17:36:26.859424 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:36:26.859392    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-b88bx"
Sep 10 17:36:26.859669 ip-10-0-141-180 kubenswrapper[2581]: E0910 17:36:26.859483    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-7ptnb" podUID="766766d2-fa56-49c2-9928-b50cbbf9173a"
Sep 10 17:36:26.859868 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:36:26.859829    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-2t2rd"
Sep 10 17:36:26.859957 ip-10-0-141-180 kubenswrapper[2581]: E0910 17:36:26.859922    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-b88bx" podUID="88d343e2-9756-438b-a19c-973b53948e68"
Sep 10 17:36:26.860047 ip-10-0-141-180 kubenswrapper[2581]: E0910 17:36:26.860011    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-2t2rd" podUID="dee7aebb-2ba6-4bfe-8fb4-371bcc4e56b1"
Sep 10 17:36:28.856179 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:36:28.856122    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-2t2rd"
Sep 10 17:36:28.856680 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:36:28.856140    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-7ptnb"
Sep 10 17:36:28.856680 ip-10-0-141-180 kubenswrapper[2581]: E0910 17:36:28.856272    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-2t2rd" podUID="dee7aebb-2ba6-4bfe-8fb4-371bcc4e56b1"
Sep 10 17:36:28.856680 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:36:28.856309    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-b88bx"
Sep 10 17:36:28.856680 ip-10-0-141-180 kubenswrapper[2581]: E0910 17:36:28.856422    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-7ptnb" podUID="766766d2-fa56-49c2-9928-b50cbbf9173a"
Sep 10 17:36:28.856680 ip-10-0-141-180 kubenswrapper[2581]: E0910 17:36:28.856509    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-b88bx" podUID="88d343e2-9756-438b-a19c-973b53948e68"
Sep 10 17:36:30.391990 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:36:30.391955    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-w28mp\" (UniqueName: \"kubernetes.io/projected/766766d2-fa56-49c2-9928-b50cbbf9173a-kube-api-access-w28mp\") pod \"network-check-target-7ptnb\" (UID: \"766766d2-fa56-49c2-9928-b50cbbf9173a\") " pod="openshift-network-diagnostics/network-check-target-7ptnb"
Sep 10 17:36:30.392529 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:36:30.392010    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/88d343e2-9756-438b-a19c-973b53948e68-metrics-certs\") pod \"network-metrics-daemon-b88bx\" (UID: \"88d343e2-9756-438b-a19c-973b53948e68\") " pod="openshift-multus/network-metrics-daemon-b88bx"
Sep 10 17:36:30.392529 ip-10-0-141-180 kubenswrapper[2581]: E0910 17:36:30.392131    2581 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 10 17:36:30.392529 ip-10-0-141-180 kubenswrapper[2581]: E0910 17:36:30.392186    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/88d343e2-9756-438b-a19c-973b53948e68-metrics-certs podName:88d343e2-9756-438b-a19c-973b53948e68 nodeName:}" failed. No retries permitted until 2026-09-10 17:36:38.392171319 +0000 UTC m=+18.046245201 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/88d343e2-9756-438b-a19c-973b53948e68-metrics-certs") pod "network-metrics-daemon-b88bx" (UID: "88d343e2-9756-438b-a19c-973b53948e68") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 10 17:36:30.392529 ip-10-0-141-180 kubenswrapper[2581]: E0910 17:36:30.392134    2581 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 10 17:36:30.392529 ip-10-0-141-180 kubenswrapper[2581]: E0910 17:36:30.392236    2581 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 10 17:36:30.392529 ip-10-0-141-180 kubenswrapper[2581]: E0910 17:36:30.392249    2581 projected.go:196] Error preparing data for projected volume kube-api-access-w28mp for pod openshift-network-diagnostics/network-check-target-7ptnb: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 10 17:36:30.392529 ip-10-0-141-180 kubenswrapper[2581]: E0910 17:36:30.392304    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/766766d2-fa56-49c2-9928-b50cbbf9173a-kube-api-access-w28mp podName:766766d2-fa56-49c2-9928-b50cbbf9173a nodeName:}" failed. No retries permitted until 2026-09-10 17:36:38.392289904 +0000 UTC m=+18.046363801 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-w28mp" (UniqueName: "kubernetes.io/projected/766766d2-fa56-49c2-9928-b50cbbf9173a-kube-api-access-w28mp") pod "network-check-target-7ptnb" (UID: "766766d2-fa56-49c2-9928-b50cbbf9173a") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 10 17:36:30.492733 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:36:30.492697    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/dee7aebb-2ba6-4bfe-8fb4-371bcc4e56b1-original-pull-secret\") pod \"global-pull-secret-syncer-2t2rd\" (UID: \"dee7aebb-2ba6-4bfe-8fb4-371bcc4e56b1\") " pod="kube-system/global-pull-secret-syncer-2t2rd"
Sep 10 17:36:30.492887 ip-10-0-141-180 kubenswrapper[2581]: E0910 17:36:30.492853    2581 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 10 17:36:30.492960 ip-10-0-141-180 kubenswrapper[2581]: E0910 17:36:30.492930    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/dee7aebb-2ba6-4bfe-8fb4-371bcc4e56b1-original-pull-secret podName:dee7aebb-2ba6-4bfe-8fb4-371bcc4e56b1 nodeName:}" failed. No retries permitted until 2026-09-10 17:36:38.492909116 +0000 UTC m=+18.146983013 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/dee7aebb-2ba6-4bfe-8fb4-371bcc4e56b1-original-pull-secret") pod "global-pull-secret-syncer-2t2rd" (UID: "dee7aebb-2ba6-4bfe-8fb4-371bcc4e56b1") : object "kube-system"/"original-pull-secret" not registered
Sep 10 17:36:30.857477 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:36:30.856693    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-b88bx"
Sep 10 17:36:30.857477 ip-10-0-141-180 kubenswrapper[2581]: E0910 17:36:30.856825    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-b88bx" podUID="88d343e2-9756-438b-a19c-973b53948e68"
Sep 10 17:36:30.857477 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:36:30.857190    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-2t2rd"
Sep 10 17:36:30.857477 ip-10-0-141-180 kubenswrapper[2581]: E0910 17:36:30.857281    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-2t2rd" podUID="dee7aebb-2ba6-4bfe-8fb4-371bcc4e56b1"
Sep 10 17:36:30.857477 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:36:30.857323    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-7ptnb"
Sep 10 17:36:30.857477 ip-10-0-141-180 kubenswrapper[2581]: E0910 17:36:30.857387    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-7ptnb" podUID="766766d2-fa56-49c2-9928-b50cbbf9173a"
Sep 10 17:36:32.856305 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:36:32.856270    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-7ptnb"
Sep 10 17:36:32.856901 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:36:32.856308    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-b88bx"
Sep 10 17:36:32.856901 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:36:32.856271    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-2t2rd"
Sep 10 17:36:32.856901 ip-10-0-141-180 kubenswrapper[2581]: E0910 17:36:32.856397    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-7ptnb" podUID="766766d2-fa56-49c2-9928-b50cbbf9173a"
Sep 10 17:36:32.856901 ip-10-0-141-180 kubenswrapper[2581]: E0910 17:36:32.856478    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-2t2rd" podUID="dee7aebb-2ba6-4bfe-8fb4-371bcc4e56b1"
Sep 10 17:36:32.856901 ip-10-0-141-180 kubenswrapper[2581]: E0910 17:36:32.856547    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-b88bx" podUID="88d343e2-9756-438b-a19c-973b53948e68"
Sep 10 17:36:34.855745 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:36:34.855711    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-b88bx"
Sep 10 17:36:34.856204 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:36:34.855711    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-7ptnb"
Sep 10 17:36:34.856204 ip-10-0-141-180 kubenswrapper[2581]: E0910 17:36:34.855853    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-b88bx" podUID="88d343e2-9756-438b-a19c-973b53948e68"
Sep 10 17:36:34.856204 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:36:34.855883    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-2t2rd"
Sep 10 17:36:34.856204 ip-10-0-141-180 kubenswrapper[2581]: E0910 17:36:34.855987    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-7ptnb" podUID="766766d2-fa56-49c2-9928-b50cbbf9173a"
Sep 10 17:36:34.856204 ip-10-0-141-180 kubenswrapper[2581]: E0910 17:36:34.856073    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-2t2rd" podUID="dee7aebb-2ba6-4bfe-8fb4-371bcc4e56b1"
Sep 10 17:36:36.856027 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:36:36.855988    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-b88bx"
Sep 10 17:36:36.856027 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:36:36.856013    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-7ptnb"
Sep 10 17:36:36.856529 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:36:36.856075    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-2t2rd"
Sep 10 17:36:36.856529 ip-10-0-141-180 kubenswrapper[2581]: E0910 17:36:36.856188    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-2t2rd" podUID="dee7aebb-2ba6-4bfe-8fb4-371bcc4e56b1"
Sep 10 17:36:36.856529 ip-10-0-141-180 kubenswrapper[2581]: E0910 17:36:36.856254    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-7ptnb" podUID="766766d2-fa56-49c2-9928-b50cbbf9173a"
Sep 10 17:36:36.856529 ip-10-0-141-180 kubenswrapper[2581]: E0910 17:36:36.856334    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-b88bx" podUID="88d343e2-9756-438b-a19c-973b53948e68"
Sep 10 17:36:38.453879 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:36:38.453842    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-w28mp\" (UniqueName: \"kubernetes.io/projected/766766d2-fa56-49c2-9928-b50cbbf9173a-kube-api-access-w28mp\") pod \"network-check-target-7ptnb\" (UID: \"766766d2-fa56-49c2-9928-b50cbbf9173a\") " pod="openshift-network-diagnostics/network-check-target-7ptnb"
Sep 10 17:36:38.454511 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:36:38.453897    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/88d343e2-9756-438b-a19c-973b53948e68-metrics-certs\") pod \"network-metrics-daemon-b88bx\" (UID: \"88d343e2-9756-438b-a19c-973b53948e68\") " pod="openshift-multus/network-metrics-daemon-b88bx"
Sep 10 17:36:38.454511 ip-10-0-141-180 kubenswrapper[2581]: E0910 17:36:38.454022    2581 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 10 17:36:38.454511 ip-10-0-141-180 kubenswrapper[2581]: E0910 17:36:38.454080    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/88d343e2-9756-438b-a19c-973b53948e68-metrics-certs podName:88d343e2-9756-438b-a19c-973b53948e68 nodeName:}" failed. No retries permitted until 2026-09-10 17:36:54.45406625 +0000 UTC m=+34.108140131 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/88d343e2-9756-438b-a19c-973b53948e68-metrics-certs") pod "network-metrics-daemon-b88bx" (UID: "88d343e2-9756-438b-a19c-973b53948e68") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 10 17:36:38.454511 ip-10-0-141-180 kubenswrapper[2581]: E0910 17:36:38.454025    2581 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 10 17:36:38.454511 ip-10-0-141-180 kubenswrapper[2581]: E0910 17:36:38.454129    2581 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 10 17:36:38.454511 ip-10-0-141-180 kubenswrapper[2581]: E0910 17:36:38.454143    2581 projected.go:196] Error preparing data for projected volume kube-api-access-w28mp for pod openshift-network-diagnostics/network-check-target-7ptnb: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 10 17:36:38.454511 ip-10-0-141-180 kubenswrapper[2581]: E0910 17:36:38.454206    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/766766d2-fa56-49c2-9928-b50cbbf9173a-kube-api-access-w28mp podName:766766d2-fa56-49c2-9928-b50cbbf9173a nodeName:}" failed. No retries permitted until 2026-09-10 17:36:54.454194528 +0000 UTC m=+34.108268433 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-w28mp" (UniqueName: "kubernetes.io/projected/766766d2-fa56-49c2-9928-b50cbbf9173a-kube-api-access-w28mp") pod "network-check-target-7ptnb" (UID: "766766d2-fa56-49c2-9928-b50cbbf9173a") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 10 17:36:38.555144 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:36:38.555108    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/dee7aebb-2ba6-4bfe-8fb4-371bcc4e56b1-original-pull-secret\") pod \"global-pull-secret-syncer-2t2rd\" (UID: \"dee7aebb-2ba6-4bfe-8fb4-371bcc4e56b1\") " pod="kube-system/global-pull-secret-syncer-2t2rd"
Sep 10 17:36:38.555325 ip-10-0-141-180 kubenswrapper[2581]: E0910 17:36:38.555268    2581 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 10 17:36:38.555384 ip-10-0-141-180 kubenswrapper[2581]: E0910 17:36:38.555363    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/dee7aebb-2ba6-4bfe-8fb4-371bcc4e56b1-original-pull-secret podName:dee7aebb-2ba6-4bfe-8fb4-371bcc4e56b1 nodeName:}" failed. No retries permitted until 2026-09-10 17:36:54.555341988 +0000 UTC m=+34.209415877 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/dee7aebb-2ba6-4bfe-8fb4-371bcc4e56b1-original-pull-secret") pod "global-pull-secret-syncer-2t2rd" (UID: "dee7aebb-2ba6-4bfe-8fb4-371bcc4e56b1") : object "kube-system"/"original-pull-secret" not registered
Sep 10 17:36:38.856207 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:36:38.856128    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-2t2rd"
Sep 10 17:36:38.856367 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:36:38.856128    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-7ptnb"
Sep 10 17:36:38.856367 ip-10-0-141-180 kubenswrapper[2581]: E0910 17:36:38.856235    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-2t2rd" podUID="dee7aebb-2ba6-4bfe-8fb4-371bcc4e56b1"
Sep 10 17:36:38.856367 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:36:38.856140    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-b88bx"
Sep 10 17:36:38.856524 ip-10-0-141-180 kubenswrapper[2581]: E0910 17:36:38.856355    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-7ptnb" podUID="766766d2-fa56-49c2-9928-b50cbbf9173a"
Sep 10 17:36:38.856524 ip-10-0-141-180 kubenswrapper[2581]: E0910 17:36:38.856488    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-b88bx" podUID="88d343e2-9756-438b-a19c-973b53948e68"
Sep 10 17:36:40.857171 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:36:40.857142    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-2t2rd"
Sep 10 17:36:40.857402 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:36:40.857243    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-b88bx"
Sep 10 17:36:40.857402 ip-10-0-141-180 kubenswrapper[2581]: E0910 17:36:40.857266    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-2t2rd" podUID="dee7aebb-2ba6-4bfe-8fb4-371bcc4e56b1"
Sep 10 17:36:40.857402 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:36:40.857331    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-7ptnb"
Sep 10 17:36:40.857488 ip-10-0-141-180 kubenswrapper[2581]: E0910 17:36:40.857327    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-b88bx" podUID="88d343e2-9756-438b-a19c-973b53948e68"
Sep 10 17:36:40.857488 ip-10-0-141-180 kubenswrapper[2581]: E0910 17:36:40.857412    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-7ptnb" podUID="766766d2-fa56-49c2-9928-b50cbbf9173a"
Sep 10 17:36:41.967928 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:36:41.967673    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-7nn7k" event={"ID":"81e433e1-460f-4d44-88dc-4ab3666e140e","Type":"ContainerStarted","Data":"91afa771d4dd5784ce3901102226ee4ef91777ab95ad54c72aeb11dda1528da2"}
Sep 10 17:36:41.968291 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:36:41.968010    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-7nn7k" event={"ID":"81e433e1-460f-4d44-88dc-4ab3666e140e","Type":"ContainerStarted","Data":"25df2f5444adf844104dd7ed825d0c65272f63d7deb62d0c180fd7304c07ecae"}
Sep 10 17:36:41.968291 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:36:41.968052    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-7nn7k" event={"ID":"81e433e1-460f-4d44-88dc-4ab3666e140e","Type":"ContainerStarted","Data":"787d1159557fc4228b131253bb26d565d9b4c5cc3f2f764ccc67af3dd58f2381"}
Sep 10 17:36:41.968291 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:36:41.968088    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-7nn7k" event={"ID":"81e433e1-460f-4d44-88dc-4ab3666e140e","Type":"ContainerStarted","Data":"2a663cd3f63ad669493ec6b69d9fb9ca8803d7d953c236ba5e9d61813584cd97"}
Sep 10 17:36:41.968291 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:36:41.968118    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-7nn7k" event={"ID":"81e433e1-460f-4d44-88dc-4ab3666e140e","Type":"ContainerStarted","Data":"a8be2e79fb852b360b6a9d8337168310fa6c0b78e60d6315f53c3f9b96682a9d"}
Sep 10 17:36:41.968291 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:36:41.968157    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-7nn7k" event={"ID":"81e433e1-460f-4d44-88dc-4ab3666e140e","Type":"ContainerStarted","Data":"a634ec785fa79454fb42e7ff976378118a10b8ac0cf72ec38c00e5291ba02050"}
Sep 10 17:36:41.969247 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:36:41.969219    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-h6rpt" event={"ID":"22cfca39-1a39-4266-9ee7-25dfc8723196","Type":"ContainerStarted","Data":"1b999314901414e69451d1d67e83aa02e943287b6434eb69b732c607d05c0f2c"}
Sep 10 17:36:41.970711 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:36:41.970683    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-7p25j" event={"ID":"459a65ca-0e07-47e2-aba9-6af10bb5387f","Type":"ContainerStarted","Data":"e0c58beeaafa67970c1a763e1e54daec1cf413cb01f013ed104602478e2b032a"}
Sep 10 17:36:41.972400 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:36:41.972378    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-z5bpf" event={"ID":"b1aa3439-0cd7-4496-b1e8-7b535c305002","Type":"ContainerStarted","Data":"1752cc5347500859c0a9acf0ffd017b5dcda623ca2a1726511edda4995daba72"}
Sep 10 17:36:41.974353 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:36:41.974310    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-jkplw" event={"ID":"84e91010-05f8-4b38-801c-8e462cd4e38d","Type":"ContainerStarted","Data":"430e9aa260de1f6b821ac7add1992de2b2c28dcc96c009eb6583e65d19a0a158"}
Sep 10 17:36:41.976054 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:36:41.976027    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-s8k6d" event={"ID":"356f6437-710e-4991-b8db-e96a35f1152d","Type":"ContainerStarted","Data":"b852370b063025be6eafd06ce5c68f48fe46c86e700c38da497f265d0b64dffc"}
Sep 10 17:36:41.977672 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:36:41.977627    2581 generic.go:356] "Generic (PLEG): container finished" podID="099b7fa9-6908-43e3-b444-73b7f327290d" containerID="760216c1327adad7bc90d547926e7bfcff656beddbbd37c35bafcf86e814f1fc" exitCode=0
Sep 10 17:36:41.977766 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:36:41.977672    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-bqj9g" event={"ID":"099b7fa9-6908-43e3-b444-73b7f327290d","Type":"ContainerDied","Data":"760216c1327adad7bc90d547926e7bfcff656beddbbd37c35bafcf86e814f1fc"}
Sep 10 17:36:41.979361 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:36:41.979344    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-w9ldt" event={"ID":"b4e9eaae-6a8f-459d-9953-f9b6a529bef4","Type":"ContainerStarted","Data":"bc000f58c94723600a2cd0eadfaedc1f1f8fcd859d95e80ebd9086828d962b6b"}
Sep 10 17:36:41.983873 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:36:41.983835    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-h6rpt" podStartSLOduration=4.793804865 podStartE2EDuration="21.983823217s" totalImagesPullingTime="17.190018352s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-10 17:36:20 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 17:36:41.983259769 +0000 UTC m=+21.637333674" watchObservedRunningTime="2026-09-10 17:36:41.983823217 +0000 UTC m=+21.637897151"
Sep 10 17:36:41.983976 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:36:41.983915    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-180.ec2.internal" podStartSLOduration=19.983908186 podStartE2EDuration="19.983908186s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-10 17:36:22 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 17:36:25.947099096 +0000 UTC m=+5.601173025" watchObservedRunningTime="2026-09-10 17:36:41.983908186 +0000 UTC m=+21.637982091"
Sep 10 17:36:42.001153 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:36:42.001105    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-s8k6d" podStartSLOduration=4.531293992 podStartE2EDuration="22.00109341s" totalImagesPullingTime="17.469799418s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-10 17:36:20 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 17:36:42.0008555 +0000 UTC m=+21.654929428" watchObservedRunningTime="2026-09-10 17:36:42.00109341 +0000 UTC m=+21.655167342"
Sep 10 17:36:42.020387 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:36:42.020332    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-jkplw" podStartSLOduration=4.558152358 podStartE2EDuration="22.020311509s" totalImagesPullingTime="17.462159151s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-10 17:36:20 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 17:36:42.016069553 +0000 UTC m=+21.670143476" watchObservedRunningTime="2026-09-10 17:36:42.020311509 +0000 UTC m=+21.674385432"
Sep 10 17:36:42.033767 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:36:42.033724    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-z5bpf" podStartSLOduration=3.532648146 podStartE2EDuration="21.033709966s" totalImagesPullingTime="17.50106182s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-10 17:36:21 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 17:36:42.033463096 +0000 UTC m=+21.687537019" watchObservedRunningTime="2026-09-10 17:36:42.033709966 +0000 UTC m=+21.687783887"
Sep 10 17:36:42.069304 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:36:42.069259    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-7p25j" podStartSLOduration=4.881109799 podStartE2EDuration="22.069244498s" totalImagesPullingTime="17.188134699s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 17:36:20 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 17:36:42.069026382 +0000 UTC m=+21.723100311" watchObservedRunningTime="2026-09-10 17:36:42.069244498 +0000 UTC m=+21.723318402"
Sep 10 17:36:42.109571 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:36:42.109537    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7p25j_459a65ca-0e07-47e2-aba9-6af10bb5387f/konnectivity-agent/0.log" limit=1024
Sep 10 17:36:42.172077 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:36:42.172020    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7p25j_459a65ca-0e07-47e2-aba9-6af10bb5387f/konnectivity-agent/0.log" limit=1024
Sep 10 17:36:42.227256 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:36:42.227226    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7p25j_459a65ca-0e07-47e2-aba9-6af10bb5387f/konnectivity-agent/0.log" limit=1024
Sep 10 17:36:42.406422 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:36:42.406395    2581 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock"
Sep 10 17:36:42.775602 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:36:42.775501    2581 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-09-10T17:36:42.40641569Z","UUID":"aabf49b6-ded9-4114-a13a-def6ea2cbb55","Handler":null,"Name":"","Endpoint":""}
Sep 10 17:36:42.777043 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:36:42.777023    2581 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0
Sep 10 17:36:42.777174 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:36:42.777056    2581 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock
Sep 10 17:36:42.782792 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:36:42.782772    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-7p25j"
Sep 10 17:36:42.783316 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:36:42.783296    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-7p25j"
Sep 10 17:36:42.809922 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:36:42.809896    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7p25j_459a65ca-0e07-47e2-aba9-6af10bb5387f/konnectivity-agent/0.log" limit=1024
Sep 10 17:36:42.855288 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:36:42.855260    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-7ptnb"
Sep 10 17:36:42.855423 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:36:42.855268    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-b88bx"
Sep 10 17:36:42.855423 ip-10-0-141-180 kubenswrapper[2581]: E0910 17:36:42.855366    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-7ptnb" podUID="766766d2-fa56-49c2-9928-b50cbbf9173a"
Sep 10 17:36:42.855423 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:36:42.855393    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-2t2rd"
Sep 10 17:36:42.855583 ip-10-0-141-180 kubenswrapper[2581]: E0910 17:36:42.855455    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-b88bx" podUID="88d343e2-9756-438b-a19c-973b53948e68"
Sep 10 17:36:42.855583 ip-10-0-141-180 kubenswrapper[2581]: E0910 17:36:42.855504    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-2t2rd" podUID="dee7aebb-2ba6-4bfe-8fb4-371bcc4e56b1"
Sep 10 17:36:42.877134 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:36:42.877112    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7p25j_459a65ca-0e07-47e2-aba9-6af10bb5387f/konnectivity-agent/0.log" limit=1024
Sep 10 17:36:42.961667 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:36:42.961622    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7p25j_459a65ca-0e07-47e2-aba9-6af10bb5387f/konnectivity-agent/0.log" limit=1024
Sep 10 17:36:42.983775 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:36:42.983741    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-tlnqf" event={"ID":"91b2593e-0a42-4735-bbd6-616678613892","Type":"ContainerStarted","Data":"795bd9efbf83de24cc8a4d2516970c969e659c2b1364416a90e95f9fbc2bab30"}
Sep 10 17:36:42.986100 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:36:42.986033    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-w9ldt" event={"ID":"b4e9eaae-6a8f-459d-9953-f9b6a529bef4","Type":"ContainerStarted","Data":"c52adadc10bd5b1a02a5e34cdb6c7df22684821348245b0a0228eb22a25531eb"}
Sep 10 17:36:43.000648 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:36:43.000585    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-tlnqf" podStartSLOduration=5.804480591 podStartE2EDuration="23.000570866s" totalImagesPullingTime="17.196090275s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-10 17:36:20 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 17:36:43.000357934 +0000 UTC m=+22.654431853" watchObservedRunningTime="2026-09-10 17:36:43.000570866 +0000 UTC m=+22.654644781"
Sep 10 17:36:43.133250 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:36:43.133181    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7p25j_459a65ca-0e07-47e2-aba9-6af10bb5387f/konnectivity-agent/0.log" limit=1024
Sep 10 17:36:43.199431 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:36:43.199397    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7p25j_459a65ca-0e07-47e2-aba9-6af10bb5387f/konnectivity-agent/0.log" limit=1024
Sep 10 17:36:43.518404 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:36:43.518374    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7p25j_459a65ca-0e07-47e2-aba9-6af10bb5387f/konnectivity-agent/0.log" limit=1024
Sep 10 17:36:43.766712 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:36:43.766676    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7p25j_459a65ca-0e07-47e2-aba9-6af10bb5387f/konnectivity-agent/0.log" limit=1024
Sep 10 17:36:43.828805 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:36:43.828727    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7p25j_459a65ca-0e07-47e2-aba9-6af10bb5387f/konnectivity-agent/0.log" limit=1024
Sep 10 17:36:43.991350 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:36:43.991319    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-w9ldt" event={"ID":"b4e9eaae-6a8f-459d-9953-f9b6a529bef4","Type":"ContainerStarted","Data":"20a0e3f24997335536c7f34a642716de6f29992dd111a02b7b5ff21f520c8d7d"}
Sep 10 17:36:43.991864 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:36:43.991334    2581 prober_manager.go:312] "Failed to trigger a manual run" probe="Readiness"
Sep 10 17:36:44.011105 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:36:44.011066    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-w9ldt" podStartSLOduration=4.737405961 podStartE2EDuration="24.01105207s" totalImagesPullingTime="19.273646109s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-10 17:36:20 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 17:36:44.010677156 +0000 UTC m=+23.664751081" watchObservedRunningTime="2026-09-10 17:36:44.01105207 +0000 UTC m=+23.665125973"
Sep 10 17:36:44.227054 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:36:44.227013    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7p25j_459a65ca-0e07-47e2-aba9-6af10bb5387f/konnectivity-agent/0.log" limit=1024
Sep 10 17:36:44.327523 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:36:44.327495    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7p25j_459a65ca-0e07-47e2-aba9-6af10bb5387f/konnectivity-agent/0.log" limit=1024
Sep 10 17:36:44.855723 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:36:44.855691    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-7ptnb"
Sep 10 17:36:44.855888 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:36:44.855691    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-2t2rd"
Sep 10 17:36:44.855888 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:36:44.855821    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-b88bx"
Sep 10 17:36:44.855888 ip-10-0-141-180 kubenswrapper[2581]: E0910 17:36:44.855818    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-7ptnb" podUID="766766d2-fa56-49c2-9928-b50cbbf9173a"
Sep 10 17:36:44.856021 ip-10-0-141-180 kubenswrapper[2581]: E0910 17:36:44.855926    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-2t2rd" podUID="dee7aebb-2ba6-4bfe-8fb4-371bcc4e56b1"
Sep 10 17:36:44.856074 ip-10-0-141-180 kubenswrapper[2581]: E0910 17:36:44.856022    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-b88bx" podUID="88d343e2-9756-438b-a19c-973b53948e68"
Sep 10 17:36:44.996487 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:36:44.996453    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-7nn7k" event={"ID":"81e433e1-460f-4d44-88dc-4ab3666e140e","Type":"ContainerStarted","Data":"b3df443ee30600604bcc84622b94fe270ca0dd4dc313142fb7201f6f69bf77e7"}
Sep 10 17:36:45.074839 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:36:45.074808    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7p25j_459a65ca-0e07-47e2-aba9-6af10bb5387f/konnectivity-agent/0.log" limit=1024
Sep 10 17:36:45.175346 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:36:45.175263    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7p25j_459a65ca-0e07-47e2-aba9-6af10bb5387f/konnectivity-agent/0.log" limit=1024
Sep 10 17:36:45.543332 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:36:45.543307    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-jkplw_84e91010-05f8-4b38-801c-8e462cd4e38d/dns-node-resolver/0.log"
Sep 10 17:36:45.927039 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:36:45.926963    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7p25j_459a65ca-0e07-47e2-aba9-6af10bb5387f/konnectivity-agent/0.log" limit=1024
Sep 10 17:36:46.027218 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:36:46.027180    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7p25j_459a65ca-0e07-47e2-aba9-6af10bb5387f/konnectivity-agent/0.log" limit=1024
Sep 10 17:36:46.127951 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:36:46.127917    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7p25j_459a65ca-0e07-47e2-aba9-6af10bb5387f/konnectivity-agent/0.log" limit=1024
Sep 10 17:36:46.743612 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:36:46.743580    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-h6rpt_22cfca39-1a39-4266-9ee7-25dfc8723196/node-ca/0.log"
Sep 10 17:36:46.856151 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:36:46.856119    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-2t2rd"
Sep 10 17:36:46.856151 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:36:46.856144    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-b88bx"
Sep 10 17:36:46.856364 ip-10-0-141-180 kubenswrapper[2581]: E0910 17:36:46.856271    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-2t2rd" podUID="dee7aebb-2ba6-4bfe-8fb4-371bcc4e56b1"
Sep 10 17:36:46.856364 ip-10-0-141-180 kubenswrapper[2581]: E0910 17:36:46.856348    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-b88bx" podUID="88d343e2-9756-438b-a19c-973b53948e68"
Sep 10 17:36:46.856452 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:36:46.856396    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-7ptnb"
Sep 10 17:36:46.856491 ip-10-0-141-180 kubenswrapper[2581]: E0910 17:36:46.856451    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-7ptnb" podUID="766766d2-fa56-49c2-9928-b50cbbf9173a"
Sep 10 17:36:48.007945 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:36:48.007809    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-7nn7k" event={"ID":"81e433e1-460f-4d44-88dc-4ab3666e140e","Type":"ContainerStarted","Data":"87c9bbe503cc622030721f237b8321b82f791142f9963d4b7e736e3b44641010"}
Sep 10 17:36:48.008369 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:36:48.008147    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-7nn7k"
Sep 10 17:36:48.008369 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:36:48.008264    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-7nn7k"
Sep 10 17:36:48.008369 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:36:48.008304    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-7nn7k"
Sep 10 17:36:48.023447 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:36:48.023426    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-7nn7k"
Sep 10 17:36:48.023626 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:36:48.023612    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-7nn7k"
Sep 10 17:36:48.040501 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:36:48.040458    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-7nn7k" podStartSLOduration=10.110608076 podStartE2EDuration="28.040446845s" totalImagesPullingTime="17.929838769s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-10 17:36:20 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 17:36:48.040293053 +0000 UTC m=+27.694366968" watchObservedRunningTime="2026-09-10 17:36:48.040446845 +0000 UTC m=+27.694520748"
Sep 10 17:36:48.855659 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:36:48.855618    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-b88bx"
Sep 10 17:36:48.855939 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:36:48.855620    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-7ptnb"
Sep 10 17:36:48.855939 ip-10-0-141-180 kubenswrapper[2581]: E0910 17:36:48.855755    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-b88bx" podUID="88d343e2-9756-438b-a19c-973b53948e68"
Sep 10 17:36:48.855939 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:36:48.855768    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-2t2rd"
Sep 10 17:36:48.855939 ip-10-0-141-180 kubenswrapper[2581]: E0910 17:36:48.855847    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-7ptnb" podUID="766766d2-fa56-49c2-9928-b50cbbf9173a"
Sep 10 17:36:48.855939 ip-10-0-141-180 kubenswrapper[2581]: E0910 17:36:48.855904    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-2t2rd" podUID="dee7aebb-2ba6-4bfe-8fb4-371bcc4e56b1"
Sep 10 17:36:49.012057 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:36:49.012028    2581 generic.go:356] "Generic (PLEG): container finished" podID="099b7fa9-6908-43e3-b444-73b7f327290d" containerID="91e88155e07a8cdc69cfbd8fb50798273987ee1c54b712677a220b94db6cf4ff" exitCode=0
Sep 10 17:36:49.012418 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:36:49.012119    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-bqj9g" event={"ID":"099b7fa9-6908-43e3-b444-73b7f327290d","Type":"ContainerDied","Data":"91e88155e07a8cdc69cfbd8fb50798273987ee1c54b712677a220b94db6cf4ff"}
Sep 10 17:36:49.913039 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:36:49.913012    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-7p25j"
Sep 10 17:36:49.913201 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:36:49.913116    2581 prober_manager.go:312] "Failed to trigger a manual run" probe="Readiness"
Sep 10 17:36:49.913668 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:36:49.913629    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-7p25j"
Sep 10 17:36:50.856361 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:36:50.856330    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-2t2rd"
Sep 10 17:36:50.856740 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:36:50.856387    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-b88bx"
Sep 10 17:36:50.856740 ip-10-0-141-180 kubenswrapper[2581]: E0910 17:36:50.856458    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-b88bx" podUID="88d343e2-9756-438b-a19c-973b53948e68"
Sep 10 17:36:50.856740 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:36:50.856488    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-7ptnb"
Sep 10 17:36:50.856740 ip-10-0-141-180 kubenswrapper[2581]: E0910 17:36:50.856571    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-2t2rd" podUID="dee7aebb-2ba6-4bfe-8fb4-371bcc4e56b1"
Sep 10 17:36:50.856740 ip-10-0-141-180 kubenswrapper[2581]: E0910 17:36:50.856680    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-7ptnb" podUID="766766d2-fa56-49c2-9928-b50cbbf9173a"
Sep 10 17:36:51.017583 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:36:51.017552    2581 generic.go:356] "Generic (PLEG): container finished" podID="099b7fa9-6908-43e3-b444-73b7f327290d" containerID="2cecd7fe03414c932a77da9253d6edb42740c6f7e3120024fa4532eee0f592e2" exitCode=0
Sep 10 17:36:51.017739 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:36:51.017611    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-bqj9g" event={"ID":"099b7fa9-6908-43e3-b444-73b7f327290d","Type":"ContainerDied","Data":"2cecd7fe03414c932a77da9253d6edb42740c6f7e3120024fa4532eee0f592e2"}
Sep 10 17:36:52.855426 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:36:52.855398    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-b88bx"
Sep 10 17:36:52.855426 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:36:52.855414    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-2t2rd"
Sep 10 17:36:52.855901 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:36:52.855418    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-7ptnb"
Sep 10 17:36:52.855901 ip-10-0-141-180 kubenswrapper[2581]: E0910 17:36:52.855491    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-b88bx" podUID="88d343e2-9756-438b-a19c-973b53948e68"
Sep 10 17:36:52.855901 ip-10-0-141-180 kubenswrapper[2581]: E0910 17:36:52.855566    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-2t2rd" podUID="dee7aebb-2ba6-4bfe-8fb4-371bcc4e56b1"
Sep 10 17:36:52.855901 ip-10-0-141-180 kubenswrapper[2581]: E0910 17:36:52.855651    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-7ptnb" podUID="766766d2-fa56-49c2-9928-b50cbbf9173a"
Sep 10 17:36:53.023930 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:36:53.023897    2581 generic.go:356] "Generic (PLEG): container finished" podID="099b7fa9-6908-43e3-b444-73b7f327290d" containerID="dada9c399f57b3a423b5fd4bdfa633fed36f2fa1dc8e99d1eb1454f33b7c0ac6" exitCode=0
Sep 10 17:36:53.024086 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:36:53.023940    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-bqj9g" event={"ID":"099b7fa9-6908-43e3-b444-73b7f327290d","Type":"ContainerDied","Data":"dada9c399f57b3a423b5fd4bdfa633fed36f2fa1dc8e99d1eb1454f33b7c0ac6"}
Sep 10 17:36:54.473045 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:36:54.473017    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-w28mp\" (UniqueName: \"kubernetes.io/projected/766766d2-fa56-49c2-9928-b50cbbf9173a-kube-api-access-w28mp\") pod \"network-check-target-7ptnb\" (UID: \"766766d2-fa56-49c2-9928-b50cbbf9173a\") " pod="openshift-network-diagnostics/network-check-target-7ptnb"
Sep 10 17:36:54.473439 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:36:54.473055    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/88d343e2-9756-438b-a19c-973b53948e68-metrics-certs\") pod \"network-metrics-daemon-b88bx\" (UID: \"88d343e2-9756-438b-a19c-973b53948e68\") " pod="openshift-multus/network-metrics-daemon-b88bx"
Sep 10 17:36:54.473439 ip-10-0-141-180 kubenswrapper[2581]: E0910 17:36:54.473147    2581 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 10 17:36:54.473439 ip-10-0-141-180 kubenswrapper[2581]: E0910 17:36:54.473160    2581 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 10 17:36:54.473439 ip-10-0-141-180 kubenswrapper[2581]: E0910 17:36:54.473178    2581 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 10 17:36:54.473439 ip-10-0-141-180 kubenswrapper[2581]: E0910 17:36:54.473187    2581 projected.go:196] Error preparing data for projected volume kube-api-access-w28mp for pod openshift-network-diagnostics/network-check-target-7ptnb: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 10 17:36:54.473439 ip-10-0-141-180 kubenswrapper[2581]: E0910 17:36:54.473191    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/88d343e2-9756-438b-a19c-973b53948e68-metrics-certs podName:88d343e2-9756-438b-a19c-973b53948e68 nodeName:}" failed. No retries permitted until 2026-09-10 17:37:26.473178866 +0000 UTC m=+66.127252748 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/88d343e2-9756-438b-a19c-973b53948e68-metrics-certs") pod "network-metrics-daemon-b88bx" (UID: "88d343e2-9756-438b-a19c-973b53948e68") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 10 17:36:54.473439 ip-10-0-141-180 kubenswrapper[2581]: E0910 17:36:54.473226    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/766766d2-fa56-49c2-9928-b50cbbf9173a-kube-api-access-w28mp podName:766766d2-fa56-49c2-9928-b50cbbf9173a nodeName:}" failed. No retries permitted until 2026-09-10 17:37:26.473213334 +0000 UTC m=+66.127287216 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-w28mp" (UniqueName: "kubernetes.io/projected/766766d2-fa56-49c2-9928-b50cbbf9173a-kube-api-access-w28mp") pod "network-check-target-7ptnb" (UID: "766766d2-fa56-49c2-9928-b50cbbf9173a") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 10 17:36:54.574279 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:36:54.574247    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/dee7aebb-2ba6-4bfe-8fb4-371bcc4e56b1-original-pull-secret\") pod \"global-pull-secret-syncer-2t2rd\" (UID: \"dee7aebb-2ba6-4bfe-8fb4-371bcc4e56b1\") " pod="kube-system/global-pull-secret-syncer-2t2rd"
Sep 10 17:36:54.574467 ip-10-0-141-180 kubenswrapper[2581]: E0910 17:36:54.574380    2581 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 10 17:36:54.574467 ip-10-0-141-180 kubenswrapper[2581]: E0910 17:36:54.574445    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/dee7aebb-2ba6-4bfe-8fb4-371bcc4e56b1-original-pull-secret podName:dee7aebb-2ba6-4bfe-8fb4-371bcc4e56b1 nodeName:}" failed. No retries permitted until 2026-09-10 17:37:26.57442725 +0000 UTC m=+66.228501145 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/dee7aebb-2ba6-4bfe-8fb4-371bcc4e56b1-original-pull-secret") pod "global-pull-secret-syncer-2t2rd" (UID: "dee7aebb-2ba6-4bfe-8fb4-371bcc4e56b1") : object "kube-system"/"original-pull-secret" not registered
Sep 10 17:36:54.855607 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:36:54.855530    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-7ptnb"
Sep 10 17:36:54.855771 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:36:54.855530    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-b88bx"
Sep 10 17:36:54.855771 ip-10-0-141-180 kubenswrapper[2581]: E0910 17:36:54.855672    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-7ptnb" podUID="766766d2-fa56-49c2-9928-b50cbbf9173a"
Sep 10 17:36:54.855895 ip-10-0-141-180 kubenswrapper[2581]: E0910 17:36:54.855780    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-b88bx" podUID="88d343e2-9756-438b-a19c-973b53948e68"
Sep 10 17:36:54.855895 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:36:54.855529    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-2t2rd"
Sep 10 17:36:54.856001 ip-10-0-141-180 kubenswrapper[2581]: E0910 17:36:54.855916    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-2t2rd" podUID="dee7aebb-2ba6-4bfe-8fb4-371bcc4e56b1"
Sep 10 17:36:56.855789 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:36:56.855750    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-2t2rd"
Sep 10 17:36:56.856352 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:36:56.855752    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-7ptnb"
Sep 10 17:36:56.856352 ip-10-0-141-180 kubenswrapper[2581]: E0910 17:36:56.855892    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-2t2rd" podUID="dee7aebb-2ba6-4bfe-8fb4-371bcc4e56b1"
Sep 10 17:36:56.856352 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:36:56.855763    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-b88bx"
Sep 10 17:36:56.856352 ip-10-0-141-180 kubenswrapper[2581]: E0910 17:36:56.855979    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-7ptnb" podUID="766766d2-fa56-49c2-9928-b50cbbf9173a"
Sep 10 17:36:56.856352 ip-10-0-141-180 kubenswrapper[2581]: E0910 17:36:56.856075    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-b88bx" podUID="88d343e2-9756-438b-a19c-973b53948e68"
Sep 10 17:36:58.855986 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:36:58.855956    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-7ptnb"
Sep 10 17:36:58.856336 ip-10-0-141-180 kubenswrapper[2581]: E0910 17:36:58.856036    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-7ptnb" podUID="766766d2-fa56-49c2-9928-b50cbbf9173a"
Sep 10 17:36:58.856336 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:36:58.855958    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-2t2rd"
Sep 10 17:36:58.856336 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:36:58.856066    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-b88bx"
Sep 10 17:36:58.856336 ip-10-0-141-180 kubenswrapper[2581]: E0910 17:36:58.856150    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-2t2rd" podUID="dee7aebb-2ba6-4bfe-8fb4-371bcc4e56b1"
Sep 10 17:36:58.856336 ip-10-0-141-180 kubenswrapper[2581]: E0910 17:36:58.856205    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-b88bx" podUID="88d343e2-9756-438b-a19c-973b53948e68"
Sep 10 17:36:59.039400 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:36:59.039371    2581 generic.go:356] "Generic (PLEG): container finished" podID="099b7fa9-6908-43e3-b444-73b7f327290d" containerID="b3c0dbd5ca0befbebe6cac28857e6f27702823e99a765da23106422b9b64b852" exitCode=0
Sep 10 17:36:59.039400 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:36:59.039403    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-bqj9g" event={"ID":"099b7fa9-6908-43e3-b444-73b7f327290d","Type":"ContainerDied","Data":"b3c0dbd5ca0befbebe6cac28857e6f27702823e99a765da23106422b9b64b852"}
Sep 10 17:37:00.044502 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:37:00.044470    2581 generic.go:356] "Generic (PLEG): container finished" podID="099b7fa9-6908-43e3-b444-73b7f327290d" containerID="dd7b57b6c7671c60525066e915121144dfde01a7afd925da54c338eb053d33b0" exitCode=0
Sep 10 17:37:00.044951 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:37:00.044527    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-bqj9g" event={"ID":"099b7fa9-6908-43e3-b444-73b7f327290d","Type":"ContainerDied","Data":"dd7b57b6c7671c60525066e915121144dfde01a7afd925da54c338eb053d33b0"}
Sep 10 17:37:00.856630 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:37:00.856579    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-7ptnb"
Sep 10 17:37:00.856818 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:37:00.856693    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-b88bx"
Sep 10 17:37:00.856818 ip-10-0-141-180 kubenswrapper[2581]: E0910 17:37:00.856704    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-7ptnb" podUID="766766d2-fa56-49c2-9928-b50cbbf9173a"
Sep 10 17:37:00.856818 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:37:00.856725    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-2t2rd"
Sep 10 17:37:00.857002 ip-10-0-141-180 kubenswrapper[2581]: E0910 17:37:00.856815    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-b88bx" podUID="88d343e2-9756-438b-a19c-973b53948e68"
Sep 10 17:37:00.857002 ip-10-0-141-180 kubenswrapper[2581]: E0910 17:37:00.856905    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-2t2rd" podUID="dee7aebb-2ba6-4bfe-8fb4-371bcc4e56b1"
Sep 10 17:37:01.049505 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:37:01.049469    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-bqj9g" event={"ID":"099b7fa9-6908-43e3-b444-73b7f327290d","Type":"ContainerStarted","Data":"13bd14f3591b4d644186ca361ac4ee208ce185beb0b41e8f4f0325444757f075"}
Sep 10 17:37:01.072090 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:37:01.072047    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-bqj9g" podStartSLOduration=10.481745483 podStartE2EDuration="41.072032586s" totalImagesPullingTime="30.590287103s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-10 17:36:20 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 17:37:01.070544718 +0000 UTC m=+40.724618640" watchObservedRunningTime="2026-09-10 17:37:01.072032586 +0000 UTC m=+40.726106490"
Sep 10 17:37:02.856113 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:37:02.856074    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-7ptnb"
Sep 10 17:37:02.856586 ip-10-0-141-180 kubenswrapper[2581]: E0910 17:37:02.856173    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-7ptnb" podUID="766766d2-fa56-49c2-9928-b50cbbf9173a"
Sep 10 17:37:02.856586 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:37:02.856203    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-b88bx"
Sep 10 17:37:02.856586 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:37:02.856261    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-2t2rd"
Sep 10 17:37:02.856586 ip-10-0-141-180 kubenswrapper[2581]: E0910 17:37:02.856364    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-2t2rd" podUID="dee7aebb-2ba6-4bfe-8fb4-371bcc4e56b1"
Sep 10 17:37:02.856586 ip-10-0-141-180 kubenswrapper[2581]: E0910 17:37:02.856462    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-b88bx" podUID="88d343e2-9756-438b-a19c-973b53948e68"
Sep 10 17:37:04.856101 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:37:04.856068    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-b88bx"
Sep 10 17:37:04.856101 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:37:04.856096    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-7ptnb"
Sep 10 17:37:04.856586 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:37:04.856068    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-2t2rd"
Sep 10 17:37:04.856586 ip-10-0-141-180 kubenswrapper[2581]: E0910 17:37:04.856191    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-b88bx" podUID="88d343e2-9756-438b-a19c-973b53948e68"
Sep 10 17:37:04.856586 ip-10-0-141-180 kubenswrapper[2581]: E0910 17:37:04.856238    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-7ptnb" podUID="766766d2-fa56-49c2-9928-b50cbbf9173a"
Sep 10 17:37:04.856586 ip-10-0-141-180 kubenswrapper[2581]: E0910 17:37:04.856352    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-2t2rd" podUID="dee7aebb-2ba6-4bfe-8fb4-371bcc4e56b1"
Sep 10 17:37:06.856038 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:37:06.856001    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-2t2rd"
Sep 10 17:37:06.856506 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:37:06.856010    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-b88bx"
Sep 10 17:37:06.856506 ip-10-0-141-180 kubenswrapper[2581]: E0910 17:37:06.856134    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-2t2rd" podUID="dee7aebb-2ba6-4bfe-8fb4-371bcc4e56b1"
Sep 10 17:37:06.856506 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:37:06.856009    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-7ptnb"
Sep 10 17:37:06.856506 ip-10-0-141-180 kubenswrapper[2581]: E0910 17:37:06.856193    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-b88bx" podUID="88d343e2-9756-438b-a19c-973b53948e68"
Sep 10 17:37:06.856506 ip-10-0-141-180 kubenswrapper[2581]: E0910 17:37:06.856254    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-7ptnb" podUID="766766d2-fa56-49c2-9928-b50cbbf9173a"
Sep 10 17:37:08.855672 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:37:08.855630    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-7ptnb"
Sep 10 17:37:08.856025 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:37:08.855631    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-b88bx"
Sep 10 17:37:08.856025 ip-10-0-141-180 kubenswrapper[2581]: E0910 17:37:08.855735    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-7ptnb" podUID="766766d2-fa56-49c2-9928-b50cbbf9173a"
Sep 10 17:37:08.856025 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:37:08.855745    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-2t2rd"
Sep 10 17:37:08.856025 ip-10-0-141-180 kubenswrapper[2581]: E0910 17:37:08.855812    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-b88bx" podUID="88d343e2-9756-438b-a19c-973b53948e68"
Sep 10 17:37:08.856025 ip-10-0-141-180 kubenswrapper[2581]: E0910 17:37:08.855876    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-2t2rd" podUID="dee7aebb-2ba6-4bfe-8fb4-371bcc4e56b1"
Sep 10 17:37:10.856799 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:37:10.856771    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-7ptnb"
Sep 10 17:37:10.857269 ip-10-0-141-180 kubenswrapper[2581]: E0910 17:37:10.856844    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-7ptnb" podUID="766766d2-fa56-49c2-9928-b50cbbf9173a"
Sep 10 17:37:10.857269 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:37:10.856869    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-b88bx"
Sep 10 17:37:10.857269 ip-10-0-141-180 kubenswrapper[2581]: E0910 17:37:10.857003    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-b88bx" podUID="88d343e2-9756-438b-a19c-973b53948e68"
Sep 10 17:37:10.857269 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:37:10.857040    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-2t2rd"
Sep 10 17:37:10.857269 ip-10-0-141-180 kubenswrapper[2581]: E0910 17:37:10.857114    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-2t2rd" podUID="dee7aebb-2ba6-4bfe-8fb4-371bcc4e56b1"
Sep 10 17:37:11.689389 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:37:11.689145    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-2t2rd"]
Sep 10 17:37:11.689538 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:37:11.689524    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-2t2rd"
Sep 10 17:37:11.689652 ip-10-0-141-180 kubenswrapper[2581]: E0910 17:37:11.689617    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-2t2rd" podUID="dee7aebb-2ba6-4bfe-8fb4-371bcc4e56b1"
Sep 10 17:37:11.692020 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:37:11.691991    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-7ptnb"]
Sep 10 17:37:11.692167 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:37:11.692097    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-7ptnb"
Sep 10 17:37:11.692230 ip-10-0-141-180 kubenswrapper[2581]: E0910 17:37:11.692192    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-7ptnb" podUID="766766d2-fa56-49c2-9928-b50cbbf9173a"
Sep 10 17:37:11.692865 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:37:11.692838    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-b88bx"]
Sep 10 17:37:11.692979 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:37:11.692943    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-b88bx"
Sep 10 17:37:11.693059 ip-10-0-141-180 kubenswrapper[2581]: E0910 17:37:11.693039    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-b88bx" podUID="88d343e2-9756-438b-a19c-973b53948e68"
Sep 10 17:37:12.855734 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:37:12.855698    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-7ptnb"
Sep 10 17:37:12.855734 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:37:12.855715    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-b88bx"
Sep 10 17:37:12.856316 ip-10-0-141-180 kubenswrapper[2581]: E0910 17:37:12.855822    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-7ptnb" podUID="766766d2-fa56-49c2-9928-b50cbbf9173a"
Sep 10 17:37:12.856316 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:37:12.855897    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-2t2rd"
Sep 10 17:37:12.856316 ip-10-0-141-180 kubenswrapper[2581]: E0910 17:37:12.855897    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-b88bx" podUID="88d343e2-9756-438b-a19c-973b53948e68"
Sep 10 17:37:12.856316 ip-10-0-141-180 kubenswrapper[2581]: E0910 17:37:12.855984    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-2t2rd" podUID="dee7aebb-2ba6-4bfe-8fb4-371bcc4e56b1"
Sep 10 17:37:14.856205 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:37:14.856173    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-2t2rd"
Sep 10 17:37:14.856538 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:37:14.856173    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-7ptnb"
Sep 10 17:37:14.856538 ip-10-0-141-180 kubenswrapper[2581]: E0910 17:37:14.856273    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-2t2rd" podUID="dee7aebb-2ba6-4bfe-8fb4-371bcc4e56b1"
Sep 10 17:37:14.856538 ip-10-0-141-180 kubenswrapper[2581]: E0910 17:37:14.856339    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-7ptnb" podUID="766766d2-fa56-49c2-9928-b50cbbf9173a"
Sep 10 17:37:14.856538 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:37:14.856189    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-b88bx"
Sep 10 17:37:14.856538 ip-10-0-141-180 kubenswrapper[2581]: E0910 17:37:14.856418    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-b88bx" podUID="88d343e2-9756-438b-a19c-973b53948e68"
Sep 10 17:37:15.097593 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:37:15.097520    2581 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-180.ec2.internal" event="NodeReady"
Sep 10 17:37:15.097748 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:37:15.097627    2581 kubelet_node_status.go:489] "Fast updating node status as it just became ready"
Sep 10 17:37:15.130474 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:37:15.130411    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-86f5bb8d74-5kxl4"]
Sep 10 17:37:15.132434 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:37:15.132412    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-86f5bb8d74-5kxl4"
Sep 10 17:37:15.135350 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:37:15.135112    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\""
Sep 10 17:37:15.135350 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:37:15.135226    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\""
Sep 10 17:37:15.135513 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:37:15.135439    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\""
Sep 10 17:37:15.137306 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:37:15.135696    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-hcd8f\""
Sep 10 17:37:15.140213 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:37:15.140186    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\""
Sep 10 17:37:15.145752 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:37:15.145731    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-86f5bb8d74-5kxl4"]
Sep 10 17:37:15.151320 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:37:15.151291    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-hxnhw"]
Sep 10 17:37:15.152919 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:37:15.152906    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-wsx6q"]
Sep 10 17:37:15.153030 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:37:15.153015    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-hxnhw"
Sep 10 17:37:15.154602 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:37:15.154588    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-wsx6q"
Sep 10 17:37:15.155300 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:37:15.155277    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\""
Sep 10 17:37:15.155383 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:37:15.155292    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-5h2r4\""
Sep 10 17:37:15.155383 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:37:15.155337    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\""
Sep 10 17:37:15.155383 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:37:15.155287    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\""
Sep 10 17:37:15.156955 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:37:15.156938    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\""
Sep 10 17:37:15.157072 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:37:15.157055    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\""
Sep 10 17:37:15.157072 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:37:15.157064    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\""
Sep 10 17:37:15.157187 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:37:15.157130    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\""
Sep 10 17:37:15.157296 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:37:15.157281    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-gx7jr\""
Sep 10 17:37:15.161103 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:37:15.161087    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-hxnhw"]
Sep 10 17:37:15.164256 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:37:15.164238    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-wsx6q"]
Sep 10 17:37:15.220143 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:37:15.220120    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/dee8eef1-d418-42eb-9e73-e37e474775ad-crio-socket\") pod \"insights-runtime-extractor-wsx6q\" (UID: \"dee8eef1-d418-42eb-9e73-e37e474775ad\") " pod="openshift-insights/insights-runtime-extractor-wsx6q"
Sep 10 17:37:15.220266 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:37:15.220146    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/dee8eef1-d418-42eb-9e73-e37e474775ad-data-volume\") pod \"insights-runtime-extractor-wsx6q\" (UID: \"dee8eef1-d418-42eb-9e73-e37e474775ad\") " pod="openshift-insights/insights-runtime-extractor-wsx6q"
Sep 10 17:37:15.220266 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:37:15.220163    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/93aaeb2d-3497-480f-b10e-9a535e3cba3c-registry-tls\") pod \"image-registry-86f5bb8d74-5kxl4\" (UID: \"93aaeb2d-3497-480f-b10e-9a535e3cba3c\") " pod="openshift-image-registry/image-registry-86f5bb8d74-5kxl4"
Sep 10 17:37:15.220266 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:37:15.220178    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gh62z\" (UniqueName: \"kubernetes.io/projected/93aaeb2d-3497-480f-b10e-9a535e3cba3c-kube-api-access-gh62z\") pod \"image-registry-86f5bb8d74-5kxl4\" (UID: \"93aaeb2d-3497-480f-b10e-9a535e3cba3c\") " pod="openshift-image-registry/image-registry-86f5bb8d74-5kxl4"
Sep 10 17:37:15.220266 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:37:15.220246    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/93aaeb2d-3497-480f-b10e-9a535e3cba3c-bound-sa-token\") pod \"image-registry-86f5bb8d74-5kxl4\" (UID: \"93aaeb2d-3497-480f-b10e-9a535e3cba3c\") " pod="openshift-image-registry/image-registry-86f5bb8d74-5kxl4"
Sep 10 17:37:15.220429 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:37:15.220338    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/dee8eef1-d418-42eb-9e73-e37e474775ad-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-wsx6q\" (UID: \"dee8eef1-d418-42eb-9e73-e37e474775ad\") " pod="openshift-insights/insights-runtime-extractor-wsx6q"
Sep 10 17:37:15.220429 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:37:15.220364    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/93aaeb2d-3497-480f-b10e-9a535e3cba3c-image-registry-private-configuration\") pod \"image-registry-86f5bb8d74-5kxl4\" (UID: \"93aaeb2d-3497-480f-b10e-9a535e3cba3c\") " pod="openshift-image-registry/image-registry-86f5bb8d74-5kxl4"
Sep 10 17:37:15.220429 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:37:15.220387    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qjq8n\" (UniqueName: \"kubernetes.io/projected/dee8eef1-d418-42eb-9e73-e37e474775ad-kube-api-access-qjq8n\") pod \"insights-runtime-extractor-wsx6q\" (UID: \"dee8eef1-d418-42eb-9e73-e37e474775ad\") " pod="openshift-insights/insights-runtime-extractor-wsx6q"
Sep 10 17:37:15.220429 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:37:15.220412    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/dee8eef1-d418-42eb-9e73-e37e474775ad-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-wsx6q\" (UID: \"dee8eef1-d418-42eb-9e73-e37e474775ad\") " pod="openshift-insights/insights-runtime-extractor-wsx6q"
Sep 10 17:37:15.220578 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:37:15.220441    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/e78eff7d-f4b5-4b89-b23b-fe0e70020758-cert\") pod \"ingress-canary-hxnhw\" (UID: \"e78eff7d-f4b5-4b89-b23b-fe0e70020758\") " pod="openshift-ingress-canary/ingress-canary-hxnhw"
Sep 10 17:37:15.220578 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:37:15.220487    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6h7j5\" (UniqueName: \"kubernetes.io/projected/e78eff7d-f4b5-4b89-b23b-fe0e70020758-kube-api-access-6h7j5\") pod \"ingress-canary-hxnhw\" (UID: \"e78eff7d-f4b5-4b89-b23b-fe0e70020758\") " pod="openshift-ingress-canary/ingress-canary-hxnhw"
Sep 10 17:37:15.220578 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:37:15.220503    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/93aaeb2d-3497-480f-b10e-9a535e3cba3c-ca-trust-extracted\") pod \"image-registry-86f5bb8d74-5kxl4\" (UID: \"93aaeb2d-3497-480f-b10e-9a535e3cba3c\") " pod="openshift-image-registry/image-registry-86f5bb8d74-5kxl4"
Sep 10 17:37:15.220578 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:37:15.220517    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/93aaeb2d-3497-480f-b10e-9a535e3cba3c-trusted-ca\") pod \"image-registry-86f5bb8d74-5kxl4\" (UID: \"93aaeb2d-3497-480f-b10e-9a535e3cba3c\") " pod="openshift-image-registry/image-registry-86f5bb8d74-5kxl4"
Sep 10 17:37:15.220578 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:37:15.220536    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/93aaeb2d-3497-480f-b10e-9a535e3cba3c-installation-pull-secrets\") pod \"image-registry-86f5bb8d74-5kxl4\" (UID: \"93aaeb2d-3497-480f-b10e-9a535e3cba3c\") " pod="openshift-image-registry/image-registry-86f5bb8d74-5kxl4"
Sep 10 17:37:15.220578 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:37:15.220554    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/93aaeb2d-3497-480f-b10e-9a535e3cba3c-registry-certificates\") pod \"image-registry-86f5bb8d74-5kxl4\" (UID: \"93aaeb2d-3497-480f-b10e-9a535e3cba3c\") " pod="openshift-image-registry/image-registry-86f5bb8d74-5kxl4"
Sep 10 17:37:15.243670 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:37:15.243628    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-99dqc"]
Sep 10 17:37:15.246072 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:37:15.246058    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-99dqc"
Sep 10 17:37:15.248446 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:37:15.248427    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-wtnr4\""
Sep 10 17:37:15.248446 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:37:15.248439    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\""
Sep 10 17:37:15.248599 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:37:15.248517    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\""
Sep 10 17:37:15.255514 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:37:15.255495    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-99dqc"]
Sep 10 17:37:15.321264 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:37:15.321229    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6h7j5\" (UniqueName: \"kubernetes.io/projected/e78eff7d-f4b5-4b89-b23b-fe0e70020758-kube-api-access-6h7j5\") pod \"ingress-canary-hxnhw\" (UID: \"e78eff7d-f4b5-4b89-b23b-fe0e70020758\") " pod="openshift-ingress-canary/ingress-canary-hxnhw"
Sep 10 17:37:15.321264 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:37:15.321261    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/93aaeb2d-3497-480f-b10e-9a535e3cba3c-ca-trust-extracted\") pod \"image-registry-86f5bb8d74-5kxl4\" (UID: \"93aaeb2d-3497-480f-b10e-9a535e3cba3c\") " pod="openshift-image-registry/image-registry-86f5bb8d74-5kxl4"
Sep 10 17:37:15.321487 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:37:15.321280    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/93aaeb2d-3497-480f-b10e-9a535e3cba3c-trusted-ca\") pod \"image-registry-86f5bb8d74-5kxl4\" (UID: \"93aaeb2d-3497-480f-b10e-9a535e3cba3c\") " pod="openshift-image-registry/image-registry-86f5bb8d74-5kxl4"
Sep 10 17:37:15.321487 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:37:15.321300    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/93aaeb2d-3497-480f-b10e-9a535e3cba3c-installation-pull-secrets\") pod \"image-registry-86f5bb8d74-5kxl4\" (UID: \"93aaeb2d-3497-480f-b10e-9a535e3cba3c\") " pod="openshift-image-registry/image-registry-86f5bb8d74-5kxl4"
Sep 10 17:37:15.321487 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:37:15.321316    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/93aaeb2d-3497-480f-b10e-9a535e3cba3c-registry-certificates\") pod \"image-registry-86f5bb8d74-5kxl4\" (UID: \"93aaeb2d-3497-480f-b10e-9a535e3cba3c\") " pod="openshift-image-registry/image-registry-86f5bb8d74-5kxl4"
Sep 10 17:37:15.321487 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:37:15.321339    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mhm8n\" (UniqueName: \"kubernetes.io/projected/d73b14cf-7f49-458f-989d-3488354f3aa7-kube-api-access-mhm8n\") pod \"dns-default-99dqc\" (UID: \"d73b14cf-7f49-458f-989d-3488354f3aa7\") " pod="openshift-dns/dns-default-99dqc"
Sep 10 17:37:15.321487 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:37:15.321364    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/dee8eef1-d418-42eb-9e73-e37e474775ad-crio-socket\") pod \"insights-runtime-extractor-wsx6q\" (UID: \"dee8eef1-d418-42eb-9e73-e37e474775ad\") " pod="openshift-insights/insights-runtime-extractor-wsx6q"
Sep 10 17:37:15.321487 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:37:15.321446    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/dee8eef1-d418-42eb-9e73-e37e474775ad-data-volume\") pod \"insights-runtime-extractor-wsx6q\" (UID: \"dee8eef1-d418-42eb-9e73-e37e474775ad\") " pod="openshift-insights/insights-runtime-extractor-wsx6q"
Sep 10 17:37:15.321487 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:37:15.321472    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/93aaeb2d-3497-480f-b10e-9a535e3cba3c-registry-tls\") pod \"image-registry-86f5bb8d74-5kxl4\" (UID: \"93aaeb2d-3497-480f-b10e-9a535e3cba3c\") " pod="openshift-image-registry/image-registry-86f5bb8d74-5kxl4"
Sep 10 17:37:15.321815 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:37:15.321494    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gh62z\" (UniqueName: \"kubernetes.io/projected/93aaeb2d-3497-480f-b10e-9a535e3cba3c-kube-api-access-gh62z\") pod \"image-registry-86f5bb8d74-5kxl4\" (UID: \"93aaeb2d-3497-480f-b10e-9a535e3cba3c\") " pod="openshift-image-registry/image-registry-86f5bb8d74-5kxl4"
Sep 10 17:37:15.321815 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:37:15.321515    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/d73b14cf-7f49-458f-989d-3488354f3aa7-config-volume\") pod \"dns-default-99dqc\" (UID: \"d73b14cf-7f49-458f-989d-3488354f3aa7\") " pod="openshift-dns/dns-default-99dqc"
Sep 10 17:37:15.321815 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:37:15.321545    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/dee8eef1-d418-42eb-9e73-e37e474775ad-crio-socket\") pod \"insights-runtime-extractor-wsx6q\" (UID: \"dee8eef1-d418-42eb-9e73-e37e474775ad\") " pod="openshift-insights/insights-runtime-extractor-wsx6q"
Sep 10 17:37:15.321815 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:37:15.321563    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/93aaeb2d-3497-480f-b10e-9a535e3cba3c-bound-sa-token\") pod \"image-registry-86f5bb8d74-5kxl4\" (UID: \"93aaeb2d-3497-480f-b10e-9a535e3cba3c\") " pod="openshift-image-registry/image-registry-86f5bb8d74-5kxl4"
Sep 10 17:37:15.321815 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:37:15.321599    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/d73b14cf-7f49-458f-989d-3488354f3aa7-metrics-tls\") pod \"dns-default-99dqc\" (UID: \"d73b14cf-7f49-458f-989d-3488354f3aa7\") " pod="openshift-dns/dns-default-99dqc"
Sep 10 17:37:15.321815 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:37:15.321679    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/dee8eef1-d418-42eb-9e73-e37e474775ad-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-wsx6q\" (UID: \"dee8eef1-d418-42eb-9e73-e37e474775ad\") " pod="openshift-insights/insights-runtime-extractor-wsx6q"
Sep 10 17:37:15.321815 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:37:15.321712    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/93aaeb2d-3497-480f-b10e-9a535e3cba3c-image-registry-private-configuration\") pod \"image-registry-86f5bb8d74-5kxl4\" (UID: \"93aaeb2d-3497-480f-b10e-9a535e3cba3c\") " pod="openshift-image-registry/image-registry-86f5bb8d74-5kxl4"
Sep 10 17:37:15.321815 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:37:15.321744    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d73b14cf-7f49-458f-989d-3488354f3aa7-tmp-dir\") pod \"dns-default-99dqc\" (UID: \"d73b14cf-7f49-458f-989d-3488354f3aa7\") " pod="openshift-dns/dns-default-99dqc"
Sep 10 17:37:15.321815 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:37:15.321764    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/93aaeb2d-3497-480f-b10e-9a535e3cba3c-ca-trust-extracted\") pod \"image-registry-86f5bb8d74-5kxl4\" (UID: \"93aaeb2d-3497-480f-b10e-9a535e3cba3c\") " pod="openshift-image-registry/image-registry-86f5bb8d74-5kxl4"
Sep 10 17:37:15.321815 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:37:15.321774    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qjq8n\" (UniqueName: \"kubernetes.io/projected/dee8eef1-d418-42eb-9e73-e37e474775ad-kube-api-access-qjq8n\") pod \"insights-runtime-extractor-wsx6q\" (UID: \"dee8eef1-d418-42eb-9e73-e37e474775ad\") " pod="openshift-insights/insights-runtime-extractor-wsx6q"
Sep 10 17:37:15.322222 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:37:15.321867    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/dee8eef1-d418-42eb-9e73-e37e474775ad-data-volume\") pod \"insights-runtime-extractor-wsx6q\" (UID: \"dee8eef1-d418-42eb-9e73-e37e474775ad\") " pod="openshift-insights/insights-runtime-extractor-wsx6q"
Sep 10 17:37:15.322222 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:37:15.321975    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/dee8eef1-d418-42eb-9e73-e37e474775ad-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-wsx6q\" (UID: \"dee8eef1-d418-42eb-9e73-e37e474775ad\") " pod="openshift-insights/insights-runtime-extractor-wsx6q"
Sep 10 17:37:15.322222 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:37:15.322010    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/e78eff7d-f4b5-4b89-b23b-fe0e70020758-cert\") pod \"ingress-canary-hxnhw\" (UID: \"e78eff7d-f4b5-4b89-b23b-fe0e70020758\") " pod="openshift-ingress-canary/ingress-canary-hxnhw"
Sep 10 17:37:15.322372 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:37:15.322290    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/93aaeb2d-3497-480f-b10e-9a535e3cba3c-registry-certificates\") pod \"image-registry-86f5bb8d74-5kxl4\" (UID: \"93aaeb2d-3497-480f-b10e-9a535e3cba3c\") " pod="openshift-image-registry/image-registry-86f5bb8d74-5kxl4"
Sep 10 17:37:15.322541 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:37:15.322521    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/dee8eef1-d418-42eb-9e73-e37e474775ad-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-wsx6q\" (UID: \"dee8eef1-d418-42eb-9e73-e37e474775ad\") " pod="openshift-insights/insights-runtime-extractor-wsx6q"
Sep 10 17:37:15.323033 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:37:15.323013    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/93aaeb2d-3497-480f-b10e-9a535e3cba3c-trusted-ca\") pod \"image-registry-86f5bb8d74-5kxl4\" (UID: \"93aaeb2d-3497-480f-b10e-9a535e3cba3c\") " pod="openshift-image-registry/image-registry-86f5bb8d74-5kxl4"
Sep 10 17:37:15.325742 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:37:15.325715    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/93aaeb2d-3497-480f-b10e-9a535e3cba3c-installation-pull-secrets\") pod \"image-registry-86f5bb8d74-5kxl4\" (UID: \"93aaeb2d-3497-480f-b10e-9a535e3cba3c\") " pod="openshift-image-registry/image-registry-86f5bb8d74-5kxl4"
Sep 10 17:37:15.325897 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:37:15.325877    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/93aaeb2d-3497-480f-b10e-9a535e3cba3c-image-registry-private-configuration\") pod \"image-registry-86f5bb8d74-5kxl4\" (UID: \"93aaeb2d-3497-480f-b10e-9a535e3cba3c\") " pod="openshift-image-registry/image-registry-86f5bb8d74-5kxl4"
Sep 10 17:37:15.325937 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:37:15.325882    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/93aaeb2d-3497-480f-b10e-9a535e3cba3c-registry-tls\") pod \"image-registry-86f5bb8d74-5kxl4\" (UID: \"93aaeb2d-3497-480f-b10e-9a535e3cba3c\") " pod="openshift-image-registry/image-registry-86f5bb8d74-5kxl4"
Sep 10 17:37:15.326019 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:37:15.326006    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/dee8eef1-d418-42eb-9e73-e37e474775ad-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-wsx6q\" (UID: \"dee8eef1-d418-42eb-9e73-e37e474775ad\") " pod="openshift-insights/insights-runtime-extractor-wsx6q"
Sep 10 17:37:15.326351 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:37:15.326332    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/e78eff7d-f4b5-4b89-b23b-fe0e70020758-cert\") pod \"ingress-canary-hxnhw\" (UID: \"e78eff7d-f4b5-4b89-b23b-fe0e70020758\") " pod="openshift-ingress-canary/ingress-canary-hxnhw"
Sep 10 17:37:15.329174 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:37:15.329152    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qjq8n\" (UniqueName: \"kubernetes.io/projected/dee8eef1-d418-42eb-9e73-e37e474775ad-kube-api-access-qjq8n\") pod \"insights-runtime-extractor-wsx6q\" (UID: \"dee8eef1-d418-42eb-9e73-e37e474775ad\") " pod="openshift-insights/insights-runtime-extractor-wsx6q"
Sep 10 17:37:15.329246 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:37:15.329150    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gh62z\" (UniqueName: \"kubernetes.io/projected/93aaeb2d-3497-480f-b10e-9a535e3cba3c-kube-api-access-gh62z\") pod \"image-registry-86f5bb8d74-5kxl4\" (UID: \"93aaeb2d-3497-480f-b10e-9a535e3cba3c\") " pod="openshift-image-registry/image-registry-86f5bb8d74-5kxl4"
Sep 10 17:37:15.329707 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:37:15.329687    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/93aaeb2d-3497-480f-b10e-9a535e3cba3c-bound-sa-token\") pod \"image-registry-86f5bb8d74-5kxl4\" (UID: \"93aaeb2d-3497-480f-b10e-9a535e3cba3c\") " pod="openshift-image-registry/image-registry-86f5bb8d74-5kxl4"
Sep 10 17:37:15.329813 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:37:15.329796    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6h7j5\" (UniqueName: \"kubernetes.io/projected/e78eff7d-f4b5-4b89-b23b-fe0e70020758-kube-api-access-6h7j5\") pod \"ingress-canary-hxnhw\" (UID: \"e78eff7d-f4b5-4b89-b23b-fe0e70020758\") " pod="openshift-ingress-canary/ingress-canary-hxnhw"
Sep 10 17:37:15.422976 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:37:15.422892    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mhm8n\" (UniqueName: \"kubernetes.io/projected/d73b14cf-7f49-458f-989d-3488354f3aa7-kube-api-access-mhm8n\") pod \"dns-default-99dqc\" (UID: \"d73b14cf-7f49-458f-989d-3488354f3aa7\") " pod="openshift-dns/dns-default-99dqc"
Sep 10 17:37:15.422976 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:37:15.422935    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/d73b14cf-7f49-458f-989d-3488354f3aa7-config-volume\") pod \"dns-default-99dqc\" (UID: \"d73b14cf-7f49-458f-989d-3488354f3aa7\") " pod="openshift-dns/dns-default-99dqc"
Sep 10 17:37:15.422976 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:37:15.422966    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/d73b14cf-7f49-458f-989d-3488354f3aa7-metrics-tls\") pod \"dns-default-99dqc\" (UID: \"d73b14cf-7f49-458f-989d-3488354f3aa7\") " pod="openshift-dns/dns-default-99dqc"
Sep 10 17:37:15.423324 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:37:15.423009    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d73b14cf-7f49-458f-989d-3488354f3aa7-tmp-dir\") pod \"dns-default-99dqc\" (UID: \"d73b14cf-7f49-458f-989d-3488354f3aa7\") " pod="openshift-dns/dns-default-99dqc"
Sep 10 17:37:15.423362 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:37:15.423325    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d73b14cf-7f49-458f-989d-3488354f3aa7-tmp-dir\") pod \"dns-default-99dqc\" (UID: \"d73b14cf-7f49-458f-989d-3488354f3aa7\") " pod="openshift-dns/dns-default-99dqc"
Sep 10 17:37:15.423431 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:37:15.423411    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/d73b14cf-7f49-458f-989d-3488354f3aa7-config-volume\") pod \"dns-default-99dqc\" (UID: \"d73b14cf-7f49-458f-989d-3488354f3aa7\") " pod="openshift-dns/dns-default-99dqc"
Sep 10 17:37:15.425102 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:37:15.425086    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/d73b14cf-7f49-458f-989d-3488354f3aa7-metrics-tls\") pod \"dns-default-99dqc\" (UID: \"d73b14cf-7f49-458f-989d-3488354f3aa7\") " pod="openshift-dns/dns-default-99dqc"
Sep 10 17:37:15.431340 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:37:15.431318    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mhm8n\" (UniqueName: \"kubernetes.io/projected/d73b14cf-7f49-458f-989d-3488354f3aa7-kube-api-access-mhm8n\") pod \"dns-default-99dqc\" (UID: \"d73b14cf-7f49-458f-989d-3488354f3aa7\") " pod="openshift-dns/dns-default-99dqc"
Sep 10 17:37:15.444294 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:37:15.444275    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-86f5bb8d74-5kxl4"
Sep 10 17:37:15.463055 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:37:15.463032    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-hxnhw"
Sep 10 17:37:15.468103 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:37:15.468084    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-wsx6q"
Sep 10 17:37:15.553980 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:37:15.553915    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-99dqc"
Sep 10 17:37:15.728342 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:37:15.728311    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-86f5bb8d74-5kxl4"]
Sep 10 17:37:15.743804 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:37:15.743777    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-hxnhw"]
Sep 10 17:37:15.746599 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:37:15.746574    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-wsx6q"]
Sep 10 17:37:15.746715 ip-10-0-141-180 kubenswrapper[2581]: W0910 17:37:15.746633    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode78eff7d_f4b5_4b89_b23b_fe0e70020758.slice/crio-a3493042f9a83d6e5d483a93d0f5bb54a7d49e1e2f846af46a818ffb06691bd2 WatchSource:0}: Error finding container a3493042f9a83d6e5d483a93d0f5bb54a7d49e1e2f846af46a818ffb06691bd2: Status 404 returned error can't find the container with id a3493042f9a83d6e5d483a93d0f5bb54a7d49e1e2f846af46a818ffb06691bd2
Sep 10 17:37:15.750431 ip-10-0-141-180 kubenswrapper[2581]: W0910 17:37:15.750403    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddee8eef1_d418_42eb_9e73_e37e474775ad.slice/crio-18d564dd45651a39a123f813da407d247f5f81f4454fc67f3cb73c659b7979e4 WatchSource:0}: Error finding container 18d564dd45651a39a123f813da407d247f5f81f4454fc67f3cb73c659b7979e4: Status 404 returned error can't find the container with id 18d564dd45651a39a123f813da407d247f5f81f4454fc67f3cb73c659b7979e4
Sep 10 17:37:15.825921 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:37:15.825896    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-99dqc"]
Sep 10 17:37:15.829780 ip-10-0-141-180 kubenswrapper[2581]: W0910 17:37:15.829754    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd73b14cf_7f49_458f_989d_3488354f3aa7.slice/crio-561eb3a001714dc4a6fee0c1d6146f97412cd0b1d13a65c5d631e8b3894c70c5 WatchSource:0}: Error finding container 561eb3a001714dc4a6fee0c1d6146f97412cd0b1d13a65c5d631e8b3894c70c5: Status 404 returned error can't find the container with id 561eb3a001714dc4a6fee0c1d6146f97412cd0b1d13a65c5d631e8b3894c70c5
Sep 10 17:37:16.083054 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:37:16.082979    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-99dqc" event={"ID":"d73b14cf-7f49-458f-989d-3488354f3aa7","Type":"ContainerStarted","Data":"561eb3a001714dc4a6fee0c1d6146f97412cd0b1d13a65c5d631e8b3894c70c5"}
Sep 10 17:37:16.084557 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:37:16.084535    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-wsx6q" event={"ID":"dee8eef1-d418-42eb-9e73-e37e474775ad","Type":"ContainerStarted","Data":"f483d94fbbd1457f9c7bd59182ea96f6a48fa9a7caf4c726e54e6ce20ba6ad04"}
Sep 10 17:37:16.084698 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:37:16.084583    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-wsx6q" event={"ID":"dee8eef1-d418-42eb-9e73-e37e474775ad","Type":"ContainerStarted","Data":"18d564dd45651a39a123f813da407d247f5f81f4454fc67f3cb73c659b7979e4"}
Sep 10 17:37:16.085775 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:37:16.085737    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-hxnhw" event={"ID":"e78eff7d-f4b5-4b89-b23b-fe0e70020758","Type":"ContainerStarted","Data":"a3493042f9a83d6e5d483a93d0f5bb54a7d49e1e2f846af46a818ffb06691bd2"}
Sep 10 17:37:16.087226 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:37:16.087205    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-86f5bb8d74-5kxl4" event={"ID":"93aaeb2d-3497-480f-b10e-9a535e3cba3c","Type":"ContainerStarted","Data":"33fbe7ac07aa612d74b535327e23a9c854bb0727265a05fcbc7575efcba8d29c"}
Sep 10 17:37:16.087316 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:37:16.087247    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-86f5bb8d74-5kxl4" event={"ID":"93aaeb2d-3497-480f-b10e-9a535e3cba3c","Type":"ContainerStarted","Data":"984f24d081133f0dae007fcee694ca7850a067ae4191d42732513f35df7c3bed"}
Sep 10 17:37:16.087366 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:37:16.087345    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-86f5bb8d74-5kxl4"
Sep 10 17:37:16.104621 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:37:16.104557    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/image-registry-86f5bb8d74-5kxl4" podStartSLOduration=6.104537874 podStartE2EDuration="6.104537874s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 17:37:10 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 17:37:16.10403089 +0000 UTC m=+55.758104819" watchObservedRunningTime="2026-09-10 17:37:16.104537874 +0000 UTC m=+55.758611779"
Sep 10 17:37:16.856229 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:37:16.856194    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-b88bx"
Sep 10 17:37:16.856411 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:37:16.856243    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-2t2rd"
Sep 10 17:37:16.856476 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:37:16.856427    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-7ptnb"
Sep 10 17:37:16.860003 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:37:16.859979    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Sep 10 17:37:16.860160 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:37:16.860143    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-sh4f6\""
Sep 10 17:37:16.860219 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:37:16.860207    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Sep 10 17:37:16.860269 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:37:16.860235    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Sep 10 17:37:16.860432 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:37:16.860416    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-7lms2\""
Sep 10 17:37:16.860490 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:37:16.860416    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Sep 10 17:37:17.092826 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:37:17.092676    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-wsx6q" event={"ID":"dee8eef1-d418-42eb-9e73-e37e474775ad","Type":"ContainerStarted","Data":"250a61d370fa7822e0b293c153307e72c54c9b65341e8e102eeaaedd77806ea4"}
Sep 10 17:37:19.100766 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:37:19.100679    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-wsx6q" event={"ID":"dee8eef1-d418-42eb-9e73-e37e474775ad","Type":"ContainerStarted","Data":"f23585e47e5203c4888fcc4030937873fbeb52a4fc29abd4426311fdf6e9aa73"}
Sep 10 17:37:19.102289 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:37:19.102260    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-hxnhw" event={"ID":"e78eff7d-f4b5-4b89-b23b-fe0e70020758","Type":"ContainerStarted","Data":"1a6afd23bf55d166552f25e2539961c7b34bca9f5ff9e3c2124dabd4f24d1a5d"}
Sep 10 17:37:19.104174 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:37:19.104150    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-99dqc" event={"ID":"d73b14cf-7f49-458f-989d-3488354f3aa7","Type":"ContainerStarted","Data":"23d4081383e097b6fa1dc2690c70ad51e443afada4e62ca4db6eacfce8aef8d9"}
Sep 10 17:37:19.104261 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:37:19.104197    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-99dqc" event={"ID":"d73b14cf-7f49-458f-989d-3488354f3aa7","Type":"ContainerStarted","Data":"a7229236f7ecde0b77ed0483d5176b21f33f7becda9dc0b956a579f4666c66fa"}
Sep 10 17:37:19.104261 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:37:19.104219    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-99dqc"
Sep 10 17:37:19.120812 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:37:19.120770    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-wsx6q" podStartSLOduration=1.155251136 podStartE2EDuration="4.120757216s" totalImagesPullingTime="2.96550608s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-10 17:37:15 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 17:37:19.119991877 +0000 UTC m=+58.774065801" watchObservedRunningTime="2026-09-10 17:37:19.120757216 +0000 UTC m=+58.774831111"
Sep 10 17:37:19.134058 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:37:19.133997    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-hxnhw" podStartSLOduration=1.020869579 podStartE2EDuration="4.133982826s" totalImagesPullingTime="3.113113247s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 17:37:15 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 17:37:19.133808972 +0000 UTC m=+58.787882875" watchObservedRunningTime="2026-09-10 17:37:19.133982826 +0000 UTC m=+58.788056730"
Sep 10 17:37:19.150469 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:37:19.150430    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-99dqc" podStartSLOduration=1.128811575 podStartE2EDuration="4.15041658s" totalImagesPullingTime="3.021605005s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 17:37:15 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 17:37:19.149537681 +0000 UTC m=+58.803611595" watchObservedRunningTime="2026-09-10 17:37:19.15041658 +0000 UTC m=+58.804490485"
Sep 10 17:37:20.024791 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:37:20.024762    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-7nn7k"
Sep 10 17:37:21.174729 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:37:21.174688    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-operator-6f4946db56-qxdfm"]
Sep 10 17:37:21.213295 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:37:21.213265    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-6f4946db56-qxdfm"]
Sep 10 17:37:21.213412 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:37:21.213393    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-6f4946db56-qxdfm"
Sep 10 17:37:21.216324 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:37:21.216303    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\""
Sep 10 17:37:21.216441 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:37:21.216304    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\""
Sep 10 17:37:21.217465 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:37:21.217449    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-dockercfg-ltffx\""
Sep 10 17:37:21.218298 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:37:21.218180    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\""
Sep 10 17:37:21.218456 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:37:21.218188    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-kube-rbac-proxy-config\""
Sep 10 17:37:21.218773 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:37:21.218621    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-tls\""
Sep 10 17:37:21.270417 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:37:21.270391    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/f32f968e-0732-4abd-a2ac-099f0cde2db6-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-6f4946db56-qxdfm\" (UID: \"f32f968e-0732-4abd-a2ac-099f0cde2db6\") " pod="openshift-monitoring/prometheus-operator-6f4946db56-qxdfm"
Sep 10 17:37:21.270586 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:37:21.270428    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/f32f968e-0732-4abd-a2ac-099f0cde2db6-metrics-client-ca\") pod \"prometheus-operator-6f4946db56-qxdfm\" (UID: \"f32f968e-0732-4abd-a2ac-099f0cde2db6\") " pod="openshift-monitoring/prometheus-operator-6f4946db56-qxdfm"
Sep 10 17:37:21.270586 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:37:21.270455    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ptjww\" (UniqueName: \"kubernetes.io/projected/f32f968e-0732-4abd-a2ac-099f0cde2db6-kube-api-access-ptjww\") pod \"prometheus-operator-6f4946db56-qxdfm\" (UID: \"f32f968e-0732-4abd-a2ac-099f0cde2db6\") " pod="openshift-monitoring/prometheus-operator-6f4946db56-qxdfm"
Sep 10 17:37:21.270719 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:37:21.270604    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/f32f968e-0732-4abd-a2ac-099f0cde2db6-prometheus-operator-tls\") pod \"prometheus-operator-6f4946db56-qxdfm\" (UID: \"f32f968e-0732-4abd-a2ac-099f0cde2db6\") " pod="openshift-monitoring/prometheus-operator-6f4946db56-qxdfm"
Sep 10 17:37:21.371522 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:37:21.371485    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/f32f968e-0732-4abd-a2ac-099f0cde2db6-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-6f4946db56-qxdfm\" (UID: \"f32f968e-0732-4abd-a2ac-099f0cde2db6\") " pod="openshift-monitoring/prometheus-operator-6f4946db56-qxdfm"
Sep 10 17:37:21.371522 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:37:21.371532    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/f32f968e-0732-4abd-a2ac-099f0cde2db6-metrics-client-ca\") pod \"prometheus-operator-6f4946db56-qxdfm\" (UID: \"f32f968e-0732-4abd-a2ac-099f0cde2db6\") " pod="openshift-monitoring/prometheus-operator-6f4946db56-qxdfm"
Sep 10 17:37:21.371798 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:37:21.371564    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ptjww\" (UniqueName: \"kubernetes.io/projected/f32f968e-0732-4abd-a2ac-099f0cde2db6-kube-api-access-ptjww\") pod \"prometheus-operator-6f4946db56-qxdfm\" (UID: \"f32f968e-0732-4abd-a2ac-099f0cde2db6\") " pod="openshift-monitoring/prometheus-operator-6f4946db56-qxdfm"
Sep 10 17:37:21.371798 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:37:21.371613    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/f32f968e-0732-4abd-a2ac-099f0cde2db6-prometheus-operator-tls\") pod \"prometheus-operator-6f4946db56-qxdfm\" (UID: \"f32f968e-0732-4abd-a2ac-099f0cde2db6\") " pod="openshift-monitoring/prometheus-operator-6f4946db56-qxdfm"
Sep 10 17:37:21.372223 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:37:21.372200    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/f32f968e-0732-4abd-a2ac-099f0cde2db6-metrics-client-ca\") pod \"prometheus-operator-6f4946db56-qxdfm\" (UID: \"f32f968e-0732-4abd-a2ac-099f0cde2db6\") " pod="openshift-monitoring/prometheus-operator-6f4946db56-qxdfm"
Sep 10 17:37:21.374106 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:37:21.374072    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/f32f968e-0732-4abd-a2ac-099f0cde2db6-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-6f4946db56-qxdfm\" (UID: \"f32f968e-0732-4abd-a2ac-099f0cde2db6\") " pod="openshift-monitoring/prometheus-operator-6f4946db56-qxdfm"
Sep 10 17:37:21.374208 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:37:21.374114    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/f32f968e-0732-4abd-a2ac-099f0cde2db6-prometheus-operator-tls\") pod \"prometheus-operator-6f4946db56-qxdfm\" (UID: \"f32f968e-0732-4abd-a2ac-099f0cde2db6\") " pod="openshift-monitoring/prometheus-operator-6f4946db56-qxdfm"
Sep 10 17:37:21.380494 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:37:21.380475    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ptjww\" (UniqueName: \"kubernetes.io/projected/f32f968e-0732-4abd-a2ac-099f0cde2db6-kube-api-access-ptjww\") pod \"prometheus-operator-6f4946db56-qxdfm\" (UID: \"f32f968e-0732-4abd-a2ac-099f0cde2db6\") " pod="openshift-monitoring/prometheus-operator-6f4946db56-qxdfm"
Sep 10 17:37:21.523749 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:37:21.523717    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-6f4946db56-qxdfm"
Sep 10 17:37:21.788850 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:37:21.788772    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-6f4946db56-qxdfm"]
Sep 10 17:37:21.791774 ip-10-0-141-180 kubenswrapper[2581]: W0910 17:37:21.791746    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf32f968e_0732_4abd_a2ac_099f0cde2db6.slice/crio-b18e6a53522d045737d4eedd2287b0c85db541a54bb939ae8f4068bd03e89416 WatchSource:0}: Error finding container b18e6a53522d045737d4eedd2287b0c85db541a54bb939ae8f4068bd03e89416: Status 404 returned error can't find the container with id b18e6a53522d045737d4eedd2287b0c85db541a54bb939ae8f4068bd03e89416
Sep 10 17:37:22.114616 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:37:22.114534    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-6f4946db56-qxdfm" event={"ID":"f32f968e-0732-4abd-a2ac-099f0cde2db6","Type":"ContainerStarted","Data":"b18e6a53522d045737d4eedd2287b0c85db541a54bb939ae8f4068bd03e89416"}
Sep 10 17:37:24.122955 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:37:24.122916    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-6f4946db56-qxdfm" event={"ID":"f32f968e-0732-4abd-a2ac-099f0cde2db6","Type":"ContainerStarted","Data":"44ebb6add7eea02e9faa9d6dee95b00d712151e1524559a522d7d939f69f1f95"}
Sep 10 17:37:24.123325 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:37:24.122972    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-6f4946db56-qxdfm" event={"ID":"f32f968e-0732-4abd-a2ac-099f0cde2db6","Type":"ContainerStarted","Data":"4edaebc0b3ac6f3f307c1c298b79b5e342be777b1b422d1a62dd379f51d6407d"}
Sep 10 17:37:24.139733 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:37:24.139692    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-operator-6f4946db56-qxdfm" podStartSLOduration=1.7443265449999998 podStartE2EDuration="3.139677227s" totalImagesPullingTime="1.395350682s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 17:37:21 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 17:37:24.138428522 +0000 UTC m=+63.792502445" watchObservedRunningTime="2026-09-10 17:37:24.139677227 +0000 UTC m=+63.793751130"
Sep 10 17:37:26.508240 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:37:26.508202    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-w28mp\" (UniqueName: \"kubernetes.io/projected/766766d2-fa56-49c2-9928-b50cbbf9173a-kube-api-access-w28mp\") pod \"network-check-target-7ptnb\" (UID: \"766766d2-fa56-49c2-9928-b50cbbf9173a\") " pod="openshift-network-diagnostics/network-check-target-7ptnb"
Sep 10 17:37:26.508240 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:37:26.508246    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/88d343e2-9756-438b-a19c-973b53948e68-metrics-certs\") pod \"network-metrics-daemon-b88bx\" (UID: \"88d343e2-9756-438b-a19c-973b53948e68\") " pod="openshift-multus/network-metrics-daemon-b88bx"
Sep 10 17:37:26.510928 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:37:26.510906    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Sep 10 17:37:26.511828 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:37:26.511813    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Sep 10 17:37:26.521001 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:37:26.520956    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/88d343e2-9756-438b-a19c-973b53948e68-metrics-certs\") pod \"network-metrics-daemon-b88bx\" (UID: \"88d343e2-9756-438b-a19c-973b53948e68\") " pod="openshift-multus/network-metrics-daemon-b88bx"
Sep 10 17:37:26.521165 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:37:26.521150    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Sep 10 17:37:26.531054 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:37:26.531032    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-w28mp\" (UniqueName: \"kubernetes.io/projected/766766d2-fa56-49c2-9928-b50cbbf9173a-kube-api-access-w28mp\") pod \"network-check-target-7ptnb\" (UID: \"766766d2-fa56-49c2-9928-b50cbbf9173a\") " pod="openshift-network-diagnostics/network-check-target-7ptnb"
Sep 10 17:37:26.555278 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:37:26.555251    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-vz9b7"]
Sep 10 17:37:26.561438 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:37:26.560964    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-vz9b7"
Sep 10 17:37:26.561438 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:37:26.561096    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/kube-state-metrics-74cb57df7-9hv2h"]
Sep 10 17:37:26.563390 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:37:26.563363    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/kube-state-metrics-74cb57df7-9hv2h"
Sep 10 17:37:26.563743 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:37:26.563707    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\""
Sep 10 17:37:26.563844 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:37:26.563791    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\""
Sep 10 17:37:26.563899 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:37:26.563886    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\""
Sep 10 17:37:26.563959 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:37:26.563945    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-rtt87\""
Sep 10 17:37:26.565549 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:37:26.565531    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-tls\""
Sep 10 17:37:26.565892 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:37:26.565874    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-kube-rbac-proxy-config\""
Sep 10 17:37:26.566351 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:37:26.566331    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-custom-resource-state-configmap\""
Sep 10 17:37:26.566438 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:37:26.566360    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-dockercfg-4lmr9\""
Sep 10 17:37:26.576375 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:37:26.576353    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/kube-state-metrics-74cb57df7-9hv2h"]
Sep 10 17:37:26.608730 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:37:26.608706    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/dcd0f9c6-c02f-437a-afe7-9dd75c827467-root\") pod \"node-exporter-vz9b7\" (UID: \"dcd0f9c6-c02f-437a-afe7-9dd75c827467\") " pod="openshift-monitoring/node-exporter-vz9b7"
Sep 10 17:37:26.608839 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:37:26.608743    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/df97b1f6-ca1d-481e-892c-68579c1dbfc4-volume-directive-shadow\") pod \"kube-state-metrics-74cb57df7-9hv2h\" (UID: \"df97b1f6-ca1d-481e-892c-68579c1dbfc4\") " pod="openshift-monitoring/kube-state-metrics-74cb57df7-9hv2h"
Sep 10 17:37:26.608839 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:37:26.608773    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/dcd0f9c6-c02f-437a-afe7-9dd75c827467-sys\") pod \"node-exporter-vz9b7\" (UID: \"dcd0f9c6-c02f-437a-afe7-9dd75c827467\") " pod="openshift-monitoring/node-exporter-vz9b7"
Sep 10 17:37:26.608839 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:37:26.608797    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/dcd0f9c6-c02f-437a-afe7-9dd75c827467-node-exporter-textfile\") pod \"node-exporter-vz9b7\" (UID: \"dcd0f9c6-c02f-437a-afe7-9dd75c827467\") " pod="openshift-monitoring/node-exporter-vz9b7"
Sep 10 17:37:26.608956 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:37:26.608843    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/dcd0f9c6-c02f-437a-afe7-9dd75c827467-node-exporter-accelerators-collector-config\") pod \"node-exporter-vz9b7\" (UID: \"dcd0f9c6-c02f-437a-afe7-9dd75c827467\") " pod="openshift-monitoring/node-exporter-vz9b7"
Sep 10 17:37:26.608956 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:37:26.608871    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/dcd0f9c6-c02f-437a-afe7-9dd75c827467-node-exporter-tls\") pod \"node-exporter-vz9b7\" (UID: \"dcd0f9c6-c02f-437a-afe7-9dd75c827467\") " pod="openshift-monitoring/node-exporter-vz9b7"
Sep 10 17:37:26.608956 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:37:26.608902    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/dcd0f9c6-c02f-437a-afe7-9dd75c827467-node-exporter-wtmp\") pod \"node-exporter-vz9b7\" (UID: \"dcd0f9c6-c02f-437a-afe7-9dd75c827467\") " pod="openshift-monitoring/node-exporter-vz9b7"
Sep 10 17:37:26.608956 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:37:26.608921    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/dcd0f9c6-c02f-437a-afe7-9dd75c827467-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-vz9b7\" (UID: \"dcd0f9c6-c02f-437a-afe7-9dd75c827467\") " pod="openshift-monitoring/node-exporter-vz9b7"
Sep 10 17:37:26.609156 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:37:26.609003    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nvclm\" (UniqueName: \"kubernetes.io/projected/df97b1f6-ca1d-481e-892c-68579c1dbfc4-kube-api-access-nvclm\") pod \"kube-state-metrics-74cb57df7-9hv2h\" (UID: \"df97b1f6-ca1d-481e-892c-68579c1dbfc4\") " pod="openshift-monitoring/kube-state-metrics-74cb57df7-9hv2h"
Sep 10 17:37:26.609156 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:37:26.609035    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/df97b1f6-ca1d-481e-892c-68579c1dbfc4-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-74cb57df7-9hv2h\" (UID: \"df97b1f6-ca1d-481e-892c-68579c1dbfc4\") " pod="openshift-monitoring/kube-state-metrics-74cb57df7-9hv2h"
Sep 10 17:37:26.609156 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:37:26.609074    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/df97b1f6-ca1d-481e-892c-68579c1dbfc4-kube-state-metrics-tls\") pod \"kube-state-metrics-74cb57df7-9hv2h\" (UID: \"df97b1f6-ca1d-481e-892c-68579c1dbfc4\") " pod="openshift-monitoring/kube-state-metrics-74cb57df7-9hv2h"
Sep 10 17:37:26.609156 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:37:26.609107    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/dee7aebb-2ba6-4bfe-8fb4-371bcc4e56b1-original-pull-secret\") pod \"global-pull-secret-syncer-2t2rd\" (UID: \"dee7aebb-2ba6-4bfe-8fb4-371bcc4e56b1\") " pod="kube-system/global-pull-secret-syncer-2t2rd"
Sep 10 17:37:26.609156 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:37:26.609144    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/df97b1f6-ca1d-481e-892c-68579c1dbfc4-metrics-client-ca\") pod \"kube-state-metrics-74cb57df7-9hv2h\" (UID: \"df97b1f6-ca1d-481e-892c-68579c1dbfc4\") " pod="openshift-monitoring/kube-state-metrics-74cb57df7-9hv2h"
Sep 10 17:37:26.609349 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:37:26.609169    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/df97b1f6-ca1d-481e-892c-68579c1dbfc4-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-74cb57df7-9hv2h\" (UID: \"df97b1f6-ca1d-481e-892c-68579c1dbfc4\") " pod="openshift-monitoring/kube-state-metrics-74cb57df7-9hv2h"
Sep 10 17:37:26.609349 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:37:26.609194    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7svrl\" (UniqueName: \"kubernetes.io/projected/dcd0f9c6-c02f-437a-afe7-9dd75c827467-kube-api-access-7svrl\") pod \"node-exporter-vz9b7\" (UID: \"dcd0f9c6-c02f-437a-afe7-9dd75c827467\") " pod="openshift-monitoring/node-exporter-vz9b7"
Sep 10 17:37:26.609349 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:37:26.609220    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/dcd0f9c6-c02f-437a-afe7-9dd75c827467-metrics-client-ca\") pod \"node-exporter-vz9b7\" (UID: \"dcd0f9c6-c02f-437a-afe7-9dd75c827467\") " pod="openshift-monitoring/node-exporter-vz9b7"
Sep 10 17:37:26.611510 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:37:26.611494    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Sep 10 17:37:26.621684 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:37:26.621666    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/dee7aebb-2ba6-4bfe-8fb4-371bcc4e56b1-original-pull-secret\") pod \"global-pull-secret-syncer-2t2rd\" (UID: \"dee7aebb-2ba6-4bfe-8fb4-371bcc4e56b1\") " pod="kube-system/global-pull-secret-syncer-2t2rd"
Sep 10 17:37:26.710353 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:37:26.710316    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/dcd0f9c6-c02f-437a-afe7-9dd75c827467-node-exporter-wtmp\") pod \"node-exporter-vz9b7\" (UID: \"dcd0f9c6-c02f-437a-afe7-9dd75c827467\") " pod="openshift-monitoring/node-exporter-vz9b7"
Sep 10 17:37:26.710353 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:37:26.710357    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/dcd0f9c6-c02f-437a-afe7-9dd75c827467-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-vz9b7\" (UID: \"dcd0f9c6-c02f-437a-afe7-9dd75c827467\") " pod="openshift-monitoring/node-exporter-vz9b7"
Sep 10 17:37:26.710568 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:37:26.710382    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nvclm\" (UniqueName: \"kubernetes.io/projected/df97b1f6-ca1d-481e-892c-68579c1dbfc4-kube-api-access-nvclm\") pod \"kube-state-metrics-74cb57df7-9hv2h\" (UID: \"df97b1f6-ca1d-481e-892c-68579c1dbfc4\") " pod="openshift-monitoring/kube-state-metrics-74cb57df7-9hv2h"
Sep 10 17:37:26.710568 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:37:26.710399    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/df97b1f6-ca1d-481e-892c-68579c1dbfc4-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-74cb57df7-9hv2h\" (UID: \"df97b1f6-ca1d-481e-892c-68579c1dbfc4\") " pod="openshift-monitoring/kube-state-metrics-74cb57df7-9hv2h"
Sep 10 17:37:26.710568 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:37:26.710422    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/df97b1f6-ca1d-481e-892c-68579c1dbfc4-kube-state-metrics-tls\") pod \"kube-state-metrics-74cb57df7-9hv2h\" (UID: \"df97b1f6-ca1d-481e-892c-68579c1dbfc4\") " pod="openshift-monitoring/kube-state-metrics-74cb57df7-9hv2h"
Sep 10 17:37:26.710568 ip-10-0-141-180 kubenswrapper[2581]: E0910 17:37:26.710520    2581 secret.go:189] Couldn't get secret openshift-monitoring/kube-state-metrics-tls: secret "kube-state-metrics-tls" not found
Sep 10 17:37:26.710568 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:37:26.710543    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/dcd0f9c6-c02f-437a-afe7-9dd75c827467-node-exporter-wtmp\") pod \"node-exporter-vz9b7\" (UID: \"dcd0f9c6-c02f-437a-afe7-9dd75c827467\") " pod="openshift-monitoring/node-exporter-vz9b7"
Sep 10 17:37:26.710856 ip-10-0-141-180 kubenswrapper[2581]: E0910 17:37:26.710598    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/df97b1f6-ca1d-481e-892c-68579c1dbfc4-kube-state-metrics-tls podName:df97b1f6-ca1d-481e-892c-68579c1dbfc4 nodeName:}" failed. No retries permitted until 2026-09-10 17:37:27.21057926 +0000 UTC m=+66.864653158 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-state-metrics-tls" (UniqueName: "kubernetes.io/secret/df97b1f6-ca1d-481e-892c-68579c1dbfc4-kube-state-metrics-tls") pod "kube-state-metrics-74cb57df7-9hv2h" (UID: "df97b1f6-ca1d-481e-892c-68579c1dbfc4") : secret "kube-state-metrics-tls" not found
Sep 10 17:37:26.710856 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:37:26.710622    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/df97b1f6-ca1d-481e-892c-68579c1dbfc4-metrics-client-ca\") pod \"kube-state-metrics-74cb57df7-9hv2h\" (UID: \"df97b1f6-ca1d-481e-892c-68579c1dbfc4\") " pod="openshift-monitoring/kube-state-metrics-74cb57df7-9hv2h"
Sep 10 17:37:26.710856 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:37:26.710675    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/df97b1f6-ca1d-481e-892c-68579c1dbfc4-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-74cb57df7-9hv2h\" (UID: \"df97b1f6-ca1d-481e-892c-68579c1dbfc4\") " pod="openshift-monitoring/kube-state-metrics-74cb57df7-9hv2h"
Sep 10 17:37:26.710856 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:37:26.710705    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7svrl\" (UniqueName: \"kubernetes.io/projected/dcd0f9c6-c02f-437a-afe7-9dd75c827467-kube-api-access-7svrl\") pod \"node-exporter-vz9b7\" (UID: \"dcd0f9c6-c02f-437a-afe7-9dd75c827467\") " pod="openshift-monitoring/node-exporter-vz9b7"
Sep 10 17:37:26.710856 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:37:26.710731    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/dcd0f9c6-c02f-437a-afe7-9dd75c827467-metrics-client-ca\") pod \"node-exporter-vz9b7\" (UID: \"dcd0f9c6-c02f-437a-afe7-9dd75c827467\") " pod="openshift-monitoring/node-exporter-vz9b7"
Sep 10 17:37:26.710856 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:37:26.710759    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/dcd0f9c6-c02f-437a-afe7-9dd75c827467-root\") pod \"node-exporter-vz9b7\" (UID: \"dcd0f9c6-c02f-437a-afe7-9dd75c827467\") " pod="openshift-monitoring/node-exporter-vz9b7"
Sep 10 17:37:26.710856 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:37:26.710794    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/df97b1f6-ca1d-481e-892c-68579c1dbfc4-volume-directive-shadow\") pod \"kube-state-metrics-74cb57df7-9hv2h\" (UID: \"df97b1f6-ca1d-481e-892c-68579c1dbfc4\") " pod="openshift-monitoring/kube-state-metrics-74cb57df7-9hv2h"
Sep 10 17:37:26.710856 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:37:26.710849    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/dcd0f9c6-c02f-437a-afe7-9dd75c827467-sys\") pod \"node-exporter-vz9b7\" (UID: \"dcd0f9c6-c02f-437a-afe7-9dd75c827467\") " pod="openshift-monitoring/node-exporter-vz9b7"
Sep 10 17:37:26.711214 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:37:26.710873    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/dcd0f9c6-c02f-437a-afe7-9dd75c827467-node-exporter-textfile\") pod \"node-exporter-vz9b7\" (UID: \"dcd0f9c6-c02f-437a-afe7-9dd75c827467\") " pod="openshift-monitoring/node-exporter-vz9b7"
Sep 10 17:37:26.711214 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:37:26.710897    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/dcd0f9c6-c02f-437a-afe7-9dd75c827467-root\") pod \"node-exporter-vz9b7\" (UID: \"dcd0f9c6-c02f-437a-afe7-9dd75c827467\") " pod="openshift-monitoring/node-exporter-vz9b7"
Sep 10 17:37:26.711214 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:37:26.710909    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/dcd0f9c6-c02f-437a-afe7-9dd75c827467-node-exporter-accelerators-collector-config\") pod \"node-exporter-vz9b7\" (UID: \"dcd0f9c6-c02f-437a-afe7-9dd75c827467\") " pod="openshift-monitoring/node-exporter-vz9b7"
Sep 10 17:37:26.711214 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:37:26.710935    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/dcd0f9c6-c02f-437a-afe7-9dd75c827467-node-exporter-tls\") pod \"node-exporter-vz9b7\" (UID: \"dcd0f9c6-c02f-437a-afe7-9dd75c827467\") " pod="openshift-monitoring/node-exporter-vz9b7"
Sep 10 17:37:26.711214 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:37:26.710986    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/dcd0f9c6-c02f-437a-afe7-9dd75c827467-sys\") pod \"node-exporter-vz9b7\" (UID: \"dcd0f9c6-c02f-437a-afe7-9dd75c827467\") " pod="openshift-monitoring/node-exporter-vz9b7"
Sep 10 17:37:26.711214 ip-10-0-141-180 kubenswrapper[2581]: E0910 17:37:26.711063    2581 secret.go:189] Couldn't get secret openshift-monitoring/node-exporter-tls: secret "node-exporter-tls" not found
Sep 10 17:37:26.711214 ip-10-0-141-180 kubenswrapper[2581]: E0910 17:37:26.711129    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/dcd0f9c6-c02f-437a-afe7-9dd75c827467-node-exporter-tls podName:dcd0f9c6-c02f-437a-afe7-9dd75c827467 nodeName:}" failed. No retries permitted until 2026-09-10 17:37:27.211113788 +0000 UTC m=+66.865187687 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "node-exporter-tls" (UniqueName: "kubernetes.io/secret/dcd0f9c6-c02f-437a-afe7-9dd75c827467-node-exporter-tls") pod "node-exporter-vz9b7" (UID: "dcd0f9c6-c02f-437a-afe7-9dd75c827467") : secret "node-exporter-tls" not found
Sep 10 17:37:26.711550 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:37:26.711403    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/dcd0f9c6-c02f-437a-afe7-9dd75c827467-node-exporter-textfile\") pod \"node-exporter-vz9b7\" (UID: \"dcd0f9c6-c02f-437a-afe7-9dd75c827467\") " pod="openshift-monitoring/node-exporter-vz9b7"
Sep 10 17:37:26.711550 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:37:26.711467    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/dcd0f9c6-c02f-437a-afe7-9dd75c827467-metrics-client-ca\") pod \"node-exporter-vz9b7\" (UID: \"dcd0f9c6-c02f-437a-afe7-9dd75c827467\") " pod="openshift-monitoring/node-exporter-vz9b7"
Sep 10 17:37:26.711716 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:37:26.711662    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/df97b1f6-ca1d-481e-892c-68579c1dbfc4-volume-directive-shadow\") pod \"kube-state-metrics-74cb57df7-9hv2h\" (UID: \"df97b1f6-ca1d-481e-892c-68579c1dbfc4\") " pod="openshift-monitoring/kube-state-metrics-74cb57df7-9hv2h"
Sep 10 17:37:26.711716 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:37:26.711693    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/dcd0f9c6-c02f-437a-afe7-9dd75c827467-node-exporter-accelerators-collector-config\") pod \"node-exporter-vz9b7\" (UID: \"dcd0f9c6-c02f-437a-afe7-9dd75c827467\") " pod="openshift-monitoring/node-exporter-vz9b7"
Sep 10 17:37:26.711970 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:37:26.711950    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/df97b1f6-ca1d-481e-892c-68579c1dbfc4-metrics-client-ca\") pod \"kube-state-metrics-74cb57df7-9hv2h\" (UID: \"df97b1f6-ca1d-481e-892c-68579c1dbfc4\") " pod="openshift-monitoring/kube-state-metrics-74cb57df7-9hv2h"
Sep 10 17:37:26.712201 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:37:26.712183    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/df97b1f6-ca1d-481e-892c-68579c1dbfc4-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-74cb57df7-9hv2h\" (UID: \"df97b1f6-ca1d-481e-892c-68579c1dbfc4\") " pod="openshift-monitoring/kube-state-metrics-74cb57df7-9hv2h"
Sep 10 17:37:26.712877 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:37:26.712856    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/dcd0f9c6-c02f-437a-afe7-9dd75c827467-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-vz9b7\" (UID: \"dcd0f9c6-c02f-437a-afe7-9dd75c827467\") " pod="openshift-monitoring/node-exporter-vz9b7"
Sep 10 17:37:26.713460 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:37:26.713435    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/df97b1f6-ca1d-481e-892c-68579c1dbfc4-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-74cb57df7-9hv2h\" (UID: \"df97b1f6-ca1d-481e-892c-68579c1dbfc4\") " pod="openshift-monitoring/kube-state-metrics-74cb57df7-9hv2h"
Sep 10 17:37:26.719457 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:37:26.719427    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nvclm\" (UniqueName: \"kubernetes.io/projected/df97b1f6-ca1d-481e-892c-68579c1dbfc4-kube-api-access-nvclm\") pod \"kube-state-metrics-74cb57df7-9hv2h\" (UID: \"df97b1f6-ca1d-481e-892c-68579c1dbfc4\") " pod="openshift-monitoring/kube-state-metrics-74cb57df7-9hv2h"
Sep 10 17:37:26.719542 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:37:26.719430    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7svrl\" (UniqueName: \"kubernetes.io/projected/dcd0f9c6-c02f-437a-afe7-9dd75c827467-kube-api-access-7svrl\") pod \"node-exporter-vz9b7\" (UID: \"dcd0f9c6-c02f-437a-afe7-9dd75c827467\") " pod="openshift-monitoring/node-exporter-vz9b7"
Sep 10 17:37:26.772775 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:37:26.772700    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-7lms2\""
Sep 10 17:37:26.775726 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:37:26.775709    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-2t2rd"
Sep 10 17:37:26.779259 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:37:26.779242    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-b88bx"
Sep 10 17:37:26.784173 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:37:26.784152    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-sh4f6\""
Sep 10 17:37:26.792051 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:37:26.792029    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-7ptnb"
Sep 10 17:37:27.053063 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:37:27.052969    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-b88bx"]
Sep 10 17:37:27.054250 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:37:27.054156    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-2t2rd"]
Sep 10 17:37:27.056949 ip-10-0-141-180 kubenswrapper[2581]: W0910 17:37:27.056922    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod88d343e2_9756_438b_a19c_973b53948e68.slice/crio-f845d0a4063154ea21336411f9c28b34734c9d4547b9375276a75b78b58ef911 WatchSource:0}: Error finding container f845d0a4063154ea21336411f9c28b34734c9d4547b9375276a75b78b58ef911: Status 404 returned error can't find the container with id f845d0a4063154ea21336411f9c28b34734c9d4547b9375276a75b78b58ef911
Sep 10 17:37:27.057438 ip-10-0-141-180 kubenswrapper[2581]: W0910 17:37:27.057415    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddee7aebb_2ba6_4bfe_8fb4_371bcc4e56b1.slice/crio-c34bc4f0a452f2214f554abc426f13884404a4f7d3307496a479964bd3906fc0 WatchSource:0}: Error finding container c34bc4f0a452f2214f554abc426f13884404a4f7d3307496a479964bd3906fc0: Status 404 returned error can't find the container with id c34bc4f0a452f2214f554abc426f13884404a4f7d3307496a479964bd3906fc0
Sep 10 17:37:27.091259 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:37:27.091234    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-7ptnb"]
Sep 10 17:37:27.094053 ip-10-0-141-180 kubenswrapper[2581]: W0910 17:37:27.094028    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod766766d2_fa56_49c2_9928_b50cbbf9173a.slice/crio-df78013669146e296776d1f679a90f10ed74860b03a526065031ed225c10344e WatchSource:0}: Error finding container df78013669146e296776d1f679a90f10ed74860b03a526065031ed225c10344e: Status 404 returned error can't find the container with id df78013669146e296776d1f679a90f10ed74860b03a526065031ed225c10344e
Sep 10 17:37:27.134682 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:37:27.134630    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-b88bx" event={"ID":"88d343e2-9756-438b-a19c-973b53948e68","Type":"ContainerStarted","Data":"f845d0a4063154ea21336411f9c28b34734c9d4547b9375276a75b78b58ef911"}
Sep 10 17:37:27.135767 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:37:27.135748    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-7ptnb" event={"ID":"766766d2-fa56-49c2-9928-b50cbbf9173a","Type":"ContainerStarted","Data":"df78013669146e296776d1f679a90f10ed74860b03a526065031ed225c10344e"}
Sep 10 17:37:27.136789 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:37:27.136768    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-2t2rd" event={"ID":"dee7aebb-2ba6-4bfe-8fb4-371bcc4e56b1","Type":"ContainerStarted","Data":"c34bc4f0a452f2214f554abc426f13884404a4f7d3307496a479964bd3906fc0"}
Sep 10 17:37:27.216280 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:37:27.216242    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/dcd0f9c6-c02f-437a-afe7-9dd75c827467-node-exporter-tls\") pod \"node-exporter-vz9b7\" (UID: \"dcd0f9c6-c02f-437a-afe7-9dd75c827467\") " pod="openshift-monitoring/node-exporter-vz9b7"
Sep 10 17:37:27.216451 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:37:27.216323    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/df97b1f6-ca1d-481e-892c-68579c1dbfc4-kube-state-metrics-tls\") pod \"kube-state-metrics-74cb57df7-9hv2h\" (UID: \"df97b1f6-ca1d-481e-892c-68579c1dbfc4\") " pod="openshift-monitoring/kube-state-metrics-74cb57df7-9hv2h"
Sep 10 17:37:27.218759 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:37:27.218740    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/dcd0f9c6-c02f-437a-afe7-9dd75c827467-node-exporter-tls\") pod \"node-exporter-vz9b7\" (UID: \"dcd0f9c6-c02f-437a-afe7-9dd75c827467\") " pod="openshift-monitoring/node-exporter-vz9b7"
Sep 10 17:37:27.218817 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:37:27.218758    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/df97b1f6-ca1d-481e-892c-68579c1dbfc4-kube-state-metrics-tls\") pod \"kube-state-metrics-74cb57df7-9hv2h\" (UID: \"df97b1f6-ca1d-481e-892c-68579c1dbfc4\") " pod="openshift-monitoring/kube-state-metrics-74cb57df7-9hv2h"
Sep 10 17:37:27.471133 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:37:27.471101    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-vz9b7"
Sep 10 17:37:27.476272 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:37:27.476241    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/kube-state-metrics-74cb57df7-9hv2h"
Sep 10 17:37:27.511715 ip-10-0-141-180 kubenswrapper[2581]: W0910 17:37:27.511688    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddcd0f9c6_c02f_437a_afe7_9dd75c827467.slice/crio-d594609f782abab1b1a0f0c8510031698de6b9e2613259d5e9ef1730488e97ab WatchSource:0}: Error finding container d594609f782abab1b1a0f0c8510031698de6b9e2613259d5e9ef1730488e97ab: Status 404 returned error can't find the container with id d594609f782abab1b1a0f0c8510031698de6b9e2613259d5e9ef1730488e97ab
Sep 10 17:37:27.814426 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:37:27.814350    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/kube-state-metrics-74cb57df7-9hv2h"]
Sep 10 17:37:27.820346 ip-10-0-141-180 kubenswrapper[2581]: W0910 17:37:27.820314    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddf97b1f6_ca1d_481e_892c_68579c1dbfc4.slice/crio-e9ca81ad128006bb356152a22174b264318e327723660301a86972dba80a1afe WatchSource:0}: Error finding container e9ca81ad128006bb356152a22174b264318e327723660301a86972dba80a1afe: Status 404 returned error can't find the container with id e9ca81ad128006bb356152a22174b264318e327723660301a86972dba80a1afe
Sep 10 17:37:28.142811 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:37:28.142676    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-74cb57df7-9hv2h" event={"ID":"df97b1f6-ca1d-481e-892c-68579c1dbfc4","Type":"ContainerStarted","Data":"e9ca81ad128006bb356152a22174b264318e327723660301a86972dba80a1afe"}
Sep 10 17:37:28.144389 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:37:28.144359    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-vz9b7" event={"ID":"dcd0f9c6-c02f-437a-afe7-9dd75c827467","Type":"ContainerStarted","Data":"d594609f782abab1b1a0f0c8510031698de6b9e2613259d5e9ef1730488e97ab"}
Sep 10 17:37:28.497115 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:37:28.495948    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/thanos-querier-568869598d-ns9hc"]
Sep 10 17:37:28.509478 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:37:28.509444    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/thanos-querier-568869598d-ns9hc"
Sep 10 17:37:28.513285 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:37:28.513255    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/thanos-querier-568869598d-ns9hc"]
Sep 10 17:37:28.513774 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:37:28.513701    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-metrics\""
Sep 10 17:37:28.513979 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:37:28.513841    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy\""
Sep 10 17:37:28.513979 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:37:28.513970    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-tls\""
Sep 10 17:37:28.514102 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:37:28.514010    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-web\""
Sep 10 17:37:28.514102 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:37:28.513701    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-grpc-tls-1b96a15tpt1ql\""
Sep 10 17:37:28.514220 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:37:28.514187    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-dockercfg-k8sbv\""
Sep 10 17:37:28.514508 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:37:28.514356    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-rules\""
Sep 10 17:37:28.626922 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:37:28.626883    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/6e66367a-0612-42be-948e-ebf346d9820c-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-568869598d-ns9hc\" (UID: \"6e66367a-0612-42be-948e-ebf346d9820c\") " pod="openshift-monitoring/thanos-querier-568869598d-ns9hc"
Sep 10 17:37:28.627109 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:37:28.626994    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/6e66367a-0612-42be-948e-ebf346d9820c-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-568869598d-ns9hc\" (UID: \"6e66367a-0612-42be-948e-ebf346d9820c\") " pod="openshift-monitoring/thanos-querier-568869598d-ns9hc"
Sep 10 17:37:28.627109 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:37:28.627043    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/6e66367a-0612-42be-948e-ebf346d9820c-secret-grpc-tls\") pod \"thanos-querier-568869598d-ns9hc\" (UID: \"6e66367a-0612-42be-948e-ebf346d9820c\") " pod="openshift-monitoring/thanos-querier-568869598d-ns9hc"
Sep 10 17:37:28.627109 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:37:28.627080    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tg7sf\" (UniqueName: \"kubernetes.io/projected/6e66367a-0612-42be-948e-ebf346d9820c-kube-api-access-tg7sf\") pod \"thanos-querier-568869598d-ns9hc\" (UID: \"6e66367a-0612-42be-948e-ebf346d9820c\") " pod="openshift-monitoring/thanos-querier-568869598d-ns9hc"
Sep 10 17:37:28.627264 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:37:28.627149    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/6e66367a-0612-42be-948e-ebf346d9820c-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-568869598d-ns9hc\" (UID: \"6e66367a-0612-42be-948e-ebf346d9820c\") " pod="openshift-monitoring/thanos-querier-568869598d-ns9hc"
Sep 10 17:37:28.627264 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:37:28.627179    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/6e66367a-0612-42be-948e-ebf346d9820c-metrics-client-ca\") pod \"thanos-querier-568869598d-ns9hc\" (UID: \"6e66367a-0612-42be-948e-ebf346d9820c\") " pod="openshift-monitoring/thanos-querier-568869598d-ns9hc"
Sep 10 17:37:28.627264 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:37:28.627208    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/6e66367a-0612-42be-948e-ebf346d9820c-secret-thanos-querier-tls\") pod \"thanos-querier-568869598d-ns9hc\" (UID: \"6e66367a-0612-42be-948e-ebf346d9820c\") " pod="openshift-monitoring/thanos-querier-568869598d-ns9hc"
Sep 10 17:37:28.627264 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:37:28.627238    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/6e66367a-0612-42be-948e-ebf346d9820c-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-568869598d-ns9hc\" (UID: \"6e66367a-0612-42be-948e-ebf346d9820c\") " pod="openshift-monitoring/thanos-querier-568869598d-ns9hc"
Sep 10 17:37:28.729054 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:37:28.728443    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/6e66367a-0612-42be-948e-ebf346d9820c-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-568869598d-ns9hc\" (UID: \"6e66367a-0612-42be-948e-ebf346d9820c\") " pod="openshift-monitoring/thanos-querier-568869598d-ns9hc"
Sep 10 17:37:28.729054 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:37:28.728500    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/6e66367a-0612-42be-948e-ebf346d9820c-secret-grpc-tls\") pod \"thanos-querier-568869598d-ns9hc\" (UID: \"6e66367a-0612-42be-948e-ebf346d9820c\") " pod="openshift-monitoring/thanos-querier-568869598d-ns9hc"
Sep 10 17:37:28.729054 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:37:28.728535    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tg7sf\" (UniqueName: \"kubernetes.io/projected/6e66367a-0612-42be-948e-ebf346d9820c-kube-api-access-tg7sf\") pod \"thanos-querier-568869598d-ns9hc\" (UID: \"6e66367a-0612-42be-948e-ebf346d9820c\") " pod="openshift-monitoring/thanos-querier-568869598d-ns9hc"
Sep 10 17:37:28.729054 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:37:28.728594    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/6e66367a-0612-42be-948e-ebf346d9820c-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-568869598d-ns9hc\" (UID: \"6e66367a-0612-42be-948e-ebf346d9820c\") " pod="openshift-monitoring/thanos-querier-568869598d-ns9hc"
Sep 10 17:37:28.729054 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:37:28.728631    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/6e66367a-0612-42be-948e-ebf346d9820c-metrics-client-ca\") pod \"thanos-querier-568869598d-ns9hc\" (UID: \"6e66367a-0612-42be-948e-ebf346d9820c\") " pod="openshift-monitoring/thanos-querier-568869598d-ns9hc"
Sep 10 17:37:28.729054 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:37:28.729002    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/6e66367a-0612-42be-948e-ebf346d9820c-secret-thanos-querier-tls\") pod \"thanos-querier-568869598d-ns9hc\" (UID: \"6e66367a-0612-42be-948e-ebf346d9820c\") " pod="openshift-monitoring/thanos-querier-568869598d-ns9hc"
Sep 10 17:37:28.729054 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:37:28.729039    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/6e66367a-0612-42be-948e-ebf346d9820c-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-568869598d-ns9hc\" (UID: \"6e66367a-0612-42be-948e-ebf346d9820c\") " pod="openshift-monitoring/thanos-querier-568869598d-ns9hc"
Sep 10 17:37:28.729526 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:37:28.729070    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/6e66367a-0612-42be-948e-ebf346d9820c-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-568869598d-ns9hc\" (UID: \"6e66367a-0612-42be-948e-ebf346d9820c\") " pod="openshift-monitoring/thanos-querier-568869598d-ns9hc"
Sep 10 17:37:28.730698 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:37:28.730628    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/6e66367a-0612-42be-948e-ebf346d9820c-metrics-client-ca\") pod \"thanos-querier-568869598d-ns9hc\" (UID: \"6e66367a-0612-42be-948e-ebf346d9820c\") " pod="openshift-monitoring/thanos-querier-568869598d-ns9hc"
Sep 10 17:37:28.732632 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:37:28.732268    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/6e66367a-0612-42be-948e-ebf346d9820c-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-568869598d-ns9hc\" (UID: \"6e66367a-0612-42be-948e-ebf346d9820c\") " pod="openshift-monitoring/thanos-querier-568869598d-ns9hc"
Sep 10 17:37:28.732632 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:37:28.732579    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/6e66367a-0612-42be-948e-ebf346d9820c-secret-grpc-tls\") pod \"thanos-querier-568869598d-ns9hc\" (UID: \"6e66367a-0612-42be-948e-ebf346d9820c\") " pod="openshift-monitoring/thanos-querier-568869598d-ns9hc"
Sep 10 17:37:28.734819 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:37:28.733936    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/6e66367a-0612-42be-948e-ebf346d9820c-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-568869598d-ns9hc\" (UID: \"6e66367a-0612-42be-948e-ebf346d9820c\") " pod="openshift-monitoring/thanos-querier-568869598d-ns9hc"
Sep 10 17:37:28.734819 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:37:28.734220    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/6e66367a-0612-42be-948e-ebf346d9820c-secret-thanos-querier-tls\") pod \"thanos-querier-568869598d-ns9hc\" (UID: \"6e66367a-0612-42be-948e-ebf346d9820c\") " pod="openshift-monitoring/thanos-querier-568869598d-ns9hc"
Sep 10 17:37:28.734819 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:37:28.734630    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/6e66367a-0612-42be-948e-ebf346d9820c-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-568869598d-ns9hc\" (UID: \"6e66367a-0612-42be-948e-ebf346d9820c\") " pod="openshift-monitoring/thanos-querier-568869598d-ns9hc"
Sep 10 17:37:28.735099 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:37:28.735055    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/6e66367a-0612-42be-948e-ebf346d9820c-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-568869598d-ns9hc\" (UID: \"6e66367a-0612-42be-948e-ebf346d9820c\") " pod="openshift-monitoring/thanos-querier-568869598d-ns9hc"
Sep 10 17:37:28.737935 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:37:28.737913    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tg7sf\" (UniqueName: \"kubernetes.io/projected/6e66367a-0612-42be-948e-ebf346d9820c-kube-api-access-tg7sf\") pod \"thanos-querier-568869598d-ns9hc\" (UID: \"6e66367a-0612-42be-948e-ebf346d9820c\") " pod="openshift-monitoring/thanos-querier-568869598d-ns9hc"
Sep 10 17:37:28.823695 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:37:28.823602    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/thanos-querier-568869598d-ns9hc"
Sep 10 17:37:29.109424 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:37:29.109334    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-99dqc"
Sep 10 17:37:29.827565 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:37:29.827146    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/thanos-querier-568869598d-ns9hc"]
Sep 10 17:37:30.156783 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:37:30.156697    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-b88bx" event={"ID":"88d343e2-9756-438b-a19c-973b53948e68","Type":"ContainerStarted","Data":"79ef33ed0c7f33e38586f5bc50ace7453ceebc41e217805143280665e61cb6de"}
Sep 10 17:37:30.156783 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:37:30.156766    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-b88bx" event={"ID":"88d343e2-9756-438b-a19c-973b53948e68","Type":"ContainerStarted","Data":"b335f395c094d3565bf1c25de7b8ec630b4013fb201aba75291db6b8fb1384b5"}
Sep 10 17:37:30.159948 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:37:30.159918    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-74cb57df7-9hv2h" event={"ID":"df97b1f6-ca1d-481e-892c-68579c1dbfc4","Type":"ContainerStarted","Data":"dee5b7bacf9cd3ad0c879f87d1f41d4ad8321d1c15cf527dfab980ab4a112026"}
Sep 10 17:37:30.160078 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:37:30.159978    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-74cb57df7-9hv2h" event={"ID":"df97b1f6-ca1d-481e-892c-68579c1dbfc4","Type":"ContainerStarted","Data":"64d2bff52e2126446222086348a838fa45582b181d27900e3afea9655df76c90"}
Sep 10 17:37:30.162666 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:37:30.162349    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-vz9b7" event={"ID":"dcd0f9c6-c02f-437a-afe7-9dd75c827467","Type":"ContainerStarted","Data":"9467c75c1eb928aece0cf1d260acfdced42ed84d549f837ef591cb02d5964e7b"}
Sep 10 17:37:30.177016 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:37:30.176964    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-b88bx" podStartSLOduration=67.708236787 podStartE2EDuration="1m10.176944214s" totalImagesPullingTime="2.468707427s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 17:36:20 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 17:37:30.175935757 +0000 UTC m=+69.830009662" watchObservedRunningTime="2026-09-10 17:37:30.176944214 +0000 UTC m=+69.831018158"
Sep 10 17:37:30.841874 ip-10-0-141-180 kubenswrapper[2581]: W0910 17:37:30.841827    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6e66367a_0612_42be_948e_ebf346d9820c.slice/crio-a702e86e9f501ac79563570b29d423f44c77f7d0755c6a3b79fba94a071eda83 WatchSource:0}: Error finding container a702e86e9f501ac79563570b29d423f44c77f7d0755c6a3b79fba94a071eda83: Status 404 returned error can't find the container with id a702e86e9f501ac79563570b29d423f44c77f7d0755c6a3b79fba94a071eda83
Sep 10 17:37:31.167359 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:37:31.167275    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-568869598d-ns9hc" event={"ID":"6e66367a-0612-42be-948e-ebf346d9820c","Type":"ContainerStarted","Data":"a702e86e9f501ac79563570b29d423f44c77f7d0755c6a3b79fba94a071eda83"}
Sep 10 17:37:31.169233 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:37:31.169197    2581 generic.go:356] "Generic (PLEG): container finished" podID="dcd0f9c6-c02f-437a-afe7-9dd75c827467" containerID="9467c75c1eb928aece0cf1d260acfdced42ed84d549f837ef591cb02d5964e7b" exitCode=0
Sep 10 17:37:31.169350 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:37:31.169284    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-vz9b7" event={"ID":"dcd0f9c6-c02f-437a-afe7-9dd75c827467","Type":"ContainerDied","Data":"9467c75c1eb928aece0cf1d260acfdced42ed84d549f837ef591cb02d5964e7b"}
Sep 10 17:37:31.750798 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:37:31.750756    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/telemeter-client-ddd5cd65c-rc8wq"]
Sep 10 17:37:31.754162 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:37:31.754131    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/telemeter-client-ddd5cd65c-rc8wq"
Sep 10 17:37:31.756379 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:37:31.756353    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"federate-client-certs\""
Sep 10 17:37:31.756672 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:37:31.756632    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-tls\""
Sep 10 17:37:31.756672 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:37:31.756662    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-dockercfg-fglzv\""
Sep 10 17:37:31.756848 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:37:31.756671    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client\""
Sep 10 17:37:31.756848 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:37:31.756689    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemeter-client-serving-certs-ca-bundle\""
Sep 10 17:37:31.756848 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:37:31.756749    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-kube-rbac-proxy-config\""
Sep 10 17:37:31.762385 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:37:31.762307    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemeter-trusted-ca-bundle-8i12ta5c71j38\""
Sep 10 17:37:31.766447 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:37:31.766385    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/telemeter-client-ddd5cd65c-rc8wq"]
Sep 10 17:37:31.864921 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:37:31.864879    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/fac9f077-8067-42f7-8930-0f28f6cb7ba6-telemeter-trusted-ca-bundle\") pod \"telemeter-client-ddd5cd65c-rc8wq\" (UID: \"fac9f077-8067-42f7-8930-0f28f6cb7ba6\") " pod="openshift-monitoring/telemeter-client-ddd5cd65c-rc8wq"
Sep 10 17:37:31.865368 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:37:31.864943    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/fac9f077-8067-42f7-8930-0f28f6cb7ba6-secret-telemeter-client\") pod \"telemeter-client-ddd5cd65c-rc8wq\" (UID: \"fac9f077-8067-42f7-8930-0f28f6cb7ba6\") " pod="openshift-monitoring/telemeter-client-ddd5cd65c-rc8wq"
Sep 10 17:37:31.865368 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:37:31.865038    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/fac9f077-8067-42f7-8930-0f28f6cb7ba6-telemeter-client-tls\") pod \"telemeter-client-ddd5cd65c-rc8wq\" (UID: \"fac9f077-8067-42f7-8930-0f28f6cb7ba6\") " pod="openshift-monitoring/telemeter-client-ddd5cd65c-rc8wq"
Sep 10 17:37:31.865368 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:37:31.865102    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/fac9f077-8067-42f7-8930-0f28f6cb7ba6-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-ddd5cd65c-rc8wq\" (UID: \"fac9f077-8067-42f7-8930-0f28f6cb7ba6\") " pod="openshift-monitoring/telemeter-client-ddd5cd65c-rc8wq"
Sep 10 17:37:31.865368 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:37:31.865161    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/fac9f077-8067-42f7-8930-0f28f6cb7ba6-serving-certs-ca-bundle\") pod \"telemeter-client-ddd5cd65c-rc8wq\" (UID: \"fac9f077-8067-42f7-8930-0f28f6cb7ba6\") " pod="openshift-monitoring/telemeter-client-ddd5cd65c-rc8wq"
Sep 10 17:37:31.865368 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:37:31.865217    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/fac9f077-8067-42f7-8930-0f28f6cb7ba6-metrics-client-ca\") pod \"telemeter-client-ddd5cd65c-rc8wq\" (UID: \"fac9f077-8067-42f7-8930-0f28f6cb7ba6\") " pod="openshift-monitoring/telemeter-client-ddd5cd65c-rc8wq"
Sep 10 17:37:31.865368 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:37:31.865248    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2xcvn\" (UniqueName: \"kubernetes.io/projected/fac9f077-8067-42f7-8930-0f28f6cb7ba6-kube-api-access-2xcvn\") pod \"telemeter-client-ddd5cd65c-rc8wq\" (UID: \"fac9f077-8067-42f7-8930-0f28f6cb7ba6\") " pod="openshift-monitoring/telemeter-client-ddd5cd65c-rc8wq"
Sep 10 17:37:31.865368 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:37:31.865296    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/fac9f077-8067-42f7-8930-0f28f6cb7ba6-federate-client-tls\") pod \"telemeter-client-ddd5cd65c-rc8wq\" (UID: \"fac9f077-8067-42f7-8930-0f28f6cb7ba6\") " pod="openshift-monitoring/telemeter-client-ddd5cd65c-rc8wq"
Sep 10 17:37:31.966177 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:37:31.966138    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/fac9f077-8067-42f7-8930-0f28f6cb7ba6-telemeter-client-tls\") pod \"telemeter-client-ddd5cd65c-rc8wq\" (UID: \"fac9f077-8067-42f7-8930-0f28f6cb7ba6\") " pod="openshift-monitoring/telemeter-client-ddd5cd65c-rc8wq"
Sep 10 17:37:31.966365 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:37:31.966202    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/fac9f077-8067-42f7-8930-0f28f6cb7ba6-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-ddd5cd65c-rc8wq\" (UID: \"fac9f077-8067-42f7-8930-0f28f6cb7ba6\") " pod="openshift-monitoring/telemeter-client-ddd5cd65c-rc8wq"
Sep 10 17:37:31.966365 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:37:31.966248    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/fac9f077-8067-42f7-8930-0f28f6cb7ba6-serving-certs-ca-bundle\") pod \"telemeter-client-ddd5cd65c-rc8wq\" (UID: \"fac9f077-8067-42f7-8930-0f28f6cb7ba6\") " pod="openshift-monitoring/telemeter-client-ddd5cd65c-rc8wq"
Sep 10 17:37:31.966365 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:37:31.966280    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/fac9f077-8067-42f7-8930-0f28f6cb7ba6-metrics-client-ca\") pod \"telemeter-client-ddd5cd65c-rc8wq\" (UID: \"fac9f077-8067-42f7-8930-0f28f6cb7ba6\") " pod="openshift-monitoring/telemeter-client-ddd5cd65c-rc8wq"
Sep 10 17:37:31.966365 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:37:31.966307    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2xcvn\" (UniqueName: \"kubernetes.io/projected/fac9f077-8067-42f7-8930-0f28f6cb7ba6-kube-api-access-2xcvn\") pod \"telemeter-client-ddd5cd65c-rc8wq\" (UID: \"fac9f077-8067-42f7-8930-0f28f6cb7ba6\") " pod="openshift-monitoring/telemeter-client-ddd5cd65c-rc8wq"
Sep 10 17:37:31.966365 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:37:31.966332    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/fac9f077-8067-42f7-8930-0f28f6cb7ba6-federate-client-tls\") pod \"telemeter-client-ddd5cd65c-rc8wq\" (UID: \"fac9f077-8067-42f7-8930-0f28f6cb7ba6\") " pod="openshift-monitoring/telemeter-client-ddd5cd65c-rc8wq"
Sep 10 17:37:31.966365 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:37:31.966356    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/fac9f077-8067-42f7-8930-0f28f6cb7ba6-telemeter-trusted-ca-bundle\") pod \"telemeter-client-ddd5cd65c-rc8wq\" (UID: \"fac9f077-8067-42f7-8930-0f28f6cb7ba6\") " pod="openshift-monitoring/telemeter-client-ddd5cd65c-rc8wq"
Sep 10 17:37:31.966681 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:37:31.966407    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/fac9f077-8067-42f7-8930-0f28f6cb7ba6-secret-telemeter-client\") pod \"telemeter-client-ddd5cd65c-rc8wq\" (UID: \"fac9f077-8067-42f7-8930-0f28f6cb7ba6\") " pod="openshift-monitoring/telemeter-client-ddd5cd65c-rc8wq"
Sep 10 17:37:31.967173 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:37:31.967142    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/fac9f077-8067-42f7-8930-0f28f6cb7ba6-serving-certs-ca-bundle\") pod \"telemeter-client-ddd5cd65c-rc8wq\" (UID: \"fac9f077-8067-42f7-8930-0f28f6cb7ba6\") " pod="openshift-monitoring/telemeter-client-ddd5cd65c-rc8wq"
Sep 10 17:37:31.967352 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:37:31.967324    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/fac9f077-8067-42f7-8930-0f28f6cb7ba6-telemeter-trusted-ca-bundle\") pod \"telemeter-client-ddd5cd65c-rc8wq\" (UID: \"fac9f077-8067-42f7-8930-0f28f6cb7ba6\") " pod="openshift-monitoring/telemeter-client-ddd5cd65c-rc8wq"
Sep 10 17:37:31.967432 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:37:31.967358    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/fac9f077-8067-42f7-8930-0f28f6cb7ba6-metrics-client-ca\") pod \"telemeter-client-ddd5cd65c-rc8wq\" (UID: \"fac9f077-8067-42f7-8930-0f28f6cb7ba6\") " pod="openshift-monitoring/telemeter-client-ddd5cd65c-rc8wq"
Sep 10 17:37:31.969374 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:37:31.969349    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/fac9f077-8067-42f7-8930-0f28f6cb7ba6-telemeter-client-tls\") pod \"telemeter-client-ddd5cd65c-rc8wq\" (UID: \"fac9f077-8067-42f7-8930-0f28f6cb7ba6\") " pod="openshift-monitoring/telemeter-client-ddd5cd65c-rc8wq"
Sep 10 17:37:31.969540 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:37:31.969423    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/fac9f077-8067-42f7-8930-0f28f6cb7ba6-federate-client-tls\") pod \"telemeter-client-ddd5cd65c-rc8wq\" (UID: \"fac9f077-8067-42f7-8930-0f28f6cb7ba6\") " pod="openshift-monitoring/telemeter-client-ddd5cd65c-rc8wq"
Sep 10 17:37:31.969540 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:37:31.969504    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/fac9f077-8067-42f7-8930-0f28f6cb7ba6-secret-telemeter-client\") pod \"telemeter-client-ddd5cd65c-rc8wq\" (UID: \"fac9f077-8067-42f7-8930-0f28f6cb7ba6\") " pod="openshift-monitoring/telemeter-client-ddd5cd65c-rc8wq"
Sep 10 17:37:31.969694 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:37:31.969589    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/fac9f077-8067-42f7-8930-0f28f6cb7ba6-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-ddd5cd65c-rc8wq\" (UID: \"fac9f077-8067-42f7-8930-0f28f6cb7ba6\") " pod="openshift-monitoring/telemeter-client-ddd5cd65c-rc8wq"
Sep 10 17:37:31.975017 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:37:31.974997    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2xcvn\" (UniqueName: \"kubernetes.io/projected/fac9f077-8067-42f7-8930-0f28f6cb7ba6-kube-api-access-2xcvn\") pod \"telemeter-client-ddd5cd65c-rc8wq\" (UID: \"fac9f077-8067-42f7-8930-0f28f6cb7ba6\") " pod="openshift-monitoring/telemeter-client-ddd5cd65c-rc8wq"
Sep 10 17:37:32.066092 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:37:32.066010    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/telemeter-client-ddd5cd65c-rc8wq"
Sep 10 17:37:32.563232 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:37:32.563092    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/telemeter-client-ddd5cd65c-rc8wq"]
Sep 10 17:37:32.567603 ip-10-0-141-180 kubenswrapper[2581]: W0910 17:37:32.567573    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfac9f077_8067_42f7_8930_0f28f6cb7ba6.slice/crio-801e13ba2367600a990681d69a3f378ea8d3e42433656f1cc7116754afc6e9ff WatchSource:0}: Error finding container 801e13ba2367600a990681d69a3f378ea8d3e42433656f1cc7116754afc6e9ff: Status 404 returned error can't find the container with id 801e13ba2367600a990681d69a3f378ea8d3e42433656f1cc7116754afc6e9ff
Sep 10 17:37:33.183842 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:37:33.183780    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-vz9b7" event={"ID":"dcd0f9c6-c02f-437a-afe7-9dd75c827467","Type":"ContainerStarted","Data":"222d60d7ba9a47412af6dfdf6f1a0811adeff6cf3f565e034cd7811ee384614d"}
Sep 10 17:37:33.184296 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:37:33.183854    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-vz9b7" event={"ID":"dcd0f9c6-c02f-437a-afe7-9dd75c827467","Type":"ContainerStarted","Data":"c622efe3ed32f4c9ead9a8b2553d9c3678444e84ac135895dd6dcce80ddd47f0"}
Sep 10 17:37:33.185715 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:37:33.185679    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-ddd5cd65c-rc8wq" event={"ID":"fac9f077-8067-42f7-8930-0f28f6cb7ba6","Type":"ContainerStarted","Data":"801e13ba2367600a990681d69a3f378ea8d3e42433656f1cc7116754afc6e9ff"}
Sep 10 17:37:33.187811 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:37:33.187778    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-7ptnb" event={"ID":"766766d2-fa56-49c2-9928-b50cbbf9173a","Type":"ContainerStarted","Data":"bb56d3fb0574505784336be94011f0dd59a10074260cba00d86ab22c46923314"}
Sep 10 17:37:33.188344 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:37:33.188317    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-7ptnb"
Sep 10 17:37:33.190856 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:37:33.190825    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-2t2rd" event={"ID":"dee7aebb-2ba6-4bfe-8fb4-371bcc4e56b1","Type":"ContainerStarted","Data":"8cdc8c57fc2bfaa98104a4f1b4c41f0d1f1b8cc86247a438f401ae0143b19856"}
Sep 10 17:37:33.193707 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:37:33.193685    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-74cb57df7-9hv2h" event={"ID":"df97b1f6-ca1d-481e-892c-68579c1dbfc4","Type":"ContainerStarted","Data":"1e826dafcd5961a05a4b618a82aff9ed5c83b6d910b699a1065e1267c2268eb2"}
Sep 10 17:37:33.211933 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:37:33.211845    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-vz9b7" podStartSLOduration=5.081077982 podStartE2EDuration="7.211824363s" totalImagesPullingTime="2.017578064s" totalInitContainerRuntime="113.168317ms" isStatefulPod=true podCreationTimestamp="2026-09-10 17:37:26 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 17:37:33.21017448 +0000 UTC m=+72.864248402" watchObservedRunningTime="2026-09-10 17:37:33.211824363 +0000 UTC m=+72.865898472"
Sep 10 17:37:33.233393 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:37:33.233226    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/kube-state-metrics-74cb57df7-9hv2h" podStartSLOduration=5.489446241 podStartE2EDuration="7.233203095s" totalImagesPullingTime="1.743756854s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 17:37:26 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 17:37:33.232726844 +0000 UTC m=+72.886800764" watchObservedRunningTime="2026-09-10 17:37:33.233203095 +0000 UTC m=+72.887277001"
Sep 10 17:37:33.255354 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:37:33.255228    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-7ptnb" podStartSLOduration=68.064324356 podStartE2EDuration="1m13.255205899s" totalImagesPullingTime="5.190881543s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 17:36:20 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 17:37:33.254940857 +0000 UTC m=+72.909014785" watchObservedRunningTime="2026-09-10 17:37:33.255205899 +0000 UTC m=+72.909279804"
Sep 10 17:37:33.276186 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:37:33.276138    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-2t2rd" podStartSLOduration=67.032724902 podStartE2EDuration="1m12.276123281s" totalImagesPullingTime="5.243398379s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-10 17:36:21 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 17:37:33.27537598 +0000 UTC m=+72.929449898" watchObservedRunningTime="2026-09-10 17:37:33.276123281 +0000 UTC m=+72.930197185"
Sep 10 17:37:35.202124 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:37:35.202028    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-568869598d-ns9hc" event={"ID":"6e66367a-0612-42be-948e-ebf346d9820c","Type":"ContainerStarted","Data":"0796bff669341fcfd0cc6c0d75f731ff6f34ffaabe2d4aff0f06acf5d16bbe8f"}
Sep 10 17:37:35.202124 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:37:35.202108    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-568869598d-ns9hc" event={"ID":"6e66367a-0612-42be-948e-ebf346d9820c","Type":"ContainerStarted","Data":"3d29e6ba03e54c5e8b3998ea063f67e4d9e7a7f1efcc31cf70c2945fa4cbfd79"}
Sep 10 17:37:35.202520 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:37:35.202136    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-568869598d-ns9hc" event={"ID":"6e66367a-0612-42be-948e-ebf346d9820c","Type":"ContainerStarted","Data":"d9c57e862da7d2db6c0687a03ae6fb178ab498d448425f1c6c1151f91c9adfe2"}
Sep 10 17:37:35.203556 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:37:35.203529    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-ddd5cd65c-rc8wq" event={"ID":"fac9f077-8067-42f7-8930-0f28f6cb7ba6","Type":"ContainerStarted","Data":"063c95241b0fad746ba53bb9f1734bbe9223259533be247f40ad0159715413b8"}
Sep 10 17:37:36.212163 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:37:36.212127    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-ddd5cd65c-rc8wq" event={"ID":"fac9f077-8067-42f7-8930-0f28f6cb7ba6","Type":"ContainerStarted","Data":"f52b63aca872e6e4a477c14890ee9d91ad78b74c057f997cccb7e5a141a11460"}
Sep 10 17:37:36.212520 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:37:36.212246    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-ddd5cd65c-rc8wq" event={"ID":"fac9f077-8067-42f7-8930-0f28f6cb7ba6","Type":"ContainerStarted","Data":"1d0d616280714b392896afe3912184e39144d2036b2269363f00bb8d39e8254e"}
Sep 10 17:37:36.216140 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:37:36.216115    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-568869598d-ns9hc" event={"ID":"6e66367a-0612-42be-948e-ebf346d9820c","Type":"ContainerStarted","Data":"28319e51260075160ee86a97bff3ea9fac24b3a69777818df640f554546620e9"}
Sep 10 17:37:36.216245 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:37:36.216182    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-568869598d-ns9hc" event={"ID":"6e66367a-0612-42be-948e-ebf346d9820c","Type":"ContainerStarted","Data":"fb04501ed62fe8ec2faccd038d4166bd41f23bd7f0fed4307941ac3492e14f49"}
Sep 10 17:37:36.216245 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:37:36.216209    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-568869598d-ns9hc" event={"ID":"6e66367a-0612-42be-948e-ebf346d9820c","Type":"ContainerStarted","Data":"2039ee13487d3e392a04f031b3897ed82fceb784e5ab211de74249c5d84a285f"}
Sep 10 17:37:36.216479 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:37:36.216464    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/thanos-querier-568869598d-ns9hc"
Sep 10 17:37:36.236429 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:37:36.236352    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/telemeter-client-ddd5cd65c-rc8wq" podStartSLOduration=1.880740872 podStartE2EDuration="5.236338772s" totalImagesPullingTime="3.3555979s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 17:37:31 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 17:37:36.235078287 +0000 UTC m=+75.889152190" watchObservedRunningTime="2026-09-10 17:37:36.236338772 +0000 UTC m=+75.890412666"
Sep 10 17:37:36.261763 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:37:36.261713    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/thanos-querier-568869598d-ns9hc" podStartSLOduration=3.721522094 podStartE2EDuration="8.261697778s" totalImagesPullingTime="4.540175684s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 17:37:28 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 17:37:36.260776635 +0000 UTC m=+75.914850549" watchObservedRunningTime="2026-09-10 17:37:36.261697778 +0000 UTC m=+75.915771699"
Sep 10 17:37:37.098498 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:37:37.098464    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-86f5bb8d74-5kxl4"
Sep 10 17:37:42.228846 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:37:42.228811    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/thanos-querier-568869598d-ns9hc"
Sep 10 17:38:05.207137 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:38:05.207029    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-7ptnb"
Sep 10 17:38:48.864554 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:38:48.864520    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Sep 10 17:38:48.867476 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:38:48.867453    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Sep 10 17:38:48.870176 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:38:48.870153    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-generated\""
Sep 10 17:38:48.870335 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:38:48.870313    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls-assets-0\""
Sep 10 17:38:48.870466 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:38:48.870154    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-dockercfg-crtdc\""
Sep 10 17:38:48.870543 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:38:48.870181    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-web\""
Sep 10 17:38:48.870596 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:38:48.870239    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls\""
Sep 10 17:38:48.870681 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:38:48.870155    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-metric\""
Sep 10 17:38:48.870681 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:38:48.870378    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-cluster-tls-config\""
Sep 10 17:38:48.870787 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:38:48.870392    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-web-config\""
Sep 10 17:38:48.872883 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:38:48.872861    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy\""
Sep 10 17:38:48.876715 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:38:48.876696    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"alertmanager-trusted-ca-bundle\""
Sep 10 17:38:48.884345 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:38:48.884320    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Sep 10 17:38:48.933487 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:38:48.933454    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/cddd9715-a16e-4393-a78f-4ece06346420-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"cddd9715-a16e-4393-a78f-4ece06346420\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 10 17:38:48.933487 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:38:48.933489    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/cddd9715-a16e-4393-a78f-4ece06346420-config-volume\") pod \"alertmanager-main-0\" (UID: \"cddd9715-a16e-4393-a78f-4ece06346420\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 10 17:38:48.933689 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:38:48.933508    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/cddd9715-a16e-4393-a78f-4ece06346420-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"cddd9715-a16e-4393-a78f-4ece06346420\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 10 17:38:48.933689 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:38:48.933559    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/cddd9715-a16e-4393-a78f-4ece06346420-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"cddd9715-a16e-4393-a78f-4ece06346420\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 10 17:38:48.933689 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:38:48.933606    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/cddd9715-a16e-4393-a78f-4ece06346420-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"cddd9715-a16e-4393-a78f-4ece06346420\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 10 17:38:48.933689 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:38:48.933632    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/cddd9715-a16e-4393-a78f-4ece06346420-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"cddd9715-a16e-4393-a78f-4ece06346420\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 10 17:38:48.933689 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:38:48.933677    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-97wgt\" (UniqueName: \"kubernetes.io/projected/cddd9715-a16e-4393-a78f-4ece06346420-kube-api-access-97wgt\") pod \"alertmanager-main-0\" (UID: \"cddd9715-a16e-4393-a78f-4ece06346420\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 10 17:38:48.933840 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:38:48.933717    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/cddd9715-a16e-4393-a78f-4ece06346420-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"cddd9715-a16e-4393-a78f-4ece06346420\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 10 17:38:48.933840 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:38:48.933733    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/cddd9715-a16e-4393-a78f-4ece06346420-tls-assets\") pod \"alertmanager-main-0\" (UID: \"cddd9715-a16e-4393-a78f-4ece06346420\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 10 17:38:48.933840 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:38:48.933764    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/cddd9715-a16e-4393-a78f-4ece06346420-web-config\") pod \"alertmanager-main-0\" (UID: \"cddd9715-a16e-4393-a78f-4ece06346420\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 10 17:38:48.933840 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:38:48.933783    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/cddd9715-a16e-4393-a78f-4ece06346420-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"cddd9715-a16e-4393-a78f-4ece06346420\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 10 17:38:48.933840 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:38:48.933798    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/cddd9715-a16e-4393-a78f-4ece06346420-config-out\") pod \"alertmanager-main-0\" (UID: \"cddd9715-a16e-4393-a78f-4ece06346420\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 10 17:38:48.933840 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:38:48.933813    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/cddd9715-a16e-4393-a78f-4ece06346420-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"cddd9715-a16e-4393-a78f-4ece06346420\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 10 17:38:49.035132 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:38:49.035097    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/cddd9715-a16e-4393-a78f-4ece06346420-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"cddd9715-a16e-4393-a78f-4ece06346420\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 10 17:38:49.035132 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:38:49.035135    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/cddd9715-a16e-4393-a78f-4ece06346420-config-volume\") pod \"alertmanager-main-0\" (UID: \"cddd9715-a16e-4393-a78f-4ece06346420\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 10 17:38:49.035335 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:38:49.035152    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/cddd9715-a16e-4393-a78f-4ece06346420-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"cddd9715-a16e-4393-a78f-4ece06346420\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 10 17:38:49.035335 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:38:49.035171    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/cddd9715-a16e-4393-a78f-4ece06346420-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"cddd9715-a16e-4393-a78f-4ece06346420\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 10 17:38:49.035335 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:38:49.035202    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/cddd9715-a16e-4393-a78f-4ece06346420-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"cddd9715-a16e-4393-a78f-4ece06346420\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 10 17:38:49.035490 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:38:49.035336    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/cddd9715-a16e-4393-a78f-4ece06346420-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"cddd9715-a16e-4393-a78f-4ece06346420\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 10 17:38:49.035490 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:38:49.035385    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-97wgt\" (UniqueName: \"kubernetes.io/projected/cddd9715-a16e-4393-a78f-4ece06346420-kube-api-access-97wgt\") pod \"alertmanager-main-0\" (UID: \"cddd9715-a16e-4393-a78f-4ece06346420\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 10 17:38:49.035490 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:38:49.035427    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/cddd9715-a16e-4393-a78f-4ece06346420-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"cddd9715-a16e-4393-a78f-4ece06346420\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 10 17:38:49.035490 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:38:49.035457    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/cddd9715-a16e-4393-a78f-4ece06346420-tls-assets\") pod \"alertmanager-main-0\" (UID: \"cddd9715-a16e-4393-a78f-4ece06346420\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 10 17:38:49.035754 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:38:49.035506    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/cddd9715-a16e-4393-a78f-4ece06346420-web-config\") pod \"alertmanager-main-0\" (UID: \"cddd9715-a16e-4393-a78f-4ece06346420\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 10 17:38:49.035754 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:38:49.035534    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/cddd9715-a16e-4393-a78f-4ece06346420-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"cddd9715-a16e-4393-a78f-4ece06346420\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 10 17:38:49.035754 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:38:49.035560    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/cddd9715-a16e-4393-a78f-4ece06346420-config-out\") pod \"alertmanager-main-0\" (UID: \"cddd9715-a16e-4393-a78f-4ece06346420\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 10 17:38:49.035754 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:38:49.035584    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/cddd9715-a16e-4393-a78f-4ece06346420-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"cddd9715-a16e-4393-a78f-4ece06346420\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 10 17:38:49.036456 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:38:49.036426    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/cddd9715-a16e-4393-a78f-4ece06346420-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"cddd9715-a16e-4393-a78f-4ece06346420\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 10 17:38:49.037231 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:38:49.037206    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/cddd9715-a16e-4393-a78f-4ece06346420-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"cddd9715-a16e-4393-a78f-4ece06346420\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 10 17:38:49.037630 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:38:49.037611    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/cddd9715-a16e-4393-a78f-4ece06346420-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"cddd9715-a16e-4393-a78f-4ece06346420\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 10 17:38:49.038369 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:38:49.038338    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/cddd9715-a16e-4393-a78f-4ece06346420-config-volume\") pod \"alertmanager-main-0\" (UID: \"cddd9715-a16e-4393-a78f-4ece06346420\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 10 17:38:49.038461 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:38:49.038393    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/cddd9715-a16e-4393-a78f-4ece06346420-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"cddd9715-a16e-4393-a78f-4ece06346420\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 10 17:38:49.038461 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:38:49.038424    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/cddd9715-a16e-4393-a78f-4ece06346420-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"cddd9715-a16e-4393-a78f-4ece06346420\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 10 17:38:49.038581 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:38:49.038479    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/cddd9715-a16e-4393-a78f-4ece06346420-web-config\") pod \"alertmanager-main-0\" (UID: \"cddd9715-a16e-4393-a78f-4ece06346420\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 10 17:38:49.038581 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:38:49.038531    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/cddd9715-a16e-4393-a78f-4ece06346420-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"cddd9715-a16e-4393-a78f-4ece06346420\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 10 17:38:49.038718 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:38:49.038698    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/cddd9715-a16e-4393-a78f-4ece06346420-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"cddd9715-a16e-4393-a78f-4ece06346420\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 10 17:38:49.039011 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:38:49.038991    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/cddd9715-a16e-4393-a78f-4ece06346420-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"cddd9715-a16e-4393-a78f-4ece06346420\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 10 17:38:49.039808 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:38:49.039788    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/cddd9715-a16e-4393-a78f-4ece06346420-config-out\") pod \"alertmanager-main-0\" (UID: \"cddd9715-a16e-4393-a78f-4ece06346420\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 10 17:38:49.040248 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:38:49.040230    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/cddd9715-a16e-4393-a78f-4ece06346420-tls-assets\") pod \"alertmanager-main-0\" (UID: \"cddd9715-a16e-4393-a78f-4ece06346420\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 10 17:38:49.044255 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:38:49.044236    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-97wgt\" (UniqueName: \"kubernetes.io/projected/cddd9715-a16e-4393-a78f-4ece06346420-kube-api-access-97wgt\") pod \"alertmanager-main-0\" (UID: \"cddd9715-a16e-4393-a78f-4ece06346420\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 10 17:38:49.178115 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:38:49.178047    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Sep 10 17:38:49.453561 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:38:49.453486    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Sep 10 17:38:49.456428 ip-10-0-141-180 kubenswrapper[2581]: W0910 17:38:49.456400    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcddd9715_a16e_4393_a78f_4ece06346420.slice/crio-94b94fca63f27a4a78a19028d1713a7788ea4bb0a289ebf4232620a391322818 WatchSource:0}: Error finding container 94b94fca63f27a4a78a19028d1713a7788ea4bb0a289ebf4232620a391322818: Status 404 returned error can't find the container with id 94b94fca63f27a4a78a19028d1713a7788ea4bb0a289ebf4232620a391322818
Sep 10 17:38:49.477925 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:38:49.477899    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"cddd9715-a16e-4393-a78f-4ece06346420","Type":"ContainerStarted","Data":"94b94fca63f27a4a78a19028d1713a7788ea4bb0a289ebf4232620a391322818"}
Sep 10 17:38:50.482767 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:38:50.482733    2581 generic.go:356] "Generic (PLEG): container finished" podID="cddd9715-a16e-4393-a78f-4ece06346420" containerID="006bc79fa818834774c5bb1060c04551e3bbb1730e3c7f5a09e5682cd0f4a1c2" exitCode=0
Sep 10 17:38:50.483131 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:38:50.482822    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"cddd9715-a16e-4393-a78f-4ece06346420","Type":"ContainerDied","Data":"006bc79fa818834774c5bb1060c04551e3bbb1730e3c7f5a09e5682cd0f4a1c2"}
Sep 10 17:38:52.495168 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:38:52.495141    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"cddd9715-a16e-4393-a78f-4ece06346420","Type":"ContainerStarted","Data":"81fddc4dd6672fbfc60b4337b5efc54263224f334db88b6ee85a028474dff0ff"}
Sep 10 17:38:52.495537 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:38:52.495228    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"cddd9715-a16e-4393-a78f-4ece06346420","Type":"ContainerStarted","Data":"7cabcd0f6efa720f514c7ec506bfa7e0b2bffcfa732b82f1049857524b1df1fa"}
Sep 10 17:38:52.495537 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:38:52.495255    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"cddd9715-a16e-4393-a78f-4ece06346420","Type":"ContainerStarted","Data":"cd4c844e3a3719495add324f1bf4e68406825609e3aa5e0fcc33fc79545e9fd0"}
Sep 10 17:38:52.495537 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:38:52.495278    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"cddd9715-a16e-4393-a78f-4ece06346420","Type":"ContainerStarted","Data":"dd793ed82ef915b87c8b4769a7a7ba5599df8740c638ff6e9ad25ef888645351"}
Sep 10 17:38:52.495537 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:38:52.495300    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"cddd9715-a16e-4393-a78f-4ece06346420","Type":"ContainerStarted","Data":"6ce2b2abfae3db079e84aed7fb8385d7d781473cf8237518e8e6f75a8152c22e"}
Sep 10 17:38:53.502055 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:38:53.502020    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"cddd9715-a16e-4393-a78f-4ece06346420","Type":"ContainerStarted","Data":"0b5959a83fa6948307309b75ae19ccf3b456587502f3366ad642c0d790f7b615"}
Sep 10 17:38:53.528813 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:38:53.528765    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/alertmanager-main-0" podStartSLOduration=3.803102482 podStartE2EDuration="5.528748914s" totalImagesPullingTime="1.710412497s" totalInitContainerRuntime="15.233935ms" isStatefulPod=false podCreationTimestamp="2026-09-10 17:38:48 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 17:38:53.527457442 +0000 UTC m=+153.181531369" watchObservedRunningTime="2026-09-10 17:38:53.528748914 +0000 UTC m=+153.182822817"
Sep 10 17:39:25.756462 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:39:25.756425    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e6754092f4m"]
Sep 10 17:39:25.762141 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:39:25.762108    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e6754092f4m"
Sep 10 17:39:25.765050 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:39:25.765026    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\""
Sep 10 17:39:25.765885 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:39:25.765821    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\""
Sep 10 17:39:25.766017 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:39:25.765837    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-2mcxw\""
Sep 10 17:39:25.767779 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:39:25.767753    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e6754092f4m"]
Sep 10 17:39:25.831728 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:39:25.831699    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/b7a7cdf0-e258-4050-8013-4b4d79fa1eee-util\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e6754092f4m\" (UID: \"b7a7cdf0-e258-4050-8013-4b4d79fa1eee\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e6754092f4m"
Sep 10 17:39:25.831866 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:39:25.831768    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/b7a7cdf0-e258-4050-8013-4b4d79fa1eee-bundle\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e6754092f4m\" (UID: \"b7a7cdf0-e258-4050-8013-4b4d79fa1eee\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e6754092f4m"
Sep 10 17:39:25.831866 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:39:25.831831    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dzztw\" (UniqueName: \"kubernetes.io/projected/b7a7cdf0-e258-4050-8013-4b4d79fa1eee-kube-api-access-dzztw\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e6754092f4m\" (UID: \"b7a7cdf0-e258-4050-8013-4b4d79fa1eee\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e6754092f4m"
Sep 10 17:39:25.932928 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:39:25.932899    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/b7a7cdf0-e258-4050-8013-4b4d79fa1eee-bundle\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e6754092f4m\" (UID: \"b7a7cdf0-e258-4050-8013-4b4d79fa1eee\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e6754092f4m"
Sep 10 17:39:25.933073 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:39:25.932943    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dzztw\" (UniqueName: \"kubernetes.io/projected/b7a7cdf0-e258-4050-8013-4b4d79fa1eee-kube-api-access-dzztw\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e6754092f4m\" (UID: \"b7a7cdf0-e258-4050-8013-4b4d79fa1eee\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e6754092f4m"
Sep 10 17:39:25.933073 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:39:25.932979    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/b7a7cdf0-e258-4050-8013-4b4d79fa1eee-util\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e6754092f4m\" (UID: \"b7a7cdf0-e258-4050-8013-4b4d79fa1eee\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e6754092f4m"
Sep 10 17:39:25.933322 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:39:25.933299    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/b7a7cdf0-e258-4050-8013-4b4d79fa1eee-bundle\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e6754092f4m\" (UID: \"b7a7cdf0-e258-4050-8013-4b4d79fa1eee\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e6754092f4m"
Sep 10 17:39:25.933381 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:39:25.933347    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/b7a7cdf0-e258-4050-8013-4b4d79fa1eee-util\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e6754092f4m\" (UID: \"b7a7cdf0-e258-4050-8013-4b4d79fa1eee\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e6754092f4m"
Sep 10 17:39:25.941672 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:39:25.941627    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dzztw\" (UniqueName: \"kubernetes.io/projected/b7a7cdf0-e258-4050-8013-4b4d79fa1eee-kube-api-access-dzztw\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e6754092f4m\" (UID: \"b7a7cdf0-e258-4050-8013-4b4d79fa1eee\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e6754092f4m"
Sep 10 17:39:26.072194 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:39:26.072127    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e6754092f4m"
Sep 10 17:39:26.344416 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:39:26.344337    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e6754092f4m"]
Sep 10 17:39:26.347738 ip-10-0-141-180 kubenswrapper[2581]: W0910 17:39:26.347711    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb7a7cdf0_e258_4050_8013_4b4d79fa1eee.slice/crio-980b2b087149691e822e535f7ca59eceea419e7c461cc7f14eff13e05250e30c WatchSource:0}: Error finding container 980b2b087149691e822e535f7ca59eceea419e7c461cc7f14eff13e05250e30c: Status 404 returned error can't find the container with id 980b2b087149691e822e535f7ca59eceea419e7c461cc7f14eff13e05250e30c
Sep 10 17:39:26.625038 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:39:26.624953    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e6754092f4m" event={"ID":"b7a7cdf0-e258-4050-8013-4b4d79fa1eee","Type":"ContainerStarted","Data":"980b2b087149691e822e535f7ca59eceea419e7c461cc7f14eff13e05250e30c"}
Sep 10 17:39:34.657217 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:39:34.657182    2581 generic.go:356] "Generic (PLEG): container finished" podID="b7a7cdf0-e258-4050-8013-4b4d79fa1eee" containerID="3bd810ab54e39bdbd6f6f64e519c94f3d5061ffe53991ec2e77168169e5f4c33" exitCode=0
Sep 10 17:39:34.657577 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:39:34.657233    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e6754092f4m" event={"ID":"b7a7cdf0-e258-4050-8013-4b4d79fa1eee","Type":"ContainerDied","Data":"3bd810ab54e39bdbd6f6f64e519c94f3d5061ffe53991ec2e77168169e5f4c33"}
Sep 10 17:39:37.670823 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:39:37.670782    2581 generic.go:356] "Generic (PLEG): container finished" podID="b7a7cdf0-e258-4050-8013-4b4d79fa1eee" containerID="11250a83bdc740cc89119501b33d80087894146d165e317bda9a3f149e5cbde0" exitCode=0
Sep 10 17:39:37.671181 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:39:37.670868    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e6754092f4m" event={"ID":"b7a7cdf0-e258-4050-8013-4b4d79fa1eee","Type":"ContainerDied","Data":"11250a83bdc740cc89119501b33d80087894146d165e317bda9a3f149e5cbde0"}
Sep 10 17:39:44.703603 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:39:44.703517    2581 generic.go:356] "Generic (PLEG): container finished" podID="b7a7cdf0-e258-4050-8013-4b4d79fa1eee" containerID="c5aba90b36fc34cabd220c25bfac7c5ff3099eb14fc8ca616d8e01f78ac8c22e" exitCode=0
Sep 10 17:39:44.703603 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:39:44.703565    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e6754092f4m" event={"ID":"b7a7cdf0-e258-4050-8013-4b4d79fa1eee","Type":"ContainerDied","Data":"c5aba90b36fc34cabd220c25bfac7c5ff3099eb14fc8ca616d8e01f78ac8c22e"}
Sep 10 17:39:45.826781 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:39:45.826758    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e6754092f4m"
Sep 10 17:39:45.911168 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:39:45.911133    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/b7a7cdf0-e258-4050-8013-4b4d79fa1eee-bundle\") pod \"b7a7cdf0-e258-4050-8013-4b4d79fa1eee\" (UID: \"b7a7cdf0-e258-4050-8013-4b4d79fa1eee\") "
Sep 10 17:39:45.911327 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:39:45.911183    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-dzztw\" (UniqueName: \"kubernetes.io/projected/b7a7cdf0-e258-4050-8013-4b4d79fa1eee-kube-api-access-dzztw\") pod \"b7a7cdf0-e258-4050-8013-4b4d79fa1eee\" (UID: \"b7a7cdf0-e258-4050-8013-4b4d79fa1eee\") "
Sep 10 17:39:45.911327 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:39:45.911215    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/b7a7cdf0-e258-4050-8013-4b4d79fa1eee-util\") pod \"b7a7cdf0-e258-4050-8013-4b4d79fa1eee\" (UID: \"b7a7cdf0-e258-4050-8013-4b4d79fa1eee\") "
Sep 10 17:39:45.912011 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:39:45.911980    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b7a7cdf0-e258-4050-8013-4b4d79fa1eee-bundle" (OuterVolumeSpecName: "bundle") pod "b7a7cdf0-e258-4050-8013-4b4d79fa1eee" (UID: "b7a7cdf0-e258-4050-8013-4b4d79fa1eee"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 17:39:45.913380 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:39:45.913347    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b7a7cdf0-e258-4050-8013-4b4d79fa1eee-kube-api-access-dzztw" (OuterVolumeSpecName: "kube-api-access-dzztw") pod "b7a7cdf0-e258-4050-8013-4b4d79fa1eee" (UID: "b7a7cdf0-e258-4050-8013-4b4d79fa1eee"). InnerVolumeSpecName "kube-api-access-dzztw". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 17:39:45.915493 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:39:45.915469    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b7a7cdf0-e258-4050-8013-4b4d79fa1eee-util" (OuterVolumeSpecName: "util") pod "b7a7cdf0-e258-4050-8013-4b4d79fa1eee" (UID: "b7a7cdf0-e258-4050-8013-4b4d79fa1eee"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 17:39:46.012453 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:39:46.012402    2581 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/b7a7cdf0-e258-4050-8013-4b4d79fa1eee-util\") on node \"ip-10-0-141-180.ec2.internal\" DevicePath \"\""
Sep 10 17:39:46.012453 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:39:46.012449    2581 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/b7a7cdf0-e258-4050-8013-4b4d79fa1eee-bundle\") on node \"ip-10-0-141-180.ec2.internal\" DevicePath \"\""
Sep 10 17:39:46.012453 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:39:46.012459    2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-dzztw\" (UniqueName: \"kubernetes.io/projected/b7a7cdf0-e258-4050-8013-4b4d79fa1eee-kube-api-access-dzztw\") on node \"ip-10-0-141-180.ec2.internal\" DevicePath \"\""
Sep 10 17:39:46.712495 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:39:46.712470    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e6754092f4m"
Sep 10 17:39:46.712683 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:39:46.712463    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e6754092f4m" event={"ID":"b7a7cdf0-e258-4050-8013-4b4d79fa1eee","Type":"ContainerDied","Data":"980b2b087149691e822e535f7ca59eceea419e7c461cc7f14eff13e05250e30c"}
Sep 10 17:39:46.712683 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:39:46.712593    2581 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="980b2b087149691e822e535f7ca59eceea419e7c461cc7f14eff13e05250e30c"
Sep 10 17:39:53.784257 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:39:53.784213    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-tt86m"]
Sep 10 17:39:53.784936 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:39:53.784700    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b7a7cdf0-e258-4050-8013-4b4d79fa1eee" containerName="extract"
Sep 10 17:39:53.784936 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:39:53.784720    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="b7a7cdf0-e258-4050-8013-4b4d79fa1eee" containerName="extract"
Sep 10 17:39:53.784936 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:39:53.784765    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b7a7cdf0-e258-4050-8013-4b4d79fa1eee" containerName="util"
Sep 10 17:39:53.784936 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:39:53.784775    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="b7a7cdf0-e258-4050-8013-4b4d79fa1eee" containerName="util"
Sep 10 17:39:53.784936 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:39:53.784784    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b7a7cdf0-e258-4050-8013-4b4d79fa1eee" containerName="pull"
Sep 10 17:39:53.784936 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:39:53.784792    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="b7a7cdf0-e258-4050-8013-4b4d79fa1eee" containerName="pull"
Sep 10 17:39:53.784936 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:39:53.784871    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="b7a7cdf0-e258-4050-8013-4b4d79fa1eee" containerName="extract"
Sep 10 17:39:53.788999 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:39:53.788978    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-tt86m"
Sep 10 17:39:53.792044 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:39:53.792020    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager-operator\"/\"kube-root-ca.crt\""
Sep 10 17:39:53.792152 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:39:53.792103    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager-operator\"/\"openshift-service-ca.crt\""
Sep 10 17:39:53.792337 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:39:53.792322    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager-operator\"/\"cert-manager-operator-controller-manager-dockercfg-z26m4\""
Sep 10 17:39:53.811680 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:39:53.811657    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-tt86m"]
Sep 10 17:39:53.870981 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:39:53.870947    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jvjdc\" (UniqueName: \"kubernetes.io/projected/48aa4f5c-d175-42dd-92df-4c416d5aea9e-kube-api-access-jvjdc\") pod \"cert-manager-operator-controller-manager-55c444bf86-tt86m\" (UID: \"48aa4f5c-d175-42dd-92df-4c416d5aea9e\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-tt86m"
Sep 10 17:39:53.871133 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:39:53.871011    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/48aa4f5c-d175-42dd-92df-4c416d5aea9e-tmp\") pod \"cert-manager-operator-controller-manager-55c444bf86-tt86m\" (UID: \"48aa4f5c-d175-42dd-92df-4c416d5aea9e\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-tt86m"
Sep 10 17:39:53.971775 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:39:53.971738    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/48aa4f5c-d175-42dd-92df-4c416d5aea9e-tmp\") pod \"cert-manager-operator-controller-manager-55c444bf86-tt86m\" (UID: \"48aa4f5c-d175-42dd-92df-4c416d5aea9e\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-tt86m"
Sep 10 17:39:53.971775 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:39:53.971778    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jvjdc\" (UniqueName: \"kubernetes.io/projected/48aa4f5c-d175-42dd-92df-4c416d5aea9e-kube-api-access-jvjdc\") pod \"cert-manager-operator-controller-manager-55c444bf86-tt86m\" (UID: \"48aa4f5c-d175-42dd-92df-4c416d5aea9e\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-tt86m"
Sep 10 17:39:53.972083 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:39:53.972065    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/48aa4f5c-d175-42dd-92df-4c416d5aea9e-tmp\") pod \"cert-manager-operator-controller-manager-55c444bf86-tt86m\" (UID: \"48aa4f5c-d175-42dd-92df-4c416d5aea9e\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-tt86m"
Sep 10 17:39:53.979324 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:39:53.979302    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jvjdc\" (UniqueName: \"kubernetes.io/projected/48aa4f5c-d175-42dd-92df-4c416d5aea9e-kube-api-access-jvjdc\") pod \"cert-manager-operator-controller-manager-55c444bf86-tt86m\" (UID: \"48aa4f5c-d175-42dd-92df-4c416d5aea9e\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-tt86m"
Sep 10 17:39:54.098575 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:39:54.098477    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-tt86m"
Sep 10 17:39:54.370463 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:39:54.370377    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-tt86m"]
Sep 10 17:39:54.374871 ip-10-0-141-180 kubenswrapper[2581]: W0910 17:39:54.374844    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod48aa4f5c_d175_42dd_92df_4c416d5aea9e.slice/crio-c223457b1424da025ccba803991be4e35afd3c2c3f786e4b1fab7e9ca57e11ab WatchSource:0}: Error finding container c223457b1424da025ccba803991be4e35afd3c2c3f786e4b1fab7e9ca57e11ab: Status 404 returned error can't find the container with id c223457b1424da025ccba803991be4e35afd3c2c3f786e4b1fab7e9ca57e11ab
Sep 10 17:39:54.746268 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:39:54.746236    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-tt86m" event={"ID":"48aa4f5c-d175-42dd-92df-4c416d5aea9e","Type":"ContainerStarted","Data":"c223457b1424da025ccba803991be4e35afd3c2c3f786e4b1fab7e9ca57e11ab"}
Sep 10 17:39:56.757415 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:39:56.757033    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-tt86m" event={"ID":"48aa4f5c-d175-42dd-92df-4c416d5aea9e","Type":"ContainerStarted","Data":"17d752f9a34f759631946fbaefe15088c44653058ce91d164ddf5226438a7bbc"}
Sep 10 17:39:56.785301 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:39:56.785246    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-tt86m" podStartSLOduration=1.8122438170000001 podStartE2EDuration="3.785230302s" totalImagesPullingTime="1.972986485s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 17:39:53 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 17:39:56.782997986 +0000 UTC m=+216.437071903" watchObservedRunningTime="2026-09-10 17:39:56.785230302 +0000 UTC m=+216.439304206"
Sep 10 17:39:57.889825 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:39:57.889790    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fpqxbk"]
Sep 10 17:39:57.892122 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:39:57.892100    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fpqxbk"
Sep 10 17:39:57.894906 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:39:57.894882    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\""
Sep 10 17:39:57.895040 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:39:57.894913    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\""
Sep 10 17:39:57.896117 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:39:57.896089    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-2mcxw\""
Sep 10 17:39:57.902630 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:39:57.902609    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fpqxbk"]
Sep 10 17:39:58.003129 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:39:58.003090    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-h6wcz\" (UniqueName: \"kubernetes.io/projected/7307e1bc-bdf2-441d-ad1d-7b3a82a23f2b-kube-api-access-h6wcz\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fpqxbk\" (UID: \"7307e1bc-bdf2-441d-ad1d-7b3a82a23f2b\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fpqxbk"
Sep 10 17:39:58.003129 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:39:58.003134    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/7307e1bc-bdf2-441d-ad1d-7b3a82a23f2b-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fpqxbk\" (UID: \"7307e1bc-bdf2-441d-ad1d-7b3a82a23f2b\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fpqxbk"
Sep 10 17:39:58.003370 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:39:58.003158    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/7307e1bc-bdf2-441d-ad1d-7b3a82a23f2b-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fpqxbk\" (UID: \"7307e1bc-bdf2-441d-ad1d-7b3a82a23f2b\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fpqxbk"
Sep 10 17:39:58.103946 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:39:58.103912    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/7307e1bc-bdf2-441d-ad1d-7b3a82a23f2b-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fpqxbk\" (UID: \"7307e1bc-bdf2-441d-ad1d-7b3a82a23f2b\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fpqxbk"
Sep 10 17:39:58.103946 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:39:58.103952    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/7307e1bc-bdf2-441d-ad1d-7b3a82a23f2b-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fpqxbk\" (UID: \"7307e1bc-bdf2-441d-ad1d-7b3a82a23f2b\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fpqxbk"
Sep 10 17:39:58.104152 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:39:58.104013    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-h6wcz\" (UniqueName: \"kubernetes.io/projected/7307e1bc-bdf2-441d-ad1d-7b3a82a23f2b-kube-api-access-h6wcz\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fpqxbk\" (UID: \"7307e1bc-bdf2-441d-ad1d-7b3a82a23f2b\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fpqxbk"
Sep 10 17:39:58.104427 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:39:58.104406    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/7307e1bc-bdf2-441d-ad1d-7b3a82a23f2b-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fpqxbk\" (UID: \"7307e1bc-bdf2-441d-ad1d-7b3a82a23f2b\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fpqxbk"
Sep 10 17:39:58.104463 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:39:58.104400    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/7307e1bc-bdf2-441d-ad1d-7b3a82a23f2b-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fpqxbk\" (UID: \"7307e1bc-bdf2-441d-ad1d-7b3a82a23f2b\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fpqxbk"
Sep 10 17:39:58.112586 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:39:58.112549    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-h6wcz\" (UniqueName: \"kubernetes.io/projected/7307e1bc-bdf2-441d-ad1d-7b3a82a23f2b-kube-api-access-h6wcz\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fpqxbk\" (UID: \"7307e1bc-bdf2-441d-ad1d-7b3a82a23f2b\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fpqxbk"
Sep 10 17:39:58.201448 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:39:58.201370    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fpqxbk"
Sep 10 17:39:58.469734 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:39:58.469698    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fpqxbk"]
Sep 10 17:39:58.471924 ip-10-0-141-180 kubenswrapper[2581]: W0910 17:39:58.471901    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7307e1bc_bdf2_441d_ad1d_7b3a82a23f2b.slice/crio-56c910ef1941d73f68a93e6fca1ea2558130ec9126a2ec170806e2021e4a89ea WatchSource:0}: Error finding container 56c910ef1941d73f68a93e6fca1ea2558130ec9126a2ec170806e2021e4a89ea: Status 404 returned error can't find the container with id 56c910ef1941d73f68a93e6fca1ea2558130ec9126a2ec170806e2021e4a89ea
Sep 10 17:39:58.766550 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:39:58.766472    2581 generic.go:356] "Generic (PLEG): container finished" podID="7307e1bc-bdf2-441d-ad1d-7b3a82a23f2b" containerID="0480adfe8cc0fecd79aa80e6ce6fa4addb741fec9ca7943f5216b52a78af22d0" exitCode=0
Sep 10 17:39:58.766550 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:39:58.766546    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fpqxbk" event={"ID":"7307e1bc-bdf2-441d-ad1d-7b3a82a23f2b","Type":"ContainerDied","Data":"0480adfe8cc0fecd79aa80e6ce6fa4addb741fec9ca7943f5216b52a78af22d0"}
Sep 10 17:39:58.766757 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:39:58.766581    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fpqxbk" event={"ID":"7307e1bc-bdf2-441d-ad1d-7b3a82a23f2b","Type":"ContainerStarted","Data":"56c910ef1941d73f68a93e6fca1ea2558130ec9126a2ec170806e2021e4a89ea"}
Sep 10 17:40:00.779781 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:40:00.779746    2581 generic.go:356] "Generic (PLEG): container finished" podID="7307e1bc-bdf2-441d-ad1d-7b3a82a23f2b" containerID="33db82c54322295589cb6f11b27f717bc8891ea81850bcc3f1b702e7819b700b" exitCode=0
Sep 10 17:40:00.780072 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:40:00.779830    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fpqxbk" event={"ID":"7307e1bc-bdf2-441d-ad1d-7b3a82a23f2b","Type":"ContainerDied","Data":"33db82c54322295589cb6f11b27f717bc8891ea81850bcc3f1b702e7819b700b"}
Sep 10 17:40:01.786832 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:40:01.786787    2581 generic.go:356] "Generic (PLEG): container finished" podID="7307e1bc-bdf2-441d-ad1d-7b3a82a23f2b" containerID="3c33eb46a2513ba6669521a407a17508057be3b476579ac694822e2dac707b3f" exitCode=0
Sep 10 17:40:01.787227 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:40:01.786846    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fpqxbk" event={"ID":"7307e1bc-bdf2-441d-ad1d-7b3a82a23f2b","Type":"ContainerDied","Data":"3c33eb46a2513ba6669521a407a17508057be3b476579ac694822e2dac707b3f"}
Sep 10 17:40:02.916864 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:40:02.916842    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fpqxbk"
Sep 10 17:40:03.044825 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:40:03.044740    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-h6wcz\" (UniqueName: \"kubernetes.io/projected/7307e1bc-bdf2-441d-ad1d-7b3a82a23f2b-kube-api-access-h6wcz\") pod \"7307e1bc-bdf2-441d-ad1d-7b3a82a23f2b\" (UID: \"7307e1bc-bdf2-441d-ad1d-7b3a82a23f2b\") "
Sep 10 17:40:03.044825 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:40:03.044792    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/7307e1bc-bdf2-441d-ad1d-7b3a82a23f2b-util\") pod \"7307e1bc-bdf2-441d-ad1d-7b3a82a23f2b\" (UID: \"7307e1bc-bdf2-441d-ad1d-7b3a82a23f2b\") "
Sep 10 17:40:03.045014 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:40:03.044840    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/7307e1bc-bdf2-441d-ad1d-7b3a82a23f2b-bundle\") pod \"7307e1bc-bdf2-441d-ad1d-7b3a82a23f2b\" (UID: \"7307e1bc-bdf2-441d-ad1d-7b3a82a23f2b\") "
Sep 10 17:40:03.045267 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:40:03.045235    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7307e1bc-bdf2-441d-ad1d-7b3a82a23f2b-bundle" (OuterVolumeSpecName: "bundle") pod "7307e1bc-bdf2-441d-ad1d-7b3a82a23f2b" (UID: "7307e1bc-bdf2-441d-ad1d-7b3a82a23f2b"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 17:40:03.046917 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:40:03.046887    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7307e1bc-bdf2-441d-ad1d-7b3a82a23f2b-kube-api-access-h6wcz" (OuterVolumeSpecName: "kube-api-access-h6wcz") pod "7307e1bc-bdf2-441d-ad1d-7b3a82a23f2b" (UID: "7307e1bc-bdf2-441d-ad1d-7b3a82a23f2b"). InnerVolumeSpecName "kube-api-access-h6wcz". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 17:40:03.050190 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:40:03.050153    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7307e1bc-bdf2-441d-ad1d-7b3a82a23f2b-util" (OuterVolumeSpecName: "util") pod "7307e1bc-bdf2-441d-ad1d-7b3a82a23f2b" (UID: "7307e1bc-bdf2-441d-ad1d-7b3a82a23f2b"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 17:40:03.145885 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:40:03.145836    2581 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/7307e1bc-bdf2-441d-ad1d-7b3a82a23f2b-util\") on node \"ip-10-0-141-180.ec2.internal\" DevicePath \"\""
Sep 10 17:40:03.145885 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:40:03.145877    2581 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/7307e1bc-bdf2-441d-ad1d-7b3a82a23f2b-bundle\") on node \"ip-10-0-141-180.ec2.internal\" DevicePath \"\""
Sep 10 17:40:03.145885 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:40:03.145889    2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-h6wcz\" (UniqueName: \"kubernetes.io/projected/7307e1bc-bdf2-441d-ad1d-7b3a82a23f2b-kube-api-access-h6wcz\") on node \"ip-10-0-141-180.ec2.internal\" DevicePath \"\""
Sep 10 17:40:03.796823 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:40:03.796790    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fpqxbk" event={"ID":"7307e1bc-bdf2-441d-ad1d-7b3a82a23f2b","Type":"ContainerDied","Data":"56c910ef1941d73f68a93e6fca1ea2558130ec9126a2ec170806e2021e4a89ea"}
Sep 10 17:40:03.796823 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:40:03.796826    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fpqxbk"
Sep 10 17:40:03.797018 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:40:03.796836    2581 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="56c910ef1941d73f68a93e6fca1ea2558130ec9126a2ec170806e2021e4a89ea"
Sep 10 17:40:05.904453 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:40:05.904416    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-webhook-6486ffd997-ds7lc"]
Sep 10 17:40:05.904935 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:40:05.904719    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7307e1bc-bdf2-441d-ad1d-7b3a82a23f2b" containerName="extract"
Sep 10 17:40:05.904935 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:40:05.904731    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="7307e1bc-bdf2-441d-ad1d-7b3a82a23f2b" containerName="extract"
Sep 10 17:40:05.904935 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:40:05.904750    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7307e1bc-bdf2-441d-ad1d-7b3a82a23f2b" containerName="util"
Sep 10 17:40:05.904935 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:40:05.904756    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="7307e1bc-bdf2-441d-ad1d-7b3a82a23f2b" containerName="util"
Sep 10 17:40:05.904935 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:40:05.904763    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7307e1bc-bdf2-441d-ad1d-7b3a82a23f2b" containerName="pull"
Sep 10 17:40:05.904935 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:40:05.904768    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="7307e1bc-bdf2-441d-ad1d-7b3a82a23f2b" containerName="pull"
Sep 10 17:40:05.904935 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:40:05.904816    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="7307e1bc-bdf2-441d-ad1d-7b3a82a23f2b" containerName="extract"
Sep 10 17:40:05.906887 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:40:05.906863    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-webhook-6486ffd997-ds7lc"
Sep 10 17:40:05.909707 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:40:05.909683    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"kube-root-ca.crt\""
Sep 10 17:40:05.910694 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:40:05.910671    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-webhook-dockercfg-szwff\""
Sep 10 17:40:05.910793 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:40:05.910731    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"openshift-service-ca.crt\""
Sep 10 17:40:05.918058 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:40:05.918033    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-webhook-6486ffd997-ds7lc"]
Sep 10 17:40:06.073904 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:40:06.073867    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/3027e67b-44e0-4cbd-b1aa-a4354a6d371c-bound-sa-token\") pod \"cert-manager-webhook-6486ffd997-ds7lc\" (UID: \"3027e67b-44e0-4cbd-b1aa-a4354a6d371c\") " pod="cert-manager/cert-manager-webhook-6486ffd997-ds7lc"
Sep 10 17:40:06.073904 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:40:06.073907    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7pn7k\" (UniqueName: \"kubernetes.io/projected/3027e67b-44e0-4cbd-b1aa-a4354a6d371c-kube-api-access-7pn7k\") pod \"cert-manager-webhook-6486ffd997-ds7lc\" (UID: \"3027e67b-44e0-4cbd-b1aa-a4354a6d371c\") " pod="cert-manager/cert-manager-webhook-6486ffd997-ds7lc"
Sep 10 17:40:06.175342 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:40:06.175263    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/3027e67b-44e0-4cbd-b1aa-a4354a6d371c-bound-sa-token\") pod \"cert-manager-webhook-6486ffd997-ds7lc\" (UID: \"3027e67b-44e0-4cbd-b1aa-a4354a6d371c\") " pod="cert-manager/cert-manager-webhook-6486ffd997-ds7lc"
Sep 10 17:40:06.175342 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:40:06.175305    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7pn7k\" (UniqueName: \"kubernetes.io/projected/3027e67b-44e0-4cbd-b1aa-a4354a6d371c-kube-api-access-7pn7k\") pod \"cert-manager-webhook-6486ffd997-ds7lc\" (UID: \"3027e67b-44e0-4cbd-b1aa-a4354a6d371c\") " pod="cert-manager/cert-manager-webhook-6486ffd997-ds7lc"
Sep 10 17:40:06.183627 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:40:06.183591    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/3027e67b-44e0-4cbd-b1aa-a4354a6d371c-bound-sa-token\") pod \"cert-manager-webhook-6486ffd997-ds7lc\" (UID: \"3027e67b-44e0-4cbd-b1aa-a4354a6d371c\") " pod="cert-manager/cert-manager-webhook-6486ffd997-ds7lc"
Sep 10 17:40:06.183800 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:40:06.183734    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7pn7k\" (UniqueName: \"kubernetes.io/projected/3027e67b-44e0-4cbd-b1aa-a4354a6d371c-kube-api-access-7pn7k\") pod \"cert-manager-webhook-6486ffd997-ds7lc\" (UID: \"3027e67b-44e0-4cbd-b1aa-a4354a6d371c\") " pod="cert-manager/cert-manager-webhook-6486ffd997-ds7lc"
Sep 10 17:40:06.225762 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:40:06.225727    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-webhook-6486ffd997-ds7lc"
Sep 10 17:40:06.495480 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:40:06.495455    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-webhook-6486ffd997-ds7lc"]
Sep 10 17:40:06.497695 ip-10-0-141-180 kubenswrapper[2581]: W0910 17:40:06.497661    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod3027e67b_44e0_4cbd_b1aa_a4354a6d371c.slice/crio-c76d091ec2cbc380ba96873784420d13766637d115ff298e56fbf03ca2498845 WatchSource:0}: Error finding container c76d091ec2cbc380ba96873784420d13766637d115ff298e56fbf03ca2498845: Status 404 returned error can't find the container with id c76d091ec2cbc380ba96873784420d13766637d115ff298e56fbf03ca2498845
Sep 10 17:40:06.810544 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:40:06.810461    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-webhook-6486ffd997-ds7lc" event={"ID":"3027e67b-44e0-4cbd-b1aa-a4354a6d371c","Type":"ContainerStarted","Data":"c76d091ec2cbc380ba96873784420d13766637d115ff298e56fbf03ca2498845"}
Sep 10 17:40:09.262884 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:40:09.262849    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-55b4d954d5-lmgbn"]
Sep 10 17:40:09.265043 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:40:09.265021    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-55b4d954d5-lmgbn"
Sep 10 17:40:09.267629 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:40:09.267609    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-dockercfg-t5fpj\""
Sep 10 17:40:09.275987 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:40:09.275961    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-55b4d954d5-lmgbn"]
Sep 10 17:40:09.299527 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:40:09.299496    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2sw8s\" (UniqueName: \"kubernetes.io/projected/39b35720-8d1d-4e7a-a96f-8a8fa924c979-kube-api-access-2sw8s\") pod \"cert-manager-55b4d954d5-lmgbn\" (UID: \"39b35720-8d1d-4e7a-a96f-8a8fa924c979\") " pod="cert-manager/cert-manager-55b4d954d5-lmgbn"
Sep 10 17:40:09.299714 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:40:09.299537    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/39b35720-8d1d-4e7a-a96f-8a8fa924c979-bound-sa-token\") pod \"cert-manager-55b4d954d5-lmgbn\" (UID: \"39b35720-8d1d-4e7a-a96f-8a8fa924c979\") " pod="cert-manager/cert-manager-55b4d954d5-lmgbn"
Sep 10 17:40:09.400881 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:40:09.400787    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2sw8s\" (UniqueName: \"kubernetes.io/projected/39b35720-8d1d-4e7a-a96f-8a8fa924c979-kube-api-access-2sw8s\") pod \"cert-manager-55b4d954d5-lmgbn\" (UID: \"39b35720-8d1d-4e7a-a96f-8a8fa924c979\") " pod="cert-manager/cert-manager-55b4d954d5-lmgbn"
Sep 10 17:40:09.400881 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:40:09.400831    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/39b35720-8d1d-4e7a-a96f-8a8fa924c979-bound-sa-token\") pod \"cert-manager-55b4d954d5-lmgbn\" (UID: \"39b35720-8d1d-4e7a-a96f-8a8fa924c979\") " pod="cert-manager/cert-manager-55b4d954d5-lmgbn"
Sep 10 17:40:09.408455 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:40:09.408426    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/39b35720-8d1d-4e7a-a96f-8a8fa924c979-bound-sa-token\") pod \"cert-manager-55b4d954d5-lmgbn\" (UID: \"39b35720-8d1d-4e7a-a96f-8a8fa924c979\") " pod="cert-manager/cert-manager-55b4d954d5-lmgbn"
Sep 10 17:40:09.408562 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:40:09.408523    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2sw8s\" (UniqueName: \"kubernetes.io/projected/39b35720-8d1d-4e7a-a96f-8a8fa924c979-kube-api-access-2sw8s\") pod \"cert-manager-55b4d954d5-lmgbn\" (UID: \"39b35720-8d1d-4e7a-a96f-8a8fa924c979\") " pod="cert-manager/cert-manager-55b4d954d5-lmgbn"
Sep 10 17:40:09.573755 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:40:09.573724    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-55b4d954d5-lmgbn"
Sep 10 17:40:09.828697 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:40:09.828664    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-webhook-6486ffd997-ds7lc" event={"ID":"3027e67b-44e0-4cbd-b1aa-a4354a6d371c","Type":"ContainerStarted","Data":"9f4f80e89fdd35fa4cec6431dd10e68b9117b98c85f11e46685b5fc5c24db4df"}
Sep 10 17:40:09.828844 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:40:09.828730    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="cert-manager/cert-manager-webhook-6486ffd997-ds7lc"
Sep 10 17:40:09.842686 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:40:09.842633    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-55b4d954d5-lmgbn"]
Sep 10 17:40:09.845115 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:40:09.845065    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-webhook-6486ffd997-ds7lc" podStartSLOduration=2.245832888 podStartE2EDuration="4.845045707s" totalImagesPullingTime="2.599212819s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 17:40:05 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 17:40:09.843611259 +0000 UTC m=+229.497685180" watchObservedRunningTime="2026-09-10 17:40:09.845045707 +0000 UTC m=+229.499119612"
Sep 10 17:40:09.845867 ip-10-0-141-180 kubenswrapper[2581]: W0910 17:40:09.845847    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod39b35720_8d1d_4e7a_a96f_8a8fa924c979.slice/crio-e8fe61b28c135e359b0fce21a44712e0652ac7ae9c6141d6816dbd1da8c2729f WatchSource:0}: Error finding container e8fe61b28c135e359b0fce21a44712e0652ac7ae9c6141d6816dbd1da8c2729f: Status 404 returned error can't find the container with id e8fe61b28c135e359b0fce21a44712e0652ac7ae9c6141d6816dbd1da8c2729f
Sep 10 17:40:10.053807 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:40:10.053769    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/openshift-lws-operator-fd8ccff4c-fgffh"]
Sep 10 17:40:10.057746 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:40:10.057705    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-fgffh"
Sep 10 17:40:10.061022 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:40:10.060988    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"openshift-lws-operator-dockercfg-w7s2j\""
Sep 10 17:40:10.061177 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:40:10.060988    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"openshift-service-ca.crt\""
Sep 10 17:40:10.061177 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:40:10.061170    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"kube-root-ca.crt\""
Sep 10 17:40:10.069227 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:40:10.069204    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/openshift-lws-operator-fd8ccff4c-fgffh"]
Sep 10 17:40:10.108745 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:40:10.108653    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/9b90c56a-f146-4d31-be95-dd480b2ba2b8-tmp\") pod \"openshift-lws-operator-fd8ccff4c-fgffh\" (UID: \"9b90c56a-f146-4d31-be95-dd480b2ba2b8\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-fgffh"
Sep 10 17:40:10.108915 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:40:10.108802    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ntdxs\" (UniqueName: \"kubernetes.io/projected/9b90c56a-f146-4d31-be95-dd480b2ba2b8-kube-api-access-ntdxs\") pod \"openshift-lws-operator-fd8ccff4c-fgffh\" (UID: \"9b90c56a-f146-4d31-be95-dd480b2ba2b8\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-fgffh"
Sep 10 17:40:10.210222 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:40:10.210172    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/9b90c56a-f146-4d31-be95-dd480b2ba2b8-tmp\") pod \"openshift-lws-operator-fd8ccff4c-fgffh\" (UID: \"9b90c56a-f146-4d31-be95-dd480b2ba2b8\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-fgffh"
Sep 10 17:40:10.210405 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:40:10.210289    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ntdxs\" (UniqueName: \"kubernetes.io/projected/9b90c56a-f146-4d31-be95-dd480b2ba2b8-kube-api-access-ntdxs\") pod \"openshift-lws-operator-fd8ccff4c-fgffh\" (UID: \"9b90c56a-f146-4d31-be95-dd480b2ba2b8\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-fgffh"
Sep 10 17:40:10.210683 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:40:10.210626    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/9b90c56a-f146-4d31-be95-dd480b2ba2b8-tmp\") pod \"openshift-lws-operator-fd8ccff4c-fgffh\" (UID: \"9b90c56a-f146-4d31-be95-dd480b2ba2b8\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-fgffh"
Sep 10 17:40:10.219080 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:40:10.219053    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ntdxs\" (UniqueName: \"kubernetes.io/projected/9b90c56a-f146-4d31-be95-dd480b2ba2b8-kube-api-access-ntdxs\") pod \"openshift-lws-operator-fd8ccff4c-fgffh\" (UID: \"9b90c56a-f146-4d31-be95-dd480b2ba2b8\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-fgffh"
Sep 10 17:40:10.368429 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:40:10.368336    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-fgffh"
Sep 10 17:40:10.635654 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:40:10.635614    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/openshift-lws-operator-fd8ccff4c-fgffh"]
Sep 10 17:40:10.637727 ip-10-0-141-180 kubenswrapper[2581]: W0910 17:40:10.637700    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod9b90c56a_f146_4d31_be95_dd480b2ba2b8.slice/crio-2d1d2e9d60f5fa9cd9f0daf95f6443d29fa98047461866115fe69a11145cce99 WatchSource:0}: Error finding container 2d1d2e9d60f5fa9cd9f0daf95f6443d29fa98047461866115fe69a11145cce99: Status 404 returned error can't find the container with id 2d1d2e9d60f5fa9cd9f0daf95f6443d29fa98047461866115fe69a11145cce99
Sep 10 17:40:10.833298 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:40:10.833262    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-fgffh" event={"ID":"9b90c56a-f146-4d31-be95-dd480b2ba2b8","Type":"ContainerStarted","Data":"2d1d2e9d60f5fa9cd9f0daf95f6443d29fa98047461866115fe69a11145cce99"}
Sep 10 17:40:10.834634 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:40:10.834600    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-55b4d954d5-lmgbn" event={"ID":"39b35720-8d1d-4e7a-a96f-8a8fa924c979","Type":"ContainerStarted","Data":"baf4c911f3efa5da4caf635396b06558dae5766c62b27712ec33481f0303e599"}
Sep 10 17:40:10.834771 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:40:10.834663    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-55b4d954d5-lmgbn" event={"ID":"39b35720-8d1d-4e7a-a96f-8a8fa924c979","Type":"ContainerStarted","Data":"e8fe61b28c135e359b0fce21a44712e0652ac7ae9c6141d6816dbd1da8c2729f"}
Sep 10 17:40:10.852065 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:40:10.852021    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-55b4d954d5-lmgbn" podStartSLOduration=1.852004465 podStartE2EDuration="1.852004465s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 17:40:09 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 17:40:10.850072953 +0000 UTC m=+230.504146858" watchObservedRunningTime="2026-09-10 17:40:10.852004465 +0000 UTC m=+230.506078369"
Sep 10 17:40:12.845532 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:40:12.845452    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-fgffh" event={"ID":"9b90c56a-f146-4d31-be95-dd480b2ba2b8","Type":"ContainerStarted","Data":"bb67829ebe13ae1fcc2cd26fc438807dcc080026d10aa80a7a86f538e6595acf"}
Sep 10 17:40:12.861657 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:40:12.861597    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-fgffh" podStartSLOduration=1.004781229 podStartE2EDuration="2.8615812s" totalImagesPullingTime="1.856799971s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 17:40:10 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 17:40:12.860710748 +0000 UTC m=+232.514784657" watchObservedRunningTime="2026-09-10 17:40:12.8615812 +0000 UTC m=+232.515655103"
Sep 10 17:40:15.827195 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:40:15.827164    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afqzd8j"]
Sep 10 17:40:15.829711 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:40:15.829693    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afqzd8j"
Sep 10 17:40:15.832398 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:40:15.832375    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\""
Sep 10 17:40:15.832520 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:40:15.832378    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-2mcxw\""
Sep 10 17:40:15.833259 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:40:15.833237    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\""
Sep 10 17:40:15.837005 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:40:15.836985    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="cert-manager/cert-manager-webhook-6486ffd997-ds7lc"
Sep 10 17:40:15.839540 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:40:15.839517    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afqzd8j"]
Sep 10 17:40:15.856019 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:40:15.855996    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/d9900edd-afaf-4548-a0d3-4fdf56a2f0c1-bundle\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afqzd8j\" (UID: \"d9900edd-afaf-4548-a0d3-4fdf56a2f0c1\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afqzd8j"
Sep 10 17:40:15.856098 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:40:15.856059    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6z2r5\" (UniqueName: \"kubernetes.io/projected/d9900edd-afaf-4548-a0d3-4fdf56a2f0c1-kube-api-access-6z2r5\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afqzd8j\" (UID: \"d9900edd-afaf-4548-a0d3-4fdf56a2f0c1\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afqzd8j"
Sep 10 17:40:15.856151 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:40:15.856100    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/d9900edd-afaf-4548-a0d3-4fdf56a2f0c1-util\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afqzd8j\" (UID: \"d9900edd-afaf-4548-a0d3-4fdf56a2f0c1\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afqzd8j"
Sep 10 17:40:15.956705 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:40:15.956657    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6z2r5\" (UniqueName: \"kubernetes.io/projected/d9900edd-afaf-4548-a0d3-4fdf56a2f0c1-kube-api-access-6z2r5\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afqzd8j\" (UID: \"d9900edd-afaf-4548-a0d3-4fdf56a2f0c1\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afqzd8j"
Sep 10 17:40:15.956892 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:40:15.956738    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/d9900edd-afaf-4548-a0d3-4fdf56a2f0c1-util\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afqzd8j\" (UID: \"d9900edd-afaf-4548-a0d3-4fdf56a2f0c1\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afqzd8j"
Sep 10 17:40:15.957002 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:40:15.956979    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/d9900edd-afaf-4548-a0d3-4fdf56a2f0c1-bundle\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afqzd8j\" (UID: \"d9900edd-afaf-4548-a0d3-4fdf56a2f0c1\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afqzd8j"
Sep 10 17:40:15.957279 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:40:15.957259    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/d9900edd-afaf-4548-a0d3-4fdf56a2f0c1-util\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afqzd8j\" (UID: \"d9900edd-afaf-4548-a0d3-4fdf56a2f0c1\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afqzd8j"
Sep 10 17:40:15.957329 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:40:15.957280    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/d9900edd-afaf-4548-a0d3-4fdf56a2f0c1-bundle\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afqzd8j\" (UID: \"d9900edd-afaf-4548-a0d3-4fdf56a2f0c1\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afqzd8j"
Sep 10 17:40:15.965420 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:40:15.965391    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6z2r5\" (UniqueName: \"kubernetes.io/projected/d9900edd-afaf-4548-a0d3-4fdf56a2f0c1-kube-api-access-6z2r5\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afqzd8j\" (UID: \"d9900edd-afaf-4548-a0d3-4fdf56a2f0c1\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afqzd8j"
Sep 10 17:40:16.139196 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:40:16.139112    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afqzd8j"
Sep 10 17:40:16.407246 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:40:16.407215    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afqzd8j"]
Sep 10 17:40:16.409601 ip-10-0-141-180 kubenswrapper[2581]: W0910 17:40:16.409573    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd9900edd_afaf_4548_a0d3_4fdf56a2f0c1.slice/crio-f6cd2ac486d5c97c5e296ce22b32bacb96a8c4a438bfb303b2b95f377e83579c WatchSource:0}: Error finding container f6cd2ac486d5c97c5e296ce22b32bacb96a8c4a438bfb303b2b95f377e83579c: Status 404 returned error can't find the container with id f6cd2ac486d5c97c5e296ce22b32bacb96a8c4a438bfb303b2b95f377e83579c
Sep 10 17:40:16.864031 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:40:16.864002    2581 generic.go:356] "Generic (PLEG): container finished" podID="d9900edd-afaf-4548-a0d3-4fdf56a2f0c1" containerID="e5af7720b3779fd2404ee1202cdd29331a8b2dfe26ac8fc14b322e7a36d20e6f" exitCode=0
Sep 10 17:40:16.864479 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:40:16.864079    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afqzd8j" event={"ID":"d9900edd-afaf-4548-a0d3-4fdf56a2f0c1","Type":"ContainerDied","Data":"e5af7720b3779fd2404ee1202cdd29331a8b2dfe26ac8fc14b322e7a36d20e6f"}
Sep 10 17:40:16.864479 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:40:16.864122    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afqzd8j" event={"ID":"d9900edd-afaf-4548-a0d3-4fdf56a2f0c1","Type":"ContainerStarted","Data":"f6cd2ac486d5c97c5e296ce22b32bacb96a8c4a438bfb303b2b95f377e83579c"}
Sep 10 17:40:18.874176 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:40:18.874144    2581 generic.go:356] "Generic (PLEG): container finished" podID="d9900edd-afaf-4548-a0d3-4fdf56a2f0c1" containerID="214e54fa08558d242d8dedb705faf516249d8e0645e846d064145687d6c70094" exitCode=0
Sep 10 17:40:18.874585 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:40:18.874224    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afqzd8j" event={"ID":"d9900edd-afaf-4548-a0d3-4fdf56a2f0c1","Type":"ContainerDied","Data":"214e54fa08558d242d8dedb705faf516249d8e0645e846d064145687d6c70094"}
Sep 10 17:40:19.883991 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:40:19.883955    2581 generic.go:356] "Generic (PLEG): container finished" podID="d9900edd-afaf-4548-a0d3-4fdf56a2f0c1" containerID="a3c8ece985fa86788af16d0320bf73c22c52e32345ea89968c98d797134b499b" exitCode=0
Sep 10 17:40:19.884362 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:40:19.884016    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afqzd8j" event={"ID":"d9900edd-afaf-4548-a0d3-4fdf56a2f0c1","Type":"ContainerDied","Data":"a3c8ece985fa86788af16d0320bf73c22c52e32345ea89968c98d797134b499b"}
Sep 10 17:40:21.004911 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:40:21.004888    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afqzd8j"
Sep 10 17:40:21.102852 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:40:21.102821    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-6z2r5\" (UniqueName: \"kubernetes.io/projected/d9900edd-afaf-4548-a0d3-4fdf56a2f0c1-kube-api-access-6z2r5\") pod \"d9900edd-afaf-4548-a0d3-4fdf56a2f0c1\" (UID: \"d9900edd-afaf-4548-a0d3-4fdf56a2f0c1\") "
Sep 10 17:40:21.102994 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:40:21.102866    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/d9900edd-afaf-4548-a0d3-4fdf56a2f0c1-util\") pod \"d9900edd-afaf-4548-a0d3-4fdf56a2f0c1\" (UID: \"d9900edd-afaf-4548-a0d3-4fdf56a2f0c1\") "
Sep 10 17:40:21.102994 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:40:21.102906    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/d9900edd-afaf-4548-a0d3-4fdf56a2f0c1-bundle\") pod \"d9900edd-afaf-4548-a0d3-4fdf56a2f0c1\" (UID: \"d9900edd-afaf-4548-a0d3-4fdf56a2f0c1\") "
Sep 10 17:40:21.103587 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:40:21.103564    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d9900edd-afaf-4548-a0d3-4fdf56a2f0c1-bundle" (OuterVolumeSpecName: "bundle") pod "d9900edd-afaf-4548-a0d3-4fdf56a2f0c1" (UID: "d9900edd-afaf-4548-a0d3-4fdf56a2f0c1"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 17:40:21.104874 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:40:21.104852    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d9900edd-afaf-4548-a0d3-4fdf56a2f0c1-kube-api-access-6z2r5" (OuterVolumeSpecName: "kube-api-access-6z2r5") pod "d9900edd-afaf-4548-a0d3-4fdf56a2f0c1" (UID: "d9900edd-afaf-4548-a0d3-4fdf56a2f0c1"). InnerVolumeSpecName "kube-api-access-6z2r5". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 17:40:21.108103 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:40:21.108066    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d9900edd-afaf-4548-a0d3-4fdf56a2f0c1-util" (OuterVolumeSpecName: "util") pod "d9900edd-afaf-4548-a0d3-4fdf56a2f0c1" (UID: "d9900edd-afaf-4548-a0d3-4fdf56a2f0c1"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 17:40:21.203478 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:40:21.203454    2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6z2r5\" (UniqueName: \"kubernetes.io/projected/d9900edd-afaf-4548-a0d3-4fdf56a2f0c1-kube-api-access-6z2r5\") on node \"ip-10-0-141-180.ec2.internal\" DevicePath \"\""
Sep 10 17:40:21.203539 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:40:21.203481    2581 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/d9900edd-afaf-4548-a0d3-4fdf56a2f0c1-util\") on node \"ip-10-0-141-180.ec2.internal\" DevicePath \"\""
Sep 10 17:40:21.203539 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:40:21.203491    2581 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/d9900edd-afaf-4548-a0d3-4fdf56a2f0c1-bundle\") on node \"ip-10-0-141-180.ec2.internal\" DevicePath \"\""
Sep 10 17:40:21.894518 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:40:21.894490    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afqzd8j"
Sep 10 17:40:21.894518 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:40:21.894504    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afqzd8j" event={"ID":"d9900edd-afaf-4548-a0d3-4fdf56a2f0c1","Type":"ContainerDied","Data":"f6cd2ac486d5c97c5e296ce22b32bacb96a8c4a438bfb303b2b95f377e83579c"}
Sep 10 17:40:21.894743 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:40:21.894548    2581 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="f6cd2ac486d5c97c5e296ce22b32bacb96a8c4a438bfb303b2b95f377e83579c"
Sep 10 17:40:32.636948 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:40:32.636905    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b07cxqw"]
Sep 10 17:40:32.637433 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:40:32.637347    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d9900edd-afaf-4548-a0d3-4fdf56a2f0c1" containerName="util"
Sep 10 17:40:32.637433 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:40:32.637366    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="d9900edd-afaf-4548-a0d3-4fdf56a2f0c1" containerName="util"
Sep 10 17:40:32.637433 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:40:32.637380    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d9900edd-afaf-4548-a0d3-4fdf56a2f0c1" containerName="extract"
Sep 10 17:40:32.637433 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:40:32.637388    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="d9900edd-afaf-4548-a0d3-4fdf56a2f0c1" containerName="extract"
Sep 10 17:40:32.637433 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:40:32.637409    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d9900edd-afaf-4548-a0d3-4fdf56a2f0c1" containerName="pull"
Sep 10 17:40:32.637433 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:40:32.637417    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="d9900edd-afaf-4548-a0d3-4fdf56a2f0c1" containerName="pull"
Sep 10 17:40:32.637766 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:40:32.637504    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="d9900edd-afaf-4548-a0d3-4fdf56a2f0c1" containerName="extract"
Sep 10 17:40:32.641840 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:40:32.641818    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b07cxqw"
Sep 10 17:40:32.644593 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:40:32.644575    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\""
Sep 10 17:40:32.645555 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:40:32.645540    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\""
Sep 10 17:40:32.645661 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:40:32.645562    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-2mcxw\""
Sep 10 17:40:32.649033 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:40:32.649005    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b07cxqw"]
Sep 10 17:40:32.707587 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:40:32.707549    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/955f5e73-4be9-4181-af7f-aad43740a4c8-util\") pod \"8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b07cxqw\" (UID: \"955f5e73-4be9-4181-af7f-aad43740a4c8\") " pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b07cxqw"
Sep 10 17:40:32.707587 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:40:32.707589    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/955f5e73-4be9-4181-af7f-aad43740a4c8-bundle\") pod \"8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b07cxqw\" (UID: \"955f5e73-4be9-4181-af7f-aad43740a4c8\") " pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b07cxqw"
Sep 10 17:40:32.707819 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:40:32.707613    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lvh69\" (UniqueName: \"kubernetes.io/projected/955f5e73-4be9-4181-af7f-aad43740a4c8-kube-api-access-lvh69\") pod \"8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b07cxqw\" (UID: \"955f5e73-4be9-4181-af7f-aad43740a4c8\") " pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b07cxqw"
Sep 10 17:40:32.808835 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:40:32.808802    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lvh69\" (UniqueName: \"kubernetes.io/projected/955f5e73-4be9-4181-af7f-aad43740a4c8-kube-api-access-lvh69\") pod \"8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b07cxqw\" (UID: \"955f5e73-4be9-4181-af7f-aad43740a4c8\") " pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b07cxqw"
Sep 10 17:40:32.809018 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:40:32.808912    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/955f5e73-4be9-4181-af7f-aad43740a4c8-util\") pod \"8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b07cxqw\" (UID: \"955f5e73-4be9-4181-af7f-aad43740a4c8\") " pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b07cxqw"
Sep 10 17:40:32.809018 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:40:32.808943    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/955f5e73-4be9-4181-af7f-aad43740a4c8-bundle\") pod \"8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b07cxqw\" (UID: \"955f5e73-4be9-4181-af7f-aad43740a4c8\") " pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b07cxqw"
Sep 10 17:40:32.809309 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:40:32.809290    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/955f5e73-4be9-4181-af7f-aad43740a4c8-bundle\") pod \"8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b07cxqw\" (UID: \"955f5e73-4be9-4181-af7f-aad43740a4c8\") " pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b07cxqw"
Sep 10 17:40:32.809406 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:40:32.809363    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/955f5e73-4be9-4181-af7f-aad43740a4c8-util\") pod \"8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b07cxqw\" (UID: \"955f5e73-4be9-4181-af7f-aad43740a4c8\") " pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b07cxqw"
Sep 10 17:40:32.821390 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:40:32.821363    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lvh69\" (UniqueName: \"kubernetes.io/projected/955f5e73-4be9-4181-af7f-aad43740a4c8-kube-api-access-lvh69\") pod \"8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b07cxqw\" (UID: \"955f5e73-4be9-4181-af7f-aad43740a4c8\") " pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b07cxqw"
Sep 10 17:40:32.950835 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:40:32.950755    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b07cxqw"
Sep 10 17:40:33.222091 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:40:33.222064    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b07cxqw"]
Sep 10 17:40:33.224356 ip-10-0-141-180 kubenswrapper[2581]: W0910 17:40:33.224328    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod955f5e73_4be9_4181_af7f_aad43740a4c8.slice/crio-1408b59289ca089e4a66aac19bb5f81764bc45ac0e5931abdef5bffa7fa9da75 WatchSource:0}: Error finding container 1408b59289ca089e4a66aac19bb5f81764bc45ac0e5931abdef5bffa7fa9da75: Status 404 returned error can't find the container with id 1408b59289ca089e4a66aac19bb5f81764bc45ac0e5931abdef5bffa7fa9da75
Sep 10 17:40:33.882154 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:40:33.882120    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-dbf885bd4-qqm77"]
Sep 10 17:40:33.884715 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:40:33.884695    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-dbf885bd4-qqm77"
Sep 10 17:40:33.888310 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:40:33.888293    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\""
Sep 10 17:40:33.888737 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:40:33.888719    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\""
Sep 10 17:40:33.888818 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:40:33.888801    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-c9dnv\""
Sep 10 17:40:33.888862 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:40:33.888805    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\""
Sep 10 17:40:33.888862 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:40:33.888834    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\""
Sep 10 17:40:33.904448 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:40:33.904423    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-dbf885bd4-qqm77"]
Sep 10 17:40:33.947689 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:40:33.947659    2581 generic.go:356] "Generic (PLEG): container finished" podID="955f5e73-4be9-4181-af7f-aad43740a4c8" containerID="5b621679f03b5304a8d663d90ad7973d4492f5479fcaa26052a3fde142403a6c" exitCode=0
Sep 10 17:40:33.947820 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:40:33.947697    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b07cxqw" event={"ID":"955f5e73-4be9-4181-af7f-aad43740a4c8","Type":"ContainerDied","Data":"5b621679f03b5304a8d663d90ad7973d4492f5479fcaa26052a3fde142403a6c"}
Sep 10 17:40:33.947820 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:40:33.947730    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b07cxqw" event={"ID":"955f5e73-4be9-4181-af7f-aad43740a4c8","Type":"ContainerStarted","Data":"1408b59289ca089e4a66aac19bb5f81764bc45ac0e5931abdef5bffa7fa9da75"}
Sep 10 17:40:34.020004 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:40:34.019969    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/53a5b6c5-f6e4-455f-b06e-25b72432e1a6-webhook-cert\") pod \"opendatahub-operator-controller-manager-dbf885bd4-qqm77\" (UID: \"53a5b6c5-f6e4-455f-b06e-25b72432e1a6\") " pod="opendatahub/opendatahub-operator-controller-manager-dbf885bd4-qqm77"
Sep 10 17:40:34.020277 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:40:34.020251    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/53a5b6c5-f6e4-455f-b06e-25b72432e1a6-manifests\") pod \"opendatahub-operator-controller-manager-dbf885bd4-qqm77\" (UID: \"53a5b6c5-f6e4-455f-b06e-25b72432e1a6\") " pod="opendatahub/opendatahub-operator-controller-manager-dbf885bd4-qqm77"
Sep 10 17:40:34.020363 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:40:34.020303    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/53a5b6c5-f6e4-455f-b06e-25b72432e1a6-apiservice-cert\") pod \"opendatahub-operator-controller-manager-dbf885bd4-qqm77\" (UID: \"53a5b6c5-f6e4-455f-b06e-25b72432e1a6\") " pod="opendatahub/opendatahub-operator-controller-manager-dbf885bd4-qqm77"
Sep 10 17:40:34.020403 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:40:34.020373    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5q8xt\" (UniqueName: \"kubernetes.io/projected/53a5b6c5-f6e4-455f-b06e-25b72432e1a6-kube-api-access-5q8xt\") pod \"opendatahub-operator-controller-manager-dbf885bd4-qqm77\" (UID: \"53a5b6c5-f6e4-455f-b06e-25b72432e1a6\") " pod="opendatahub/opendatahub-operator-controller-manager-dbf885bd4-qqm77"
Sep 10 17:40:34.020453 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:40:34.020438    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/53a5b6c5-f6e4-455f-b06e-25b72432e1a6-tmp\") pod \"opendatahub-operator-controller-manager-dbf885bd4-qqm77\" (UID: \"53a5b6c5-f6e4-455f-b06e-25b72432e1a6\") " pod="opendatahub/opendatahub-operator-controller-manager-dbf885bd4-qqm77"
Sep 10 17:40:34.121199 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:40:34.121132    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/53a5b6c5-f6e4-455f-b06e-25b72432e1a6-tmp\") pod \"opendatahub-operator-controller-manager-dbf885bd4-qqm77\" (UID: \"53a5b6c5-f6e4-455f-b06e-25b72432e1a6\") " pod="opendatahub/opendatahub-operator-controller-manager-dbf885bd4-qqm77"
Sep 10 17:40:34.121199 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:40:34.121190    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/53a5b6c5-f6e4-455f-b06e-25b72432e1a6-webhook-cert\") pod \"opendatahub-operator-controller-manager-dbf885bd4-qqm77\" (UID: \"53a5b6c5-f6e4-455f-b06e-25b72432e1a6\") " pod="opendatahub/opendatahub-operator-controller-manager-dbf885bd4-qqm77"
Sep 10 17:40:34.121354 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:40:34.121217    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/53a5b6c5-f6e4-455f-b06e-25b72432e1a6-manifests\") pod \"opendatahub-operator-controller-manager-dbf885bd4-qqm77\" (UID: \"53a5b6c5-f6e4-455f-b06e-25b72432e1a6\") " pod="opendatahub/opendatahub-operator-controller-manager-dbf885bd4-qqm77"
Sep 10 17:40:34.121354 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:40:34.121237    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/53a5b6c5-f6e4-455f-b06e-25b72432e1a6-apiservice-cert\") pod \"opendatahub-operator-controller-manager-dbf885bd4-qqm77\" (UID: \"53a5b6c5-f6e4-455f-b06e-25b72432e1a6\") " pod="opendatahub/opendatahub-operator-controller-manager-dbf885bd4-qqm77"
Sep 10 17:40:34.121354 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:40:34.121252    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5q8xt\" (UniqueName: \"kubernetes.io/projected/53a5b6c5-f6e4-455f-b06e-25b72432e1a6-kube-api-access-5q8xt\") pod \"opendatahub-operator-controller-manager-dbf885bd4-qqm77\" (UID: \"53a5b6c5-f6e4-455f-b06e-25b72432e1a6\") " pod="opendatahub/opendatahub-operator-controller-manager-dbf885bd4-qqm77"
Sep 10 17:40:34.121616 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:40:34.121594    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/53a5b6c5-f6e4-455f-b06e-25b72432e1a6-manifests\") pod \"opendatahub-operator-controller-manager-dbf885bd4-qqm77\" (UID: \"53a5b6c5-f6e4-455f-b06e-25b72432e1a6\") " pod="opendatahub/opendatahub-operator-controller-manager-dbf885bd4-qqm77"
Sep 10 17:40:34.123427 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:40:34.123404    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/53a5b6c5-f6e4-455f-b06e-25b72432e1a6-tmp\") pod \"opendatahub-operator-controller-manager-dbf885bd4-qqm77\" (UID: \"53a5b6c5-f6e4-455f-b06e-25b72432e1a6\") " pod="opendatahub/opendatahub-operator-controller-manager-dbf885bd4-qqm77"
Sep 10 17:40:34.123721 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:40:34.123699    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/53a5b6c5-f6e4-455f-b06e-25b72432e1a6-webhook-cert\") pod \"opendatahub-operator-controller-manager-dbf885bd4-qqm77\" (UID: \"53a5b6c5-f6e4-455f-b06e-25b72432e1a6\") " pod="opendatahub/opendatahub-operator-controller-manager-dbf885bd4-qqm77"
Sep 10 17:40:34.123793 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:40:34.123705    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/53a5b6c5-f6e4-455f-b06e-25b72432e1a6-apiservice-cert\") pod \"opendatahub-operator-controller-manager-dbf885bd4-qqm77\" (UID: \"53a5b6c5-f6e4-455f-b06e-25b72432e1a6\") " pod="opendatahub/opendatahub-operator-controller-manager-dbf885bd4-qqm77"
Sep 10 17:40:34.130300 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:40:34.130274    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5q8xt\" (UniqueName: \"kubernetes.io/projected/53a5b6c5-f6e4-455f-b06e-25b72432e1a6-kube-api-access-5q8xt\") pod \"opendatahub-operator-controller-manager-dbf885bd4-qqm77\" (UID: \"53a5b6c5-f6e4-455f-b06e-25b72432e1a6\") " pod="opendatahub/opendatahub-operator-controller-manager-dbf885bd4-qqm77"
Sep 10 17:40:34.194308 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:40:34.194278    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-dbf885bd4-qqm77"
Sep 10 17:40:34.469271 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:40:34.469250    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-dbf885bd4-qqm77"]
Sep 10 17:40:34.471926 ip-10-0-141-180 kubenswrapper[2581]: W0910 17:40:34.471900    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod53a5b6c5_f6e4_455f_b06e_25b72432e1a6.slice/crio-9e68bd3de9962b5f78033bb7ea7a6c34ee4cc3babb30682058c7731c22ff22ef WatchSource:0}: Error finding container 9e68bd3de9962b5f78033bb7ea7a6c34ee4cc3babb30682058c7731c22ff22ef: Status 404 returned error can't find the container with id 9e68bd3de9962b5f78033bb7ea7a6c34ee4cc3babb30682058c7731c22ff22ef
Sep 10 17:40:34.954044 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:40:34.954011    2581 generic.go:356] "Generic (PLEG): container finished" podID="955f5e73-4be9-4181-af7f-aad43740a4c8" containerID="5538f6f80ceadf14507a487df0250d8924df607eb5cffdce2c47d3b847fc6c65" exitCode=0
Sep 10 17:40:34.954481 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:40:34.954082    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b07cxqw" event={"ID":"955f5e73-4be9-4181-af7f-aad43740a4c8","Type":"ContainerDied","Data":"5538f6f80ceadf14507a487df0250d8924df607eb5cffdce2c47d3b847fc6c65"}
Sep 10 17:40:34.955502 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:40:34.955479    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-dbf885bd4-qqm77" event={"ID":"53a5b6c5-f6e4-455f-b06e-25b72432e1a6","Type":"ContainerStarted","Data":"9e68bd3de9962b5f78033bb7ea7a6c34ee4cc3babb30682058c7731c22ff22ef"}
Sep 10 17:40:35.964630 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:40:35.964592    2581 generic.go:356] "Generic (PLEG): container finished" podID="955f5e73-4be9-4181-af7f-aad43740a4c8" containerID="da22ca3e626111d1ed3da888104c6e34c5651f2c12e584e0803b16cfba0616e6" exitCode=0
Sep 10 17:40:35.965052 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:40:35.964687    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b07cxqw" event={"ID":"955f5e73-4be9-4181-af7f-aad43740a4c8","Type":"ContainerDied","Data":"da22ca3e626111d1ed3da888104c6e34c5651f2c12e584e0803b16cfba0616e6"}
Sep 10 17:40:37.460921 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:40:37.460897    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b07cxqw"
Sep 10 17:40:37.555076 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:40:37.555049    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-lvh69\" (UniqueName: \"kubernetes.io/projected/955f5e73-4be9-4181-af7f-aad43740a4c8-kube-api-access-lvh69\") pod \"955f5e73-4be9-4181-af7f-aad43740a4c8\" (UID: \"955f5e73-4be9-4181-af7f-aad43740a4c8\") "
Sep 10 17:40:37.555258 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:40:37.555089    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/955f5e73-4be9-4181-af7f-aad43740a4c8-util\") pod \"955f5e73-4be9-4181-af7f-aad43740a4c8\" (UID: \"955f5e73-4be9-4181-af7f-aad43740a4c8\") "
Sep 10 17:40:37.555258 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:40:37.555123    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/955f5e73-4be9-4181-af7f-aad43740a4c8-bundle\") pod \"955f5e73-4be9-4181-af7f-aad43740a4c8\" (UID: \"955f5e73-4be9-4181-af7f-aad43740a4c8\") "
Sep 10 17:40:37.555858 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:40:37.555830    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/955f5e73-4be9-4181-af7f-aad43740a4c8-bundle" (OuterVolumeSpecName: "bundle") pod "955f5e73-4be9-4181-af7f-aad43740a4c8" (UID: "955f5e73-4be9-4181-af7f-aad43740a4c8"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 17:40:37.557106 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:40:37.557077    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/955f5e73-4be9-4181-af7f-aad43740a4c8-kube-api-access-lvh69" (OuterVolumeSpecName: "kube-api-access-lvh69") pod "955f5e73-4be9-4181-af7f-aad43740a4c8" (UID: "955f5e73-4be9-4181-af7f-aad43740a4c8"). InnerVolumeSpecName "kube-api-access-lvh69". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 17:40:37.560306 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:40:37.560269    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/955f5e73-4be9-4181-af7f-aad43740a4c8-util" (OuterVolumeSpecName: "util") pod "955f5e73-4be9-4181-af7f-aad43740a4c8" (UID: "955f5e73-4be9-4181-af7f-aad43740a4c8"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 17:40:37.655771 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:40:37.655735    2581 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/955f5e73-4be9-4181-af7f-aad43740a4c8-bundle\") on node \"ip-10-0-141-180.ec2.internal\" DevicePath \"\""
Sep 10 17:40:37.655771 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:40:37.655763    2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-lvh69\" (UniqueName: \"kubernetes.io/projected/955f5e73-4be9-4181-af7f-aad43740a4c8-kube-api-access-lvh69\") on node \"ip-10-0-141-180.ec2.internal\" DevicePath \"\""
Sep 10 17:40:37.655771 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:40:37.655773    2581 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/955f5e73-4be9-4181-af7f-aad43740a4c8-util\") on node \"ip-10-0-141-180.ec2.internal\" DevicePath \"\""
Sep 10 17:40:37.978496 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:40:37.978453    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b07cxqw" event={"ID":"955f5e73-4be9-4181-af7f-aad43740a4c8","Type":"ContainerDied","Data":"1408b59289ca089e4a66aac19bb5f81764bc45ac0e5931abdef5bffa7fa9da75"}
Sep 10 17:40:37.978710 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:40:37.978514    2581 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="1408b59289ca089e4a66aac19bb5f81764bc45ac0e5931abdef5bffa7fa9da75"
Sep 10 17:40:37.978710 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:40:37.978465    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b07cxqw"
Sep 10 17:40:37.980333 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:40:37.980295    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-dbf885bd4-qqm77" event={"ID":"53a5b6c5-f6e4-455f-b06e-25b72432e1a6","Type":"ContainerStarted","Data":"18fa865e86f0c519df96f3fa7b004bfa576b4e01afe4e361ffbbd19a08d1f0fc"}
Sep 10 17:40:38.986140 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:40:38.986103    2581 generic.go:356] "Generic (PLEG): container finished" podID="53a5b6c5-f6e4-455f-b06e-25b72432e1a6" containerID="18fa865e86f0c519df96f3fa7b004bfa576b4e01afe4e361ffbbd19a08d1f0fc" exitCode=0
Sep 10 17:40:38.986537 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:40:38.986191    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-dbf885bd4-qqm77" event={"ID":"53a5b6c5-f6e4-455f-b06e-25b72432e1a6","Type":"ContainerDied","Data":"18fa865e86f0c519df96f3fa7b004bfa576b4e01afe4e361ffbbd19a08d1f0fc"}
Sep 10 17:40:39.992589 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:40:39.992537    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-dbf885bd4-qqm77" event={"ID":"53a5b6c5-f6e4-455f-b06e-25b72432e1a6","Type":"ContainerStarted","Data":"f651e81628de0746054f3a15087f43e2528cd56a0a398c0e02575c89de4efc02"}
Sep 10 17:40:39.993016 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:40:39.992744    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-dbf885bd4-qqm77"
Sep 10 17:40:40.020875 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:40:40.020828    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-dbf885bd4-qqm77" podStartSLOduration=2.5157173520000002 podStartE2EDuration="7.020813224s" totalImagesPullingTime="3.371680661s" totalInitContainerRuntime="1.133415211s" isStatefulPod=false podCreationTimestamp="2026-09-10 17:40:33 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 17:40:40.018425776 +0000 UTC m=+259.672499695" watchObservedRunningTime="2026-09-10 17:40:40.020813224 +0000 UTC m=+259.674887128"
Sep 10 17:40:50.999830 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:40:50.999753    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-dbf885bd4-qqm77"
Sep 10 17:40:53.448372 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:40:53.448342    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/kube-auth-proxy-7cd5576685-rd9h7"]
Sep 10 17:40:53.448755 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:40:53.448663    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="955f5e73-4be9-4181-af7f-aad43740a4c8" containerName="util"
Sep 10 17:40:53.448755 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:40:53.448676    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="955f5e73-4be9-4181-af7f-aad43740a4c8" containerName="util"
Sep 10 17:40:53.448755 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:40:53.448698    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="955f5e73-4be9-4181-af7f-aad43740a4c8" containerName="pull"
Sep 10 17:40:53.448755 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:40:53.448703    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="955f5e73-4be9-4181-af7f-aad43740a4c8" containerName="pull"
Sep 10 17:40:53.448755 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:40:53.448710    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="955f5e73-4be9-4181-af7f-aad43740a4c8" containerName="extract"
Sep 10 17:40:53.448755 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:40:53.448715    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="955f5e73-4be9-4181-af7f-aad43740a4c8" containerName="extract"
Sep 10 17:40:53.448937 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:40:53.448766    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="955f5e73-4be9-4181-af7f-aad43740a4c8" containerName="extract"
Sep 10 17:40:53.454839 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:40:53.454820    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-7cd5576685-rd9h7"
Sep 10 17:40:53.457349 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:40:53.457327    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-creds\""
Sep 10 17:40:53.458373 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:40:53.458351    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-tls\""
Sep 10 17:40:53.458497 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:40:53.458375    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-dockercfg-n5lq4\""
Sep 10 17:40:53.458497 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:40:53.458415    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\""
Sep 10 17:40:53.458497 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:40:53.458428    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\""
Sep 10 17:40:53.463406 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:40:53.463383    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-7cd5576685-rd9h7"]
Sep 10 17:40:53.487166 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:40:53.487141    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/bc13dda3-b684-423a-bf98-165fa717027c-tmp\") pod \"kube-auth-proxy-7cd5576685-rd9h7\" (UID: \"bc13dda3-b684-423a-bf98-165fa717027c\") " pod="openshift-ingress/kube-auth-proxy-7cd5576685-rd9h7"
Sep 10 17:40:53.487283 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:40:53.487177    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ddh98\" (UniqueName: \"kubernetes.io/projected/bc13dda3-b684-423a-bf98-165fa717027c-kube-api-access-ddh98\") pod \"kube-auth-proxy-7cd5576685-rd9h7\" (UID: \"bc13dda3-b684-423a-bf98-165fa717027c\") " pod="openshift-ingress/kube-auth-proxy-7cd5576685-rd9h7"
Sep 10 17:40:53.487283 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:40:53.487212    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/bc13dda3-b684-423a-bf98-165fa717027c-tls-certs\") pod \"kube-auth-proxy-7cd5576685-rd9h7\" (UID: \"bc13dda3-b684-423a-bf98-165fa717027c\") " pod="openshift-ingress/kube-auth-proxy-7cd5576685-rd9h7"
Sep 10 17:40:53.587581 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:40:53.587547    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/bc13dda3-b684-423a-bf98-165fa717027c-tmp\") pod \"kube-auth-proxy-7cd5576685-rd9h7\" (UID: \"bc13dda3-b684-423a-bf98-165fa717027c\") " pod="openshift-ingress/kube-auth-proxy-7cd5576685-rd9h7"
Sep 10 17:40:53.587734 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:40:53.587586    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ddh98\" (UniqueName: \"kubernetes.io/projected/bc13dda3-b684-423a-bf98-165fa717027c-kube-api-access-ddh98\") pod \"kube-auth-proxy-7cd5576685-rd9h7\" (UID: \"bc13dda3-b684-423a-bf98-165fa717027c\") " pod="openshift-ingress/kube-auth-proxy-7cd5576685-rd9h7"
Sep 10 17:40:53.587734 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:40:53.587620    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/bc13dda3-b684-423a-bf98-165fa717027c-tls-certs\") pod \"kube-auth-proxy-7cd5576685-rd9h7\" (UID: \"bc13dda3-b684-423a-bf98-165fa717027c\") " pod="openshift-ingress/kube-auth-proxy-7cd5576685-rd9h7"
Sep 10 17:40:53.589787 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:40:53.589766    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/bc13dda3-b684-423a-bf98-165fa717027c-tmp\") pod \"kube-auth-proxy-7cd5576685-rd9h7\" (UID: \"bc13dda3-b684-423a-bf98-165fa717027c\") " pod="openshift-ingress/kube-auth-proxy-7cd5576685-rd9h7"
Sep 10 17:40:53.590001 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:40:53.589982    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/bc13dda3-b684-423a-bf98-165fa717027c-tls-certs\") pod \"kube-auth-proxy-7cd5576685-rd9h7\" (UID: \"bc13dda3-b684-423a-bf98-165fa717027c\") " pod="openshift-ingress/kube-auth-proxy-7cd5576685-rd9h7"
Sep 10 17:40:53.602071 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:40:53.602049    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ddh98\" (UniqueName: \"kubernetes.io/projected/bc13dda3-b684-423a-bf98-165fa717027c-kube-api-access-ddh98\") pod \"kube-auth-proxy-7cd5576685-rd9h7\" (UID: \"bc13dda3-b684-423a-bf98-165fa717027c\") " pod="openshift-ingress/kube-auth-proxy-7cd5576685-rd9h7"
Sep 10 17:40:53.764453 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:40:53.764425    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-7cd5576685-rd9h7"
Sep 10 17:40:54.038143 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:40:54.038121    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-7cd5576685-rd9h7"]
Sep 10 17:40:54.040035 ip-10-0-141-180 kubenswrapper[2581]: W0910 17:40:54.040007    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbc13dda3_b684_423a_bf98_165fa717027c.slice/crio-6ed033b7c78f90be4c73183dd691c5e3fba0612de0afa7ae1b23d58c17890078 WatchSource:0}: Error finding container 6ed033b7c78f90be4c73183dd691c5e3fba0612de0afa7ae1b23d58c17890078: Status 404 returned error can't find the container with id 6ed033b7c78f90be4c73183dd691c5e3fba0612de0afa7ae1b23d58c17890078
Sep 10 17:40:54.056511 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:40:54.056485    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-7cd5576685-rd9h7" event={"ID":"bc13dda3-b684-423a-bf98-165fa717027c","Type":"ContainerStarted","Data":"6ed033b7c78f90be4c73183dd691c5e3fba0612de0afa7ae1b23d58c17890078"}
Sep 10 17:40:57.073297 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:40:57.073260    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-7cd5576685-rd9h7" event={"ID":"bc13dda3-b684-423a-bf98-165fa717027c","Type":"ContainerStarted","Data":"544829924f6ab15020d6a12f42d810bc4ecac80f200316715fe34bface9bea32"}
Sep 10 17:40:57.090316 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:40:57.090263    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/kube-auth-proxy-7cd5576685-rd9h7" podStartSLOduration=1.443009347 podStartE2EDuration="4.090249697s" totalImagesPullingTime="2.64724035s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 17:40:53 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 17:40:57.088961995 +0000 UTC m=+276.743035900" watchObservedRunningTime="2026-09-10 17:40:57.090249697 +0000 UTC m=+276.744323602"
Sep 10 17:40:59.879396 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:40:59.879363    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-kb6rp"]
Sep 10 17:40:59.882897 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:40:59.882878    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-kb6rp"
Sep 10 17:40:59.885710 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:40:59.885687    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"data-science-gateway-data-science-gateway-class-dockercfg-gwbcl\""
Sep 10 17:40:59.885838 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:40:59.885777    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\""
Sep 10 17:40:59.894487 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:40:59.894464    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-kb6rp"]
Sep 10 17:40:59.938550 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:40:59.938514    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9097a0bb-c99e-4edb-b9a2-5c140c8c2773-istio-token\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-kb6rp\" (UID: \"9097a0bb-c99e-4edb-b9a2-5c140c8c2773\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-kb6rp"
Sep 10 17:40:59.938699 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:40:59.938559    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9097a0bb-c99e-4edb-b9a2-5c140c8c2773-istio-data\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-kb6rp\" (UID: \"9097a0bb-c99e-4edb-b9a2-5c140c8c2773\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-kb6rp"
Sep 10 17:40:59.938699 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:40:59.938632    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9097a0bb-c99e-4edb-b9a2-5c140c8c2773-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-kb6rp\" (UID: \"9097a0bb-c99e-4edb-b9a2-5c140c8c2773\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-kb6rp"
Sep 10 17:40:59.938699 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:40:59.938678    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9097a0bb-c99e-4edb-b9a2-5c140c8c2773-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-kb6rp\" (UID: \"9097a0bb-c99e-4edb-b9a2-5c140c8c2773\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-kb6rp"
Sep 10 17:40:59.938867 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:40:59.938703    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9097a0bb-c99e-4edb-b9a2-5c140c8c2773-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-kb6rp\" (UID: \"9097a0bb-c99e-4edb-b9a2-5c140c8c2773\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-kb6rp"
Sep 10 17:40:59.938867 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:40:59.938754    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9097a0bb-c99e-4edb-b9a2-5c140c8c2773-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-kb6rp\" (UID: \"9097a0bb-c99e-4edb-b9a2-5c140c8c2773\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-kb6rp"
Sep 10 17:40:59.938867 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:40:59.938770    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zxccc\" (UniqueName: \"kubernetes.io/projected/9097a0bb-c99e-4edb-b9a2-5c140c8c2773-kube-api-access-zxccc\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-kb6rp\" (UID: \"9097a0bb-c99e-4edb-b9a2-5c140c8c2773\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-kb6rp"
Sep 10 17:40:59.938867 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:40:59.938807    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9097a0bb-c99e-4edb-b9a2-5c140c8c2773-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-kb6rp\" (UID: \"9097a0bb-c99e-4edb-b9a2-5c140c8c2773\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-kb6rp"
Sep 10 17:40:59.938867 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:40:59.938840    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9097a0bb-c99e-4edb-b9a2-5c140c8c2773-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-kb6rp\" (UID: \"9097a0bb-c99e-4edb-b9a2-5c140c8c2773\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-kb6rp"
Sep 10 17:41:00.039949 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:41:00.039910    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9097a0bb-c99e-4edb-b9a2-5c140c8c2773-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-kb6rp\" (UID: \"9097a0bb-c99e-4edb-b9a2-5c140c8c2773\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-kb6rp"
Sep 10 17:41:00.040128 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:41:00.039964    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9097a0bb-c99e-4edb-b9a2-5c140c8c2773-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-kb6rp\" (UID: \"9097a0bb-c99e-4edb-b9a2-5c140c8c2773\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-kb6rp"
Sep 10 17:41:00.040128 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:41:00.039996    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9097a0bb-c99e-4edb-b9a2-5c140c8c2773-istio-token\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-kb6rp\" (UID: \"9097a0bb-c99e-4edb-b9a2-5c140c8c2773\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-kb6rp"
Sep 10 17:41:00.040128 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:41:00.040025    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9097a0bb-c99e-4edb-b9a2-5c140c8c2773-istio-data\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-kb6rp\" (UID: \"9097a0bb-c99e-4edb-b9a2-5c140c8c2773\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-kb6rp"
Sep 10 17:41:00.040128 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:41:00.040066    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9097a0bb-c99e-4edb-b9a2-5c140c8c2773-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-kb6rp\" (UID: \"9097a0bb-c99e-4edb-b9a2-5c140c8c2773\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-kb6rp"
Sep 10 17:41:00.040128 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:41:00.040089    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9097a0bb-c99e-4edb-b9a2-5c140c8c2773-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-kb6rp\" (UID: \"9097a0bb-c99e-4edb-b9a2-5c140c8c2773\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-kb6rp"
Sep 10 17:41:00.040128 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:41:00.040122    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9097a0bb-c99e-4edb-b9a2-5c140c8c2773-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-kb6rp\" (UID: \"9097a0bb-c99e-4edb-b9a2-5c140c8c2773\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-kb6rp"
Sep 10 17:41:00.040431 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:41:00.040170    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9097a0bb-c99e-4edb-b9a2-5c140c8c2773-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-kb6rp\" (UID: \"9097a0bb-c99e-4edb-b9a2-5c140c8c2773\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-kb6rp"
Sep 10 17:41:00.040431 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:41:00.040196    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zxccc\" (UniqueName: \"kubernetes.io/projected/9097a0bb-c99e-4edb-b9a2-5c140c8c2773-kube-api-access-zxccc\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-kb6rp\" (UID: \"9097a0bb-c99e-4edb-b9a2-5c140c8c2773\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-kb6rp"
Sep 10 17:41:00.040431 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:41:00.040369    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9097a0bb-c99e-4edb-b9a2-5c140c8c2773-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-kb6rp\" (UID: \"9097a0bb-c99e-4edb-b9a2-5c140c8c2773\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-kb6rp"
Sep 10 17:41:00.040431 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:41:00.040384    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9097a0bb-c99e-4edb-b9a2-5c140c8c2773-istio-data\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-kb6rp\" (UID: \"9097a0bb-c99e-4edb-b9a2-5c140c8c2773\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-kb6rp"
Sep 10 17:41:00.040666 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:41:00.040430    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9097a0bb-c99e-4edb-b9a2-5c140c8c2773-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-kb6rp\" (UID: \"9097a0bb-c99e-4edb-b9a2-5c140c8c2773\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-kb6rp"
Sep 10 17:41:00.040666 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:41:00.040521    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9097a0bb-c99e-4edb-b9a2-5c140c8c2773-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-kb6rp\" (UID: \"9097a0bb-c99e-4edb-b9a2-5c140c8c2773\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-kb6rp"
Sep 10 17:41:00.040945 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:41:00.040923    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9097a0bb-c99e-4edb-b9a2-5c140c8c2773-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-kb6rp\" (UID: \"9097a0bb-c99e-4edb-b9a2-5c140c8c2773\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-kb6rp"
Sep 10 17:41:00.042452 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:41:00.042429    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9097a0bb-c99e-4edb-b9a2-5c140c8c2773-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-kb6rp\" (UID: \"9097a0bb-c99e-4edb-b9a2-5c140c8c2773\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-kb6rp"
Sep 10 17:41:00.042544 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:41:00.042529    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9097a0bb-c99e-4edb-b9a2-5c140c8c2773-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-kb6rp\" (UID: \"9097a0bb-c99e-4edb-b9a2-5c140c8c2773\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-kb6rp"
Sep 10 17:41:00.055514 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:41:00.055490    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zxccc\" (UniqueName: \"kubernetes.io/projected/9097a0bb-c99e-4edb-b9a2-5c140c8c2773-kube-api-access-zxccc\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-kb6rp\" (UID: \"9097a0bb-c99e-4edb-b9a2-5c140c8c2773\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-kb6rp"
Sep 10 17:41:00.058208 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:41:00.058192    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9097a0bb-c99e-4edb-b9a2-5c140c8c2773-istio-token\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-kb6rp\" (UID: \"9097a0bb-c99e-4edb-b9a2-5c140c8c2773\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-kb6rp"
Sep 10 17:41:00.194025 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:41:00.193945    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-kb6rp"
Sep 10 17:41:00.474246 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:41:00.474222    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-kb6rp"]
Sep 10 17:41:00.476257 ip-10-0-141-180 kubenswrapper[2581]: W0910 17:41:00.476232    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9097a0bb_c99e_4edb_b9a2_5c140c8c2773.slice/crio-ccfee9f1badc2bd775414b0dee63909a6fdeaf5a93987211278123ddaa89d080 WatchSource:0}: Error finding container ccfee9f1badc2bd775414b0dee63909a6fdeaf5a93987211278123ddaa89d080: Status 404 returned error can't find the container with id ccfee9f1badc2bd775414b0dee63909a6fdeaf5a93987211278123ddaa89d080
Sep 10 17:41:01.091343 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:41:01.091300    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-kb6rp" event={"ID":"9097a0bb-c99e-4edb-b9a2-5c140c8c2773","Type":"ContainerStarted","Data":"ccfee9f1badc2bd775414b0dee63909a6fdeaf5a93987211278123ddaa89d080"}
Sep 10 17:41:03.263482 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:41:03.263439    2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 10 17:41:03.263867 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:41:03.263516    2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 10 17:41:03.263867 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:41:03.263546    2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 10 17:41:03.906126 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:41:03.906092    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/odh-model-controller-7998c67dc4-g8jts"]
Sep 10 17:41:03.909574 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:41:03.909547    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/odh-model-controller-7998c67dc4-g8jts"
Sep 10 17:41:03.912033 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:41:03.912010    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"odh-model-controller-dockercfg-kbcpc\""
Sep 10 17:41:03.912244 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:41:03.912226    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"odh-model-controller-webhook-cert\""
Sep 10 17:41:03.925067 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:41:03.925039    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/odh-model-controller-7998c67dc4-g8jts"]
Sep 10 17:41:03.973918 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:41:03.973792    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mzbcm\" (UniqueName: \"kubernetes.io/projected/9b5d3298-2120-43a2-b2e5-50434143c3df-kube-api-access-mzbcm\") pod \"odh-model-controller-7998c67dc4-g8jts\" (UID: \"9b5d3298-2120-43a2-b2e5-50434143c3df\") " pod="opendatahub/odh-model-controller-7998c67dc4-g8jts"
Sep 10 17:41:03.973918 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:41:03.973878    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/9b5d3298-2120-43a2-b2e5-50434143c3df-cert\") pod \"odh-model-controller-7998c67dc4-g8jts\" (UID: \"9b5d3298-2120-43a2-b2e5-50434143c3df\") " pod="opendatahub/odh-model-controller-7998c67dc4-g8jts"
Sep 10 17:41:04.074414 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:41:04.074373    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/9b5d3298-2120-43a2-b2e5-50434143c3df-cert\") pod \"odh-model-controller-7998c67dc4-g8jts\" (UID: \"9b5d3298-2120-43a2-b2e5-50434143c3df\") " pod="opendatahub/odh-model-controller-7998c67dc4-g8jts"
Sep 10 17:41:04.074572 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:41:04.074438    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mzbcm\" (UniqueName: \"kubernetes.io/projected/9b5d3298-2120-43a2-b2e5-50434143c3df-kube-api-access-mzbcm\") pod \"odh-model-controller-7998c67dc4-g8jts\" (UID: \"9b5d3298-2120-43a2-b2e5-50434143c3df\") " pod="opendatahub/odh-model-controller-7998c67dc4-g8jts"
Sep 10 17:41:04.077035 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:41:04.076999    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/9b5d3298-2120-43a2-b2e5-50434143c3df-cert\") pod \"odh-model-controller-7998c67dc4-g8jts\" (UID: \"9b5d3298-2120-43a2-b2e5-50434143c3df\") " pod="opendatahub/odh-model-controller-7998c67dc4-g8jts"
Sep 10 17:41:04.083653 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:41:04.083612    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mzbcm\" (UniqueName: \"kubernetes.io/projected/9b5d3298-2120-43a2-b2e5-50434143c3df-kube-api-access-mzbcm\") pod \"odh-model-controller-7998c67dc4-g8jts\" (UID: \"9b5d3298-2120-43a2-b2e5-50434143c3df\") " pod="opendatahub/odh-model-controller-7998c67dc4-g8jts"
Sep 10 17:41:04.106909 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:41:04.106881    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-kb6rp" event={"ID":"9097a0bb-c99e-4edb-b9a2-5c140c8c2773","Type":"ContainerStarted","Data":"d38f2927857ae7a564bbeb5e6640884d3f22ae096f765b950c733b6c3bb9a444"}
Sep 10 17:41:04.126263 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:41:04.126221    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-kb6rp" podStartSLOduration=2.341277938 podStartE2EDuration="5.12620421s" totalImagesPullingTime="2.784926272s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 17:40:59 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 17:41:04.125405548 +0000 UTC m=+283.779479462" watchObservedRunningTime="2026-09-10 17:41:04.12620421 +0000 UTC m=+283.780278115"
Sep 10 17:41:04.194585 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:41:04.194504    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-kb6rp"
Sep 10 17:41:04.199719 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:41:04.199696    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-kb6rp"
Sep 10 17:41:04.220153 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:41:04.220123    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/odh-model-controller-7998c67dc4-g8jts"
Sep 10 17:41:04.431947 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:41:04.431908    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-j5fcj"]
Sep 10 17:41:04.436337 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:41:04.436313    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/model-serving-api-54d89bd5fc-j5fcj"
Sep 10 17:41:04.438889 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:41:04.438866    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"model-serving-api-tls\""
Sep 10 17:41:04.444959 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:41:04.444916    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-j5fcj"]
Sep 10 17:41:04.478499 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:41:04.478470    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/aa43a8cc-acb7-4845-bca6-e90b5848d533-tls-certs\") pod \"model-serving-api-54d89bd5fc-j5fcj\" (UID: \"aa43a8cc-acb7-4845-bca6-e90b5848d533\") " pod="opendatahub/model-serving-api-54d89bd5fc-j5fcj"
Sep 10 17:41:04.478617 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:41:04.478512    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xmbsb\" (UniqueName: \"kubernetes.io/projected/aa43a8cc-acb7-4845-bca6-e90b5848d533-kube-api-access-xmbsb\") pod \"model-serving-api-54d89bd5fc-j5fcj\" (UID: \"aa43a8cc-acb7-4845-bca6-e90b5848d533\") " pod="opendatahub/model-serving-api-54d89bd5fc-j5fcj"
Sep 10 17:41:04.515519 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:41:04.515495    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/odh-model-controller-7998c67dc4-g8jts"]
Sep 10 17:41:04.517840 ip-10-0-141-180 kubenswrapper[2581]: W0910 17:41:04.517815    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9b5d3298_2120_43a2_b2e5_50434143c3df.slice/crio-5eff08941d738cfe3c9c28c4b8344fc48f8abfbf5390b60327194424755d5f95 WatchSource:0}: Error finding container 5eff08941d738cfe3c9c28c4b8344fc48f8abfbf5390b60327194424755d5f95: Status 404 returned error can't find the container with id 5eff08941d738cfe3c9c28c4b8344fc48f8abfbf5390b60327194424755d5f95
Sep 10 17:41:04.579891 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:41:04.579847    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/aa43a8cc-acb7-4845-bca6-e90b5848d533-tls-certs\") pod \"model-serving-api-54d89bd5fc-j5fcj\" (UID: \"aa43a8cc-acb7-4845-bca6-e90b5848d533\") " pod="opendatahub/model-serving-api-54d89bd5fc-j5fcj"
Sep 10 17:41:04.580011 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:41:04.579903    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xmbsb\" (UniqueName: \"kubernetes.io/projected/aa43a8cc-acb7-4845-bca6-e90b5848d533-kube-api-access-xmbsb\") pod \"model-serving-api-54d89bd5fc-j5fcj\" (UID: \"aa43a8cc-acb7-4845-bca6-e90b5848d533\") " pod="opendatahub/model-serving-api-54d89bd5fc-j5fcj"
Sep 10 17:41:04.582184 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:41:04.582166    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/aa43a8cc-acb7-4845-bca6-e90b5848d533-tls-certs\") pod \"model-serving-api-54d89bd5fc-j5fcj\" (UID: \"aa43a8cc-acb7-4845-bca6-e90b5848d533\") " pod="opendatahub/model-serving-api-54d89bd5fc-j5fcj"
Sep 10 17:41:04.588261 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:41:04.588242    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xmbsb\" (UniqueName: \"kubernetes.io/projected/aa43a8cc-acb7-4845-bca6-e90b5848d533-kube-api-access-xmbsb\") pod \"model-serving-api-54d89bd5fc-j5fcj\" (UID: \"aa43a8cc-acb7-4845-bca6-e90b5848d533\") " pod="opendatahub/model-serving-api-54d89bd5fc-j5fcj"
Sep 10 17:41:04.748777 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:41:04.748743    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/model-serving-api-54d89bd5fc-j5fcj"
Sep 10 17:41:05.016163 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:41:05.016137    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-j5fcj"]
Sep 10 17:41:05.017632 ip-10-0-141-180 kubenswrapper[2581]: W0910 17:41:05.017609    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podaa43a8cc_acb7_4845_bca6_e90b5848d533.slice/crio-fcb2964f45c0fe2db5ee1089aba12d39bc678bf819087ca9ad5a1fb06bab9be7 WatchSource:0}: Error finding container fcb2964f45c0fe2db5ee1089aba12d39bc678bf819087ca9ad5a1fb06bab9be7: Status 404 returned error can't find the container with id fcb2964f45c0fe2db5ee1089aba12d39bc678bf819087ca9ad5a1fb06bab9be7
Sep 10 17:41:05.112928 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:41:05.112899    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/odh-model-controller-7998c67dc4-g8jts" event={"ID":"9b5d3298-2120-43a2-b2e5-50434143c3df","Type":"ContainerStarted","Data":"5eff08941d738cfe3c9c28c4b8344fc48f8abfbf5390b60327194424755d5f95"}
Sep 10 17:41:05.114059 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:41:05.114033    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/model-serving-api-54d89bd5fc-j5fcj" event={"ID":"aa43a8cc-acb7-4845-bca6-e90b5848d533","Type":"ContainerStarted","Data":"fcb2964f45c0fe2db5ee1089aba12d39bc678bf819087ca9ad5a1fb06bab9be7"}
Sep 10 17:41:05.114349 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:41:05.114330    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-kb6rp"
Sep 10 17:41:05.115295 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:41:05.115274    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-kb6rp"
Sep 10 17:41:08.131035 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:41:08.130996    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/odh-model-controller-7998c67dc4-g8jts" event={"ID":"9b5d3298-2120-43a2-b2e5-50434143c3df","Type":"ContainerStarted","Data":"9c0e30a80c46b2ca754248c1be4b063895f3aa389bac39aee14371346191df0c"}
Sep 10 17:41:08.131471 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:41:08.131097    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/odh-model-controller-7998c67dc4-g8jts"
Sep 10 17:41:08.132478 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:41:08.132447    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/model-serving-api-54d89bd5fc-j5fcj" event={"ID":"aa43a8cc-acb7-4845-bca6-e90b5848d533","Type":"ContainerStarted","Data":"491e0a880cbf192454fc8577cd9ff0cd01d643dd843de6fc7c0b5630f2d768bc"}
Sep 10 17:41:08.132768 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:41:08.132504    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/model-serving-api-54d89bd5fc-j5fcj"
Sep 10 17:41:08.147522 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:41:08.147463    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/odh-model-controller-7998c67dc4-g8jts" podStartSLOduration=1.702938042 podStartE2EDuration="5.147441418s" totalImagesPullingTime="3.444503376s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 17:41:03 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 17:41:08.146957551 +0000 UTC m=+287.801031484" watchObservedRunningTime="2026-09-10 17:41:08.147441418 +0000 UTC m=+287.801515342"
Sep 10 17:41:08.164196 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:41:08.164146    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/model-serving-api-54d89bd5fc-j5fcj" podStartSLOduration=1.230676309 podStartE2EDuration="4.164127529s" totalImagesPullingTime="2.93345122s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 17:41:04 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 17:41:08.162929005 +0000 UTC m=+287.817002909" watchObservedRunningTime="2026-09-10 17:41:08.164127529 +0000 UTC m=+287.818201432"
Sep 10 17:41:10.916548 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:41:10.916511    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/istiod-openshift-gateway-5677ccb49b-5jhrh"]
Sep 10 17:41:10.923404 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:41:10.923372    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-5jhrh"
Sep 10 17:41:10.926487 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:41:10.926463    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-tls\""
Sep 10 17:41:10.926631 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:41:10.926575    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-openshift-gateway-dockercfg-tn85x\""
Sep 10 17:41:10.927310 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:41:10.927294    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"cacerts\""
Sep 10 17:41:10.927388 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:41:10.927357    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istio-kubeconfig\""
Sep 10 17:41:10.952978 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:41:10.952917    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-5677ccb49b-5jhrh"]
Sep 10 17:41:11.036306 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:41:11.036274    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1cc585ba-5015-43e9-9752-1b4a288933bd-istio-token\") pod \"istiod-openshift-gateway-5677ccb49b-5jhrh\" (UID: \"1cc585ba-5015-43e9-9752-1b4a288933bd\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-5jhrh"
Sep 10 17:41:11.036306 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:41:11.036310    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/1cc585ba-5015-43e9-9752-1b4a288933bd-local-certs\") pod \"istiod-openshift-gateway-5677ccb49b-5jhrh\" (UID: \"1cc585ba-5015-43e9-9752-1b4a288933bd\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-5jhrh"
Sep 10 17:41:11.036504 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:41:11.036332    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/1cc585ba-5015-43e9-9752-1b4a288933bd-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-5677ccb49b-5jhrh\" (UID: \"1cc585ba-5015-43e9-9752-1b4a288933bd\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-5jhrh"
Sep 10 17:41:11.036504 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:41:11.036365    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/1cc585ba-5015-43e9-9752-1b4a288933bd-istio-kubeconfig\") pod \"istiod-openshift-gateway-5677ccb49b-5jhrh\" (UID: \"1cc585ba-5015-43e9-9752-1b4a288933bd\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-5jhrh"
Sep 10 17:41:11.036504 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:41:11.036427    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/1cc585ba-5015-43e9-9752-1b4a288933bd-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-5677ccb49b-5jhrh\" (UID: \"1cc585ba-5015-43e9-9752-1b4a288933bd\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-5jhrh"
Sep 10 17:41:11.036609 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:41:11.036507    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-frpmh\" (UniqueName: \"kubernetes.io/projected/1cc585ba-5015-43e9-9752-1b4a288933bd-kube-api-access-frpmh\") pod \"istiod-openshift-gateway-5677ccb49b-5jhrh\" (UID: \"1cc585ba-5015-43e9-9752-1b4a288933bd\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-5jhrh"
Sep 10 17:41:11.036609 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:41:11.036570    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/1cc585ba-5015-43e9-9752-1b4a288933bd-cacerts\") pod \"istiod-openshift-gateway-5677ccb49b-5jhrh\" (UID: \"1cc585ba-5015-43e9-9752-1b4a288933bd\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-5jhrh"
Sep 10 17:41:11.138069 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:41:11.138033    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/1cc585ba-5015-43e9-9752-1b4a288933bd-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-5677ccb49b-5jhrh\" (UID: \"1cc585ba-5015-43e9-9752-1b4a288933bd\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-5jhrh"
Sep 10 17:41:11.138069 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:41:11.138071    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-frpmh\" (UniqueName: \"kubernetes.io/projected/1cc585ba-5015-43e9-9752-1b4a288933bd-kube-api-access-frpmh\") pod \"istiod-openshift-gateway-5677ccb49b-5jhrh\" (UID: \"1cc585ba-5015-43e9-9752-1b4a288933bd\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-5jhrh"
Sep 10 17:41:11.138305 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:41:11.138092    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/1cc585ba-5015-43e9-9752-1b4a288933bd-cacerts\") pod \"istiod-openshift-gateway-5677ccb49b-5jhrh\" (UID: \"1cc585ba-5015-43e9-9752-1b4a288933bd\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-5jhrh"
Sep 10 17:41:11.138305 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:41:11.138132    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1cc585ba-5015-43e9-9752-1b4a288933bd-istio-token\") pod \"istiod-openshift-gateway-5677ccb49b-5jhrh\" (UID: \"1cc585ba-5015-43e9-9752-1b4a288933bd\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-5jhrh"
Sep 10 17:41:11.138395 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:41:11.138298    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/1cc585ba-5015-43e9-9752-1b4a288933bd-local-certs\") pod \"istiod-openshift-gateway-5677ccb49b-5jhrh\" (UID: \"1cc585ba-5015-43e9-9752-1b4a288933bd\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-5jhrh"
Sep 10 17:41:11.138395 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:41:11.138337    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/1cc585ba-5015-43e9-9752-1b4a288933bd-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-5677ccb49b-5jhrh\" (UID: \"1cc585ba-5015-43e9-9752-1b4a288933bd\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-5jhrh"
Sep 10 17:41:11.138395 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:41:11.138376    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/1cc585ba-5015-43e9-9752-1b4a288933bd-istio-kubeconfig\") pod \"istiod-openshift-gateway-5677ccb49b-5jhrh\" (UID: \"1cc585ba-5015-43e9-9752-1b4a288933bd\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-5jhrh"
Sep 10 17:41:11.139122 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:41:11.139091    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/1cc585ba-5015-43e9-9752-1b4a288933bd-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-5677ccb49b-5jhrh\" (UID: \"1cc585ba-5015-43e9-9752-1b4a288933bd\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-5jhrh"
Sep 10 17:41:11.140538 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:41:11.140512    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/1cc585ba-5015-43e9-9752-1b4a288933bd-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-5677ccb49b-5jhrh\" (UID: \"1cc585ba-5015-43e9-9752-1b4a288933bd\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-5jhrh"
Sep 10 17:41:11.140695 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:41:11.140515    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/1cc585ba-5015-43e9-9752-1b4a288933bd-local-certs\") pod \"istiod-openshift-gateway-5677ccb49b-5jhrh\" (UID: \"1cc585ba-5015-43e9-9752-1b4a288933bd\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-5jhrh"
Sep 10 17:41:11.140695 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:41:11.140573    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/1cc585ba-5015-43e9-9752-1b4a288933bd-cacerts\") pod \"istiod-openshift-gateway-5677ccb49b-5jhrh\" (UID: \"1cc585ba-5015-43e9-9752-1b4a288933bd\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-5jhrh"
Sep 10 17:41:11.140838 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:41:11.140730    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/1cc585ba-5015-43e9-9752-1b4a288933bd-istio-kubeconfig\") pod \"istiod-openshift-gateway-5677ccb49b-5jhrh\" (UID: \"1cc585ba-5015-43e9-9752-1b4a288933bd\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-5jhrh"
Sep 10 17:41:11.147865 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:41:11.147840    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1cc585ba-5015-43e9-9752-1b4a288933bd-istio-token\") pod \"istiod-openshift-gateway-5677ccb49b-5jhrh\" (UID: \"1cc585ba-5015-43e9-9752-1b4a288933bd\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-5jhrh"
Sep 10 17:41:11.147970 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:41:11.147872    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-frpmh\" (UniqueName: \"kubernetes.io/projected/1cc585ba-5015-43e9-9752-1b4a288933bd-kube-api-access-frpmh\") pod \"istiod-openshift-gateway-5677ccb49b-5jhrh\" (UID: \"1cc585ba-5015-43e9-9752-1b4a288933bd\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-5jhrh"
Sep 10 17:41:11.234816 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:41:11.234788    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-5jhrh"
Sep 10 17:41:11.536840 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:41:11.536812    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-5677ccb49b-5jhrh"]
Sep 10 17:41:11.538173 ip-10-0-141-180 kubenswrapper[2581]: W0910 17:41:11.538148    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1cc585ba_5015_43e9_9752_1b4a288933bd.slice/crio-0fcd62062dbce58764d440194a9b6cdfa4f6e743e8a28bfe2e6bb76944780144 WatchSource:0}: Error finding container 0fcd62062dbce58764d440194a9b6cdfa4f6e743e8a28bfe2e6bb76944780144: Status 404 returned error can't find the container with id 0fcd62062dbce58764d440194a9b6cdfa4f6e743e8a28bfe2e6bb76944780144
Sep 10 17:41:12.154290 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:41:12.154245    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-5jhrh" event={"ID":"1cc585ba-5015-43e9-9752-1b4a288933bd","Type":"ContainerStarted","Data":"0fcd62062dbce58764d440194a9b6cdfa4f6e743e8a28bfe2e6bb76944780144"}
Sep 10 17:41:13.811005 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:41:13.810951    2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 10 17:41:13.811282 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:41:13.811036    2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 10 17:41:13.838244 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:41:13.838220    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/kserve-controller-manager-d6f4b65b8-jpmlv"]
Sep 10 17:41:13.842774 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:41:13.842756    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/kserve-controller-manager-d6f4b65b8-jpmlv"
Sep 10 17:41:13.848275 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:41:13.848257    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"kserve-webhook-server-cert\""
Sep 10 17:41:13.848381 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:41:13.848270    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"kserve-controller-manager-dockercfg-vnhzt\""
Sep 10 17:41:13.859014 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:41:13.858990    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/kserve-controller-manager-d6f4b65b8-jpmlv"]
Sep 10 17:41:13.874212 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:41:13.874188    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-8cr6w"]
Sep 10 17:41:13.878653 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:41:13.878625    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-8cr6w"
Sep 10 17:41:13.881418 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:41:13.881395    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"llmisvc-webhook-server-cert\""
Sep 10 17:41:13.881517 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:41:13.881468    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"llmisvc-controller-manager-dockercfg-7pjkg\""
Sep 10 17:41:13.891158 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:41:13.891103    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-8cr6w"]
Sep 10 17:41:13.966730 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:41:13.966685    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bx88m\" (UniqueName: \"kubernetes.io/projected/726ce400-99e3-4b7b-a9ae-b8c3b0875f5f-kube-api-access-bx88m\") pod \"llmisvc-controller-manager-777c6bd7d5-8cr6w\" (UID: \"726ce400-99e3-4b7b-a9ae-b8c3b0875f5f\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-8cr6w"
Sep 10 17:41:13.966908 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:41:13.966733    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dj7tc\" (UniqueName: \"kubernetes.io/projected/051c5f70-fb7c-4155-bc93-e6354409ac5e-kube-api-access-dj7tc\") pod \"kserve-controller-manager-d6f4b65b8-jpmlv\" (UID: \"051c5f70-fb7c-4155-bc93-e6354409ac5e\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-jpmlv"
Sep 10 17:41:13.966908 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:41:13.966806    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/726ce400-99e3-4b7b-a9ae-b8c3b0875f5f-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-8cr6w\" (UID: \"726ce400-99e3-4b7b-a9ae-b8c3b0875f5f\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-8cr6w"
Sep 10 17:41:13.966908 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:41:13.966864    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/051c5f70-fb7c-4155-bc93-e6354409ac5e-cert\") pod \"kserve-controller-manager-d6f4b65b8-jpmlv\" (UID: \"051c5f70-fb7c-4155-bc93-e6354409ac5e\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-jpmlv"
Sep 10 17:41:14.067897 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:41:14.067802    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bx88m\" (UniqueName: \"kubernetes.io/projected/726ce400-99e3-4b7b-a9ae-b8c3b0875f5f-kube-api-access-bx88m\") pod \"llmisvc-controller-manager-777c6bd7d5-8cr6w\" (UID: \"726ce400-99e3-4b7b-a9ae-b8c3b0875f5f\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-8cr6w"
Sep 10 17:41:14.067897 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:41:14.067839    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dj7tc\" (UniqueName: \"kubernetes.io/projected/051c5f70-fb7c-4155-bc93-e6354409ac5e-kube-api-access-dj7tc\") pod \"kserve-controller-manager-d6f4b65b8-jpmlv\" (UID: \"051c5f70-fb7c-4155-bc93-e6354409ac5e\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-jpmlv"
Sep 10 17:41:14.068132 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:41:14.067945    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/726ce400-99e3-4b7b-a9ae-b8c3b0875f5f-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-8cr6w\" (UID: \"726ce400-99e3-4b7b-a9ae-b8c3b0875f5f\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-8cr6w"
Sep 10 17:41:14.068132 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:41:14.067979    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/051c5f70-fb7c-4155-bc93-e6354409ac5e-cert\") pod \"kserve-controller-manager-d6f4b65b8-jpmlv\" (UID: \"051c5f70-fb7c-4155-bc93-e6354409ac5e\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-jpmlv"
Sep 10 17:41:14.068132 ip-10-0-141-180 kubenswrapper[2581]: E0910 17:41:14.068094    2581 secret.go:189] Couldn't get secret opendatahub/llmisvc-webhook-server-cert: secret "llmisvc-webhook-server-cert" not found
Sep 10 17:41:14.068283 ip-10-0-141-180 kubenswrapper[2581]: E0910 17:41:14.068188    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/726ce400-99e3-4b7b-a9ae-b8c3b0875f5f-cert podName:726ce400-99e3-4b7b-a9ae-b8c3b0875f5f nodeName:}" failed. No retries permitted until 2026-09-10 17:41:14.568166552 +0000 UTC m=+294.222240443 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/726ce400-99e3-4b7b-a9ae-b8c3b0875f5f-cert") pod "llmisvc-controller-manager-777c6bd7d5-8cr6w" (UID: "726ce400-99e3-4b7b-a9ae-b8c3b0875f5f") : secret "llmisvc-webhook-server-cert" not found
Sep 10 17:41:14.070504 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:41:14.070478    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/051c5f70-fb7c-4155-bc93-e6354409ac5e-cert\") pod \"kserve-controller-manager-d6f4b65b8-jpmlv\" (UID: \"051c5f70-fb7c-4155-bc93-e6354409ac5e\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-jpmlv"
Sep 10 17:41:14.082358 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:41:14.082331    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bx88m\" (UniqueName: \"kubernetes.io/projected/726ce400-99e3-4b7b-a9ae-b8c3b0875f5f-kube-api-access-bx88m\") pod \"llmisvc-controller-manager-777c6bd7d5-8cr6w\" (UID: \"726ce400-99e3-4b7b-a9ae-b8c3b0875f5f\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-8cr6w"
Sep 10 17:41:14.082469 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:41:14.082335    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dj7tc\" (UniqueName: \"kubernetes.io/projected/051c5f70-fb7c-4155-bc93-e6354409ac5e-kube-api-access-dj7tc\") pod \"kserve-controller-manager-d6f4b65b8-jpmlv\" (UID: \"051c5f70-fb7c-4155-bc93-e6354409ac5e\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-jpmlv"
Sep 10 17:41:14.162452 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:41:14.162420    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/kserve-controller-manager-d6f4b65b8-jpmlv"
Sep 10 17:41:14.168134 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:41:14.168102    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-5jhrh" event={"ID":"1cc585ba-5015-43e9-9752-1b4a288933bd","Type":"ContainerStarted","Data":"0c37220d190137b8ba510c34a394f137ab853eb405dc1440ea92d8fd94db2898"}
Sep 10 17:41:14.168343 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:41:14.168324    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-5jhrh"
Sep 10 17:41:14.170039 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:41:14.170013    2581 patch_prober.go:28] interesting pod/istiod-openshift-gateway-5677ccb49b-5jhrh container/discovery namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Sep 10 17:41:14.170119 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:41:14.170069    2581 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-5jhrh" podUID="1cc585ba-5015-43e9-9752-1b4a288933bd" containerName="discovery" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Sep 10 17:41:14.207882 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:41:14.207826    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-5jhrh" podStartSLOduration=1.937507497 podStartE2EDuration="4.207805219s" totalImagesPullingTime="2.270297722s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 17:41:10 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 17:41:14.204917419 +0000 UTC m=+293.858991323" watchObservedRunningTime="2026-09-10 17:41:14.207805219 +0000 UTC m=+293.861879124"
Sep 10 17:41:14.437273 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:41:14.437238    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/kserve-controller-manager-d6f4b65b8-jpmlv"]
Sep 10 17:41:14.439661 ip-10-0-141-180 kubenswrapper[2581]: W0910 17:41:14.439611    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod051c5f70_fb7c_4155_bc93_e6354409ac5e.slice/crio-0c1343981f0c11b6ccfe72305682b52d118526789041b85e4a1838955839fa1a WatchSource:0}: Error finding container 0c1343981f0c11b6ccfe72305682b52d118526789041b85e4a1838955839fa1a: Status 404 returned error can't find the container with id 0c1343981f0c11b6ccfe72305682b52d118526789041b85e4a1838955839fa1a
Sep 10 17:41:14.572765 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:41:14.572711    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/726ce400-99e3-4b7b-a9ae-b8c3b0875f5f-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-8cr6w\" (UID: \"726ce400-99e3-4b7b-a9ae-b8c3b0875f5f\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-8cr6w"
Sep 10 17:41:14.575179 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:41:14.575156    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/726ce400-99e3-4b7b-a9ae-b8c3b0875f5f-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-8cr6w\" (UID: \"726ce400-99e3-4b7b-a9ae-b8c3b0875f5f\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-8cr6w"
Sep 10 17:41:14.789447 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:41:14.789413    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-8cr6w"
Sep 10 17:41:15.087358 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:41:15.087325    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-8cr6w"]
Sep 10 17:41:15.089874 ip-10-0-141-180 kubenswrapper[2581]: W0910 17:41:15.089846    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pod726ce400_99e3_4b7b_a9ae_b8c3b0875f5f.slice/crio-9d78a03508d8bf5eb8b0d979295b1841a8c4db9f7b8ad75ddf6bfe31efe137c3 WatchSource:0}: Error finding container 9d78a03508d8bf5eb8b0d979295b1841a8c4db9f7b8ad75ddf6bfe31efe137c3: Status 404 returned error can't find the container with id 9d78a03508d8bf5eb8b0d979295b1841a8c4db9f7b8ad75ddf6bfe31efe137c3
Sep 10 17:41:15.174960 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:41:15.174923    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/kserve-controller-manager-d6f4b65b8-jpmlv" event={"ID":"051c5f70-fb7c-4155-bc93-e6354409ac5e","Type":"ContainerStarted","Data":"0c1343981f0c11b6ccfe72305682b52d118526789041b85e4a1838955839fa1a"}
Sep 10 17:41:15.176050 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:41:15.176025    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-8cr6w" event={"ID":"726ce400-99e3-4b7b-a9ae-b8c3b0875f5f","Type":"ContainerStarted","Data":"9d78a03508d8bf5eb8b0d979295b1841a8c4db9f7b8ad75ddf6bfe31efe137c3"}
Sep 10 17:41:15.176906 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:41:15.176889    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-5jhrh"
Sep 10 17:41:17.191157 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:41:17.191111    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/kserve-controller-manager-d6f4b65b8-jpmlv" event={"ID":"051c5f70-fb7c-4155-bc93-e6354409ac5e","Type":"ContainerStarted","Data":"e407135498ba26b194641b2975a1100c3870e563aee82b88bf2a8326b078f0ee"}
Sep 10 17:41:17.191550 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:41:17.191224    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/kserve-controller-manager-d6f4b65b8-jpmlv"
Sep 10 17:41:17.215244 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:41:17.215193    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/kserve-controller-manager-d6f4b65b8-jpmlv" podStartSLOduration=1.7509034479999999 podStartE2EDuration="4.215177894s" totalImagesPullingTime="2.464274446s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 17:41:13 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 17:41:17.214431653 +0000 UTC m=+296.868505579" watchObservedRunningTime="2026-09-10 17:41:17.215177894 +0000 UTC m=+296.869251798"
Sep 10 17:41:18.197894 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:41:18.197790    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-8cr6w" event={"ID":"726ce400-99e3-4b7b-a9ae-b8c3b0875f5f","Type":"ContainerStarted","Data":"312e9202f37ee87e93e0199b576b06fbf688793988ba709dfb75ce9c736e957c"}
Sep 10 17:41:18.216207 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:41:18.216159    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-8cr6w" podStartSLOduration=2.589790687 podStartE2EDuration="5.216144332s" totalImagesPullingTime="2.626353645s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 17:41:13 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 17:41:18.214802004 +0000 UTC m=+297.868875935" watchObservedRunningTime="2026-09-10 17:41:18.216144332 +0000 UTC m=+297.870218272"
Sep 10 17:41:19.139721 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:41:19.139686    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/odh-model-controller-7998c67dc4-g8jts"
Sep 10 17:41:19.141564 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:41:19.141538    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/model-serving-api-54d89bd5fc-j5fcj"
Sep 10 17:41:19.204335 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:41:19.204307    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-8cr6w"
Sep 10 17:41:20.763789 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:41:20.763766    2581 kubelet.go:1648] "Image garbage collection succeeded"
Sep 10 17:41:48.202854 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:41:48.202823    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/kserve-controller-manager-d6f4b65b8-jpmlv"
Sep 10 17:41:50.211418 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:41:50.211386    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-8cr6w"
Sep 10 17:42:09.071824 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:42:09.071794    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847a98dnt"]
Sep 10 17:42:09.083349 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:42:09.083307    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847a98dnt"
Sep 10 17:42:09.085571 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:42:09.085546    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847a98dnt"]
Sep 10 17:42:09.086177 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:42:09.086159    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\""
Sep 10 17:42:09.087295 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:42:09.087272    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-2mcxw\""
Sep 10 17:42:09.087388 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:42:09.087335    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\""
Sep 10 17:42:09.142573 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:42:09.142546    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/b7d115cc-a234-426d-8809-0dcd578f5ea8-bundle\") pod \"6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847a98dnt\" (UID: \"b7d115cc-a234-426d-8809-0dcd578f5ea8\") " pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847a98dnt"
Sep 10 17:42:09.142733 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:42:09.142582    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/b7d115cc-a234-426d-8809-0dcd578f5ea8-util\") pod \"6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847a98dnt\" (UID: \"b7d115cc-a234-426d-8809-0dcd578f5ea8\") " pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847a98dnt"
Sep 10 17:42:09.142733 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:42:09.142718    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bh4zw\" (UniqueName: \"kubernetes.io/projected/b7d115cc-a234-426d-8809-0dcd578f5ea8-kube-api-access-bh4zw\") pod \"6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847a98dnt\" (UID: \"b7d115cc-a234-426d-8809-0dcd578f5ea8\") " pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847a98dnt"
Sep 10 17:42:09.171373 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:42:09.171345    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cchkct"]
Sep 10 17:42:09.173892 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:42:09.173873    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cchkct"
Sep 10 17:42:09.184123 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:42:09.184098    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cchkct"]
Sep 10 17:42:09.243750 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:42:09.243722    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/614c026a-3bd8-46f1-be37-809eb47ddc5d-bundle\") pod \"e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cchkct\" (UID: \"614c026a-3bd8-46f1-be37-809eb47ddc5d\") " pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cchkct"
Sep 10 17:42:09.243885 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:42:09.243769    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bh4zw\" (UniqueName: \"kubernetes.io/projected/b7d115cc-a234-426d-8809-0dcd578f5ea8-kube-api-access-bh4zw\") pod \"6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847a98dnt\" (UID: \"b7d115cc-a234-426d-8809-0dcd578f5ea8\") " pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847a98dnt"
Sep 10 17:42:09.243885 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:42:09.243819    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mkqwb\" (UniqueName: \"kubernetes.io/projected/614c026a-3bd8-46f1-be37-809eb47ddc5d-kube-api-access-mkqwb\") pod \"e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cchkct\" (UID: \"614c026a-3bd8-46f1-be37-809eb47ddc5d\") " pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cchkct"
Sep 10 17:42:09.244000 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:42:09.243889    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/b7d115cc-a234-426d-8809-0dcd578f5ea8-bundle\") pod \"6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847a98dnt\" (UID: \"b7d115cc-a234-426d-8809-0dcd578f5ea8\") " pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847a98dnt"
Sep 10 17:42:09.244000 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:42:09.243922    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/b7d115cc-a234-426d-8809-0dcd578f5ea8-util\") pod \"6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847a98dnt\" (UID: \"b7d115cc-a234-426d-8809-0dcd578f5ea8\") " pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847a98dnt"
Sep 10 17:42:09.244000 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:42:09.243950    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/614c026a-3bd8-46f1-be37-809eb47ddc5d-util\") pod \"e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cchkct\" (UID: \"614c026a-3bd8-46f1-be37-809eb47ddc5d\") " pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cchkct"
Sep 10 17:42:09.244346 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:42:09.244322    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/b7d115cc-a234-426d-8809-0dcd578f5ea8-bundle\") pod \"6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847a98dnt\" (UID: \"b7d115cc-a234-426d-8809-0dcd578f5ea8\") " pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847a98dnt"
Sep 10 17:42:09.244440 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:42:09.244359    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/b7d115cc-a234-426d-8809-0dcd578f5ea8-util\") pod \"6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847a98dnt\" (UID: \"b7d115cc-a234-426d-8809-0dcd578f5ea8\") " pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847a98dnt"
Sep 10 17:42:09.254377 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:42:09.254350    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bh4zw\" (UniqueName: \"kubernetes.io/projected/b7d115cc-a234-426d-8809-0dcd578f5ea8-kube-api-access-bh4zw\") pod \"6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847a98dnt\" (UID: \"b7d115cc-a234-426d-8809-0dcd578f5ea8\") " pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847a98dnt"
Sep 10 17:42:09.264750 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:42:09.264727    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7pw94q"]
Sep 10 17:42:09.267340 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:42:09.267326    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7pw94q"
Sep 10 17:42:09.277254 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:42:09.277233    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7pw94q"]
Sep 10 17:42:09.344973 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:42:09.344897    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/614c026a-3bd8-46f1-be37-809eb47ddc5d-bundle\") pod \"e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cchkct\" (UID: \"614c026a-3bd8-46f1-be37-809eb47ddc5d\") " pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cchkct"
Sep 10 17:42:09.344973 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:42:09.344938    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/c8dcfe49-71da-4417-b97c-23d2b271f305-bundle\") pod \"da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7pw94q\" (UID: \"c8dcfe49-71da-4417-b97c-23d2b271f305\") " pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7pw94q"
Sep 10 17:42:09.344973 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:42:09.344960    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/c8dcfe49-71da-4417-b97c-23d2b271f305-util\") pod \"da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7pw94q\" (UID: \"c8dcfe49-71da-4417-b97c-23d2b271f305\") " pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7pw94q"
Sep 10 17:42:09.345240 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:42:09.344985    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5hwx9\" (UniqueName: \"kubernetes.io/projected/c8dcfe49-71da-4417-b97c-23d2b271f305-kube-api-access-5hwx9\") pod \"da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7pw94q\" (UID: \"c8dcfe49-71da-4417-b97c-23d2b271f305\") " pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7pw94q"
Sep 10 17:42:09.345240 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:42:09.345035    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mkqwb\" (UniqueName: \"kubernetes.io/projected/614c026a-3bd8-46f1-be37-809eb47ddc5d-kube-api-access-mkqwb\") pod \"e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cchkct\" (UID: \"614c026a-3bd8-46f1-be37-809eb47ddc5d\") " pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cchkct"
Sep 10 17:42:09.345240 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:42:09.345098    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/614c026a-3bd8-46f1-be37-809eb47ddc5d-util\") pod \"e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cchkct\" (UID: \"614c026a-3bd8-46f1-be37-809eb47ddc5d\") " pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cchkct"
Sep 10 17:42:09.345387 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:42:09.345353    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/614c026a-3bd8-46f1-be37-809eb47ddc5d-bundle\") pod \"e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cchkct\" (UID: \"614c026a-3bd8-46f1-be37-809eb47ddc5d\") " pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cchkct"
Sep 10 17:42:09.345449 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:42:09.345429    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/614c026a-3bd8-46f1-be37-809eb47ddc5d-util\") pod \"e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cchkct\" (UID: \"614c026a-3bd8-46f1-be37-809eb47ddc5d\") " pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cchkct"
Sep 10 17:42:09.357325 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:42:09.357297    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mkqwb\" (UniqueName: \"kubernetes.io/projected/614c026a-3bd8-46f1-be37-809eb47ddc5d-kube-api-access-mkqwb\") pod \"e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cchkct\" (UID: \"614c026a-3bd8-46f1-be37-809eb47ddc5d\") " pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cchkct"
Sep 10 17:42:09.363199 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:42:09.363177    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c64xdcq"]
Sep 10 17:42:09.366200 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:42:09.366184    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c64xdcq"
Sep 10 17:42:09.375895 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:42:09.375873    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c64xdcq"]
Sep 10 17:42:09.393510 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:42:09.393486    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847a98dnt"
Sep 10 17:42:09.447104 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:42:09.445666    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qnhcv\" (UniqueName: \"kubernetes.io/projected/dfba1da7-d2bd-4e2e-904e-d600b53bb8d4-kube-api-access-qnhcv\") pod \"ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c64xdcq\" (UID: \"dfba1da7-d2bd-4e2e-904e-d600b53bb8d4\") " pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c64xdcq"
Sep 10 17:42:09.447104 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:42:09.445753    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/c8dcfe49-71da-4417-b97c-23d2b271f305-bundle\") pod \"da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7pw94q\" (UID: \"c8dcfe49-71da-4417-b97c-23d2b271f305\") " pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7pw94q"
Sep 10 17:42:09.447104 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:42:09.445777    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/c8dcfe49-71da-4417-b97c-23d2b271f305-util\") pod \"da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7pw94q\" (UID: \"c8dcfe49-71da-4417-b97c-23d2b271f305\") " pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7pw94q"
Sep 10 17:42:09.447104 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:42:09.445808    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5hwx9\" (UniqueName: \"kubernetes.io/projected/c8dcfe49-71da-4417-b97c-23d2b271f305-kube-api-access-5hwx9\") pod \"da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7pw94q\" (UID: \"c8dcfe49-71da-4417-b97c-23d2b271f305\") " pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7pw94q"
Sep 10 17:42:09.447104 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:42:09.445851    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/dfba1da7-d2bd-4e2e-904e-d600b53bb8d4-bundle\") pod \"ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c64xdcq\" (UID: \"dfba1da7-d2bd-4e2e-904e-d600b53bb8d4\") " pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c64xdcq"
Sep 10 17:42:09.447104 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:42:09.445878    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/dfba1da7-d2bd-4e2e-904e-d600b53bb8d4-util\") pod \"ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c64xdcq\" (UID: \"dfba1da7-d2bd-4e2e-904e-d600b53bb8d4\") " pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c64xdcq"
Sep 10 17:42:09.447104 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:42:09.446315    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/c8dcfe49-71da-4417-b97c-23d2b271f305-bundle\") pod \"da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7pw94q\" (UID: \"c8dcfe49-71da-4417-b97c-23d2b271f305\") " pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7pw94q"
Sep 10 17:42:09.447104 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:42:09.446542    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/c8dcfe49-71da-4417-b97c-23d2b271f305-util\") pod \"da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7pw94q\" (UID: \"c8dcfe49-71da-4417-b97c-23d2b271f305\") " pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7pw94q"
Sep 10 17:42:09.457360 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:42:09.457294    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5hwx9\" (UniqueName: \"kubernetes.io/projected/c8dcfe49-71da-4417-b97c-23d2b271f305-kube-api-access-5hwx9\") pod \"da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7pw94q\" (UID: \"c8dcfe49-71da-4417-b97c-23d2b271f305\") " pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7pw94q"
Sep 10 17:42:09.482717 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:42:09.482690    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cchkct"
Sep 10 17:42:09.547278 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:42:09.547230    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qnhcv\" (UniqueName: \"kubernetes.io/projected/dfba1da7-d2bd-4e2e-904e-d600b53bb8d4-kube-api-access-qnhcv\") pod \"ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c64xdcq\" (UID: \"dfba1da7-d2bd-4e2e-904e-d600b53bb8d4\") " pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c64xdcq"
Sep 10 17:42:09.547406 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:42:09.547333    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/dfba1da7-d2bd-4e2e-904e-d600b53bb8d4-bundle\") pod \"ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c64xdcq\" (UID: \"dfba1da7-d2bd-4e2e-904e-d600b53bb8d4\") " pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c64xdcq"
Sep 10 17:42:09.547406 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:42:09.547349    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/dfba1da7-d2bd-4e2e-904e-d600b53bb8d4-util\") pod \"ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c64xdcq\" (UID: \"dfba1da7-d2bd-4e2e-904e-d600b53bb8d4\") " pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c64xdcq"
Sep 10 17:42:09.547723 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:42:09.547703    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/dfba1da7-d2bd-4e2e-904e-d600b53bb8d4-util\") pod \"ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c64xdcq\" (UID: \"dfba1da7-d2bd-4e2e-904e-d600b53bb8d4\") " pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c64xdcq"
Sep 10 17:42:09.547808 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:42:09.547737    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/dfba1da7-d2bd-4e2e-904e-d600b53bb8d4-bundle\") pod \"ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c64xdcq\" (UID: \"dfba1da7-d2bd-4e2e-904e-d600b53bb8d4\") " pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c64xdcq"
Sep 10 17:42:09.555503 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:42:09.555478    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qnhcv\" (UniqueName: \"kubernetes.io/projected/dfba1da7-d2bd-4e2e-904e-d600b53bb8d4-kube-api-access-qnhcv\") pod \"ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c64xdcq\" (UID: \"dfba1da7-d2bd-4e2e-904e-d600b53bb8d4\") " pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c64xdcq"
Sep 10 17:42:09.577404 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:42:09.577382    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7pw94q"
Sep 10 17:42:09.673151 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:42:09.673128    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847a98dnt"]
Sep 10 17:42:09.674943 ip-10-0-141-180 kubenswrapper[2581]: W0910 17:42:09.674920    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb7d115cc_a234_426d_8809_0dcd578f5ea8.slice/crio-82892fa442dff635d0f6677fa368fe67ad4c8f8ab88763ac9a07b4123a26fa9c WatchSource:0}: Error finding container 82892fa442dff635d0f6677fa368fe67ad4c8f8ab88763ac9a07b4123a26fa9c: Status 404 returned error can't find the container with id 82892fa442dff635d0f6677fa368fe67ad4c8f8ab88763ac9a07b4123a26fa9c
Sep 10 17:42:09.675681 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:42:09.675663    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c64xdcq"
Sep 10 17:42:09.758551 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:42:09.758519    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cchkct"]
Sep 10 17:42:09.759183 ip-10-0-141-180 kubenswrapper[2581]: W0910 17:42:09.759159    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod614c026a_3bd8_46f1_be37_809eb47ddc5d.slice/crio-522689c06c87198abc312ac8bf75994c8f8c3f40348ca5bb8c50e7a09dadcb7a WatchSource:0}: Error finding container 522689c06c87198abc312ac8bf75994c8f8c3f40348ca5bb8c50e7a09dadcb7a: Status 404 returned error can't find the container with id 522689c06c87198abc312ac8bf75994c8f8c3f40348ca5bb8c50e7a09dadcb7a
Sep 10 17:42:09.851694 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:42:09.851630    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7pw94q"]
Sep 10 17:42:09.856817 ip-10-0-141-180 kubenswrapper[2581]: W0910 17:42:09.856791    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc8dcfe49_71da_4417_b97c_23d2b271f305.slice/crio-4fa4394f9010e9496e0962a505c95a50f7ec25aefc0ae4c87ad6d52ec0ac9832 WatchSource:0}: Error finding container 4fa4394f9010e9496e0962a505c95a50f7ec25aefc0ae4c87ad6d52ec0ac9832: Status 404 returned error can't find the container with id 4fa4394f9010e9496e0962a505c95a50f7ec25aefc0ae4c87ad6d52ec0ac9832
Sep 10 17:42:09.952152 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:42:09.952120    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c64xdcq"]
Sep 10 17:42:10.027569 ip-10-0-141-180 kubenswrapper[2581]: W0910 17:42:10.027523    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddfba1da7_d2bd_4e2e_904e_d600b53bb8d4.slice/crio-66cbc8a687e42f8e728496af8b028d7ac6ba73907c0fed52295ceb7a3a18d1bd WatchSource:0}: Error finding container 66cbc8a687e42f8e728496af8b028d7ac6ba73907c0fed52295ceb7a3a18d1bd: Status 404 returned error can't find the container with id 66cbc8a687e42f8e728496af8b028d7ac6ba73907c0fed52295ceb7a3a18d1bd
Sep 10 17:42:10.454830 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:42:10.454790    2581 generic.go:356] "Generic (PLEG): container finished" podID="614c026a-3bd8-46f1-be37-809eb47ddc5d" containerID="eab6b936c84d385045c78cf43f472b3ad1a127a7447c5c0cd77fe57a4e79d77d" exitCode=0
Sep 10 17:42:10.455252 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:42:10.454866    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cchkct" event={"ID":"614c026a-3bd8-46f1-be37-809eb47ddc5d","Type":"ContainerDied","Data":"eab6b936c84d385045c78cf43f472b3ad1a127a7447c5c0cd77fe57a4e79d77d"}
Sep 10 17:42:10.455252 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:42:10.454921    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cchkct" event={"ID":"614c026a-3bd8-46f1-be37-809eb47ddc5d","Type":"ContainerStarted","Data":"522689c06c87198abc312ac8bf75994c8f8c3f40348ca5bb8c50e7a09dadcb7a"}
Sep 10 17:42:10.456034 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:42:10.456013    2581 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 10 17:42:10.456769 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:42:10.456747    2581 generic.go:356] "Generic (PLEG): container finished" podID="dfba1da7-d2bd-4e2e-904e-d600b53bb8d4" containerID="4e6224c86a81fcb42ec0015643c9a6a1d7cd17723dc566dd17c0fb93ada67ee3" exitCode=0
Sep 10 17:42:10.456869 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:42:10.456829    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c64xdcq" event={"ID":"dfba1da7-d2bd-4e2e-904e-d600b53bb8d4","Type":"ContainerDied","Data":"4e6224c86a81fcb42ec0015643c9a6a1d7cd17723dc566dd17c0fb93ada67ee3"}
Sep 10 17:42:10.456929 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:42:10.456881    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c64xdcq" event={"ID":"dfba1da7-d2bd-4e2e-904e-d600b53bb8d4","Type":"ContainerStarted","Data":"66cbc8a687e42f8e728496af8b028d7ac6ba73907c0fed52295ceb7a3a18d1bd"}
Sep 10 17:42:10.458412 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:42:10.458387    2581 generic.go:356] "Generic (PLEG): container finished" podID="c8dcfe49-71da-4417-b97c-23d2b271f305" containerID="279a31ca74a45eb2fb7a4552e101294d034d6c75785cb2e8abebc9b67a76961b" exitCode=0
Sep 10 17:42:10.458522 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:42:10.458463    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7pw94q" event={"ID":"c8dcfe49-71da-4417-b97c-23d2b271f305","Type":"ContainerDied","Data":"279a31ca74a45eb2fb7a4552e101294d034d6c75785cb2e8abebc9b67a76961b"}
Sep 10 17:42:10.458522 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:42:10.458504    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7pw94q" event={"ID":"c8dcfe49-71da-4417-b97c-23d2b271f305","Type":"ContainerStarted","Data":"4fa4394f9010e9496e0962a505c95a50f7ec25aefc0ae4c87ad6d52ec0ac9832"}
Sep 10 17:42:10.460239 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:42:10.460218    2581 generic.go:356] "Generic (PLEG): container finished" podID="b7d115cc-a234-426d-8809-0dcd578f5ea8" containerID="ebd875b83e5f677cb56ca550ba67e25cf6fc0f14ba550f530339942c5e938761" exitCode=0
Sep 10 17:42:10.460326 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:42:10.460300    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847a98dnt" event={"ID":"b7d115cc-a234-426d-8809-0dcd578f5ea8","Type":"ContainerDied","Data":"ebd875b83e5f677cb56ca550ba67e25cf6fc0f14ba550f530339942c5e938761"}
Sep 10 17:42:10.460384 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:42:10.460335    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847a98dnt" event={"ID":"b7d115cc-a234-426d-8809-0dcd578f5ea8","Type":"ContainerStarted","Data":"82892fa442dff635d0f6677fa368fe67ad4c8f8ab88763ac9a07b4123a26fa9c"}
Sep 10 17:42:11.468430 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:42:11.468403    2581 generic.go:356] "Generic (PLEG): container finished" podID="614c026a-3bd8-46f1-be37-809eb47ddc5d" containerID="2fb1977cb68b3fb6aaaa899e0e6d77104752eaa6dfb7622ca98396c621bdb9a5" exitCode=0
Sep 10 17:42:11.468752 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:42:11.468477    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cchkct" event={"ID":"614c026a-3bd8-46f1-be37-809eb47ddc5d","Type":"ContainerDied","Data":"2fb1977cb68b3fb6aaaa899e0e6d77104752eaa6dfb7622ca98396c621bdb9a5"}
Sep 10 17:42:11.470492 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:42:11.470472    2581 generic.go:356] "Generic (PLEG): container finished" podID="dfba1da7-d2bd-4e2e-904e-d600b53bb8d4" containerID="0ad3363558babfecb8feca9d4e441978f009edb09a280d678b958258158370d6" exitCode=0
Sep 10 17:42:11.470580 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:42:11.470546    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c64xdcq" event={"ID":"dfba1da7-d2bd-4e2e-904e-d600b53bb8d4","Type":"ContainerDied","Data":"0ad3363558babfecb8feca9d4e441978f009edb09a280d678b958258158370d6"}
Sep 10 17:42:11.472492 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:42:11.472469    2581 generic.go:356] "Generic (PLEG): container finished" podID="c8dcfe49-71da-4417-b97c-23d2b271f305" containerID="24d0250a04a888b40665aefd08256768fcbbc5279c36e0bcd28a7ae476c53596" exitCode=0
Sep 10 17:42:11.472595 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:42:11.472542    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7pw94q" event={"ID":"c8dcfe49-71da-4417-b97c-23d2b271f305","Type":"ContainerDied","Data":"24d0250a04a888b40665aefd08256768fcbbc5279c36e0bcd28a7ae476c53596"}
Sep 10 17:42:11.570539 ip-10-0-141-180 kubenswrapper[2581]: E0910 17:42:11.570432    2581 cadvisor_stats_provider.go:567] "Partial failure issuing cadvisor.ContainerInfoV2" err="partial failures: [\"/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb7d115cc_a234_426d_8809_0dcd578f5ea8.slice/crio-a2cfa4866f5c9dcff33354de05dc5769aa607eeba55abc025a22fa44aa7e9292.scope\": RecentStats: unable to find data in memory cache], [\"/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb7d115cc_a234_426d_8809_0dcd578f5ea8.slice/crio-conmon-a2cfa4866f5c9dcff33354de05dc5769aa607eeba55abc025a22fa44aa7e9292.scope\": RecentStats: unable to find data in memory cache]"
Sep 10 17:42:12.480156 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:42:12.480118    2581 generic.go:356] "Generic (PLEG): container finished" podID="dfba1da7-d2bd-4e2e-904e-d600b53bb8d4" containerID="f0fb8b60f9f4939460ab984f29604fa7dbbaa0460f2e20ba219ae519feabeb8d" exitCode=0
Sep 10 17:42:12.480534 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:42:12.480204    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c64xdcq" event={"ID":"dfba1da7-d2bd-4e2e-904e-d600b53bb8d4","Type":"ContainerDied","Data":"f0fb8b60f9f4939460ab984f29604fa7dbbaa0460f2e20ba219ae519feabeb8d"}
Sep 10 17:42:12.482158 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:42:12.482134    2581 generic.go:356] "Generic (PLEG): container finished" podID="c8dcfe49-71da-4417-b97c-23d2b271f305" containerID="864b65ad2361ac0a4c613045dcf196f4814022ae94a8dee1e97a87f1f0a39ab5" exitCode=0
Sep 10 17:42:12.482256 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:42:12.482210    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7pw94q" event={"ID":"c8dcfe49-71da-4417-b97c-23d2b271f305","Type":"ContainerDied","Data":"864b65ad2361ac0a4c613045dcf196f4814022ae94a8dee1e97a87f1f0a39ab5"}
Sep 10 17:42:12.483772 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:42:12.483754    2581 generic.go:356] "Generic (PLEG): container finished" podID="b7d115cc-a234-426d-8809-0dcd578f5ea8" containerID="a2cfa4866f5c9dcff33354de05dc5769aa607eeba55abc025a22fa44aa7e9292" exitCode=0
Sep 10 17:42:12.483879 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:42:12.483862    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847a98dnt" event={"ID":"b7d115cc-a234-426d-8809-0dcd578f5ea8","Type":"ContainerDied","Data":"a2cfa4866f5c9dcff33354de05dc5769aa607eeba55abc025a22fa44aa7e9292"}
Sep 10 17:42:12.486184 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:42:12.486168    2581 generic.go:356] "Generic (PLEG): container finished" podID="614c026a-3bd8-46f1-be37-809eb47ddc5d" containerID="60172b7aeac070f0accb7736ee937fa578684e02d8cc15a4e53b1636978f5516" exitCode=0
Sep 10 17:42:12.486264 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:42:12.486203    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cchkct" event={"ID":"614c026a-3bd8-46f1-be37-809eb47ddc5d","Type":"ContainerDied","Data":"60172b7aeac070f0accb7736ee937fa578684e02d8cc15a4e53b1636978f5516"}
Sep 10 17:42:13.492945 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:42:13.492911    2581 generic.go:356] "Generic (PLEG): container finished" podID="b7d115cc-a234-426d-8809-0dcd578f5ea8" containerID="486f459c5bec20e34e460e1e56872547edf9d4e4270e18f6fc3726b178bc351e" exitCode=0
Sep 10 17:42:13.493367 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:42:13.493004    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847a98dnt" event={"ID":"b7d115cc-a234-426d-8809-0dcd578f5ea8","Type":"ContainerDied","Data":"486f459c5bec20e34e460e1e56872547edf9d4e4270e18f6fc3726b178bc351e"}
Sep 10 17:42:13.632836 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:42:13.632813    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c64xdcq"
Sep 10 17:42:13.668480 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:42:13.668454    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7pw94q"
Sep 10 17:42:13.684777 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:42:13.684754    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cchkct"
Sep 10 17:42:13.688884 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:42:13.688862    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-qnhcv\" (UniqueName: \"kubernetes.io/projected/dfba1da7-d2bd-4e2e-904e-d600b53bb8d4-kube-api-access-qnhcv\") pod \"dfba1da7-d2bd-4e2e-904e-d600b53bb8d4\" (UID: \"dfba1da7-d2bd-4e2e-904e-d600b53bb8d4\") "
Sep 10 17:42:13.688984 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:42:13.688954    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/dfba1da7-d2bd-4e2e-904e-d600b53bb8d4-util\") pod \"dfba1da7-d2bd-4e2e-904e-d600b53bb8d4\" (UID: \"dfba1da7-d2bd-4e2e-904e-d600b53bb8d4\") "
Sep 10 17:42:13.689038 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:42:13.689000    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/dfba1da7-d2bd-4e2e-904e-d600b53bb8d4-bundle\") pod \"dfba1da7-d2bd-4e2e-904e-d600b53bb8d4\" (UID: \"dfba1da7-d2bd-4e2e-904e-d600b53bb8d4\") "
Sep 10 17:42:13.689469 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:42:13.689439    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/dfba1da7-d2bd-4e2e-904e-d600b53bb8d4-bundle" (OuterVolumeSpecName: "bundle") pod "dfba1da7-d2bd-4e2e-904e-d600b53bb8d4" (UID: "dfba1da7-d2bd-4e2e-904e-d600b53bb8d4"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 17:42:13.690860 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:42:13.690835    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/dfba1da7-d2bd-4e2e-904e-d600b53bb8d4-kube-api-access-qnhcv" (OuterVolumeSpecName: "kube-api-access-qnhcv") pod "dfba1da7-d2bd-4e2e-904e-d600b53bb8d4" (UID: "dfba1da7-d2bd-4e2e-904e-d600b53bb8d4"). InnerVolumeSpecName "kube-api-access-qnhcv". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 17:42:13.694122 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:42:13.694088    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/dfba1da7-d2bd-4e2e-904e-d600b53bb8d4-util" (OuterVolumeSpecName: "util") pod "dfba1da7-d2bd-4e2e-904e-d600b53bb8d4" (UID: "dfba1da7-d2bd-4e2e-904e-d600b53bb8d4"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 17:42:13.789514 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:42:13.789431    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-5hwx9\" (UniqueName: \"kubernetes.io/projected/c8dcfe49-71da-4417-b97c-23d2b271f305-kube-api-access-5hwx9\") pod \"c8dcfe49-71da-4417-b97c-23d2b271f305\" (UID: \"c8dcfe49-71da-4417-b97c-23d2b271f305\") "
Sep 10 17:42:13.789671 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:42:13.789522    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/614c026a-3bd8-46f1-be37-809eb47ddc5d-bundle\") pod \"614c026a-3bd8-46f1-be37-809eb47ddc5d\" (UID: \"614c026a-3bd8-46f1-be37-809eb47ddc5d\") "
Sep 10 17:42:13.789671 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:42:13.789540    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-mkqwb\" (UniqueName: \"kubernetes.io/projected/614c026a-3bd8-46f1-be37-809eb47ddc5d-kube-api-access-mkqwb\") pod \"614c026a-3bd8-46f1-be37-809eb47ddc5d\" (UID: \"614c026a-3bd8-46f1-be37-809eb47ddc5d\") "
Sep 10 17:42:13.789671 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:42:13.789575    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/614c026a-3bd8-46f1-be37-809eb47ddc5d-util\") pod \"614c026a-3bd8-46f1-be37-809eb47ddc5d\" (UID: \"614c026a-3bd8-46f1-be37-809eb47ddc5d\") "
Sep 10 17:42:13.789671 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:42:13.789608    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/c8dcfe49-71da-4417-b97c-23d2b271f305-bundle\") pod \"c8dcfe49-71da-4417-b97c-23d2b271f305\" (UID: \"c8dcfe49-71da-4417-b97c-23d2b271f305\") "
Sep 10 17:42:13.789671 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:42:13.789630    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/c8dcfe49-71da-4417-b97c-23d2b271f305-util\") pod \"c8dcfe49-71da-4417-b97c-23d2b271f305\" (UID: \"c8dcfe49-71da-4417-b97c-23d2b271f305\") "
Sep 10 17:42:13.789935 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:42:13.789866    2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qnhcv\" (UniqueName: \"kubernetes.io/projected/dfba1da7-d2bd-4e2e-904e-d600b53bb8d4-kube-api-access-qnhcv\") on node \"ip-10-0-141-180.ec2.internal\" DevicePath \"\""
Sep 10 17:42:13.789935 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:42:13.789885    2581 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/dfba1da7-d2bd-4e2e-904e-d600b53bb8d4-util\") on node \"ip-10-0-141-180.ec2.internal\" DevicePath \"\""
Sep 10 17:42:13.789935 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:42:13.789899    2581 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/dfba1da7-d2bd-4e2e-904e-d600b53bb8d4-bundle\") on node \"ip-10-0-141-180.ec2.internal\" DevicePath \"\""
Sep 10 17:42:13.790214 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:42:13.790182    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c8dcfe49-71da-4417-b97c-23d2b271f305-bundle" (OuterVolumeSpecName: "bundle") pod "c8dcfe49-71da-4417-b97c-23d2b271f305" (UID: "c8dcfe49-71da-4417-b97c-23d2b271f305"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 17:42:13.790280 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:42:13.790210    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/614c026a-3bd8-46f1-be37-809eb47ddc5d-bundle" (OuterVolumeSpecName: "bundle") pod "614c026a-3bd8-46f1-be37-809eb47ddc5d" (UID: "614c026a-3bd8-46f1-be37-809eb47ddc5d"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 17:42:13.791624 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:42:13.791594    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c8dcfe49-71da-4417-b97c-23d2b271f305-kube-api-access-5hwx9" (OuterVolumeSpecName: "kube-api-access-5hwx9") pod "c8dcfe49-71da-4417-b97c-23d2b271f305" (UID: "c8dcfe49-71da-4417-b97c-23d2b271f305"). InnerVolumeSpecName "kube-api-access-5hwx9". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 17:42:13.791785 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:42:13.791765    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/614c026a-3bd8-46f1-be37-809eb47ddc5d-kube-api-access-mkqwb" (OuterVolumeSpecName: "kube-api-access-mkqwb") pod "614c026a-3bd8-46f1-be37-809eb47ddc5d" (UID: "614c026a-3bd8-46f1-be37-809eb47ddc5d"). InnerVolumeSpecName "kube-api-access-mkqwb". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 17:42:13.795505 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:42:13.795487    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/614c026a-3bd8-46f1-be37-809eb47ddc5d-util" (OuterVolumeSpecName: "util") pod "614c026a-3bd8-46f1-be37-809eb47ddc5d" (UID: "614c026a-3bd8-46f1-be37-809eb47ddc5d"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 17:42:13.795677 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:42:13.795654    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c8dcfe49-71da-4417-b97c-23d2b271f305-util" (OuterVolumeSpecName: "util") pod "c8dcfe49-71da-4417-b97c-23d2b271f305" (UID: "c8dcfe49-71da-4417-b97c-23d2b271f305"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 17:42:13.890906 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:42:13.890879    2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mkqwb\" (UniqueName: \"kubernetes.io/projected/614c026a-3bd8-46f1-be37-809eb47ddc5d-kube-api-access-mkqwb\") on node \"ip-10-0-141-180.ec2.internal\" DevicePath \"\""
Sep 10 17:42:13.890906 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:42:13.890902    2581 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/614c026a-3bd8-46f1-be37-809eb47ddc5d-util\") on node \"ip-10-0-141-180.ec2.internal\" DevicePath \"\""
Sep 10 17:42:13.891040 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:42:13.890913    2581 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/c8dcfe49-71da-4417-b97c-23d2b271f305-bundle\") on node \"ip-10-0-141-180.ec2.internal\" DevicePath \"\""
Sep 10 17:42:13.891040 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:42:13.890921    2581 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/c8dcfe49-71da-4417-b97c-23d2b271f305-util\") on node \"ip-10-0-141-180.ec2.internal\" DevicePath \"\""
Sep 10 17:42:13.891040 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:42:13.890928    2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5hwx9\" (UniqueName: \"kubernetes.io/projected/c8dcfe49-71da-4417-b97c-23d2b271f305-kube-api-access-5hwx9\") on node \"ip-10-0-141-180.ec2.internal\" DevicePath \"\""
Sep 10 17:42:13.891040 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:42:13.890936    2581 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/614c026a-3bd8-46f1-be37-809eb47ddc5d-bundle\") on node \"ip-10-0-141-180.ec2.internal\" DevicePath \"\""
Sep 10 17:42:14.499655 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:42:14.499619    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cchkct"
Sep 10 17:42:14.499655 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:42:14.499633    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cchkct" event={"ID":"614c026a-3bd8-46f1-be37-809eb47ddc5d","Type":"ContainerDied","Data":"522689c06c87198abc312ac8bf75994c8f8c3f40348ca5bb8c50e7a09dadcb7a"}
Sep 10 17:42:14.500104 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:42:14.499695    2581 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="522689c06c87198abc312ac8bf75994c8f8c3f40348ca5bb8c50e7a09dadcb7a"
Sep 10 17:42:14.501679 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:42:14.501662    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c64xdcq"
Sep 10 17:42:14.501784 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:42:14.501674    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c64xdcq" event={"ID":"dfba1da7-d2bd-4e2e-904e-d600b53bb8d4","Type":"ContainerDied","Data":"66cbc8a687e42f8e728496af8b028d7ac6ba73907c0fed52295ceb7a3a18d1bd"}
Sep 10 17:42:14.501784 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:42:14.501708    2581 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="66cbc8a687e42f8e728496af8b028d7ac6ba73907c0fed52295ceb7a3a18d1bd"
Sep 10 17:42:14.503682 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:42:14.503665    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7pw94q"
Sep 10 17:42:14.503770 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:42:14.503700    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7pw94q" event={"ID":"c8dcfe49-71da-4417-b97c-23d2b271f305","Type":"ContainerDied","Data":"4fa4394f9010e9496e0962a505c95a50f7ec25aefc0ae4c87ad6d52ec0ac9832"}
Sep 10 17:42:14.503770 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:42:14.503740    2581 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="4fa4394f9010e9496e0962a505c95a50f7ec25aefc0ae4c87ad6d52ec0ac9832"
Sep 10 17:42:14.629719 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:42:14.629699    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847a98dnt"
Sep 10 17:42:14.699266 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:42:14.699241    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-bh4zw\" (UniqueName: \"kubernetes.io/projected/b7d115cc-a234-426d-8809-0dcd578f5ea8-kube-api-access-bh4zw\") pod \"b7d115cc-a234-426d-8809-0dcd578f5ea8\" (UID: \"b7d115cc-a234-426d-8809-0dcd578f5ea8\") "
Sep 10 17:42:14.699399 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:42:14.699283    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/b7d115cc-a234-426d-8809-0dcd578f5ea8-bundle\") pod \"b7d115cc-a234-426d-8809-0dcd578f5ea8\" (UID: \"b7d115cc-a234-426d-8809-0dcd578f5ea8\") "
Sep 10 17:42:14.699399 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:42:14.699344    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/b7d115cc-a234-426d-8809-0dcd578f5ea8-util\") pod \"b7d115cc-a234-426d-8809-0dcd578f5ea8\" (UID: \"b7d115cc-a234-426d-8809-0dcd578f5ea8\") "
Sep 10 17:42:14.699971 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:42:14.699944    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b7d115cc-a234-426d-8809-0dcd578f5ea8-bundle" (OuterVolumeSpecName: "bundle") pod "b7d115cc-a234-426d-8809-0dcd578f5ea8" (UID: "b7d115cc-a234-426d-8809-0dcd578f5ea8"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 17:42:14.701409 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:42:14.701379    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b7d115cc-a234-426d-8809-0dcd578f5ea8-kube-api-access-bh4zw" (OuterVolumeSpecName: "kube-api-access-bh4zw") pod "b7d115cc-a234-426d-8809-0dcd578f5ea8" (UID: "b7d115cc-a234-426d-8809-0dcd578f5ea8"). InnerVolumeSpecName "kube-api-access-bh4zw". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 17:42:14.704482 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:42:14.704449    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b7d115cc-a234-426d-8809-0dcd578f5ea8-util" (OuterVolumeSpecName: "util") pod "b7d115cc-a234-426d-8809-0dcd578f5ea8" (UID: "b7d115cc-a234-426d-8809-0dcd578f5ea8"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 17:42:14.800385 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:42:14.800321    2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-bh4zw\" (UniqueName: \"kubernetes.io/projected/b7d115cc-a234-426d-8809-0dcd578f5ea8-kube-api-access-bh4zw\") on node \"ip-10-0-141-180.ec2.internal\" DevicePath \"\""
Sep 10 17:42:14.800385 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:42:14.800345    2581 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/b7d115cc-a234-426d-8809-0dcd578f5ea8-bundle\") on node \"ip-10-0-141-180.ec2.internal\" DevicePath \"\""
Sep 10 17:42:14.800385 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:42:14.800354    2581 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/b7d115cc-a234-426d-8809-0dcd578f5ea8-util\") on node \"ip-10-0-141-180.ec2.internal\" DevicePath \"\""
Sep 10 17:42:15.510557 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:42:15.510522    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847a98dnt" event={"ID":"b7d115cc-a234-426d-8809-0dcd578f5ea8","Type":"ContainerDied","Data":"82892fa442dff635d0f6677fa368fe67ad4c8f8ab88763ac9a07b4123a26fa9c"}
Sep 10 17:42:15.510908 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:42:15.510571    2581 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="82892fa442dff635d0f6677fa368fe67ad4c8f8ab88763ac9a07b4123a26fa9c"
Sep 10 17:42:15.510908 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:42:15.510538    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847a98dnt"
Sep 10 17:42:24.150761 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:42:24.150719    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/dns-operator-controller-manager-7b84cd7c9c-dsd84"]
Sep 10 17:42:24.151172 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:42:24.151070    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c8dcfe49-71da-4417-b97c-23d2b271f305" containerName="pull"
Sep 10 17:42:24.151172 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:42:24.151081    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="c8dcfe49-71da-4417-b97c-23d2b271f305" containerName="pull"
Sep 10 17:42:24.151172 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:42:24.151090    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="614c026a-3bd8-46f1-be37-809eb47ddc5d" containerName="pull"
Sep 10 17:42:24.151172 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:42:24.151096    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="614c026a-3bd8-46f1-be37-809eb47ddc5d" containerName="pull"
Sep 10 17:42:24.151172 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:42:24.151103    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="614c026a-3bd8-46f1-be37-809eb47ddc5d" containerName="extract"
Sep 10 17:42:24.151172 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:42:24.151108    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="614c026a-3bd8-46f1-be37-809eb47ddc5d" containerName="extract"
Sep 10 17:42:24.151172 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:42:24.151117    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="dfba1da7-d2bd-4e2e-904e-d600b53bb8d4" containerName="util"
Sep 10 17:42:24.151172 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:42:24.151122    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="dfba1da7-d2bd-4e2e-904e-d600b53bb8d4" containerName="util"
Sep 10 17:42:24.151172 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:42:24.151127    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c8dcfe49-71da-4417-b97c-23d2b271f305" containerName="extract"
Sep 10 17:42:24.151172 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:42:24.151132    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="c8dcfe49-71da-4417-b97c-23d2b271f305" containerName="extract"
Sep 10 17:42:24.151172 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:42:24.151140    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b7d115cc-a234-426d-8809-0dcd578f5ea8" containerName="pull"
Sep 10 17:42:24.151172 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:42:24.151144    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="b7d115cc-a234-426d-8809-0dcd578f5ea8" containerName="pull"
Sep 10 17:42:24.151172 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:42:24.151152    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b7d115cc-a234-426d-8809-0dcd578f5ea8" containerName="util"
Sep 10 17:42:24.151172 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:42:24.151157    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="b7d115cc-a234-426d-8809-0dcd578f5ea8" containerName="util"
Sep 10 17:42:24.151172 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:42:24.151162    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="dfba1da7-d2bd-4e2e-904e-d600b53bb8d4" containerName="extract"
Sep 10 17:42:24.151172 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:42:24.151167    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="dfba1da7-d2bd-4e2e-904e-d600b53bb8d4" containerName="extract"
Sep 10 17:42:24.151172 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:42:24.151175    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="614c026a-3bd8-46f1-be37-809eb47ddc5d" containerName="util"
Sep 10 17:42:24.151172 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:42:24.151180    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="614c026a-3bd8-46f1-be37-809eb47ddc5d" containerName="util"
Sep 10 17:42:24.151734 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:42:24.151190    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="dfba1da7-d2bd-4e2e-904e-d600b53bb8d4" containerName="pull"
Sep 10 17:42:24.151734 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:42:24.151195    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="dfba1da7-d2bd-4e2e-904e-d600b53bb8d4" containerName="pull"
Sep 10 17:42:24.151734 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:42:24.151201    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c8dcfe49-71da-4417-b97c-23d2b271f305" containerName="util"
Sep 10 17:42:24.151734 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:42:24.151205    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="c8dcfe49-71da-4417-b97c-23d2b271f305" containerName="util"
Sep 10 17:42:24.151734 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:42:24.151213    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b7d115cc-a234-426d-8809-0dcd578f5ea8" containerName="extract"
Sep 10 17:42:24.151734 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:42:24.151218    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="b7d115cc-a234-426d-8809-0dcd578f5ea8" containerName="extract"
Sep 10 17:42:24.151734 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:42:24.151270    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="614c026a-3bd8-46f1-be37-809eb47ddc5d" containerName="extract"
Sep 10 17:42:24.151734 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:42:24.151276    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="c8dcfe49-71da-4417-b97c-23d2b271f305" containerName="extract"
Sep 10 17:42:24.151734 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:42:24.151283    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="dfba1da7-d2bd-4e2e-904e-d600b53bb8d4" containerName="extract"
Sep 10 17:42:24.151734 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:42:24.151292    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="b7d115cc-a234-426d-8809-0dcd578f5ea8" containerName="extract"
Sep 10 17:42:24.153911 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:42:24.153893    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/dns-operator-controller-manager-7b84cd7c9c-dsd84"
Sep 10 17:42:24.156817 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:42:24.156801    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"dns-operator-controller-env\""
Sep 10 17:42:24.157051 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:42:24.157035    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\""
Sep 10 17:42:24.157096 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:42:24.157047    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"dns-operator-controller-manager-dockercfg-k8wmb\""
Sep 10 17:42:24.157860 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:42:24.157845    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\""
Sep 10 17:42:24.169219 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:42:24.169194    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/dns-operator-controller-manager-7b84cd7c9c-dsd84"]
Sep 10 17:42:24.281008 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:42:24.280973    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-szp5s\" (UniqueName: \"kubernetes.io/projected/aa8daf81-2eee-4880-b323-fcbfa7420927-kube-api-access-szp5s\") pod \"dns-operator-controller-manager-7b84cd7c9c-dsd84\" (UID: \"aa8daf81-2eee-4880-b323-fcbfa7420927\") " pod="kuadrant-system/dns-operator-controller-manager-7b84cd7c9c-dsd84"
Sep 10 17:42:24.382327 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:42:24.382284    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-szp5s\" (UniqueName: \"kubernetes.io/projected/aa8daf81-2eee-4880-b323-fcbfa7420927-kube-api-access-szp5s\") pod \"dns-operator-controller-manager-7b84cd7c9c-dsd84\" (UID: \"aa8daf81-2eee-4880-b323-fcbfa7420927\") " pod="kuadrant-system/dns-operator-controller-manager-7b84cd7c9c-dsd84"
Sep 10 17:42:24.393010 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:42:24.392986    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-szp5s\" (UniqueName: \"kubernetes.io/projected/aa8daf81-2eee-4880-b323-fcbfa7420927-kube-api-access-szp5s\") pod \"dns-operator-controller-manager-7b84cd7c9c-dsd84\" (UID: \"aa8daf81-2eee-4880-b323-fcbfa7420927\") " pod="kuadrant-system/dns-operator-controller-manager-7b84cd7c9c-dsd84"
Sep 10 17:42:24.463747 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:42:24.463711    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/dns-operator-controller-manager-7b84cd7c9c-dsd84"
Sep 10 17:42:24.739157 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:42:24.739126    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/dns-operator-controller-manager-7b84cd7c9c-dsd84"]
Sep 10 17:42:24.741146 ip-10-0-141-180 kubenswrapper[2581]: W0910 17:42:24.741122    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podaa8daf81_2eee_4880_b323_fcbfa7420927.slice/crio-baa87349c58cf3fccd5ef8be898bf23c5617650b60603511c31f30a690a74b9a WatchSource:0}: Error finding container baa87349c58cf3fccd5ef8be898bf23c5617650b60603511c31f30a690a74b9a: Status 404 returned error can't find the container with id baa87349c58cf3fccd5ef8be898bf23c5617650b60603511c31f30a690a74b9a
Sep 10 17:42:25.565549 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:42:25.565510    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/dns-operator-controller-manager-7b84cd7c9c-dsd84" event={"ID":"aa8daf81-2eee-4880-b323-fcbfa7420927","Type":"ContainerStarted","Data":"baa87349c58cf3fccd5ef8be898bf23c5617650b60603511c31f30a690a74b9a"}
Sep 10 17:42:27.578350 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:42:27.578319    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/dns-operator-controller-manager-7b84cd7c9c-dsd84" event={"ID":"aa8daf81-2eee-4880-b323-fcbfa7420927","Type":"ContainerStarted","Data":"cd1a512a9cde09af9dc0c270ef53fd5674ede69a1bd282b8bde4e7fdf7e32366"}
Sep 10 17:42:27.578736 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:42:27.578465    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/dns-operator-controller-manager-7b84cd7c9c-dsd84"
Sep 10 17:42:29.299560 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:42:29.299502    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/dns-operator-controller-manager-7b84cd7c9c-dsd84" podStartSLOduration=2.710827743 podStartE2EDuration="5.299486025s" totalImagesPullingTime="2.588658282s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 17:42:24 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 17:42:27.607546524 +0000 UTC m=+367.261620427" watchObservedRunningTime="2026-09-10 17:42:29.299486025 +0000 UTC m=+368.953559933"
Sep 10 17:42:29.299974 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:42:29.299631    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-operator-76b478bd78-hx2rv"]
Sep 10 17:42:29.302086 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:42:29.302067    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-operator-76b478bd78-hx2rv"
Sep 10 17:42:29.305107 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:42:29.305089    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-operator-dockercfg-8wjwz\""
Sep 10 17:42:29.314011 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:42:29.313988    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-operator-76b478bd78-hx2rv"]
Sep 10 17:42:29.433355 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:42:29.433321    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kx7vd\" (UniqueName: \"kubernetes.io/projected/bcaa2b00-1542-404e-87da-ce9345a99c3f-kube-api-access-kx7vd\") pod \"authorino-operator-76b478bd78-hx2rv\" (UID: \"bcaa2b00-1542-404e-87da-ce9345a99c3f\") " pod="kuadrant-system/authorino-operator-76b478bd78-hx2rv"
Sep 10 17:42:29.534782 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:42:29.534748    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kx7vd\" (UniqueName: \"kubernetes.io/projected/bcaa2b00-1542-404e-87da-ce9345a99c3f-kube-api-access-kx7vd\") pod \"authorino-operator-76b478bd78-hx2rv\" (UID: \"bcaa2b00-1542-404e-87da-ce9345a99c3f\") " pod="kuadrant-system/authorino-operator-76b478bd78-hx2rv"
Sep 10 17:42:29.557365 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:42:29.557299    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kx7vd\" (UniqueName: \"kubernetes.io/projected/bcaa2b00-1542-404e-87da-ce9345a99c3f-kube-api-access-kx7vd\") pod \"authorino-operator-76b478bd78-hx2rv\" (UID: \"bcaa2b00-1542-404e-87da-ce9345a99c3f\") " pod="kuadrant-system/authorino-operator-76b478bd78-hx2rv"
Sep 10 17:42:29.612579 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:42:29.612539    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-operator-76b478bd78-hx2rv"
Sep 10 17:42:29.889323 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:42:29.889292    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-operator-76b478bd78-hx2rv"]
Sep 10 17:42:29.890891 ip-10-0-141-180 kubenswrapper[2581]: W0910 17:42:29.890856    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbcaa2b00_1542_404e_87da_ce9345a99c3f.slice/crio-9369c21474c45eb03fd15b1b007fabca7c204a7793a6108e047722976369f111 WatchSource:0}: Error finding container 9369c21474c45eb03fd15b1b007fabca7c204a7793a6108e047722976369f111: Status 404 returned error can't find the container with id 9369c21474c45eb03fd15b1b007fabca7c204a7793a6108e047722976369f111
Sep 10 17:42:30.599443 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:42:30.599403    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-operator-76b478bd78-hx2rv" event={"ID":"bcaa2b00-1542-404e-87da-ce9345a99c3f","Type":"ContainerStarted","Data":"9369c21474c45eb03fd15b1b007fabca7c204a7793a6108e047722976369f111"}
Sep 10 17:42:31.605911 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:42:31.605879    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-operator-76b478bd78-hx2rv" event={"ID":"bcaa2b00-1542-404e-87da-ce9345a99c3f","Type":"ContainerStarted","Data":"4b4c95bff30dfc141b8af1d324e82140ca8d212c61cf1c181f9f2f2b6d0b6125"}
Sep 10 17:42:31.606298 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:42:31.605994    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/authorino-operator-76b478bd78-hx2rv"
Sep 10 17:42:31.627394 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:42:31.627351    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-operator-76b478bd78-hx2rv" podStartSLOduration=1.46645942 podStartE2EDuration="2.627336555s" totalImagesPullingTime="1.160877135s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 17:42:29 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 17:42:31.627097417 +0000 UTC m=+371.281171320" watchObservedRunningTime="2026-09-10 17:42:31.627336555 +0000 UTC m=+371.281410463"
Sep 10 17:42:38.588090 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:42:38.588057    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/dns-operator-controller-manager-7b84cd7c9c-dsd84"
Sep 10 17:42:42.614195 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:42:42.614159    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/authorino-operator-76b478bd78-hx2rv"
Sep 10 17:43:11.141048 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:43:11.141012    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-nbnx9"]
Sep 10 17:43:11.145877 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:43:11.145851    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-nbnx9"
Sep 10 17:43:11.149380 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:43:11.149362    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"maas-default-gateway-openshift-default-dockercfg-9np92\""
Sep 10 17:43:11.160591 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:43:11.160557    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-nbnx9"]
Sep 10 17:43:11.295816 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:43:11.295780    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a5425b39-1732-47b5-90b4-774966781b06-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-78bb78ff78-nbnx9\" (UID: \"a5425b39-1732-47b5-90b4-774966781b06\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-nbnx9"
Sep 10 17:43:11.295816 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:43:11.295821    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a5425b39-1732-47b5-90b4-774966781b06-istio-podinfo\") pod \"maas-default-gateway-openshift-default-78bb78ff78-nbnx9\" (UID: \"a5425b39-1732-47b5-90b4-774966781b06\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-nbnx9"
Sep 10 17:43:11.296013 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:43:11.295847    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a5425b39-1732-47b5-90b4-774966781b06-istio-token\") pod \"maas-default-gateway-openshift-default-78bb78ff78-nbnx9\" (UID: \"a5425b39-1732-47b5-90b4-774966781b06\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-nbnx9"
Sep 10 17:43:11.296013 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:43:11.295895    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a5425b39-1732-47b5-90b4-774966781b06-credential-socket\") pod \"maas-default-gateway-openshift-default-78bb78ff78-nbnx9\" (UID: \"a5425b39-1732-47b5-90b4-774966781b06\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-nbnx9"
Sep 10 17:43:11.296013 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:43:11.295912    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a5425b39-1732-47b5-90b4-774966781b06-istio-envoy\") pod \"maas-default-gateway-openshift-default-78bb78ff78-nbnx9\" (UID: \"a5425b39-1732-47b5-90b4-774966781b06\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-nbnx9"
Sep 10 17:43:11.296013 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:43:11.295935    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a5425b39-1732-47b5-90b4-774966781b06-istio-data\") pod \"maas-default-gateway-openshift-default-78bb78ff78-nbnx9\" (UID: \"a5425b39-1732-47b5-90b4-774966781b06\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-nbnx9"
Sep 10 17:43:11.296013 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:43:11.295964    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a5425b39-1732-47b5-90b4-774966781b06-workload-socket\") pod \"maas-default-gateway-openshift-default-78bb78ff78-nbnx9\" (UID: \"a5425b39-1732-47b5-90b4-774966781b06\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-nbnx9"
Sep 10 17:43:11.296013 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:43:11.295984    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sfdst\" (UniqueName: \"kubernetes.io/projected/a5425b39-1732-47b5-90b4-774966781b06-kube-api-access-sfdst\") pod \"maas-default-gateway-openshift-default-78bb78ff78-nbnx9\" (UID: \"a5425b39-1732-47b5-90b4-774966781b06\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-nbnx9"
Sep 10 17:43:11.296013 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:43:11.296013    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a5425b39-1732-47b5-90b4-774966781b06-workload-certs\") pod \"maas-default-gateway-openshift-default-78bb78ff78-nbnx9\" (UID: \"a5425b39-1732-47b5-90b4-774966781b06\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-nbnx9"
Sep 10 17:43:11.397513 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:43:11.397436    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a5425b39-1732-47b5-90b4-774966781b06-credential-socket\") pod \"maas-default-gateway-openshift-default-78bb78ff78-nbnx9\" (UID: \"a5425b39-1732-47b5-90b4-774966781b06\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-nbnx9"
Sep 10 17:43:11.397513 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:43:11.397471    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a5425b39-1732-47b5-90b4-774966781b06-istio-envoy\") pod \"maas-default-gateway-openshift-default-78bb78ff78-nbnx9\" (UID: \"a5425b39-1732-47b5-90b4-774966781b06\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-nbnx9"
Sep 10 17:43:11.397513 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:43:11.397486    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a5425b39-1732-47b5-90b4-774966781b06-istio-data\") pod \"maas-default-gateway-openshift-default-78bb78ff78-nbnx9\" (UID: \"a5425b39-1732-47b5-90b4-774966781b06\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-nbnx9"
Sep 10 17:43:11.397513 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:43:11.397505    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a5425b39-1732-47b5-90b4-774966781b06-workload-socket\") pod \"maas-default-gateway-openshift-default-78bb78ff78-nbnx9\" (UID: \"a5425b39-1732-47b5-90b4-774966781b06\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-nbnx9"
Sep 10 17:43:11.397876 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:43:11.397521    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sfdst\" (UniqueName: \"kubernetes.io/projected/a5425b39-1732-47b5-90b4-774966781b06-kube-api-access-sfdst\") pod \"maas-default-gateway-openshift-default-78bb78ff78-nbnx9\" (UID: \"a5425b39-1732-47b5-90b4-774966781b06\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-nbnx9"
Sep 10 17:43:11.397876 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:43:11.397607    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a5425b39-1732-47b5-90b4-774966781b06-workload-certs\") pod \"maas-default-gateway-openshift-default-78bb78ff78-nbnx9\" (UID: \"a5425b39-1732-47b5-90b4-774966781b06\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-nbnx9"
Sep 10 17:43:11.397876 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:43:11.397717    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a5425b39-1732-47b5-90b4-774966781b06-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-78bb78ff78-nbnx9\" (UID: \"a5425b39-1732-47b5-90b4-774966781b06\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-nbnx9"
Sep 10 17:43:11.397876 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:43:11.397748    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a5425b39-1732-47b5-90b4-774966781b06-istio-podinfo\") pod \"maas-default-gateway-openshift-default-78bb78ff78-nbnx9\" (UID: \"a5425b39-1732-47b5-90b4-774966781b06\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-nbnx9"
Sep 10 17:43:11.397876 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:43:11.397796    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a5425b39-1732-47b5-90b4-774966781b06-istio-token\") pod \"maas-default-gateway-openshift-default-78bb78ff78-nbnx9\" (UID: \"a5425b39-1732-47b5-90b4-774966781b06\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-nbnx9"
Sep 10 17:43:11.398118 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:43:11.398009    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a5425b39-1732-47b5-90b4-774966781b06-istio-data\") pod \"maas-default-gateway-openshift-default-78bb78ff78-nbnx9\" (UID: \"a5425b39-1732-47b5-90b4-774966781b06\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-nbnx9"
Sep 10 17:43:11.398118 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:43:11.398043    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a5425b39-1732-47b5-90b4-774966781b06-credential-socket\") pod \"maas-default-gateway-openshift-default-78bb78ff78-nbnx9\" (UID: \"a5425b39-1732-47b5-90b4-774966781b06\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-nbnx9"
Sep 10 17:43:11.398194 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:43:11.398166    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a5425b39-1732-47b5-90b4-774966781b06-workload-certs\") pod \"maas-default-gateway-openshift-default-78bb78ff78-nbnx9\" (UID: \"a5425b39-1732-47b5-90b4-774966781b06\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-nbnx9"
Sep 10 17:43:11.398300 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:43:11.398276    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a5425b39-1732-47b5-90b4-774966781b06-workload-socket\") pod \"maas-default-gateway-openshift-default-78bb78ff78-nbnx9\" (UID: \"a5425b39-1732-47b5-90b4-774966781b06\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-nbnx9"
Sep 10 17:43:11.398530 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:43:11.398512    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a5425b39-1732-47b5-90b4-774966781b06-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-78bb78ff78-nbnx9\" (UID: \"a5425b39-1732-47b5-90b4-774966781b06\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-nbnx9"
Sep 10 17:43:11.399826 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:43:11.399807    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a5425b39-1732-47b5-90b4-774966781b06-istio-envoy\") pod \"maas-default-gateway-openshift-default-78bb78ff78-nbnx9\" (UID: \"a5425b39-1732-47b5-90b4-774966781b06\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-nbnx9"
Sep 10 17:43:11.400305 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:43:11.400285    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a5425b39-1732-47b5-90b4-774966781b06-istio-podinfo\") pod \"maas-default-gateway-openshift-default-78bb78ff78-nbnx9\" (UID: \"a5425b39-1732-47b5-90b4-774966781b06\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-nbnx9"
Sep 10 17:43:11.412063 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:43:11.412033    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-sfdst\" (UniqueName: \"kubernetes.io/projected/a5425b39-1732-47b5-90b4-774966781b06-kube-api-access-sfdst\") pod \"maas-default-gateway-openshift-default-78bb78ff78-nbnx9\" (UID: \"a5425b39-1732-47b5-90b4-774966781b06\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-nbnx9"
Sep 10 17:43:11.415811 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:43:11.415790    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a5425b39-1732-47b5-90b4-774966781b06-istio-token\") pod \"maas-default-gateway-openshift-default-78bb78ff78-nbnx9\" (UID: \"a5425b39-1732-47b5-90b4-774966781b06\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-nbnx9"
Sep 10 17:43:11.455841 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:43:11.455812    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-nbnx9"
Sep 10 17:43:11.732550 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:43:11.732522    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-nbnx9"]
Sep 10 17:43:11.734304 ip-10-0-141-180 kubenswrapper[2581]: W0910 17:43:11.734274    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda5425b39_1732_47b5_90b4_774966781b06.slice/crio-9c6d7ae4266fc04b0f6c702ca1db86c274f512c99bba144caa6a56385c1d2544 WatchSource:0}: Error finding container 9c6d7ae4266fc04b0f6c702ca1db86c274f512c99bba144caa6a56385c1d2544: Status 404 returned error can't find the container with id 9c6d7ae4266fc04b0f6c702ca1db86c274f512c99bba144caa6a56385c1d2544
Sep 10 17:43:11.736612 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:43:11.736564    2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 10 17:43:11.736708 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:43:11.736676    2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 10 17:43:11.736752 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:43:11.736725    2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 10 17:43:11.831852 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:43:11.831813    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-nbnx9" event={"ID":"a5425b39-1732-47b5-90b4-774966781b06","Type":"ContainerStarted","Data":"f342e72adc49b0856442520f88a6c948d768d4819cf2e1bd2510be8e0d0a5922"}
Sep 10 17:43:11.832031 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:43:11.831880    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-nbnx9" event={"ID":"a5425b39-1732-47b5-90b4-774966781b06","Type":"ContainerStarted","Data":"9c6d7ae4266fc04b0f6c702ca1db86c274f512c99bba144caa6a56385c1d2544"}
Sep 10 17:43:11.862264 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:43:11.862209    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-nbnx9" podStartSLOduration=0.862194788 podStartE2EDuration="862.194788ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 17:43:11 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 17:43:11.85940187 +0000 UTC m=+411.513475807" watchObservedRunningTime="2026-09-10 17:43:11.862194788 +0000 UTC m=+411.516268691"
Sep 10 17:43:12.456008 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:43:12.455971    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-nbnx9"
Sep 10 17:43:12.460377 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:43:12.460351    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-nbnx9"
Sep 10 17:43:12.837465 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:43:12.837430    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-nbnx9"
Sep 10 17:43:12.838420 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:43:12.838398    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-nbnx9"
Sep 10 17:43:50.289541 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:43:50.289469    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-75895f5d8-524zp"]
Sep 10 17:43:50.292928 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:43:50.292911    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-75895f5d8-524zp"
Sep 10 17:43:50.298067 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:43:50.298040    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-metrics-tls\""
Sep 10 17:43:50.298259 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:43:50.298144    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-bbp7v\""
Sep 10 17:43:50.298987 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:43:50.298965    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\""
Sep 10 17:43:50.299089 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:43:50.298969    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\""
Sep 10 17:43:50.314100 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:43:50.312735    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-75895f5d8-524zp"]
Sep 10 17:43:50.429573 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:43:50.429535    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/3f80838f-1a03-4dea-8e29-4aedb344ceb2-webhook-cert\") pod \"maas-controller-75895f5d8-524zp\" (UID: \"3f80838f-1a03-4dea-8e29-4aedb344ceb2\") " pod="opendatahub/maas-controller-75895f5d8-524zp"
Sep 10 17:43:50.429750 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:43:50.429580    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8mqtl\" (UniqueName: \"kubernetes.io/projected/3f80838f-1a03-4dea-8e29-4aedb344ceb2-kube-api-access-8mqtl\") pod \"maas-controller-75895f5d8-524zp\" (UID: \"3f80838f-1a03-4dea-8e29-4aedb344ceb2\") " pod="opendatahub/maas-controller-75895f5d8-524zp"
Sep 10 17:43:50.429750 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:43:50.429615    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3f80838f-1a03-4dea-8e29-4aedb344ceb2-metrics-certs\") pod \"maas-controller-75895f5d8-524zp\" (UID: \"3f80838f-1a03-4dea-8e29-4aedb344ceb2\") " pod="opendatahub/maas-controller-75895f5d8-524zp"
Sep 10 17:43:50.530218 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:43:50.530172    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3f80838f-1a03-4dea-8e29-4aedb344ceb2-metrics-certs\") pod \"maas-controller-75895f5d8-524zp\" (UID: \"3f80838f-1a03-4dea-8e29-4aedb344ceb2\") " pod="opendatahub/maas-controller-75895f5d8-524zp"
Sep 10 17:43:50.530401 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:43:50.530277    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/3f80838f-1a03-4dea-8e29-4aedb344ceb2-webhook-cert\") pod \"maas-controller-75895f5d8-524zp\" (UID: \"3f80838f-1a03-4dea-8e29-4aedb344ceb2\") " pod="opendatahub/maas-controller-75895f5d8-524zp"
Sep 10 17:43:50.530401 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:43:50.530301    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8mqtl\" (UniqueName: \"kubernetes.io/projected/3f80838f-1a03-4dea-8e29-4aedb344ceb2-kube-api-access-8mqtl\") pod \"maas-controller-75895f5d8-524zp\" (UID: \"3f80838f-1a03-4dea-8e29-4aedb344ceb2\") " pod="opendatahub/maas-controller-75895f5d8-524zp"
Sep 10 17:43:50.530401 ip-10-0-141-180 kubenswrapper[2581]: E0910 17:43:50.530322    2581 secret.go:189] Couldn't get secret opendatahub/maas-controller-metrics-tls: secret "maas-controller-metrics-tls" not found
Sep 10 17:43:50.530401 ip-10-0-141-180 kubenswrapper[2581]: E0910 17:43:50.530400    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3f80838f-1a03-4dea-8e29-4aedb344ceb2-metrics-certs podName:3f80838f-1a03-4dea-8e29-4aedb344ceb2 nodeName:}" failed. No retries permitted until 2026-09-10 17:43:51.030383455 +0000 UTC m=+450.684457336 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/3f80838f-1a03-4dea-8e29-4aedb344ceb2-metrics-certs") pod "maas-controller-75895f5d8-524zp" (UID: "3f80838f-1a03-4dea-8e29-4aedb344ceb2") : secret "maas-controller-metrics-tls" not found
Sep 10 17:43:50.532719 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:43:50.532697    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/3f80838f-1a03-4dea-8e29-4aedb344ceb2-webhook-cert\") pod \"maas-controller-75895f5d8-524zp\" (UID: \"3f80838f-1a03-4dea-8e29-4aedb344ceb2\") " pod="opendatahub/maas-controller-75895f5d8-524zp"
Sep 10 17:43:50.538766 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:43:50.538747    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8mqtl\" (UniqueName: \"kubernetes.io/projected/3f80838f-1a03-4dea-8e29-4aedb344ceb2-kube-api-access-8mqtl\") pod \"maas-controller-75895f5d8-524zp\" (UID: \"3f80838f-1a03-4dea-8e29-4aedb344ceb2\") " pod="opendatahub/maas-controller-75895f5d8-524zp"
Sep 10 17:43:51.036042 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:43:51.035999    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3f80838f-1a03-4dea-8e29-4aedb344ceb2-metrics-certs\") pod \"maas-controller-75895f5d8-524zp\" (UID: \"3f80838f-1a03-4dea-8e29-4aedb344ceb2\") " pod="opendatahub/maas-controller-75895f5d8-524zp"
Sep 10 17:43:51.038426 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:43:51.038398    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3f80838f-1a03-4dea-8e29-4aedb344ceb2-metrics-certs\") pod \"maas-controller-75895f5d8-524zp\" (UID: \"3f80838f-1a03-4dea-8e29-4aedb344ceb2\") " pod="opendatahub/maas-controller-75895f5d8-524zp"
Sep 10 17:43:51.203457 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:43:51.203426    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-75895f5d8-524zp"
Sep 10 17:43:51.478273 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:43:51.478233    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-75895f5d8-524zp"]
Sep 10 17:43:52.054280 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:43:52.054232    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-75895f5d8-524zp" event={"ID":"3f80838f-1a03-4dea-8e29-4aedb344ceb2","Type":"ContainerStarted","Data":"a86821e2288032daa3c766f4972e7db93cb206c047324a0de16621f41d5bcb4e"}
Sep 10 17:43:54.070363 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:43:54.070276    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-75895f5d8-524zp" event={"ID":"3f80838f-1a03-4dea-8e29-4aedb344ceb2","Type":"ContainerStarted","Data":"b374738c5642ddfad89ae1603ff593c2a92dfdbdd2313de1216553dcfc06b017"}
Sep 10 17:43:54.070754 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:43:54.070394    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-75895f5d8-524zp"
Sep 10 17:43:54.097712 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:43:54.097667    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-75895f5d8-524zp" podStartSLOduration=1.8323200800000001 podStartE2EDuration="4.097652388s" totalImagesPullingTime="2.265332308s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 17:43:50 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 17:43:54.095105866 +0000 UTC m=+453.749179816" watchObservedRunningTime="2026-09-10 17:43:54.097652388 +0000 UTC m=+453.751726283"
Sep 10 17:44:05.077484 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:44:05.077453    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-75895f5d8-524zp"
Sep 10 17:44:28.367468 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:44:28.367431    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-59fbc47bf8-jdd25"]
Sep 10 17:44:28.371115 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:44:28.371092    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-59fbc47bf8-jdd25"
Sep 10 17:44:28.374017 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:44:28.373997    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins\""
Sep 10 17:44:28.390476 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:44:28.390439    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-59fbc47bf8-jdd25"]
Sep 10 17:44:28.446707 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:44:28.446677    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/cc45ca90-2458-4f31-aeb6-cef3e8a9cf2a-plugins-config-volume\") pod \"payload-pre-processing-59fbc47bf8-jdd25\" (UID: \"cc45ca90-2458-4f31-aeb6-cef3e8a9cf2a\") " pod="openshift-ingress/payload-pre-processing-59fbc47bf8-jdd25"
Sep 10 17:44:28.446860 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:44:28.446764    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xp8bb\" (UniqueName: \"kubernetes.io/projected/cc45ca90-2458-4f31-aeb6-cef3e8a9cf2a-kube-api-access-xp8bb\") pod \"payload-pre-processing-59fbc47bf8-jdd25\" (UID: \"cc45ca90-2458-4f31-aeb6-cef3e8a9cf2a\") " pod="openshift-ingress/payload-pre-processing-59fbc47bf8-jdd25"
Sep 10 17:44:28.548285 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:44:28.548243    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/cc45ca90-2458-4f31-aeb6-cef3e8a9cf2a-plugins-config-volume\") pod \"payload-pre-processing-59fbc47bf8-jdd25\" (UID: \"cc45ca90-2458-4f31-aeb6-cef3e8a9cf2a\") " pod="openshift-ingress/payload-pre-processing-59fbc47bf8-jdd25"
Sep 10 17:44:28.548612 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:44:28.548572    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xp8bb\" (UniqueName: \"kubernetes.io/projected/cc45ca90-2458-4f31-aeb6-cef3e8a9cf2a-kube-api-access-xp8bb\") pod \"payload-pre-processing-59fbc47bf8-jdd25\" (UID: \"cc45ca90-2458-4f31-aeb6-cef3e8a9cf2a\") " pod="openshift-ingress/payload-pre-processing-59fbc47bf8-jdd25"
Sep 10 17:44:28.553541 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:44:28.549834    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/cc45ca90-2458-4f31-aeb6-cef3e8a9cf2a-plugins-config-volume\") pod \"payload-pre-processing-59fbc47bf8-jdd25\" (UID: \"cc45ca90-2458-4f31-aeb6-cef3e8a9cf2a\") " pod="openshift-ingress/payload-pre-processing-59fbc47bf8-jdd25"
Sep 10 17:44:28.561027 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:44:28.560998    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xp8bb\" (UniqueName: \"kubernetes.io/projected/cc45ca90-2458-4f31-aeb6-cef3e8a9cf2a-kube-api-access-xp8bb\") pod \"payload-pre-processing-59fbc47bf8-jdd25\" (UID: \"cc45ca90-2458-4f31-aeb6-cef3e8a9cf2a\") " pod="openshift-ingress/payload-pre-processing-59fbc47bf8-jdd25"
Sep 10 17:44:28.681014 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:44:28.680928    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-59fbc47bf8-jdd25"
Sep 10 17:44:28.961700 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:44:28.961675    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-59fbc47bf8-jdd25"]
Sep 10 17:44:28.963766 ip-10-0-141-180 kubenswrapper[2581]: W0910 17:44:28.963736    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcc45ca90_2458_4f31_aeb6_cef3e8a9cf2a.slice/crio-32da6188e85aca9e5934dc0c12bb65b16c96f3fb38198e8d15492b84a7ef6471 WatchSource:0}: Error finding container 32da6188e85aca9e5934dc0c12bb65b16c96f3fb38198e8d15492b84a7ef6471: Status 404 returned error can't find the container with id 32da6188e85aca9e5934dc0c12bb65b16c96f3fb38198e8d15492b84a7ef6471
Sep 10 17:44:29.266172 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:44:29.266081    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-59fbc47bf8-jdd25" event={"ID":"cc45ca90-2458-4f31-aeb6-cef3e8a9cf2a","Type":"ContainerStarted","Data":"32da6188e85aca9e5934dc0c12bb65b16c96f3fb38198e8d15492b84a7ef6471"}
Sep 10 17:44:31.283771 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:44:31.283731    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-59fbc47bf8-jdd25" event={"ID":"cc45ca90-2458-4f31-aeb6-cef3e8a9cf2a","Type":"ContainerStarted","Data":"c5bb0af2565bb5d16427672dae5fc2b442872c9ab2b68effbec2843205180242"}
Sep 10 17:44:31.284246 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:44:31.283986    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-59fbc47bf8-jdd25"
Sep 10 17:44:31.304676 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:44:31.304613    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-59fbc47bf8-jdd25" podStartSLOduration=1.460678454 podStartE2EDuration="3.304597747s" totalImagesPullingTime="1.843919293s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 17:44:28 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 17:44:31.302159776 +0000 UTC m=+490.956233699" watchObservedRunningTime="2026-09-10 17:44:31.304597747 +0000 UTC m=+490.958671642"
Sep 10 17:44:38.330873 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:44:38.330789    2581 generic.go:356] "Generic (PLEG): container finished" podID="a5425b39-1732-47b5-90b4-774966781b06" containerID="f342e72adc49b0856442520f88a6c948d768d4819cf2e1bd2510be8e0d0a5922" exitCode=0
Sep 10 17:44:38.331257 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:44:38.330865    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-nbnx9" event={"ID":"a5425b39-1732-47b5-90b4-774966781b06","Type":"ContainerDied","Data":"f342e72adc49b0856442520f88a6c948d768d4819cf2e1bd2510be8e0d0a5922"}
Sep 10 17:44:38.331301 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:44:38.331272    2581 scope.go:117] "RemoveContainer" containerID="f342e72adc49b0856442520f88a6c948d768d4819cf2e1bd2510be8e0d0a5922"
Sep 10 17:44:38.332231 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:44:38.332206    2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 10 17:44:38.332280 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:44:38.332268    2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 10 17:44:38.332316 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:44:38.332299    2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 10 17:44:39.338726 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:44:39.338691    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-nbnx9" event={"ID":"a5425b39-1732-47b5-90b4-774966781b06","Type":"ContainerStarted","Data":"28cabb745d67f7509a9278af2acc99b88d8c76167fcb74120b3c6b87bfb1f29a"}
Sep 10 17:44:39.456331 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:44:39.456288    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-nbnx9"
Sep 10 17:44:39.457494 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:44:39.457453    2581 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-78bb78ff78-nbnx9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.37:15021/healthz/ready\": dial tcp 10.133.0.37:15021: connect: connection refused" start-of-body=
Sep 10 17:44:39.457735 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:44:39.457501    2581 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-nbnx9" podUID="a5425b39-1732-47b5-90b4-774966781b06" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.37:15021/healthz/ready\": dial tcp 10.133.0.37:15021: connect: connection refused"
Sep 10 17:44:40.456599 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:44:40.456557    2581 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-78bb78ff78-nbnx9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.37:15021/healthz/ready\": dial tcp 10.133.0.37:15021: connect: connection refused" start-of-body=
Sep 10 17:44:40.457078 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:44:40.456632    2581 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-nbnx9" podUID="a5425b39-1732-47b5-90b4-774966781b06" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.37:15021/healthz/ready\": dial tcp 10.133.0.37:15021: connect: connection refused"
Sep 10 17:44:41.457057 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:44:41.457023    2581 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-78bb78ff78-nbnx9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.37:15021/healthz/ready\": dial tcp 10.133.0.37:15021: connect: connection refused" start-of-body=
Sep 10 17:44:41.457432 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:44:41.457088    2581 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-nbnx9" podUID="a5425b39-1732-47b5-90b4-774966781b06" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.37:15021/healthz/ready\": dial tcp 10.133.0.37:15021: connect: connection refused"
Sep 10 17:44:42.295858 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:44:42.295827    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-59fbc47bf8-jdd25"
Sep 10 17:44:42.456463 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:44:42.456428    2581 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-78bb78ff78-nbnx9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.37:15021/healthz/ready\": dial tcp 10.133.0.37:15021: connect: connection refused" start-of-body=
Sep 10 17:44:42.456673 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:44:42.456482    2581 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-nbnx9" podUID="a5425b39-1732-47b5-90b4-774966781b06" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.37:15021/healthz/ready\": dial tcp 10.133.0.37:15021: connect: connection refused"
Sep 10 17:44:42.838164 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:44:42.838130    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-nbnx9"
Sep 10 17:44:43.456860 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:44:43.456821    2581 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-78bb78ff78-nbnx9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.37:15021/healthz/ready\": dial tcp 10.133.0.37:15021: connect: connection refused" start-of-body=
Sep 10 17:44:43.457037 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:44:43.456882    2581 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-nbnx9" podUID="a5425b39-1732-47b5-90b4-774966781b06" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.37:15021/healthz/ready\": dial tcp 10.133.0.37:15021: connect: connection refused"
Sep 10 17:44:44.456437 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:44:44.456392    2581 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-78bb78ff78-nbnx9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.37:15021/healthz/ready\": dial tcp 10.133.0.37:15021: connect: connection refused" start-of-body=
Sep 10 17:44:44.456851 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:44:44.456467    2581 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-nbnx9" podUID="a5425b39-1732-47b5-90b4-774966781b06" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.37:15021/healthz/ready\": dial tcp 10.133.0.37:15021: connect: connection refused"
Sep 10 17:44:45.460245 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:44:45.460214    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-nbnx9"
Sep 10 17:44:45.461115 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:44:45.461095    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-nbnx9"
Sep 10 17:44:58.262941 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:44:58.262910    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-dmnd6"]
Sep 10 17:44:58.266662 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:44:58.266631    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-dmnd6"
Sep 10 17:44:58.269731 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:44:58.269710    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-distinct-2-simulated-kserve-self-signed-certs\""
Sep 10 17:44:58.269882 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:44:58.269863    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-pntqt\""
Sep 10 17:44:58.270691 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:44:58.270670    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\""
Sep 10 17:44:58.270801 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:44:58.270723    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\""
Sep 10 17:44:58.285406 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:44:58.285382    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-dmnd6"]
Sep 10 17:44:58.418917 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:44:58.418874    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f3313d0c-b37f-4758-a78e-d28b750ffab5-dshm\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-dmnd6\" (UID: \"f3313d0c-b37f-4758-a78e-d28b750ffab5\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-dmnd6"
Sep 10 17:44:58.419091 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:44:58.418971    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f3313d0c-b37f-4758-a78e-d28b750ffab5-home\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-dmnd6\" (UID: \"f3313d0c-b37f-4758-a78e-d28b750ffab5\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-dmnd6"
Sep 10 17:44:58.419091 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:44:58.418992    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f3313d0c-b37f-4758-a78e-d28b750ffab5-model-cache\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-dmnd6\" (UID: \"f3313d0c-b37f-4758-a78e-d28b750ffab5\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-dmnd6"
Sep 10 17:44:58.419091 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:44:58.419015    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7cx54\" (UniqueName: \"kubernetes.io/projected/f3313d0c-b37f-4758-a78e-d28b750ffab5-kube-api-access-7cx54\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-dmnd6\" (UID: \"f3313d0c-b37f-4758-a78e-d28b750ffab5\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-dmnd6"
Sep 10 17:44:58.419091 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:44:58.419043    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f3313d0c-b37f-4758-a78e-d28b750ffab5-tmp-dir\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-dmnd6\" (UID: \"f3313d0c-b37f-4758-a78e-d28b750ffab5\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-dmnd6"
Sep 10 17:44:58.419091 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:44:58.419086    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f3313d0c-b37f-4758-a78e-d28b750ffab5-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-dmnd6\" (UID: \"f3313d0c-b37f-4758-a78e-d28b750ffab5\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-dmnd6"
Sep 10 17:44:58.519576 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:44:58.519498    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f3313d0c-b37f-4758-a78e-d28b750ffab5-home\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-dmnd6\" (UID: \"f3313d0c-b37f-4758-a78e-d28b750ffab5\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-dmnd6"
Sep 10 17:44:58.519576 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:44:58.519535    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f3313d0c-b37f-4758-a78e-d28b750ffab5-model-cache\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-dmnd6\" (UID: \"f3313d0c-b37f-4758-a78e-d28b750ffab5\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-dmnd6"
Sep 10 17:44:58.519576 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:44:58.519555    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7cx54\" (UniqueName: \"kubernetes.io/projected/f3313d0c-b37f-4758-a78e-d28b750ffab5-kube-api-access-7cx54\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-dmnd6\" (UID: \"f3313d0c-b37f-4758-a78e-d28b750ffab5\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-dmnd6"
Sep 10 17:44:58.519576 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:44:58.519574    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f3313d0c-b37f-4758-a78e-d28b750ffab5-tmp-dir\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-dmnd6\" (UID: \"f3313d0c-b37f-4758-a78e-d28b750ffab5\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-dmnd6"
Sep 10 17:44:58.519860 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:44:58.519592    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f3313d0c-b37f-4758-a78e-d28b750ffab5-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-dmnd6\" (UID: \"f3313d0c-b37f-4758-a78e-d28b750ffab5\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-dmnd6"
Sep 10 17:44:58.519860 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:44:58.519670    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f3313d0c-b37f-4758-a78e-d28b750ffab5-dshm\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-dmnd6\" (UID: \"f3313d0c-b37f-4758-a78e-d28b750ffab5\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-dmnd6"
Sep 10 17:44:58.520018 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:44:58.519994    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f3313d0c-b37f-4758-a78e-d28b750ffab5-model-cache\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-dmnd6\" (UID: \"f3313d0c-b37f-4758-a78e-d28b750ffab5\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-dmnd6"
Sep 10 17:44:58.520054 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:44:58.520019    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f3313d0c-b37f-4758-a78e-d28b750ffab5-home\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-dmnd6\" (UID: \"f3313d0c-b37f-4758-a78e-d28b750ffab5\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-dmnd6"
Sep 10 17:44:58.520089 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:44:58.520070    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f3313d0c-b37f-4758-a78e-d28b750ffab5-tmp-dir\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-dmnd6\" (UID: \"f3313d0c-b37f-4758-a78e-d28b750ffab5\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-dmnd6"
Sep 10 17:44:58.521895 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:44:58.521869    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f3313d0c-b37f-4758-a78e-d28b750ffab5-dshm\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-dmnd6\" (UID: \"f3313d0c-b37f-4758-a78e-d28b750ffab5\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-dmnd6"
Sep 10 17:44:58.522188 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:44:58.522168    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f3313d0c-b37f-4758-a78e-d28b750ffab5-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-dmnd6\" (UID: \"f3313d0c-b37f-4758-a78e-d28b750ffab5\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-dmnd6"
Sep 10 17:44:58.527674 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:44:58.527633    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7cx54\" (UniqueName: \"kubernetes.io/projected/f3313d0c-b37f-4758-a78e-d28b750ffab5-kube-api-access-7cx54\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-dmnd6\" (UID: \"f3313d0c-b37f-4758-a78e-d28b750ffab5\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-dmnd6"
Sep 10 17:44:58.577681 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:44:58.577632    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-dmnd6"
Sep 10 17:44:58.854145 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:44:58.854117    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-dmnd6"]
Sep 10 17:44:58.855549 ip-10-0-141-180 kubenswrapper[2581]: W0910 17:44:58.855520    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf3313d0c_b37f_4758_a78e_d28b750ffab5.slice/crio-b813c7514b19c8b7ba3d7012ce95238ffd033fde12fb0d10a2c78f2914715a73 WatchSource:0}: Error finding container b813c7514b19c8b7ba3d7012ce95238ffd033fde12fb0d10a2c78f2914715a73: Status 404 returned error can't find the container with id b813c7514b19c8b7ba3d7012ce95238ffd033fde12fb0d10a2c78f2914715a73
Sep 10 17:44:59.454371 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:44:59.454327    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-dmnd6" event={"ID":"f3313d0c-b37f-4758-a78e-d28b750ffab5","Type":"ContainerStarted","Data":"b813c7514b19c8b7ba3d7012ce95238ffd033fde12fb0d10a2c78f2914715a73"}
Sep 10 17:45:00.152385 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:45:00.152351    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29817705-9x9dh"]
Sep 10 17:45:00.156016 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:45:00.155993    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29817705-9x9dh"
Sep 10 17:45:00.159145 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:45:00.159126    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-k7zj6\""
Sep 10 17:45:00.167321 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:45:00.167298    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29817705-9x9dh"
Sep 10 17:45:00.169707 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:45:00.169681    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29817705-9x9dh"]
Sep 10 17:45:00.446874 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:45:00.446849    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29817705-9x9dh"]
Sep 10 17:45:00.449023 ip-10-0-141-180 kubenswrapper[2581]: W0910 17:45:00.448994    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2cdabae7_bf8c_41c5_813a_fa8fc475eeeb.slice/crio-8188655cd9732c51c8e7762dd3aea5df728f18a0d87931ecf3b62ed6d27a64c9 WatchSource:0}: Error finding container 8188655cd9732c51c8e7762dd3aea5df728f18a0d87931ecf3b62ed6d27a64c9: Status 404 returned error can't find the container with id 8188655cd9732c51c8e7762dd3aea5df728f18a0d87931ecf3b62ed6d27a64c9
Sep 10 17:45:00.462277 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:45:00.462247    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-dmnd6" event={"ID":"f3313d0c-b37f-4758-a78e-d28b750ffab5","Type":"ContainerStarted","Data":"3b596012ff786aae12792b3d27079d0b5c6eea1456d6c70240c128913042106b"}
Sep 10 17:45:00.463561 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:45:00.463539    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29817705-9x9dh" event={"ID":"2cdabae7-bf8c-41c5-813a-fa8fc475eeeb","Type":"ContainerStarted","Data":"8188655cd9732c51c8e7762dd3aea5df728f18a0d87931ecf3b62ed6d27a64c9"}
Sep 10 17:45:00.495600 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:45:00.495559    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-dmnd6" podStartSLOduration=1.310844417 podStartE2EDuration="2.495544965s" totalImagesPullingTime="1.184700548s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 17:44:58 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 17:45:00.493253232 +0000 UTC m=+520.147327146" watchObservedRunningTime="2026-09-10 17:45:00.495544965 +0000 UTC m=+520.149618869"
Sep 10 17:45:03.491943 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:45:03.491902    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29817705-9x9dh" event={"ID":"2cdabae7-bf8c-41c5-813a-fa8fc475eeeb","Type":"ContainerStarted","Data":"fc806e4006b20e2143e1e8c50f96772a82cb461d65788bccfd55f5af4371198e"}
Sep 10 17:45:03.528874 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:45:03.528813    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29817705-9x9dh" podStartSLOduration=1.093180548 podStartE2EDuration="3.528793667s" totalImagesPullingTime="2.435613119s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 17:45:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 17:45:03.528059594 +0000 UTC m=+523.182133728" watchObservedRunningTime="2026-09-10 17:45:03.528793667 +0000 UTC m=+523.182867608"
Sep 10 17:45:04.499016 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:45:04.498979    2581 generic.go:356] "Generic (PLEG): container finished" podID="2cdabae7-bf8c-41c5-813a-fa8fc475eeeb" containerID="fc806e4006b20e2143e1e8c50f96772a82cb461d65788bccfd55f5af4371198e" exitCode=0
Sep 10 17:45:04.499421 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:45:04.499040    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29817705-9x9dh" event={"ID":"2cdabae7-bf8c-41c5-813a-fa8fc475eeeb","Type":"ContainerDied","Data":"fc806e4006b20e2143e1e8c50f96772a82cb461d65788bccfd55f5af4371198e"}
Sep 10 17:45:05.637300 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:45:05.637272    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29817705-9x9dh"
Sep 10 17:45:06.511340 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:45:06.511308    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29817705-9x9dh"
Sep 10 17:45:06.511537 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:45:06.511343    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29817705-9x9dh" event={"ID":"2cdabae7-bf8c-41c5-813a-fa8fc475eeeb","Type":"ContainerDied","Data":"8188655cd9732c51c8e7762dd3aea5df728f18a0d87931ecf3b62ed6d27a64c9"}
Sep 10 17:45:06.511537 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:45:06.511392    2581 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="8188655cd9732c51c8e7762dd3aea5df728f18a0d87931ecf3b62ed6d27a64c9"
Sep 10 17:45:08.578735 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:45:08.578686    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-dmnd6"
Sep 10 17:45:08.579220 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:45:08.578770    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-dmnd6"
Sep 10 17:45:08.591363 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:45:08.591334    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-dmnd6"
Sep 10 17:45:09.539744 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:45:09.539716    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-dmnd6"
Sep 10 17:45:12.527476 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:45:12.527444    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-distinct-simulated-kserve-59975d69f9-7m6ws"]
Sep 10 17:45:12.527946 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:45:12.527931    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2cdabae7-bf8c-41c5-813a-fa8fc475eeeb" containerName="cleanup"
Sep 10 17:45:12.527997 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:45:12.527948    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="2cdabae7-bf8c-41c5-813a-fa8fc475eeeb" containerName="cleanup"
Sep 10 17:45:12.528032 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:45:12.528013    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="2cdabae7-bf8c-41c5-813a-fa8fc475eeeb" containerName="cleanup"
Sep 10 17:45:12.531751 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:45:12.531727    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-7m6ws"
Sep 10 17:45:12.535035 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:45:12.535013    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-distinct-simulated-kserve-self-signed-certs\""
Sep 10 17:45:12.550048 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:45:12.550023    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-simulated-kserve-59975d69f9-7m6ws"]
Sep 10 17:45:12.654925 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:45:12.654886    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2m2hz\" (UniqueName: \"kubernetes.io/projected/efe38876-314b-48ab-814a-e512a1dd0608-kube-api-access-2m2hz\") pod \"e2e-distinct-simulated-kserve-59975d69f9-7m6ws\" (UID: \"efe38876-314b-48ab-814a-e512a1dd0608\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-7m6ws"
Sep 10 17:45:12.655110 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:45:12.654986    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/efe38876-314b-48ab-814a-e512a1dd0608-model-cache\") pod \"e2e-distinct-simulated-kserve-59975d69f9-7m6ws\" (UID: \"efe38876-314b-48ab-814a-e512a1dd0608\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-7m6ws"
Sep 10 17:45:12.655110 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:45:12.655033    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/efe38876-314b-48ab-814a-e512a1dd0608-dshm\") pod \"e2e-distinct-simulated-kserve-59975d69f9-7m6ws\" (UID: \"efe38876-314b-48ab-814a-e512a1dd0608\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-7m6ws"
Sep 10 17:45:12.655237 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:45:12.655190    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/efe38876-314b-48ab-814a-e512a1dd0608-home\") pod \"e2e-distinct-simulated-kserve-59975d69f9-7m6ws\" (UID: \"efe38876-314b-48ab-814a-e512a1dd0608\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-7m6ws"
Sep 10 17:45:12.655296 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:45:12.655241    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/efe38876-314b-48ab-814a-e512a1dd0608-tls-certs\") pod \"e2e-distinct-simulated-kserve-59975d69f9-7m6ws\" (UID: \"efe38876-314b-48ab-814a-e512a1dd0608\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-7m6ws"
Sep 10 17:45:12.655349 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:45:12.655315    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/efe38876-314b-48ab-814a-e512a1dd0608-tmp-dir\") pod \"e2e-distinct-simulated-kserve-59975d69f9-7m6ws\" (UID: \"efe38876-314b-48ab-814a-e512a1dd0608\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-7m6ws"
Sep 10 17:45:12.756427 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:45:12.756381    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2m2hz\" (UniqueName: \"kubernetes.io/projected/efe38876-314b-48ab-814a-e512a1dd0608-kube-api-access-2m2hz\") pod \"e2e-distinct-simulated-kserve-59975d69f9-7m6ws\" (UID: \"efe38876-314b-48ab-814a-e512a1dd0608\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-7m6ws"
Sep 10 17:45:12.756625 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:45:12.756452    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/efe38876-314b-48ab-814a-e512a1dd0608-model-cache\") pod \"e2e-distinct-simulated-kserve-59975d69f9-7m6ws\" (UID: \"efe38876-314b-48ab-814a-e512a1dd0608\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-7m6ws"
Sep 10 17:45:12.756625 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:45:12.756490    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/efe38876-314b-48ab-814a-e512a1dd0608-dshm\") pod \"e2e-distinct-simulated-kserve-59975d69f9-7m6ws\" (UID: \"efe38876-314b-48ab-814a-e512a1dd0608\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-7m6ws"
Sep 10 17:45:12.756625 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:45:12.756571    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/efe38876-314b-48ab-814a-e512a1dd0608-home\") pod \"e2e-distinct-simulated-kserve-59975d69f9-7m6ws\" (UID: \"efe38876-314b-48ab-814a-e512a1dd0608\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-7m6ws"
Sep 10 17:45:12.756850 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:45:12.756665    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/efe38876-314b-48ab-814a-e512a1dd0608-tls-certs\") pod \"e2e-distinct-simulated-kserve-59975d69f9-7m6ws\" (UID: \"efe38876-314b-48ab-814a-e512a1dd0608\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-7m6ws"
Sep 10 17:45:12.756850 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:45:12.756704    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/efe38876-314b-48ab-814a-e512a1dd0608-tmp-dir\") pod \"e2e-distinct-simulated-kserve-59975d69f9-7m6ws\" (UID: \"efe38876-314b-48ab-814a-e512a1dd0608\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-7m6ws"
Sep 10 17:45:12.756958 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:45:12.756931    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/efe38876-314b-48ab-814a-e512a1dd0608-model-cache\") pod \"e2e-distinct-simulated-kserve-59975d69f9-7m6ws\" (UID: \"efe38876-314b-48ab-814a-e512a1dd0608\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-7m6ws"
Sep 10 17:45:12.757018 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:45:12.756965    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/efe38876-314b-48ab-814a-e512a1dd0608-home\") pod \"e2e-distinct-simulated-kserve-59975d69f9-7m6ws\" (UID: \"efe38876-314b-48ab-814a-e512a1dd0608\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-7m6ws"
Sep 10 17:45:12.757080 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:45:12.757062    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/efe38876-314b-48ab-814a-e512a1dd0608-tmp-dir\") pod \"e2e-distinct-simulated-kserve-59975d69f9-7m6ws\" (UID: \"efe38876-314b-48ab-814a-e512a1dd0608\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-7m6ws"
Sep 10 17:45:12.758888 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:45:12.758870    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/efe38876-314b-48ab-814a-e512a1dd0608-dshm\") pod \"e2e-distinct-simulated-kserve-59975d69f9-7m6ws\" (UID: \"efe38876-314b-48ab-814a-e512a1dd0608\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-7m6ws"
Sep 10 17:45:12.759053 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:45:12.759034    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/efe38876-314b-48ab-814a-e512a1dd0608-tls-certs\") pod \"e2e-distinct-simulated-kserve-59975d69f9-7m6ws\" (UID: \"efe38876-314b-48ab-814a-e512a1dd0608\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-7m6ws"
Sep 10 17:45:12.765047 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:45:12.765024    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2m2hz\" (UniqueName: \"kubernetes.io/projected/efe38876-314b-48ab-814a-e512a1dd0608-kube-api-access-2m2hz\") pod \"e2e-distinct-simulated-kserve-59975d69f9-7m6ws\" (UID: \"efe38876-314b-48ab-814a-e512a1dd0608\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-7m6ws"
Sep 10 17:45:12.843125 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:45:12.843060    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-7m6ws"
Sep 10 17:45:13.123690 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:45:13.123661    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-simulated-kserve-59975d69f9-7m6ws"]
Sep 10 17:45:13.123803 ip-10-0-141-180 kubenswrapper[2581]: W0910 17:45:13.123723    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podefe38876_314b_48ab_814a_e512a1dd0608.slice/crio-1213eb05df48dcfc75ca7dfd73a4108d35538d67edf29f1750d1296de986f67e WatchSource:0}: Error finding container 1213eb05df48dcfc75ca7dfd73a4108d35538d67edf29f1750d1296de986f67e: Status 404 returned error can't find the container with id 1213eb05df48dcfc75ca7dfd73a4108d35538d67edf29f1750d1296de986f67e
Sep 10 17:45:13.555541 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:45:13.555499    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-7m6ws" event={"ID":"efe38876-314b-48ab-814a-e512a1dd0608","Type":"ContainerStarted","Data":"b6288ed53b660d7a103f6bbe9d58916a8724a2e5de2cf33657f4631c2d34dc81"}
Sep 10 17:45:13.555992 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:45:13.555564    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-7m6ws" event={"ID":"efe38876-314b-48ab-814a-e512a1dd0608","Type":"ContainerStarted","Data":"1213eb05df48dcfc75ca7dfd73a4108d35538d67edf29f1750d1296de986f67e"}
Sep 10 17:45:13.579510 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:45:13.579445    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-7m6ws" podStartSLOduration=1.355396162 podStartE2EDuration="1.579425044s" totalImagesPullingTime="224.028882ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 17:45:12 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 17:45:13.577000829 +0000 UTC m=+533.231074749" watchObservedRunningTime="2026-09-10 17:45:13.579425044 +0000 UTC m=+533.233498949"
Sep 10 17:45:22.843774 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:45:22.843738    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-7m6ws"
Sep 10 17:45:22.844142 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:45:22.843823    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-7m6ws"
Sep 10 17:45:22.862481 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:45:22.862453    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-7m6ws"
Sep 10 17:45:23.622349 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:45:23.622319    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-7m6ws"
Sep 10 17:48:04.414224 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:48:04.414177    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-inf-a-48d00b-openshift-default-6cc74dc546-kjh62"]
Sep 10 17:48:04.418157 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:48:04.418132    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-48d00b-openshift-default-6cc74dc546-kjh62"
Sep 10 17:48:04.421536 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:48:04.421512    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-inf-a-48d00b-openshift-default-dockercfg-hgqmj\""
Sep 10 17:48:04.427120 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:48:04.427089    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/20ec55f7-fa46-4372-bf1b-791406c99a10-istio-token\") pod \"e2e-inf-a-48d00b-openshift-default-6cc74dc546-kjh62\" (UID: \"20ec55f7-fa46-4372-bf1b-791406c99a10\") " pod="openshift-ingress/e2e-inf-a-48d00b-openshift-default-6cc74dc546-kjh62"
Sep 10 17:48:04.427234 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:48:04.427121    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/20ec55f7-fa46-4372-bf1b-791406c99a10-workload-socket\") pod \"e2e-inf-a-48d00b-openshift-default-6cc74dc546-kjh62\" (UID: \"20ec55f7-fa46-4372-bf1b-791406c99a10\") " pod="openshift-ingress/e2e-inf-a-48d00b-openshift-default-6cc74dc546-kjh62"
Sep 10 17:48:04.427234 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:48:04.427156    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/20ec55f7-fa46-4372-bf1b-791406c99a10-workload-certs\") pod \"e2e-inf-a-48d00b-openshift-default-6cc74dc546-kjh62\" (UID: \"20ec55f7-fa46-4372-bf1b-791406c99a10\") " pod="openshift-ingress/e2e-inf-a-48d00b-openshift-default-6cc74dc546-kjh62"
Sep 10 17:48:04.427234 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:48:04.427197    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/20ec55f7-fa46-4372-bf1b-791406c99a10-istio-envoy\") pod \"e2e-inf-a-48d00b-openshift-default-6cc74dc546-kjh62\" (UID: \"20ec55f7-fa46-4372-bf1b-791406c99a10\") " pod="openshift-ingress/e2e-inf-a-48d00b-openshift-default-6cc74dc546-kjh62"
Sep 10 17:48:04.427234 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:48:04.427219    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/20ec55f7-fa46-4372-bf1b-791406c99a10-istio-data\") pod \"e2e-inf-a-48d00b-openshift-default-6cc74dc546-kjh62\" (UID: \"20ec55f7-fa46-4372-bf1b-791406c99a10\") " pod="openshift-ingress/e2e-inf-a-48d00b-openshift-default-6cc74dc546-kjh62"
Sep 10 17:48:04.427369 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:48:04.427282    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9lnh7\" (UniqueName: \"kubernetes.io/projected/20ec55f7-fa46-4372-bf1b-791406c99a10-kube-api-access-9lnh7\") pod \"e2e-inf-a-48d00b-openshift-default-6cc74dc546-kjh62\" (UID: \"20ec55f7-fa46-4372-bf1b-791406c99a10\") " pod="openshift-ingress/e2e-inf-a-48d00b-openshift-default-6cc74dc546-kjh62"
Sep 10 17:48:04.427369 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:48:04.427315    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/20ec55f7-fa46-4372-bf1b-791406c99a10-istiod-ca-cert\") pod \"e2e-inf-a-48d00b-openshift-default-6cc74dc546-kjh62\" (UID: \"20ec55f7-fa46-4372-bf1b-791406c99a10\") " pod="openshift-ingress/e2e-inf-a-48d00b-openshift-default-6cc74dc546-kjh62"
Sep 10 17:48:04.427369 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:48:04.427339    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/20ec55f7-fa46-4372-bf1b-791406c99a10-istio-podinfo\") pod \"e2e-inf-a-48d00b-openshift-default-6cc74dc546-kjh62\" (UID: \"20ec55f7-fa46-4372-bf1b-791406c99a10\") " pod="openshift-ingress/e2e-inf-a-48d00b-openshift-default-6cc74dc546-kjh62"
Sep 10 17:48:04.427485 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:48:04.427373    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/20ec55f7-fa46-4372-bf1b-791406c99a10-credential-socket\") pod \"e2e-inf-a-48d00b-openshift-default-6cc74dc546-kjh62\" (UID: \"20ec55f7-fa46-4372-bf1b-791406c99a10\") " pod="openshift-ingress/e2e-inf-a-48d00b-openshift-default-6cc74dc546-kjh62"
Sep 10 17:48:04.433097 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:48:04.433070    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-a-48d00b-openshift-default-6cc74dc546-kjh62"]
Sep 10 17:48:04.527822 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:48:04.527787    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/20ec55f7-fa46-4372-bf1b-791406c99a10-credential-socket\") pod \"e2e-inf-a-48d00b-openshift-default-6cc74dc546-kjh62\" (UID: \"20ec55f7-fa46-4372-bf1b-791406c99a10\") " pod="openshift-ingress/e2e-inf-a-48d00b-openshift-default-6cc74dc546-kjh62"
Sep 10 17:48:04.527985 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:48:04.527863    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/20ec55f7-fa46-4372-bf1b-791406c99a10-istio-token\") pod \"e2e-inf-a-48d00b-openshift-default-6cc74dc546-kjh62\" (UID: \"20ec55f7-fa46-4372-bf1b-791406c99a10\") " pod="openshift-ingress/e2e-inf-a-48d00b-openshift-default-6cc74dc546-kjh62"
Sep 10 17:48:04.527985 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:48:04.527886    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/20ec55f7-fa46-4372-bf1b-791406c99a10-workload-socket\") pod \"e2e-inf-a-48d00b-openshift-default-6cc74dc546-kjh62\" (UID: \"20ec55f7-fa46-4372-bf1b-791406c99a10\") " pod="openshift-ingress/e2e-inf-a-48d00b-openshift-default-6cc74dc546-kjh62"
Sep 10 17:48:04.527985 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:48:04.527924    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/20ec55f7-fa46-4372-bf1b-791406c99a10-workload-certs\") pod \"e2e-inf-a-48d00b-openshift-default-6cc74dc546-kjh62\" (UID: \"20ec55f7-fa46-4372-bf1b-791406c99a10\") " pod="openshift-ingress/e2e-inf-a-48d00b-openshift-default-6cc74dc546-kjh62"
Sep 10 17:48:04.527985 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:48:04.527946    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/20ec55f7-fa46-4372-bf1b-791406c99a10-istio-envoy\") pod \"e2e-inf-a-48d00b-openshift-default-6cc74dc546-kjh62\" (UID: \"20ec55f7-fa46-4372-bf1b-791406c99a10\") " pod="openshift-ingress/e2e-inf-a-48d00b-openshift-default-6cc74dc546-kjh62"
Sep 10 17:48:04.527985 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:48:04.527962    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/20ec55f7-fa46-4372-bf1b-791406c99a10-istio-data\") pod \"e2e-inf-a-48d00b-openshift-default-6cc74dc546-kjh62\" (UID: \"20ec55f7-fa46-4372-bf1b-791406c99a10\") " pod="openshift-ingress/e2e-inf-a-48d00b-openshift-default-6cc74dc546-kjh62"
Sep 10 17:48:04.528231 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:48:04.527994    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9lnh7\" (UniqueName: \"kubernetes.io/projected/20ec55f7-fa46-4372-bf1b-791406c99a10-kube-api-access-9lnh7\") pod \"e2e-inf-a-48d00b-openshift-default-6cc74dc546-kjh62\" (UID: \"20ec55f7-fa46-4372-bf1b-791406c99a10\") " pod="openshift-ingress/e2e-inf-a-48d00b-openshift-default-6cc74dc546-kjh62"
Sep 10 17:48:04.528231 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:48:04.528011    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/20ec55f7-fa46-4372-bf1b-791406c99a10-istiod-ca-cert\") pod \"e2e-inf-a-48d00b-openshift-default-6cc74dc546-kjh62\" (UID: \"20ec55f7-fa46-4372-bf1b-791406c99a10\") " pod="openshift-ingress/e2e-inf-a-48d00b-openshift-default-6cc74dc546-kjh62"
Sep 10 17:48:04.528231 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:48:04.528041    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/20ec55f7-fa46-4372-bf1b-791406c99a10-istio-podinfo\") pod \"e2e-inf-a-48d00b-openshift-default-6cc74dc546-kjh62\" (UID: \"20ec55f7-fa46-4372-bf1b-791406c99a10\") " pod="openshift-ingress/e2e-inf-a-48d00b-openshift-default-6cc74dc546-kjh62"
Sep 10 17:48:04.528231 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:48:04.528222    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/20ec55f7-fa46-4372-bf1b-791406c99a10-credential-socket\") pod \"e2e-inf-a-48d00b-openshift-default-6cc74dc546-kjh62\" (UID: \"20ec55f7-fa46-4372-bf1b-791406c99a10\") " pod="openshift-ingress/e2e-inf-a-48d00b-openshift-default-6cc74dc546-kjh62"
Sep 10 17:48:04.528465 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:48:04.528439    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/20ec55f7-fa46-4372-bf1b-791406c99a10-workload-certs\") pod \"e2e-inf-a-48d00b-openshift-default-6cc74dc546-kjh62\" (UID: \"20ec55f7-fa46-4372-bf1b-791406c99a10\") " pod="openshift-ingress/e2e-inf-a-48d00b-openshift-default-6cc74dc546-kjh62"
Sep 10 17:48:04.528529 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:48:04.528479    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/20ec55f7-fa46-4372-bf1b-791406c99a10-istio-data\") pod \"e2e-inf-a-48d00b-openshift-default-6cc74dc546-kjh62\" (UID: \"20ec55f7-fa46-4372-bf1b-791406c99a10\") " pod="openshift-ingress/e2e-inf-a-48d00b-openshift-default-6cc74dc546-kjh62"
Sep 10 17:48:04.528657 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:48:04.528612    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/20ec55f7-fa46-4372-bf1b-791406c99a10-workload-socket\") pod \"e2e-inf-a-48d00b-openshift-default-6cc74dc546-kjh62\" (UID: \"20ec55f7-fa46-4372-bf1b-791406c99a10\") " pod="openshift-ingress/e2e-inf-a-48d00b-openshift-default-6cc74dc546-kjh62"
Sep 10 17:48:04.528908 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:48:04.528889    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/20ec55f7-fa46-4372-bf1b-791406c99a10-istiod-ca-cert\") pod \"e2e-inf-a-48d00b-openshift-default-6cc74dc546-kjh62\" (UID: \"20ec55f7-fa46-4372-bf1b-791406c99a10\") " pod="openshift-ingress/e2e-inf-a-48d00b-openshift-default-6cc74dc546-kjh62"
Sep 10 17:48:04.530455 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:48:04.530432    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/20ec55f7-fa46-4372-bf1b-791406c99a10-istio-envoy\") pod \"e2e-inf-a-48d00b-openshift-default-6cc74dc546-kjh62\" (UID: \"20ec55f7-fa46-4372-bf1b-791406c99a10\") " pod="openshift-ingress/e2e-inf-a-48d00b-openshift-default-6cc74dc546-kjh62"
Sep 10 17:48:04.530720 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:48:04.530697    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/20ec55f7-fa46-4372-bf1b-791406c99a10-istio-podinfo\") pod \"e2e-inf-a-48d00b-openshift-default-6cc74dc546-kjh62\" (UID: \"20ec55f7-fa46-4372-bf1b-791406c99a10\") " pod="openshift-ingress/e2e-inf-a-48d00b-openshift-default-6cc74dc546-kjh62"
Sep 10 17:48:04.538205 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:48:04.538185    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/20ec55f7-fa46-4372-bf1b-791406c99a10-istio-token\") pod \"e2e-inf-a-48d00b-openshift-default-6cc74dc546-kjh62\" (UID: \"20ec55f7-fa46-4372-bf1b-791406c99a10\") " pod="openshift-ingress/e2e-inf-a-48d00b-openshift-default-6cc74dc546-kjh62"
Sep 10 17:48:04.538787 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:48:04.538757    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9lnh7\" (UniqueName: \"kubernetes.io/projected/20ec55f7-fa46-4372-bf1b-791406c99a10-kube-api-access-9lnh7\") pod \"e2e-inf-a-48d00b-openshift-default-6cc74dc546-kjh62\" (UID: \"20ec55f7-fa46-4372-bf1b-791406c99a10\") " pod="openshift-ingress/e2e-inf-a-48d00b-openshift-default-6cc74dc546-kjh62"
Sep 10 17:48:04.728990 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:48:04.728957    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-48d00b-openshift-default-6cc74dc546-kjh62"
Sep 10 17:48:05.012335 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:48:05.012302    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-a-48d00b-openshift-default-6cc74dc546-kjh62"]
Sep 10 17:48:05.013793 ip-10-0-141-180 kubenswrapper[2581]: W0910 17:48:05.013760    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod20ec55f7_fa46_4372_bf1b_791406c99a10.slice/crio-9cde959c748ee5b9356027f255d04830d9144237ae707d240090a62d60b18c75 WatchSource:0}: Error finding container 9cde959c748ee5b9356027f255d04830d9144237ae707d240090a62d60b18c75: Status 404 returned error can't find the container with id 9cde959c748ee5b9356027f255d04830d9144237ae707d240090a62d60b18c75
Sep 10 17:48:05.016616 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:48:05.016391    2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 10 17:48:05.016616 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:48:05.016479    2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 10 17:48:05.016616 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:48:05.016522    2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 10 17:48:05.541536 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:48:05.541498    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-48d00b-openshift-default-6cc74dc546-kjh62" event={"ID":"20ec55f7-fa46-4372-bf1b-791406c99a10","Type":"ContainerStarted","Data":"a992fe6c27c2c21e201c10a259fbcbcf8850ea7283b2cdbc34a9974033bf49a9"}
Sep 10 17:48:05.541967 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:48:05.541706    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-48d00b-openshift-default-6cc74dc546-kjh62" event={"ID":"20ec55f7-fa46-4372-bf1b-791406c99a10","Type":"ContainerStarted","Data":"9cde959c748ee5b9356027f255d04830d9144237ae707d240090a62d60b18c75"}
Sep 10 17:48:05.571607 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:48:05.571558    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-inf-a-48d00b-openshift-default-6cc74dc546-kjh62" podStartSLOduration=1.571541239 podStartE2EDuration="1.571541239s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 17:48:04 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 17:48:05.570269474 +0000 UTC m=+705.224343377" watchObservedRunningTime="2026-09-10 17:48:05.571541239 +0000 UTC m=+705.225615142"
Sep 10 17:48:05.729560 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:48:05.729524    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-a-48d00b-openshift-default-6cc74dc546-kjh62"
Sep 10 17:48:06.734898 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:48:06.734866    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-a-48d00b-openshift-default-6cc74dc546-kjh62"
Sep 10 17:48:07.552969 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:48:07.552935    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-a-48d00b-openshift-default-6cc74dc546-kjh62"
Sep 10 17:48:07.553724 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:48:07.553704    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-a-48d00b-openshift-default-6cc74dc546-kjh62"
Sep 10 17:48:09.608940 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:48:09.608898    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-a-48d00b-5fdb8c96fc-kgm8l"]
Sep 10 17:48:09.613023 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:48:09.612997    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-a-48d00b-5fdb8c96fc-kgm8l"
Sep 10 17:48:09.616322 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:48:09.616297    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-inf-a-48d00b\""
Sep 10 17:48:09.616431 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:48:09.616369    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-inf-a-48d00b-dockercfg-qf7tp\""
Sep 10 17:48:09.626286 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:48:09.626257    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-a-48d00b-5fdb8c96fc-kgm8l"]
Sep 10 17:48:09.675909 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:48:09.675876    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c884e60f-66ba-4308-94a0-e43e15fb1bfe-plugins-config-volume\") pod \"payload-processing-e2e-inf-a-48d00b-5fdb8c96fc-kgm8l\" (UID: \"c884e60f-66ba-4308-94a0-e43e15fb1bfe\") " pod="openshift-ingress/payload-processing-e2e-inf-a-48d00b-5fdb8c96fc-kgm8l"
Sep 10 17:48:09.676070 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:48:09.675918    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hn2w6\" (UniqueName: \"kubernetes.io/projected/c884e60f-66ba-4308-94a0-e43e15fb1bfe-kube-api-access-hn2w6\") pod \"payload-processing-e2e-inf-a-48d00b-5fdb8c96fc-kgm8l\" (UID: \"c884e60f-66ba-4308-94a0-e43e15fb1bfe\") " pod="openshift-ingress/payload-processing-e2e-inf-a-48d00b-5fdb8c96fc-kgm8l"
Sep 10 17:48:09.694913 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:48:09.694883    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-a-48d00b-5cbd8f7fcd-ngxnj"]
Sep 10 17:48:09.699278 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:48:09.699259    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-48d00b-5cbd8f7fcd-ngxnj"
Sep 10 17:48:09.718022 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:48:09.717995    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-a-48d00b-5cbd8f7fcd-ngxnj"]
Sep 10 17:48:09.777368 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:48:09.777288    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-n48z5\" (UniqueName: \"kubernetes.io/projected/e02ee03f-8b48-409f-971f-60058b269148-kube-api-access-n48z5\") pod \"payload-pre-processing-e2e-inf-a-48d00b-5cbd8f7fcd-ngxnj\" (UID: \"e02ee03f-8b48-409f-971f-60058b269148\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-a-48d00b-5cbd8f7fcd-ngxnj"
Sep 10 17:48:09.777368 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:48:09.777367    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e02ee03f-8b48-409f-971f-60058b269148-plugins-config-volume\") pod \"payload-pre-processing-e2e-inf-a-48d00b-5cbd8f7fcd-ngxnj\" (UID: \"e02ee03f-8b48-409f-971f-60058b269148\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-a-48d00b-5cbd8f7fcd-ngxnj"
Sep 10 17:48:09.777617 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:48:09.777434    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c884e60f-66ba-4308-94a0-e43e15fb1bfe-plugins-config-volume\") pod \"payload-processing-e2e-inf-a-48d00b-5fdb8c96fc-kgm8l\" (UID: \"c884e60f-66ba-4308-94a0-e43e15fb1bfe\") " pod="openshift-ingress/payload-processing-e2e-inf-a-48d00b-5fdb8c96fc-kgm8l"
Sep 10 17:48:09.777617 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:48:09.777462    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hn2w6\" (UniqueName: \"kubernetes.io/projected/c884e60f-66ba-4308-94a0-e43e15fb1bfe-kube-api-access-hn2w6\") pod \"payload-processing-e2e-inf-a-48d00b-5fdb8c96fc-kgm8l\" (UID: \"c884e60f-66ba-4308-94a0-e43e15fb1bfe\") " pod="openshift-ingress/payload-processing-e2e-inf-a-48d00b-5fdb8c96fc-kgm8l"
Sep 10 17:48:09.778914 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:48:09.778750    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c884e60f-66ba-4308-94a0-e43e15fb1bfe-plugins-config-volume\") pod \"payload-processing-e2e-inf-a-48d00b-5fdb8c96fc-kgm8l\" (UID: \"c884e60f-66ba-4308-94a0-e43e15fb1bfe\") " pod="openshift-ingress/payload-processing-e2e-inf-a-48d00b-5fdb8c96fc-kgm8l"
Sep 10 17:48:09.788060 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:48:09.788026    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hn2w6\" (UniqueName: \"kubernetes.io/projected/c884e60f-66ba-4308-94a0-e43e15fb1bfe-kube-api-access-hn2w6\") pod \"payload-processing-e2e-inf-a-48d00b-5fdb8c96fc-kgm8l\" (UID: \"c884e60f-66ba-4308-94a0-e43e15fb1bfe\") " pod="openshift-ingress/payload-processing-e2e-inf-a-48d00b-5fdb8c96fc-kgm8l"
Sep 10 17:48:09.878879 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:48:09.878791    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e02ee03f-8b48-409f-971f-60058b269148-plugins-config-volume\") pod \"payload-pre-processing-e2e-inf-a-48d00b-5cbd8f7fcd-ngxnj\" (UID: \"e02ee03f-8b48-409f-971f-60058b269148\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-a-48d00b-5cbd8f7fcd-ngxnj"
Sep 10 17:48:09.879055 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:48:09.878976    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-n48z5\" (UniqueName: \"kubernetes.io/projected/e02ee03f-8b48-409f-971f-60058b269148-kube-api-access-n48z5\") pod \"payload-pre-processing-e2e-inf-a-48d00b-5cbd8f7fcd-ngxnj\" (UID: \"e02ee03f-8b48-409f-971f-60058b269148\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-a-48d00b-5cbd8f7fcd-ngxnj"
Sep 10 17:48:09.879715 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:48:09.879689    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e02ee03f-8b48-409f-971f-60058b269148-plugins-config-volume\") pod \"payload-pre-processing-e2e-inf-a-48d00b-5cbd8f7fcd-ngxnj\" (UID: \"e02ee03f-8b48-409f-971f-60058b269148\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-a-48d00b-5cbd8f7fcd-ngxnj"
Sep 10 17:48:09.888241 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:48:09.888215    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-n48z5\" (UniqueName: \"kubernetes.io/projected/e02ee03f-8b48-409f-971f-60058b269148-kube-api-access-n48z5\") pod \"payload-pre-processing-e2e-inf-a-48d00b-5cbd8f7fcd-ngxnj\" (UID: \"e02ee03f-8b48-409f-971f-60058b269148\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-a-48d00b-5cbd8f7fcd-ngxnj"
Sep 10 17:48:09.927226 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:48:09.927193    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-a-48d00b-5fdb8c96fc-kgm8l"
Sep 10 17:48:10.018280 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:48:10.018001    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-48d00b-5cbd8f7fcd-ngxnj"
Sep 10 17:48:10.227782 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:48:10.227756    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-a-48d00b-5fdb8c96fc-kgm8l"]
Sep 10 17:48:10.228870 ip-10-0-141-180 kubenswrapper[2581]: W0910 17:48:10.228843    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc884e60f_66ba_4308_94a0_e43e15fb1bfe.slice/crio-25ce47b30857e47c39e72658ee1bc66195b45c67f6c529c5abe324407609efc4 WatchSource:0}: Error finding container 25ce47b30857e47c39e72658ee1bc66195b45c67f6c529c5abe324407609efc4: Status 404 returned error can't find the container with id 25ce47b30857e47c39e72658ee1bc66195b45c67f6c529c5abe324407609efc4
Sep 10 17:48:10.322429 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:48:10.322402    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-a-48d00b-5cbd8f7fcd-ngxnj"]
Sep 10 17:48:10.323264 ip-10-0-141-180 kubenswrapper[2581]: W0910 17:48:10.323238    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode02ee03f_8b48_409f_971f_60058b269148.slice/crio-e44d48980f20434b226d9f44f28dfb53f8262150ecd53d64eb408f86b004111c WatchSource:0}: Error finding container e44d48980f20434b226d9f44f28dfb53f8262150ecd53d64eb408f86b004111c: Status 404 returned error can't find the container with id e44d48980f20434b226d9f44f28dfb53f8262150ecd53d64eb408f86b004111c
Sep 10 17:48:10.574742 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:48:10.574651    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-48d00b-5cbd8f7fcd-ngxnj" event={"ID":"e02ee03f-8b48-409f-971f-60058b269148","Type":"ContainerStarted","Data":"a44cc7cdcd63fb6e4f3011b4977a38903e7a288ce207e1d08c2c12952fbc8ab2"}
Sep 10 17:48:10.574742 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:48:10.574719    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-48d00b-5cbd8f7fcd-ngxnj" event={"ID":"e02ee03f-8b48-409f-971f-60058b269148","Type":"ContainerStarted","Data":"e44d48980f20434b226d9f44f28dfb53f8262150ecd53d64eb408f86b004111c"}
Sep 10 17:48:10.574977 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:48:10.574887    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-48d00b-5cbd8f7fcd-ngxnj"
Sep 10 17:48:10.576520 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:48:10.576491    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-a-48d00b-5fdb8c96fc-kgm8l" event={"ID":"c884e60f-66ba-4308-94a0-e43e15fb1bfe","Type":"ContainerStarted","Data":"c85c0aff340b6badbb9ebcdec56472a2ad1503252208f7c749a9326e8b7e31b9"}
Sep 10 17:48:10.576664 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:48:10.576536    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-a-48d00b-5fdb8c96fc-kgm8l" event={"ID":"c884e60f-66ba-4308-94a0-e43e15fb1bfe","Type":"ContainerStarted","Data":"25ce47b30857e47c39e72658ee1bc66195b45c67f6c529c5abe324407609efc4"}
Sep 10 17:48:10.576718 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:48:10.576686    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-inf-a-48d00b-5fdb8c96fc-kgm8l"
Sep 10 17:48:10.634087 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:48:10.634040    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-48d00b-5cbd8f7fcd-ngxnj" podStartSLOduration=1.634023595 podStartE2EDuration="1.634023595s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 17:48:09 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 17:48:10.629931656 +0000 UTC m=+710.284005562" watchObservedRunningTime="2026-09-10 17:48:10.634023595 +0000 UTC m=+710.288097532"
Sep 10 17:48:10.673198 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:48:10.673134    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-inf-a-48d00b-5fdb8c96fc-kgm8l" podStartSLOduration=1.6731074910000001 podStartE2EDuration="1.673107491s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 17:48:09 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 17:48:10.668305866 +0000 UTC m=+710.322379790" watchObservedRunningTime="2026-09-10 17:48:10.673107491 +0000 UTC m=+710.327181397"
Sep 10 17:48:12.648194 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:48:12.648163    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-autores-51ed8d-openshift-default-74f9875b8-4fftk"]
Sep 10 17:48:12.652549 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:48:12.652524    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-autores-51ed8d-openshift-default-74f9875b8-4fftk"
Sep 10 17:48:12.655634 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:48:12.655613    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-autores-51ed8d-openshift-default-dockercfg-wb7xc\""
Sep 10 17:48:12.666995 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:48:12.666968    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-autores-51ed8d-openshift-default-74f9875b8-4fftk"]
Sep 10 17:48:12.709507 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:48:12.709475    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wrhtg\" (UniqueName: \"kubernetes.io/projected/8a2891cb-620d-4e36-984b-0cc72cc6f5b9-kube-api-access-wrhtg\") pod \"e2e-autores-51ed8d-openshift-default-74f9875b8-4fftk\" (UID: \"8a2891cb-620d-4e36-984b-0cc72cc6f5b9\") " pod="openshift-ingress/e2e-autores-51ed8d-openshift-default-74f9875b8-4fftk"
Sep 10 17:48:12.709507 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:48:12.709508    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8a2891cb-620d-4e36-984b-0cc72cc6f5b9-istio-podinfo\") pod \"e2e-autores-51ed8d-openshift-default-74f9875b8-4fftk\" (UID: \"8a2891cb-620d-4e36-984b-0cc72cc6f5b9\") " pod="openshift-ingress/e2e-autores-51ed8d-openshift-default-74f9875b8-4fftk"
Sep 10 17:48:12.709727 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:48:12.709529    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8a2891cb-620d-4e36-984b-0cc72cc6f5b9-istio-data\") pod \"e2e-autores-51ed8d-openshift-default-74f9875b8-4fftk\" (UID: \"8a2891cb-620d-4e36-984b-0cc72cc6f5b9\") " pod="openshift-ingress/e2e-autores-51ed8d-openshift-default-74f9875b8-4fftk"
Sep 10 17:48:12.709727 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:48:12.709570    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8a2891cb-620d-4e36-984b-0cc72cc6f5b9-istio-token\") pod \"e2e-autores-51ed8d-openshift-default-74f9875b8-4fftk\" (UID: \"8a2891cb-620d-4e36-984b-0cc72cc6f5b9\") " pod="openshift-ingress/e2e-autores-51ed8d-openshift-default-74f9875b8-4fftk"
Sep 10 17:48:12.709727 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:48:12.709617    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8a2891cb-620d-4e36-984b-0cc72cc6f5b9-istiod-ca-cert\") pod \"e2e-autores-51ed8d-openshift-default-74f9875b8-4fftk\" (UID: \"8a2891cb-620d-4e36-984b-0cc72cc6f5b9\") " pod="openshift-ingress/e2e-autores-51ed8d-openshift-default-74f9875b8-4fftk"
Sep 10 17:48:12.709727 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:48:12.709676    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8a2891cb-620d-4e36-984b-0cc72cc6f5b9-workload-socket\") pod \"e2e-autores-51ed8d-openshift-default-74f9875b8-4fftk\" (UID: \"8a2891cb-620d-4e36-984b-0cc72cc6f5b9\") " pod="openshift-ingress/e2e-autores-51ed8d-openshift-default-74f9875b8-4fftk"
Sep 10 17:48:12.709856 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:48:12.709755    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8a2891cb-620d-4e36-984b-0cc72cc6f5b9-istio-envoy\") pod \"e2e-autores-51ed8d-openshift-default-74f9875b8-4fftk\" (UID: \"8a2891cb-620d-4e36-984b-0cc72cc6f5b9\") " pod="openshift-ingress/e2e-autores-51ed8d-openshift-default-74f9875b8-4fftk"
Sep 10 17:48:12.709856 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:48:12.709839    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8a2891cb-620d-4e36-984b-0cc72cc6f5b9-workload-certs\") pod \"e2e-autores-51ed8d-openshift-default-74f9875b8-4fftk\" (UID: \"8a2891cb-620d-4e36-984b-0cc72cc6f5b9\") " pod="openshift-ingress/e2e-autores-51ed8d-openshift-default-74f9875b8-4fftk"
Sep 10 17:48:12.709915 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:48:12.709863    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8a2891cb-620d-4e36-984b-0cc72cc6f5b9-credential-socket\") pod \"e2e-autores-51ed8d-openshift-default-74f9875b8-4fftk\" (UID: \"8a2891cb-620d-4e36-984b-0cc72cc6f5b9\") " pod="openshift-ingress/e2e-autores-51ed8d-openshift-default-74f9875b8-4fftk"
Sep 10 17:48:12.810986 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:48:12.810945    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8a2891cb-620d-4e36-984b-0cc72cc6f5b9-workload-certs\") pod \"e2e-autores-51ed8d-openshift-default-74f9875b8-4fftk\" (UID: \"8a2891cb-620d-4e36-984b-0cc72cc6f5b9\") " pod="openshift-ingress/e2e-autores-51ed8d-openshift-default-74f9875b8-4fftk"
Sep 10 17:48:12.810986 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:48:12.810988    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8a2891cb-620d-4e36-984b-0cc72cc6f5b9-credential-socket\") pod \"e2e-autores-51ed8d-openshift-default-74f9875b8-4fftk\" (UID: \"8a2891cb-620d-4e36-984b-0cc72cc6f5b9\") " pod="openshift-ingress/e2e-autores-51ed8d-openshift-default-74f9875b8-4fftk"
Sep 10 17:48:12.811205 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:48:12.811023    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wrhtg\" (UniqueName: \"kubernetes.io/projected/8a2891cb-620d-4e36-984b-0cc72cc6f5b9-kube-api-access-wrhtg\") pod \"e2e-autores-51ed8d-openshift-default-74f9875b8-4fftk\" (UID: \"8a2891cb-620d-4e36-984b-0cc72cc6f5b9\") " pod="openshift-ingress/e2e-autores-51ed8d-openshift-default-74f9875b8-4fftk"
Sep 10 17:48:12.811205 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:48:12.811043    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8a2891cb-620d-4e36-984b-0cc72cc6f5b9-istio-podinfo\") pod \"e2e-autores-51ed8d-openshift-default-74f9875b8-4fftk\" (UID: \"8a2891cb-620d-4e36-984b-0cc72cc6f5b9\") " pod="openshift-ingress/e2e-autores-51ed8d-openshift-default-74f9875b8-4fftk"
Sep 10 17:48:12.811205 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:48:12.811061    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8a2891cb-620d-4e36-984b-0cc72cc6f5b9-istio-data\") pod \"e2e-autores-51ed8d-openshift-default-74f9875b8-4fftk\" (UID: \"8a2891cb-620d-4e36-984b-0cc72cc6f5b9\") " pod="openshift-ingress/e2e-autores-51ed8d-openshift-default-74f9875b8-4fftk"
Sep 10 17:48:12.811205 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:48:12.811079    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8a2891cb-620d-4e36-984b-0cc72cc6f5b9-istio-token\") pod \"e2e-autores-51ed8d-openshift-default-74f9875b8-4fftk\" (UID: \"8a2891cb-620d-4e36-984b-0cc72cc6f5b9\") " pod="openshift-ingress/e2e-autores-51ed8d-openshift-default-74f9875b8-4fftk"
Sep 10 17:48:12.811205 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:48:12.811100    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8a2891cb-620d-4e36-984b-0cc72cc6f5b9-istiod-ca-cert\") pod \"e2e-autores-51ed8d-openshift-default-74f9875b8-4fftk\" (UID: \"8a2891cb-620d-4e36-984b-0cc72cc6f5b9\") " pod="openshift-ingress/e2e-autores-51ed8d-openshift-default-74f9875b8-4fftk"
Sep 10 17:48:12.811205 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:48:12.811138    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8a2891cb-620d-4e36-984b-0cc72cc6f5b9-workload-socket\") pod \"e2e-autores-51ed8d-openshift-default-74f9875b8-4fftk\" (UID: \"8a2891cb-620d-4e36-984b-0cc72cc6f5b9\") " pod="openshift-ingress/e2e-autores-51ed8d-openshift-default-74f9875b8-4fftk"
Sep 10 17:48:12.811205 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:48:12.811201    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8a2891cb-620d-4e36-984b-0cc72cc6f5b9-istio-envoy\") pod \"e2e-autores-51ed8d-openshift-default-74f9875b8-4fftk\" (UID: \"8a2891cb-620d-4e36-984b-0cc72cc6f5b9\") " pod="openshift-ingress/e2e-autores-51ed8d-openshift-default-74f9875b8-4fftk"
Sep 10 17:48:12.811490 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:48:12.811397    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8a2891cb-620d-4e36-984b-0cc72cc6f5b9-workload-certs\") pod \"e2e-autores-51ed8d-openshift-default-74f9875b8-4fftk\" (UID: \"8a2891cb-620d-4e36-984b-0cc72cc6f5b9\") " pod="openshift-ingress/e2e-autores-51ed8d-openshift-default-74f9875b8-4fftk"
Sep 10 17:48:12.811801 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:48:12.811761    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8a2891cb-620d-4e36-984b-0cc72cc6f5b9-istiod-ca-cert\") pod \"e2e-autores-51ed8d-openshift-default-74f9875b8-4fftk\" (UID: \"8a2891cb-620d-4e36-984b-0cc72cc6f5b9\") " pod="openshift-ingress/e2e-autores-51ed8d-openshift-default-74f9875b8-4fftk"
Sep 10 17:48:12.811907 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:48:12.811887    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8a2891cb-620d-4e36-984b-0cc72cc6f5b9-credential-socket\") pod \"e2e-autores-51ed8d-openshift-default-74f9875b8-4fftk\" (UID: \"8a2891cb-620d-4e36-984b-0cc72cc6f5b9\") " pod="openshift-ingress/e2e-autores-51ed8d-openshift-default-74f9875b8-4fftk"
Sep 10 17:48:12.811984 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:48:12.811909    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8a2891cb-620d-4e36-984b-0cc72cc6f5b9-istio-data\") pod \"e2e-autores-51ed8d-openshift-default-74f9875b8-4fftk\" (UID: \"8a2891cb-620d-4e36-984b-0cc72cc6f5b9\") " pod="openshift-ingress/e2e-autores-51ed8d-openshift-default-74f9875b8-4fftk"
Sep 10 17:48:12.812089 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:48:12.812063    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8a2891cb-620d-4e36-984b-0cc72cc6f5b9-workload-socket\") pod \"e2e-autores-51ed8d-openshift-default-74f9875b8-4fftk\" (UID: \"8a2891cb-620d-4e36-984b-0cc72cc6f5b9\") " pod="openshift-ingress/e2e-autores-51ed8d-openshift-default-74f9875b8-4fftk"
Sep 10 17:48:12.813793 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:48:12.813767    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8a2891cb-620d-4e36-984b-0cc72cc6f5b9-istio-podinfo\") pod \"e2e-autores-51ed8d-openshift-default-74f9875b8-4fftk\" (UID: \"8a2891cb-620d-4e36-984b-0cc72cc6f5b9\") " pod="openshift-ingress/e2e-autores-51ed8d-openshift-default-74f9875b8-4fftk"
Sep 10 17:48:12.813892 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:48:12.813876    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8a2891cb-620d-4e36-984b-0cc72cc6f5b9-istio-envoy\") pod \"e2e-autores-51ed8d-openshift-default-74f9875b8-4fftk\" (UID: \"8a2891cb-620d-4e36-984b-0cc72cc6f5b9\") " pod="openshift-ingress/e2e-autores-51ed8d-openshift-default-74f9875b8-4fftk"
Sep 10 17:48:12.821910 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:48:12.821871    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8a2891cb-620d-4e36-984b-0cc72cc6f5b9-istio-token\") pod \"e2e-autores-51ed8d-openshift-default-74f9875b8-4fftk\" (UID: \"8a2891cb-620d-4e36-984b-0cc72cc6f5b9\") " pod="openshift-ingress/e2e-autores-51ed8d-openshift-default-74f9875b8-4fftk"
Sep 10 17:48:12.822335 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:48:12.822310    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wrhtg\" (UniqueName: \"kubernetes.io/projected/8a2891cb-620d-4e36-984b-0cc72cc6f5b9-kube-api-access-wrhtg\") pod \"e2e-autores-51ed8d-openshift-default-74f9875b8-4fftk\" (UID: \"8a2891cb-620d-4e36-984b-0cc72cc6f5b9\") " pod="openshift-ingress/e2e-autores-51ed8d-openshift-default-74f9875b8-4fftk"
Sep 10 17:48:12.963198 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:48:12.963171    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-autores-51ed8d-openshift-default-74f9875b8-4fftk"
Sep 10 17:48:13.249439 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:48:13.249413    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-autores-51ed8d-openshift-default-74f9875b8-4fftk"]
Sep 10 17:48:13.250951 ip-10-0-141-180 kubenswrapper[2581]: W0910 17:48:13.250925    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8a2891cb_620d_4e36_984b_0cc72cc6f5b9.slice/crio-595b7ecd4ccd05c7d1faba07533aa483a4f2339de37a74db73a9f95041590212 WatchSource:0}: Error finding container 595b7ecd4ccd05c7d1faba07533aa483a4f2339de37a74db73a9f95041590212: Status 404 returned error can't find the container with id 595b7ecd4ccd05c7d1faba07533aa483a4f2339de37a74db73a9f95041590212
Sep 10 17:48:13.253659 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:48:13.253613    2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 10 17:48:13.253751 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:48:13.253693    2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 10 17:48:13.253751 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:48:13.253723    2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 10 17:48:13.599714 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:48:13.599593    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-autores-51ed8d-openshift-default-74f9875b8-4fftk" event={"ID":"8a2891cb-620d-4e36-984b-0cc72cc6f5b9","Type":"ContainerStarted","Data":"40882ffc414bf233886f1847bd73ca1e47ef3701710d891111d02bd939ec63e0"}
Sep 10 17:48:13.599714 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:48:13.599668    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-autores-51ed8d-openshift-default-74f9875b8-4fftk" event={"ID":"8a2891cb-620d-4e36-984b-0cc72cc6f5b9","Type":"ContainerStarted","Data":"595b7ecd4ccd05c7d1faba07533aa483a4f2339de37a74db73a9f95041590212"}
Sep 10 17:48:13.634391 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:48:13.634327    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-autores-51ed8d-openshift-default-74f9875b8-4fftk" podStartSLOduration=1.634306473 podStartE2EDuration="1.634306473s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 17:48:12 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 17:48:13.630180013 +0000 UTC m=+713.284253943" watchObservedRunningTime="2026-09-10 17:48:13.634306473 +0000 UTC m=+713.288380378"
Sep 10 17:48:13.963582 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:48:13.963552    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-autores-51ed8d-openshift-default-74f9875b8-4fftk"
Sep 10 17:48:14.970226 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:48:14.970192    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-autores-51ed8d-openshift-default-74f9875b8-4fftk"
Sep 10 17:48:15.614009 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:48:15.613974    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-autores-51ed8d-openshift-default-74f9875b8-4fftk"
Sep 10 17:48:15.614850 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:48:15.614825    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-autores-51ed8d-openshift-default-74f9875b8-4fftk"
Sep 10 17:48:17.576263 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:48:17.576227    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-autores-51ed8d-7d4f74f6ff-m2dh4"]
Sep 10 17:48:17.592889 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:48:17.592860    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-autores-51ed8d-7d4f74f6ff-m2dh4"]
Sep 10 17:48:17.593178 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:48:17.593162    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-autores-51ed8d-7d4f74f6ff-m2dh4"
Sep 10 17:48:17.604332 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:48:17.603023    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-autores-51ed8d-dockercfg-sdqfk\""
Sep 10 17:48:17.604332 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:48:17.603430    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-autores-51ed8d\""
Sep 10 17:48:17.665037 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:48:17.664912    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/bf92bad5-3350-4228-bf29-c83360577fc2-plugins-config-volume\") pod \"payload-processing-e2e-autores-51ed8d-7d4f74f6ff-m2dh4\" (UID: \"bf92bad5-3350-4228-bf29-c83360577fc2\") " pod="openshift-ingress/payload-processing-e2e-autores-51ed8d-7d4f74f6ff-m2dh4"
Sep 10 17:48:17.665538 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:48:17.665516    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bhgk5\" (UniqueName: \"kubernetes.io/projected/bf92bad5-3350-4228-bf29-c83360577fc2-kube-api-access-bhgk5\") pod \"payload-processing-e2e-autores-51ed8d-7d4f74f6ff-m2dh4\" (UID: \"bf92bad5-3350-4228-bf29-c83360577fc2\") " pod="openshift-ingress/payload-processing-e2e-autores-51ed8d-7d4f74f6ff-m2dh4"
Sep 10 17:48:17.766974 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:48:17.766939    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/bf92bad5-3350-4228-bf29-c83360577fc2-plugins-config-volume\") pod \"payload-processing-e2e-autores-51ed8d-7d4f74f6ff-m2dh4\" (UID: \"bf92bad5-3350-4228-bf29-c83360577fc2\") " pod="openshift-ingress/payload-processing-e2e-autores-51ed8d-7d4f74f6ff-m2dh4"
Sep 10 17:48:17.767136 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:48:17.767004    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bhgk5\" (UniqueName: \"kubernetes.io/projected/bf92bad5-3350-4228-bf29-c83360577fc2-kube-api-access-bhgk5\") pod \"payload-processing-e2e-autores-51ed8d-7d4f74f6ff-m2dh4\" (UID: \"bf92bad5-3350-4228-bf29-c83360577fc2\") " pod="openshift-ingress/payload-processing-e2e-autores-51ed8d-7d4f74f6ff-m2dh4"
Sep 10 17:48:17.767632 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:48:17.767610    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/bf92bad5-3350-4228-bf29-c83360577fc2-plugins-config-volume\") pod \"payload-processing-e2e-autores-51ed8d-7d4f74f6ff-m2dh4\" (UID: \"bf92bad5-3350-4228-bf29-c83360577fc2\") " pod="openshift-ingress/payload-processing-e2e-autores-51ed8d-7d4f74f6ff-m2dh4"
Sep 10 17:48:17.784138 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:48:17.784110    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bhgk5\" (UniqueName: \"kubernetes.io/projected/bf92bad5-3350-4228-bf29-c83360577fc2-kube-api-access-bhgk5\") pod \"payload-processing-e2e-autores-51ed8d-7d4f74f6ff-m2dh4\" (UID: \"bf92bad5-3350-4228-bf29-c83360577fc2\") " pod="openshift-ingress/payload-processing-e2e-autores-51ed8d-7d4f74f6ff-m2dh4"
Sep 10 17:48:17.914762 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:48:17.914670    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-autores-51ed8d-7d4f74f6ff-m2dh4"
Sep 10 17:48:18.196156 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:48:18.194770    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-autores-51ed8d-7d4f74f6ff-m2dh4"]
Sep 10 17:48:18.639052 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:48:18.639008    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-autores-51ed8d-7d4f74f6ff-m2dh4" event={"ID":"bf92bad5-3350-4228-bf29-c83360577fc2","Type":"ContainerStarted","Data":"4ca1c9cd982293601eb4f985bbb797145166fe0043741f66dea6cdcde999fb07"}
Sep 10 17:48:18.639523 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:48:18.639083    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-autores-51ed8d-7d4f74f6ff-m2dh4" event={"ID":"bf92bad5-3350-4228-bf29-c83360577fc2","Type":"ContainerStarted","Data":"3ea5d4ef550d3a5e6d522130ac606a2d443f5f08ce39487f03373a02d66069ef"}
Sep 10 17:48:18.639523 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:48:18.639195    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-autores-51ed8d-7d4f74f6ff-m2dh4"
Sep 10 17:48:18.662066 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:48:18.662011    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-autores-51ed8d-7d4f74f6ff-m2dh4" podStartSLOduration=1.661997199 podStartE2EDuration="1.661997199s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 17:48:17 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 17:48:18.659236167 +0000 UTC m=+718.313310090" watchObservedRunningTime="2026-09-10 17:48:18.661997199 +0000 UTC m=+718.316071104"
Sep 10 17:48:21.139217 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:48:21.139172    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-4601bf83-fd5cdb559-rfp9s"]
Sep 10 17:48:21.143369 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:48:21.143334    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-4601bf83-fd5cdb559-rfp9s"
Sep 10 17:48:21.146141 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:48:21.146119    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-ait-4601bf83\""
Sep 10 17:48:21.146288 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:48:21.146123    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-ait-4601bf83-dockercfg-zjjmx\""
Sep 10 17:48:21.153097 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:48:21.153073    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-4601bf83-fd5cdb559-rfp9s"]
Sep 10 17:48:21.206102 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:48:21.206071    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/5b4452ff-d15f-493c-b9e6-9704144bf575-plugins-config-volume\") pod \"payload-processing-e2e-ait-4601bf83-fd5cdb559-rfp9s\" (UID: \"5b4452ff-d15f-493c-b9e6-9704144bf575\") " pod="openshift-ingress/payload-processing-e2e-ait-4601bf83-fd5cdb559-rfp9s"
Sep 10 17:48:21.206237 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:48:21.206180    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kz8v8\" (UniqueName: \"kubernetes.io/projected/5b4452ff-d15f-493c-b9e6-9704144bf575-kube-api-access-kz8v8\") pod \"payload-processing-e2e-ait-4601bf83-fd5cdb559-rfp9s\" (UID: \"5b4452ff-d15f-493c-b9e6-9704144bf575\") " pod="openshift-ingress/payload-processing-e2e-ait-4601bf83-fd5cdb559-rfp9s"
Sep 10 17:48:21.307394 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:48:21.307352    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kz8v8\" (UniqueName: \"kubernetes.io/projected/5b4452ff-d15f-493c-b9e6-9704144bf575-kube-api-access-kz8v8\") pod \"payload-processing-e2e-ait-4601bf83-fd5cdb559-rfp9s\" (UID: \"5b4452ff-d15f-493c-b9e6-9704144bf575\") " pod="openshift-ingress/payload-processing-e2e-ait-4601bf83-fd5cdb559-rfp9s"
Sep 10 17:48:21.307558 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:48:21.307439    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/5b4452ff-d15f-493c-b9e6-9704144bf575-plugins-config-volume\") pod \"payload-processing-e2e-ait-4601bf83-fd5cdb559-rfp9s\" (UID: \"5b4452ff-d15f-493c-b9e6-9704144bf575\") " pod="openshift-ingress/payload-processing-e2e-ait-4601bf83-fd5cdb559-rfp9s"
Sep 10 17:48:21.308789 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:48:21.308768    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/5b4452ff-d15f-493c-b9e6-9704144bf575-plugins-config-volume\") pod \"payload-processing-e2e-ait-4601bf83-fd5cdb559-rfp9s\" (UID: \"5b4452ff-d15f-493c-b9e6-9704144bf575\") " pod="openshift-ingress/payload-processing-e2e-ait-4601bf83-fd5cdb559-rfp9s"
Sep 10 17:48:21.316253 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:48:21.316232    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kz8v8\" (UniqueName: \"kubernetes.io/projected/5b4452ff-d15f-493c-b9e6-9704144bf575-kube-api-access-kz8v8\") pod \"payload-processing-e2e-ait-4601bf83-fd5cdb559-rfp9s\" (UID: \"5b4452ff-d15f-493c-b9e6-9704144bf575\") " pod="openshift-ingress/payload-processing-e2e-ait-4601bf83-fd5cdb559-rfp9s"
Sep 10 17:48:21.456271 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:48:21.456186    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-4601bf83-fd5cdb559-rfp9s"
Sep 10 17:48:21.598193 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:48:21.596780    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-48d00b-5cbd8f7fcd-ngxnj"
Sep 10 17:48:21.598193 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:48:21.596862    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-inf-a-48d00b-5fdb8c96fc-kgm8l"
Sep 10 17:48:21.776710 ip-10-0-141-180 kubenswrapper[2581]: W0910 17:48:21.775767    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5b4452ff_d15f_493c_b9e6_9704144bf575.slice/crio-6b525e6f582b6e1370d184069010def58626a0583b056399b1cc8c81357474e5 WatchSource:0}: Error finding container 6b525e6f582b6e1370d184069010def58626a0583b056399b1cc8c81357474e5: Status 404 returned error can't find the container with id 6b525e6f582b6e1370d184069010def58626a0583b056399b1cc8c81357474e5
Sep 10 17:48:21.778382 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:48:21.778350    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-4601bf83-fd5cdb559-rfp9s"]
Sep 10 17:48:22.669488 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:48:22.669442    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-4601bf83-fd5cdb559-rfp9s" event={"ID":"5b4452ff-d15f-493c-b9e6-9704144bf575","Type":"ContainerStarted","Data":"ee061dc611eb3e957ed326b457e9471453a1e6a54edad32558ef39398e3683e0"}
Sep 10 17:48:22.669980 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:48:22.669569    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-ait-4601bf83-fd5cdb559-rfp9s"
Sep 10 17:48:22.669980 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:48:22.669593    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-4601bf83-fd5cdb559-rfp9s" event={"ID":"5b4452ff-d15f-493c-b9e6-9704144bf575","Type":"ContainerStarted","Data":"6b525e6f582b6e1370d184069010def58626a0583b056399b1cc8c81357474e5"}
Sep 10 17:48:22.699604 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:48:22.699550    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-ait-4601bf83-fd5cdb559-rfp9s" podStartSLOduration=1.6995335630000001 podStartE2EDuration="1.699533563s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 17:48:21 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 17:48:22.697896103 +0000 UTC m=+722.351970030" watchObservedRunningTime="2026-09-10 17:48:22.699533563 +0000 UTC m=+722.353607500"
Sep 10 17:48:23.593841 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:48:23.593804    2581 patch_prober.go:28] interesting pod/e2e-inf-a-48d00b-openshift-default-6cc74dc546-kjh62 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.43:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Sep 10 17:48:23.594029 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:48:23.593862    2581 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-inf-a-48d00b-openshift-default-6cc74dc546-kjh62" podUID="20ec55f7-fa46-4372-bf1b-791406c99a10" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.43:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Sep 10 17:48:25.699711 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:48:25.699670    2581 generic.go:356] "Generic (PLEG): container finished" podID="20ec55f7-fa46-4372-bf1b-791406c99a10" containerID="a992fe6c27c2c21e201c10a259fbcbcf8850ea7283b2cdbc34a9974033bf49a9" exitCode=0
Sep 10 17:48:25.700146 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:48:25.699730    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-48d00b-openshift-default-6cc74dc546-kjh62" event={"ID":"20ec55f7-fa46-4372-bf1b-791406c99a10","Type":"ContainerDied","Data":"a992fe6c27c2c21e201c10a259fbcbcf8850ea7283b2cdbc34a9974033bf49a9"}
Sep 10 17:48:25.700146 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:48:25.700144    2581 scope.go:117] "RemoveContainer" containerID="a992fe6c27c2c21e201c10a259fbcbcf8850ea7283b2cdbc34a9974033bf49a9"
Sep 10 17:48:25.701286 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:48:25.701260    2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 10 17:48:25.701387 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:48:25.701328    2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 10 17:48:25.701425 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:48:25.701415    2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 10 17:48:26.714241 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:48:26.714200    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-48d00b-openshift-default-6cc74dc546-kjh62" event={"ID":"20ec55f7-fa46-4372-bf1b-791406c99a10","Type":"ContainerStarted","Data":"32b2827a6a7383e227807bbec57841e55b9ce266bdf59eb614f1d72536568ee7"}
Sep 10 17:48:26.729490 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:48:26.729457    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-a-48d00b-openshift-default-6cc74dc546-kjh62"
Sep 10 17:48:26.729711 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:48:26.729686    2581 patch_prober.go:28] interesting pod/e2e-inf-a-48d00b-openshift-default-6cc74dc546-kjh62 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.43:15021/healthz/ready\": dial tcp 10.133.0.43:15021: connect: connection refused" start-of-body=
Sep 10 17:48:26.729764 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:48:26.729739    2581 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-48d00b-openshift-default-6cc74dc546-kjh62" podUID="20ec55f7-fa46-4372-bf1b-791406c99a10" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.43:15021/healthz/ready\": dial tcp 10.133.0.43:15021: connect: connection refused"
Sep 10 17:48:27.729708 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:48:27.729668    2581 patch_prober.go:28] interesting pod/e2e-inf-a-48d00b-openshift-default-6cc74dc546-kjh62 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.43:15021/healthz/ready\": dial tcp 10.133.0.43:15021: connect: connection refused" start-of-body=
Sep 10 17:48:27.730137 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:48:27.729728    2581 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-48d00b-openshift-default-6cc74dc546-kjh62" podUID="20ec55f7-fa46-4372-bf1b-791406c99a10" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.43:15021/healthz/ready\": dial tcp 10.133.0.43:15021: connect: connection refused"
Sep 10 17:48:28.729600 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:48:28.729557    2581 patch_prober.go:28] interesting pod/e2e-inf-a-48d00b-openshift-default-6cc74dc546-kjh62 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.43:15021/healthz/ready\": dial tcp 10.133.0.43:15021: connect: connection refused" start-of-body=
Sep 10 17:48:28.729806 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:48:28.729622    2581 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-48d00b-openshift-default-6cc74dc546-kjh62" podUID="20ec55f7-fa46-4372-bf1b-791406c99a10" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.43:15021/healthz/ready\": dial tcp 10.133.0.43:15021: connect: connection refused"
Sep 10 17:48:29.648565 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:48:29.648472    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-autores-51ed8d-7d4f74f6ff-m2dh4"
Sep 10 17:48:29.729968 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:48:29.729927    2581 patch_prober.go:28] interesting pod/e2e-inf-a-48d00b-openshift-default-6cc74dc546-kjh62 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.43:15021/healthz/ready\": dial tcp 10.133.0.43:15021: connect: connection refused" start-of-body=
Sep 10 17:48:29.730401 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:48:29.730000    2581 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-48d00b-openshift-default-6cc74dc546-kjh62" podUID="20ec55f7-fa46-4372-bf1b-791406c99a10" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.43:15021/healthz/ready\": dial tcp 10.133.0.43:15021: connect: connection refused"
Sep 10 17:48:30.729349 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:48:30.729303    2581 patch_prober.go:28] interesting pod/e2e-inf-a-48d00b-openshift-default-6cc74dc546-kjh62 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.43:15021/healthz/ready\": dial tcp 10.133.0.43:15021: connect: connection refused" start-of-body=
Sep 10 17:48:30.729526 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:48:30.729376    2581 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-48d00b-openshift-default-6cc74dc546-kjh62" podUID="20ec55f7-fa46-4372-bf1b-791406c99a10" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.43:15021/healthz/ready\": dial tcp 10.133.0.43:15021: connect: connection refused"
Sep 10 17:48:31.655809 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:48:31.655765    2581 patch_prober.go:28] interesting pod/e2e-autores-51ed8d-openshift-default-74f9875b8-4fftk container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.46:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Sep 10 17:48:31.656209 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:48:31.655828    2581 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-autores-51ed8d-openshift-default-74f9875b8-4fftk" podUID="8a2891cb-620d-4e36-984b-0cc72cc6f5b9" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.46:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Sep 10 17:48:31.729980 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:48:31.729942    2581 patch_prober.go:28] interesting pod/e2e-inf-a-48d00b-openshift-default-6cc74dc546-kjh62 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.43:15021/healthz/ready\": dial tcp 10.133.0.43:15021: connect: connection refused" start-of-body=
Sep 10 17:48:31.730151 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:48:31.730001    2581 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-48d00b-openshift-default-6cc74dc546-kjh62" podUID="20ec55f7-fa46-4372-bf1b-791406c99a10" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.43:15021/healthz/ready\": dial tcp 10.133.0.43:15021: connect: connection refused"
Sep 10 17:48:32.733675 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:48:32.733628    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-a-48d00b-openshift-default-6cc74dc546-kjh62"
Sep 10 17:48:32.734160 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:48:32.734133    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-a-48d00b-openshift-default-6cc74dc546-kjh62"
Sep 10 17:48:32.734861 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:48:32.734840    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-a-48d00b-openshift-default-6cc74dc546-kjh62"
Sep 10 17:48:33.445217 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:48:33.445183    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-inf-b-783a5e-openshift-default-5dd94f65d-bxhrc"]
Sep 10 17:48:33.448143 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:48:33.448122    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-783a5e-openshift-default-5dd94f65d-bxhrc"
Sep 10 17:48:33.451076 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:48:33.451054    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-inf-b-783a5e-openshift-default-dockercfg-9km46\""
Sep 10 17:48:33.467669 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:48:33.467627    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-b-783a5e-openshift-default-5dd94f65d-bxhrc"]
Sep 10 17:48:33.539991 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:48:33.539947    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a628ea2c-8ec6-4d36-99b5-db4f7927de9d-workload-certs\") pod \"e2e-inf-b-783a5e-openshift-default-5dd94f65d-bxhrc\" (UID: \"a628ea2c-8ec6-4d36-99b5-db4f7927de9d\") " pod="openshift-ingress/e2e-inf-b-783a5e-openshift-default-5dd94f65d-bxhrc"
Sep 10 17:48:33.540172 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:48:33.540011    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a628ea2c-8ec6-4d36-99b5-db4f7927de9d-istiod-ca-cert\") pod \"e2e-inf-b-783a5e-openshift-default-5dd94f65d-bxhrc\" (UID: \"a628ea2c-8ec6-4d36-99b5-db4f7927de9d\") " pod="openshift-ingress/e2e-inf-b-783a5e-openshift-default-5dd94f65d-bxhrc"
Sep 10 17:48:33.540172 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:48:33.540068    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a628ea2c-8ec6-4d36-99b5-db4f7927de9d-istio-token\") pod \"e2e-inf-b-783a5e-openshift-default-5dd94f65d-bxhrc\" (UID: \"a628ea2c-8ec6-4d36-99b5-db4f7927de9d\") " pod="openshift-ingress/e2e-inf-b-783a5e-openshift-default-5dd94f65d-bxhrc"
Sep 10 17:48:33.540172 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:48:33.540116    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5qxt4\" (UniqueName: \"kubernetes.io/projected/a628ea2c-8ec6-4d36-99b5-db4f7927de9d-kube-api-access-5qxt4\") pod \"e2e-inf-b-783a5e-openshift-default-5dd94f65d-bxhrc\" (UID: \"a628ea2c-8ec6-4d36-99b5-db4f7927de9d\") " pod="openshift-ingress/e2e-inf-b-783a5e-openshift-default-5dd94f65d-bxhrc"
Sep 10 17:48:33.540172 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:48:33.540134    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a628ea2c-8ec6-4d36-99b5-db4f7927de9d-istio-podinfo\") pod \"e2e-inf-b-783a5e-openshift-default-5dd94f65d-bxhrc\" (UID: \"a628ea2c-8ec6-4d36-99b5-db4f7927de9d\") " pod="openshift-ingress/e2e-inf-b-783a5e-openshift-default-5dd94f65d-bxhrc"
Sep 10 17:48:33.540172 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:48:33.540153    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a628ea2c-8ec6-4d36-99b5-db4f7927de9d-workload-socket\") pod \"e2e-inf-b-783a5e-openshift-default-5dd94f65d-bxhrc\" (UID: \"a628ea2c-8ec6-4d36-99b5-db4f7927de9d\") " pod="openshift-ingress/e2e-inf-b-783a5e-openshift-default-5dd94f65d-bxhrc"
Sep 10 17:48:33.540440 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:48:33.540217    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a628ea2c-8ec6-4d36-99b5-db4f7927de9d-credential-socket\") pod \"e2e-inf-b-783a5e-openshift-default-5dd94f65d-bxhrc\" (UID: \"a628ea2c-8ec6-4d36-99b5-db4f7927de9d\") " pod="openshift-ingress/e2e-inf-b-783a5e-openshift-default-5dd94f65d-bxhrc"
Sep 10 17:48:33.540440 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:48:33.540281    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a628ea2c-8ec6-4d36-99b5-db4f7927de9d-istio-data\") pod \"e2e-inf-b-783a5e-openshift-default-5dd94f65d-bxhrc\" (UID: \"a628ea2c-8ec6-4d36-99b5-db4f7927de9d\") " pod="openshift-ingress/e2e-inf-b-783a5e-openshift-default-5dd94f65d-bxhrc"
Sep 10 17:48:33.540440 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:48:33.540310    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a628ea2c-8ec6-4d36-99b5-db4f7927de9d-istio-envoy\") pod \"e2e-inf-b-783a5e-openshift-default-5dd94f65d-bxhrc\" (UID: \"a628ea2c-8ec6-4d36-99b5-db4f7927de9d\") " pod="openshift-ingress/e2e-inf-b-783a5e-openshift-default-5dd94f65d-bxhrc"
Sep 10 17:48:33.641568 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:48:33.641531    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a628ea2c-8ec6-4d36-99b5-db4f7927de9d-istio-token\") pod \"e2e-inf-b-783a5e-openshift-default-5dd94f65d-bxhrc\" (UID: \"a628ea2c-8ec6-4d36-99b5-db4f7927de9d\") " pod="openshift-ingress/e2e-inf-b-783a5e-openshift-default-5dd94f65d-bxhrc"
Sep 10 17:48:33.641785 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:48:33.641592    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5qxt4\" (UniqueName: \"kubernetes.io/projected/a628ea2c-8ec6-4d36-99b5-db4f7927de9d-kube-api-access-5qxt4\") pod \"e2e-inf-b-783a5e-openshift-default-5dd94f65d-bxhrc\" (UID: \"a628ea2c-8ec6-4d36-99b5-db4f7927de9d\") " pod="openshift-ingress/e2e-inf-b-783a5e-openshift-default-5dd94f65d-bxhrc"
Sep 10 17:48:33.641785 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:48:33.641618    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a628ea2c-8ec6-4d36-99b5-db4f7927de9d-istio-podinfo\") pod \"e2e-inf-b-783a5e-openshift-default-5dd94f65d-bxhrc\" (UID: \"a628ea2c-8ec6-4d36-99b5-db4f7927de9d\") " pod="openshift-ingress/e2e-inf-b-783a5e-openshift-default-5dd94f65d-bxhrc"
Sep 10 17:48:33.641785 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:48:33.641670    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a628ea2c-8ec6-4d36-99b5-db4f7927de9d-workload-socket\") pod \"e2e-inf-b-783a5e-openshift-default-5dd94f65d-bxhrc\" (UID: \"a628ea2c-8ec6-4d36-99b5-db4f7927de9d\") " pod="openshift-ingress/e2e-inf-b-783a5e-openshift-default-5dd94f65d-bxhrc"
Sep 10 17:48:33.641967 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:48:33.641791    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a628ea2c-8ec6-4d36-99b5-db4f7927de9d-credential-socket\") pod \"e2e-inf-b-783a5e-openshift-default-5dd94f65d-bxhrc\" (UID: \"a628ea2c-8ec6-4d36-99b5-db4f7927de9d\") " pod="openshift-ingress/e2e-inf-b-783a5e-openshift-default-5dd94f65d-bxhrc"
Sep 10 17:48:33.641967 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:48:33.641833    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a628ea2c-8ec6-4d36-99b5-db4f7927de9d-istio-data\") pod \"e2e-inf-b-783a5e-openshift-default-5dd94f65d-bxhrc\" (UID: \"a628ea2c-8ec6-4d36-99b5-db4f7927de9d\") " pod="openshift-ingress/e2e-inf-b-783a5e-openshift-default-5dd94f65d-bxhrc"
Sep 10 17:48:33.641967 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:48:33.641861    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a628ea2c-8ec6-4d36-99b5-db4f7927de9d-istio-envoy\") pod \"e2e-inf-b-783a5e-openshift-default-5dd94f65d-bxhrc\" (UID: \"a628ea2c-8ec6-4d36-99b5-db4f7927de9d\") " pod="openshift-ingress/e2e-inf-b-783a5e-openshift-default-5dd94f65d-bxhrc"
Sep 10 17:48:33.641967 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:48:33.641897    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a628ea2c-8ec6-4d36-99b5-db4f7927de9d-workload-certs\") pod \"e2e-inf-b-783a5e-openshift-default-5dd94f65d-bxhrc\" (UID: \"a628ea2c-8ec6-4d36-99b5-db4f7927de9d\") " pod="openshift-ingress/e2e-inf-b-783a5e-openshift-default-5dd94f65d-bxhrc"
Sep 10 17:48:33.641967 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:48:33.641964    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a628ea2c-8ec6-4d36-99b5-db4f7927de9d-istiod-ca-cert\") pod \"e2e-inf-b-783a5e-openshift-default-5dd94f65d-bxhrc\" (UID: \"a628ea2c-8ec6-4d36-99b5-db4f7927de9d\") " pod="openshift-ingress/e2e-inf-b-783a5e-openshift-default-5dd94f65d-bxhrc"
Sep 10 17:48:33.642221 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:48:33.642063    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a628ea2c-8ec6-4d36-99b5-db4f7927de9d-workload-socket\") pod \"e2e-inf-b-783a5e-openshift-default-5dd94f65d-bxhrc\" (UID: \"a628ea2c-8ec6-4d36-99b5-db4f7927de9d\") " pod="openshift-ingress/e2e-inf-b-783a5e-openshift-default-5dd94f65d-bxhrc"
Sep 10 17:48:33.642221 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:48:33.642081    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a628ea2c-8ec6-4d36-99b5-db4f7927de9d-credential-socket\") pod \"e2e-inf-b-783a5e-openshift-default-5dd94f65d-bxhrc\" (UID: \"a628ea2c-8ec6-4d36-99b5-db4f7927de9d\") " pod="openshift-ingress/e2e-inf-b-783a5e-openshift-default-5dd94f65d-bxhrc"
Sep 10 17:48:33.642221 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:48:33.642204    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a628ea2c-8ec6-4d36-99b5-db4f7927de9d-istio-data\") pod \"e2e-inf-b-783a5e-openshift-default-5dd94f65d-bxhrc\" (UID: \"a628ea2c-8ec6-4d36-99b5-db4f7927de9d\") " pod="openshift-ingress/e2e-inf-b-783a5e-openshift-default-5dd94f65d-bxhrc"
Sep 10 17:48:33.642333 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:48:33.642311    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a628ea2c-8ec6-4d36-99b5-db4f7927de9d-workload-certs\") pod \"e2e-inf-b-783a5e-openshift-default-5dd94f65d-bxhrc\" (UID: \"a628ea2c-8ec6-4d36-99b5-db4f7927de9d\") " pod="openshift-ingress/e2e-inf-b-783a5e-openshift-default-5dd94f65d-bxhrc"
Sep 10 17:48:33.642671 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:48:33.642628    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a628ea2c-8ec6-4d36-99b5-db4f7927de9d-istiod-ca-cert\") pod \"e2e-inf-b-783a5e-openshift-default-5dd94f65d-bxhrc\" (UID: \"a628ea2c-8ec6-4d36-99b5-db4f7927de9d\") " pod="openshift-ingress/e2e-inf-b-783a5e-openshift-default-5dd94f65d-bxhrc"
Sep 10 17:48:33.644224 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:48:33.644198    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a628ea2c-8ec6-4d36-99b5-db4f7927de9d-istio-envoy\") pod \"e2e-inf-b-783a5e-openshift-default-5dd94f65d-bxhrc\" (UID: \"a628ea2c-8ec6-4d36-99b5-db4f7927de9d\") " pod="openshift-ingress/e2e-inf-b-783a5e-openshift-default-5dd94f65d-bxhrc"
Sep 10 17:48:33.644330 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:48:33.644243    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a628ea2c-8ec6-4d36-99b5-db4f7927de9d-istio-podinfo\") pod \"e2e-inf-b-783a5e-openshift-default-5dd94f65d-bxhrc\" (UID: \"a628ea2c-8ec6-4d36-99b5-db4f7927de9d\") " pod="openshift-ingress/e2e-inf-b-783a5e-openshift-default-5dd94f65d-bxhrc"
Sep 10 17:48:33.650681 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:48:33.650634    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a628ea2c-8ec6-4d36-99b5-db4f7927de9d-istio-token\") pod \"e2e-inf-b-783a5e-openshift-default-5dd94f65d-bxhrc\" (UID: \"a628ea2c-8ec6-4d36-99b5-db4f7927de9d\") " pod="openshift-ingress/e2e-inf-b-783a5e-openshift-default-5dd94f65d-bxhrc"
Sep 10 17:48:33.650805 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:48:33.650788    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5qxt4\" (UniqueName: \"kubernetes.io/projected/a628ea2c-8ec6-4d36-99b5-db4f7927de9d-kube-api-access-5qxt4\") pod \"e2e-inf-b-783a5e-openshift-default-5dd94f65d-bxhrc\" (UID: \"a628ea2c-8ec6-4d36-99b5-db4f7927de9d\") " pod="openshift-ingress/e2e-inf-b-783a5e-openshift-default-5dd94f65d-bxhrc"
Sep 10 17:48:33.683242 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:48:33.683209    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-ait-4601bf83-fd5cdb559-rfp9s"
Sep 10 17:48:33.759147 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:48:33.759114    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-783a5e-openshift-default-5dd94f65d-bxhrc"
Sep 10 17:48:33.780614 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:48:33.780584    2581 generic.go:356] "Generic (PLEG): container finished" podID="8a2891cb-620d-4e36-984b-0cc72cc6f5b9" containerID="40882ffc414bf233886f1847bd73ca1e47ef3701710d891111d02bd939ec63e0" exitCode=0
Sep 10 17:48:33.780778 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:48:33.780676    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-autores-51ed8d-openshift-default-74f9875b8-4fftk" event={"ID":"8a2891cb-620d-4e36-984b-0cc72cc6f5b9","Type":"ContainerDied","Data":"40882ffc414bf233886f1847bd73ca1e47ef3701710d891111d02bd939ec63e0"}
Sep 10 17:48:33.781266 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:48:33.781247    2581 scope.go:117] "RemoveContainer" containerID="40882ffc414bf233886f1847bd73ca1e47ef3701710d891111d02bd939ec63e0"
Sep 10 17:48:33.782362 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:48:33.782332    2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 10 17:48:33.782463 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:48:33.782419    2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 10 17:48:33.782526 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:48:33.782467    2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 10 17:48:34.048992 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:48:34.048962    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-b-783a5e-openshift-default-5dd94f65d-bxhrc"]
Sep 10 17:48:34.051140 ip-10-0-141-180 kubenswrapper[2581]: W0910 17:48:34.051070    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda628ea2c_8ec6_4d36_99b5_db4f7927de9d.slice/crio-9aa497f6994c6eb15a1fb52fb684caaae9e1bb2f6d2524a24086fd5e7257e3ac WatchSource:0}: Error finding container 9aa497f6994c6eb15a1fb52fb684caaae9e1bb2f6d2524a24086fd5e7257e3ac: Status 404 returned error can't find the container with id 9aa497f6994c6eb15a1fb52fb684caaae9e1bb2f6d2524a24086fd5e7257e3ac
Sep 10 17:48:34.053783 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:48:34.053747    2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 10 17:48:34.053892 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:48:34.053833    2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 10 17:48:34.053892 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:48:34.053875    2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 10 17:48:34.794146 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:48:34.794101    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-783a5e-openshift-default-5dd94f65d-bxhrc" event={"ID":"a628ea2c-8ec6-4d36-99b5-db4f7927de9d","Type":"ContainerStarted","Data":"0e01bafb578bfa16f30a63f5ef6a4856daee5fb7a5a772c95c9dac27505996d7"}
Sep 10 17:48:34.794686 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:48:34.794191    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-783a5e-openshift-default-5dd94f65d-bxhrc" event={"ID":"a628ea2c-8ec6-4d36-99b5-db4f7927de9d","Type":"ContainerStarted","Data":"9aa497f6994c6eb15a1fb52fb684caaae9e1bb2f6d2524a24086fd5e7257e3ac"}
Sep 10 17:48:34.798161 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:48:34.798120    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-autores-51ed8d-openshift-default-74f9875b8-4fftk" event={"ID":"8a2891cb-620d-4e36-984b-0cc72cc6f5b9","Type":"ContainerStarted","Data":"1f9c700929ee0ca6c7f05ce88ed52f4ac6959c12d970938074baa18fede068c8"}
Sep 10 17:48:34.828160 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:48:34.828106    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-inf-b-783a5e-openshift-default-5dd94f65d-bxhrc" podStartSLOduration=1.828089503 podStartE2EDuration="1.828089503s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 17:48:33 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 17:48:34.824629916 +0000 UTC m=+734.478703823" watchObservedRunningTime="2026-09-10 17:48:34.828089503 +0000 UTC m=+734.482163786"
Sep 10 17:48:34.964122 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:48:34.964097    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-autores-51ed8d-openshift-default-74f9875b8-4fftk"
Sep 10 17:48:34.964308 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:48:34.964280    2581 patch_prober.go:28] interesting pod/e2e-autores-51ed8d-openshift-default-74f9875b8-4fftk container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.46:15021/healthz/ready\": dial tcp 10.133.0.46:15021: connect: connection refused" start-of-body=
Sep 10 17:48:34.964383 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:48:34.964340    2581 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-autores-51ed8d-openshift-default-74f9875b8-4fftk" podUID="8a2891cb-620d-4e36-984b-0cc72cc6f5b9" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.46:15021/healthz/ready\": dial tcp 10.133.0.46:15021: connect: connection refused"
Sep 10 17:48:35.760239 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:48:35.760205    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-b-783a5e-openshift-default-5dd94f65d-bxhrc"
Sep 10 17:48:35.765348 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:48:35.765317    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-b-783a5e-openshift-default-5dd94f65d-bxhrc"
Sep 10 17:48:35.804582 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:48:35.804558    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-b-783a5e-openshift-default-5dd94f65d-bxhrc"
Sep 10 17:48:35.805401 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:48:35.805381    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-b-783a5e-openshift-default-5dd94f65d-bxhrc"
Sep 10 17:48:35.964165 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:48:35.964131    2581 patch_prober.go:28] interesting pod/e2e-autores-51ed8d-openshift-default-74f9875b8-4fftk container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.46:15021/healthz/ready\": dial tcp 10.133.0.46:15021: connect: connection refused" start-of-body=
Sep 10 17:48:35.964321 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:48:35.964190    2581 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-autores-51ed8d-openshift-default-74f9875b8-4fftk" podUID="8a2891cb-620d-4e36-984b-0cc72cc6f5b9" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.46:15021/healthz/ready\": dial tcp 10.133.0.46:15021: connect: connection refused"
Sep 10 17:48:36.963973 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:48:36.963935    2581 patch_prober.go:28] interesting pod/e2e-autores-51ed8d-openshift-default-74f9875b8-4fftk container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.46:15021/healthz/ready\": dial tcp 10.133.0.46:15021: connect: connection refused" start-of-body=
Sep 10 17:48:36.964381 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:48:36.963990    2581 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-autores-51ed8d-openshift-default-74f9875b8-4fftk" podUID="8a2891cb-620d-4e36-984b-0cc72cc6f5b9" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.46:15021/healthz/ready\": dial tcp 10.133.0.46:15021: connect: connection refused"
Sep 10 17:48:37.964554 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:48:37.964509    2581 patch_prober.go:28] interesting pod/e2e-autores-51ed8d-openshift-default-74f9875b8-4fftk container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.46:15021/healthz/ready\": dial tcp 10.133.0.46:15021: connect: connection refused" start-of-body=
Sep 10 17:48:37.964967 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:48:37.964578    2581 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-autores-51ed8d-openshift-default-74f9875b8-4fftk" podUID="8a2891cb-620d-4e36-984b-0cc72cc6f5b9" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.46:15021/healthz/ready\": dial tcp 10.133.0.46:15021: connect: connection refused"
Sep 10 17:48:38.963816 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:48:38.963777    2581 patch_prober.go:28] interesting pod/e2e-autores-51ed8d-openshift-default-74f9875b8-4fftk container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.46:15021/healthz/ready\": dial tcp 10.133.0.46:15021: connect: connection refused" start-of-body=
Sep 10 17:48:38.963977 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:48:38.963843    2581 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-autores-51ed8d-openshift-default-74f9875b8-4fftk" podUID="8a2891cb-620d-4e36-984b-0cc72cc6f5b9" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.46:15021/healthz/ready\": dial tcp 10.133.0.46:15021: connect: connection refused"
Sep 10 17:48:39.098711 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:48:39.098671    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-b-783a5e-7b8b7f74fd-9r6bx"]
Sep 10 17:48:39.102252 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:48:39.102233    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-783a5e-7b8b7f74fd-9r6bx"
Sep 10 17:48:39.106083 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:48:39.106057    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-inf-b-783a5e-dockercfg-4864v\""
Sep 10 17:48:39.106270 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:48:39.106253    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-inf-b-783a5e\""
Sep 10 17:48:39.113958 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:48:39.113924    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-b-783a5e-7b8b7f74fd-9r6bx"]
Sep 10 17:48:39.219438 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:48:39.219355    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zh5zv\" (UniqueName: \"kubernetes.io/projected/787f82f8-1081-42c7-b950-763bcbae5d2c-kube-api-access-zh5zv\") pod \"payload-pre-processing-e2e-inf-b-783a5e-7b8b7f74fd-9r6bx\" (UID: \"787f82f8-1081-42c7-b950-763bcbae5d2c\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-b-783a5e-7b8b7f74fd-9r6bx"
Sep 10 17:48:39.219575 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:48:39.219469    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/787f82f8-1081-42c7-b950-763bcbae5d2c-plugins-config-volume\") pod \"payload-pre-processing-e2e-inf-b-783a5e-7b8b7f74fd-9r6bx\" (UID: \"787f82f8-1081-42c7-b950-763bcbae5d2c\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-b-783a5e-7b8b7f74fd-9r6bx"
Sep 10 17:48:39.320274 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:48:39.320228    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/787f82f8-1081-42c7-b950-763bcbae5d2c-plugins-config-volume\") pod \"payload-pre-processing-e2e-inf-b-783a5e-7b8b7f74fd-9r6bx\" (UID: \"787f82f8-1081-42c7-b950-763bcbae5d2c\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-b-783a5e-7b8b7f74fd-9r6bx"
Sep 10 17:48:39.320429 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:48:39.320310    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zh5zv\" (UniqueName: \"kubernetes.io/projected/787f82f8-1081-42c7-b950-763bcbae5d2c-kube-api-access-zh5zv\") pod \"payload-pre-processing-e2e-inf-b-783a5e-7b8b7f74fd-9r6bx\" (UID: \"787f82f8-1081-42c7-b950-763bcbae5d2c\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-b-783a5e-7b8b7f74fd-9r6bx"
Sep 10 17:48:39.321065 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:48:39.321040    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/787f82f8-1081-42c7-b950-763bcbae5d2c-plugins-config-volume\") pod \"payload-pre-processing-e2e-inf-b-783a5e-7b8b7f74fd-9r6bx\" (UID: \"787f82f8-1081-42c7-b950-763bcbae5d2c\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-b-783a5e-7b8b7f74fd-9r6bx"
Sep 10 17:48:39.329908 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:48:39.329871    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zh5zv\" (UniqueName: \"kubernetes.io/projected/787f82f8-1081-42c7-b950-763bcbae5d2c-kube-api-access-zh5zv\") pod \"payload-pre-processing-e2e-inf-b-783a5e-7b8b7f74fd-9r6bx\" (UID: \"787f82f8-1081-42c7-b950-763bcbae5d2c\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-b-783a5e-7b8b7f74fd-9r6bx"
Sep 10 17:48:39.414881 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:48:39.414844    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-783a5e-7b8b7f74fd-9r6bx"
Sep 10 17:48:39.729354 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:48:39.729318    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-b-783a5e-7b8b7f74fd-9r6bx"]
Sep 10 17:48:39.730062 ip-10-0-141-180 kubenswrapper[2581]: W0910 17:48:39.730037    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod787f82f8_1081_42c7_b950_763bcbae5d2c.slice/crio-81e3d68fd924d1882444f7bf98aa0b2104b29d72d1176e10910142a9592bae44 WatchSource:0}: Error finding container 81e3d68fd924d1882444f7bf98aa0b2104b29d72d1176e10910142a9592bae44: Status 404 returned error can't find the container with id 81e3d68fd924d1882444f7bf98aa0b2104b29d72d1176e10910142a9592bae44
Sep 10 17:48:39.830716 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:48:39.830679    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-783a5e-7b8b7f74fd-9r6bx" event={"ID":"787f82f8-1081-42c7-b950-763bcbae5d2c","Type":"ContainerStarted","Data":"ae08db3b74825ce340cca9f23fc367319a4618733e8af4a9a3b3e025ca2de72b"}
Sep 10 17:48:39.830903 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:48:39.830742    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-783a5e-7b8b7f74fd-9r6bx" event={"ID":"787f82f8-1081-42c7-b950-763bcbae5d2c","Type":"ContainerStarted","Data":"81e3d68fd924d1882444f7bf98aa0b2104b29d72d1176e10910142a9592bae44"}
Sep 10 17:48:39.830903 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:48:39.830834    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-783a5e-7b8b7f74fd-9r6bx"
Sep 10 17:48:39.855206 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:48:39.855158    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-783a5e-7b8b7f74fd-9r6bx" podStartSLOduration=0.855143032 podStartE2EDuration="855.143032ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 17:48:39 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 17:48:39.852001049 +0000 UTC m=+739.506074964" watchObservedRunningTime="2026-09-10 17:48:39.855143032 +0000 UTC m=+739.509216936"
Sep 10 17:48:39.964012 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:48:39.963971    2581 patch_prober.go:28] interesting pod/e2e-autores-51ed8d-openshift-default-74f9875b8-4fftk container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.46:15021/healthz/ready\": dial tcp 10.133.0.46:15021: connect: connection refused" start-of-body=
Sep 10 17:48:39.964203 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:48:39.964028    2581 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-autores-51ed8d-openshift-default-74f9875b8-4fftk" podUID="8a2891cb-620d-4e36-984b-0cc72cc6f5b9" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.46:15021/healthz/ready\": dial tcp 10.133.0.46:15021: connect: connection refused"
Sep 10 17:48:41.005948 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:48:41.005919    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-autores-51ed8d-openshift-default-74f9875b8-4fftk"
Sep 10 17:48:41.006541 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:48:41.006397    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-autores-51ed8d-openshift-default-74f9875b8-4fftk"
Sep 10 17:48:41.007057 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:48:41.007039    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-autores-51ed8d-openshift-default-74f9875b8-4fftk"
Sep 10 17:48:49.369068 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:48:49.369032    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-4601bf83-ntm9s"]
Sep 10 17:48:49.371878 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:48:49.371852    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-4601bf83-ntm9s"
Sep 10 17:48:49.378367 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:48:49.377973    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-k7zj6\""
Sep 10 17:48:49.378367 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:48:49.378055    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 10 17:48:49.393136 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:48:49.393109    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-4601bf83-ntm9s"]
Sep 10 17:48:49.415087 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:48:49.415057    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/e8ee9eb2-44a9-4f94-84e2-c8ef43761d60-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-4601bf83-ntm9s\" (UID: \"e8ee9eb2-44a9-4f94-84e2-c8ef43761d60\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-4601bf83-ntm9s"
Sep 10 17:48:49.516022 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:48:49.515980    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/e8ee9eb2-44a9-4f94-84e2-c8ef43761d60-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-4601bf83-ntm9s\" (UID: \"e8ee9eb2-44a9-4f94-84e2-c8ef43761d60\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-4601bf83-ntm9s"
Sep 10 17:48:49.516607 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:48:49.516587    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/e8ee9eb2-44a9-4f94-84e2-c8ef43761d60-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-4601bf83-ntm9s\" (UID: \"e8ee9eb2-44a9-4f94-84e2-c8ef43761d60\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-4601bf83-ntm9s"
Sep 10 17:48:49.696234 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:48:49.696152    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-4601bf83-ntm9s"
Sep 10 17:48:49.987443 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:48:49.987403    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-4601bf83-ntm9s"]
Sep 10 17:48:49.998112 ip-10-0-141-180 kubenswrapper[2581]: W0910 17:48:49.997885    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode8ee9eb2_44a9_4f94_84e2_c8ef43761d60.slice/crio-0e1ce9cc09203798ea9f70972381920d8687f06cd5bbf320ac197a93d9151dda WatchSource:0}: Error finding container 0e1ce9cc09203798ea9f70972381920d8687f06cd5bbf320ac197a93d9151dda: Status 404 returned error can't find the container with id 0e1ce9cc09203798ea9f70972381920d8687f06cd5bbf320ac197a93d9151dda
Sep 10 17:48:50.841043 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:48:50.841013    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-783a5e-7b8b7f74fd-9r6bx"
Sep 10 17:48:50.914768 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:48:50.914729    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-4601bf83-ntm9s" event={"ID":"e8ee9eb2-44a9-4f94-84e2-c8ef43761d60","Type":"ContainerStarted","Data":"b75b0ea151068376533342fecfbef5908cd0876a562389172dc38515c9e13916"}
Sep 10 17:48:50.914947 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:48:50.914785    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-4601bf83-ntm9s" event={"ID":"e8ee9eb2-44a9-4f94-84e2-c8ef43761d60","Type":"ContainerStarted","Data":"0e1ce9cc09203798ea9f70972381920d8687f06cd5bbf320ac197a93d9151dda"}
Sep 10 17:48:50.936363 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:48:50.936314    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-4601bf83-ntm9s" podStartSLOduration=1.9362983919999999 podStartE2EDuration="1.936298392s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 17:48:49 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 17:48:50.935620307 +0000 UTC m=+750.589694231" watchObservedRunningTime="2026-09-10 17:48:50.936298392 +0000 UTC m=+750.590372306"
Sep 10 17:48:51.921995 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:48:51.921957    2581 generic.go:356] "Generic (PLEG): container finished" podID="e8ee9eb2-44a9-4f94-84e2-c8ef43761d60" containerID="b75b0ea151068376533342fecfbef5908cd0876a562389172dc38515c9e13916" exitCode=0
Sep 10 17:48:51.922365 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:48:51.922038    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-4601bf83-ntm9s" event={"ID":"e8ee9eb2-44a9-4f94-84e2-c8ef43761d60","Type":"ContainerDied","Data":"b75b0ea151068376533342fecfbef5908cd0876a562389172dc38515c9e13916"}
Sep 10 17:48:53.102966 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:48:53.102940    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-4601bf83-ntm9s"
Sep 10 17:48:53.152332 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:48:53.152287    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/e8ee9eb2-44a9-4f94-84e2-c8ef43761d60-maas-api-service-ca\") pod \"e8ee9eb2-44a9-4f94-84e2-c8ef43761d60\" (UID: \"e8ee9eb2-44a9-4f94-84e2-c8ef43761d60\") "
Sep 10 17:48:53.152731 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:48:53.152702    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e8ee9eb2-44a9-4f94-84e2-c8ef43761d60-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "e8ee9eb2-44a9-4f94-84e2-c8ef43761d60" (UID: "e8ee9eb2-44a9-4f94-84e2-c8ef43761d60"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 17:48:53.253040 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:48:53.253006    2581 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/e8ee9eb2-44a9-4f94-84e2-c8ef43761d60-maas-api-service-ca\") on node \"ip-10-0-141-180.ec2.internal\" DevicePath \"\""
Sep 10 17:48:53.936537 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:48:53.936461    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-4601bf83-ntm9s"
Sep 10 17:48:53.936711 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:48:53.936467    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-4601bf83-ntm9s" event={"ID":"e8ee9eb2-44a9-4f94-84e2-c8ef43761d60","Type":"ContainerDied","Data":"0e1ce9cc09203798ea9f70972381920d8687f06cd5bbf320ac197a93d9151dda"}
Sep 10 17:48:53.936711 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:48:53.936590    2581 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="0e1ce9cc09203798ea9f70972381920d8687f06cd5bbf320ac197a93d9151dda"
Sep 10 17:48:56.955377 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:48:56.955341    2581 generic.go:356] "Generic (PLEG): container finished" podID="a628ea2c-8ec6-4d36-99b5-db4f7927de9d" containerID="0e01bafb578bfa16f30a63f5ef6a4856daee5fb7a5a772c95c9dac27505996d7" exitCode=0
Sep 10 17:48:56.955768 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:48:56.955413    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-783a5e-openshift-default-5dd94f65d-bxhrc" event={"ID":"a628ea2c-8ec6-4d36-99b5-db4f7927de9d","Type":"ContainerDied","Data":"0e01bafb578bfa16f30a63f5ef6a4856daee5fb7a5a772c95c9dac27505996d7"}
Sep 10 17:48:56.955859 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:48:56.955845    2581 scope.go:117] "RemoveContainer" containerID="0e01bafb578bfa16f30a63f5ef6a4856daee5fb7a5a772c95c9dac27505996d7"
Sep 10 17:48:56.957095 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:48:56.957069    2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 10 17:48:56.957171 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:48:56.957137    2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 10 17:48:56.957171 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:48:56.957167    2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 10 17:48:57.963442 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:48:57.963410    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-783a5e-openshift-default-5dd94f65d-bxhrc" event={"ID":"a628ea2c-8ec6-4d36-99b5-db4f7927de9d","Type":"ContainerStarted","Data":"eaf96441f2fb4a1737394476495cc0ac2d6535ab6fa866d3089441ac827239a3"}
Sep 10 17:48:58.760228 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:48:58.760189    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-b-783a5e-openshift-default-5dd94f65d-bxhrc"
Sep 10 17:48:58.760496 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:48:58.760473    2581 patch_prober.go:28] interesting pod/e2e-inf-b-783a5e-openshift-default-5dd94f65d-bxhrc container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.49:15021/healthz/ready\": dial tcp 10.133.0.49:15021: connect: connection refused" start-of-body=
Sep 10 17:48:58.760549 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:48:58.760521    2581 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-783a5e-openshift-default-5dd94f65d-bxhrc" podUID="a628ea2c-8ec6-4d36-99b5-db4f7927de9d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.49:15021/healthz/ready\": dial tcp 10.133.0.49:15021: connect: connection refused"
Sep 10 17:48:59.760493 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:48:59.760455    2581 patch_prober.go:28] interesting pod/e2e-inf-b-783a5e-openshift-default-5dd94f65d-bxhrc container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.49:15021/healthz/ready\": dial tcp 10.133.0.49:15021: connect: connection refused" start-of-body=
Sep 10 17:48:59.760913 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:48:59.760512    2581 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-783a5e-openshift-default-5dd94f65d-bxhrc" podUID="a628ea2c-8ec6-4d36-99b5-db4f7927de9d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.49:15021/healthz/ready\": dial tcp 10.133.0.49:15021: connect: connection refused"
Sep 10 17:49:00.349285 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:49:00.349247    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-4601bf83-fd5cdb559-rfp9s"]
Sep 10 17:49:00.349572 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:49:00.349542    2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-ait-4601bf83-fd5cdb559-rfp9s" podUID="5b4452ff-d15f-493c-b9e6-9704144bf575" containerName="payload-processing" containerID="cri-o://ee061dc611eb3e957ed326b457e9471453a1e6a54edad32558ef39398e3683e0" gracePeriod=30
Sep 10 17:49:00.714657 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:49:00.714620    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-4601bf83-fd5cdb559-rfp9s"
Sep 10 17:49:00.758576 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:49:00.758542    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-4601bf83-ntm9s"]
Sep 10 17:49:00.759582 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:49:00.759562    2581 patch_prober.go:28] interesting pod/e2e-inf-b-783a5e-openshift-default-5dd94f65d-bxhrc container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.49:15021/healthz/ready\": dial tcp 10.133.0.49:15021: connect: connection refused" start-of-body=
Sep 10 17:49:00.759657 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:49:00.759605    2581 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-783a5e-openshift-default-5dd94f65d-bxhrc" podUID="a628ea2c-8ec6-4d36-99b5-db4f7927de9d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.49:15021/healthz/ready\": dial tcp 10.133.0.49:15021: connect: connection refused"
Sep 10 17:49:00.804958 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:49:00.804920    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-4601bf83-ntm9s"]
Sep 10 17:49:00.828349 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:49:00.828318    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-kz8v8\" (UniqueName: \"kubernetes.io/projected/5b4452ff-d15f-493c-b9e6-9704144bf575-kube-api-access-kz8v8\") pod \"5b4452ff-d15f-493c-b9e6-9704144bf575\" (UID: \"5b4452ff-d15f-493c-b9e6-9704144bf575\") "
Sep 10 17:49:00.828549 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:49:00.828470    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/5b4452ff-d15f-493c-b9e6-9704144bf575-plugins-config-volume\") pod \"5b4452ff-d15f-493c-b9e6-9704144bf575\" (UID: \"5b4452ff-d15f-493c-b9e6-9704144bf575\") "
Sep 10 17:49:00.828846 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:49:00.828822    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5b4452ff-d15f-493c-b9e6-9704144bf575-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "5b4452ff-d15f-493c-b9e6-9704144bf575" (UID: "5b4452ff-d15f-493c-b9e6-9704144bf575"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 17:49:00.830653 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:49:00.830617    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5b4452ff-d15f-493c-b9e6-9704144bf575-kube-api-access-kz8v8" (OuterVolumeSpecName: "kube-api-access-kz8v8") pod "5b4452ff-d15f-493c-b9e6-9704144bf575" (UID: "5b4452ff-d15f-493c-b9e6-9704144bf575"). InnerVolumeSpecName "kube-api-access-kz8v8". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 17:49:00.875945 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:49:00.875907    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e8ee9eb2-44a9-4f94-84e2-c8ef43761d60" path="/var/lib/kubelet/pods/e8ee9eb2-44a9-4f94-84e2-c8ef43761d60/volumes"
Sep 10 17:49:00.930520 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:49:00.930434    2581 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/5b4452ff-d15f-493c-b9e6-9704144bf575-plugins-config-volume\") on node \"ip-10-0-141-180.ec2.internal\" DevicePath \"\""
Sep 10 17:49:00.930520 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:49:00.930470    2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-kz8v8\" (UniqueName: \"kubernetes.io/projected/5b4452ff-d15f-493c-b9e6-9704144bf575-kube-api-access-kz8v8\") on node \"ip-10-0-141-180.ec2.internal\" DevicePath \"\""
Sep 10 17:49:01.009343 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:49:01.009309    2581 generic.go:356] "Generic (PLEG): container finished" podID="5b4452ff-d15f-493c-b9e6-9704144bf575" containerID="ee061dc611eb3e957ed326b457e9471453a1e6a54edad32558ef39398e3683e0" exitCode=0
Sep 10 17:49:01.009864 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:49:01.009507    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-4601bf83-fd5cdb559-rfp9s" event={"ID":"5b4452ff-d15f-493c-b9e6-9704144bf575","Type":"ContainerDied","Data":"ee061dc611eb3e957ed326b457e9471453a1e6a54edad32558ef39398e3683e0"}
Sep 10 17:49:01.009986 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:49:01.009914    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-4601bf83-fd5cdb559-rfp9s" event={"ID":"5b4452ff-d15f-493c-b9e6-9704144bf575","Type":"ContainerDied","Data":"6b525e6f582b6e1370d184069010def58626a0583b056399b1cc8c81357474e5"}
Sep 10 17:49:01.009986 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:49:01.009599    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-4601bf83-fd5cdb559-rfp9s"
Sep 10 17:49:01.009986 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:49:01.009961    2581 scope.go:117] "RemoveContainer" containerID="ee061dc611eb3e957ed326b457e9471453a1e6a54edad32558ef39398e3683e0"
Sep 10 17:49:01.023617 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:49:01.023575    2581 scope.go:117] "RemoveContainer" containerID="ee061dc611eb3e957ed326b457e9471453a1e6a54edad32558ef39398e3683e0"
Sep 10 17:49:01.024011 ip-10-0-141-180 kubenswrapper[2581]: E0910 17:49:01.023981    2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ee061dc611eb3e957ed326b457e9471453a1e6a54edad32558ef39398e3683e0\": container with ID starting with ee061dc611eb3e957ed326b457e9471453a1e6a54edad32558ef39398e3683e0 not found: ID does not exist" containerID="ee061dc611eb3e957ed326b457e9471453a1e6a54edad32558ef39398e3683e0"
Sep 10 17:49:01.024106 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:49:01.024020    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ee061dc611eb3e957ed326b457e9471453a1e6a54edad32558ef39398e3683e0"} err="failed to get container status \"ee061dc611eb3e957ed326b457e9471453a1e6a54edad32558ef39398e3683e0\": rpc error: code = NotFound desc = could not find container \"ee061dc611eb3e957ed326b457e9471453a1e6a54edad32558ef39398e3683e0\": container with ID starting with ee061dc611eb3e957ed326b457e9471453a1e6a54edad32558ef39398e3683e0 not found: ID does not exist"
Sep 10 17:49:01.040198 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:49:01.039869    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-4601bf83-fd5cdb559-rfp9s"]
Sep 10 17:49:01.048801 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:49:01.048782    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-4601bf83-fd5cdb559-rfp9s"]
Sep 10 17:49:01.760417 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:49:01.760380    2581 patch_prober.go:28] interesting pod/e2e-inf-b-783a5e-openshift-default-5dd94f65d-bxhrc container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.49:15021/healthz/ready\": dial tcp 10.133.0.49:15021: connect: connection refused" start-of-body=
Sep 10 17:49:01.760593 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:49:01.760434    2581 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-783a5e-openshift-default-5dd94f65d-bxhrc" podUID="a628ea2c-8ec6-4d36-99b5-db4f7927de9d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.49:15021/healthz/ready\": dial tcp 10.133.0.49:15021: connect: connection refused"
Sep 10 17:49:02.759380 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:49:02.759347    2581 patch_prober.go:28] interesting pod/e2e-inf-b-783a5e-openshift-default-5dd94f65d-bxhrc container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.49:15021/healthz/ready\": dial tcp 10.133.0.49:15021: connect: connection refused" start-of-body=
Sep 10 17:49:02.759789 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:49:02.759394    2581 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-783a5e-openshift-default-5dd94f65d-bxhrc" podUID="a628ea2c-8ec6-4d36-99b5-db4f7927de9d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.49:15021/healthz/ready\": dial tcp 10.133.0.49:15021: connect: connection refused"
Sep 10 17:49:02.863757 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:49:02.863723    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5b4452ff-d15f-493c-b9e6-9704144bf575" path="/var/lib/kubelet/pods/5b4452ff-d15f-493c-b9e6-9704144bf575/volumes"
Sep 10 17:49:03.763447 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:49:03.763417    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-b-783a5e-openshift-default-5dd94f65d-bxhrc"
Sep 10 17:49:03.763903 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:49:03.763888    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-b-783a5e-openshift-default-5dd94f65d-bxhrc"
Sep 10 17:49:03.764521 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:49:03.764498    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-b-783a5e-openshift-default-5dd94f65d-bxhrc"
Sep 10 17:49:29.963039 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:49:29.963000    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-624b5c35-w6ggf"]
Sep 10 17:49:29.963673 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:49:29.963652    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5b4452ff-d15f-493c-b9e6-9704144bf575" containerName="payload-processing"
Sep 10 17:49:29.963730 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:49:29.963675    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="5b4452ff-d15f-493c-b9e6-9704144bf575" containerName="payload-processing"
Sep 10 17:49:29.963730 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:49:29.963698    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e8ee9eb2-44a9-4f94-84e2-c8ef43761d60" containerName="revoke-keys"
Sep 10 17:49:29.963730 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:49:29.963705    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="e8ee9eb2-44a9-4f94-84e2-c8ef43761d60" containerName="revoke-keys"
Sep 10 17:49:29.963827 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:49:29.963819    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="5b4452ff-d15f-493c-b9e6-9704144bf575" containerName="payload-processing"
Sep 10 17:49:29.963861 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:49:29.963835    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="e8ee9eb2-44a9-4f94-84e2-c8ef43761d60" containerName="revoke-keys"
Sep 10 17:49:29.967069 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:49:29.967050    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-624b5c35-w6ggf"
Sep 10 17:49:29.969897 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:49:29.969867    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-k7zj6\""
Sep 10 17:49:29.970006 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:49:29.969865    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 10 17:49:29.977810 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:49:29.977781    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-624b5c35-w6ggf"]
Sep 10 17:49:30.027105 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:49:30.027075    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/e136c0dd-3ef3-4857-a728-bba0a3cb41f0-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-624b5c35-w6ggf\" (UID: \"e136c0dd-3ef3-4857-a728-bba0a3cb41f0\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-624b5c35-w6ggf"
Sep 10 17:49:30.128075 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:49:30.128044    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/e136c0dd-3ef3-4857-a728-bba0a3cb41f0-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-624b5c35-w6ggf\" (UID: \"e136c0dd-3ef3-4857-a728-bba0a3cb41f0\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-624b5c35-w6ggf"
Sep 10 17:49:30.128732 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:49:30.128713    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/e136c0dd-3ef3-4857-a728-bba0a3cb41f0-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-624b5c35-w6ggf\" (UID: \"e136c0dd-3ef3-4857-a728-bba0a3cb41f0\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-624b5c35-w6ggf"
Sep 10 17:49:30.279977 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:49:30.279885    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-624b5c35-w6ggf"
Sep 10 17:49:30.564194 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:49:30.564155    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-624b5c35-w6ggf"]
Sep 10 17:49:30.564984 ip-10-0-141-180 kubenswrapper[2581]: W0910 17:49:30.564955    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode136c0dd_3ef3_4857_a728_bba0a3cb41f0.slice/crio-08371bff5f9d27c97dfc651f784e145547e149c0e688adefdca085340c01eee7 WatchSource:0}: Error finding container 08371bff5f9d27c97dfc651f784e145547e149c0e688adefdca085340c01eee7: Status 404 returned error can't find the container with id 08371bff5f9d27c97dfc651f784e145547e149c0e688adefdca085340c01eee7
Sep 10 17:49:31.215130 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:49:31.215096    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-624b5c35-w6ggf" event={"ID":"e136c0dd-3ef3-4857-a728-bba0a3cb41f0","Type":"ContainerStarted","Data":"12982feb88d8846d024d793c7b2e542eb3004cb8d0d37a778939f56e9829f902"}
Sep 10 17:49:31.215505 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:49:31.215159    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-624b5c35-w6ggf" event={"ID":"e136c0dd-3ef3-4857-a728-bba0a3cb41f0","Type":"ContainerStarted","Data":"08371bff5f9d27c97dfc651f784e145547e149c0e688adefdca085340c01eee7"}
Sep 10 17:49:31.239242 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:49:31.239191    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-624b5c35-w6ggf" podStartSLOduration=2.239174896 podStartE2EDuration="2.239174896s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 17:49:29 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 17:49:31.238374837 +0000 UTC m=+790.892448740" watchObservedRunningTime="2026-09-10 17:49:31.239174896 +0000 UTC m=+790.893248829"
Sep 10 17:49:32.223953 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:49:32.223916    2581 generic.go:356] "Generic (PLEG): container finished" podID="e136c0dd-3ef3-4857-a728-bba0a3cb41f0" containerID="12982feb88d8846d024d793c7b2e542eb3004cb8d0d37a778939f56e9829f902" exitCode=0
Sep 10 17:49:32.224493 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:49:32.224145    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-624b5c35-w6ggf" event={"ID":"e136c0dd-3ef3-4857-a728-bba0a3cb41f0","Type":"ContainerDied","Data":"12982feb88d8846d024d793c7b2e542eb3004cb8d0d37a778939f56e9829f902"}
Sep 10 17:49:33.368596 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:49:33.368570    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-624b5c35-w6ggf"
Sep 10 17:49:33.459287 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:49:33.459249    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/e136c0dd-3ef3-4857-a728-bba0a3cb41f0-maas-api-service-ca\") pod \"e136c0dd-3ef3-4857-a728-bba0a3cb41f0\" (UID: \"e136c0dd-3ef3-4857-a728-bba0a3cb41f0\") "
Sep 10 17:49:33.459590 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:49:33.459567    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e136c0dd-3ef3-4857-a728-bba0a3cb41f0-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "e136c0dd-3ef3-4857-a728-bba0a3cb41f0" (UID: "e136c0dd-3ef3-4857-a728-bba0a3cb41f0"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 17:49:33.560110 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:49:33.560025    2581 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/e136c0dd-3ef3-4857-a728-bba0a3cb41f0-maas-api-service-ca\") on node \"ip-10-0-141-180.ec2.internal\" DevicePath \"\""
Sep 10 17:49:34.241758 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:49:34.241727    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-624b5c35-w6ggf"
Sep 10 17:49:34.241758 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:49:34.241744    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-624b5c35-w6ggf" event={"ID":"e136c0dd-3ef3-4857-a728-bba0a3cb41f0","Type":"ContainerDied","Data":"08371bff5f9d27c97dfc651f784e145547e149c0e688adefdca085340c01eee7"}
Sep 10 17:49:34.241982 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:49:34.241796    2581 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="08371bff5f9d27c97dfc651f784e145547e149c0e688adefdca085340c01eee7"
Sep 10 17:49:37.226821 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:49:37.226781    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-624b5c35-w6ggf"]
Sep 10 17:49:37.231058 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:49:37.231026    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-624b5c35-w6ggf"]
Sep 10 17:49:38.862561 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:49:38.862527    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e136c0dd-3ef3-4857-a728-bba0a3cb41f0" path="/var/lib/kubelet/pods/e136c0dd-3ef3-4857-a728-bba0a3cb41f0/volumes"
Sep 10 17:49:42.996184 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:49:42.996106    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-0ae95d09-96b97ffc-6mbd9"]
Sep 10 17:49:42.996615 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:49:42.996600    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e136c0dd-3ef3-4857-a728-bba0a3cb41f0" containerName="revoke-keys"
Sep 10 17:49:42.996700 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:49:42.996617    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="e136c0dd-3ef3-4857-a728-bba0a3cb41f0" containerName="revoke-keys"
Sep 10 17:49:42.996745 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:49:42.996722    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="e136c0dd-3ef3-4857-a728-bba0a3cb41f0" containerName="revoke-keys"
Sep 10 17:49:42.998999 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:49:42.998981    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-0ae95d09-96b97ffc-6mbd9"
Sep 10 17:49:43.001714 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:49:43.001690    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-ait-0ae95d09-dockercfg-jbgts\""
Sep 10 17:49:43.001863 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:49:43.001843    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-ait-0ae95d09\""
Sep 10 17:49:43.012037 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:49:43.012015    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-0ae95d09-96b97ffc-6mbd9"]
Sep 10 17:49:43.058286 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:49:43.058251    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/09cf427e-942e-4183-b063-18ac4d36ec2a-plugins-config-volume\") pod \"payload-processing-e2e-ait-0ae95d09-96b97ffc-6mbd9\" (UID: \"09cf427e-942e-4183-b063-18ac4d36ec2a\") " pod="openshift-ingress/payload-processing-e2e-ait-0ae95d09-96b97ffc-6mbd9"
Sep 10 17:49:43.058407 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:49:43.058307    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kp8d4\" (UniqueName: \"kubernetes.io/projected/09cf427e-942e-4183-b063-18ac4d36ec2a-kube-api-access-kp8d4\") pod \"payload-processing-e2e-ait-0ae95d09-96b97ffc-6mbd9\" (UID: \"09cf427e-942e-4183-b063-18ac4d36ec2a\") " pod="openshift-ingress/payload-processing-e2e-ait-0ae95d09-96b97ffc-6mbd9"
Sep 10 17:49:43.159887 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:49:43.159848    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/09cf427e-942e-4183-b063-18ac4d36ec2a-plugins-config-volume\") pod \"payload-processing-e2e-ait-0ae95d09-96b97ffc-6mbd9\" (UID: \"09cf427e-942e-4183-b063-18ac4d36ec2a\") " pod="openshift-ingress/payload-processing-e2e-ait-0ae95d09-96b97ffc-6mbd9"
Sep 10 17:49:43.160058 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:49:43.159936    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kp8d4\" (UniqueName: \"kubernetes.io/projected/09cf427e-942e-4183-b063-18ac4d36ec2a-kube-api-access-kp8d4\") pod \"payload-processing-e2e-ait-0ae95d09-96b97ffc-6mbd9\" (UID: \"09cf427e-942e-4183-b063-18ac4d36ec2a\") " pod="openshift-ingress/payload-processing-e2e-ait-0ae95d09-96b97ffc-6mbd9"
Sep 10 17:49:43.160625 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:49:43.160606    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/09cf427e-942e-4183-b063-18ac4d36ec2a-plugins-config-volume\") pod \"payload-processing-e2e-ait-0ae95d09-96b97ffc-6mbd9\" (UID: \"09cf427e-942e-4183-b063-18ac4d36ec2a\") " pod="openshift-ingress/payload-processing-e2e-ait-0ae95d09-96b97ffc-6mbd9"
Sep 10 17:49:43.170026 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:49:43.169998    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kp8d4\" (UniqueName: \"kubernetes.io/projected/09cf427e-942e-4183-b063-18ac4d36ec2a-kube-api-access-kp8d4\") pod \"payload-processing-e2e-ait-0ae95d09-96b97ffc-6mbd9\" (UID: \"09cf427e-942e-4183-b063-18ac4d36ec2a\") " pod="openshift-ingress/payload-processing-e2e-ait-0ae95d09-96b97ffc-6mbd9"
Sep 10 17:49:43.310682 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:49:43.310577    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-0ae95d09-96b97ffc-6mbd9"
Sep 10 17:49:43.588289 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:49:43.588258    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-0ae95d09-96b97ffc-6mbd9"]
Sep 10 17:49:43.588759 ip-10-0-141-180 kubenswrapper[2581]: W0910 17:49:43.588728    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod09cf427e_942e_4183_b063_18ac4d36ec2a.slice/crio-29b6e095b0b1e9d2765815fe1185d98d809d34929ed6623539149bb7b7816d57 WatchSource:0}: Error finding container 29b6e095b0b1e9d2765815fe1185d98d809d34929ed6623539149bb7b7816d57: Status 404 returned error can't find the container with id 29b6e095b0b1e9d2765815fe1185d98d809d34929ed6623539149bb7b7816d57
Sep 10 17:49:44.304474 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:49:44.304435    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-0ae95d09-96b97ffc-6mbd9" event={"ID":"09cf427e-942e-4183-b063-18ac4d36ec2a","Type":"ContainerStarted","Data":"cd977e3c3e768f0c1e9fb8a9d788dbb0580f681fe9bf163459daebc6bd0100b5"}
Sep 10 17:49:44.304895 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:49:44.304508    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-ait-0ae95d09-96b97ffc-6mbd9"
Sep 10 17:49:44.304895 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:49:44.304536    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-0ae95d09-96b97ffc-6mbd9" event={"ID":"09cf427e-942e-4183-b063-18ac4d36ec2a","Type":"ContainerStarted","Data":"29b6e095b0b1e9d2765815fe1185d98d809d34929ed6623539149bb7b7816d57"}
Sep 10 17:49:44.335108 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:49:44.335060    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-ait-0ae95d09-96b97ffc-6mbd9" podStartSLOduration=2.335044499 podStartE2EDuration="2.335044499s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 17:49:42 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 17:49:44.332632791 +0000 UTC m=+803.986706723" watchObservedRunningTime="2026-09-10 17:49:44.335044499 +0000 UTC m=+803.989118466"
Sep 10 17:49:53.768587 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:49:53.768552    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-autores-51ed8d-znfkv"]
Sep 10 17:49:53.772036 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:49:53.772011    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-autores-51ed8d-znfkv"
Sep 10 17:49:53.775420 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:49:53.775063    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 10 17:49:53.775420 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:49:53.775395    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-k7zj6\""
Sep 10 17:49:53.777854 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:49:53.777832    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-autores-51ed8d-znfkv"]
Sep 10 17:49:53.867896 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:49:53.867865    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/3a54a06a-4ef1-4abc-a578-fcb18ced422b-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-autores-51ed8d-znfkv\" (UID: \"3a54a06a-4ef1-4abc-a578-fcb18ced422b\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-autores-51ed8d-znfkv"
Sep 10 17:49:53.968410 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:49:53.968377    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/3a54a06a-4ef1-4abc-a578-fcb18ced422b-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-autores-51ed8d-znfkv\" (UID: \"3a54a06a-4ef1-4abc-a578-fcb18ced422b\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-autores-51ed8d-znfkv"
Sep 10 17:49:53.969044 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:49:53.969022    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/3a54a06a-4ef1-4abc-a578-fcb18ced422b-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-autores-51ed8d-znfkv\" (UID: \"3a54a06a-4ef1-4abc-a578-fcb18ced422b\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-autores-51ed8d-znfkv"
Sep 10 17:49:54.087305 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:49:54.087227    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-autores-51ed8d-znfkv"
Sep 10 17:49:54.362584 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:49:54.360374    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-autores-51ed8d-znfkv"]
Sep 10 17:49:54.379807 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:49:54.379776    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-autores-51ed8d-znfkv" event={"ID":"3a54a06a-4ef1-4abc-a578-fcb18ced422b","Type":"ContainerStarted","Data":"678daf6f399baccda80ebee6ab9c77228d6f76b68f349c272b4063dbf181a4d6"}
Sep 10 17:49:55.313211 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:49:55.313175    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-ait-0ae95d09-96b97ffc-6mbd9"
Sep 10 17:49:55.388017 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:49:55.387988    2581 generic.go:356] "Generic (PLEG): container finished" podID="3a54a06a-4ef1-4abc-a578-fcb18ced422b" containerID="fce53936a17f1bed97660b5655f737a98f3a85f7ca469637e97e64d46d4d0c0d" exitCode=0
Sep 10 17:49:55.388177 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:49:55.388043    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-autores-51ed8d-znfkv" event={"ID":"3a54a06a-4ef1-4abc-a578-fcb18ced422b","Type":"ContainerDied","Data":"fce53936a17f1bed97660b5655f737a98f3a85f7ca469637e97e64d46d4d0c0d"}
Sep 10 17:49:56.530947 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:49:56.530918    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-autores-51ed8d-znfkv"
Sep 10 17:49:56.695988 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:49:56.695909    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/3a54a06a-4ef1-4abc-a578-fcb18ced422b-maas-api-service-ca\") pod \"3a54a06a-4ef1-4abc-a578-fcb18ced422b\" (UID: \"3a54a06a-4ef1-4abc-a578-fcb18ced422b\") "
Sep 10 17:49:56.696320 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:49:56.696296    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3a54a06a-4ef1-4abc-a578-fcb18ced422b-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "3a54a06a-4ef1-4abc-a578-fcb18ced422b" (UID: "3a54a06a-4ef1-4abc-a578-fcb18ced422b"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 17:49:56.797497 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:49:56.797458    2581 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/3a54a06a-4ef1-4abc-a578-fcb18ced422b-maas-api-service-ca\") on node \"ip-10-0-141-180.ec2.internal\" DevicePath \"\""
Sep 10 17:49:57.402285 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:49:57.402251    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-autores-51ed8d-znfkv"
Sep 10 17:49:57.402459 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:49:57.402281    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-autores-51ed8d-znfkv" event={"ID":"3a54a06a-4ef1-4abc-a578-fcb18ced422b","Type":"ContainerDied","Data":"678daf6f399baccda80ebee6ab9c77228d6f76b68f349c272b4063dbf181a4d6"}
Sep 10 17:49:57.402459 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:49:57.402330    2581 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="678daf6f399baccda80ebee6ab9c77228d6f76b68f349c272b4063dbf181a4d6"
Sep 10 17:50:05.541413 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:50:05.541379    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-autores-51ed8d-7d4f74f6ff-m2dh4"]
Sep 10 17:50:05.541947 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:50:05.541591    2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-autores-51ed8d-7d4f74f6ff-m2dh4" podUID="bf92bad5-3350-4228-bf29-c83360577fc2" containerName="payload-processing" containerID="cri-o://4ca1c9cd982293601eb4f985bbb797145166fe0043741f66dea6cdcde999fb07" gracePeriod=30
Sep 10 17:50:05.813892 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:50:05.813779    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-autores-51ed8d-znfkv"]
Sep 10 17:50:05.818349 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:50:05.818321    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-autores-51ed8d-znfkv"]
Sep 10 17:50:05.845361 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:50:05.845335    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-autores-51ed8d-7d4f74f6ff-m2dh4"
Sep 10 17:50:05.988786 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:50:05.988754    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-bhgk5\" (UniqueName: \"kubernetes.io/projected/bf92bad5-3350-4228-bf29-c83360577fc2-kube-api-access-bhgk5\") pod \"bf92bad5-3350-4228-bf29-c83360577fc2\" (UID: \"bf92bad5-3350-4228-bf29-c83360577fc2\") "
Sep 10 17:50:05.988956 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:50:05.988856    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/bf92bad5-3350-4228-bf29-c83360577fc2-plugins-config-volume\") pod \"bf92bad5-3350-4228-bf29-c83360577fc2\" (UID: \"bf92bad5-3350-4228-bf29-c83360577fc2\") "
Sep 10 17:50:05.989229 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:50:05.989199    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/bf92bad5-3350-4228-bf29-c83360577fc2-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "bf92bad5-3350-4228-bf29-c83360577fc2" (UID: "bf92bad5-3350-4228-bf29-c83360577fc2"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 17:50:05.989346 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:50:05.989317    2581 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/bf92bad5-3350-4228-bf29-c83360577fc2-plugins-config-volume\") on node \"ip-10-0-141-180.ec2.internal\" DevicePath \"\""
Sep 10 17:50:05.991139 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:50:05.991118    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/bf92bad5-3350-4228-bf29-c83360577fc2-kube-api-access-bhgk5" (OuterVolumeSpecName: "kube-api-access-bhgk5") pod "bf92bad5-3350-4228-bf29-c83360577fc2" (UID: "bf92bad5-3350-4228-bf29-c83360577fc2"). InnerVolumeSpecName "kube-api-access-bhgk5". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 17:50:06.090284 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:50:06.090200    2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-bhgk5\" (UniqueName: \"kubernetes.io/projected/bf92bad5-3350-4228-bf29-c83360577fc2-kube-api-access-bhgk5\") on node \"ip-10-0-141-180.ec2.internal\" DevicePath \"\""
Sep 10 17:50:06.470485 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:50:06.470444    2581 generic.go:356] "Generic (PLEG): container finished" podID="bf92bad5-3350-4228-bf29-c83360577fc2" containerID="4ca1c9cd982293601eb4f985bbb797145166fe0043741f66dea6cdcde999fb07" exitCode=0
Sep 10 17:50:06.470724 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:50:06.470513    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-autores-51ed8d-7d4f74f6ff-m2dh4"
Sep 10 17:50:06.470724 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:50:06.470525    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-autores-51ed8d-7d4f74f6ff-m2dh4" event={"ID":"bf92bad5-3350-4228-bf29-c83360577fc2","Type":"ContainerDied","Data":"4ca1c9cd982293601eb4f985bbb797145166fe0043741f66dea6cdcde999fb07"}
Sep 10 17:50:06.470724 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:50:06.470612    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-autores-51ed8d-7d4f74f6ff-m2dh4" event={"ID":"bf92bad5-3350-4228-bf29-c83360577fc2","Type":"ContainerDied","Data":"3ea5d4ef550d3a5e6d522130ac606a2d443f5f08ce39487f03373a02d66069ef"}
Sep 10 17:50:06.470724 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:50:06.470654    2581 scope.go:117] "RemoveContainer" containerID="4ca1c9cd982293601eb4f985bbb797145166fe0043741f66dea6cdcde999fb07"
Sep 10 17:50:06.481474 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:50:06.481452    2581 scope.go:117] "RemoveContainer" containerID="4ca1c9cd982293601eb4f985bbb797145166fe0043741f66dea6cdcde999fb07"
Sep 10 17:50:06.481823 ip-10-0-141-180 kubenswrapper[2581]: E0910 17:50:06.481803    2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4ca1c9cd982293601eb4f985bbb797145166fe0043741f66dea6cdcde999fb07\": container with ID starting with 4ca1c9cd982293601eb4f985bbb797145166fe0043741f66dea6cdcde999fb07 not found: ID does not exist" containerID="4ca1c9cd982293601eb4f985bbb797145166fe0043741f66dea6cdcde999fb07"
Sep 10 17:50:06.481887 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:50:06.481834    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4ca1c9cd982293601eb4f985bbb797145166fe0043741f66dea6cdcde999fb07"} err="failed to get container status \"4ca1c9cd982293601eb4f985bbb797145166fe0043741f66dea6cdcde999fb07\": rpc error: code = NotFound desc = could not find container \"4ca1c9cd982293601eb4f985bbb797145166fe0043741f66dea6cdcde999fb07\": container with ID starting with 4ca1c9cd982293601eb4f985bbb797145166fe0043741f66dea6cdcde999fb07 not found: ID does not exist"
Sep 10 17:50:06.496843 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:50:06.496809    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-autores-51ed8d-7d4f74f6ff-m2dh4"]
Sep 10 17:50:06.516773 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:50:06.516740    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-autores-51ed8d-7d4f74f6ff-m2dh4"]
Sep 10 17:50:06.862609 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:50:06.862530    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3a54a06a-4ef1-4abc-a578-fcb18ced422b" path="/var/lib/kubelet/pods/3a54a06a-4ef1-4abc-a578-fcb18ced422b/volumes"
Sep 10 17:50:06.863001 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:50:06.862919    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="bf92bad5-3350-4228-bf29-c83360577fc2" path="/var/lib/kubelet/pods/bf92bad5-3350-4228-bf29-c83360577fc2/volumes"
Sep 10 17:50:06.955221 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:50:06.955183    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-0ae95d09-vbbwf"]
Sep 10 17:50:06.955739 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:50:06.955721    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3a54a06a-4ef1-4abc-a578-fcb18ced422b" containerName="revoke-keys"
Sep 10 17:50:06.955840 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:50:06.955741    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="3a54a06a-4ef1-4abc-a578-fcb18ced422b" containerName="revoke-keys"
Sep 10 17:50:06.955840 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:50:06.955762    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="bf92bad5-3350-4228-bf29-c83360577fc2" containerName="payload-processing"
Sep 10 17:50:06.955840 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:50:06.955770    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="bf92bad5-3350-4228-bf29-c83360577fc2" containerName="payload-processing"
Sep 10 17:50:06.956001 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:50:06.955915    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="bf92bad5-3350-4228-bf29-c83360577fc2" containerName="payload-processing"
Sep 10 17:50:06.956001 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:50:06.955930    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="3a54a06a-4ef1-4abc-a578-fcb18ced422b" containerName="revoke-keys"
Sep 10 17:50:06.959084 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:50:06.959058    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-0ae95d09-vbbwf"
Sep 10 17:50:06.961812 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:50:06.961793    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 10 17:50:06.961812 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:50:06.961805    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-k7zj6\""
Sep 10 17:50:06.967765 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:50:06.967737    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-0ae95d09-vbbwf"]
Sep 10 17:50:06.999505 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:50:06.999471    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/561c30ba-aea0-4f22-91de-153b496033c1-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-0ae95d09-vbbwf\" (UID: \"561c30ba-aea0-4f22-91de-153b496033c1\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-0ae95d09-vbbwf"
Sep 10 17:50:07.100102 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:50:07.100074    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/561c30ba-aea0-4f22-91de-153b496033c1-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-0ae95d09-vbbwf\" (UID: \"561c30ba-aea0-4f22-91de-153b496033c1\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-0ae95d09-vbbwf"
Sep 10 17:50:07.100669 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:50:07.100616    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/561c30ba-aea0-4f22-91de-153b496033c1-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-0ae95d09-vbbwf\" (UID: \"561c30ba-aea0-4f22-91de-153b496033c1\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-0ae95d09-vbbwf"
Sep 10 17:50:07.272255 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:50:07.272221    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-0ae95d09-vbbwf"
Sep 10 17:50:07.545784 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:50:07.545755    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-0ae95d09-vbbwf"]
Sep 10 17:50:07.547113 ip-10-0-141-180 kubenswrapper[2581]: W0910 17:50:07.547088    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod561c30ba_aea0_4f22_91de_153b496033c1.slice/crio-3aead621c2553119a3ee3dbba6b0f33eaaa84e11479df3343ae4d39f1f0ab527 WatchSource:0}: Error finding container 3aead621c2553119a3ee3dbba6b0f33eaaa84e11479df3343ae4d39f1f0ab527: Status 404 returned error can't find the container with id 3aead621c2553119a3ee3dbba6b0f33eaaa84e11479df3343ae4d39f1f0ab527
Sep 10 17:50:08.486553 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:50:08.486516    2581 generic.go:356] "Generic (PLEG): container finished" podID="561c30ba-aea0-4f22-91de-153b496033c1" containerID="e8b8a4a94ad0e94c71d9fc7fa65815c0d652113b37059323a8e84f329ef4a567" exitCode=0
Sep 10 17:50:08.486961 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:50:08.486601    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-0ae95d09-vbbwf" event={"ID":"561c30ba-aea0-4f22-91de-153b496033c1","Type":"ContainerDied","Data":"e8b8a4a94ad0e94c71d9fc7fa65815c0d652113b37059323a8e84f329ef4a567"}
Sep 10 17:50:08.486961 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:50:08.486684    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-0ae95d09-vbbwf" event={"ID":"561c30ba-aea0-4f22-91de-153b496033c1","Type":"ContainerStarted","Data":"3aead621c2553119a3ee3dbba6b0f33eaaa84e11479df3343ae4d39f1f0ab527"}
Sep 10 17:50:09.632154 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:50:09.632126    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-0ae95d09-vbbwf"
Sep 10 17:50:09.725612 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:50:09.725577    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/561c30ba-aea0-4f22-91de-153b496033c1-maas-api-service-ca\") pod \"561c30ba-aea0-4f22-91de-153b496033c1\" (UID: \"561c30ba-aea0-4f22-91de-153b496033c1\") "
Sep 10 17:50:09.725966 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:50:09.725943    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/561c30ba-aea0-4f22-91de-153b496033c1-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "561c30ba-aea0-4f22-91de-153b496033c1" (UID: "561c30ba-aea0-4f22-91de-153b496033c1"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 17:50:09.827475 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:50:09.827380    2581 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/561c30ba-aea0-4f22-91de-153b496033c1-maas-api-service-ca\") on node \"ip-10-0-141-180.ec2.internal\" DevicePath \"\""
Sep 10 17:50:10.499661 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:50:10.499606    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-0ae95d09-vbbwf" event={"ID":"561c30ba-aea0-4f22-91de-153b496033c1","Type":"ContainerDied","Data":"3aead621c2553119a3ee3dbba6b0f33eaaa84e11479df3343ae4d39f1f0ab527"}
Sep 10 17:50:10.499807 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:50:10.499678    2581 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="3aead621c2553119a3ee3dbba6b0f33eaaa84e11479df3343ae4d39f1f0ab527"
Sep 10 17:50:10.499807 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:50:10.499622    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-0ae95d09-vbbwf"
Sep 10 17:50:12.986008 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:50:12.985932    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-0ae95d09-96b97ffc-6mbd9"]
Sep 10 17:50:12.986531 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:50:12.986235    2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-ait-0ae95d09-96b97ffc-6mbd9" podUID="09cf427e-942e-4183-b063-18ac4d36ec2a" containerName="payload-processing" containerID="cri-o://cd977e3c3e768f0c1e9fb8a9d788dbb0580f681fe9bf163459daebc6bd0100b5" gracePeriod=30
Sep 10 17:50:13.274468 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:50:13.274442    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-0ae95d09-96b97ffc-6mbd9"
Sep 10 17:50:13.331077 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:50:13.331046    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-0ae95d09-vbbwf"]
Sep 10 17:50:13.339968 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:50:13.339933    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-0ae95d09-vbbwf"]
Sep 10 17:50:13.367548 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:50:13.367518    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/09cf427e-942e-4183-b063-18ac4d36ec2a-plugins-config-volume\") pod \"09cf427e-942e-4183-b063-18ac4d36ec2a\" (UID: \"09cf427e-942e-4183-b063-18ac4d36ec2a\") "
Sep 10 17:50:13.367700 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:50:13.367563    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-kp8d4\" (UniqueName: \"kubernetes.io/projected/09cf427e-942e-4183-b063-18ac4d36ec2a-kube-api-access-kp8d4\") pod \"09cf427e-942e-4183-b063-18ac4d36ec2a\" (UID: \"09cf427e-942e-4183-b063-18ac4d36ec2a\") "
Sep 10 17:50:13.367922 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:50:13.367898    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/09cf427e-942e-4183-b063-18ac4d36ec2a-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "09cf427e-942e-4183-b063-18ac4d36ec2a" (UID: "09cf427e-942e-4183-b063-18ac4d36ec2a"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 17:50:13.369632 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:50:13.369612    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/09cf427e-942e-4183-b063-18ac4d36ec2a-kube-api-access-kp8d4" (OuterVolumeSpecName: "kube-api-access-kp8d4") pod "09cf427e-942e-4183-b063-18ac4d36ec2a" (UID: "09cf427e-942e-4183-b063-18ac4d36ec2a"). InnerVolumeSpecName "kube-api-access-kp8d4". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 17:50:13.468658 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:50:13.468614    2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-kp8d4\" (UniqueName: \"kubernetes.io/projected/09cf427e-942e-4183-b063-18ac4d36ec2a-kube-api-access-kp8d4\") on node \"ip-10-0-141-180.ec2.internal\" DevicePath \"\""
Sep 10 17:50:13.468816 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:50:13.468680    2581 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/09cf427e-942e-4183-b063-18ac4d36ec2a-plugins-config-volume\") on node \"ip-10-0-141-180.ec2.internal\" DevicePath \"\""
Sep 10 17:50:13.519752 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:50:13.519671    2581 generic.go:356] "Generic (PLEG): container finished" podID="09cf427e-942e-4183-b063-18ac4d36ec2a" containerID="cd977e3c3e768f0c1e9fb8a9d788dbb0580f681fe9bf163459daebc6bd0100b5" exitCode=0
Sep 10 17:50:13.519752 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:50:13.519720    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-0ae95d09-96b97ffc-6mbd9"
Sep 10 17:50:13.519968 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:50:13.519720    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-0ae95d09-96b97ffc-6mbd9" event={"ID":"09cf427e-942e-4183-b063-18ac4d36ec2a","Type":"ContainerDied","Data":"cd977e3c3e768f0c1e9fb8a9d788dbb0580f681fe9bf163459daebc6bd0100b5"}
Sep 10 17:50:13.519968 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:50:13.519866    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-0ae95d09-96b97ffc-6mbd9" event={"ID":"09cf427e-942e-4183-b063-18ac4d36ec2a","Type":"ContainerDied","Data":"29b6e095b0b1e9d2765815fe1185d98d809d34929ed6623539149bb7b7816d57"}
Sep 10 17:50:13.519968 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:50:13.519906    2581 scope.go:117] "RemoveContainer" containerID="cd977e3c3e768f0c1e9fb8a9d788dbb0580f681fe9bf163459daebc6bd0100b5"
Sep 10 17:50:13.530218 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:50:13.530194    2581 scope.go:117] "RemoveContainer" containerID="cd977e3c3e768f0c1e9fb8a9d788dbb0580f681fe9bf163459daebc6bd0100b5"
Sep 10 17:50:13.530491 ip-10-0-141-180 kubenswrapper[2581]: E0910 17:50:13.530466    2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"cd977e3c3e768f0c1e9fb8a9d788dbb0580f681fe9bf163459daebc6bd0100b5\": container with ID starting with cd977e3c3e768f0c1e9fb8a9d788dbb0580f681fe9bf163459daebc6bd0100b5 not found: ID does not exist" containerID="cd977e3c3e768f0c1e9fb8a9d788dbb0580f681fe9bf163459daebc6bd0100b5"
Sep 10 17:50:13.530549 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:50:13.530501    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"cd977e3c3e768f0c1e9fb8a9d788dbb0580f681fe9bf163459daebc6bd0100b5"} err="failed to get container status \"cd977e3c3e768f0c1e9fb8a9d788dbb0580f681fe9bf163459daebc6bd0100b5\": rpc error: code = NotFound desc = could not find container \"cd977e3c3e768f0c1e9fb8a9d788dbb0580f681fe9bf163459daebc6bd0100b5\": container with ID starting with cd977e3c3e768f0c1e9fb8a9d788dbb0580f681fe9bf163459daebc6bd0100b5 not found: ID does not exist"
Sep 10 17:50:13.546396 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:50:13.546375    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-0ae95d09-96b97ffc-6mbd9"]
Sep 10 17:50:13.552681 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:50:13.552659    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-0ae95d09-96b97ffc-6mbd9"]
Sep 10 17:50:14.862391 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:50:14.862356    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="09cf427e-942e-4183-b063-18ac4d36ec2a" path="/var/lib/kubelet/pods/09cf427e-942e-4183-b063-18ac4d36ec2a/volumes"
Sep 10 17:50:14.862769 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:50:14.862755    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="561c30ba-aea0-4f22-91de-153b496033c1" path="/var/lib/kubelet/pods/561c30ba-aea0-4f22-91de-153b496033c1/volumes"
Sep 10 17:50:20.420145 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:50:20.420109    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-derive-d2a8a5f7-56554bddbb-vntsd"]
Sep 10 17:50:20.420876 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:50:20.420855    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="09cf427e-942e-4183-b063-18ac4d36ec2a" containerName="payload-processing"
Sep 10 17:50:20.420931 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:50:20.420881    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="09cf427e-942e-4183-b063-18ac4d36ec2a" containerName="payload-processing"
Sep 10 17:50:20.420931 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:50:20.420906    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="561c30ba-aea0-4f22-91de-153b496033c1" containerName="revoke-keys"
Sep 10 17:50:20.420931 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:50:20.420915    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="561c30ba-aea0-4f22-91de-153b496033c1" containerName="revoke-keys"
Sep 10 17:50:20.421071 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:50:20.421050    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="561c30ba-aea0-4f22-91de-153b496033c1" containerName="revoke-keys"
Sep 10 17:50:20.421071 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:50:20.421072    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="09cf427e-942e-4183-b063-18ac4d36ec2a" containerName="payload-processing"
Sep 10 17:50:20.423846 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:50:20.423828    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-derive-d2a8a5f7-56554bddbb-vntsd"
Sep 10 17:50:20.427231 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:50:20.427212    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-derive-d2a8a5f7\""
Sep 10 17:50:20.428364 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:50:20.428345    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-derive-d2a8a5f7-dockercfg-l6qtw\""
Sep 10 17:50:20.435865 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:50:20.435839    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-derive-d2a8a5f7-56554bddbb-vntsd"]
Sep 10 17:50:20.543894 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:50:20.543848    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tn5zd\" (UniqueName: \"kubernetes.io/projected/d6e2640c-1099-48e3-a2cb-7546cbc76d8f-kube-api-access-tn5zd\") pod \"payload-pre-processing-e2e-derive-d2a8a5f7-56554bddbb-vntsd\" (UID: \"d6e2640c-1099-48e3-a2cb-7546cbc76d8f\") " pod="openshift-ingress/payload-pre-processing-e2e-derive-d2a8a5f7-56554bddbb-vntsd"
Sep 10 17:50:20.544087 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:50:20.543968    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d6e2640c-1099-48e3-a2cb-7546cbc76d8f-plugins-config-volume\") pod \"payload-pre-processing-e2e-derive-d2a8a5f7-56554bddbb-vntsd\" (UID: \"d6e2640c-1099-48e3-a2cb-7546cbc76d8f\") " pod="openshift-ingress/payload-pre-processing-e2e-derive-d2a8a5f7-56554bddbb-vntsd"
Sep 10 17:50:20.645512 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:50:20.645473    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tn5zd\" (UniqueName: \"kubernetes.io/projected/d6e2640c-1099-48e3-a2cb-7546cbc76d8f-kube-api-access-tn5zd\") pod \"payload-pre-processing-e2e-derive-d2a8a5f7-56554bddbb-vntsd\" (UID: \"d6e2640c-1099-48e3-a2cb-7546cbc76d8f\") " pod="openshift-ingress/payload-pre-processing-e2e-derive-d2a8a5f7-56554bddbb-vntsd"
Sep 10 17:50:20.645689 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:50:20.645527    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d6e2640c-1099-48e3-a2cb-7546cbc76d8f-plugins-config-volume\") pod \"payload-pre-processing-e2e-derive-d2a8a5f7-56554bddbb-vntsd\" (UID: \"d6e2640c-1099-48e3-a2cb-7546cbc76d8f\") " pod="openshift-ingress/payload-pre-processing-e2e-derive-d2a8a5f7-56554bddbb-vntsd"
Sep 10 17:50:20.646213 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:50:20.646194    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d6e2640c-1099-48e3-a2cb-7546cbc76d8f-plugins-config-volume\") pod \"payload-pre-processing-e2e-derive-d2a8a5f7-56554bddbb-vntsd\" (UID: \"d6e2640c-1099-48e3-a2cb-7546cbc76d8f\") " pod="openshift-ingress/payload-pre-processing-e2e-derive-d2a8a5f7-56554bddbb-vntsd"
Sep 10 17:50:20.654240 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:50:20.654212    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tn5zd\" (UniqueName: \"kubernetes.io/projected/d6e2640c-1099-48e3-a2cb-7546cbc76d8f-kube-api-access-tn5zd\") pod \"payload-pre-processing-e2e-derive-d2a8a5f7-56554bddbb-vntsd\" (UID: \"d6e2640c-1099-48e3-a2cb-7546cbc76d8f\") " pod="openshift-ingress/payload-pre-processing-e2e-derive-d2a8a5f7-56554bddbb-vntsd"
Sep 10 17:50:20.738912 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:50:20.738880    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-derive-d2a8a5f7-dockercfg-l6qtw\""
Sep 10 17:50:20.746820 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:50:20.746793    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-derive-d2a8a5f7-56554bddbb-vntsd"
Sep 10 17:50:21.027567 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:50:21.027531    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-derive-d2a8a5f7-56554bddbb-vntsd"]
Sep 10 17:50:21.029465 ip-10-0-141-180 kubenswrapper[2581]: W0910 17:50:21.029426    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd6e2640c_1099_48e3_a2cb_7546cbc76d8f.slice/crio-48208b1f6601fc49c57c267f03600c0af577ef0e986d974527c3312ce8a4691b WatchSource:0}: Error finding container 48208b1f6601fc49c57c267f03600c0af577ef0e986d974527c3312ce8a4691b: Status 404 returned error can't find the container with id 48208b1f6601fc49c57c267f03600c0af577ef0e986d974527c3312ce8a4691b
Sep 10 17:50:21.576980 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:50:21.576935    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-derive-d2a8a5f7-56554bddbb-vntsd" event={"ID":"d6e2640c-1099-48e3-a2cb-7546cbc76d8f","Type":"ContainerStarted","Data":"942e4da42671d38fb98ed58bbec2d240abbb375ccc7265ca733e2662d4430dd4"}
Sep 10 17:50:21.577380 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:50:21.577004    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-derive-d2a8a5f7-56554bddbb-vntsd" event={"ID":"d6e2640c-1099-48e3-a2cb-7546cbc76d8f","Type":"ContainerStarted","Data":"48208b1f6601fc49c57c267f03600c0af577ef0e986d974527c3312ce8a4691b"}
Sep 10 17:50:21.577380 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:50:21.577054    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-derive-d2a8a5f7-56554bddbb-vntsd"
Sep 10 17:50:21.595770 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:50:21.595721    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-derive-d2a8a5f7-56554bddbb-vntsd" podStartSLOduration=1.595704569 podStartE2EDuration="1.595704569s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 17:50:20 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 17:50:21.593088109 +0000 UTC m=+841.247162010" watchObservedRunningTime="2026-09-10 17:50:21.595704569 +0000 UTC m=+841.249778473"
Sep 10 17:50:32.585489 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:50:32.585453    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-derive-d2a8a5f7-56554bddbb-vntsd"
Sep 10 17:50:43.216959 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:50:43.216923    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-d2a8a5f7-crhsf"]
Sep 10 17:50:43.221601 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:50:43.221583    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-d2a8a5f7-crhsf"
Sep 10 17:50:43.225012 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:50:43.224990    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-k7zj6\""
Sep 10 17:50:43.225107 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:50:43.225061    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 10 17:50:43.231186 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:50:43.231164    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-d2a8a5f7-crhsf"]
Sep 10 17:50:43.263215 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:50:43.263192    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/2c67c17a-a18d-480f-8437-1f0679c1d9c8-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-d2a8a5f7-crhsf\" (UID: \"2c67c17a-a18d-480f-8437-1f0679c1d9c8\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-d2a8a5f7-crhsf"
Sep 10 17:50:43.364346 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:50:43.364303    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/2c67c17a-a18d-480f-8437-1f0679c1d9c8-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-d2a8a5f7-crhsf\" (UID: \"2c67c17a-a18d-480f-8437-1f0679c1d9c8\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-d2a8a5f7-crhsf"
Sep 10 17:50:43.364962 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:50:43.364941    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/2c67c17a-a18d-480f-8437-1f0679c1d9c8-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-d2a8a5f7-crhsf\" (UID: \"2c67c17a-a18d-480f-8437-1f0679c1d9c8\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-d2a8a5f7-crhsf"
Sep 10 17:50:43.534274 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:50:43.534177    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-d2a8a5f7-crhsf"
Sep 10 17:50:43.810736 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:50:43.810708    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-d2a8a5f7-crhsf"]
Sep 10 17:50:43.812119 ip-10-0-141-180 kubenswrapper[2581]: W0910 17:50:43.812092    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2c67c17a_a18d_480f_8437_1f0679c1d9c8.slice/crio-6da12613e046c7cf3f25a3a63adf4027d8a90d0221f6dba479dea0488eb181a4 WatchSource:0}: Error finding container 6da12613e046c7cf3f25a3a63adf4027d8a90d0221f6dba479dea0488eb181a4: Status 404 returned error can't find the container with id 6da12613e046c7cf3f25a3a63adf4027d8a90d0221f6dba479dea0488eb181a4
Sep 10 17:50:44.729660 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:50:44.729575    2581 generic.go:356] "Generic (PLEG): container finished" podID="2c67c17a-a18d-480f-8437-1f0679c1d9c8" containerID="5718f0eb8ee5c7d49c213e7feadd7770c478cba1b2e051aa55380f85f815c65f" exitCode=0
Sep 10 17:50:44.730027 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:50:44.729668    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-d2a8a5f7-crhsf" event={"ID":"2c67c17a-a18d-480f-8437-1f0679c1d9c8","Type":"ContainerDied","Data":"5718f0eb8ee5c7d49c213e7feadd7770c478cba1b2e051aa55380f85f815c65f"}
Sep 10 17:50:44.730027 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:50:44.729724    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-d2a8a5f7-crhsf" event={"ID":"2c67c17a-a18d-480f-8437-1f0679c1d9c8","Type":"ContainerStarted","Data":"6da12613e046c7cf3f25a3a63adf4027d8a90d0221f6dba479dea0488eb181a4"}
Sep 10 17:50:45.873289 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:50:45.873266    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-d2a8a5f7-crhsf"
Sep 10 17:50:45.888608 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:50:45.888588    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/2c67c17a-a18d-480f-8437-1f0679c1d9c8-maas-api-service-ca\") pod \"2c67c17a-a18d-480f-8437-1f0679c1d9c8\" (UID: \"2c67c17a-a18d-480f-8437-1f0679c1d9c8\") "
Sep 10 17:50:45.888924 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:50:45.888899    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2c67c17a-a18d-480f-8437-1f0679c1d9c8-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "2c67c17a-a18d-480f-8437-1f0679c1d9c8" (UID: "2c67c17a-a18d-480f-8437-1f0679c1d9c8"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 17:50:45.990115 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:50:45.990039    2581 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/2c67c17a-a18d-480f-8437-1f0679c1d9c8-maas-api-service-ca\") on node \"ip-10-0-141-180.ec2.internal\" DevicePath \"\""
Sep 10 17:50:46.742538 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:50:46.742505    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-d2a8a5f7-crhsf"
Sep 10 17:50:46.742736 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:50:46.742543    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-d2a8a5f7-crhsf" event={"ID":"2c67c17a-a18d-480f-8437-1f0679c1d9c8","Type":"ContainerDied","Data":"6da12613e046c7cf3f25a3a63adf4027d8a90d0221f6dba479dea0488eb181a4"}
Sep 10 17:50:46.742736 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:50:46.742597    2581 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="6da12613e046c7cf3f25a3a63adf4027d8a90d0221f6dba479dea0488eb181a4"
Sep 10 17:50:49.316376 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:50:49.316329    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-autores-51ed8d-openshift-default-74f9875b8-4fftk"]
Sep 10 17:50:49.316901 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:50:49.316625    2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-autores-51ed8d-openshift-default-74f9875b8-4fftk" podUID="8a2891cb-620d-4e36-984b-0cc72cc6f5b9" containerName="istio-proxy" containerID="cri-o://1f9c700929ee0ca6c7f05ce88ed52f4ac6959c12d970938074baa18fede068c8" gracePeriod=30
Sep 10 17:50:50.248986 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:50:50.248951    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-explicit-c7995b-openshift-default-6ccb9db69b-6h6xh"]
Sep 10 17:50:50.249994 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:50:50.249959    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2c67c17a-a18d-480f-8437-1f0679c1d9c8" containerName="revoke-keys"
Sep 10 17:50:50.249994 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:50:50.249987    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="2c67c17a-a18d-480f-8437-1f0679c1d9c8" containerName="revoke-keys"
Sep 10 17:50:50.250181 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:50:50.250122    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="2c67c17a-a18d-480f-8437-1f0679c1d9c8" containerName="revoke-keys"
Sep 10 17:50:50.253474 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:50:50.253456    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-explicit-c7995b-openshift-default-6ccb9db69b-6h6xh"
Sep 10 17:50:50.258949 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:50:50.258929    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-explicit-c7995b-openshift-default-dockercfg-wzr44\""
Sep 10 17:50:50.265342 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:50:50.265295    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-derive-d2a8a5f7-56554bddbb-vntsd"]
Sep 10 17:50:50.265517 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:50:50.265497    2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-derive-d2a8a5f7-56554bddbb-vntsd" podUID="d6e2640c-1099-48e3-a2cb-7546cbc76d8f" containerName="payload-pre-processing" containerID="cri-o://942e4da42671d38fb98ed58bbec2d240abbb375ccc7265ca733e2662d4430dd4" gracePeriod=30
Sep 10 17:50:50.278588 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:50:50.278562    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-explicit-c7995b-openshift-default-6ccb9db69b-6h6xh"]
Sep 10 17:50:50.334022 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:50:50.333991    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/200bec9f-4a17-4e71-9d38-939cbb027551-workload-certs\") pod \"e2e-explicit-c7995b-openshift-default-6ccb9db69b-6h6xh\" (UID: \"200bec9f-4a17-4e71-9d38-939cbb027551\") " pod="openshift-ingress/e2e-explicit-c7995b-openshift-default-6ccb9db69b-6h6xh"
Sep 10 17:50:50.334373 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:50:50.334053    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/200bec9f-4a17-4e71-9d38-939cbb027551-workload-socket\") pod \"e2e-explicit-c7995b-openshift-default-6ccb9db69b-6h6xh\" (UID: \"200bec9f-4a17-4e71-9d38-939cbb027551\") " pod="openshift-ingress/e2e-explicit-c7995b-openshift-default-6ccb9db69b-6h6xh"
Sep 10 17:50:50.334373 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:50:50.334113    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/200bec9f-4a17-4e71-9d38-939cbb027551-istio-podinfo\") pod \"e2e-explicit-c7995b-openshift-default-6ccb9db69b-6h6xh\" (UID: \"200bec9f-4a17-4e71-9d38-939cbb027551\") " pod="openshift-ingress/e2e-explicit-c7995b-openshift-default-6ccb9db69b-6h6xh"
Sep 10 17:50:50.334373 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:50:50.334171    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/200bec9f-4a17-4e71-9d38-939cbb027551-istio-data\") pod \"e2e-explicit-c7995b-openshift-default-6ccb9db69b-6h6xh\" (UID: \"200bec9f-4a17-4e71-9d38-939cbb027551\") " pod="openshift-ingress/e2e-explicit-c7995b-openshift-default-6ccb9db69b-6h6xh"
Sep 10 17:50:50.334373 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:50:50.334205    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/200bec9f-4a17-4e71-9d38-939cbb027551-istio-envoy\") pod \"e2e-explicit-c7995b-openshift-default-6ccb9db69b-6h6xh\" (UID: \"200bec9f-4a17-4e71-9d38-939cbb027551\") " pod="openshift-ingress/e2e-explicit-c7995b-openshift-default-6ccb9db69b-6h6xh"
Sep 10 17:50:50.334373 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:50:50.334225    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/200bec9f-4a17-4e71-9d38-939cbb027551-istio-token\") pod \"e2e-explicit-c7995b-openshift-default-6ccb9db69b-6h6xh\" (UID: \"200bec9f-4a17-4e71-9d38-939cbb027551\") " pod="openshift-ingress/e2e-explicit-c7995b-openshift-default-6ccb9db69b-6h6xh"
Sep 10 17:50:50.334373 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:50:50.334254    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/200bec9f-4a17-4e71-9d38-939cbb027551-credential-socket\") pod \"e2e-explicit-c7995b-openshift-default-6ccb9db69b-6h6xh\" (UID: \"200bec9f-4a17-4e71-9d38-939cbb027551\") " pod="openshift-ingress/e2e-explicit-c7995b-openshift-default-6ccb9db69b-6h6xh"
Sep 10 17:50:50.334373 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:50:50.334275    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/200bec9f-4a17-4e71-9d38-939cbb027551-istiod-ca-cert\") pod \"e2e-explicit-c7995b-openshift-default-6ccb9db69b-6h6xh\" (UID: \"200bec9f-4a17-4e71-9d38-939cbb027551\") " pod="openshift-ingress/e2e-explicit-c7995b-openshift-default-6ccb9db69b-6h6xh"
Sep 10 17:50:50.334373 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:50:50.334330    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-w5l9b\" (UniqueName: \"kubernetes.io/projected/200bec9f-4a17-4e71-9d38-939cbb027551-kube-api-access-w5l9b\") pod \"e2e-explicit-c7995b-openshift-default-6ccb9db69b-6h6xh\" (UID: \"200bec9f-4a17-4e71-9d38-939cbb027551\") " pod="openshift-ingress/e2e-explicit-c7995b-openshift-default-6ccb9db69b-6h6xh"
Sep 10 17:50:50.435410 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:50:50.435374    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/200bec9f-4a17-4e71-9d38-939cbb027551-istio-data\") pod \"e2e-explicit-c7995b-openshift-default-6ccb9db69b-6h6xh\" (UID: \"200bec9f-4a17-4e71-9d38-939cbb027551\") " pod="openshift-ingress/e2e-explicit-c7995b-openshift-default-6ccb9db69b-6h6xh"
Sep 10 17:50:50.435410 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:50:50.435419    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/200bec9f-4a17-4e71-9d38-939cbb027551-istio-envoy\") pod \"e2e-explicit-c7995b-openshift-default-6ccb9db69b-6h6xh\" (UID: \"200bec9f-4a17-4e71-9d38-939cbb027551\") " pod="openshift-ingress/e2e-explicit-c7995b-openshift-default-6ccb9db69b-6h6xh"
Sep 10 17:50:50.435680 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:50:50.435440    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/200bec9f-4a17-4e71-9d38-939cbb027551-istio-token\") pod \"e2e-explicit-c7995b-openshift-default-6ccb9db69b-6h6xh\" (UID: \"200bec9f-4a17-4e71-9d38-939cbb027551\") " pod="openshift-ingress/e2e-explicit-c7995b-openshift-default-6ccb9db69b-6h6xh"
Sep 10 17:50:50.435680 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:50:50.435484    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/200bec9f-4a17-4e71-9d38-939cbb027551-credential-socket\") pod \"e2e-explicit-c7995b-openshift-default-6ccb9db69b-6h6xh\" (UID: \"200bec9f-4a17-4e71-9d38-939cbb027551\") " pod="openshift-ingress/e2e-explicit-c7995b-openshift-default-6ccb9db69b-6h6xh"
Sep 10 17:50:50.435680 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:50:50.435518    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/200bec9f-4a17-4e71-9d38-939cbb027551-istiod-ca-cert\") pod \"e2e-explicit-c7995b-openshift-default-6ccb9db69b-6h6xh\" (UID: \"200bec9f-4a17-4e71-9d38-939cbb027551\") " pod="openshift-ingress/e2e-explicit-c7995b-openshift-default-6ccb9db69b-6h6xh"
Sep 10 17:50:50.435680 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:50:50.435581    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-w5l9b\" (UniqueName: \"kubernetes.io/projected/200bec9f-4a17-4e71-9d38-939cbb027551-kube-api-access-w5l9b\") pod \"e2e-explicit-c7995b-openshift-default-6ccb9db69b-6h6xh\" (UID: \"200bec9f-4a17-4e71-9d38-939cbb027551\") " pod="openshift-ingress/e2e-explicit-c7995b-openshift-default-6ccb9db69b-6h6xh"
Sep 10 17:50:50.435680 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:50:50.435635    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/200bec9f-4a17-4e71-9d38-939cbb027551-workload-certs\") pod \"e2e-explicit-c7995b-openshift-default-6ccb9db69b-6h6xh\" (UID: \"200bec9f-4a17-4e71-9d38-939cbb027551\") " pod="openshift-ingress/e2e-explicit-c7995b-openshift-default-6ccb9db69b-6h6xh"
Sep 10 17:50:50.435968 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:50:50.435729    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/200bec9f-4a17-4e71-9d38-939cbb027551-workload-socket\") pod \"e2e-explicit-c7995b-openshift-default-6ccb9db69b-6h6xh\" (UID: \"200bec9f-4a17-4e71-9d38-939cbb027551\") " pod="openshift-ingress/e2e-explicit-c7995b-openshift-default-6ccb9db69b-6h6xh"
Sep 10 17:50:50.435968 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:50:50.435775    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/200bec9f-4a17-4e71-9d38-939cbb027551-istio-podinfo\") pod \"e2e-explicit-c7995b-openshift-default-6ccb9db69b-6h6xh\" (UID: \"200bec9f-4a17-4e71-9d38-939cbb027551\") " pod="openshift-ingress/e2e-explicit-c7995b-openshift-default-6ccb9db69b-6h6xh"
Sep 10 17:50:50.435968 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:50:50.435914    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/200bec9f-4a17-4e71-9d38-939cbb027551-istio-data\") pod \"e2e-explicit-c7995b-openshift-default-6ccb9db69b-6h6xh\" (UID: \"200bec9f-4a17-4e71-9d38-939cbb027551\") " pod="openshift-ingress/e2e-explicit-c7995b-openshift-default-6ccb9db69b-6h6xh"
Sep 10 17:50:50.435968 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:50:50.435936    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/200bec9f-4a17-4e71-9d38-939cbb027551-credential-socket\") pod \"e2e-explicit-c7995b-openshift-default-6ccb9db69b-6h6xh\" (UID: \"200bec9f-4a17-4e71-9d38-939cbb027551\") " pod="openshift-ingress/e2e-explicit-c7995b-openshift-default-6ccb9db69b-6h6xh"
Sep 10 17:50:50.436267 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:50:50.436133    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/200bec9f-4a17-4e71-9d38-939cbb027551-workload-certs\") pod \"e2e-explicit-c7995b-openshift-default-6ccb9db69b-6h6xh\" (UID: \"200bec9f-4a17-4e71-9d38-939cbb027551\") " pod="openshift-ingress/e2e-explicit-c7995b-openshift-default-6ccb9db69b-6h6xh"
Sep 10 17:50:50.436267 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:50:50.436178    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/200bec9f-4a17-4e71-9d38-939cbb027551-workload-socket\") pod \"e2e-explicit-c7995b-openshift-default-6ccb9db69b-6h6xh\" (UID: \"200bec9f-4a17-4e71-9d38-939cbb027551\") " pod="openshift-ingress/e2e-explicit-c7995b-openshift-default-6ccb9db69b-6h6xh"
Sep 10 17:50:50.436449 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:50:50.436426    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/200bec9f-4a17-4e71-9d38-939cbb027551-istiod-ca-cert\") pod \"e2e-explicit-c7995b-openshift-default-6ccb9db69b-6h6xh\" (UID: \"200bec9f-4a17-4e71-9d38-939cbb027551\") " pod="openshift-ingress/e2e-explicit-c7995b-openshift-default-6ccb9db69b-6h6xh"
Sep 10 17:50:50.438658 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:50:50.438616    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/200bec9f-4a17-4e71-9d38-939cbb027551-istio-envoy\") pod \"e2e-explicit-c7995b-openshift-default-6ccb9db69b-6h6xh\" (UID: \"200bec9f-4a17-4e71-9d38-939cbb027551\") " pod="openshift-ingress/e2e-explicit-c7995b-openshift-default-6ccb9db69b-6h6xh"
Sep 10 17:50:50.439862 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:50:50.438889    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/200bec9f-4a17-4e71-9d38-939cbb027551-istio-podinfo\") pod \"e2e-explicit-c7995b-openshift-default-6ccb9db69b-6h6xh\" (UID: \"200bec9f-4a17-4e71-9d38-939cbb027551\") " pod="openshift-ingress/e2e-explicit-c7995b-openshift-default-6ccb9db69b-6h6xh"
Sep 10 17:50:50.449683 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:50:50.449648    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/200bec9f-4a17-4e71-9d38-939cbb027551-istio-token\") pod \"e2e-explicit-c7995b-openshift-default-6ccb9db69b-6h6xh\" (UID: \"200bec9f-4a17-4e71-9d38-939cbb027551\") " pod="openshift-ingress/e2e-explicit-c7995b-openshift-default-6ccb9db69b-6h6xh"
Sep 10 17:50:50.450198 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:50:50.450156    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-w5l9b\" (UniqueName: \"kubernetes.io/projected/200bec9f-4a17-4e71-9d38-939cbb027551-kube-api-access-w5l9b\") pod \"e2e-explicit-c7995b-openshift-default-6ccb9db69b-6h6xh\" (UID: \"200bec9f-4a17-4e71-9d38-939cbb027551\") " pod="openshift-ingress/e2e-explicit-c7995b-openshift-default-6ccb9db69b-6h6xh"
Sep 10 17:50:50.528485 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:50:50.528457    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-derive-d2a8a5f7-56554bddbb-vntsd"
Sep 10 17:50:50.541649 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:50:50.541600    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-d2a8a5f7-crhsf"]
Sep 10 17:50:50.547503 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:50:50.547473    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-d2a8a5f7-crhsf"]
Sep 10 17:50:50.564611 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:50:50.564582    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-explicit-c7995b-openshift-default-6ccb9db69b-6h6xh"
Sep 10 17:50:50.637135 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:50:50.637109    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-tn5zd\" (UniqueName: \"kubernetes.io/projected/d6e2640c-1099-48e3-a2cb-7546cbc76d8f-kube-api-access-tn5zd\") pod \"d6e2640c-1099-48e3-a2cb-7546cbc76d8f\" (UID: \"d6e2640c-1099-48e3-a2cb-7546cbc76d8f\") "
Sep 10 17:50:50.637270 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:50:50.637156    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d6e2640c-1099-48e3-a2cb-7546cbc76d8f-plugins-config-volume\") pod \"d6e2640c-1099-48e3-a2cb-7546cbc76d8f\" (UID: \"d6e2640c-1099-48e3-a2cb-7546cbc76d8f\") "
Sep 10 17:50:50.637567 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:50:50.637543    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d6e2640c-1099-48e3-a2cb-7546cbc76d8f-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "d6e2640c-1099-48e3-a2cb-7546cbc76d8f" (UID: "d6e2640c-1099-48e3-a2cb-7546cbc76d8f"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 17:50:50.639515 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:50:50.639494    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d6e2640c-1099-48e3-a2cb-7546cbc76d8f-kube-api-access-tn5zd" (OuterVolumeSpecName: "kube-api-access-tn5zd") pod "d6e2640c-1099-48e3-a2cb-7546cbc76d8f" (UID: "d6e2640c-1099-48e3-a2cb-7546cbc76d8f"). InnerVolumeSpecName "kube-api-access-tn5zd". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 17:50:50.738803 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:50:50.738766    2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-tn5zd\" (UniqueName: \"kubernetes.io/projected/d6e2640c-1099-48e3-a2cb-7546cbc76d8f-kube-api-access-tn5zd\") on node \"ip-10-0-141-180.ec2.internal\" DevicePath \"\""
Sep 10 17:50:50.738803 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:50:50.738797    2581 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d6e2640c-1099-48e3-a2cb-7546cbc76d8f-plugins-config-volume\") on node \"ip-10-0-141-180.ec2.internal\" DevicePath \"\""
Sep 10 17:50:50.768722 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:50:50.768631    2581 generic.go:356] "Generic (PLEG): container finished" podID="d6e2640c-1099-48e3-a2cb-7546cbc76d8f" containerID="942e4da42671d38fb98ed58bbec2d240abbb375ccc7265ca733e2662d4430dd4" exitCode=0
Sep 10 17:50:50.768722 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:50:50.768699    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-derive-d2a8a5f7-56554bddbb-vntsd"
Sep 10 17:50:50.768872 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:50:50.768696    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-derive-d2a8a5f7-56554bddbb-vntsd" event={"ID":"d6e2640c-1099-48e3-a2cb-7546cbc76d8f","Type":"ContainerDied","Data":"942e4da42671d38fb98ed58bbec2d240abbb375ccc7265ca733e2662d4430dd4"}
Sep 10 17:50:50.768872 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:50:50.768824    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-derive-d2a8a5f7-56554bddbb-vntsd" event={"ID":"d6e2640c-1099-48e3-a2cb-7546cbc76d8f","Type":"ContainerDied","Data":"48208b1f6601fc49c57c267f03600c0af577ef0e986d974527c3312ce8a4691b"}
Sep 10 17:50:50.768872 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:50:50.768846    2581 scope.go:117] "RemoveContainer" containerID="942e4da42671d38fb98ed58bbec2d240abbb375ccc7265ca733e2662d4430dd4"
Sep 10 17:50:50.783204 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:50:50.782217    2581 scope.go:117] "RemoveContainer" containerID="942e4da42671d38fb98ed58bbec2d240abbb375ccc7265ca733e2662d4430dd4"
Sep 10 17:50:50.785600 ip-10-0-141-180 kubenswrapper[2581]: E0910 17:50:50.784068    2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"942e4da42671d38fb98ed58bbec2d240abbb375ccc7265ca733e2662d4430dd4\": container with ID starting with 942e4da42671d38fb98ed58bbec2d240abbb375ccc7265ca733e2662d4430dd4 not found: ID does not exist" containerID="942e4da42671d38fb98ed58bbec2d240abbb375ccc7265ca733e2662d4430dd4"
Sep 10 17:50:50.785600 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:50:50.784103    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"942e4da42671d38fb98ed58bbec2d240abbb375ccc7265ca733e2662d4430dd4"} err="failed to get container status \"942e4da42671d38fb98ed58bbec2d240abbb375ccc7265ca733e2662d4430dd4\": rpc error: code = NotFound desc = could not find container \"942e4da42671d38fb98ed58bbec2d240abbb375ccc7265ca733e2662d4430dd4\": container with ID starting with 942e4da42671d38fb98ed58bbec2d240abbb375ccc7265ca733e2662d4430dd4 not found: ID does not exist"
Sep 10 17:50:50.800555 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:50:50.800519    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-derive-d2a8a5f7-56554bddbb-vntsd"]
Sep 10 17:50:50.806617 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:50:50.806469    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-derive-d2a8a5f7-56554bddbb-vntsd"]
Sep 10 17:50:50.848760 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:50:50.848732    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-explicit-c7995b-openshift-default-6ccb9db69b-6h6xh"]
Sep 10 17:50:50.850165 ip-10-0-141-180 kubenswrapper[2581]: W0910 17:50:50.850139    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod200bec9f_4a17_4e71_9d38_939cbb027551.slice/crio-c4401e895921198659cc4333fdd41725d4e3189f29169ef66374700e540d9c76 WatchSource:0}: Error finding container c4401e895921198659cc4333fdd41725d4e3189f29169ef66374700e540d9c76: Status 404 returned error can't find the container with id c4401e895921198659cc4333fdd41725d4e3189f29169ef66374700e540d9c76
Sep 10 17:50:50.852312 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:50:50.852282    2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 10 17:50:50.852404 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:50:50.852356    2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 10 17:50:50.852404 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:50:50.852397    2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 10 17:50:50.863730 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:50:50.863704    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2c67c17a-a18d-480f-8437-1f0679c1d9c8" path="/var/lib/kubelet/pods/2c67c17a-a18d-480f-8437-1f0679c1d9c8/volumes"
Sep 10 17:50:50.864277 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:50:50.864258    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d6e2640c-1099-48e3-a2cb-7546cbc76d8f" path="/var/lib/kubelet/pods/d6e2640c-1099-48e3-a2cb-7546cbc76d8f/volumes"
Sep 10 17:50:51.779249 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:50:51.779211    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-explicit-c7995b-openshift-default-6ccb9db69b-6h6xh" event={"ID":"200bec9f-4a17-4e71-9d38-939cbb027551","Type":"ContainerStarted","Data":"5d3c3fa004e8c77d810be71221df1a4ad209af09aa40cdd27081a7a081123866"}
Sep 10 17:50:51.779629 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:50:51.779308    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-explicit-c7995b-openshift-default-6ccb9db69b-6h6xh" event={"ID":"200bec9f-4a17-4e71-9d38-939cbb027551","Type":"ContainerStarted","Data":"c4401e895921198659cc4333fdd41725d4e3189f29169ef66374700e540d9c76"}
Sep 10 17:50:51.816197 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:50:51.816139    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-explicit-c7995b-openshift-default-6ccb9db69b-6h6xh" podStartSLOduration=1.816122792 podStartE2EDuration="1.816122792s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 17:50:50 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 17:50:51.814018461 +0000 UTC m=+871.468092389" watchObservedRunningTime="2026-09-10 17:50:51.816122792 +0000 UTC m=+871.470196695"
Sep 10 17:50:52.565113 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:50:52.565073    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-explicit-c7995b-openshift-default-6ccb9db69b-6h6xh"
Sep 10 17:50:52.570441 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:50:52.570417    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-explicit-c7995b-openshift-default-6ccb9db69b-6h6xh"
Sep 10 17:50:52.786455 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:50:52.786421    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-explicit-c7995b-openshift-default-6ccb9db69b-6h6xh"
Sep 10 17:50:52.787495 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:50:52.787477    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-explicit-c7995b-openshift-default-6ccb9db69b-6h6xh"
Sep 10 17:50:54.579947 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:50:54.579893    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-autores-51ed8d-openshift-default-74f9875b8-4fftk"
Sep 10 17:50:54.674412 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:50:54.674375    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8a2891cb-620d-4e36-984b-0cc72cc6f5b9-credential-socket\") pod \"8a2891cb-620d-4e36-984b-0cc72cc6f5b9\" (UID: \"8a2891cb-620d-4e36-984b-0cc72cc6f5b9\") "
Sep 10 17:50:54.674577 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:50:54.674438    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8a2891cb-620d-4e36-984b-0cc72cc6f5b9-istio-podinfo\") pod \"8a2891cb-620d-4e36-984b-0cc72cc6f5b9\" (UID: \"8a2891cb-620d-4e36-984b-0cc72cc6f5b9\") "
Sep 10 17:50:54.674577 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:50:54.674463    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8a2891cb-620d-4e36-984b-0cc72cc6f5b9-istio-envoy\") pod \"8a2891cb-620d-4e36-984b-0cc72cc6f5b9\" (UID: \"8a2891cb-620d-4e36-984b-0cc72cc6f5b9\") "
Sep 10 17:50:54.674577 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:50:54.674494    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8a2891cb-620d-4e36-984b-0cc72cc6f5b9-istio-token\") pod \"8a2891cb-620d-4e36-984b-0cc72cc6f5b9\" (UID: \"8a2891cb-620d-4e36-984b-0cc72cc6f5b9\") "
Sep 10 17:50:54.674577 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:50:54.674518    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8a2891cb-620d-4e36-984b-0cc72cc6f5b9-istiod-ca-cert\") pod \"8a2891cb-620d-4e36-984b-0cc72cc6f5b9\" (UID: \"8a2891cb-620d-4e36-984b-0cc72cc6f5b9\") "
Sep 10 17:50:54.674809 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:50:54.674590    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8a2891cb-620d-4e36-984b-0cc72cc6f5b9-istio-data\") pod \"8a2891cb-620d-4e36-984b-0cc72cc6f5b9\" (UID: \"8a2891cb-620d-4e36-984b-0cc72cc6f5b9\") "
Sep 10 17:50:54.674809 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:50:54.674620    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8a2891cb-620d-4e36-984b-0cc72cc6f5b9-workload-socket\") pod \"8a2891cb-620d-4e36-984b-0cc72cc6f5b9\" (UID: \"8a2891cb-620d-4e36-984b-0cc72cc6f5b9\") "
Sep 10 17:50:54.674809 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:50:54.674680    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-wrhtg\" (UniqueName: \"kubernetes.io/projected/8a2891cb-620d-4e36-984b-0cc72cc6f5b9-kube-api-access-wrhtg\") pod \"8a2891cb-620d-4e36-984b-0cc72cc6f5b9\" (UID: \"8a2891cb-620d-4e36-984b-0cc72cc6f5b9\") "
Sep 10 17:50:54.674809 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:50:54.674719    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8a2891cb-620d-4e36-984b-0cc72cc6f5b9-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "8a2891cb-620d-4e36-984b-0cc72cc6f5b9" (UID: "8a2891cb-620d-4e36-984b-0cc72cc6f5b9"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 17:50:54.674809 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:50:54.674787    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8a2891cb-620d-4e36-984b-0cc72cc6f5b9-workload-certs\") pod \"8a2891cb-620d-4e36-984b-0cc72cc6f5b9\" (UID: \"8a2891cb-620d-4e36-984b-0cc72cc6f5b9\") "
Sep 10 17:50:54.675126 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:50:54.675100    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8a2891cb-620d-4e36-984b-0cc72cc6f5b9-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "8a2891cb-620d-4e36-984b-0cc72cc6f5b9" (UID: "8a2891cb-620d-4e36-984b-0cc72cc6f5b9"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 17:50:54.675275 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:50:54.675249    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8a2891cb-620d-4e36-984b-0cc72cc6f5b9-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "8a2891cb-620d-4e36-984b-0cc72cc6f5b9" (UID: "8a2891cb-620d-4e36-984b-0cc72cc6f5b9"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 17:50:54.675357 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:50:54.675274    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8a2891cb-620d-4e36-984b-0cc72cc6f5b9-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "8a2891cb-620d-4e36-984b-0cc72cc6f5b9" (UID: "8a2891cb-620d-4e36-984b-0cc72cc6f5b9"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 17:50:54.675357 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:50:54.675281    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8a2891cb-620d-4e36-984b-0cc72cc6f5b9-istio-data" (OuterVolumeSpecName: "istio-data") pod "8a2891cb-620d-4e36-984b-0cc72cc6f5b9" (UID: "8a2891cb-620d-4e36-984b-0cc72cc6f5b9"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 17:50:54.675447 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:50:54.675372    2581 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8a2891cb-620d-4e36-984b-0cc72cc6f5b9-istiod-ca-cert\") on node \"ip-10-0-141-180.ec2.internal\" DevicePath \"\""
Sep 10 17:50:54.675447 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:50:54.675399    2581 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8a2891cb-620d-4e36-984b-0cc72cc6f5b9-istio-data\") on node \"ip-10-0-141-180.ec2.internal\" DevicePath \"\""
Sep 10 17:50:54.675447 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:50:54.675427    2581 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8a2891cb-620d-4e36-984b-0cc72cc6f5b9-workload-socket\") on node \"ip-10-0-141-180.ec2.internal\" DevicePath \"\""
Sep 10 17:50:54.675447 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:50:54.675442    2581 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8a2891cb-620d-4e36-984b-0cc72cc6f5b9-workload-certs\") on node \"ip-10-0-141-180.ec2.internal\" DevicePath \"\""
Sep 10 17:50:54.675673 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:50:54.675455    2581 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8a2891cb-620d-4e36-984b-0cc72cc6f5b9-credential-socket\") on node \"ip-10-0-141-180.ec2.internal\" DevicePath \"\""
Sep 10 17:50:54.677028 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:50:54.677003    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8a2891cb-620d-4e36-984b-0cc72cc6f5b9-istio-token" (OuterVolumeSpecName: "istio-token") pod "8a2891cb-620d-4e36-984b-0cc72cc6f5b9" (UID: "8a2891cb-620d-4e36-984b-0cc72cc6f5b9"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 17:50:54.677155 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:50:54.677104    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8a2891cb-620d-4e36-984b-0cc72cc6f5b9-kube-api-access-wrhtg" (OuterVolumeSpecName: "kube-api-access-wrhtg") pod "8a2891cb-620d-4e36-984b-0cc72cc6f5b9" (UID: "8a2891cb-620d-4e36-984b-0cc72cc6f5b9"). InnerVolumeSpecName "kube-api-access-wrhtg". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 17:50:54.677225 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:50:54.677195    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8a2891cb-620d-4e36-984b-0cc72cc6f5b9-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "8a2891cb-620d-4e36-984b-0cc72cc6f5b9" (UID: "8a2891cb-620d-4e36-984b-0cc72cc6f5b9"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 17:50:54.677225 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:50:54.677197    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/8a2891cb-620d-4e36-984b-0cc72cc6f5b9-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "8a2891cb-620d-4e36-984b-0cc72cc6f5b9" (UID: "8a2891cb-620d-4e36-984b-0cc72cc6f5b9"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 10 17:50:54.776177 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:50:54.776142    2581 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8a2891cb-620d-4e36-984b-0cc72cc6f5b9-istio-token\") on node \"ip-10-0-141-180.ec2.internal\" DevicePath \"\""
Sep 10 17:50:54.776177 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:50:54.776172    2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wrhtg\" (UniqueName: \"kubernetes.io/projected/8a2891cb-620d-4e36-984b-0cc72cc6f5b9-kube-api-access-wrhtg\") on node \"ip-10-0-141-180.ec2.internal\" DevicePath \"\""
Sep 10 17:50:54.776177 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:50:54.776182    2581 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8a2891cb-620d-4e36-984b-0cc72cc6f5b9-istio-podinfo\") on node \"ip-10-0-141-180.ec2.internal\" DevicePath \"\""
Sep 10 17:50:54.776384 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:50:54.776190    2581 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8a2891cb-620d-4e36-984b-0cc72cc6f5b9-istio-envoy\") on node \"ip-10-0-141-180.ec2.internal\" DevicePath \"\""
Sep 10 17:50:54.800476 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:50:54.800440    2581 generic.go:356] "Generic (PLEG): container finished" podID="8a2891cb-620d-4e36-984b-0cc72cc6f5b9" containerID="1f9c700929ee0ca6c7f05ce88ed52f4ac6959c12d970938074baa18fede068c8" exitCode=0
Sep 10 17:50:54.800682 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:50:54.800492    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-autores-51ed8d-openshift-default-74f9875b8-4fftk"
Sep 10 17:50:54.800682 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:50:54.800522    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-autores-51ed8d-openshift-default-74f9875b8-4fftk" event={"ID":"8a2891cb-620d-4e36-984b-0cc72cc6f5b9","Type":"ContainerDied","Data":"1f9c700929ee0ca6c7f05ce88ed52f4ac6959c12d970938074baa18fede068c8"}
Sep 10 17:50:54.800682 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:50:54.800572    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-autores-51ed8d-openshift-default-74f9875b8-4fftk" event={"ID":"8a2891cb-620d-4e36-984b-0cc72cc6f5b9","Type":"ContainerDied","Data":"595b7ecd4ccd05c7d1faba07533aa483a4f2339de37a74db73a9f95041590212"}
Sep 10 17:50:54.800682 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:50:54.800596    2581 scope.go:117] "RemoveContainer" containerID="1f9c700929ee0ca6c7f05ce88ed52f4ac6959c12d970938074baa18fede068c8"
Sep 10 17:50:54.811652 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:50:54.811621    2581 scope.go:117] "RemoveContainer" containerID="40882ffc414bf233886f1847bd73ca1e47ef3701710d891111d02bd939ec63e0"
Sep 10 17:50:54.820192 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:50:54.820172    2581 scope.go:117] "RemoveContainer" containerID="1f9c700929ee0ca6c7f05ce88ed52f4ac6959c12d970938074baa18fede068c8"
Sep 10 17:50:54.820458 ip-10-0-141-180 kubenswrapper[2581]: E0910 17:50:54.820440    2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1f9c700929ee0ca6c7f05ce88ed52f4ac6959c12d970938074baa18fede068c8\": container with ID starting with 1f9c700929ee0ca6c7f05ce88ed52f4ac6959c12d970938074baa18fede068c8 not found: ID does not exist" containerID="1f9c700929ee0ca6c7f05ce88ed52f4ac6959c12d970938074baa18fede068c8"
Sep 10 17:50:54.820530 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:50:54.820470    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1f9c700929ee0ca6c7f05ce88ed52f4ac6959c12d970938074baa18fede068c8"} err="failed to get container status \"1f9c700929ee0ca6c7f05ce88ed52f4ac6959c12d970938074baa18fede068c8\": rpc error: code = NotFound desc = could not find container \"1f9c700929ee0ca6c7f05ce88ed52f4ac6959c12d970938074baa18fede068c8\": container with ID starting with 1f9c700929ee0ca6c7f05ce88ed52f4ac6959c12d970938074baa18fede068c8 not found: ID does not exist"
Sep 10 17:50:54.820530 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:50:54.820495    2581 scope.go:117] "RemoveContainer" containerID="40882ffc414bf233886f1847bd73ca1e47ef3701710d891111d02bd939ec63e0"
Sep 10 17:50:54.820788 ip-10-0-141-180 kubenswrapper[2581]: E0910 17:50:54.820769    2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"40882ffc414bf233886f1847bd73ca1e47ef3701710d891111d02bd939ec63e0\": container with ID starting with 40882ffc414bf233886f1847bd73ca1e47ef3701710d891111d02bd939ec63e0 not found: ID does not exist" containerID="40882ffc414bf233886f1847bd73ca1e47ef3701710d891111d02bd939ec63e0"
Sep 10 17:50:54.820866 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:50:54.820796    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"40882ffc414bf233886f1847bd73ca1e47ef3701710d891111d02bd939ec63e0"} err="failed to get container status \"40882ffc414bf233886f1847bd73ca1e47ef3701710d891111d02bd939ec63e0\": rpc error: code = NotFound desc = could not find container \"40882ffc414bf233886f1847bd73ca1e47ef3701710d891111d02bd939ec63e0\": container with ID starting with 40882ffc414bf233886f1847bd73ca1e47ef3701710d891111d02bd939ec63e0 not found: ID does not exist"
Sep 10 17:50:54.824803 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:50:54.824782    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-autores-51ed8d-openshift-default-74f9875b8-4fftk"]
Sep 10 17:50:54.829927 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:50:54.829907    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-autores-51ed8d-openshift-default-74f9875b8-4fftk"]
Sep 10 17:50:54.862138 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:50:54.862088    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8a2891cb-620d-4e36-984b-0cc72cc6f5b9" path="/var/lib/kubelet/pods/8a2891cb-620d-4e36-984b-0cc72cc6f5b9/volumes"
Sep 10 17:50:55.789681 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:50:55.789620    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-explicit-c7995b-6f7c94747c-hq8tm"]
Sep 10 17:50:55.790188 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:50:55.790171    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d6e2640c-1099-48e3-a2cb-7546cbc76d8f" containerName="payload-pre-processing"
Sep 10 17:50:55.790188 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:50:55.790189    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="d6e2640c-1099-48e3-a2cb-7546cbc76d8f" containerName="payload-pre-processing"
Sep 10 17:50:55.790282 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:50:55.790206    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8a2891cb-620d-4e36-984b-0cc72cc6f5b9" containerName="istio-proxy"
Sep 10 17:50:55.790282 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:50:55.790212    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="8a2891cb-620d-4e36-984b-0cc72cc6f5b9" containerName="istio-proxy"
Sep 10 17:50:55.790349 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:50:55.790282    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="8a2891cb-620d-4e36-984b-0cc72cc6f5b9" containerName="istio-proxy"
Sep 10 17:50:55.790349 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:50:55.790290    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="8a2891cb-620d-4e36-984b-0cc72cc6f5b9" containerName="istio-proxy"
Sep 10 17:50:55.790349 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:50:55.790296    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="d6e2640c-1099-48e3-a2cb-7546cbc76d8f" containerName="payload-pre-processing"
Sep 10 17:50:55.794865 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:50:55.794848    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-explicit-c7995b-6f7c94747c-hq8tm"
Sep 10 17:50:55.797841 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:50:55.797820    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-explicit-c7995b-dockercfg-rz5zm\""
Sep 10 17:50:55.797948 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:50:55.797825    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-explicit-c7995b\""
Sep 10 17:50:55.804715 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:50:55.804695    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-explicit-c7995b-6f7c94747c-hq8tm"]
Sep 10 17:50:55.888110 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:50:55.888071    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-57ckr\" (UniqueName: \"kubernetes.io/projected/8d92e2f4-f71b-495e-9632-5555a6ffe864-kube-api-access-57ckr\") pod \"payload-processing-e2e-explicit-c7995b-6f7c94747c-hq8tm\" (UID: \"8d92e2f4-f71b-495e-9632-5555a6ffe864\") " pod="openshift-ingress/payload-processing-e2e-explicit-c7995b-6f7c94747c-hq8tm"
Sep 10 17:50:55.888397 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:50:55.888373    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/8d92e2f4-f71b-495e-9632-5555a6ffe864-plugins-config-volume\") pod \"payload-processing-e2e-explicit-c7995b-6f7c94747c-hq8tm\" (UID: \"8d92e2f4-f71b-495e-9632-5555a6ffe864\") " pod="openshift-ingress/payload-processing-e2e-explicit-c7995b-6f7c94747c-hq8tm"
Sep 10 17:50:55.989564 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:50:55.989514    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-57ckr\" (UniqueName: \"kubernetes.io/projected/8d92e2f4-f71b-495e-9632-5555a6ffe864-kube-api-access-57ckr\") pod \"payload-processing-e2e-explicit-c7995b-6f7c94747c-hq8tm\" (UID: \"8d92e2f4-f71b-495e-9632-5555a6ffe864\") " pod="openshift-ingress/payload-processing-e2e-explicit-c7995b-6f7c94747c-hq8tm"
Sep 10 17:50:55.989975 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:50:55.989938    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/8d92e2f4-f71b-495e-9632-5555a6ffe864-plugins-config-volume\") pod \"payload-processing-e2e-explicit-c7995b-6f7c94747c-hq8tm\" (UID: \"8d92e2f4-f71b-495e-9632-5555a6ffe864\") " pod="openshift-ingress/payload-processing-e2e-explicit-c7995b-6f7c94747c-hq8tm"
Sep 10 17:50:55.990772 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:50:55.990749    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/8d92e2f4-f71b-495e-9632-5555a6ffe864-plugins-config-volume\") pod \"payload-processing-e2e-explicit-c7995b-6f7c94747c-hq8tm\" (UID: \"8d92e2f4-f71b-495e-9632-5555a6ffe864\") " pod="openshift-ingress/payload-processing-e2e-explicit-c7995b-6f7c94747c-hq8tm"
Sep 10 17:50:55.999105 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:50:55.999084    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-57ckr\" (UniqueName: \"kubernetes.io/projected/8d92e2f4-f71b-495e-9632-5555a6ffe864-kube-api-access-57ckr\") pod \"payload-processing-e2e-explicit-c7995b-6f7c94747c-hq8tm\" (UID: \"8d92e2f4-f71b-495e-9632-5555a6ffe864\") " pod="openshift-ingress/payload-processing-e2e-explicit-c7995b-6f7c94747c-hq8tm"
Sep 10 17:50:56.107538 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:50:56.107457    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-explicit-c7995b-6f7c94747c-hq8tm"
Sep 10 17:50:56.389596 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:50:56.389564    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-explicit-c7995b-6f7c94747c-hq8tm"]
Sep 10 17:50:56.391203 ip-10-0-141-180 kubenswrapper[2581]: W0910 17:50:56.391177    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8d92e2f4_f71b_495e_9632_5555a6ffe864.slice/crio-3e3ebab071df5a1778517436fd4b61d2695b0dfc45906493f5098df68503e526 WatchSource:0}: Error finding container 3e3ebab071df5a1778517436fd4b61d2695b0dfc45906493f5098df68503e526: Status 404 returned error can't find the container with id 3e3ebab071df5a1778517436fd4b61d2695b0dfc45906493f5098df68503e526
Sep 10 17:50:56.818407 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:50:56.818360    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-explicit-c7995b-6f7c94747c-hq8tm" event={"ID":"8d92e2f4-f71b-495e-9632-5555a6ffe864","Type":"ContainerStarted","Data":"bab77bb4cfd1bc9cc2322b0678579883787a0801f49a9b1f3c5702e8bb0a345b"}
Sep 10 17:50:56.818839 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:50:56.818421    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-explicit-c7995b-6f7c94747c-hq8tm" event={"ID":"8d92e2f4-f71b-495e-9632-5555a6ffe864","Type":"ContainerStarted","Data":"3e3ebab071df5a1778517436fd4b61d2695b0dfc45906493f5098df68503e526"}
Sep 10 17:50:56.818839 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:50:56.818534    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-explicit-c7995b-6f7c94747c-hq8tm"
Sep 10 17:50:56.842432 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:50:56.842375    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-explicit-c7995b-6f7c94747c-hq8tm" podStartSLOduration=1.84236205 podStartE2EDuration="1.84236205s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 17:50:55 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 17:50:56.840007553 +0000 UTC m=+876.494081480" watchObservedRunningTime="2026-09-10 17:50:56.84236205 +0000 UTC m=+876.496435955"
Sep 10 17:51:07.831199 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:51:07.831163    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-explicit-c7995b-6f7c94747c-hq8tm"
Sep 10 17:51:14.953501 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:51:14.953463    2581 generic.go:356] "Generic (PLEG): container finished" podID="200bec9f-4a17-4e71-9d38-939cbb027551" containerID="5d3c3fa004e8c77d810be71221df1a4ad209af09aa40cdd27081a7a081123866" exitCode=0
Sep 10 17:51:14.954002 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:51:14.953534    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-explicit-c7995b-openshift-default-6ccb9db69b-6h6xh" event={"ID":"200bec9f-4a17-4e71-9d38-939cbb027551","Type":"ContainerDied","Data":"5d3c3fa004e8c77d810be71221df1a4ad209af09aa40cdd27081a7a081123866"}
Sep 10 17:51:14.954002 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:51:14.953975    2581 scope.go:117] "RemoveContainer" containerID="5d3c3fa004e8c77d810be71221df1a4ad209af09aa40cdd27081a7a081123866"
Sep 10 17:51:14.955057 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:51:14.955026    2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 10 17:51:14.955152 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:51:14.955095    2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 10 17:51:14.955152 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:51:14.955126    2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 10 17:51:15.964518 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:51:15.964484    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-explicit-c7995b-openshift-default-6ccb9db69b-6h6xh" event={"ID":"200bec9f-4a17-4e71-9d38-939cbb027551","Type":"ContainerStarted","Data":"b8cc280ca38dbccbfae9bc816a156437a22151e0c52d44fc141209f39b941bca"}
Sep 10 17:51:16.564821 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:51:16.564783    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-explicit-c7995b-openshift-default-6ccb9db69b-6h6xh"
Sep 10 17:51:16.565032 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:51:16.564988    2581 patch_prober.go:28] interesting pod/e2e-explicit-c7995b-openshift-default-6ccb9db69b-6h6xh container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.58:15021/healthz/ready\": dial tcp 10.133.0.58:15021: connect: connection refused" start-of-body=
Sep 10 17:51:16.565109 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:51:16.565047    2581 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-explicit-c7995b-openshift-default-6ccb9db69b-6h6xh" podUID="200bec9f-4a17-4e71-9d38-939cbb027551" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.58:15021/healthz/ready\": dial tcp 10.133.0.58:15021: connect: connection refused"
Sep 10 17:51:17.565488 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:51:17.565446    2581 patch_prober.go:28] interesting pod/e2e-explicit-c7995b-openshift-default-6ccb9db69b-6h6xh container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.58:15021/healthz/ready\": dial tcp 10.133.0.58:15021: connect: connection refused" start-of-body=
Sep 10 17:51:17.565960 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:51:17.565505    2581 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-explicit-c7995b-openshift-default-6ccb9db69b-6h6xh" podUID="200bec9f-4a17-4e71-9d38-939cbb027551" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.58:15021/healthz/ready\": dial tcp 10.133.0.58:15021: connect: connection refused"
Sep 10 17:51:18.565612 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:51:18.565572    2581 patch_prober.go:28] interesting pod/e2e-explicit-c7995b-openshift-default-6ccb9db69b-6h6xh container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.58:15021/healthz/ready\": dial tcp 10.133.0.58:15021: connect: connection refused" start-of-body=
Sep 10 17:51:18.566029 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:51:18.565678    2581 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-explicit-c7995b-openshift-default-6ccb9db69b-6h6xh" podUID="200bec9f-4a17-4e71-9d38-939cbb027551" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.58:15021/healthz/ready\": dial tcp 10.133.0.58:15021: connect: connection refused"
Sep 10 17:51:19.565927 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:51:19.565889    2581 patch_prober.go:28] interesting pod/e2e-explicit-c7995b-openshift-default-6ccb9db69b-6h6xh container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.58:15021/healthz/ready\": dial tcp 10.133.0.58:15021: connect: connection refused" start-of-body=
Sep 10 17:51:19.566305 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:51:19.565946    2581 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-explicit-c7995b-openshift-default-6ccb9db69b-6h6xh" podUID="200bec9f-4a17-4e71-9d38-939cbb027551" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.58:15021/healthz/ready\": dial tcp 10.133.0.58:15021: connect: connection refused"
Sep 10 17:51:20.565581 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:51:20.565539    2581 patch_prober.go:28] interesting pod/e2e-explicit-c7995b-openshift-default-6ccb9db69b-6h6xh container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.58:15021/healthz/ready\": dial tcp 10.133.0.58:15021: connect: connection refused" start-of-body=
Sep 10 17:51:20.565825 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:51:20.565613    2581 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-explicit-c7995b-openshift-default-6ccb9db69b-6h6xh" podUID="200bec9f-4a17-4e71-9d38-939cbb027551" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.58:15021/healthz/ready\": dial tcp 10.133.0.58:15021: connect: connection refused"
Sep 10 17:51:21.674262 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:51:21.674215    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-explicit-c7995b-openshift-default-6ccb9db69b-6h6xh"
Sep 10 17:51:21.674805 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:51:21.674784    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-explicit-c7995b-openshift-default-6ccb9db69b-6h6xh"
Sep 10 17:51:21.675736 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:51:21.675708    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-explicit-c7995b-openshift-default-6ccb9db69b-6h6xh"
Sep 10 17:52:02.799406 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:52:02.799367    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-explicit-c7995b-vhxrd"]
Sep 10 17:52:02.800118 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:52:02.800099    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8a2891cb-620d-4e36-984b-0cc72cc6f5b9" containerName="istio-proxy"
Sep 10 17:52:02.800168 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:52:02.800122    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="8a2891cb-620d-4e36-984b-0cc72cc6f5b9" containerName="istio-proxy"
Sep 10 17:52:02.802948 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:52:02.802930    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-explicit-c7995b-vhxrd"
Sep 10 17:52:02.806011 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:52:02.805987    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-k7zj6\""
Sep 10 17:52:02.806278 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:52:02.806262    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 10 17:52:02.828320 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:52:02.828283    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-explicit-c7995b-vhxrd"]
Sep 10 17:52:02.874153 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:52:02.872146    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/851e55d1-da64-45ce-874a-b48e24c3433f-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-explicit-c7995b-vhxrd\" (UID: \"851e55d1-da64-45ce-874a-b48e24c3433f\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-explicit-c7995b-vhxrd"
Sep 10 17:52:02.973059 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:52:02.973013    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/851e55d1-da64-45ce-874a-b48e24c3433f-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-explicit-c7995b-vhxrd\" (UID: \"851e55d1-da64-45ce-874a-b48e24c3433f\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-explicit-c7995b-vhxrd"
Sep 10 17:52:02.973918 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:52:02.973887    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/851e55d1-da64-45ce-874a-b48e24c3433f-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-explicit-c7995b-vhxrd\" (UID: \"851e55d1-da64-45ce-874a-b48e24c3433f\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-explicit-c7995b-vhxrd"
Sep 10 17:52:03.114498 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:52:03.114387    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-explicit-c7995b-vhxrd"
Sep 10 17:52:03.407968 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:52:03.407552    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-explicit-c7995b-vhxrd"]
Sep 10 17:52:03.410567 ip-10-0-141-180 kubenswrapper[2581]: W0910 17:52:03.410539    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod851e55d1_da64_45ce_874a_b48e24c3433f.slice/crio-5973b77d9997e9f58d736f489bd0b687ec81481a092dd95baeff0d94db65c77e WatchSource:0}: Error finding container 5973b77d9997e9f58d736f489bd0b687ec81481a092dd95baeff0d94db65c77e: Status 404 returned error can't find the container with id 5973b77d9997e9f58d736f489bd0b687ec81481a092dd95baeff0d94db65c77e
Sep 10 17:52:04.295014 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:52:04.294913    2581 generic.go:356] "Generic (PLEG): container finished" podID="851e55d1-da64-45ce-874a-b48e24c3433f" containerID="d8976504ad86a25d1998bae7e03ca50d584c2410a8c5151eade93752dffdb563" exitCode=0
Sep 10 17:52:04.295370 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:52:04.295001    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-explicit-c7995b-vhxrd" event={"ID":"851e55d1-da64-45ce-874a-b48e24c3433f","Type":"ContainerDied","Data":"d8976504ad86a25d1998bae7e03ca50d584c2410a8c5151eade93752dffdb563"}
Sep 10 17:52:04.295370 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:52:04.295093    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-explicit-c7995b-vhxrd" event={"ID":"851e55d1-da64-45ce-874a-b48e24c3433f","Type":"ContainerStarted","Data":"5973b77d9997e9f58d736f489bd0b687ec81481a092dd95baeff0d94db65c77e"}
Sep 10 17:52:05.441195 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:52:05.441171    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-explicit-c7995b-vhxrd"
Sep 10 17:52:05.501684 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:52:05.501633    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/851e55d1-da64-45ce-874a-b48e24c3433f-maas-api-service-ca\") pod \"851e55d1-da64-45ce-874a-b48e24c3433f\" (UID: \"851e55d1-da64-45ce-874a-b48e24c3433f\") "
Sep 10 17:52:05.501952 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:52:05.501930    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/851e55d1-da64-45ce-874a-b48e24c3433f-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "851e55d1-da64-45ce-874a-b48e24c3433f" (UID: "851e55d1-da64-45ce-874a-b48e24c3433f"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 17:52:05.603271 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:52:05.603200    2581 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/851e55d1-da64-45ce-874a-b48e24c3433f-maas-api-service-ca\") on node \"ip-10-0-141-180.ec2.internal\" DevicePath \"\""
Sep 10 17:52:06.309069 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:52:06.309029    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-explicit-c7995b-vhxrd" event={"ID":"851e55d1-da64-45ce-874a-b48e24c3433f","Type":"ContainerDied","Data":"5973b77d9997e9f58d736f489bd0b687ec81481a092dd95baeff0d94db65c77e"}
Sep 10 17:52:06.309069 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:52:06.309054    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-explicit-c7995b-vhxrd"
Sep 10 17:52:06.309283 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:52:06.309084    2581 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="5973b77d9997e9f58d736f489bd0b687ec81481a092dd95baeff0d94db65c77e"
Sep 10 17:52:13.791259 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:52:13.791227    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-explicit-c7995b-6f7c94747c-hq8tm"]
Sep 10 17:52:13.791623 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:52:13.791461    2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-explicit-c7995b-6f7c94747c-hq8tm" podUID="8d92e2f4-f71b-495e-9632-5555a6ffe864" containerName="payload-processing" containerID="cri-o://bab77bb4cfd1bc9cc2322b0678579883787a0801f49a9b1f3c5702e8bb0a345b" gracePeriod=30
Sep 10 17:52:14.047238 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:52:14.047172    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-explicit-c7995b-vhxrd"]
Sep 10 17:52:14.058758 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:52:14.058730    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-explicit-c7995b-vhxrd"]
Sep 10 17:52:14.063849 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:52:14.063828    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-explicit-c7995b-6f7c94747c-hq8tm"
Sep 10 17:52:14.189321 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:52:14.189288    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/8d92e2f4-f71b-495e-9632-5555a6ffe864-plugins-config-volume\") pod \"8d92e2f4-f71b-495e-9632-5555a6ffe864\" (UID: \"8d92e2f4-f71b-495e-9632-5555a6ffe864\") "
Sep 10 17:52:14.189493 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:52:14.189398    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-57ckr\" (UniqueName: \"kubernetes.io/projected/8d92e2f4-f71b-495e-9632-5555a6ffe864-kube-api-access-57ckr\") pod \"8d92e2f4-f71b-495e-9632-5555a6ffe864\" (UID: \"8d92e2f4-f71b-495e-9632-5555a6ffe864\") "
Sep 10 17:52:14.189752 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:52:14.189723    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8d92e2f4-f71b-495e-9632-5555a6ffe864-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "8d92e2f4-f71b-495e-9632-5555a6ffe864" (UID: "8d92e2f4-f71b-495e-9632-5555a6ffe864"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 17:52:14.191381 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:52:14.191361    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8d92e2f4-f71b-495e-9632-5555a6ffe864-kube-api-access-57ckr" (OuterVolumeSpecName: "kube-api-access-57ckr") pod "8d92e2f4-f71b-495e-9632-5555a6ffe864" (UID: "8d92e2f4-f71b-495e-9632-5555a6ffe864"). InnerVolumeSpecName "kube-api-access-57ckr". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 17:52:14.290833 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:52:14.290798    2581 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/8d92e2f4-f71b-495e-9632-5555a6ffe864-plugins-config-volume\") on node \"ip-10-0-141-180.ec2.internal\" DevicePath \"\""
Sep 10 17:52:14.290833 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:52:14.290824    2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-57ckr\" (UniqueName: \"kubernetes.io/projected/8d92e2f4-f71b-495e-9632-5555a6ffe864-kube-api-access-57ckr\") on node \"ip-10-0-141-180.ec2.internal\" DevicePath \"\""
Sep 10 17:52:14.362041 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:52:14.361965    2581 generic.go:356] "Generic (PLEG): container finished" podID="8d92e2f4-f71b-495e-9632-5555a6ffe864" containerID="bab77bb4cfd1bc9cc2322b0678579883787a0801f49a9b1f3c5702e8bb0a345b" exitCode=0
Sep 10 17:52:14.362041 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:52:14.362011    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-explicit-c7995b-6f7c94747c-hq8tm"
Sep 10 17:52:14.362205 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:52:14.362043    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-explicit-c7995b-6f7c94747c-hq8tm" event={"ID":"8d92e2f4-f71b-495e-9632-5555a6ffe864","Type":"ContainerDied","Data":"bab77bb4cfd1bc9cc2322b0678579883787a0801f49a9b1f3c5702e8bb0a345b"}
Sep 10 17:52:14.362205 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:52:14.362104    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-explicit-c7995b-6f7c94747c-hq8tm" event={"ID":"8d92e2f4-f71b-495e-9632-5555a6ffe864","Type":"ContainerDied","Data":"3e3ebab071df5a1778517436fd4b61d2695b0dfc45906493f5098df68503e526"}
Sep 10 17:52:14.362205 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:52:14.362125    2581 scope.go:117] "RemoveContainer" containerID="bab77bb4cfd1bc9cc2322b0678579883787a0801f49a9b1f3c5702e8bb0a345b"
Sep 10 17:52:14.373222 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:52:14.373203    2581 scope.go:117] "RemoveContainer" containerID="bab77bb4cfd1bc9cc2322b0678579883787a0801f49a9b1f3c5702e8bb0a345b"
Sep 10 17:52:14.373491 ip-10-0-141-180 kubenswrapper[2581]: E0910 17:52:14.373470    2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"bab77bb4cfd1bc9cc2322b0678579883787a0801f49a9b1f3c5702e8bb0a345b\": container with ID starting with bab77bb4cfd1bc9cc2322b0678579883787a0801f49a9b1f3c5702e8bb0a345b not found: ID does not exist" containerID="bab77bb4cfd1bc9cc2322b0678579883787a0801f49a9b1f3c5702e8bb0a345b"
Sep 10 17:52:14.373555 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:52:14.373498    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"bab77bb4cfd1bc9cc2322b0678579883787a0801f49a9b1f3c5702e8bb0a345b"} err="failed to get container status \"bab77bb4cfd1bc9cc2322b0678579883787a0801f49a9b1f3c5702e8bb0a345b\": rpc error: code = NotFound desc = could not find container \"bab77bb4cfd1bc9cc2322b0678579883787a0801f49a9b1f3c5702e8bb0a345b\": container with ID starting with bab77bb4cfd1bc9cc2322b0678579883787a0801f49a9b1f3c5702e8bb0a345b not found: ID does not exist"
Sep 10 17:52:14.410458 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:52:14.410432    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-explicit-c7995b-6f7c94747c-hq8tm"]
Sep 10 17:52:14.424922 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:52:14.424898    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-explicit-c7995b-6f7c94747c-hq8tm"]
Sep 10 17:52:14.864903 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:52:14.864871    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="851e55d1-da64-45ce-874a-b48e24c3433f" path="/var/lib/kubelet/pods/851e55d1-da64-45ce-874a-b48e24c3433f/volumes"
Sep 10 17:52:14.865269 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:52:14.865258    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8d92e2f4-f71b-495e-9632-5555a6ffe864" path="/var/lib/kubelet/pods/8d92e2f4-f71b-495e-9632-5555a6ffe864/volumes"
Sep 10 17:52:21.177366 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:52:21.177279    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-explicit-c7995b-openshift-default-6ccb9db69b-6h6xh"]
Sep 10 17:52:21.177838 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:52:21.177781    2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-explicit-c7995b-openshift-default-6ccb9db69b-6h6xh" podUID="200bec9f-4a17-4e71-9d38-939cbb027551" containerName="istio-proxy" containerID="cri-o://b8cc280ca38dbccbfae9bc816a156437a22151e0c52d44fc141209f39b941bca" gracePeriod=30
Sep 10 17:52:21.677499 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:52:21.677456    2581 patch_prober.go:28] interesting pod/e2e-explicit-c7995b-openshift-default-6ccb9db69b-6h6xh container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Sep 10 17:52:21.677670 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:52:21.677520    2581 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-explicit-c7995b-openshift-default-6ccb9db69b-6h6xh" podUID="200bec9f-4a17-4e71-9d38-939cbb027551" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Sep 10 17:52:26.446878 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:52:26.446846    2581 generic.go:356] "Generic (PLEG): container finished" podID="200bec9f-4a17-4e71-9d38-939cbb027551" containerID="b8cc280ca38dbccbfae9bc816a156437a22151e0c52d44fc141209f39b941bca" exitCode=0
Sep 10 17:52:26.447215 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:52:26.446921    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-explicit-c7995b-openshift-default-6ccb9db69b-6h6xh" event={"ID":"200bec9f-4a17-4e71-9d38-939cbb027551","Type":"ContainerDied","Data":"b8cc280ca38dbccbfae9bc816a156437a22151e0c52d44fc141209f39b941bca"}
Sep 10 17:52:26.447215 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:52:26.447009    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-explicit-c7995b-openshift-default-6ccb9db69b-6h6xh" event={"ID":"200bec9f-4a17-4e71-9d38-939cbb027551","Type":"ContainerDied","Data":"c4401e895921198659cc4333fdd41725d4e3189f29169ef66374700e540d9c76"}
Sep 10 17:52:26.447215 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:52:26.447028    2581 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="c4401e895921198659cc4333fdd41725d4e3189f29169ef66374700e540d9c76"
Sep 10 17:52:26.447215 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:52:26.447043    2581 scope.go:117] "RemoveContainer" containerID="5d3c3fa004e8c77d810be71221df1a4ad209af09aa40cdd27081a7a081123866"
Sep 10 17:52:26.454714 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:52:26.454694    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-explicit-c7995b-openshift-default-6ccb9db69b-6h6xh"
Sep 10 17:52:26.510239 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:52:26.510212    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/200bec9f-4a17-4e71-9d38-939cbb027551-istio-data\") pod \"200bec9f-4a17-4e71-9d38-939cbb027551\" (UID: \"200bec9f-4a17-4e71-9d38-939cbb027551\") "
Sep 10 17:52:26.510392 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:52:26.510312    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/200bec9f-4a17-4e71-9d38-939cbb027551-istio-envoy\") pod \"200bec9f-4a17-4e71-9d38-939cbb027551\" (UID: \"200bec9f-4a17-4e71-9d38-939cbb027551\") "
Sep 10 17:52:26.510392 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:52:26.510380    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/200bec9f-4a17-4e71-9d38-939cbb027551-workload-socket\") pod \"200bec9f-4a17-4e71-9d38-939cbb027551\" (UID: \"200bec9f-4a17-4e71-9d38-939cbb027551\") "
Sep 10 17:52:26.510509 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:52:26.510421    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/200bec9f-4a17-4e71-9d38-939cbb027551-istio-podinfo\") pod \"200bec9f-4a17-4e71-9d38-939cbb027551\" (UID: \"200bec9f-4a17-4e71-9d38-939cbb027551\") "
Sep 10 17:52:26.510509 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:52:26.510448    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/200bec9f-4a17-4e71-9d38-939cbb027551-istio-token\") pod \"200bec9f-4a17-4e71-9d38-939cbb027551\" (UID: \"200bec9f-4a17-4e71-9d38-939cbb027551\") "
Sep 10 17:52:26.510509 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:52:26.510473    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/200bec9f-4a17-4e71-9d38-939cbb027551-credential-socket\") pod \"200bec9f-4a17-4e71-9d38-939cbb027551\" (UID: \"200bec9f-4a17-4e71-9d38-939cbb027551\") "
Sep 10 17:52:26.510730 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:52:26.510515    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/200bec9f-4a17-4e71-9d38-939cbb027551-istio-data" (OuterVolumeSpecName: "istio-data") pod "200bec9f-4a17-4e71-9d38-939cbb027551" (UID: "200bec9f-4a17-4e71-9d38-939cbb027551"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 17:52:26.510730 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:52:26.510546    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/200bec9f-4a17-4e71-9d38-939cbb027551-istiod-ca-cert\") pod \"200bec9f-4a17-4e71-9d38-939cbb027551\" (UID: \"200bec9f-4a17-4e71-9d38-939cbb027551\") "
Sep 10 17:52:26.510730 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:52:26.510586    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/200bec9f-4a17-4e71-9d38-939cbb027551-workload-certs\") pod \"200bec9f-4a17-4e71-9d38-939cbb027551\" (UID: \"200bec9f-4a17-4e71-9d38-939cbb027551\") "
Sep 10 17:52:26.510730 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:52:26.510621    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-w5l9b\" (UniqueName: \"kubernetes.io/projected/200bec9f-4a17-4e71-9d38-939cbb027551-kube-api-access-w5l9b\") pod \"200bec9f-4a17-4e71-9d38-939cbb027551\" (UID: \"200bec9f-4a17-4e71-9d38-939cbb027551\") "
Sep 10 17:52:26.510730 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:52:26.510683    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/200bec9f-4a17-4e71-9d38-939cbb027551-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "200bec9f-4a17-4e71-9d38-939cbb027551" (UID: "200bec9f-4a17-4e71-9d38-939cbb027551"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 17:52:26.510972 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:52:26.510900    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/200bec9f-4a17-4e71-9d38-939cbb027551-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "200bec9f-4a17-4e71-9d38-939cbb027551" (UID: "200bec9f-4a17-4e71-9d38-939cbb027551"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 17:52:26.510972 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:52:26.510948    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/200bec9f-4a17-4e71-9d38-939cbb027551-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "200bec9f-4a17-4e71-9d38-939cbb027551" (UID: "200bec9f-4a17-4e71-9d38-939cbb027551"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 17:52:26.511108 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:52:26.511081    2581 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/200bec9f-4a17-4e71-9d38-939cbb027551-istiod-ca-cert\") on node \"ip-10-0-141-180.ec2.internal\" DevicePath \"\""
Sep 10 17:52:26.511171 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:52:26.511107    2581 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/200bec9f-4a17-4e71-9d38-939cbb027551-workload-certs\") on node \"ip-10-0-141-180.ec2.internal\" DevicePath \"\""
Sep 10 17:52:26.511171 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:52:26.511120    2581 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/200bec9f-4a17-4e71-9d38-939cbb027551-istio-data\") on node \"ip-10-0-141-180.ec2.internal\" DevicePath \"\""
Sep 10 17:52:26.511171 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:52:26.511133    2581 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/200bec9f-4a17-4e71-9d38-939cbb027551-workload-socket\") on node \"ip-10-0-141-180.ec2.internal\" DevicePath \"\""
Sep 10 17:52:26.511342 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:52:26.511322    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/200bec9f-4a17-4e71-9d38-939cbb027551-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "200bec9f-4a17-4e71-9d38-939cbb027551" (UID: "200bec9f-4a17-4e71-9d38-939cbb027551"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 17:52:26.512873 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:52:26.512845    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/200bec9f-4a17-4e71-9d38-939cbb027551-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "200bec9f-4a17-4e71-9d38-939cbb027551" (UID: "200bec9f-4a17-4e71-9d38-939cbb027551"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 17:52:26.513081 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:52:26.513058    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/200bec9f-4a17-4e71-9d38-939cbb027551-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "200bec9f-4a17-4e71-9d38-939cbb027551" (UID: "200bec9f-4a17-4e71-9d38-939cbb027551"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 10 17:52:26.513081 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:52:26.513074    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/200bec9f-4a17-4e71-9d38-939cbb027551-istio-token" (OuterVolumeSpecName: "istio-token") pod "200bec9f-4a17-4e71-9d38-939cbb027551" (UID: "200bec9f-4a17-4e71-9d38-939cbb027551"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 17:52:26.513254 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:52:26.513237    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/200bec9f-4a17-4e71-9d38-939cbb027551-kube-api-access-w5l9b" (OuterVolumeSpecName: "kube-api-access-w5l9b") pod "200bec9f-4a17-4e71-9d38-939cbb027551" (UID: "200bec9f-4a17-4e71-9d38-939cbb027551"). InnerVolumeSpecName "kube-api-access-w5l9b". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 17:52:26.612012 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:52:26.611975    2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-w5l9b\" (UniqueName: \"kubernetes.io/projected/200bec9f-4a17-4e71-9d38-939cbb027551-kube-api-access-w5l9b\") on node \"ip-10-0-141-180.ec2.internal\" DevicePath \"\""
Sep 10 17:52:26.612012 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:52:26.612008    2581 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/200bec9f-4a17-4e71-9d38-939cbb027551-istio-envoy\") on node \"ip-10-0-141-180.ec2.internal\" DevicePath \"\""
Sep 10 17:52:26.612012 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:52:26.612019    2581 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/200bec9f-4a17-4e71-9d38-939cbb027551-istio-podinfo\") on node \"ip-10-0-141-180.ec2.internal\" DevicePath \"\""
Sep 10 17:52:26.612379 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:52:26.612026    2581 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/200bec9f-4a17-4e71-9d38-939cbb027551-istio-token\") on node \"ip-10-0-141-180.ec2.internal\" DevicePath \"\""
Sep 10 17:52:26.612379 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:52:26.612035    2581 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/200bec9f-4a17-4e71-9d38-939cbb027551-credential-socket\") on node \"ip-10-0-141-180.ec2.internal\" DevicePath \"\""
Sep 10 17:52:27.454503 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:52:27.454470    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-explicit-c7995b-openshift-default-6ccb9db69b-6h6xh"
Sep 10 17:52:27.492627 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:52:27.492598    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-explicit-c7995b-openshift-default-6ccb9db69b-6h6xh"]
Sep 10 17:52:27.497517 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:52:27.497491    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-explicit-c7995b-openshift-default-6ccb9db69b-6h6xh"]
Sep 10 17:52:28.862184 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:52:28.862150    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="200bec9f-4a17-4e71-9d38-939cbb027551" path="/var/lib/kubelet/pods/200bec9f-4a17-4e71-9d38-939cbb027551/volumes"
Sep 10 17:52:30.590357 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:52:30.590321    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-mt-c523e8a0-openshift-default-6b4468db8-s7fjk"]
Sep 10 17:52:30.591007 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:52:30.590988    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="851e55d1-da64-45ce-874a-b48e24c3433f" containerName="revoke-keys"
Sep 10 17:52:30.591056 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:52:30.591010    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="851e55d1-da64-45ce-874a-b48e24c3433f" containerName="revoke-keys"
Sep 10 17:52:30.591056 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:52:30.591029    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="200bec9f-4a17-4e71-9d38-939cbb027551" containerName="istio-proxy"
Sep 10 17:52:30.591056 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:52:30.591035    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="200bec9f-4a17-4e71-9d38-939cbb027551" containerName="istio-proxy"
Sep 10 17:52:30.591056 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:52:30.591044    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8d92e2f4-f71b-495e-9632-5555a6ffe864" containerName="payload-processing"
Sep 10 17:52:30.591056 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:52:30.591050    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="8d92e2f4-f71b-495e-9632-5555a6ffe864" containerName="payload-processing"
Sep 10 17:52:30.591210 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:52:30.591063    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="200bec9f-4a17-4e71-9d38-939cbb027551" containerName="istio-proxy"
Sep 10 17:52:30.591210 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:52:30.591069    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="200bec9f-4a17-4e71-9d38-939cbb027551" containerName="istio-proxy"
Sep 10 17:52:30.591210 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:52:30.591166    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="200bec9f-4a17-4e71-9d38-939cbb027551" containerName="istio-proxy"
Sep 10 17:52:30.591210 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:52:30.591177    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="851e55d1-da64-45ce-874a-b48e24c3433f" containerName="revoke-keys"
Sep 10 17:52:30.591210 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:52:30.591192    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="200bec9f-4a17-4e71-9d38-939cbb027551" containerName="istio-proxy"
Sep 10 17:52:30.591210 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:52:30.591202    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="8d92e2f4-f71b-495e-9632-5555a6ffe864" containerName="payload-processing"
Sep 10 17:52:30.595814 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:52:30.595794    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-c523e8a0-openshift-default-6b4468db8-s7fjk"
Sep 10 17:52:30.598797 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:52:30.598777    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-mt-c523e8a0-openshift-default-dockercfg-s2frb\""
Sep 10 17:52:30.620783 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:52:30.620760    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-c523e8a0-openshift-default-6b4468db8-s7fjk"]
Sep 10 17:52:30.640906 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:52:30.640872    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/97d2c5cd-d503-4f66-97c6-6ca97cc789a8-istio-podinfo\") pod \"e2e-mt-c523e8a0-openshift-default-6b4468db8-s7fjk\" (UID: \"97d2c5cd-d503-4f66-97c6-6ca97cc789a8\") " pod="openshift-ingress/e2e-mt-c523e8a0-openshift-default-6b4468db8-s7fjk"
Sep 10 17:52:30.641032 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:52:30.640921    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/97d2c5cd-d503-4f66-97c6-6ca97cc789a8-istio-data\") pod \"e2e-mt-c523e8a0-openshift-default-6b4468db8-s7fjk\" (UID: \"97d2c5cd-d503-4f66-97c6-6ca97cc789a8\") " pod="openshift-ingress/e2e-mt-c523e8a0-openshift-default-6b4468db8-s7fjk"
Sep 10 17:52:30.641032 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:52:30.641002    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/97d2c5cd-d503-4f66-97c6-6ca97cc789a8-istiod-ca-cert\") pod \"e2e-mt-c523e8a0-openshift-default-6b4468db8-s7fjk\" (UID: \"97d2c5cd-d503-4f66-97c6-6ca97cc789a8\") " pod="openshift-ingress/e2e-mt-c523e8a0-openshift-default-6b4468db8-s7fjk"
Sep 10 17:52:30.641170 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:52:30.641036    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rrn8f\" (UniqueName: \"kubernetes.io/projected/97d2c5cd-d503-4f66-97c6-6ca97cc789a8-kube-api-access-rrn8f\") pod \"e2e-mt-c523e8a0-openshift-default-6b4468db8-s7fjk\" (UID: \"97d2c5cd-d503-4f66-97c6-6ca97cc789a8\") " pod="openshift-ingress/e2e-mt-c523e8a0-openshift-default-6b4468db8-s7fjk"
Sep 10 17:52:30.641170 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:52:30.641093    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/97d2c5cd-d503-4f66-97c6-6ca97cc789a8-istio-envoy\") pod \"e2e-mt-c523e8a0-openshift-default-6b4468db8-s7fjk\" (UID: \"97d2c5cd-d503-4f66-97c6-6ca97cc789a8\") " pod="openshift-ingress/e2e-mt-c523e8a0-openshift-default-6b4468db8-s7fjk"
Sep 10 17:52:30.641170 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:52:30.641139    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/97d2c5cd-d503-4f66-97c6-6ca97cc789a8-workload-socket\") pod \"e2e-mt-c523e8a0-openshift-default-6b4468db8-s7fjk\" (UID: \"97d2c5cd-d503-4f66-97c6-6ca97cc789a8\") " pod="openshift-ingress/e2e-mt-c523e8a0-openshift-default-6b4468db8-s7fjk"
Sep 10 17:52:30.641323 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:52:30.641204    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/97d2c5cd-d503-4f66-97c6-6ca97cc789a8-credential-socket\") pod \"e2e-mt-c523e8a0-openshift-default-6b4468db8-s7fjk\" (UID: \"97d2c5cd-d503-4f66-97c6-6ca97cc789a8\") " pod="openshift-ingress/e2e-mt-c523e8a0-openshift-default-6b4468db8-s7fjk"
Sep 10 17:52:30.641323 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:52:30.641238    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/97d2c5cd-d503-4f66-97c6-6ca97cc789a8-istio-token\") pod \"e2e-mt-c523e8a0-openshift-default-6b4468db8-s7fjk\" (UID: \"97d2c5cd-d503-4f66-97c6-6ca97cc789a8\") " pod="openshift-ingress/e2e-mt-c523e8a0-openshift-default-6b4468db8-s7fjk"
Sep 10 17:52:30.641323 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:52:30.641270    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/97d2c5cd-d503-4f66-97c6-6ca97cc789a8-workload-certs\") pod \"e2e-mt-c523e8a0-openshift-default-6b4468db8-s7fjk\" (UID: \"97d2c5cd-d503-4f66-97c6-6ca97cc789a8\") " pod="openshift-ingress/e2e-mt-c523e8a0-openshift-default-6b4468db8-s7fjk"
Sep 10 17:52:30.742630 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:52:30.742586    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/97d2c5cd-d503-4f66-97c6-6ca97cc789a8-istiod-ca-cert\") pod \"e2e-mt-c523e8a0-openshift-default-6b4468db8-s7fjk\" (UID: \"97d2c5cd-d503-4f66-97c6-6ca97cc789a8\") " pod="openshift-ingress/e2e-mt-c523e8a0-openshift-default-6b4468db8-s7fjk"
Sep 10 17:52:30.742630 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:52:30.742632    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rrn8f\" (UniqueName: \"kubernetes.io/projected/97d2c5cd-d503-4f66-97c6-6ca97cc789a8-kube-api-access-rrn8f\") pod \"e2e-mt-c523e8a0-openshift-default-6b4468db8-s7fjk\" (UID: \"97d2c5cd-d503-4f66-97c6-6ca97cc789a8\") " pod="openshift-ingress/e2e-mt-c523e8a0-openshift-default-6b4468db8-s7fjk"
Sep 10 17:52:30.742898 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:52:30.742701    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/97d2c5cd-d503-4f66-97c6-6ca97cc789a8-istio-envoy\") pod \"e2e-mt-c523e8a0-openshift-default-6b4468db8-s7fjk\" (UID: \"97d2c5cd-d503-4f66-97c6-6ca97cc789a8\") " pod="openshift-ingress/e2e-mt-c523e8a0-openshift-default-6b4468db8-s7fjk"
Sep 10 17:52:30.742898 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:52:30.742828    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/97d2c5cd-d503-4f66-97c6-6ca97cc789a8-workload-socket\") pod \"e2e-mt-c523e8a0-openshift-default-6b4468db8-s7fjk\" (UID: \"97d2c5cd-d503-4f66-97c6-6ca97cc789a8\") " pod="openshift-ingress/e2e-mt-c523e8a0-openshift-default-6b4468db8-s7fjk"
Sep 10 17:52:30.743021 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:52:30.742926    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/97d2c5cd-d503-4f66-97c6-6ca97cc789a8-credential-socket\") pod \"e2e-mt-c523e8a0-openshift-default-6b4468db8-s7fjk\" (UID: \"97d2c5cd-d503-4f66-97c6-6ca97cc789a8\") " pod="openshift-ingress/e2e-mt-c523e8a0-openshift-default-6b4468db8-s7fjk"
Sep 10 17:52:30.743021 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:52:30.742965    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/97d2c5cd-d503-4f66-97c6-6ca97cc789a8-istio-token\") pod \"e2e-mt-c523e8a0-openshift-default-6b4468db8-s7fjk\" (UID: \"97d2c5cd-d503-4f66-97c6-6ca97cc789a8\") " pod="openshift-ingress/e2e-mt-c523e8a0-openshift-default-6b4468db8-s7fjk"
Sep 10 17:52:30.743021 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:52:30.743014    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/97d2c5cd-d503-4f66-97c6-6ca97cc789a8-workload-certs\") pod \"e2e-mt-c523e8a0-openshift-default-6b4468db8-s7fjk\" (UID: \"97d2c5cd-d503-4f66-97c6-6ca97cc789a8\") " pod="openshift-ingress/e2e-mt-c523e8a0-openshift-default-6b4468db8-s7fjk"
Sep 10 17:52:30.743187 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:52:30.743107    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/97d2c5cd-d503-4f66-97c6-6ca97cc789a8-istio-podinfo\") pod \"e2e-mt-c523e8a0-openshift-default-6b4468db8-s7fjk\" (UID: \"97d2c5cd-d503-4f66-97c6-6ca97cc789a8\") " pod="openshift-ingress/e2e-mt-c523e8a0-openshift-default-6b4468db8-s7fjk"
Sep 10 17:52:30.743187 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:52:30.743136    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/97d2c5cd-d503-4f66-97c6-6ca97cc789a8-istio-data\") pod \"e2e-mt-c523e8a0-openshift-default-6b4468db8-s7fjk\" (UID: \"97d2c5cd-d503-4f66-97c6-6ca97cc789a8\") " pod="openshift-ingress/e2e-mt-c523e8a0-openshift-default-6b4468db8-s7fjk"
Sep 10 17:52:30.743289 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:52:30.743188    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/97d2c5cd-d503-4f66-97c6-6ca97cc789a8-workload-socket\") pod \"e2e-mt-c523e8a0-openshift-default-6b4468db8-s7fjk\" (UID: \"97d2c5cd-d503-4f66-97c6-6ca97cc789a8\") " pod="openshift-ingress/e2e-mt-c523e8a0-openshift-default-6b4468db8-s7fjk"
Sep 10 17:52:30.743289 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:52:30.743275    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/97d2c5cd-d503-4f66-97c6-6ca97cc789a8-credential-socket\") pod \"e2e-mt-c523e8a0-openshift-default-6b4468db8-s7fjk\" (UID: \"97d2c5cd-d503-4f66-97c6-6ca97cc789a8\") " pod="openshift-ingress/e2e-mt-c523e8a0-openshift-default-6b4468db8-s7fjk"
Sep 10 17:52:30.743395 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:52:30.743378    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/97d2c5cd-d503-4f66-97c6-6ca97cc789a8-istiod-ca-cert\") pod \"e2e-mt-c523e8a0-openshift-default-6b4468db8-s7fjk\" (UID: \"97d2c5cd-d503-4f66-97c6-6ca97cc789a8\") " pod="openshift-ingress/e2e-mt-c523e8a0-openshift-default-6b4468db8-s7fjk"
Sep 10 17:52:30.743456 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:52:30.743393    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/97d2c5cd-d503-4f66-97c6-6ca97cc789a8-workload-certs\") pod \"e2e-mt-c523e8a0-openshift-default-6b4468db8-s7fjk\" (UID: \"97d2c5cd-d503-4f66-97c6-6ca97cc789a8\") " pod="openshift-ingress/e2e-mt-c523e8a0-openshift-default-6b4468db8-s7fjk"
Sep 10 17:52:30.743510 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:52:30.743464    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/97d2c5cd-d503-4f66-97c6-6ca97cc789a8-istio-data\") pod \"e2e-mt-c523e8a0-openshift-default-6b4468db8-s7fjk\" (UID: \"97d2c5cd-d503-4f66-97c6-6ca97cc789a8\") " pod="openshift-ingress/e2e-mt-c523e8a0-openshift-default-6b4468db8-s7fjk"
Sep 10 17:52:30.745056 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:52:30.745032    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/97d2c5cd-d503-4f66-97c6-6ca97cc789a8-istio-envoy\") pod \"e2e-mt-c523e8a0-openshift-default-6b4468db8-s7fjk\" (UID: \"97d2c5cd-d503-4f66-97c6-6ca97cc789a8\") " pod="openshift-ingress/e2e-mt-c523e8a0-openshift-default-6b4468db8-s7fjk"
Sep 10 17:52:30.745417 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:52:30.745399    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/97d2c5cd-d503-4f66-97c6-6ca97cc789a8-istio-podinfo\") pod \"e2e-mt-c523e8a0-openshift-default-6b4468db8-s7fjk\" (UID: \"97d2c5cd-d503-4f66-97c6-6ca97cc789a8\") " pod="openshift-ingress/e2e-mt-c523e8a0-openshift-default-6b4468db8-s7fjk"
Sep 10 17:52:30.755287 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:52:30.755261    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/97d2c5cd-d503-4f66-97c6-6ca97cc789a8-istio-token\") pod \"e2e-mt-c523e8a0-openshift-default-6b4468db8-s7fjk\" (UID: \"97d2c5cd-d503-4f66-97c6-6ca97cc789a8\") " pod="openshift-ingress/e2e-mt-c523e8a0-openshift-default-6b4468db8-s7fjk"
Sep 10 17:52:30.756436 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:52:30.756412    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rrn8f\" (UniqueName: \"kubernetes.io/projected/97d2c5cd-d503-4f66-97c6-6ca97cc789a8-kube-api-access-rrn8f\") pod \"e2e-mt-c523e8a0-openshift-default-6b4468db8-s7fjk\" (UID: \"97d2c5cd-d503-4f66-97c6-6ca97cc789a8\") " pod="openshift-ingress/e2e-mt-c523e8a0-openshift-default-6b4468db8-s7fjk"
Sep 10 17:52:30.905800 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:52:30.905725    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-c523e8a0-openshift-default-6b4468db8-s7fjk"
Sep 10 17:52:31.190992 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:52:31.190961    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-c523e8a0-openshift-default-6b4468db8-s7fjk"]
Sep 10 17:52:31.193314 ip-10-0-141-180 kubenswrapper[2581]: W0910 17:52:31.193287    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod97d2c5cd_d503_4f66_97c6_6ca97cc789a8.slice/crio-43139501329c411e62e677d4749200fa54b6bf2a161ec270b0fffdffd2cd08cc WatchSource:0}: Error finding container 43139501329c411e62e677d4749200fa54b6bf2a161ec270b0fffdffd2cd08cc: Status 404 returned error can't find the container with id 43139501329c411e62e677d4749200fa54b6bf2a161ec270b0fffdffd2cd08cc
Sep 10 17:52:31.195584 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:52:31.195549    2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 10 17:52:31.195684 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:52:31.195619    2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 10 17:52:31.195684 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:52:31.195660    2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 10 17:52:31.485864 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:52:31.485822    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-c523e8a0-openshift-default-6b4468db8-s7fjk" event={"ID":"97d2c5cd-d503-4f66-97c6-6ca97cc789a8","Type":"ContainerStarted","Data":"72a695053135e51e74da7ae6371ab567ae4906007f4e328633147acc462482e3"}
Sep 10 17:52:31.486035 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:52:31.485883    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-c523e8a0-openshift-default-6b4468db8-s7fjk" event={"ID":"97d2c5cd-d503-4f66-97c6-6ca97cc789a8","Type":"ContainerStarted","Data":"43139501329c411e62e677d4749200fa54b6bf2a161ec270b0fffdffd2cd08cc"}
Sep 10 17:52:31.508861 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:52:31.508799    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-mt-c523e8a0-openshift-default-6b4468db8-s7fjk" podStartSLOduration=1.50877785 podStartE2EDuration="1.50877785s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 17:52:30 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 17:52:31.506878719 +0000 UTC m=+971.160952647" watchObservedRunningTime="2026-09-10 17:52:31.50877785 +0000 UTC m=+971.162851752"
Sep 10 17:52:31.905889 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:52:31.905811    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-c523e8a0-openshift-default-6b4468db8-s7fjk"
Sep 10 17:52:32.911521 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:52:32.911494    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-c523e8a0-openshift-default-6b4468db8-s7fjk"
Sep 10 17:52:33.501330 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:52:33.501297    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-c523e8a0-openshift-default-6b4468db8-s7fjk"
Sep 10 17:52:33.502489 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:52:33.502472    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-c523e8a0-openshift-default-6b4468db8-s7fjk"
Sep 10 17:52:35.663843 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:52:35.663808    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-c523e8a0-67b96f7577-ds7xz"]
Sep 10 17:52:35.666609 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:52:35.666592    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-c523e8a0-67b96f7577-ds7xz"
Sep 10 17:52:35.669229 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:52:35.669206    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-mt-c523e8a0-dockercfg-hhm7j\""
Sep 10 17:52:35.669333 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:52:35.669217    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-mt-c523e8a0\""
Sep 10 17:52:35.680162 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:52:35.680135    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-c523e8a0-67b96f7577-ds7xz"]
Sep 10 17:52:35.693827 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:52:35.693798    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5rwch\" (UniqueName: \"kubernetes.io/projected/4b171014-a09d-4221-baa7-2afc58afa8b9-kube-api-access-5rwch\") pod \"payload-processing-e2e-mt-c523e8a0-67b96f7577-ds7xz\" (UID: \"4b171014-a09d-4221-baa7-2afc58afa8b9\") " pod="openshift-ingress/payload-processing-e2e-mt-c523e8a0-67b96f7577-ds7xz"
Sep 10 17:52:35.693965 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:52:35.693946    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/4b171014-a09d-4221-baa7-2afc58afa8b9-plugins-config-volume\") pod \"payload-processing-e2e-mt-c523e8a0-67b96f7577-ds7xz\" (UID: \"4b171014-a09d-4221-baa7-2afc58afa8b9\") " pod="openshift-ingress/payload-processing-e2e-mt-c523e8a0-67b96f7577-ds7xz"
Sep 10 17:52:35.795322 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:52:35.795278    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/4b171014-a09d-4221-baa7-2afc58afa8b9-plugins-config-volume\") pod \"payload-processing-e2e-mt-c523e8a0-67b96f7577-ds7xz\" (UID: \"4b171014-a09d-4221-baa7-2afc58afa8b9\") " pod="openshift-ingress/payload-processing-e2e-mt-c523e8a0-67b96f7577-ds7xz"
Sep 10 17:52:35.795540 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:52:35.795421    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5rwch\" (UniqueName: \"kubernetes.io/projected/4b171014-a09d-4221-baa7-2afc58afa8b9-kube-api-access-5rwch\") pod \"payload-processing-e2e-mt-c523e8a0-67b96f7577-ds7xz\" (UID: \"4b171014-a09d-4221-baa7-2afc58afa8b9\") " pod="openshift-ingress/payload-processing-e2e-mt-c523e8a0-67b96f7577-ds7xz"
Sep 10 17:52:35.796376 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:52:35.796346    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/4b171014-a09d-4221-baa7-2afc58afa8b9-plugins-config-volume\") pod \"payload-processing-e2e-mt-c523e8a0-67b96f7577-ds7xz\" (UID: \"4b171014-a09d-4221-baa7-2afc58afa8b9\") " pod="openshift-ingress/payload-processing-e2e-mt-c523e8a0-67b96f7577-ds7xz"
Sep 10 17:52:35.804402 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:52:35.804376    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5rwch\" (UniqueName: \"kubernetes.io/projected/4b171014-a09d-4221-baa7-2afc58afa8b9-kube-api-access-5rwch\") pod \"payload-processing-e2e-mt-c523e8a0-67b96f7577-ds7xz\" (UID: \"4b171014-a09d-4221-baa7-2afc58afa8b9\") " pod="openshift-ingress/payload-processing-e2e-mt-c523e8a0-67b96f7577-ds7xz"
Sep 10 17:52:35.978077 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:52:35.978043    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-c523e8a0-67b96f7577-ds7xz"
Sep 10 17:52:36.253245 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:52:36.253221    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-c523e8a0-67b96f7577-ds7xz"]
Sep 10 17:52:36.255391 ip-10-0-141-180 kubenswrapper[2581]: W0910 17:52:36.255361    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4b171014_a09d_4221_baa7_2afc58afa8b9.slice/crio-fcf1c40f0de7124e6ccba6d3fca186acb69a0b4da2082415f8d88ff100257f31 WatchSource:0}: Error finding container fcf1c40f0de7124e6ccba6d3fca186acb69a0b4da2082415f8d88ff100257f31: Status 404 returned error can't find the container with id fcf1c40f0de7124e6ccba6d3fca186acb69a0b4da2082415f8d88ff100257f31
Sep 10 17:52:36.524397 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:52:36.524302    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-c523e8a0-67b96f7577-ds7xz" event={"ID":"4b171014-a09d-4221-baa7-2afc58afa8b9","Type":"ContainerStarted","Data":"0005efe1164262e0aeb4d244c7d8a98e4524fc47a4f795cfd6161d5a1e46ba8c"}
Sep 10 17:52:36.524397 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:52:36.524359    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-c523e8a0-67b96f7577-ds7xz" event={"ID":"4b171014-a09d-4221-baa7-2afc58afa8b9","Type":"ContainerStarted","Data":"fcf1c40f0de7124e6ccba6d3fca186acb69a0b4da2082415f8d88ff100257f31"}
Sep 10 17:52:36.524397 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:52:36.524390    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-mt-c523e8a0-67b96f7577-ds7xz"
Sep 10 17:52:36.545757 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:52:36.545703    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-mt-c523e8a0-67b96f7577-ds7xz" podStartSLOduration=1.545688291 podStartE2EDuration="1.545688291s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 17:52:35 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 17:52:36.54231278 +0000 UTC m=+976.196386695" watchObservedRunningTime="2026-09-10 17:52:36.545688291 +0000 UTC m=+976.199762194"
Sep 10 17:52:40.416921 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:52:40.416882    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-48d00b-bgmm7"]
Sep 10 17:52:40.419771 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:52:40.419753    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-48d00b-bgmm7"
Sep 10 17:52:40.422794 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:52:40.422772    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 10 17:52:40.422910 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:52:40.422796    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-k7zj6\""
Sep 10 17:52:40.430011 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:52:40.429985    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-48d00b-bgmm7"]
Sep 10 17:52:40.548206 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:52:40.548172    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/8d4a56bb-c337-4f33-9a73-b2e5dcd2decd-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-a-48d00b-bgmm7\" (UID: \"8d4a56bb-c337-4f33-9a73-b2e5dcd2decd\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-48d00b-bgmm7"
Sep 10 17:52:40.649100 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:52:40.649067    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/8d4a56bb-c337-4f33-9a73-b2e5dcd2decd-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-a-48d00b-bgmm7\" (UID: \"8d4a56bb-c337-4f33-9a73-b2e5dcd2decd\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-48d00b-bgmm7"
Sep 10 17:52:40.649722 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:52:40.649701    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/8d4a56bb-c337-4f33-9a73-b2e5dcd2decd-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-a-48d00b-bgmm7\" (UID: \"8d4a56bb-c337-4f33-9a73-b2e5dcd2decd\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-48d00b-bgmm7"
Sep 10 17:52:40.731779 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:52:40.731711    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-48d00b-bgmm7"
Sep 10 17:52:41.006799 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:52:41.006766    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-48d00b-bgmm7"]
Sep 10 17:52:41.007277 ip-10-0-141-180 kubenswrapper[2581]: W0910 17:52:41.007254    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8d4a56bb_c337_4f33_9a73_b2e5dcd2decd.slice/crio-d61c1c02d36b1dcb60ce37a11b15b5ee538877e00a56a8e050b66843d70adbb2 WatchSource:0}: Error finding container d61c1c02d36b1dcb60ce37a11b15b5ee538877e00a56a8e050b66843d70adbb2: Status 404 returned error can't find the container with id d61c1c02d36b1dcb60ce37a11b15b5ee538877e00a56a8e050b66843d70adbb2
Sep 10 17:52:41.558044 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:52:41.557945    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-48d00b-bgmm7" event={"ID":"8d4a56bb-c337-4f33-9a73-b2e5dcd2decd","Type":"ContainerStarted","Data":"81e88b8cea6194560b7c059d7b328b127d40ac95277169faff5e475e9dba176c"}
Sep 10 17:52:41.558044 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:52:41.558007    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-48d00b-bgmm7" event={"ID":"8d4a56bb-c337-4f33-9a73-b2e5dcd2decd","Type":"ContainerStarted","Data":"d61c1c02d36b1dcb60ce37a11b15b5ee538877e00a56a8e050b66843d70adbb2"}
Sep 10 17:52:41.576997 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:52:41.576954    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-48d00b-bgmm7" podStartSLOduration=1.5769385489999999 podStartE2EDuration="1.576938549s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 17:52:40 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 17:52:41.57316485 +0000 UTC m=+981.227238767" watchObservedRunningTime="2026-09-10 17:52:41.576938549 +0000 UTC m=+981.231012452"
Sep 10 17:52:42.565443 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:52:42.565406    2581 generic.go:356] "Generic (PLEG): container finished" podID="8d4a56bb-c337-4f33-9a73-b2e5dcd2decd" containerID="81e88b8cea6194560b7c059d7b328b127d40ac95277169faff5e475e9dba176c" exitCode=0
Sep 10 17:52:42.565850 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:52:42.565497    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-48d00b-bgmm7" event={"ID":"8d4a56bb-c337-4f33-9a73-b2e5dcd2decd","Type":"ContainerDied","Data":"81e88b8cea6194560b7c059d7b328b127d40ac95277169faff5e475e9dba176c"}
Sep 10 17:52:43.710464 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:52:43.710441    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-48d00b-bgmm7"
Sep 10 17:52:43.779383 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:52:43.779347    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/8d4a56bb-c337-4f33-9a73-b2e5dcd2decd-maas-api-service-ca\") pod \"8d4a56bb-c337-4f33-9a73-b2e5dcd2decd\" (UID: \"8d4a56bb-c337-4f33-9a73-b2e5dcd2decd\") "
Sep 10 17:52:43.779742 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:52:43.779717    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8d4a56bb-c337-4f33-9a73-b2e5dcd2decd-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "8d4a56bb-c337-4f33-9a73-b2e5dcd2decd" (UID: "8d4a56bb-c337-4f33-9a73-b2e5dcd2decd"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 17:52:43.881131 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:52:43.881043    2581 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/8d4a56bb-c337-4f33-9a73-b2e5dcd2decd-maas-api-service-ca\") on node \"ip-10-0-141-180.ec2.internal\" DevicePath \"\""
Sep 10 17:52:44.584902 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:52:44.584862    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-48d00b-bgmm7" event={"ID":"8d4a56bb-c337-4f33-9a73-b2e5dcd2decd","Type":"ContainerDied","Data":"d61c1c02d36b1dcb60ce37a11b15b5ee538877e00a56a8e050b66843d70adbb2"}
Sep 10 17:52:44.585171 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:52:44.585155    2581 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="d61c1c02d36b1dcb60ce37a11b15b5ee538877e00a56a8e050b66843d70adbb2"
Sep 10 17:52:44.585329 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:52:44.585316    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-48d00b-bgmm7"
Sep 10 17:52:47.533836 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:52:47.533803    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-mt-c523e8a0-67b96f7577-ds7xz"
Sep 10 17:52:51.028469 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:52:51.028427    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-a-48d00b-5fdb8c96fc-kgm8l"]
Sep 10 17:52:51.028955 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:52:51.028775    2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-inf-a-48d00b-5fdb8c96fc-kgm8l" podUID="c884e60f-66ba-4308-94a0-e43e15fb1bfe" containerName="payload-processing" containerID="cri-o://c85c0aff340b6badbb9ebcdec56472a2ad1503252208f7c749a9326e8b7e31b9" gracePeriod=30
Sep 10 17:52:51.046172 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:52:51.046134    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-a-48d00b-5cbd8f7fcd-ngxnj"]
Sep 10 17:52:51.046574 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:52:51.046546    2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-48d00b-5cbd8f7fcd-ngxnj" podUID="e02ee03f-8b48-409f-971f-60058b269148" containerName="payload-pre-processing" containerID="cri-o://a44cc7cdcd63fb6e4f3011b4977a38903e7a288ce207e1d08c2c12952fbc8ab2" gracePeriod=30
Sep 10 17:52:51.353456 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:52:51.353415    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-48d00b-bgmm7"]
Sep 10 17:52:51.358237 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:52:51.358213    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-48d00b-bgmm7"]
Sep 10 17:52:51.369018 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:52:51.368991    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-a-48d00b-5fdb8c96fc-kgm8l"
Sep 10 17:52:51.373101 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:52:51.373082    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-48d00b-5cbd8f7fcd-ngxnj"
Sep 10 17:52:51.465589 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:52:51.465553    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-n48z5\" (UniqueName: \"kubernetes.io/projected/e02ee03f-8b48-409f-971f-60058b269148-kube-api-access-n48z5\") pod \"e02ee03f-8b48-409f-971f-60058b269148\" (UID: \"e02ee03f-8b48-409f-971f-60058b269148\") "
Sep 10 17:52:51.465589 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:52:51.465602    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c884e60f-66ba-4308-94a0-e43e15fb1bfe-plugins-config-volume\") pod \"c884e60f-66ba-4308-94a0-e43e15fb1bfe\" (UID: \"c884e60f-66ba-4308-94a0-e43e15fb1bfe\") "
Sep 10 17:52:51.465879 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:52:51.465659    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e02ee03f-8b48-409f-971f-60058b269148-plugins-config-volume\") pod \"e02ee03f-8b48-409f-971f-60058b269148\" (UID: \"e02ee03f-8b48-409f-971f-60058b269148\") "
Sep 10 17:52:51.465879 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:52:51.465738    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-hn2w6\" (UniqueName: \"kubernetes.io/projected/c884e60f-66ba-4308-94a0-e43e15fb1bfe-kube-api-access-hn2w6\") pod \"c884e60f-66ba-4308-94a0-e43e15fb1bfe\" (UID: \"c884e60f-66ba-4308-94a0-e43e15fb1bfe\") "
Sep 10 17:52:51.466054 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:52:51.466026    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c884e60f-66ba-4308-94a0-e43e15fb1bfe-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "c884e60f-66ba-4308-94a0-e43e15fb1bfe" (UID: "c884e60f-66ba-4308-94a0-e43e15fb1bfe"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 17:52:51.466385 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:52:51.466352    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e02ee03f-8b48-409f-971f-60058b269148-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "e02ee03f-8b48-409f-971f-60058b269148" (UID: "e02ee03f-8b48-409f-971f-60058b269148"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 17:52:51.468045 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:52:51.468021    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c884e60f-66ba-4308-94a0-e43e15fb1bfe-kube-api-access-hn2w6" (OuterVolumeSpecName: "kube-api-access-hn2w6") pod "c884e60f-66ba-4308-94a0-e43e15fb1bfe" (UID: "c884e60f-66ba-4308-94a0-e43e15fb1bfe"). InnerVolumeSpecName "kube-api-access-hn2w6". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 17:52:51.468367 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:52:51.468346    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e02ee03f-8b48-409f-971f-60058b269148-kube-api-access-n48z5" (OuterVolumeSpecName: "kube-api-access-n48z5") pod "e02ee03f-8b48-409f-971f-60058b269148" (UID: "e02ee03f-8b48-409f-971f-60058b269148"). InnerVolumeSpecName "kube-api-access-n48z5". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 17:52:51.566593 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:52:51.566557    2581 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e02ee03f-8b48-409f-971f-60058b269148-plugins-config-volume\") on node \"ip-10-0-141-180.ec2.internal\" DevicePath \"\""
Sep 10 17:52:51.566593 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:52:51.566589    2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hn2w6\" (UniqueName: \"kubernetes.io/projected/c884e60f-66ba-4308-94a0-e43e15fb1bfe-kube-api-access-hn2w6\") on node \"ip-10-0-141-180.ec2.internal\" DevicePath \"\""
Sep 10 17:52:51.566593 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:52:51.566599    2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-n48z5\" (UniqueName: \"kubernetes.io/projected/e02ee03f-8b48-409f-971f-60058b269148-kube-api-access-n48z5\") on node \"ip-10-0-141-180.ec2.internal\" DevicePath \"\""
Sep 10 17:52:51.566856 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:52:51.566609    2581 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c884e60f-66ba-4308-94a0-e43e15fb1bfe-plugins-config-volume\") on node \"ip-10-0-141-180.ec2.internal\" DevicePath \"\""
Sep 10 17:52:51.637907 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:52:51.637823    2581 generic.go:356] "Generic (PLEG): container finished" podID="e02ee03f-8b48-409f-971f-60058b269148" containerID="a44cc7cdcd63fb6e4f3011b4977a38903e7a288ce207e1d08c2c12952fbc8ab2" exitCode=0
Sep 10 17:52:51.637907 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:52:51.637862    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-48d00b-5cbd8f7fcd-ngxnj"
Sep 10 17:52:51.638118 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:52:51.637900    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-48d00b-5cbd8f7fcd-ngxnj" event={"ID":"e02ee03f-8b48-409f-971f-60058b269148","Type":"ContainerDied","Data":"a44cc7cdcd63fb6e4f3011b4977a38903e7a288ce207e1d08c2c12952fbc8ab2"}
Sep 10 17:52:51.638118 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:52:51.637959    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-48d00b-5cbd8f7fcd-ngxnj" event={"ID":"e02ee03f-8b48-409f-971f-60058b269148","Type":"ContainerDied","Data":"e44d48980f20434b226d9f44f28dfb53f8262150ecd53d64eb408f86b004111c"}
Sep 10 17:52:51.638118 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:52:51.637990    2581 scope.go:117] "RemoveContainer" containerID="a44cc7cdcd63fb6e4f3011b4977a38903e7a288ce207e1d08c2c12952fbc8ab2"
Sep 10 17:52:51.639589 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:52:51.639565    2581 generic.go:356] "Generic (PLEG): container finished" podID="c884e60f-66ba-4308-94a0-e43e15fb1bfe" containerID="c85c0aff340b6badbb9ebcdec56472a2ad1503252208f7c749a9326e8b7e31b9" exitCode=0
Sep 10 17:52:51.639704 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:52:51.639662    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-a-48d00b-5fdb8c96fc-kgm8l" event={"ID":"c884e60f-66ba-4308-94a0-e43e15fb1bfe","Type":"ContainerDied","Data":"c85c0aff340b6badbb9ebcdec56472a2ad1503252208f7c749a9326e8b7e31b9"}
Sep 10 17:52:51.639782 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:52:51.639731    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-a-48d00b-5fdb8c96fc-kgm8l" event={"ID":"c884e60f-66ba-4308-94a0-e43e15fb1bfe","Type":"ContainerDied","Data":"25ce47b30857e47c39e72658ee1bc66195b45c67f6c529c5abe324407609efc4"}
Sep 10 17:52:51.639782 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:52:51.639675    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-a-48d00b-5fdb8c96fc-kgm8l"
Sep 10 17:52:51.648041 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:52:51.648025    2581 scope.go:117] "RemoveContainer" containerID="a44cc7cdcd63fb6e4f3011b4977a38903e7a288ce207e1d08c2c12952fbc8ab2"
Sep 10 17:52:51.648322 ip-10-0-141-180 kubenswrapper[2581]: E0910 17:52:51.648305    2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a44cc7cdcd63fb6e4f3011b4977a38903e7a288ce207e1d08c2c12952fbc8ab2\": container with ID starting with a44cc7cdcd63fb6e4f3011b4977a38903e7a288ce207e1d08c2c12952fbc8ab2 not found: ID does not exist" containerID="a44cc7cdcd63fb6e4f3011b4977a38903e7a288ce207e1d08c2c12952fbc8ab2"
Sep 10 17:52:51.648389 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:52:51.648329    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a44cc7cdcd63fb6e4f3011b4977a38903e7a288ce207e1d08c2c12952fbc8ab2"} err="failed to get container status \"a44cc7cdcd63fb6e4f3011b4977a38903e7a288ce207e1d08c2c12952fbc8ab2\": rpc error: code = NotFound desc = could not find container \"a44cc7cdcd63fb6e4f3011b4977a38903e7a288ce207e1d08c2c12952fbc8ab2\": container with ID starting with a44cc7cdcd63fb6e4f3011b4977a38903e7a288ce207e1d08c2c12952fbc8ab2 not found: ID does not exist"
Sep 10 17:52:51.648389 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:52:51.648345    2581 scope.go:117] "RemoveContainer" containerID="c85c0aff340b6badbb9ebcdec56472a2ad1503252208f7c749a9326e8b7e31b9"
Sep 10 17:52:51.657359 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:52:51.657340    2581 scope.go:117] "RemoveContainer" containerID="c85c0aff340b6badbb9ebcdec56472a2ad1503252208f7c749a9326e8b7e31b9"
Sep 10 17:52:51.657590 ip-10-0-141-180 kubenswrapper[2581]: E0910 17:52:51.657573    2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c85c0aff340b6badbb9ebcdec56472a2ad1503252208f7c749a9326e8b7e31b9\": container with ID starting with c85c0aff340b6badbb9ebcdec56472a2ad1503252208f7c749a9326e8b7e31b9 not found: ID does not exist" containerID="c85c0aff340b6badbb9ebcdec56472a2ad1503252208f7c749a9326e8b7e31b9"
Sep 10 17:52:51.657682 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:52:51.657601    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c85c0aff340b6badbb9ebcdec56472a2ad1503252208f7c749a9326e8b7e31b9"} err="failed to get container status \"c85c0aff340b6badbb9ebcdec56472a2ad1503252208f7c749a9326e8b7e31b9\": rpc error: code = NotFound desc = could not find container \"c85c0aff340b6badbb9ebcdec56472a2ad1503252208f7c749a9326e8b7e31b9\": container with ID starting with c85c0aff340b6badbb9ebcdec56472a2ad1503252208f7c749a9326e8b7e31b9 not found: ID does not exist"
Sep 10 17:52:51.668226 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:52:51.668205    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-a-48d00b-5cbd8f7fcd-ngxnj"]
Sep 10 17:52:51.672913 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:52:51.672881    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-a-48d00b-5cbd8f7fcd-ngxnj"]
Sep 10 17:52:51.689121 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:52:51.689098    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-a-48d00b-5fdb8c96fc-kgm8l"]
Sep 10 17:52:51.694569 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:52:51.694550    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-a-48d00b-5fdb8c96fc-kgm8l"]
Sep 10 17:52:52.862698 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:52:52.862658    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8d4a56bb-c337-4f33-9a73-b2e5dcd2decd" path="/var/lib/kubelet/pods/8d4a56bb-c337-4f33-9a73-b2e5dcd2decd/volumes"
Sep 10 17:52:52.863232 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:52:52.863212    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c884e60f-66ba-4308-94a0-e43e15fb1bfe" path="/var/lib/kubelet/pods/c884e60f-66ba-4308-94a0-e43e15fb1bfe/volumes"
Sep 10 17:52:52.863603 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:52:52.863585    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e02ee03f-8b48-409f-971f-60058b269148" path="/var/lib/kubelet/pods/e02ee03f-8b48-409f-971f-60058b269148/volumes"
Sep 10 17:52:53.655633 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:52:53.655599    2581 generic.go:356] "Generic (PLEG): container finished" podID="97d2c5cd-d503-4f66-97c6-6ca97cc789a8" containerID="72a695053135e51e74da7ae6371ab567ae4906007f4e328633147acc462482e3" exitCode=0
Sep 10 17:52:53.655865 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:52:53.655676    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-c523e8a0-openshift-default-6b4468db8-s7fjk" event={"ID":"97d2c5cd-d503-4f66-97c6-6ca97cc789a8","Type":"ContainerDied","Data":"72a695053135e51e74da7ae6371ab567ae4906007f4e328633147acc462482e3"}
Sep 10 17:52:53.656096 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:52:53.656081    2581 scope.go:117] "RemoveContainer" containerID="72a695053135e51e74da7ae6371ab567ae4906007f4e328633147acc462482e3"
Sep 10 17:52:53.657145 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:52:53.657122    2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 10 17:52:53.657196 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:52:53.657188    2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 10 17:52:53.657232 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:52:53.657216    2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 10 17:52:54.664368 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:52:54.664331    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-c523e8a0-openshift-default-6b4468db8-s7fjk" event={"ID":"97d2c5cd-d503-4f66-97c6-6ca97cc789a8","Type":"ContainerStarted","Data":"efdcb46794b47abed151efea1717cb5f1a9251d77800558f7f93ccb655b64b7b"}
Sep 10 17:52:54.906562 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:52:54.906523    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-c523e8a0-openshift-default-6b4468db8-s7fjk"
Sep 10 17:52:54.906798 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:52:54.906773    2581 patch_prober.go:28] interesting pod/e2e-mt-c523e8a0-openshift-default-6b4468db8-s7fjk container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.61:15021/healthz/ready\": dial tcp 10.133.0.61:15021: connect: connection refused" start-of-body=
Sep 10 17:52:54.906848 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:52:54.906828    2581 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-c523e8a0-openshift-default-6b4468db8-s7fjk" podUID="97d2c5cd-d503-4f66-97c6-6ca97cc789a8" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.61:15021/healthz/ready\": dial tcp 10.133.0.61:15021: connect: connection refused"
Sep 10 17:52:55.906517 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:52:55.906482    2581 patch_prober.go:28] interesting pod/e2e-mt-c523e8a0-openshift-default-6b4468db8-s7fjk container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.61:15021/healthz/ready\": dial tcp 10.133.0.61:15021: connect: connection refused" start-of-body=
Sep 10 17:52:55.906930 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:52:55.906540    2581 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-c523e8a0-openshift-default-6b4468db8-s7fjk" podUID="97d2c5cd-d503-4f66-97c6-6ca97cc789a8" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.61:15021/healthz/ready\": dial tcp 10.133.0.61:15021: connect: connection refused"
Sep 10 17:52:56.906692 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:52:56.906620    2581 patch_prober.go:28] interesting pod/e2e-mt-c523e8a0-openshift-default-6b4468db8-s7fjk container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.61:15021/healthz/ready\": dial tcp 10.133.0.61:15021: connect: connection refused" start-of-body=
Sep 10 17:52:56.907172 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:52:56.906723    2581 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-c523e8a0-openshift-default-6b4468db8-s7fjk" podUID="97d2c5cd-d503-4f66-97c6-6ca97cc789a8" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.61:15021/healthz/ready\": dial tcp 10.133.0.61:15021: connect: connection refused"
Sep 10 17:52:57.906491 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:52:57.906452    2581 patch_prober.go:28] interesting pod/e2e-mt-c523e8a0-openshift-default-6b4468db8-s7fjk container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.61:15021/healthz/ready\": dial tcp 10.133.0.61:15021: connect: connection refused" start-of-body=
Sep 10 17:52:57.906697 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:52:57.906513    2581 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-c523e8a0-openshift-default-6b4468db8-s7fjk" podUID="97d2c5cd-d503-4f66-97c6-6ca97cc789a8" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.61:15021/healthz/ready\": dial tcp 10.133.0.61:15021: connect: connection refused"
Sep 10 17:52:58.573456 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:52:58.573405    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-a-48d00b-openshift-default-6cc74dc546-kjh62"]
Sep 10 17:52:58.582776 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:52:58.582552    2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-inf-a-48d00b-openshift-default-6cc74dc546-kjh62" podUID="20ec55f7-fa46-4372-bf1b-791406c99a10" containerName="istio-proxy" containerID="cri-o://32b2827a6a7383e227807bbec57841e55b9ce266bdf59eb614f1d72536568ee7" gracePeriod=30
Sep 10 17:52:58.906458 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:52:58.906374    2581 patch_prober.go:28] interesting pod/e2e-mt-c523e8a0-openshift-default-6b4468db8-s7fjk container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.61:15021/healthz/ready\": dial tcp 10.133.0.61:15021: connect: connection refused" start-of-body=
Sep 10 17:52:58.906458 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:52:58.906428    2581 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-c523e8a0-openshift-default-6b4468db8-s7fjk" podUID="97d2c5cd-d503-4f66-97c6-6ca97cc789a8" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.61:15021/healthz/ready\": dial tcp 10.133.0.61:15021: connect: connection refused"
Sep 10 17:52:59.015899 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:52:59.015865    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-783a5e-lw6fv"]
Sep 10 17:52:59.016311 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:52:59.016298    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8d4a56bb-c337-4f33-9a73-b2e5dcd2decd" containerName="revoke-keys"
Sep 10 17:52:59.016365 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:52:59.016313    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="8d4a56bb-c337-4f33-9a73-b2e5dcd2decd" containerName="revoke-keys"
Sep 10 17:52:59.016365 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:52:59.016334    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c884e60f-66ba-4308-94a0-e43e15fb1bfe" containerName="payload-processing"
Sep 10 17:52:59.016365 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:52:59.016341    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="c884e60f-66ba-4308-94a0-e43e15fb1bfe" containerName="payload-processing"
Sep 10 17:52:59.016365 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:52:59.016354    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e02ee03f-8b48-409f-971f-60058b269148" containerName="payload-pre-processing"
Sep 10 17:52:59.016365 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:52:59.016360    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="e02ee03f-8b48-409f-971f-60058b269148" containerName="payload-pre-processing"
Sep 10 17:52:59.016535 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:52:59.016431    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="e02ee03f-8b48-409f-971f-60058b269148" containerName="payload-pre-processing"
Sep 10 17:52:59.016535 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:52:59.016441    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="c884e60f-66ba-4308-94a0-e43e15fb1bfe" containerName="payload-processing"
Sep 10 17:52:59.016535 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:52:59.016451    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="8d4a56bb-c337-4f33-9a73-b2e5dcd2decd" containerName="revoke-keys"
Sep 10 17:52:59.018767 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:52:59.018736    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-783a5e-lw6fv"
Sep 10 17:52:59.022501 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:52:59.022481    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 10 17:52:59.022624 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:52:59.022544    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-k7zj6\""
Sep 10 17:52:59.038097 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:52:59.038067    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-783a5e-lw6fv"]
Sep 10 17:52:59.141368 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:52:59.141328    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/3f5843ad-da03-4167-9537-b5665deeca7d-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-b-783a5e-lw6fv\" (UID: \"3f5843ad-da03-4167-9537-b5665deeca7d\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-783a5e-lw6fv"
Sep 10 17:52:59.242442 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:52:59.242398    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/3f5843ad-da03-4167-9537-b5665deeca7d-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-b-783a5e-lw6fv\" (UID: \"3f5843ad-da03-4167-9537-b5665deeca7d\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-783a5e-lw6fv"
Sep 10 17:52:59.243080 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:52:59.243058    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/3f5843ad-da03-4167-9537-b5665deeca7d-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-b-783a5e-lw6fv\" (UID: \"3f5843ad-da03-4167-9537-b5665deeca7d\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-783a5e-lw6fv"
Sep 10 17:52:59.330828 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:52:59.330794    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-783a5e-lw6fv"
Sep 10 17:52:59.727348 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:52:59.727319    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-783a5e-lw6fv"]
Sep 10 17:52:59.728260 ip-10-0-141-180 kubenswrapper[2581]: W0910 17:52:59.728237    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3f5843ad_da03_4167_9537_b5665deeca7d.slice/crio-0787f5e28726f4ce14976a92627469ff3d39165399353c29dc30cd2fbb5a930c WatchSource:0}: Error finding container 0787f5e28726f4ce14976a92627469ff3d39165399353c29dc30cd2fbb5a930c: Status 404 returned error can't find the container with id 0787f5e28726f4ce14976a92627469ff3d39165399353c29dc30cd2fbb5a930c
Sep 10 17:52:59.911146 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:52:59.911118    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-c523e8a0-openshift-default-6b4468db8-s7fjk"
Sep 10 17:52:59.911541 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:52:59.911516    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-c523e8a0-openshift-default-6b4468db8-s7fjk"
Sep 10 17:52:59.912272 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:52:59.912255    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-c523e8a0-openshift-default-6b4468db8-s7fjk"
Sep 10 17:53:00.717308 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:53:00.717278    2581 generic.go:356] "Generic (PLEG): container finished" podID="3f5843ad-da03-4167-9537-b5665deeca7d" containerID="2dfa6189317915e2c311f129f578bcaf81268a7cc0931fe5cc8dfb4adf0fa714" exitCode=0
Sep 10 17:53:00.717498 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:53:00.717361    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-783a5e-lw6fv" event={"ID":"3f5843ad-da03-4167-9537-b5665deeca7d","Type":"ContainerDied","Data":"2dfa6189317915e2c311f129f578bcaf81268a7cc0931fe5cc8dfb4adf0fa714"}
Sep 10 17:53:00.717498 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:53:00.717431    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-783a5e-lw6fv" event={"ID":"3f5843ad-da03-4167-9537-b5665deeca7d","Type":"ContainerStarted","Data":"0787f5e28726f4ce14976a92627469ff3d39165399353c29dc30cd2fbb5a930c"}
Sep 10 17:53:01.859327 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:53:01.859303    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-783a5e-lw6fv"
Sep 10 17:53:01.967593 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:53:01.967565    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/3f5843ad-da03-4167-9537-b5665deeca7d-maas-api-service-ca\") pod \"3f5843ad-da03-4167-9537-b5665deeca7d\" (UID: \"3f5843ad-da03-4167-9537-b5665deeca7d\") "
Sep 10 17:53:01.967903 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:53:01.967881    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3f5843ad-da03-4167-9537-b5665deeca7d-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "3f5843ad-da03-4167-9537-b5665deeca7d" (UID: "3f5843ad-da03-4167-9537-b5665deeca7d"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 17:53:02.069143 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:53:02.069074    2581 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/3f5843ad-da03-4167-9537-b5665deeca7d-maas-api-service-ca\") on node \"ip-10-0-141-180.ec2.internal\" DevicePath \"\""
Sep 10 17:53:02.731385 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:53:02.731342    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-783a5e-lw6fv" event={"ID":"3f5843ad-da03-4167-9537-b5665deeca7d","Type":"ContainerDied","Data":"0787f5e28726f4ce14976a92627469ff3d39165399353c29dc30cd2fbb5a930c"}
Sep 10 17:53:02.731385 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:53:02.731392    2581 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="0787f5e28726f4ce14976a92627469ff3d39165399353c29dc30cd2fbb5a930c"
Sep 10 17:53:02.731613 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:53:02.731355    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-783a5e-lw6fv"
Sep 10 17:53:02.736635 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:53:02.736603    2581 patch_prober.go:28] interesting pod/e2e-inf-a-48d00b-openshift-default-6cc74dc546-kjh62 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Sep 10 17:53:02.736779 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:53:02.736688    2581 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-inf-a-48d00b-openshift-default-6cc74dc546-kjh62" podUID="20ec55f7-fa46-4372-bf1b-791406c99a10" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Sep 10 17:53:03.740314 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:53:03.740280    2581 generic.go:356] "Generic (PLEG): container finished" podID="20ec55f7-fa46-4372-bf1b-791406c99a10" containerID="32b2827a6a7383e227807bbec57841e55b9ce266bdf59eb614f1d72536568ee7" exitCode=0
Sep 10 17:53:03.740708 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:53:03.740344    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-48d00b-openshift-default-6cc74dc546-kjh62" event={"ID":"20ec55f7-fa46-4372-bf1b-791406c99a10","Type":"ContainerDied","Data":"32b2827a6a7383e227807bbec57841e55b9ce266bdf59eb614f1d72536568ee7"}
Sep 10 17:53:03.740708 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:53:03.740398    2581 scope.go:117] "RemoveContainer" containerID="a992fe6c27c2c21e201c10a259fbcbcf8850ea7283b2cdbc34a9974033bf49a9"
Sep 10 17:53:03.848073 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:53:03.848046    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-48d00b-openshift-default-6cc74dc546-kjh62"
Sep 10 17:53:03.988808 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:53:03.988767    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/20ec55f7-fa46-4372-bf1b-791406c99a10-istio-podinfo\") pod \"20ec55f7-fa46-4372-bf1b-791406c99a10\" (UID: \"20ec55f7-fa46-4372-bf1b-791406c99a10\") "
Sep 10 17:53:03.988808 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:53:03.988817    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/20ec55f7-fa46-4372-bf1b-791406c99a10-credential-socket\") pod \"20ec55f7-fa46-4372-bf1b-791406c99a10\" (UID: \"20ec55f7-fa46-4372-bf1b-791406c99a10\") "
Sep 10 17:53:03.989048 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:53:03.988843    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/20ec55f7-fa46-4372-bf1b-791406c99a10-istio-envoy\") pod \"20ec55f7-fa46-4372-bf1b-791406c99a10\" (UID: \"20ec55f7-fa46-4372-bf1b-791406c99a10\") "
Sep 10 17:53:03.989048 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:53:03.988877    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/20ec55f7-fa46-4372-bf1b-791406c99a10-istio-data\") pod \"20ec55f7-fa46-4372-bf1b-791406c99a10\" (UID: \"20ec55f7-fa46-4372-bf1b-791406c99a10\") "
Sep 10 17:53:03.989048 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:53:03.988920    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/20ec55f7-fa46-4372-bf1b-791406c99a10-istio-token\") pod \"20ec55f7-fa46-4372-bf1b-791406c99a10\" (UID: \"20ec55f7-fa46-4372-bf1b-791406c99a10\") "
Sep 10 17:53:03.989048 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:53:03.988944    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/20ec55f7-fa46-4372-bf1b-791406c99a10-istiod-ca-cert\") pod \"20ec55f7-fa46-4372-bf1b-791406c99a10\" (UID: \"20ec55f7-fa46-4372-bf1b-791406c99a10\") "
Sep 10 17:53:03.989048 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:53:03.988987    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-9lnh7\" (UniqueName: \"kubernetes.io/projected/20ec55f7-fa46-4372-bf1b-791406c99a10-kube-api-access-9lnh7\") pod \"20ec55f7-fa46-4372-bf1b-791406c99a10\" (UID: \"20ec55f7-fa46-4372-bf1b-791406c99a10\") "
Sep 10 17:53:03.989048 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:53:03.989033    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/20ec55f7-fa46-4372-bf1b-791406c99a10-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "20ec55f7-fa46-4372-bf1b-791406c99a10" (UID: "20ec55f7-fa46-4372-bf1b-791406c99a10"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 17:53:03.989375 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:53:03.989067    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/20ec55f7-fa46-4372-bf1b-791406c99a10-workload-socket\") pod \"20ec55f7-fa46-4372-bf1b-791406c99a10\" (UID: \"20ec55f7-fa46-4372-bf1b-791406c99a10\") "
Sep 10 17:53:03.989375 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:53:03.989117    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/20ec55f7-fa46-4372-bf1b-791406c99a10-workload-certs\") pod \"20ec55f7-fa46-4372-bf1b-791406c99a10\" (UID: \"20ec55f7-fa46-4372-bf1b-791406c99a10\") "
Sep 10 17:53:03.989483 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:53:03.989374    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/20ec55f7-fa46-4372-bf1b-791406c99a10-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "20ec55f7-fa46-4372-bf1b-791406c99a10" (UID: "20ec55f7-fa46-4372-bf1b-791406c99a10"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 17:53:03.989483 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:53:03.989391    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/20ec55f7-fa46-4372-bf1b-791406c99a10-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "20ec55f7-fa46-4372-bf1b-791406c99a10" (UID: "20ec55f7-fa46-4372-bf1b-791406c99a10"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 17:53:03.989595 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:53:03.989579    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/20ec55f7-fa46-4372-bf1b-791406c99a10-istio-data" (OuterVolumeSpecName: "istio-data") pod "20ec55f7-fa46-4372-bf1b-791406c99a10" (UID: "20ec55f7-fa46-4372-bf1b-791406c99a10"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 17:53:03.989681 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:53:03.989583    2581 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/20ec55f7-fa46-4372-bf1b-791406c99a10-workload-socket\") on node \"ip-10-0-141-180.ec2.internal\" DevicePath \"\""
Sep 10 17:53:03.989681 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:53:03.989611    2581 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/20ec55f7-fa46-4372-bf1b-791406c99a10-credential-socket\") on node \"ip-10-0-141-180.ec2.internal\" DevicePath \"\""
Sep 10 17:53:03.989681 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:53:03.989626    2581 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/20ec55f7-fa46-4372-bf1b-791406c99a10-istiod-ca-cert\") on node \"ip-10-0-141-180.ec2.internal\" DevicePath \"\""
Sep 10 17:53:03.990226 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:53:03.990202    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/20ec55f7-fa46-4372-bf1b-791406c99a10-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "20ec55f7-fa46-4372-bf1b-791406c99a10" (UID: "20ec55f7-fa46-4372-bf1b-791406c99a10"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 17:53:03.991303 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:53:03.991272    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/20ec55f7-fa46-4372-bf1b-791406c99a10-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "20ec55f7-fa46-4372-bf1b-791406c99a10" (UID: "20ec55f7-fa46-4372-bf1b-791406c99a10"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 17:53:03.991303 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:53:03.991294    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/20ec55f7-fa46-4372-bf1b-791406c99a10-kube-api-access-9lnh7" (OuterVolumeSpecName: "kube-api-access-9lnh7") pod "20ec55f7-fa46-4372-bf1b-791406c99a10" (UID: "20ec55f7-fa46-4372-bf1b-791406c99a10"). InnerVolumeSpecName "kube-api-access-9lnh7". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 17:53:03.991527 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:53:03.991502    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/20ec55f7-fa46-4372-bf1b-791406c99a10-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "20ec55f7-fa46-4372-bf1b-791406c99a10" (UID: "20ec55f7-fa46-4372-bf1b-791406c99a10"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 10 17:53:03.991527 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:53:03.991511    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/20ec55f7-fa46-4372-bf1b-791406c99a10-istio-token" (OuterVolumeSpecName: "istio-token") pod "20ec55f7-fa46-4372-bf1b-791406c99a10" (UID: "20ec55f7-fa46-4372-bf1b-791406c99a10"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 17:53:04.090330 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:53:04.090276    2581 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/20ec55f7-fa46-4372-bf1b-791406c99a10-workload-certs\") on node \"ip-10-0-141-180.ec2.internal\" DevicePath \"\""
Sep 10 17:53:04.090330 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:53:04.090322    2581 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/20ec55f7-fa46-4372-bf1b-791406c99a10-istio-podinfo\") on node \"ip-10-0-141-180.ec2.internal\" DevicePath \"\""
Sep 10 17:53:04.090330 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:53:04.090334    2581 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/20ec55f7-fa46-4372-bf1b-791406c99a10-istio-envoy\") on node \"ip-10-0-141-180.ec2.internal\" DevicePath \"\""
Sep 10 17:53:04.090568 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:53:04.090346    2581 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/20ec55f7-fa46-4372-bf1b-791406c99a10-istio-data\") on node \"ip-10-0-141-180.ec2.internal\" DevicePath \"\""
Sep 10 17:53:04.090568 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:53:04.090357    2581 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/20ec55f7-fa46-4372-bf1b-791406c99a10-istio-token\") on node \"ip-10-0-141-180.ec2.internal\" DevicePath \"\""
Sep 10 17:53:04.090568 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:53:04.090368    2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9lnh7\" (UniqueName: \"kubernetes.io/projected/20ec55f7-fa46-4372-bf1b-791406c99a10-kube-api-access-9lnh7\") on node \"ip-10-0-141-180.ec2.internal\" DevicePath \"\""
Sep 10 17:53:04.754727 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:53:04.754698    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-48d00b-openshift-default-6cc74dc546-kjh62"
Sep 10 17:53:04.754727 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:53:04.754707    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-48d00b-openshift-default-6cc74dc546-kjh62" event={"ID":"20ec55f7-fa46-4372-bf1b-791406c99a10","Type":"ContainerDied","Data":"9cde959c748ee5b9356027f255d04830d9144237ae707d240090a62d60b18c75"}
Sep 10 17:53:04.755211 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:53:04.754786    2581 scope.go:117] "RemoveContainer" containerID="32b2827a6a7383e227807bbec57841e55b9ce266bdf59eb614f1d72536568ee7"
Sep 10 17:53:04.779399 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:53:04.779375    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-a-48d00b-openshift-default-6cc74dc546-kjh62"]
Sep 10 17:53:04.783793 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:53:04.783771    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-inf-a-48d00b-openshift-default-6cc74dc546-kjh62"]
Sep 10 17:53:04.863095 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:53:04.863068    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="20ec55f7-fa46-4372-bf1b-791406c99a10" path="/var/lib/kubelet/pods/20ec55f7-fa46-4372-bf1b-791406c99a10/volumes"
Sep 10 17:53:06.258950 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:53:06.258916    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-c523e8a0-n7w8k"]
Sep 10 17:53:06.259364 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:53:06.259351    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3f5843ad-da03-4167-9537-b5665deeca7d" containerName="revoke-keys"
Sep 10 17:53:06.259426 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:53:06.259366    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="3f5843ad-da03-4167-9537-b5665deeca7d" containerName="revoke-keys"
Sep 10 17:53:06.259426 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:53:06.259375    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="20ec55f7-fa46-4372-bf1b-791406c99a10" containerName="istio-proxy"
Sep 10 17:53:06.259426 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:53:06.259381    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="20ec55f7-fa46-4372-bf1b-791406c99a10" containerName="istio-proxy"
Sep 10 17:53:06.259426 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:53:06.259390    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="20ec55f7-fa46-4372-bf1b-791406c99a10" containerName="istio-proxy"
Sep 10 17:53:06.259426 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:53:06.259397    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="20ec55f7-fa46-4372-bf1b-791406c99a10" containerName="istio-proxy"
Sep 10 17:53:06.259598 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:53:06.259473    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="20ec55f7-fa46-4372-bf1b-791406c99a10" containerName="istio-proxy"
Sep 10 17:53:06.259598 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:53:06.259480    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="20ec55f7-fa46-4372-bf1b-791406c99a10" containerName="istio-proxy"
Sep 10 17:53:06.259598 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:53:06.259489    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="3f5843ad-da03-4167-9537-b5665deeca7d" containerName="revoke-keys"
Sep 10 17:53:06.262711 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:53:06.262691    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-c523e8a0-n7w8k"
Sep 10 17:53:06.268674 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:53:06.268650    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 10 17:53:06.268824 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:53:06.268732    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-k7zj6\""
Sep 10 17:53:06.278438 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:53:06.278410    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-c523e8a0-n7w8k"]
Sep 10 17:53:06.412106 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:53:06.412069    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/3bfeae9c-6656-4fe4-8232-921f5e208c8d-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-c523e8a0-n7w8k\" (UID: \"3bfeae9c-6656-4fe4-8232-921f5e208c8d\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-c523e8a0-n7w8k"
Sep 10 17:53:06.513448 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:53:06.513366    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/3bfeae9c-6656-4fe4-8232-921f5e208c8d-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-c523e8a0-n7w8k\" (UID: \"3bfeae9c-6656-4fe4-8232-921f5e208c8d\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-c523e8a0-n7w8k"
Sep 10 17:53:06.514060 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:53:06.514028    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/3bfeae9c-6656-4fe4-8232-921f5e208c8d-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-c523e8a0-n7w8k\" (UID: \"3bfeae9c-6656-4fe4-8232-921f5e208c8d\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-c523e8a0-n7w8k"
Sep 10 17:53:06.575379 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:53:06.575347    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-c523e8a0-n7w8k"
Sep 10 17:53:06.863451 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:53:06.863423    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-c523e8a0-n7w8k"]
Sep 10 17:53:06.864422 ip-10-0-141-180 kubenswrapper[2581]: W0910 17:53:06.864396    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3bfeae9c_6656_4fe4_8232_921f5e208c8d.slice/crio-6e55e3e537835396b3c78b30cddca1c7d352f27d10d93d9e3a130f973b63f5e0 WatchSource:0}: Error finding container 6e55e3e537835396b3c78b30cddca1c7d352f27d10d93d9e3a130f973b63f5e0: Status 404 returned error can't find the container with id 6e55e3e537835396b3c78b30cddca1c7d352f27d10d93d9e3a130f973b63f5e0
Sep 10 17:53:07.777946 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:53:07.777902    2581 generic.go:356] "Generic (PLEG): container finished" podID="3bfeae9c-6656-4fe4-8232-921f5e208c8d" containerID="2d74a641ab858634fdd44da7c4223c98256496a7ad9745e8889f52c4f600cba1" exitCode=0
Sep 10 17:53:07.778398 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:53:07.778082    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-c523e8a0-n7w8k" event={"ID":"3bfeae9c-6656-4fe4-8232-921f5e208c8d","Type":"ContainerDied","Data":"2d74a641ab858634fdd44da7c4223c98256496a7ad9745e8889f52c4f600cba1"}
Sep 10 17:53:07.778398 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:53:07.778141    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-c523e8a0-n7w8k" event={"ID":"3bfeae9c-6656-4fe4-8232-921f5e208c8d","Type":"ContainerStarted","Data":"6e55e3e537835396b3c78b30cddca1c7d352f27d10d93d9e3a130f973b63f5e0"}
Sep 10 17:53:08.920171 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:53:08.920141    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-c523e8a0-n7w8k"
Sep 10 17:53:09.040469 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:53:09.040385    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/3bfeae9c-6656-4fe4-8232-921f5e208c8d-maas-api-service-ca\") pod \"3bfeae9c-6656-4fe4-8232-921f5e208c8d\" (UID: \"3bfeae9c-6656-4fe4-8232-921f5e208c8d\") "
Sep 10 17:53:09.040769 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:53:09.040745    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3bfeae9c-6656-4fe4-8232-921f5e208c8d-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "3bfeae9c-6656-4fe4-8232-921f5e208c8d" (UID: "3bfeae9c-6656-4fe4-8232-921f5e208c8d"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 17:53:09.141410 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:53:09.141365    2581 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/3bfeae9c-6656-4fe4-8232-921f5e208c8d-maas-api-service-ca\") on node \"ip-10-0-141-180.ec2.internal\" DevicePath \"\""
Sep 10 17:53:09.796435 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:53:09.796396    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-c523e8a0-n7w8k" event={"ID":"3bfeae9c-6656-4fe4-8232-921f5e208c8d","Type":"ContainerDied","Data":"6e55e3e537835396b3c78b30cddca1c7d352f27d10d93d9e3a130f973b63f5e0"}
Sep 10 17:53:09.796602 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:53:09.796477    2581 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="6e55e3e537835396b3c78b30cddca1c7d352f27d10d93d9e3a130f973b63f5e0"
Sep 10 17:53:09.796602 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:53:09.796412    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-c523e8a0-n7w8k"
Sep 10 17:53:09.849014 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:53:09.848979    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-b-783a5e-7b8b7f74fd-9r6bx"]
Sep 10 17:53:09.849252 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:53:09.849224    2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-783a5e-7b8b7f74fd-9r6bx" podUID="787f82f8-1081-42c7-b950-763bcbae5d2c" containerName="payload-pre-processing" containerID="cri-o://ae08db3b74825ce340cca9f23fc367319a4618733e8af4a9a3b3e025ca2de72b" gracePeriod=30
Sep 10 17:53:10.108375 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:53:10.108351    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-783a5e-7b8b7f74fd-9r6bx"
Sep 10 17:53:10.125152 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:53:10.125126    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-783a5e-lw6fv"]
Sep 10 17:53:10.134302 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:53:10.134278    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-783a5e-lw6fv"]
Sep 10 17:53:10.253798 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:53:10.253767    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/787f82f8-1081-42c7-b950-763bcbae5d2c-plugins-config-volume\") pod \"787f82f8-1081-42c7-b950-763bcbae5d2c\" (UID: \"787f82f8-1081-42c7-b950-763bcbae5d2c\") "
Sep 10 17:53:10.253997 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:53:10.253824    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-zh5zv\" (UniqueName: \"kubernetes.io/projected/787f82f8-1081-42c7-b950-763bcbae5d2c-kube-api-access-zh5zv\") pod \"787f82f8-1081-42c7-b950-763bcbae5d2c\" (UID: \"787f82f8-1081-42c7-b950-763bcbae5d2c\") "
Sep 10 17:53:10.254205 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:53:10.254178    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/787f82f8-1081-42c7-b950-763bcbae5d2c-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "787f82f8-1081-42c7-b950-763bcbae5d2c" (UID: "787f82f8-1081-42c7-b950-763bcbae5d2c"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 17:53:10.255893 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:53:10.255872    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/787f82f8-1081-42c7-b950-763bcbae5d2c-kube-api-access-zh5zv" (OuterVolumeSpecName: "kube-api-access-zh5zv") pod "787f82f8-1081-42c7-b950-763bcbae5d2c" (UID: "787f82f8-1081-42c7-b950-763bcbae5d2c"). InnerVolumeSpecName "kube-api-access-zh5zv". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 17:53:10.355336 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:53:10.355247    2581 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/787f82f8-1081-42c7-b950-763bcbae5d2c-plugins-config-volume\") on node \"ip-10-0-141-180.ec2.internal\" DevicePath \"\""
Sep 10 17:53:10.355336 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:53:10.355278    2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-zh5zv\" (UniqueName: \"kubernetes.io/projected/787f82f8-1081-42c7-b950-763bcbae5d2c-kube-api-access-zh5zv\") on node \"ip-10-0-141-180.ec2.internal\" DevicePath \"\""
Sep 10 17:53:10.804613 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:53:10.804575    2581 generic.go:356] "Generic (PLEG): container finished" podID="787f82f8-1081-42c7-b950-763bcbae5d2c" containerID="ae08db3b74825ce340cca9f23fc367319a4618733e8af4a9a3b3e025ca2de72b" exitCode=0
Sep 10 17:53:10.804811 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:53:10.804621    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-783a5e-7b8b7f74fd-9r6bx"
Sep 10 17:53:10.804811 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:53:10.804661    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-783a5e-7b8b7f74fd-9r6bx" event={"ID":"787f82f8-1081-42c7-b950-763bcbae5d2c","Type":"ContainerDied","Data":"ae08db3b74825ce340cca9f23fc367319a4618733e8af4a9a3b3e025ca2de72b"}
Sep 10 17:53:10.804811 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:53:10.804728    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-783a5e-7b8b7f74fd-9r6bx" event={"ID":"787f82f8-1081-42c7-b950-763bcbae5d2c","Type":"ContainerDied","Data":"81e3d68fd924d1882444f7bf98aa0b2104b29d72d1176e10910142a9592bae44"}
Sep 10 17:53:10.804811 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:53:10.804757    2581 scope.go:117] "RemoveContainer" containerID="ae08db3b74825ce340cca9f23fc367319a4618733e8af4a9a3b3e025ca2de72b"
Sep 10 17:53:10.815182 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:53:10.815161    2581 scope.go:117] "RemoveContainer" containerID="ae08db3b74825ce340cca9f23fc367319a4618733e8af4a9a3b3e025ca2de72b"
Sep 10 17:53:10.815453 ip-10-0-141-180 kubenswrapper[2581]: E0910 17:53:10.815434    2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ae08db3b74825ce340cca9f23fc367319a4618733e8af4a9a3b3e025ca2de72b\": container with ID starting with ae08db3b74825ce340cca9f23fc367319a4618733e8af4a9a3b3e025ca2de72b not found: ID does not exist" containerID="ae08db3b74825ce340cca9f23fc367319a4618733e8af4a9a3b3e025ca2de72b"
Sep 10 17:53:10.815503 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:53:10.815462    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ae08db3b74825ce340cca9f23fc367319a4618733e8af4a9a3b3e025ca2de72b"} err="failed to get container status \"ae08db3b74825ce340cca9f23fc367319a4618733e8af4a9a3b3e025ca2de72b\": rpc error: code = NotFound desc = could not find container \"ae08db3b74825ce340cca9f23fc367319a4618733e8af4a9a3b3e025ca2de72b\": container with ID starting with ae08db3b74825ce340cca9f23fc367319a4618733e8af4a9a3b3e025ca2de72b not found: ID does not exist"
Sep 10 17:53:10.831200 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:53:10.831169    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-b-783a5e-7b8b7f74fd-9r6bx"]
Sep 10 17:53:10.837358 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:53:10.837336    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-b-783a5e-7b8b7f74fd-9r6bx"]
Sep 10 17:53:10.862878 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:53:10.862849    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3f5843ad-da03-4167-9537-b5665deeca7d" path="/var/lib/kubelet/pods/3f5843ad-da03-4167-9537-b5665deeca7d/volumes"
Sep 10 17:53:10.863225 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:53:10.863211    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="787f82f8-1081-42c7-b950-763bcbae5d2c" path="/var/lib/kubelet/pods/787f82f8-1081-42c7-b950-763bcbae5d2c/volumes"
Sep 10 17:53:17.275242 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:53:17.275206    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-c523e8a0-67b96f7577-ds7xz"]
Sep 10 17:53:17.275732 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:53:17.275460    2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-mt-c523e8a0-67b96f7577-ds7xz" podUID="4b171014-a09d-4221-baa7-2afc58afa8b9" containerName="payload-processing" containerID="cri-o://0005efe1164262e0aeb4d244c7d8a98e4524fc47a4f795cfd6161d5a1e46ba8c" gracePeriod=30
Sep 10 17:53:17.528128 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:53:17.528065    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-c523e8a0-67b96f7577-ds7xz"
Sep 10 17:53:17.603306 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:53:17.603207    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-c523e8a0-n7w8k"]
Sep 10 17:53:17.608403 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:53:17.608343    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-c523e8a0-n7w8k"]
Sep 10 17:53:17.629476 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:53:17.629428    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/4b171014-a09d-4221-baa7-2afc58afa8b9-plugins-config-volume\") pod \"4b171014-a09d-4221-baa7-2afc58afa8b9\" (UID: \"4b171014-a09d-4221-baa7-2afc58afa8b9\") "
Sep 10 17:53:17.629476 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:53:17.629488    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-5rwch\" (UniqueName: \"kubernetes.io/projected/4b171014-a09d-4221-baa7-2afc58afa8b9-kube-api-access-5rwch\") pod \"4b171014-a09d-4221-baa7-2afc58afa8b9\" (UID: \"4b171014-a09d-4221-baa7-2afc58afa8b9\") "
Sep 10 17:53:17.629926 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:53:17.629898    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4b171014-a09d-4221-baa7-2afc58afa8b9-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "4b171014-a09d-4221-baa7-2afc58afa8b9" (UID: "4b171014-a09d-4221-baa7-2afc58afa8b9"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 17:53:17.631692 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:53:17.631663    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4b171014-a09d-4221-baa7-2afc58afa8b9-kube-api-access-5rwch" (OuterVolumeSpecName: "kube-api-access-5rwch") pod "4b171014-a09d-4221-baa7-2afc58afa8b9" (UID: "4b171014-a09d-4221-baa7-2afc58afa8b9"). InnerVolumeSpecName "kube-api-access-5rwch". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 17:53:17.731078 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:53:17.731045    2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5rwch\" (UniqueName: \"kubernetes.io/projected/4b171014-a09d-4221-baa7-2afc58afa8b9-kube-api-access-5rwch\") on node \"ip-10-0-141-180.ec2.internal\" DevicePath \"\""
Sep 10 17:53:17.731078 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:53:17.731072    2581 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/4b171014-a09d-4221-baa7-2afc58afa8b9-plugins-config-volume\") on node \"ip-10-0-141-180.ec2.internal\" DevicePath \"\""
Sep 10 17:53:17.853128 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:53:17.853042    2581 generic.go:356] "Generic (PLEG): container finished" podID="4b171014-a09d-4221-baa7-2afc58afa8b9" containerID="0005efe1164262e0aeb4d244c7d8a98e4524fc47a4f795cfd6161d5a1e46ba8c" exitCode=0
Sep 10 17:53:17.853128 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:53:17.853087    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-c523e8a0-67b96f7577-ds7xz"
Sep 10 17:53:17.853309 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:53:17.853124    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-c523e8a0-67b96f7577-ds7xz" event={"ID":"4b171014-a09d-4221-baa7-2afc58afa8b9","Type":"ContainerDied","Data":"0005efe1164262e0aeb4d244c7d8a98e4524fc47a4f795cfd6161d5a1e46ba8c"}
Sep 10 17:53:17.853309 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:53:17.853181    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-c523e8a0-67b96f7577-ds7xz" event={"ID":"4b171014-a09d-4221-baa7-2afc58afa8b9","Type":"ContainerDied","Data":"fcf1c40f0de7124e6ccba6d3fca186acb69a0b4da2082415f8d88ff100257f31"}
Sep 10 17:53:17.853309 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:53:17.853204    2581 scope.go:117] "RemoveContainer" containerID="0005efe1164262e0aeb4d244c7d8a98e4524fc47a4f795cfd6161d5a1e46ba8c"
Sep 10 17:53:17.862710 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:53:17.862692    2581 scope.go:117] "RemoveContainer" containerID="0005efe1164262e0aeb4d244c7d8a98e4524fc47a4f795cfd6161d5a1e46ba8c"
Sep 10 17:53:17.862982 ip-10-0-141-180 kubenswrapper[2581]: E0910 17:53:17.862959    2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0005efe1164262e0aeb4d244c7d8a98e4524fc47a4f795cfd6161d5a1e46ba8c\": container with ID starting with 0005efe1164262e0aeb4d244c7d8a98e4524fc47a4f795cfd6161d5a1e46ba8c not found: ID does not exist" containerID="0005efe1164262e0aeb4d244c7d8a98e4524fc47a4f795cfd6161d5a1e46ba8c"
Sep 10 17:53:17.863034 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:53:17.862990    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0005efe1164262e0aeb4d244c7d8a98e4524fc47a4f795cfd6161d5a1e46ba8c"} err="failed to get container status \"0005efe1164262e0aeb4d244c7d8a98e4524fc47a4f795cfd6161d5a1e46ba8c\": rpc error: code = NotFound desc = could not find container \"0005efe1164262e0aeb4d244c7d8a98e4524fc47a4f795cfd6161d5a1e46ba8c\": container with ID starting with 0005efe1164262e0aeb4d244c7d8a98e4524fc47a4f795cfd6161d5a1e46ba8c not found: ID does not exist"
Sep 10 17:53:17.903579 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:53:17.903553    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-c523e8a0-67b96f7577-ds7xz"]
Sep 10 17:53:17.918564 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:53:17.918538    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-c523e8a0-67b96f7577-ds7xz"]
Sep 10 17:53:18.027213 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:53:18.027172    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-b-783a5e-openshift-default-5dd94f65d-bxhrc"]
Sep 10 17:53:18.027479 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:53:18.027449    2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-inf-b-783a5e-openshift-default-5dd94f65d-bxhrc" podUID="a628ea2c-8ec6-4d36-99b5-db4f7927de9d" containerName="istio-proxy" containerID="cri-o://eaf96441f2fb4a1737394476495cc0ac2d6535ab6fa866d3089441ac827239a3" gracePeriod=30
Sep 10 17:53:18.766260 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:53:18.766219    2581 patch_prober.go:28] interesting pod/e2e-inf-b-783a5e-openshift-default-5dd94f65d-bxhrc container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Sep 10 17:53:18.766720 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:53:18.766279    2581 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-inf-b-783a5e-openshift-default-5dd94f65d-bxhrc" podUID="a628ea2c-8ec6-4d36-99b5-db4f7927de9d" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Sep 10 17:53:18.862936 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:53:18.862902    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3bfeae9c-6656-4fe4-8232-921f5e208c8d" path="/var/lib/kubelet/pods/3bfeae9c-6656-4fe4-8232-921f5e208c8d/volumes"
Sep 10 17:53:18.863293 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:53:18.863279    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4b171014-a09d-4221-baa7-2afc58afa8b9" path="/var/lib/kubelet/pods/4b171014-a09d-4221-baa7-2afc58afa8b9/volumes"
Sep 10 17:53:23.283454 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:53:23.283431    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-783a5e-openshift-default-5dd94f65d-bxhrc"
Sep 10 17:53:23.382737 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:53:23.382634    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-5qxt4\" (UniqueName: \"kubernetes.io/projected/a628ea2c-8ec6-4d36-99b5-db4f7927de9d-kube-api-access-5qxt4\") pod \"a628ea2c-8ec6-4d36-99b5-db4f7927de9d\" (UID: \"a628ea2c-8ec6-4d36-99b5-db4f7927de9d\") "
Sep 10 17:53:23.382888 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:53:23.382743    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a628ea2c-8ec6-4d36-99b5-db4f7927de9d-workload-socket\") pod \"a628ea2c-8ec6-4d36-99b5-db4f7927de9d\" (UID: \"a628ea2c-8ec6-4d36-99b5-db4f7927de9d\") "
Sep 10 17:53:23.382888 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:53:23.382773    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a628ea2c-8ec6-4d36-99b5-db4f7927de9d-credential-socket\") pod \"a628ea2c-8ec6-4d36-99b5-db4f7927de9d\" (UID: \"a628ea2c-8ec6-4d36-99b5-db4f7927de9d\") "
Sep 10 17:53:23.382888 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:53:23.382805    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a628ea2c-8ec6-4d36-99b5-db4f7927de9d-istio-token\") pod \"a628ea2c-8ec6-4d36-99b5-db4f7927de9d\" (UID: \"a628ea2c-8ec6-4d36-99b5-db4f7927de9d\") "
Sep 10 17:53:23.382888 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:53:23.382841    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a628ea2c-8ec6-4d36-99b5-db4f7927de9d-istio-envoy\") pod \"a628ea2c-8ec6-4d36-99b5-db4f7927de9d\" (UID: \"a628ea2c-8ec6-4d36-99b5-db4f7927de9d\") "
Sep 10 17:53:23.383113 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:53:23.382890    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a628ea2c-8ec6-4d36-99b5-db4f7927de9d-workload-certs\") pod \"a628ea2c-8ec6-4d36-99b5-db4f7927de9d\" (UID: \"a628ea2c-8ec6-4d36-99b5-db4f7927de9d\") "
Sep 10 17:53:23.383113 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:53:23.382932    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a628ea2c-8ec6-4d36-99b5-db4f7927de9d-istio-podinfo\") pod \"a628ea2c-8ec6-4d36-99b5-db4f7927de9d\" (UID: \"a628ea2c-8ec6-4d36-99b5-db4f7927de9d\") "
Sep 10 17:53:23.383113 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:53:23.382957    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a628ea2c-8ec6-4d36-99b5-db4f7927de9d-istiod-ca-cert\") pod \"a628ea2c-8ec6-4d36-99b5-db4f7927de9d\" (UID: \"a628ea2c-8ec6-4d36-99b5-db4f7927de9d\") "
Sep 10 17:53:23.383113 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:53:23.382998    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a628ea2c-8ec6-4d36-99b5-db4f7927de9d-istio-data\") pod \"a628ea2c-8ec6-4d36-99b5-db4f7927de9d\" (UID: \"a628ea2c-8ec6-4d36-99b5-db4f7927de9d\") "
Sep 10 17:53:23.383113 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:53:23.383087    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a628ea2c-8ec6-4d36-99b5-db4f7927de9d-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "a628ea2c-8ec6-4d36-99b5-db4f7927de9d" (UID: "a628ea2c-8ec6-4d36-99b5-db4f7927de9d"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 17:53:23.383370 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:53:23.383082    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a628ea2c-8ec6-4d36-99b5-db4f7927de9d-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "a628ea2c-8ec6-4d36-99b5-db4f7927de9d" (UID: "a628ea2c-8ec6-4d36-99b5-db4f7927de9d"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 17:53:23.383370 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:53:23.383156    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a628ea2c-8ec6-4d36-99b5-db4f7927de9d-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "a628ea2c-8ec6-4d36-99b5-db4f7927de9d" (UID: "a628ea2c-8ec6-4d36-99b5-db4f7927de9d"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 17:53:23.383370 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:53:23.383323    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a628ea2c-8ec6-4d36-99b5-db4f7927de9d-istio-data" (OuterVolumeSpecName: "istio-data") pod "a628ea2c-8ec6-4d36-99b5-db4f7927de9d" (UID: "a628ea2c-8ec6-4d36-99b5-db4f7927de9d"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 17:53:23.383370 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:53:23.383346    2581 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a628ea2c-8ec6-4d36-99b5-db4f7927de9d-workload-socket\") on node \"ip-10-0-141-180.ec2.internal\" DevicePath \"\""
Sep 10 17:53:23.383370 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:53:23.383365    2581 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a628ea2c-8ec6-4d36-99b5-db4f7927de9d-credential-socket\") on node \"ip-10-0-141-180.ec2.internal\" DevicePath \"\""
Sep 10 17:53:23.383671 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:53:23.383379    2581 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a628ea2c-8ec6-4d36-99b5-db4f7927de9d-workload-certs\") on node \"ip-10-0-141-180.ec2.internal\" DevicePath \"\""
Sep 10 17:53:23.383671 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:53:23.383418    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a628ea2c-8ec6-4d36-99b5-db4f7927de9d-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "a628ea2c-8ec6-4d36-99b5-db4f7927de9d" (UID: "a628ea2c-8ec6-4d36-99b5-db4f7927de9d"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 17:53:23.385071 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:53:23.385046    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a628ea2c-8ec6-4d36-99b5-db4f7927de9d-istio-token" (OuterVolumeSpecName: "istio-token") pod "a628ea2c-8ec6-4d36-99b5-db4f7927de9d" (UID: "a628ea2c-8ec6-4d36-99b5-db4f7927de9d"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 17:53:23.385198 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:53:23.385140    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/a628ea2c-8ec6-4d36-99b5-db4f7927de9d-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "a628ea2c-8ec6-4d36-99b5-db4f7927de9d" (UID: "a628ea2c-8ec6-4d36-99b5-db4f7927de9d"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 10 17:53:23.385272 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:53:23.385249    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a628ea2c-8ec6-4d36-99b5-db4f7927de9d-kube-api-access-5qxt4" (OuterVolumeSpecName: "kube-api-access-5qxt4") pod "a628ea2c-8ec6-4d36-99b5-db4f7927de9d" (UID: "a628ea2c-8ec6-4d36-99b5-db4f7927de9d"). InnerVolumeSpecName "kube-api-access-5qxt4". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 17:53:23.385382 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:53:23.385359    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a628ea2c-8ec6-4d36-99b5-db4f7927de9d-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "a628ea2c-8ec6-4d36-99b5-db4f7927de9d" (UID: "a628ea2c-8ec6-4d36-99b5-db4f7927de9d"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 17:53:23.484547 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:53:23.484514    2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5qxt4\" (UniqueName: \"kubernetes.io/projected/a628ea2c-8ec6-4d36-99b5-db4f7927de9d-kube-api-access-5qxt4\") on node \"ip-10-0-141-180.ec2.internal\" DevicePath \"\""
Sep 10 17:53:23.484547 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:53:23.484542    2581 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a628ea2c-8ec6-4d36-99b5-db4f7927de9d-istio-token\") on node \"ip-10-0-141-180.ec2.internal\" DevicePath \"\""
Sep 10 17:53:23.484547 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:53:23.484552    2581 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a628ea2c-8ec6-4d36-99b5-db4f7927de9d-istio-envoy\") on node \"ip-10-0-141-180.ec2.internal\" DevicePath \"\""
Sep 10 17:53:23.484806 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:53:23.484559    2581 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a628ea2c-8ec6-4d36-99b5-db4f7927de9d-istio-podinfo\") on node \"ip-10-0-141-180.ec2.internal\" DevicePath \"\""
Sep 10 17:53:23.484806 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:53:23.484568    2581 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a628ea2c-8ec6-4d36-99b5-db4f7927de9d-istiod-ca-cert\") on node \"ip-10-0-141-180.ec2.internal\" DevicePath \"\""
Sep 10 17:53:23.484806 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:53:23.484575    2581 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a628ea2c-8ec6-4d36-99b5-db4f7927de9d-istio-data\") on node \"ip-10-0-141-180.ec2.internal\" DevicePath \"\""
Sep 10 17:53:23.902624 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:53:23.902588    2581 generic.go:356] "Generic (PLEG): container finished" podID="a628ea2c-8ec6-4d36-99b5-db4f7927de9d" containerID="eaf96441f2fb4a1737394476495cc0ac2d6535ab6fa866d3089441ac827239a3" exitCode=0
Sep 10 17:53:23.902832 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:53:23.902662    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-783a5e-openshift-default-5dd94f65d-bxhrc"
Sep 10 17:53:23.902832 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:53:23.902687    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-783a5e-openshift-default-5dd94f65d-bxhrc" event={"ID":"a628ea2c-8ec6-4d36-99b5-db4f7927de9d","Type":"ContainerDied","Data":"eaf96441f2fb4a1737394476495cc0ac2d6535ab6fa866d3089441ac827239a3"}
Sep 10 17:53:23.902832 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:53:23.902773    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-783a5e-openshift-default-5dd94f65d-bxhrc" event={"ID":"a628ea2c-8ec6-4d36-99b5-db4f7927de9d","Type":"ContainerDied","Data":"9aa497f6994c6eb15a1fb52fb684caaae9e1bb2f6d2524a24086fd5e7257e3ac"}
Sep 10 17:53:23.902832 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:53:23.902802    2581 scope.go:117] "RemoveContainer" containerID="eaf96441f2fb4a1737394476495cc0ac2d6535ab6fa866d3089441ac827239a3"
Sep 10 17:53:23.913171 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:53:23.913152    2581 scope.go:117] "RemoveContainer" containerID="0e01bafb578bfa16f30a63f5ef6a4856daee5fb7a5a772c95c9dac27505996d7"
Sep 10 17:53:23.921554 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:53:23.921540    2581 scope.go:117] "RemoveContainer" containerID="eaf96441f2fb4a1737394476495cc0ac2d6535ab6fa866d3089441ac827239a3"
Sep 10 17:53:23.921834 ip-10-0-141-180 kubenswrapper[2581]: E0910 17:53:23.921814    2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"eaf96441f2fb4a1737394476495cc0ac2d6535ab6fa866d3089441ac827239a3\": container with ID starting with eaf96441f2fb4a1737394476495cc0ac2d6535ab6fa866d3089441ac827239a3 not found: ID does not exist" containerID="eaf96441f2fb4a1737394476495cc0ac2d6535ab6fa866d3089441ac827239a3"
Sep 10 17:53:23.921915 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:53:23.921845    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"eaf96441f2fb4a1737394476495cc0ac2d6535ab6fa866d3089441ac827239a3"} err="failed to get container status \"eaf96441f2fb4a1737394476495cc0ac2d6535ab6fa866d3089441ac827239a3\": rpc error: code = NotFound desc = could not find container \"eaf96441f2fb4a1737394476495cc0ac2d6535ab6fa866d3089441ac827239a3\": container with ID starting with eaf96441f2fb4a1737394476495cc0ac2d6535ab6fa866d3089441ac827239a3 not found: ID does not exist"
Sep 10 17:53:23.921915 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:53:23.921868    2581 scope.go:117] "RemoveContainer" containerID="0e01bafb578bfa16f30a63f5ef6a4856daee5fb7a5a772c95c9dac27505996d7"
Sep 10 17:53:23.922138 ip-10-0-141-180 kubenswrapper[2581]: E0910 17:53:23.922116    2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0e01bafb578bfa16f30a63f5ef6a4856daee5fb7a5a772c95c9dac27505996d7\": container with ID starting with 0e01bafb578bfa16f30a63f5ef6a4856daee5fb7a5a772c95c9dac27505996d7 not found: ID does not exist" containerID="0e01bafb578bfa16f30a63f5ef6a4856daee5fb7a5a772c95c9dac27505996d7"
Sep 10 17:53:23.922249 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:53:23.922138    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0e01bafb578bfa16f30a63f5ef6a4856daee5fb7a5a772c95c9dac27505996d7"} err="failed to get container status \"0e01bafb578bfa16f30a63f5ef6a4856daee5fb7a5a772c95c9dac27505996d7\": rpc error: code = NotFound desc = could not find container \"0e01bafb578bfa16f30a63f5ef6a4856daee5fb7a5a772c95c9dac27505996d7\": container with ID starting with 0e01bafb578bfa16f30a63f5ef6a4856daee5fb7a5a772c95c9dac27505996d7 not found: ID does not exist"
Sep 10 17:53:23.937350 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:53:23.937328    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-b-783a5e-openshift-default-5dd94f65d-bxhrc"]
Sep 10 17:53:23.967412 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:53:23.967390    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-inf-b-783a5e-openshift-default-5dd94f65d-bxhrc"]
Sep 10 17:53:24.186039 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:53:24.185966    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w3-18ecb2-5cc65989c4-2v267"]
Sep 10 17:53:24.186460 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:53:24.186439    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a628ea2c-8ec6-4d36-99b5-db4f7927de9d" containerName="istio-proxy"
Sep 10 17:53:24.186460 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:53:24.186458    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="a628ea2c-8ec6-4d36-99b5-db4f7927de9d" containerName="istio-proxy"
Sep 10 17:53:24.186625 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:53:24.186476    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="787f82f8-1081-42c7-b950-763bcbae5d2c" containerName="payload-pre-processing"
Sep 10 17:53:24.186625 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:53:24.186482    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="787f82f8-1081-42c7-b950-763bcbae5d2c" containerName="payload-pre-processing"
Sep 10 17:53:24.186625 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:53:24.186489    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3bfeae9c-6656-4fe4-8232-921f5e208c8d" containerName="revoke-keys"
Sep 10 17:53:24.186625 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:53:24.186495    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="3bfeae9c-6656-4fe4-8232-921f5e208c8d" containerName="revoke-keys"
Sep 10 17:53:24.186625 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:53:24.186508    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a628ea2c-8ec6-4d36-99b5-db4f7927de9d" containerName="istio-proxy"
Sep 10 17:53:24.186625 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:53:24.186513    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="a628ea2c-8ec6-4d36-99b5-db4f7927de9d" containerName="istio-proxy"
Sep 10 17:53:24.186625 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:53:24.186523    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4b171014-a09d-4221-baa7-2afc58afa8b9" containerName="payload-processing"
Sep 10 17:53:24.186625 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:53:24.186529    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="4b171014-a09d-4221-baa7-2afc58afa8b9" containerName="payload-processing"
Sep 10 17:53:24.186625 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:53:24.186588    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="4b171014-a09d-4221-baa7-2afc58afa8b9" containerName="payload-processing"
Sep 10 17:53:24.186625 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:53:24.186597    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="a628ea2c-8ec6-4d36-99b5-db4f7927de9d" containerName="istio-proxy"
Sep 10 17:53:24.186625 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:53:24.186604    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="787f82f8-1081-42c7-b950-763bcbae5d2c" containerName="payload-pre-processing"
Sep 10 17:53:24.186625 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:53:24.186610    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="a628ea2c-8ec6-4d36-99b5-db4f7927de9d" containerName="istio-proxy"
Sep 10 17:53:24.186625 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:53:24.186619    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="3bfeae9c-6656-4fe4-8232-921f5e208c8d" containerName="revoke-keys"
Sep 10 17:53:24.189423 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:53:24.189406    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-18ecb2-5cc65989c4-2v267"
Sep 10 17:53:24.194565 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:53:24.194545    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-shared-a-w3-18ecb2\""
Sep 10 17:53:24.194723 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:53:24.194594    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-shared-a-w3-18ecb2-dockercfg-hwfvr\""
Sep 10 17:53:24.195196 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:53:24.195173    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w3-18ecb2-5d975ddfbf-7hcpz"]
Sep 10 17:53:24.197800 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:53:24.197782    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-a-w3-18ecb2-5d975ddfbf-7hcpz"
Sep 10 17:53:24.208675 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:53:24.208651    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w3-18ecb2-5cc65989c4-2v267"]
Sep 10 17:53:24.217567 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:53:24.217545    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w3-18ecb2-5d975ddfbf-7hcpz"]
Sep 10 17:53:24.291429 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:53:24.291396    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/bb569e16-1fa4-4e68-abe0-6cb0983c966c-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-a-w3-18ecb2-5cc65989c4-2v267\" (UID: \"bb569e16-1fa4-4e68-abe0-6cb0983c966c\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-18ecb2-5cc65989c4-2v267"
Sep 10 17:53:24.291831 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:53:24.291511    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e50a465a-d483-413a-9dbe-d165d1b844b2-plugins-config-volume\") pod \"payload-processing-e2e-shared-a-w3-18ecb2-5d975ddfbf-7hcpz\" (UID: \"e50a465a-d483-413a-9dbe-d165d1b844b2\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w3-18ecb2-5d975ddfbf-7hcpz"
Sep 10 17:53:24.291831 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:53:24.291544    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-k9rhw\" (UniqueName: \"kubernetes.io/projected/e50a465a-d483-413a-9dbe-d165d1b844b2-kube-api-access-k9rhw\") pod \"payload-processing-e2e-shared-a-w3-18ecb2-5d975ddfbf-7hcpz\" (UID: \"e50a465a-d483-413a-9dbe-d165d1b844b2\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w3-18ecb2-5d975ddfbf-7hcpz"
Sep 10 17:53:24.291831 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:53:24.291569    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2jqcx\" (UniqueName: \"kubernetes.io/projected/bb569e16-1fa4-4e68-abe0-6cb0983c966c-kube-api-access-2jqcx\") pod \"payload-pre-processing-e2e-shared-a-w3-18ecb2-5cc65989c4-2v267\" (UID: \"bb569e16-1fa4-4e68-abe0-6cb0983c966c\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-18ecb2-5cc65989c4-2v267"
Sep 10 17:53:24.393161 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:53:24.393114    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e50a465a-d483-413a-9dbe-d165d1b844b2-plugins-config-volume\") pod \"payload-processing-e2e-shared-a-w3-18ecb2-5d975ddfbf-7hcpz\" (UID: \"e50a465a-d483-413a-9dbe-d165d1b844b2\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w3-18ecb2-5d975ddfbf-7hcpz"
Sep 10 17:53:24.393319 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:53:24.393170    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-k9rhw\" (UniqueName: \"kubernetes.io/projected/e50a465a-d483-413a-9dbe-d165d1b844b2-kube-api-access-k9rhw\") pod \"payload-processing-e2e-shared-a-w3-18ecb2-5d975ddfbf-7hcpz\" (UID: \"e50a465a-d483-413a-9dbe-d165d1b844b2\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w3-18ecb2-5d975ddfbf-7hcpz"
Sep 10 17:53:24.393319 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:53:24.393205    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2jqcx\" (UniqueName: \"kubernetes.io/projected/bb569e16-1fa4-4e68-abe0-6cb0983c966c-kube-api-access-2jqcx\") pod \"payload-pre-processing-e2e-shared-a-w3-18ecb2-5cc65989c4-2v267\" (UID: \"bb569e16-1fa4-4e68-abe0-6cb0983c966c\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-18ecb2-5cc65989c4-2v267"
Sep 10 17:53:24.393411 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:53:24.393323    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/bb569e16-1fa4-4e68-abe0-6cb0983c966c-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-a-w3-18ecb2-5cc65989c4-2v267\" (UID: \"bb569e16-1fa4-4e68-abe0-6cb0983c966c\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-18ecb2-5cc65989c4-2v267"
Sep 10 17:53:24.393977 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:53:24.393950    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e50a465a-d483-413a-9dbe-d165d1b844b2-plugins-config-volume\") pod \"payload-processing-e2e-shared-a-w3-18ecb2-5d975ddfbf-7hcpz\" (UID: \"e50a465a-d483-413a-9dbe-d165d1b844b2\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w3-18ecb2-5d975ddfbf-7hcpz"
Sep 10 17:53:24.394090 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:53:24.394070    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/bb569e16-1fa4-4e68-abe0-6cb0983c966c-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-a-w3-18ecb2-5cc65989c4-2v267\" (UID: \"bb569e16-1fa4-4e68-abe0-6cb0983c966c\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-18ecb2-5cc65989c4-2v267"
Sep 10 17:53:24.402604 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:53:24.402572    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2jqcx\" (UniqueName: \"kubernetes.io/projected/bb569e16-1fa4-4e68-abe0-6cb0983c966c-kube-api-access-2jqcx\") pod \"payload-pre-processing-e2e-shared-a-w3-18ecb2-5cc65989c4-2v267\" (UID: \"bb569e16-1fa4-4e68-abe0-6cb0983c966c\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-18ecb2-5cc65989c4-2v267"
Sep 10 17:53:24.402763 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:53:24.402672    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-k9rhw\" (UniqueName: \"kubernetes.io/projected/e50a465a-d483-413a-9dbe-d165d1b844b2-kube-api-access-k9rhw\") pod \"payload-processing-e2e-shared-a-w3-18ecb2-5d975ddfbf-7hcpz\" (UID: \"e50a465a-d483-413a-9dbe-d165d1b844b2\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w3-18ecb2-5d975ddfbf-7hcpz"
Sep 10 17:53:24.501453 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:53:24.501416    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-18ecb2-5cc65989c4-2v267"
Sep 10 17:53:24.510134 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:53:24.510112    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-a-w3-18ecb2-5d975ddfbf-7hcpz"
Sep 10 17:53:24.778130 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:53:24.778098    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w3-18ecb2-5cc65989c4-2v267"]
Sep 10 17:53:24.778964 ip-10-0-141-180 kubenswrapper[2581]: W0910 17:53:24.778934    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbb569e16_1fa4_4e68_abe0_6cb0983c966c.slice/crio-1f491da253d18756d55d0ae5bd3e1551762a00934219158c47a88d62b37ad289 WatchSource:0}: Error finding container 1f491da253d18756d55d0ae5bd3e1551762a00934219158c47a88d62b37ad289: Status 404 returned error can't find the container with id 1f491da253d18756d55d0ae5bd3e1551762a00934219158c47a88d62b37ad289
Sep 10 17:53:24.794454 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:53:24.794430    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w3-18ecb2-5d975ddfbf-7hcpz"]
Sep 10 17:53:24.796204 ip-10-0-141-180 kubenswrapper[2581]: W0910 17:53:24.796178    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode50a465a_d483_413a_9dbe_d165d1b844b2.slice/crio-e96d7bfcba5efd675c5e53874adc243ddd5c372dc61de6cd132c9374f16b5164 WatchSource:0}: Error finding container e96d7bfcba5efd675c5e53874adc243ddd5c372dc61de6cd132c9374f16b5164: Status 404 returned error can't find the container with id e96d7bfcba5efd675c5e53874adc243ddd5c372dc61de6cd132c9374f16b5164
Sep 10 17:53:24.864818 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:53:24.864780    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a628ea2c-8ec6-4d36-99b5-db4f7927de9d" path="/var/lib/kubelet/pods/a628ea2c-8ec6-4d36-99b5-db4f7927de9d/volumes"
Sep 10 17:53:24.911808 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:53:24.911772    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-18ecb2-5cc65989c4-2v267" event={"ID":"bb569e16-1fa4-4e68-abe0-6cb0983c966c","Type":"ContainerStarted","Data":"6c70e3e090bb1ec1008eca234e88d6ba28353ce3557e1c3591a32dbc5987b291"}
Sep 10 17:53:24.911992 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:53:24.911843    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-18ecb2-5cc65989c4-2v267"
Sep 10 17:53:24.911992 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:53:24.911872    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-18ecb2-5cc65989c4-2v267" event={"ID":"bb569e16-1fa4-4e68-abe0-6cb0983c966c","Type":"ContainerStarted","Data":"1f491da253d18756d55d0ae5bd3e1551762a00934219158c47a88d62b37ad289"}
Sep 10 17:53:24.913718 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:53:24.913684    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-a-w3-18ecb2-5d975ddfbf-7hcpz" event={"ID":"e50a465a-d483-413a-9dbe-d165d1b844b2","Type":"ContainerStarted","Data":"d76f81c2b86d3fda1967b5abd9297e55c6e7f14a5577881d62dbb9f9dd120441"}
Sep 10 17:53:24.913843 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:53:24.913738    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-shared-a-w3-18ecb2-5d975ddfbf-7hcpz"
Sep 10 17:53:24.913843 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:53:24.913765    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-a-w3-18ecb2-5d975ddfbf-7hcpz" event={"ID":"e50a465a-d483-413a-9dbe-d165d1b844b2","Type":"ContainerStarted","Data":"e96d7bfcba5efd675c5e53874adc243ddd5c372dc61de6cd132c9374f16b5164"}
Sep 10 17:53:24.938333 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:53:24.938276    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-18ecb2-5cc65989c4-2v267" podStartSLOduration=0.938260368 podStartE2EDuration="938.260368ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 17:53:24 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 17:53:24.935391635 +0000 UTC m=+1024.589465589" watchObservedRunningTime="2026-09-10 17:53:24.938260368 +0000 UTC m=+1024.592334271"
Sep 10 17:53:24.959714 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:53:24.959657    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-shared-a-w3-18ecb2-5d975ddfbf-7hcpz" podStartSLOduration=0.959626698 podStartE2EDuration="959.626698ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 17:53:24 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 17:53:24.955489161 +0000 UTC m=+1024.609563083" watchObservedRunningTime="2026-09-10 17:53:24.959626698 +0000 UTC m=+1024.613700622"
Sep 10 17:53:25.905429 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:53:25.905395    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-c523e8a0-openshift-default-6b4468db8-s7fjk"]
Sep 10 17:53:25.905899 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:53:25.905614    2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-mt-c523e8a0-openshift-default-6b4468db8-s7fjk" podUID="97d2c5cd-d503-4f66-97c6-6ca97cc789a8" containerName="istio-proxy" containerID="cri-o://efdcb46794b47abed151efea1717cb5f1a9251d77800558f7f93ccb655b64b7b" gracePeriod=30
Sep 10 17:53:29.913919 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:53:29.913882    2581 patch_prober.go:28] interesting pod/e2e-mt-c523e8a0-openshift-default-6b4468db8-s7fjk container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Sep 10 17:53:29.914280 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:53:29.913936    2581 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-mt-c523e8a0-openshift-default-6b4468db8-s7fjk" podUID="97d2c5cd-d503-4f66-97c6-6ca97cc789a8" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Sep 10 17:53:31.158305 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:53:31.158279    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-c523e8a0-openshift-default-6b4468db8-s7fjk"
Sep 10 17:53:31.265199 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:53:31.265169    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/97d2c5cd-d503-4f66-97c6-6ca97cc789a8-istio-envoy\") pod \"97d2c5cd-d503-4f66-97c6-6ca97cc789a8\" (UID: \"97d2c5cd-d503-4f66-97c6-6ca97cc789a8\") "
Sep 10 17:53:31.265360 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:53:31.265219    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/97d2c5cd-d503-4f66-97c6-6ca97cc789a8-workload-socket\") pod \"97d2c5cd-d503-4f66-97c6-6ca97cc789a8\" (UID: \"97d2c5cd-d503-4f66-97c6-6ca97cc789a8\") "
Sep 10 17:53:31.265360 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:53:31.265246    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/97d2c5cd-d503-4f66-97c6-6ca97cc789a8-istio-token\") pod \"97d2c5cd-d503-4f66-97c6-6ca97cc789a8\" (UID: \"97d2c5cd-d503-4f66-97c6-6ca97cc789a8\") "
Sep 10 17:53:31.265360 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:53:31.265274    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-rrn8f\" (UniqueName: \"kubernetes.io/projected/97d2c5cd-d503-4f66-97c6-6ca97cc789a8-kube-api-access-rrn8f\") pod \"97d2c5cd-d503-4f66-97c6-6ca97cc789a8\" (UID: \"97d2c5cd-d503-4f66-97c6-6ca97cc789a8\") "
Sep 10 17:53:31.265360 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:53:31.265296    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/97d2c5cd-d503-4f66-97c6-6ca97cc789a8-istio-podinfo\") pod \"97d2c5cd-d503-4f66-97c6-6ca97cc789a8\" (UID: \"97d2c5cd-d503-4f66-97c6-6ca97cc789a8\") "
Sep 10 17:53:31.265360 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:53:31.265328    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/97d2c5cd-d503-4f66-97c6-6ca97cc789a8-istiod-ca-cert\") pod \"97d2c5cd-d503-4f66-97c6-6ca97cc789a8\" (UID: \"97d2c5cd-d503-4f66-97c6-6ca97cc789a8\") "
Sep 10 17:53:31.265620 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:53:31.265367    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/97d2c5cd-d503-4f66-97c6-6ca97cc789a8-workload-certs\") pod \"97d2c5cd-d503-4f66-97c6-6ca97cc789a8\" (UID: \"97d2c5cd-d503-4f66-97c6-6ca97cc789a8\") "
Sep 10 17:53:31.265620 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:53:31.265393    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/97d2c5cd-d503-4f66-97c6-6ca97cc789a8-istio-data\") pod \"97d2c5cd-d503-4f66-97c6-6ca97cc789a8\" (UID: \"97d2c5cd-d503-4f66-97c6-6ca97cc789a8\") "
Sep 10 17:53:31.265620 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:53:31.265432    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/97d2c5cd-d503-4f66-97c6-6ca97cc789a8-credential-socket\") pod \"97d2c5cd-d503-4f66-97c6-6ca97cc789a8\" (UID: \"97d2c5cd-d503-4f66-97c6-6ca97cc789a8\") "
Sep 10 17:53:31.265620 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:53:31.265483    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/97d2c5cd-d503-4f66-97c6-6ca97cc789a8-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "97d2c5cd-d503-4f66-97c6-6ca97cc789a8" (UID: "97d2c5cd-d503-4f66-97c6-6ca97cc789a8"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 17:53:31.265846 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:53:31.265701    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/97d2c5cd-d503-4f66-97c6-6ca97cc789a8-istio-data" (OuterVolumeSpecName: "istio-data") pod "97d2c5cd-d503-4f66-97c6-6ca97cc789a8" (UID: "97d2c5cd-d503-4f66-97c6-6ca97cc789a8"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 17:53:31.265846 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:53:31.265750    2581 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/97d2c5cd-d503-4f66-97c6-6ca97cc789a8-workload-socket\") on node \"ip-10-0-141-180.ec2.internal\" DevicePath \"\""
Sep 10 17:53:31.265928 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:53:31.265899    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/97d2c5cd-d503-4f66-97c6-6ca97cc789a8-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "97d2c5cd-d503-4f66-97c6-6ca97cc789a8" (UID: "97d2c5cd-d503-4f66-97c6-6ca97cc789a8"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 17:53:31.265999 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:53:31.265957    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/97d2c5cd-d503-4f66-97c6-6ca97cc789a8-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "97d2c5cd-d503-4f66-97c6-6ca97cc789a8" (UID: "97d2c5cd-d503-4f66-97c6-6ca97cc789a8"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 17:53:31.266055 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:53:31.265994    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/97d2c5cd-d503-4f66-97c6-6ca97cc789a8-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "97d2c5cd-d503-4f66-97c6-6ca97cc789a8" (UID: "97d2c5cd-d503-4f66-97c6-6ca97cc789a8"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 17:53:31.267453 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:53:31.267420    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/97d2c5cd-d503-4f66-97c6-6ca97cc789a8-kube-api-access-rrn8f" (OuterVolumeSpecName: "kube-api-access-rrn8f") pod "97d2c5cd-d503-4f66-97c6-6ca97cc789a8" (UID: "97d2c5cd-d503-4f66-97c6-6ca97cc789a8"). InnerVolumeSpecName "kube-api-access-rrn8f". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 17:53:31.267549 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:53:31.267522    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/97d2c5cd-d503-4f66-97c6-6ca97cc789a8-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "97d2c5cd-d503-4f66-97c6-6ca97cc789a8" (UID: "97d2c5cd-d503-4f66-97c6-6ca97cc789a8"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 10 17:53:31.267808 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:53:31.267784    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/97d2c5cd-d503-4f66-97c6-6ca97cc789a8-istio-token" (OuterVolumeSpecName: "istio-token") pod "97d2c5cd-d503-4f66-97c6-6ca97cc789a8" (UID: "97d2c5cd-d503-4f66-97c6-6ca97cc789a8"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 17:53:31.267808 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:53:31.267783    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/97d2c5cd-d503-4f66-97c6-6ca97cc789a8-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "97d2c5cd-d503-4f66-97c6-6ca97cc789a8" (UID: "97d2c5cd-d503-4f66-97c6-6ca97cc789a8"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 17:53:31.367223 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:53:31.367195    2581 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/97d2c5cd-d503-4f66-97c6-6ca97cc789a8-workload-certs\") on node \"ip-10-0-141-180.ec2.internal\" DevicePath \"\""
Sep 10 17:53:31.367223 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:53:31.367218    2581 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/97d2c5cd-d503-4f66-97c6-6ca97cc789a8-istio-data\") on node \"ip-10-0-141-180.ec2.internal\" DevicePath \"\""
Sep 10 17:53:31.367223 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:53:31.367227    2581 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/97d2c5cd-d503-4f66-97c6-6ca97cc789a8-credential-socket\") on node \"ip-10-0-141-180.ec2.internal\" DevicePath \"\""
Sep 10 17:53:31.367421 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:53:31.367237    2581 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/97d2c5cd-d503-4f66-97c6-6ca97cc789a8-istio-envoy\") on node \"ip-10-0-141-180.ec2.internal\" DevicePath \"\""
Sep 10 17:53:31.367421 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:53:31.367246    2581 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/97d2c5cd-d503-4f66-97c6-6ca97cc789a8-istio-token\") on node \"ip-10-0-141-180.ec2.internal\" DevicePath \"\""
Sep 10 17:53:31.367421 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:53:31.367254    2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rrn8f\" (UniqueName: \"kubernetes.io/projected/97d2c5cd-d503-4f66-97c6-6ca97cc789a8-kube-api-access-rrn8f\") on node \"ip-10-0-141-180.ec2.internal\" DevicePath \"\""
Sep 10 17:53:31.367421 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:53:31.367262    2581 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/97d2c5cd-d503-4f66-97c6-6ca97cc789a8-istio-podinfo\") on node \"ip-10-0-141-180.ec2.internal\" DevicePath \"\""
Sep 10 17:53:31.367421 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:53:31.367270    2581 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/97d2c5cd-d503-4f66-97c6-6ca97cc789a8-istiod-ca-cert\") on node \"ip-10-0-141-180.ec2.internal\" DevicePath \"\""
Sep 10 17:53:31.961473 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:53:31.961439    2581 generic.go:356] "Generic (PLEG): container finished" podID="97d2c5cd-d503-4f66-97c6-6ca97cc789a8" containerID="efdcb46794b47abed151efea1717cb5f1a9251d77800558f7f93ccb655b64b7b" exitCode=0
Sep 10 17:53:31.961670 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:53:31.961481    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-c523e8a0-openshift-default-6b4468db8-s7fjk"
Sep 10 17:53:31.961670 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:53:31.961525    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-c523e8a0-openshift-default-6b4468db8-s7fjk" event={"ID":"97d2c5cd-d503-4f66-97c6-6ca97cc789a8","Type":"ContainerDied","Data":"efdcb46794b47abed151efea1717cb5f1a9251d77800558f7f93ccb655b64b7b"}
Sep 10 17:53:31.961670 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:53:31.961574    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-c523e8a0-openshift-default-6b4468db8-s7fjk" event={"ID":"97d2c5cd-d503-4f66-97c6-6ca97cc789a8","Type":"ContainerDied","Data":"43139501329c411e62e677d4749200fa54b6bf2a161ec270b0fffdffd2cd08cc"}
Sep 10 17:53:31.961670 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:53:31.961599    2581 scope.go:117] "RemoveContainer" containerID="efdcb46794b47abed151efea1717cb5f1a9251d77800558f7f93ccb655b64b7b"
Sep 10 17:53:31.978237 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:53:31.978218    2581 scope.go:117] "RemoveContainer" containerID="72a695053135e51e74da7ae6371ab567ae4906007f4e328633147acc462482e3"
Sep 10 17:53:31.987590 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:53:31.987566    2581 scope.go:117] "RemoveContainer" containerID="efdcb46794b47abed151efea1717cb5f1a9251d77800558f7f93ccb655b64b7b"
Sep 10 17:53:31.987936 ip-10-0-141-180 kubenswrapper[2581]: E0910 17:53:31.987907    2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"efdcb46794b47abed151efea1717cb5f1a9251d77800558f7f93ccb655b64b7b\": container with ID starting with efdcb46794b47abed151efea1717cb5f1a9251d77800558f7f93ccb655b64b7b not found: ID does not exist" containerID="efdcb46794b47abed151efea1717cb5f1a9251d77800558f7f93ccb655b64b7b"
Sep 10 17:53:31.988017 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:53:31.987947    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"efdcb46794b47abed151efea1717cb5f1a9251d77800558f7f93ccb655b64b7b"} err="failed to get container status \"efdcb46794b47abed151efea1717cb5f1a9251d77800558f7f93ccb655b64b7b\": rpc error: code = NotFound desc = could not find container \"efdcb46794b47abed151efea1717cb5f1a9251d77800558f7f93ccb655b64b7b\": container with ID starting with efdcb46794b47abed151efea1717cb5f1a9251d77800558f7f93ccb655b64b7b not found: ID does not exist"
Sep 10 17:53:31.988017 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:53:31.987969    2581 scope.go:117] "RemoveContainer" containerID="72a695053135e51e74da7ae6371ab567ae4906007f4e328633147acc462482e3"
Sep 10 17:53:31.988746 ip-10-0-141-180 kubenswrapper[2581]: E0910 17:53:31.988694    2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"72a695053135e51e74da7ae6371ab567ae4906007f4e328633147acc462482e3\": container with ID starting with 72a695053135e51e74da7ae6371ab567ae4906007f4e328633147acc462482e3 not found: ID does not exist" containerID="72a695053135e51e74da7ae6371ab567ae4906007f4e328633147acc462482e3"
Sep 10 17:53:31.988847 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:53:31.988751    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"72a695053135e51e74da7ae6371ab567ae4906007f4e328633147acc462482e3"} err="failed to get container status \"72a695053135e51e74da7ae6371ab567ae4906007f4e328633147acc462482e3\": rpc error: code = NotFound desc = could not find container \"72a695053135e51e74da7ae6371ab567ae4906007f4e328633147acc462482e3\": container with ID starting with 72a695053135e51e74da7ae6371ab567ae4906007f4e328633147acc462482e3 not found: ID does not exist"
Sep 10 17:53:31.991288 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:53:31.991267    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-c523e8a0-openshift-default-6b4468db8-s7fjk"]
Sep 10 17:53:31.995633 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:53:31.995612    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-mt-c523e8a0-openshift-default-6b4468db8-s7fjk"]
Sep 10 17:53:32.863770 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:53:32.863735    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="97d2c5cd-d503-4f66-97c6-6ca97cc789a8" path="/var/lib/kubelet/pods/97d2c5cd-d503-4f66-97c6-6ca97cc789a8/volumes"
Sep 10 17:53:35.925546 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:53:35.925515    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-18ecb2-5cc65989c4-2v267"
Sep 10 17:53:35.926023 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:53:35.925581    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-shared-a-w3-18ecb2-5d975ddfbf-7hcpz"
Sep 10 17:53:41.093095 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:53:41.093057    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-shared-a-w2-9391d2-openshift-default-57c888f898-dwd2h"]
Sep 10 17:53:41.093710 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:53:41.093575    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="97d2c5cd-d503-4f66-97c6-6ca97cc789a8" containerName="istio-proxy"
Sep 10 17:53:41.093710 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:53:41.093593    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="97d2c5cd-d503-4f66-97c6-6ca97cc789a8" containerName="istio-proxy"
Sep 10 17:53:41.093710 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:53:41.093604    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="97d2c5cd-d503-4f66-97c6-6ca97cc789a8" containerName="istio-proxy"
Sep 10 17:53:41.093710 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:53:41.093609    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="97d2c5cd-d503-4f66-97c6-6ca97cc789a8" containerName="istio-proxy"
Sep 10 17:53:41.093710 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:53:41.093708    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="97d2c5cd-d503-4f66-97c6-6ca97cc789a8" containerName="istio-proxy"
Sep 10 17:53:41.093971 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:53:41.093723    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="97d2c5cd-d503-4f66-97c6-6ca97cc789a8" containerName="istio-proxy"
Sep 10 17:53:41.098686 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:53:41.098626    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-w2-9391d2-openshift-default-57c888f898-dwd2h"
Sep 10 17:53:41.101259 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:53:41.101238    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-shared-a-w2-9391d2-openshift-default-dockercfg-fhckl\""
Sep 10 17:53:41.123567 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:53:41.123542    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-a-w2-9391d2-openshift-default-57c888f898-dwd2h"]
Sep 10 17:53:41.156123 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:53:41.156086    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d406cdfe-2228-4c17-ba8f-01a86277b666-credential-socket\") pod \"e2e-shared-a-w2-9391d2-openshift-default-57c888f898-dwd2h\" (UID: \"d406cdfe-2228-4c17-ba8f-01a86277b666\") " pod="openshift-ingress/e2e-shared-a-w2-9391d2-openshift-default-57c888f898-dwd2h"
Sep 10 17:53:41.156268 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:53:41.156132    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d406cdfe-2228-4c17-ba8f-01a86277b666-istio-token\") pod \"e2e-shared-a-w2-9391d2-openshift-default-57c888f898-dwd2h\" (UID: \"d406cdfe-2228-4c17-ba8f-01a86277b666\") " pod="openshift-ingress/e2e-shared-a-w2-9391d2-openshift-default-57c888f898-dwd2h"
Sep 10 17:53:41.156268 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:53:41.156163    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d406cdfe-2228-4c17-ba8f-01a86277b666-workload-socket\") pod \"e2e-shared-a-w2-9391d2-openshift-default-57c888f898-dwd2h\" (UID: \"d406cdfe-2228-4c17-ba8f-01a86277b666\") " pod="openshift-ingress/e2e-shared-a-w2-9391d2-openshift-default-57c888f898-dwd2h"
Sep 10 17:53:41.156268 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:53:41.156185    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d406cdfe-2228-4c17-ba8f-01a86277b666-istio-data\") pod \"e2e-shared-a-w2-9391d2-openshift-default-57c888f898-dwd2h\" (UID: \"d406cdfe-2228-4c17-ba8f-01a86277b666\") " pod="openshift-ingress/e2e-shared-a-w2-9391d2-openshift-default-57c888f898-dwd2h"
Sep 10 17:53:41.156268 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:53:41.156251    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d406cdfe-2228-4c17-ba8f-01a86277b666-workload-certs\") pod \"e2e-shared-a-w2-9391d2-openshift-default-57c888f898-dwd2h\" (UID: \"d406cdfe-2228-4c17-ba8f-01a86277b666\") " pod="openshift-ingress/e2e-shared-a-w2-9391d2-openshift-default-57c888f898-dwd2h"
Sep 10 17:53:41.156496 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:53:41.156271    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d406cdfe-2228-4c17-ba8f-01a86277b666-istio-envoy\") pod \"e2e-shared-a-w2-9391d2-openshift-default-57c888f898-dwd2h\" (UID: \"d406cdfe-2228-4c17-ba8f-01a86277b666\") " pod="openshift-ingress/e2e-shared-a-w2-9391d2-openshift-default-57c888f898-dwd2h"
Sep 10 17:53:41.156496 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:53:41.156381    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8qsxq\" (UniqueName: \"kubernetes.io/projected/d406cdfe-2228-4c17-ba8f-01a86277b666-kube-api-access-8qsxq\") pod \"e2e-shared-a-w2-9391d2-openshift-default-57c888f898-dwd2h\" (UID: \"d406cdfe-2228-4c17-ba8f-01a86277b666\") " pod="openshift-ingress/e2e-shared-a-w2-9391d2-openshift-default-57c888f898-dwd2h"
Sep 10 17:53:41.156496 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:53:41.156443    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d406cdfe-2228-4c17-ba8f-01a86277b666-istio-podinfo\") pod \"e2e-shared-a-w2-9391d2-openshift-default-57c888f898-dwd2h\" (UID: \"d406cdfe-2228-4c17-ba8f-01a86277b666\") " pod="openshift-ingress/e2e-shared-a-w2-9391d2-openshift-default-57c888f898-dwd2h"
Sep 10 17:53:41.156496 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:53:41.156470    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d406cdfe-2228-4c17-ba8f-01a86277b666-istiod-ca-cert\") pod \"e2e-shared-a-w2-9391d2-openshift-default-57c888f898-dwd2h\" (UID: \"d406cdfe-2228-4c17-ba8f-01a86277b666\") " pod="openshift-ingress/e2e-shared-a-w2-9391d2-openshift-default-57c888f898-dwd2h"
Sep 10 17:53:41.257302 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:53:41.257267    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d406cdfe-2228-4c17-ba8f-01a86277b666-workload-certs\") pod \"e2e-shared-a-w2-9391d2-openshift-default-57c888f898-dwd2h\" (UID: \"d406cdfe-2228-4c17-ba8f-01a86277b666\") " pod="openshift-ingress/e2e-shared-a-w2-9391d2-openshift-default-57c888f898-dwd2h"
Sep 10 17:53:41.257302 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:53:41.257305    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d406cdfe-2228-4c17-ba8f-01a86277b666-istio-envoy\") pod \"e2e-shared-a-w2-9391d2-openshift-default-57c888f898-dwd2h\" (UID: \"d406cdfe-2228-4c17-ba8f-01a86277b666\") " pod="openshift-ingress/e2e-shared-a-w2-9391d2-openshift-default-57c888f898-dwd2h"
Sep 10 17:53:41.257539 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:53:41.257347    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8qsxq\" (UniqueName: \"kubernetes.io/projected/d406cdfe-2228-4c17-ba8f-01a86277b666-kube-api-access-8qsxq\") pod \"e2e-shared-a-w2-9391d2-openshift-default-57c888f898-dwd2h\" (UID: \"d406cdfe-2228-4c17-ba8f-01a86277b666\") " pod="openshift-ingress/e2e-shared-a-w2-9391d2-openshift-default-57c888f898-dwd2h"
Sep 10 17:53:41.257539 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:53:41.257374    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d406cdfe-2228-4c17-ba8f-01a86277b666-istio-podinfo\") pod \"e2e-shared-a-w2-9391d2-openshift-default-57c888f898-dwd2h\" (UID: \"d406cdfe-2228-4c17-ba8f-01a86277b666\") " pod="openshift-ingress/e2e-shared-a-w2-9391d2-openshift-default-57c888f898-dwd2h"
Sep 10 17:53:41.257539 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:53:41.257390    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d406cdfe-2228-4c17-ba8f-01a86277b666-istiod-ca-cert\") pod \"e2e-shared-a-w2-9391d2-openshift-default-57c888f898-dwd2h\" (UID: \"d406cdfe-2228-4c17-ba8f-01a86277b666\") " pod="openshift-ingress/e2e-shared-a-w2-9391d2-openshift-default-57c888f898-dwd2h"
Sep 10 17:53:41.257539 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:53:41.257419    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d406cdfe-2228-4c17-ba8f-01a86277b666-credential-socket\") pod \"e2e-shared-a-w2-9391d2-openshift-default-57c888f898-dwd2h\" (UID: \"d406cdfe-2228-4c17-ba8f-01a86277b666\") " pod="openshift-ingress/e2e-shared-a-w2-9391d2-openshift-default-57c888f898-dwd2h"
Sep 10 17:53:41.257539 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:53:41.257447    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d406cdfe-2228-4c17-ba8f-01a86277b666-istio-token\") pod \"e2e-shared-a-w2-9391d2-openshift-default-57c888f898-dwd2h\" (UID: \"d406cdfe-2228-4c17-ba8f-01a86277b666\") " pod="openshift-ingress/e2e-shared-a-w2-9391d2-openshift-default-57c888f898-dwd2h"
Sep 10 17:53:41.257539 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:53:41.257487    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d406cdfe-2228-4c17-ba8f-01a86277b666-workload-socket\") pod \"e2e-shared-a-w2-9391d2-openshift-default-57c888f898-dwd2h\" (UID: \"d406cdfe-2228-4c17-ba8f-01a86277b666\") " pod="openshift-ingress/e2e-shared-a-w2-9391d2-openshift-default-57c888f898-dwd2h"
Sep 10 17:53:41.257539 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:53:41.257523    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d406cdfe-2228-4c17-ba8f-01a86277b666-istio-data\") pod \"e2e-shared-a-w2-9391d2-openshift-default-57c888f898-dwd2h\" (UID: \"d406cdfe-2228-4c17-ba8f-01a86277b666\") " pod="openshift-ingress/e2e-shared-a-w2-9391d2-openshift-default-57c888f898-dwd2h"
Sep 10 17:53:41.257973 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:53:41.257723    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d406cdfe-2228-4c17-ba8f-01a86277b666-workload-certs\") pod \"e2e-shared-a-w2-9391d2-openshift-default-57c888f898-dwd2h\" (UID: \"d406cdfe-2228-4c17-ba8f-01a86277b666\") " pod="openshift-ingress/e2e-shared-a-w2-9391d2-openshift-default-57c888f898-dwd2h"
Sep 10 17:53:41.257973 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:53:41.257802    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d406cdfe-2228-4c17-ba8f-01a86277b666-credential-socket\") pod \"e2e-shared-a-w2-9391d2-openshift-default-57c888f898-dwd2h\" (UID: \"d406cdfe-2228-4c17-ba8f-01a86277b666\") " pod="openshift-ingress/e2e-shared-a-w2-9391d2-openshift-default-57c888f898-dwd2h"
Sep 10 17:53:41.257973 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:53:41.257953    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d406cdfe-2228-4c17-ba8f-01a86277b666-istio-data\") pod \"e2e-shared-a-w2-9391d2-openshift-default-57c888f898-dwd2h\" (UID: \"d406cdfe-2228-4c17-ba8f-01a86277b666\") " pod="openshift-ingress/e2e-shared-a-w2-9391d2-openshift-default-57c888f898-dwd2h"
Sep 10 17:53:41.258103 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:53:41.257974    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d406cdfe-2228-4c17-ba8f-01a86277b666-workload-socket\") pod \"e2e-shared-a-w2-9391d2-openshift-default-57c888f898-dwd2h\" (UID: \"d406cdfe-2228-4c17-ba8f-01a86277b666\") " pod="openshift-ingress/e2e-shared-a-w2-9391d2-openshift-default-57c888f898-dwd2h"
Sep 10 17:53:41.258149 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:53:41.258133    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d406cdfe-2228-4c17-ba8f-01a86277b666-istiod-ca-cert\") pod \"e2e-shared-a-w2-9391d2-openshift-default-57c888f898-dwd2h\" (UID: \"d406cdfe-2228-4c17-ba8f-01a86277b666\") " pod="openshift-ingress/e2e-shared-a-w2-9391d2-openshift-default-57c888f898-dwd2h"
Sep 10 17:53:41.259825 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:53:41.259801    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d406cdfe-2228-4c17-ba8f-01a86277b666-istio-envoy\") pod \"e2e-shared-a-w2-9391d2-openshift-default-57c888f898-dwd2h\" (UID: \"d406cdfe-2228-4c17-ba8f-01a86277b666\") " pod="openshift-ingress/e2e-shared-a-w2-9391d2-openshift-default-57c888f898-dwd2h"
Sep 10 17:53:41.259959 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:53:41.259941    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d406cdfe-2228-4c17-ba8f-01a86277b666-istio-podinfo\") pod \"e2e-shared-a-w2-9391d2-openshift-default-57c888f898-dwd2h\" (UID: \"d406cdfe-2228-4c17-ba8f-01a86277b666\") " pod="openshift-ingress/e2e-shared-a-w2-9391d2-openshift-default-57c888f898-dwd2h"
Sep 10 17:53:41.271951 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:53:41.271924    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8qsxq\" (UniqueName: \"kubernetes.io/projected/d406cdfe-2228-4c17-ba8f-01a86277b666-kube-api-access-8qsxq\") pod \"e2e-shared-a-w2-9391d2-openshift-default-57c888f898-dwd2h\" (UID: \"d406cdfe-2228-4c17-ba8f-01a86277b666\") " pod="openshift-ingress/e2e-shared-a-w2-9391d2-openshift-default-57c888f898-dwd2h"
Sep 10 17:53:41.273860 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:53:41.273833    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d406cdfe-2228-4c17-ba8f-01a86277b666-istio-token\") pod \"e2e-shared-a-w2-9391d2-openshift-default-57c888f898-dwd2h\" (UID: \"d406cdfe-2228-4c17-ba8f-01a86277b666\") " pod="openshift-ingress/e2e-shared-a-w2-9391d2-openshift-default-57c888f898-dwd2h"
Sep 10 17:53:41.411294 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:53:41.411200    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-w2-9391d2-openshift-default-57c888f898-dwd2h"
Sep 10 17:53:41.697741 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:53:41.697712    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-a-w2-9391d2-openshift-default-57c888f898-dwd2h"]
Sep 10 17:53:41.698886 ip-10-0-141-180 kubenswrapper[2581]: W0910 17:53:41.698859    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd406cdfe_2228_4c17_ba8f_01a86277b666.slice/crio-5661824dc11979779607aea7d88155a991f4b2ce82d55e19b6f06216caa73540 WatchSource:0}: Error finding container 5661824dc11979779607aea7d88155a991f4b2ce82d55e19b6f06216caa73540: Status 404 returned error can't find the container with id 5661824dc11979779607aea7d88155a991f4b2ce82d55e19b6f06216caa73540
Sep 10 17:53:41.701259 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:53:41.701218    2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 10 17:53:41.701324 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:53:41.701308    2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 10 17:53:41.701372 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:53:41.701355    2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 10 17:53:42.029606 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:53:42.029569    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w2-9391d2-openshift-default-57c888f898-dwd2h" event={"ID":"d406cdfe-2228-4c17-ba8f-01a86277b666","Type":"ContainerStarted","Data":"691ba5c9e471b1f22f077c8147ee5334fa8b934e8b85b9b8a619ad60f1c1f2c2"}
Sep 10 17:53:42.029781 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:53:42.029623    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w2-9391d2-openshift-default-57c888f898-dwd2h" event={"ID":"d406cdfe-2228-4c17-ba8f-01a86277b666","Type":"ContainerStarted","Data":"5661824dc11979779607aea7d88155a991f4b2ce82d55e19b6f06216caa73540"}
Sep 10 17:53:42.057201 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:53:42.057139    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-shared-a-w2-9391d2-openshift-default-57c888f898-dwd2h" podStartSLOduration=1.057118321 podStartE2EDuration="1.057118321s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 17:53:41 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 17:53:42.052809074 +0000 UTC m=+1041.706883041" watchObservedRunningTime="2026-09-10 17:53:42.057118321 +0000 UTC m=+1041.711192225"
Sep 10 17:53:42.412094 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:53:42.412007    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-a-w2-9391d2-openshift-default-57c888f898-dwd2h"
Sep 10 17:53:42.417029 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:53:42.417003    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-a-w2-9391d2-openshift-default-57c888f898-dwd2h"
Sep 10 17:53:43.036536 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:53:43.036503    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-a-w2-9391d2-openshift-default-57c888f898-dwd2h"
Sep 10 17:53:43.037462 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:53:43.037440    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-a-w2-9391d2-openshift-default-57c888f898-dwd2h"
Sep 10 17:53:46.417682 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:53:46.417621    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w2-9391d2-66bd85c6c8-qwwg4"]
Sep 10 17:53:46.421477 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:53:46.421459    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-9391d2-66bd85c6c8-qwwg4"
Sep 10 17:53:46.429136 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:53:46.429065    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-shared-a-w2-9391d2-dockercfg-xlq6m\""
Sep 10 17:53:46.436351 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:53:46.436324    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-shared-a-w2-9391d2\""
Sep 10 17:53:46.439763 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:53:46.439738    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w2-9391d2-66bd85c6c8-qwwg4"]
Sep 10 17:53:46.471132 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:53:46.471101    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w2-9391d2-569f5888bf-qqks8"]
Sep 10 17:53:46.475408 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:53:46.475388    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-a-w2-9391d2-569f5888bf-qqks8"
Sep 10 17:53:46.492423 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:53:46.492399    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w2-9391d2-569f5888bf-qqks8"]
Sep 10 17:53:46.511191 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:53:46.511158    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b259c38c-d3fd-4f18-924c-fbbaf62d3624-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-a-w2-9391d2-66bd85c6c8-qwwg4\" (UID: \"b259c38c-d3fd-4f18-924c-fbbaf62d3624\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-9391d2-66bd85c6c8-qwwg4"
Sep 10 17:53:46.511369 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:53:46.511199    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-k2jdx\" (UniqueName: \"kubernetes.io/projected/b259c38c-d3fd-4f18-924c-fbbaf62d3624-kube-api-access-k2jdx\") pod \"payload-pre-processing-e2e-shared-a-w2-9391d2-66bd85c6c8-qwwg4\" (UID: \"b259c38c-d3fd-4f18-924c-fbbaf62d3624\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-9391d2-66bd85c6c8-qwwg4"
Sep 10 17:53:46.612618 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:53:46.612581    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/58758048-0e64-4870-87cf-b95eb28c72e7-plugins-config-volume\") pod \"payload-processing-e2e-shared-a-w2-9391d2-569f5888bf-qqks8\" (UID: \"58758048-0e64-4870-87cf-b95eb28c72e7\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w2-9391d2-569f5888bf-qqks8"
Sep 10 17:53:46.612827 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:53:46.612797    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b259c38c-d3fd-4f18-924c-fbbaf62d3624-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-a-w2-9391d2-66bd85c6c8-qwwg4\" (UID: \"b259c38c-d3fd-4f18-924c-fbbaf62d3624\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-9391d2-66bd85c6c8-qwwg4"
Sep 10 17:53:46.612876 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:53:46.612848    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-g7zsw\" (UniqueName: \"kubernetes.io/projected/58758048-0e64-4870-87cf-b95eb28c72e7-kube-api-access-g7zsw\") pod \"payload-processing-e2e-shared-a-w2-9391d2-569f5888bf-qqks8\" (UID: \"58758048-0e64-4870-87cf-b95eb28c72e7\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w2-9391d2-569f5888bf-qqks8"
Sep 10 17:53:46.612919 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:53:46.612884    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-k2jdx\" (UniqueName: \"kubernetes.io/projected/b259c38c-d3fd-4f18-924c-fbbaf62d3624-kube-api-access-k2jdx\") pod \"payload-pre-processing-e2e-shared-a-w2-9391d2-66bd85c6c8-qwwg4\" (UID: \"b259c38c-d3fd-4f18-924c-fbbaf62d3624\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-9391d2-66bd85c6c8-qwwg4"
Sep 10 17:53:46.613534 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:53:46.613515    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b259c38c-d3fd-4f18-924c-fbbaf62d3624-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-a-w2-9391d2-66bd85c6c8-qwwg4\" (UID: \"b259c38c-d3fd-4f18-924c-fbbaf62d3624\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-9391d2-66bd85c6c8-qwwg4"
Sep 10 17:53:46.621167 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:53:46.621144    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-k2jdx\" (UniqueName: \"kubernetes.io/projected/b259c38c-d3fd-4f18-924c-fbbaf62d3624-kube-api-access-k2jdx\") pod \"payload-pre-processing-e2e-shared-a-w2-9391d2-66bd85c6c8-qwwg4\" (UID: \"b259c38c-d3fd-4f18-924c-fbbaf62d3624\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-9391d2-66bd85c6c8-qwwg4"
Sep 10 17:53:46.713613 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:53:46.713580    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-g7zsw\" (UniqueName: \"kubernetes.io/projected/58758048-0e64-4870-87cf-b95eb28c72e7-kube-api-access-g7zsw\") pod \"payload-processing-e2e-shared-a-w2-9391d2-569f5888bf-qqks8\" (UID: \"58758048-0e64-4870-87cf-b95eb28c72e7\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w2-9391d2-569f5888bf-qqks8"
Sep 10 17:53:46.713816 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:53:46.713673    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/58758048-0e64-4870-87cf-b95eb28c72e7-plugins-config-volume\") pod \"payload-processing-e2e-shared-a-w2-9391d2-569f5888bf-qqks8\" (UID: \"58758048-0e64-4870-87cf-b95eb28c72e7\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w2-9391d2-569f5888bf-qqks8"
Sep 10 17:53:46.714341 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:53:46.714320    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/58758048-0e64-4870-87cf-b95eb28c72e7-plugins-config-volume\") pod \"payload-processing-e2e-shared-a-w2-9391d2-569f5888bf-qqks8\" (UID: \"58758048-0e64-4870-87cf-b95eb28c72e7\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w2-9391d2-569f5888bf-qqks8"
Sep 10 17:53:46.722359 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:53:46.722333    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-g7zsw\" (UniqueName: \"kubernetes.io/projected/58758048-0e64-4870-87cf-b95eb28c72e7-kube-api-access-g7zsw\") pod \"payload-processing-e2e-shared-a-w2-9391d2-569f5888bf-qqks8\" (UID: \"58758048-0e64-4870-87cf-b95eb28c72e7\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w2-9391d2-569f5888bf-qqks8"
Sep 10 17:53:46.733178 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:53:46.733147    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-9391d2-66bd85c6c8-qwwg4"
Sep 10 17:53:46.785595 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:53:46.785344    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-a-w2-9391d2-569f5888bf-qqks8"
Sep 10 17:53:47.012479 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:53:47.012388    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w2-9391d2-66bd85c6c8-qwwg4"]
Sep 10 17:53:47.014774 ip-10-0-141-180 kubenswrapper[2581]: W0910 17:53:47.014744    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb259c38c_d3fd_4f18_924c_fbbaf62d3624.slice/crio-abc1051c6b9c0629d25eb4abbe1dde23beb8ff8326fba18003c69145a2e14c4a WatchSource:0}: Error finding container abc1051c6b9c0629d25eb4abbe1dde23beb8ff8326fba18003c69145a2e14c4a: Status 404 returned error can't find the container with id abc1051c6b9c0629d25eb4abbe1dde23beb8ff8326fba18003c69145a2e14c4a
Sep 10 17:53:47.063324 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:53:47.063297    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-9391d2-66bd85c6c8-qwwg4" event={"ID":"b259c38c-d3fd-4f18-924c-fbbaf62d3624","Type":"ContainerStarted","Data":"abc1051c6b9c0629d25eb4abbe1dde23beb8ff8326fba18003c69145a2e14c4a"}
Sep 10 17:53:47.076398 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:53:47.076375    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w2-9391d2-569f5888bf-qqks8"]
Sep 10 17:53:47.077569 ip-10-0-141-180 kubenswrapper[2581]: W0910 17:53:47.077547    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod58758048_0e64_4870_87cf_b95eb28c72e7.slice/crio-4b3dd3595ebf9ca0410ffcb7c37c641451dc9a98ca5a9aa3986b5639c47977e1 WatchSource:0}: Error finding container 4b3dd3595ebf9ca0410ffcb7c37c641451dc9a98ca5a9aa3986b5639c47977e1: Status 404 returned error can't find the container with id 4b3dd3595ebf9ca0410ffcb7c37c641451dc9a98ca5a9aa3986b5639c47977e1
Sep 10 17:53:47.556219 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:53:47.556173    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-shared-b-w3-57474d-openshift-default-5c978cd6bd-d8phk"]
Sep 10 17:53:47.560612 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:53:47.560554    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-w3-57474d-openshift-default-5c978cd6bd-d8phk"
Sep 10 17:53:47.563187 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:53:47.563156    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-shared-b-w3-57474d-openshift-default-dockercfg-s7mqz\""
Sep 10 17:53:47.573069 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:53:47.573042    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-b-w3-57474d-openshift-default-5c978cd6bd-d8phk"]
Sep 10 17:53:47.723276 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:53:47.723249    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0a789c73-9891-4c17-bfa0-948af3f241eb-istiod-ca-cert\") pod \"e2e-shared-b-w3-57474d-openshift-default-5c978cd6bd-d8phk\" (UID: \"0a789c73-9891-4c17-bfa0-948af3f241eb\") " pod="openshift-ingress/e2e-shared-b-w3-57474d-openshift-default-5c978cd6bd-d8phk"
Sep 10 17:53:47.723460 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:53:47.723282    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0a789c73-9891-4c17-bfa0-948af3f241eb-credential-socket\") pod \"e2e-shared-b-w3-57474d-openshift-default-5c978cd6bd-d8phk\" (UID: \"0a789c73-9891-4c17-bfa0-948af3f241eb\") " pod="openshift-ingress/e2e-shared-b-w3-57474d-openshift-default-5c978cd6bd-d8phk"
Sep 10 17:53:47.723460 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:53:47.723303    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0a789c73-9891-4c17-bfa0-948af3f241eb-istio-token\") pod \"e2e-shared-b-w3-57474d-openshift-default-5c978cd6bd-d8phk\" (UID: \"0a789c73-9891-4c17-bfa0-948af3f241eb\") " pod="openshift-ingress/e2e-shared-b-w3-57474d-openshift-default-5c978cd6bd-d8phk"
Sep 10 17:53:47.723460 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:53:47.723395    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0a789c73-9891-4c17-bfa0-948af3f241eb-workload-socket\") pod \"e2e-shared-b-w3-57474d-openshift-default-5c978cd6bd-d8phk\" (UID: \"0a789c73-9891-4c17-bfa0-948af3f241eb\") " pod="openshift-ingress/e2e-shared-b-w3-57474d-openshift-default-5c978cd6bd-d8phk"
Sep 10 17:53:47.723667 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:53:47.723458    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0a789c73-9891-4c17-bfa0-948af3f241eb-istio-envoy\") pod \"e2e-shared-b-w3-57474d-openshift-default-5c978cd6bd-d8phk\" (UID: \"0a789c73-9891-4c17-bfa0-948af3f241eb\") " pod="openshift-ingress/e2e-shared-b-w3-57474d-openshift-default-5c978cd6bd-d8phk"
Sep 10 17:53:47.723667 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:53:47.723491    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0a789c73-9891-4c17-bfa0-948af3f241eb-istio-data\") pod \"e2e-shared-b-w3-57474d-openshift-default-5c978cd6bd-d8phk\" (UID: \"0a789c73-9891-4c17-bfa0-948af3f241eb\") " pod="openshift-ingress/e2e-shared-b-w3-57474d-openshift-default-5c978cd6bd-d8phk"
Sep 10 17:53:47.723667 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:53:47.723524    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0a789c73-9891-4c17-bfa0-948af3f241eb-workload-certs\") pod \"e2e-shared-b-w3-57474d-openshift-default-5c978cd6bd-d8phk\" (UID: \"0a789c73-9891-4c17-bfa0-948af3f241eb\") " pod="openshift-ingress/e2e-shared-b-w3-57474d-openshift-default-5c978cd6bd-d8phk"
Sep 10 17:53:47.723667 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:53:47.723570    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-c5s49\" (UniqueName: \"kubernetes.io/projected/0a789c73-9891-4c17-bfa0-948af3f241eb-kube-api-access-c5s49\") pod \"e2e-shared-b-w3-57474d-openshift-default-5c978cd6bd-d8phk\" (UID: \"0a789c73-9891-4c17-bfa0-948af3f241eb\") " pod="openshift-ingress/e2e-shared-b-w3-57474d-openshift-default-5c978cd6bd-d8phk"
Sep 10 17:53:47.723667 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:53:47.723627    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0a789c73-9891-4c17-bfa0-948af3f241eb-istio-podinfo\") pod \"e2e-shared-b-w3-57474d-openshift-default-5c978cd6bd-d8phk\" (UID: \"0a789c73-9891-4c17-bfa0-948af3f241eb\") " pod="openshift-ingress/e2e-shared-b-w3-57474d-openshift-default-5c978cd6bd-d8phk"
Sep 10 17:53:47.825050 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:53:47.824964    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0a789c73-9891-4c17-bfa0-948af3f241eb-workload-socket\") pod \"e2e-shared-b-w3-57474d-openshift-default-5c978cd6bd-d8phk\" (UID: \"0a789c73-9891-4c17-bfa0-948af3f241eb\") " pod="openshift-ingress/e2e-shared-b-w3-57474d-openshift-default-5c978cd6bd-d8phk"
Sep 10 17:53:47.825050 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:53:47.825001    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0a789c73-9891-4c17-bfa0-948af3f241eb-istio-envoy\") pod \"e2e-shared-b-w3-57474d-openshift-default-5c978cd6bd-d8phk\" (UID: \"0a789c73-9891-4c17-bfa0-948af3f241eb\") " pod="openshift-ingress/e2e-shared-b-w3-57474d-openshift-default-5c978cd6bd-d8phk"
Sep 10 17:53:47.825050 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:53:47.825016    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0a789c73-9891-4c17-bfa0-948af3f241eb-istio-data\") pod \"e2e-shared-b-w3-57474d-openshift-default-5c978cd6bd-d8phk\" (UID: \"0a789c73-9891-4c17-bfa0-948af3f241eb\") " pod="openshift-ingress/e2e-shared-b-w3-57474d-openshift-default-5c978cd6bd-d8phk"
Sep 10 17:53:47.825050 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:53:47.825037    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0a789c73-9891-4c17-bfa0-948af3f241eb-workload-certs\") pod \"e2e-shared-b-w3-57474d-openshift-default-5c978cd6bd-d8phk\" (UID: \"0a789c73-9891-4c17-bfa0-948af3f241eb\") " pod="openshift-ingress/e2e-shared-b-w3-57474d-openshift-default-5c978cd6bd-d8phk"
Sep 10 17:53:47.825388 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:53:47.825070    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-c5s49\" (UniqueName: \"kubernetes.io/projected/0a789c73-9891-4c17-bfa0-948af3f241eb-kube-api-access-c5s49\") pod \"e2e-shared-b-w3-57474d-openshift-default-5c978cd6bd-d8phk\" (UID: \"0a789c73-9891-4c17-bfa0-948af3f241eb\") " pod="openshift-ingress/e2e-shared-b-w3-57474d-openshift-default-5c978cd6bd-d8phk"
Sep 10 17:53:47.825388 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:53:47.825111    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0a789c73-9891-4c17-bfa0-948af3f241eb-istio-podinfo\") pod \"e2e-shared-b-w3-57474d-openshift-default-5c978cd6bd-d8phk\" (UID: \"0a789c73-9891-4c17-bfa0-948af3f241eb\") " pod="openshift-ingress/e2e-shared-b-w3-57474d-openshift-default-5c978cd6bd-d8phk"
Sep 10 17:53:47.825388 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:53:47.825176    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0a789c73-9891-4c17-bfa0-948af3f241eb-istiod-ca-cert\") pod \"e2e-shared-b-w3-57474d-openshift-default-5c978cd6bd-d8phk\" (UID: \"0a789c73-9891-4c17-bfa0-948af3f241eb\") " pod="openshift-ingress/e2e-shared-b-w3-57474d-openshift-default-5c978cd6bd-d8phk"
Sep 10 17:53:47.825388 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:53:47.825204    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0a789c73-9891-4c17-bfa0-948af3f241eb-credential-socket\") pod \"e2e-shared-b-w3-57474d-openshift-default-5c978cd6bd-d8phk\" (UID: \"0a789c73-9891-4c17-bfa0-948af3f241eb\") " pod="openshift-ingress/e2e-shared-b-w3-57474d-openshift-default-5c978cd6bd-d8phk"
Sep 10 17:53:47.825388 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:53:47.825229    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0a789c73-9891-4c17-bfa0-948af3f241eb-istio-token\") pod \"e2e-shared-b-w3-57474d-openshift-default-5c978cd6bd-d8phk\" (UID: \"0a789c73-9891-4c17-bfa0-948af3f241eb\") " pod="openshift-ingress/e2e-shared-b-w3-57474d-openshift-default-5c978cd6bd-d8phk"
Sep 10 17:53:47.825675 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:53:47.825427    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0a789c73-9891-4c17-bfa0-948af3f241eb-workload-socket\") pod \"e2e-shared-b-w3-57474d-openshift-default-5c978cd6bd-d8phk\" (UID: \"0a789c73-9891-4c17-bfa0-948af3f241eb\") " pod="openshift-ingress/e2e-shared-b-w3-57474d-openshift-default-5c978cd6bd-d8phk"
Sep 10 17:53:47.825675 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:53:47.825470    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0a789c73-9891-4c17-bfa0-948af3f241eb-workload-certs\") pod \"e2e-shared-b-w3-57474d-openshift-default-5c978cd6bd-d8phk\" (UID: \"0a789c73-9891-4c17-bfa0-948af3f241eb\") " pod="openshift-ingress/e2e-shared-b-w3-57474d-openshift-default-5c978cd6bd-d8phk"
Sep 10 17:53:47.825675 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:53:47.825528    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0a789c73-9891-4c17-bfa0-948af3f241eb-istio-data\") pod \"e2e-shared-b-w3-57474d-openshift-default-5c978cd6bd-d8phk\" (UID: \"0a789c73-9891-4c17-bfa0-948af3f241eb\") " pod="openshift-ingress/e2e-shared-b-w3-57474d-openshift-default-5c978cd6bd-d8phk"
Sep 10 17:53:47.825840 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:53:47.825787    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0a789c73-9891-4c17-bfa0-948af3f241eb-credential-socket\") pod \"e2e-shared-b-w3-57474d-openshift-default-5c978cd6bd-d8phk\" (UID: \"0a789c73-9891-4c17-bfa0-948af3f241eb\") " pod="openshift-ingress/e2e-shared-b-w3-57474d-openshift-default-5c978cd6bd-d8phk"
Sep 10 17:53:47.826060 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:53:47.826038    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0a789c73-9891-4c17-bfa0-948af3f241eb-istiod-ca-cert\") pod \"e2e-shared-b-w3-57474d-openshift-default-5c978cd6bd-d8phk\" (UID: \"0a789c73-9891-4c17-bfa0-948af3f241eb\") " pod="openshift-ingress/e2e-shared-b-w3-57474d-openshift-default-5c978cd6bd-d8phk"
Sep 10 17:53:47.827439 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:53:47.827421    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0a789c73-9891-4c17-bfa0-948af3f241eb-istio-envoy\") pod \"e2e-shared-b-w3-57474d-openshift-default-5c978cd6bd-d8phk\" (UID: \"0a789c73-9891-4c17-bfa0-948af3f241eb\") " pod="openshift-ingress/e2e-shared-b-w3-57474d-openshift-default-5c978cd6bd-d8phk"
Sep 10 17:53:47.827786 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:53:47.827767    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0a789c73-9891-4c17-bfa0-948af3f241eb-istio-podinfo\") pod \"e2e-shared-b-w3-57474d-openshift-default-5c978cd6bd-d8phk\" (UID: \"0a789c73-9891-4c17-bfa0-948af3f241eb\") " pod="openshift-ingress/e2e-shared-b-w3-57474d-openshift-default-5c978cd6bd-d8phk"
Sep 10 17:53:47.837464 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:53:47.837445    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0a789c73-9891-4c17-bfa0-948af3f241eb-istio-token\") pod \"e2e-shared-b-w3-57474d-openshift-default-5c978cd6bd-d8phk\" (UID: \"0a789c73-9891-4c17-bfa0-948af3f241eb\") " pod="openshift-ingress/e2e-shared-b-w3-57474d-openshift-default-5c978cd6bd-d8phk"
Sep 10 17:53:47.838300 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:53:47.838277    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-c5s49\" (UniqueName: \"kubernetes.io/projected/0a789c73-9891-4c17-bfa0-948af3f241eb-kube-api-access-c5s49\") pod \"e2e-shared-b-w3-57474d-openshift-default-5c978cd6bd-d8phk\" (UID: \"0a789c73-9891-4c17-bfa0-948af3f241eb\") " pod="openshift-ingress/e2e-shared-b-w3-57474d-openshift-default-5c978cd6bd-d8phk"
Sep 10 17:53:47.874522 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:53:47.874486    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-w3-57474d-openshift-default-5c978cd6bd-d8phk"
Sep 10 17:53:48.072449 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:53:48.072412    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-9391d2-66bd85c6c8-qwwg4" event={"ID":"b259c38c-d3fd-4f18-924c-fbbaf62d3624","Type":"ContainerStarted","Data":"6ab60620d47614239e36836204e6b9538acf9a2a2e7572ba2661c7dac6effd6c"}
Sep 10 17:53:48.072621 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:53:48.072497    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-9391d2-66bd85c6c8-qwwg4"
Sep 10 17:53:48.074295 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:53:48.074267    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-a-w2-9391d2-569f5888bf-qqks8" event={"ID":"58758048-0e64-4870-87cf-b95eb28c72e7","Type":"ContainerStarted","Data":"bdfe2ffa86b505261d17f79563bda3d27e1266b081d7c3b574d8a49e4fde3863"}
Sep 10 17:53:48.074419 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:53:48.074315    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-a-w2-9391d2-569f5888bf-qqks8" event={"ID":"58758048-0e64-4870-87cf-b95eb28c72e7","Type":"ContainerStarted","Data":"4b3dd3595ebf9ca0410ffcb7c37c641451dc9a98ca5a9aa3986b5639c47977e1"}
Sep 10 17:53:48.074419 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:53:48.074353    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-shared-a-w2-9391d2-569f5888bf-qqks8"
Sep 10 17:53:48.100043 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:53:48.099879    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-9391d2-66bd85c6c8-qwwg4" podStartSLOduration=2.099861651 podStartE2EDuration="2.099861651s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 17:53:46 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 17:53:48.099558319 +0000 UTC m=+1047.753632248" watchObservedRunningTime="2026-09-10 17:53:48.099861651 +0000 UTC m=+1047.753935554"
Sep 10 17:53:48.130725 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:53:48.130684    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-shared-a-w2-9391d2-569f5888bf-qqks8" podStartSLOduration=2.13067111 podStartE2EDuration="2.13067111s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 17:53:46 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 17:53:48.128845726 +0000 UTC m=+1047.782919652" watchObservedRunningTime="2026-09-10 17:53:48.13067111 +0000 UTC m=+1047.784745014"
Sep 10 17:53:48.165568 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:53:48.165537    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-b-w3-57474d-openshift-default-5c978cd6bd-d8phk"]
Sep 10 17:53:48.169578 ip-10-0-141-180 kubenswrapper[2581]: W0910 17:53:48.169557    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0a789c73_9891_4c17_bfa0_948af3f241eb.slice/crio-30eb3bf505b452d6e140d75ed2e7021d89d1f0bff84fba422d6ba399ecb7d966 WatchSource:0}: Error finding container 30eb3bf505b452d6e140d75ed2e7021d89d1f0bff84fba422d6ba399ecb7d966: Status 404 returned error can't find the container with id 30eb3bf505b452d6e140d75ed2e7021d89d1f0bff84fba422d6ba399ecb7d966
Sep 10 17:53:48.171952 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:53:48.171918    2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 10 17:53:48.172040 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:53:48.171979    2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 10 17:53:48.172040 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:53:48.172009    2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 10 17:53:49.082530 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:53:49.082494    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w3-57474d-openshift-default-5c978cd6bd-d8phk" event={"ID":"0a789c73-9891-4c17-bfa0-948af3f241eb","Type":"ContainerStarted","Data":"4bacf92809286f23b0dfc3f8c197832a99d190927f6517765596b6517f0b5232"}
Sep 10 17:53:49.083011 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:53:49.082546    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w3-57474d-openshift-default-5c978cd6bd-d8phk" event={"ID":"0a789c73-9891-4c17-bfa0-948af3f241eb","Type":"ContainerStarted","Data":"30eb3bf505b452d6e140d75ed2e7021d89d1f0bff84fba422d6ba399ecb7d966"}
Sep 10 17:53:49.107706 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:53:49.107659    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-shared-b-w3-57474d-openshift-default-5c978cd6bd-d8phk" podStartSLOduration=2.107633427 podStartE2EDuration="2.107633427s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 17:53:47 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 17:53:49.105293169 +0000 UTC m=+1048.759367092" watchObservedRunningTime="2026-09-10 17:53:49.107633427 +0000 UTC m=+1048.761707334"
Sep 10 17:53:49.874818 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:53:49.874783    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-b-w3-57474d-openshift-default-5c978cd6bd-d8phk"
Sep 10 17:53:49.880250 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:53:49.880223    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-b-w3-57474d-openshift-default-5c978cd6bd-d8phk"
Sep 10 17:53:50.089374 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:53:50.089342    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-b-w3-57474d-openshift-default-5c978cd6bd-d8phk"
Sep 10 17:53:50.090405 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:53:50.090386    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-b-w3-57474d-openshift-default-5c978cd6bd-d8phk"
Sep 10 17:53:59.077842 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:53:59.077797    2581 patch_prober.go:28] interesting pod/e2e-shared-a-w2-9391d2-openshift-default-57c888f898-dwd2h container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.68:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Sep 10 17:53:59.078450 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:53:59.078413    2581 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-shared-a-w2-9391d2-openshift-default-57c888f898-dwd2h" podUID="d406cdfe-2228-4c17-ba8f-01a86277b666" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.68:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Sep 10 17:53:59.084738 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:53:59.084699    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-shared-a-w2-9391d2-569f5888bf-qqks8"
Sep 10 17:53:59.085583 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:53:59.085559    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-9391d2-66bd85c6c8-qwwg4"
Sep 10 17:54:02.169292 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:54:02.169196    2581 generic.go:356] "Generic (PLEG): container finished" podID="d406cdfe-2228-4c17-ba8f-01a86277b666" containerID="691ba5c9e471b1f22f077c8147ee5334fa8b934e8b85b9b8a619ad60f1c1f2c2" exitCode=0
Sep 10 17:54:02.169292 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:54:02.169277    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w2-9391d2-openshift-default-57c888f898-dwd2h" event={"ID":"d406cdfe-2228-4c17-ba8f-01a86277b666","Type":"ContainerDied","Data":"691ba5c9e471b1f22f077c8147ee5334fa8b934e8b85b9b8a619ad60f1c1f2c2"}
Sep 10 17:54:02.169760 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:54:02.169722    2581 scope.go:117] "RemoveContainer" containerID="691ba5c9e471b1f22f077c8147ee5334fa8b934e8b85b9b8a619ad60f1c1f2c2"
Sep 10 17:54:02.171215 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:54:02.170887    2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 10 17:54:02.171215 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:54:02.170976    2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 10 17:54:02.171215 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:54:02.171022    2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 10 17:54:03.185329 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:54:03.185294    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w2-9391d2-openshift-default-57c888f898-dwd2h" event={"ID":"d406cdfe-2228-4c17-ba8f-01a86277b666","Type":"ContainerStarted","Data":"dbebc011dddf30407b12734364f786768c1eca83a4331bfa7362eda1f5f87a6a"}
Sep 10 17:54:03.412150 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:54:03.412115    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-a-w2-9391d2-openshift-default-57c888f898-dwd2h"
Sep 10 17:54:03.412552 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:54:03.412437    2581 patch_prober.go:28] interesting pod/e2e-shared-a-w2-9391d2-openshift-default-57c888f898-dwd2h container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.68:15021/healthz/ready\": dial tcp 10.133.0.68:15021: connect: connection refused" start-of-body=
Sep 10 17:54:03.412775 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:54:03.412749    2581 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w2-9391d2-openshift-default-57c888f898-dwd2h" podUID="d406cdfe-2228-4c17-ba8f-01a86277b666" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.68:15021/healthz/ready\": dial tcp 10.133.0.68:15021: connect: connection refused"
Sep 10 17:54:04.412513 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:54:04.412468    2581 patch_prober.go:28] interesting pod/e2e-shared-a-w2-9391d2-openshift-default-57c888f898-dwd2h container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.68:15021/healthz/ready\": dial tcp 10.133.0.68:15021: connect: connection refused" start-of-body=
Sep 10 17:54:04.413003 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:54:04.412522    2581 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w2-9391d2-openshift-default-57c888f898-dwd2h" podUID="d406cdfe-2228-4c17-ba8f-01a86277b666" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.68:15021/healthz/ready\": dial tcp 10.133.0.68:15021: connect: connection refused"
Sep 10 17:54:05.412109 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:54:05.412070    2581 patch_prober.go:28] interesting pod/e2e-shared-a-w2-9391d2-openshift-default-57c888f898-dwd2h container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.68:15021/healthz/ready\": dial tcp 10.133.0.68:15021: connect: connection refused" start-of-body=
Sep 10 17:54:05.412282 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:54:05.412139    2581 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w2-9391d2-openshift-default-57c888f898-dwd2h" podUID="d406cdfe-2228-4c17-ba8f-01a86277b666" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.68:15021/healthz/ready\": dial tcp 10.133.0.68:15021: connect: connection refused"
Sep 10 17:54:06.132804 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:54:06.132764    2581 patch_prober.go:28] interesting pod/e2e-shared-b-w3-57474d-openshift-default-5c978cd6bd-d8phk container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.71:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Sep 10 17:54:06.133214 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:54:06.132828    2581 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-shared-b-w3-57474d-openshift-default-5c978cd6bd-d8phk" podUID="0a789c73-9891-4c17-bfa0-948af3f241eb" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.71:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Sep 10 17:54:06.411922 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:54:06.411837    2581 patch_prober.go:28] interesting pod/e2e-shared-a-w2-9391d2-openshift-default-57c888f898-dwd2h container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.68:15021/healthz/ready\": dial tcp 10.133.0.68:15021: connect: connection refused" start-of-body=
Sep 10 17:54:06.411922 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:54:06.411908    2581 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w2-9391d2-openshift-default-57c888f898-dwd2h" podUID="d406cdfe-2228-4c17-ba8f-01a86277b666" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.68:15021/healthz/ready\": dial tcp 10.133.0.68:15021: connect: connection refused"
Sep 10 17:54:07.412382 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:54:07.412339    2581 patch_prober.go:28] interesting pod/e2e-shared-a-w2-9391d2-openshift-default-57c888f898-dwd2h container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.68:15021/healthz/ready\": dial tcp 10.133.0.68:15021: connect: connection refused" start-of-body=
Sep 10 17:54:07.412794 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:54:07.412403    2581 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w2-9391d2-openshift-default-57c888f898-dwd2h" podUID="d406cdfe-2228-4c17-ba8f-01a86277b666" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.68:15021/healthz/ready\": dial tcp 10.133.0.68:15021: connect: connection refused"
Sep 10 17:54:08.229560 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:54:08.229491    2581 generic.go:356] "Generic (PLEG): container finished" podID="0a789c73-9891-4c17-bfa0-948af3f241eb" containerID="4bacf92809286f23b0dfc3f8c197832a99d190927f6517765596b6517f0b5232" exitCode=0
Sep 10 17:54:08.229802 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:54:08.229561    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w3-57474d-openshift-default-5c978cd6bd-d8phk" event={"ID":"0a789c73-9891-4c17-bfa0-948af3f241eb","Type":"ContainerDied","Data":"4bacf92809286f23b0dfc3f8c197832a99d190927f6517765596b6517f0b5232"}
Sep 10 17:54:08.230043 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:54:08.230026    2581 scope.go:117] "RemoveContainer" containerID="4bacf92809286f23b0dfc3f8c197832a99d190927f6517765596b6517f0b5232"
Sep 10 17:54:08.231103 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:54:08.231077    2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 10 17:54:08.231210 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:54:08.231143    2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 10 17:54:08.231258 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:54:08.231210    2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 10 17:54:08.412833 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:54:08.412793    2581 patch_prober.go:28] interesting pod/e2e-shared-a-w2-9391d2-openshift-default-57c888f898-dwd2h container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.68:15021/healthz/ready\": dial tcp 10.133.0.68:15021: connect: connection refused" start-of-body=
Sep 10 17:54:08.413279 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:54:08.412883    2581 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w2-9391d2-openshift-default-57c888f898-dwd2h" podUID="d406cdfe-2228-4c17-ba8f-01a86277b666" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.68:15021/healthz/ready\": dial tcp 10.133.0.68:15021: connect: connection refused"
Sep 10 17:54:09.238214 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:54:09.238179    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w3-57474d-openshift-default-5c978cd6bd-d8phk" event={"ID":"0a789c73-9891-4c17-bfa0-948af3f241eb","Type":"ContainerStarted","Data":"4947a863b2ef1bf41ef24ab6809a60274ea684fc5176358ab3e9e5c29dab3c6a"}
Sep 10 17:54:09.415629 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:54:09.415600    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-a-w2-9391d2-openshift-default-57c888f898-dwd2h"
Sep 10 17:54:09.416016 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:54:09.416003    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-a-w2-9391d2-openshift-default-57c888f898-dwd2h"
Sep 10 17:54:09.416602 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:54:09.416582    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-a-w2-9391d2-openshift-default-57c888f898-dwd2h"
Sep 10 17:54:09.878125 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:54:09.877839    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-b-w3-57474d-openshift-default-5c978cd6bd-d8phk"
Sep 10 17:54:09.878125 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:54:09.877963    2581 patch_prober.go:28] interesting pod/e2e-shared-b-w3-57474d-openshift-default-5c978cd6bd-d8phk container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.71:15021/healthz/ready\": dial tcp 10.133.0.71:15021: connect: connection refused" start-of-body=
Sep 10 17:54:09.878125 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:54:09.878011    2581 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w3-57474d-openshift-default-5c978cd6bd-d8phk" podUID="0a789c73-9891-4c17-bfa0-948af3f241eb" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.71:15021/healthz/ready\": dial tcp 10.133.0.71:15021: connect: connection refused"
Sep 10 17:54:09.907129 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:54:09.907093    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-shared-b-w2-bdf47c-openshift-default-6d77688b7b-slw4g"]
Sep 10 17:54:09.913572 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:54:09.913544    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-w2-bdf47c-openshift-default-6d77688b7b-slw4g"
Sep 10 17:54:09.920033 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:54:09.919993    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-shared-b-w2-bdf47c-openshift-default-dockercfg-8bpt2\""
Sep 10 17:54:09.927095 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:54:09.927072    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-b-w2-bdf47c-openshift-default-6d77688b7b-slw4g"]
Sep 10 17:54:10.009248 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:54:10.009206    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-h5hlx\" (UniqueName: \"kubernetes.io/projected/08571c40-a532-4fa2-a18b-c7e583a29609-kube-api-access-h5hlx\") pod \"e2e-shared-b-w2-bdf47c-openshift-default-6d77688b7b-slw4g\" (UID: \"08571c40-a532-4fa2-a18b-c7e583a29609\") " pod="openshift-ingress/e2e-shared-b-w2-bdf47c-openshift-default-6d77688b7b-slw4g"
Sep 10 17:54:10.009519 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:54:10.009493    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/08571c40-a532-4fa2-a18b-c7e583a29609-workload-certs\") pod \"e2e-shared-b-w2-bdf47c-openshift-default-6d77688b7b-slw4g\" (UID: \"08571c40-a532-4fa2-a18b-c7e583a29609\") " pod="openshift-ingress/e2e-shared-b-w2-bdf47c-openshift-default-6d77688b7b-slw4g"
Sep 10 17:54:10.009666 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:54:10.009622    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/08571c40-a532-4fa2-a18b-c7e583a29609-istio-token\") pod \"e2e-shared-b-w2-bdf47c-openshift-default-6d77688b7b-slw4g\" (UID: \"08571c40-a532-4fa2-a18b-c7e583a29609\") " pod="openshift-ingress/e2e-shared-b-w2-bdf47c-openshift-default-6d77688b7b-slw4g"
Sep 10 17:54:10.009884 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:54:10.009865    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/08571c40-a532-4fa2-a18b-c7e583a29609-credential-socket\") pod \"e2e-shared-b-w2-bdf47c-openshift-default-6d77688b7b-slw4g\" (UID: \"08571c40-a532-4fa2-a18b-c7e583a29609\") " pod="openshift-ingress/e2e-shared-b-w2-bdf47c-openshift-default-6d77688b7b-slw4g"
Sep 10 17:54:10.010022 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:54:10.010006    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/08571c40-a532-4fa2-a18b-c7e583a29609-workload-socket\") pod \"e2e-shared-b-w2-bdf47c-openshift-default-6d77688b7b-slw4g\" (UID: \"08571c40-a532-4fa2-a18b-c7e583a29609\") " pod="openshift-ingress/e2e-shared-b-w2-bdf47c-openshift-default-6d77688b7b-slw4g"
Sep 10 17:54:10.010156 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:54:10.010140    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/08571c40-a532-4fa2-a18b-c7e583a29609-istio-data\") pod \"e2e-shared-b-w2-bdf47c-openshift-default-6d77688b7b-slw4g\" (UID: \"08571c40-a532-4fa2-a18b-c7e583a29609\") " pod="openshift-ingress/e2e-shared-b-w2-bdf47c-openshift-default-6d77688b7b-slw4g"
Sep 10 17:54:10.010281 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:54:10.010267    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/08571c40-a532-4fa2-a18b-c7e583a29609-istio-envoy\") pod \"e2e-shared-b-w2-bdf47c-openshift-default-6d77688b7b-slw4g\" (UID: \"08571c40-a532-4fa2-a18b-c7e583a29609\") " pod="openshift-ingress/e2e-shared-b-w2-bdf47c-openshift-default-6d77688b7b-slw4g"
Sep 10 17:54:10.010384 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:54:10.010371    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/08571c40-a532-4fa2-a18b-c7e583a29609-istiod-ca-cert\") pod \"e2e-shared-b-w2-bdf47c-openshift-default-6d77688b7b-slw4g\" (UID: \"08571c40-a532-4fa2-a18b-c7e583a29609\") " pod="openshift-ingress/e2e-shared-b-w2-bdf47c-openshift-default-6d77688b7b-slw4g"
Sep 10 17:54:10.010498 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:54:10.010484    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/08571c40-a532-4fa2-a18b-c7e583a29609-istio-podinfo\") pod \"e2e-shared-b-w2-bdf47c-openshift-default-6d77688b7b-slw4g\" (UID: \"08571c40-a532-4fa2-a18b-c7e583a29609\") " pod="openshift-ingress/e2e-shared-b-w2-bdf47c-openshift-default-6d77688b7b-slw4g"
Sep 10 17:54:10.111180 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:54:10.111145    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-h5hlx\" (UniqueName: \"kubernetes.io/projected/08571c40-a532-4fa2-a18b-c7e583a29609-kube-api-access-h5hlx\") pod \"e2e-shared-b-w2-bdf47c-openshift-default-6d77688b7b-slw4g\" (UID: \"08571c40-a532-4fa2-a18b-c7e583a29609\") " pod="openshift-ingress/e2e-shared-b-w2-bdf47c-openshift-default-6d77688b7b-slw4g"
Sep 10 17:54:10.111377 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:54:10.111195    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/08571c40-a532-4fa2-a18b-c7e583a29609-workload-certs\") pod \"e2e-shared-b-w2-bdf47c-openshift-default-6d77688b7b-slw4g\" (UID: \"08571c40-a532-4fa2-a18b-c7e583a29609\") " pod="openshift-ingress/e2e-shared-b-w2-bdf47c-openshift-default-6d77688b7b-slw4g"
Sep 10 17:54:10.111377 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:54:10.111254    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/08571c40-a532-4fa2-a18b-c7e583a29609-istio-token\") pod \"e2e-shared-b-w2-bdf47c-openshift-default-6d77688b7b-slw4g\" (UID: \"08571c40-a532-4fa2-a18b-c7e583a29609\") " pod="openshift-ingress/e2e-shared-b-w2-bdf47c-openshift-default-6d77688b7b-slw4g"
Sep 10 17:54:10.111377 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:54:10.111343    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/08571c40-a532-4fa2-a18b-c7e583a29609-credential-socket\") pod \"e2e-shared-b-w2-bdf47c-openshift-default-6d77688b7b-slw4g\" (UID: \"08571c40-a532-4fa2-a18b-c7e583a29609\") " pod="openshift-ingress/e2e-shared-b-w2-bdf47c-openshift-default-6d77688b7b-slw4g"
Sep 10 17:54:10.111377 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:54:10.111367    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/08571c40-a532-4fa2-a18b-c7e583a29609-workload-socket\") pod \"e2e-shared-b-w2-bdf47c-openshift-default-6d77688b7b-slw4g\" (UID: \"08571c40-a532-4fa2-a18b-c7e583a29609\") " pod="openshift-ingress/e2e-shared-b-w2-bdf47c-openshift-default-6d77688b7b-slw4g"
Sep 10 17:54:10.111626 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:54:10.111413    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/08571c40-a532-4fa2-a18b-c7e583a29609-istio-data\") pod \"e2e-shared-b-w2-bdf47c-openshift-default-6d77688b7b-slw4g\" (UID: \"08571c40-a532-4fa2-a18b-c7e583a29609\") " pod="openshift-ingress/e2e-shared-b-w2-bdf47c-openshift-default-6d77688b7b-slw4g"
Sep 10 17:54:10.111626 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:54:10.111444    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/08571c40-a532-4fa2-a18b-c7e583a29609-istio-envoy\") pod \"e2e-shared-b-w2-bdf47c-openshift-default-6d77688b7b-slw4g\" (UID: \"08571c40-a532-4fa2-a18b-c7e583a29609\") " pod="openshift-ingress/e2e-shared-b-w2-bdf47c-openshift-default-6d77688b7b-slw4g"
Sep 10 17:54:10.111626 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:54:10.111464    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/08571c40-a532-4fa2-a18b-c7e583a29609-istiod-ca-cert\") pod \"e2e-shared-b-w2-bdf47c-openshift-default-6d77688b7b-slw4g\" (UID: \"08571c40-a532-4fa2-a18b-c7e583a29609\") " pod="openshift-ingress/e2e-shared-b-w2-bdf47c-openshift-default-6d77688b7b-slw4g"
Sep 10 17:54:10.111626 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:54:10.111493    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/08571c40-a532-4fa2-a18b-c7e583a29609-istio-podinfo\") pod \"e2e-shared-b-w2-bdf47c-openshift-default-6d77688b7b-slw4g\" (UID: \"08571c40-a532-4fa2-a18b-c7e583a29609\") " pod="openshift-ingress/e2e-shared-b-w2-bdf47c-openshift-default-6d77688b7b-slw4g"
Sep 10 17:54:10.111626 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:54:10.111602    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/08571c40-a532-4fa2-a18b-c7e583a29609-workload-certs\") pod \"e2e-shared-b-w2-bdf47c-openshift-default-6d77688b7b-slw4g\" (UID: \"08571c40-a532-4fa2-a18b-c7e583a29609\") " pod="openshift-ingress/e2e-shared-b-w2-bdf47c-openshift-default-6d77688b7b-slw4g"
Sep 10 17:54:10.111851 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:54:10.111792    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/08571c40-a532-4fa2-a18b-c7e583a29609-workload-socket\") pod \"e2e-shared-b-w2-bdf47c-openshift-default-6d77688b7b-slw4g\" (UID: \"08571c40-a532-4fa2-a18b-c7e583a29609\") " pod="openshift-ingress/e2e-shared-b-w2-bdf47c-openshift-default-6d77688b7b-slw4g"
Sep 10 17:54:10.111851 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:54:10.111830    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/08571c40-a532-4fa2-a18b-c7e583a29609-istio-data\") pod \"e2e-shared-b-w2-bdf47c-openshift-default-6d77688b7b-slw4g\" (UID: \"08571c40-a532-4fa2-a18b-c7e583a29609\") " pod="openshift-ingress/e2e-shared-b-w2-bdf47c-openshift-default-6d77688b7b-slw4g"
Sep 10 17:54:10.112087 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:54:10.112065    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/08571c40-a532-4fa2-a18b-c7e583a29609-credential-socket\") pod \"e2e-shared-b-w2-bdf47c-openshift-default-6d77688b7b-slw4g\" (UID: \"08571c40-a532-4fa2-a18b-c7e583a29609\") " pod="openshift-ingress/e2e-shared-b-w2-bdf47c-openshift-default-6d77688b7b-slw4g"
Sep 10 17:54:10.112152 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:54:10.112127    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/08571c40-a532-4fa2-a18b-c7e583a29609-istiod-ca-cert\") pod \"e2e-shared-b-w2-bdf47c-openshift-default-6d77688b7b-slw4g\" (UID: \"08571c40-a532-4fa2-a18b-c7e583a29609\") " pod="openshift-ingress/e2e-shared-b-w2-bdf47c-openshift-default-6d77688b7b-slw4g"
Sep 10 17:54:10.113978 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:54:10.113960    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/08571c40-a532-4fa2-a18b-c7e583a29609-istio-envoy\") pod \"e2e-shared-b-w2-bdf47c-openshift-default-6d77688b7b-slw4g\" (UID: \"08571c40-a532-4fa2-a18b-c7e583a29609\") " pod="openshift-ingress/e2e-shared-b-w2-bdf47c-openshift-default-6d77688b7b-slw4g"
Sep 10 17:54:10.114124 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:54:10.114105    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/08571c40-a532-4fa2-a18b-c7e583a29609-istio-podinfo\") pod \"e2e-shared-b-w2-bdf47c-openshift-default-6d77688b7b-slw4g\" (UID: \"08571c40-a532-4fa2-a18b-c7e583a29609\") " pod="openshift-ingress/e2e-shared-b-w2-bdf47c-openshift-default-6d77688b7b-slw4g"
Sep 10 17:54:10.119934 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:54:10.119908    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/08571c40-a532-4fa2-a18b-c7e583a29609-istio-token\") pod \"e2e-shared-b-w2-bdf47c-openshift-default-6d77688b7b-slw4g\" (UID: \"08571c40-a532-4fa2-a18b-c7e583a29609\") " pod="openshift-ingress/e2e-shared-b-w2-bdf47c-openshift-default-6d77688b7b-slw4g"
Sep 10 17:54:10.120962 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:54:10.120938    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-h5hlx\" (UniqueName: \"kubernetes.io/projected/08571c40-a532-4fa2-a18b-c7e583a29609-kube-api-access-h5hlx\") pod \"e2e-shared-b-w2-bdf47c-openshift-default-6d77688b7b-slw4g\" (UID: \"08571c40-a532-4fa2-a18b-c7e583a29609\") " pod="openshift-ingress/e2e-shared-b-w2-bdf47c-openshift-default-6d77688b7b-slw4g"
Sep 10 17:54:10.225289 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:54:10.225255    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-w2-bdf47c-openshift-default-6d77688b7b-slw4g"
Sep 10 17:54:10.559860 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:54:10.559832    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-b-w2-bdf47c-openshift-default-6d77688b7b-slw4g"]
Sep 10 17:54:10.561565 ip-10-0-141-180 kubenswrapper[2581]: W0910 17:54:10.561539    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod08571c40_a532_4fa2_a18b_c7e583a29609.slice/crio-289e603f76d0b83729f8ce35371a35d5e6207d1ee115e454f868f3f7ac575a56 WatchSource:0}: Error finding container 289e603f76d0b83729f8ce35371a35d5e6207d1ee115e454f868f3f7ac575a56: Status 404 returned error can't find the container with id 289e603f76d0b83729f8ce35371a35d5e6207d1ee115e454f868f3f7ac575a56
Sep 10 17:54:10.564019 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:54:10.563986    2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 10 17:54:10.564122 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:54:10.564045    2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 10 17:54:10.564122 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:54:10.564073    2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 10 17:54:10.874986 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:54:10.874894    2581 patch_prober.go:28] interesting pod/e2e-shared-b-w3-57474d-openshift-default-5c978cd6bd-d8phk container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.71:15021/healthz/ready\": dial tcp 10.133.0.71:15021: connect: connection refused" start-of-body=
Sep 10 17:54:10.875236 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:54:10.875207    2581 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w3-57474d-openshift-default-5c978cd6bd-d8phk" podUID="0a789c73-9891-4c17-bfa0-948af3f241eb" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.71:15021/healthz/ready\": dial tcp 10.133.0.71:15021: connect: connection refused"
Sep 10 17:54:11.260837 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:54:11.260799    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w2-bdf47c-openshift-default-6d77688b7b-slw4g" event={"ID":"08571c40-a532-4fa2-a18b-c7e583a29609","Type":"ContainerStarted","Data":"c1b960f1ee5824b6bc7903fca91484dd280fc7bb381f4d793972530692cbcf2c"}
Sep 10 17:54:11.261125 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:54:11.261106    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w2-bdf47c-openshift-default-6d77688b7b-slw4g" event={"ID":"08571c40-a532-4fa2-a18b-c7e583a29609","Type":"ContainerStarted","Data":"289e603f76d0b83729f8ce35371a35d5e6207d1ee115e454f868f3f7ac575a56"}
Sep 10 17:54:11.876073 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:54:11.876035    2581 patch_prober.go:28] interesting pod/e2e-shared-b-w3-57474d-openshift-default-5c978cd6bd-d8phk container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.71:15021/healthz/ready\": dial tcp 10.133.0.71:15021: connect: connection refused" start-of-body=
Sep 10 17:54:11.877088 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:54:11.877054    2581 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w3-57474d-openshift-default-5c978cd6bd-d8phk" podUID="0a789c73-9891-4c17-bfa0-948af3f241eb" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.71:15021/healthz/ready\": dial tcp 10.133.0.71:15021: connect: connection refused"
Sep 10 17:54:12.225421 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:54:12.225379    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-b-w2-bdf47c-openshift-default-6d77688b7b-slw4g"
Sep 10 17:54:12.230678 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:54:12.230651    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-b-w2-bdf47c-openshift-default-6d77688b7b-slw4g"
Sep 10 17:54:12.262369 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:54:12.262314    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-shared-b-w2-bdf47c-openshift-default-6d77688b7b-slw4g" podStartSLOduration=3.262298065 podStartE2EDuration="3.262298065s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 17:54:09 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 17:54:11.29139058 +0000 UTC m=+1070.945464514" watchObservedRunningTime="2026-09-10 17:54:12.262298065 +0000 UTC m=+1071.916372016"
Sep 10 17:54:12.267258 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:54:12.267231    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-b-w2-bdf47c-openshift-default-6d77688b7b-slw4g"
Sep 10 17:54:12.268003 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:54:12.267983    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-b-w2-bdf47c-openshift-default-6d77688b7b-slw4g"
Sep 10 17:54:12.877761 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:54:12.877725    2581 patch_prober.go:28] interesting pod/e2e-shared-b-w3-57474d-openshift-default-5c978cd6bd-d8phk container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.71:15021/healthz/ready\": dial tcp 10.133.0.71:15021: connect: connection refused" start-of-body=
Sep 10 17:54:12.878330 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:54:12.878293    2581 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w3-57474d-openshift-default-5c978cd6bd-d8phk" podUID="0a789c73-9891-4c17-bfa0-948af3f241eb" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.71:15021/healthz/ready\": dial tcp 10.133.0.71:15021: connect: connection refused"
Sep 10 17:54:13.875146 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:54:13.875105    2581 patch_prober.go:28] interesting pod/e2e-shared-b-w3-57474d-openshift-default-5c978cd6bd-d8phk container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.71:15021/healthz/ready\": dial tcp 10.133.0.71:15021: connect: connection refused" start-of-body=
Sep 10 17:54:13.875462 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:54:13.875428    2581 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w3-57474d-openshift-default-5c978cd6bd-d8phk" podUID="0a789c73-9891-4c17-bfa0-948af3f241eb" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.71:15021/healthz/ready\": dial tcp 10.133.0.71:15021: connect: connection refused"
Sep 10 17:54:14.878610 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:54:14.878578    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-b-w3-57474d-openshift-default-5c978cd6bd-d8phk"
Sep 10 17:54:14.879021 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:54:14.878898    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-b-w3-57474d-openshift-default-5c978cd6bd-d8phk"
Sep 10 17:54:14.879589 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:54:14.879471    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-b-w3-57474d-openshift-default-5c978cd6bd-d8phk"
Sep 10 17:54:15.505105 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:54:15.505073    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w2-bdf47c-6dfb5579ff-sgr2m"]
Sep 10 17:54:15.509396 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:54:15.509371    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-b-w2-bdf47c-6dfb5579ff-sgr2m"
Sep 10 17:54:15.512211 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:54:15.512169    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-shared-b-w2-bdf47c\""
Sep 10 17:54:15.512330 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:54:15.512260    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-shared-b-w2-bdf47c-dockercfg-tqfs9\""
Sep 10 17:54:15.522829 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:54:15.522803    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w2-bdf47c-6dfb5579ff-sgr2m"]
Sep 10 17:54:15.668541 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:54:15.668503    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/4b2ac153-7018-4bfc-b150-079763c603e6-plugins-config-volume\") pod \"payload-processing-e2e-shared-b-w2-bdf47c-6dfb5579ff-sgr2m\" (UID: \"4b2ac153-7018-4bfc-b150-079763c603e6\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w2-bdf47c-6dfb5579ff-sgr2m"
Sep 10 17:54:15.668748 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:54:15.668613    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fssxq\" (UniqueName: \"kubernetes.io/projected/4b2ac153-7018-4bfc-b150-079763c603e6-kube-api-access-fssxq\") pod \"payload-processing-e2e-shared-b-w2-bdf47c-6dfb5579ff-sgr2m\" (UID: \"4b2ac153-7018-4bfc-b150-079763c603e6\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w2-bdf47c-6dfb5579ff-sgr2m"
Sep 10 17:54:15.769548 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:54:15.769450    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fssxq\" (UniqueName: \"kubernetes.io/projected/4b2ac153-7018-4bfc-b150-079763c603e6-kube-api-access-fssxq\") pod \"payload-processing-e2e-shared-b-w2-bdf47c-6dfb5579ff-sgr2m\" (UID: \"4b2ac153-7018-4bfc-b150-079763c603e6\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w2-bdf47c-6dfb5579ff-sgr2m"
Sep 10 17:54:15.769761 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:54:15.769562    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/4b2ac153-7018-4bfc-b150-079763c603e6-plugins-config-volume\") pod \"payload-processing-e2e-shared-b-w2-bdf47c-6dfb5579ff-sgr2m\" (UID: \"4b2ac153-7018-4bfc-b150-079763c603e6\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w2-bdf47c-6dfb5579ff-sgr2m"
Sep 10 17:54:15.770321 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:54:15.770295    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/4b2ac153-7018-4bfc-b150-079763c603e6-plugins-config-volume\") pod \"payload-processing-e2e-shared-b-w2-bdf47c-6dfb5579ff-sgr2m\" (UID: \"4b2ac153-7018-4bfc-b150-079763c603e6\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w2-bdf47c-6dfb5579ff-sgr2m"
Sep 10 17:54:15.785367 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:54:15.785334    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fssxq\" (UniqueName: \"kubernetes.io/projected/4b2ac153-7018-4bfc-b150-079763c603e6-kube-api-access-fssxq\") pod \"payload-processing-e2e-shared-b-w2-bdf47c-6dfb5579ff-sgr2m\" (UID: \"4b2ac153-7018-4bfc-b150-079763c603e6\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w2-bdf47c-6dfb5579ff-sgr2m"
Sep 10 17:54:15.822264 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:54:15.822236    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-b-w2-bdf47c-6dfb5579ff-sgr2m"
Sep 10 17:54:16.103231 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:54:16.103200    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w2-bdf47c-6dfb5579ff-sgr2m"]
Sep 10 17:54:16.104598 ip-10-0-141-180 kubenswrapper[2581]: W0910 17:54:16.104573    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4b2ac153_7018_4bfc_b150_079763c603e6.slice/crio-beb1e8f9f188880e78c55855065f9a9d4529c043489a9d7bc95e06c7891f9520 WatchSource:0}: Error finding container beb1e8f9f188880e78c55855065f9a9d4529c043489a9d7bc95e06c7891f9520: Status 404 returned error can't find the container with id beb1e8f9f188880e78c55855065f9a9d4529c043489a9d7bc95e06c7891f9520
Sep 10 17:54:16.312207 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:54:16.312172    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-b-w2-bdf47c-6dfb5579ff-sgr2m" event={"ID":"4b2ac153-7018-4bfc-b150-079763c603e6","Type":"ContainerStarted","Data":"dbf6b9714d4283c3203f1b344e8e5d474b5f34a262caaaee497271351a2d042c"}
Sep 10 17:54:16.312404 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:54:16.312233    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-b-w2-bdf47c-6dfb5579ff-sgr2m" event={"ID":"4b2ac153-7018-4bfc-b150-079763c603e6","Type":"ContainerStarted","Data":"beb1e8f9f188880e78c55855065f9a9d4529c043489a9d7bc95e06c7891f9520"}
Sep 10 17:54:16.312404 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:54:16.312275    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-shared-b-w2-bdf47c-6dfb5579ff-sgr2m"
Sep 10 17:54:16.346380 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:54:16.346330    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-shared-b-w2-bdf47c-6dfb5579ff-sgr2m" podStartSLOduration=1.3463142430000001 podStartE2EDuration="1.346314243s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 17:54:15 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 17:54:16.344982947 +0000 UTC m=+1075.999056872" watchObservedRunningTime="2026-09-10 17:54:16.346314243 +0000 UTC m=+1076.000388138"
Sep 10 17:54:27.321833 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:54:27.321798    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-shared-b-w2-bdf47c-6dfb5579ff-sgr2m"
Sep 10 17:54:31.444565 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:54:31.444526    2581 generic.go:356] "Generic (PLEG): container finished" podID="08571c40-a532-4fa2-a18b-c7e583a29609" containerID="c1b960f1ee5824b6bc7903fca91484dd280fc7bb381f4d793972530692cbcf2c" exitCode=0
Sep 10 17:54:31.445017 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:54:31.444603    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w2-bdf47c-openshift-default-6d77688b7b-slw4g" event={"ID":"08571c40-a532-4fa2-a18b-c7e583a29609","Type":"ContainerDied","Data":"c1b960f1ee5824b6bc7903fca91484dd280fc7bb381f4d793972530692cbcf2c"}
Sep 10 17:54:31.445060 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:54:31.445042    2581 scope.go:117] "RemoveContainer" containerID="c1b960f1ee5824b6bc7903fca91484dd280fc7bb381f4d793972530692cbcf2c"
Sep 10 17:54:31.446203 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:54:31.446178    2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 10 17:54:31.446259 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:54:31.446245    2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 10 17:54:31.446295 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:54:31.446282    2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 10 17:54:32.452907 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:54:32.452873    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w2-bdf47c-openshift-default-6d77688b7b-slw4g" event={"ID":"08571c40-a532-4fa2-a18b-c7e583a29609","Type":"ContainerStarted","Data":"5b7c9837102beba86605ce1da3791ab14f074b2719f75ca25e4779ef5659a90e"}
Sep 10 17:54:33.225617 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:54:33.225577    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-b-w2-bdf47c-openshift-default-6d77688b7b-slw4g"
Sep 10 17:54:33.225851 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:54:33.225796    2581 patch_prober.go:28] interesting pod/e2e-shared-b-w2-bdf47c-openshift-default-6d77688b7b-slw4g container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.72:15021/healthz/ready\": dial tcp 10.133.0.72:15021: connect: connection refused" start-of-body=
Sep 10 17:54:33.225930 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:54:33.225851    2581 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w2-bdf47c-openshift-default-6d77688b7b-slw4g" podUID="08571c40-a532-4fa2-a18b-c7e583a29609" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.72:15021/healthz/ready\": dial tcp 10.133.0.72:15021: connect: connection refused"
Sep 10 17:54:34.225938 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:54:34.225896    2581 patch_prober.go:28] interesting pod/e2e-shared-b-w2-bdf47c-openshift-default-6d77688b7b-slw4g container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.72:15021/healthz/ready\": dial tcp 10.133.0.72:15021: connect: connection refused" start-of-body=
Sep 10 17:54:34.226331 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:54:34.225966    2581 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w2-bdf47c-openshift-default-6d77688b7b-slw4g" podUID="08571c40-a532-4fa2-a18b-c7e583a29609" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.72:15021/healthz/ready\": dial tcp 10.133.0.72:15021: connect: connection refused"
Sep 10 17:54:35.226548 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:54:35.226511    2581 patch_prober.go:28] interesting pod/e2e-shared-b-w2-bdf47c-openshift-default-6d77688b7b-slw4g container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.72:15021/healthz/ready\": dial tcp 10.133.0.72:15021: connect: connection refused" start-of-body=
Sep 10 17:54:35.226947 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:54:35.226573    2581 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w2-bdf47c-openshift-default-6d77688b7b-slw4g" podUID="08571c40-a532-4fa2-a18b-c7e583a29609" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.72:15021/healthz/ready\": dial tcp 10.133.0.72:15021: connect: connection refused"
Sep 10 17:54:36.226472 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:54:36.226431    2581 patch_prober.go:28] interesting pod/e2e-shared-b-w2-bdf47c-openshift-default-6d77688b7b-slw4g container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.72:15021/healthz/ready\": dial tcp 10.133.0.72:15021: connect: connection refused" start-of-body=
Sep 10 17:54:36.226752 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:54:36.226494    2581 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w2-bdf47c-openshift-default-6d77688b7b-slw4g" podUID="08571c40-a532-4fa2-a18b-c7e583a29609" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.72:15021/healthz/ready\": dial tcp 10.133.0.72:15021: connect: connection refused"
Sep 10 17:54:37.225619 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:54:37.225580    2581 patch_prober.go:28] interesting pod/e2e-shared-b-w2-bdf47c-openshift-default-6d77688b7b-slw4g container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.72:15021/healthz/ready\": dial tcp 10.133.0.72:15021: connect: connection refused" start-of-body=
Sep 10 17:54:37.225819 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:54:37.225670    2581 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w2-bdf47c-openshift-default-6d77688b7b-slw4g" podUID="08571c40-a532-4fa2-a18b-c7e583a29609" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.72:15021/healthz/ready\": dial tcp 10.133.0.72:15021: connect: connection refused"
Sep 10 17:54:38.229944 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:54:38.229912    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-b-w2-bdf47c-openshift-default-6d77688b7b-slw4g"
Sep 10 17:54:38.230405 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:54:38.230320    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-b-w2-bdf47c-openshift-default-6d77688b7b-slw4g"
Sep 10 17:54:38.231529 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:54:38.231510    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-b-w2-bdf47c-openshift-default-6d77688b7b-slw4g"
Sep 10 17:54:54.459827 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:54:54.459792    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-655f7f0b-7cddcdcbf7-cw82g"]
Sep 10 17:54:54.463322 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:54:54.463305    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-655f7f0b-7cddcdcbf7-cw82g"
Sep 10 17:54:54.465213 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:54:54.465186    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dwrhd\" (UniqueName: \"kubernetes.io/projected/621424ca-033c-4500-93ec-14dd86344060-kube-api-access-dwrhd\") pod \"payload-processing-e2e-mt-655f7f0b-7cddcdcbf7-cw82g\" (UID: \"621424ca-033c-4500-93ec-14dd86344060\") " pod="openshift-ingress/payload-processing-e2e-mt-655f7f0b-7cddcdcbf7-cw82g"
Sep 10 17:54:54.465321 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:54:54.465300    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/621424ca-033c-4500-93ec-14dd86344060-plugins-config-volume\") pod \"payload-processing-e2e-mt-655f7f0b-7cddcdcbf7-cw82g\" (UID: \"621424ca-033c-4500-93ec-14dd86344060\") " pod="openshift-ingress/payload-processing-e2e-mt-655f7f0b-7cddcdcbf7-cw82g"
Sep 10 17:54:54.467953 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:54:54.467934    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-mt-655f7f0b\""
Sep 10 17:54:54.468057 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:54:54.468007    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-mt-655f7f0b-dockercfg-5wr9d\""
Sep 10 17:54:54.477527 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:54:54.477504    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-655f7f0b-7cddcdcbf7-cw82g"]
Sep 10 17:54:54.566064 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:54:54.566009    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/621424ca-033c-4500-93ec-14dd86344060-plugins-config-volume\") pod \"payload-processing-e2e-mt-655f7f0b-7cddcdcbf7-cw82g\" (UID: \"621424ca-033c-4500-93ec-14dd86344060\") " pod="openshift-ingress/payload-processing-e2e-mt-655f7f0b-7cddcdcbf7-cw82g"
Sep 10 17:54:54.566268 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:54:54.566126    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dwrhd\" (UniqueName: \"kubernetes.io/projected/621424ca-033c-4500-93ec-14dd86344060-kube-api-access-dwrhd\") pod \"payload-processing-e2e-mt-655f7f0b-7cddcdcbf7-cw82g\" (UID: \"621424ca-033c-4500-93ec-14dd86344060\") " pod="openshift-ingress/payload-processing-e2e-mt-655f7f0b-7cddcdcbf7-cw82g"
Sep 10 17:54:54.567065 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:54:54.567031    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/621424ca-033c-4500-93ec-14dd86344060-plugins-config-volume\") pod \"payload-processing-e2e-mt-655f7f0b-7cddcdcbf7-cw82g\" (UID: \"621424ca-033c-4500-93ec-14dd86344060\") " pod="openshift-ingress/payload-processing-e2e-mt-655f7f0b-7cddcdcbf7-cw82g"
Sep 10 17:54:54.576190 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:54:54.576160    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dwrhd\" (UniqueName: \"kubernetes.io/projected/621424ca-033c-4500-93ec-14dd86344060-kube-api-access-dwrhd\") pod \"payload-processing-e2e-mt-655f7f0b-7cddcdcbf7-cw82g\" (UID: \"621424ca-033c-4500-93ec-14dd86344060\") " pod="openshift-ingress/payload-processing-e2e-mt-655f7f0b-7cddcdcbf7-cw82g"
Sep 10 17:54:54.774229 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:54:54.774139    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-655f7f0b-7cddcdcbf7-cw82g"
Sep 10 17:54:55.063182 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:54:55.063150    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-655f7f0b-7cddcdcbf7-cw82g"]
Sep 10 17:54:55.064145 ip-10-0-141-180 kubenswrapper[2581]: W0910 17:54:55.064120    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod621424ca_033c_4500_93ec_14dd86344060.slice/crio-a9f0b3fa361b9c2ebcbb9bb5f0387e64eba954983950e6dab2eb36917be60a35 WatchSource:0}: Error finding container a9f0b3fa361b9c2ebcbb9bb5f0387e64eba954983950e6dab2eb36917be60a35: Status 404 returned error can't find the container with id a9f0b3fa361b9c2ebcbb9bb5f0387e64eba954983950e6dab2eb36917be60a35
Sep 10 17:54:55.627261 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:54:55.627225    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-655f7f0b-7cddcdcbf7-cw82g" event={"ID":"621424ca-033c-4500-93ec-14dd86344060","Type":"ContainerStarted","Data":"a854153b5ece5ed426ffc12fad53084cccfd701097d6ba7fbcd3ef8b1ef7f065"}
Sep 10 17:54:55.627665 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:54:55.627280    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-655f7f0b-7cddcdcbf7-cw82g" event={"ID":"621424ca-033c-4500-93ec-14dd86344060","Type":"ContainerStarted","Data":"a9f0b3fa361b9c2ebcbb9bb5f0387e64eba954983950e6dab2eb36917be60a35"}
Sep 10 17:54:55.627665 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:54:55.627341    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-mt-655f7f0b-7cddcdcbf7-cw82g"
Sep 10 17:54:55.646575 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:54:55.646524    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-mt-655f7f0b-7cddcdcbf7-cw82g" podStartSLOduration=1.646509242 podStartE2EDuration="1.646509242s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 17:54:54 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 17:54:55.643323947 +0000 UTC m=+1115.297397875" watchObservedRunningTime="2026-09-10 17:54:55.646509242 +0000 UTC m=+1115.300583146"
Sep 10 17:55:06.636839 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:55:06.636803    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-mt-655f7f0b-7cddcdcbf7-cw82g"
Sep 10 17:55:20.972564 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:55:20.972528    2581 scope.go:117] "RemoveContainer" containerID="b75b0ea151068376533342fecfbef5908cd0876a562389172dc38515c9e13916"
Sep 10 17:55:51.663085 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:55:51.663011    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-655f7f0b-zmmd7"]
Sep 10 17:55:51.666100 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:55:51.666084    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-655f7f0b-zmmd7"
Sep 10 17:55:51.668987 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:55:51.668965    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-k7zj6\""
Sep 10 17:55:51.669113 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:55:51.668963    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 10 17:55:51.675155 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:55:51.675132    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-655f7f0b-zmmd7"]
Sep 10 17:55:51.753310 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:55:51.753273    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/99cadc95-8d81-430c-8155-da1b362b65ca-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-655f7f0b-zmmd7\" (UID: \"99cadc95-8d81-430c-8155-da1b362b65ca\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-655f7f0b-zmmd7"
Sep 10 17:55:51.854313 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:55:51.854279    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/99cadc95-8d81-430c-8155-da1b362b65ca-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-655f7f0b-zmmd7\" (UID: \"99cadc95-8d81-430c-8155-da1b362b65ca\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-655f7f0b-zmmd7"
Sep 10 17:55:51.854969 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:55:51.854925    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/99cadc95-8d81-430c-8155-da1b362b65ca-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-655f7f0b-zmmd7\" (UID: \"99cadc95-8d81-430c-8155-da1b362b65ca\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-655f7f0b-zmmd7"
Sep 10 17:55:51.977761 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:55:51.977725    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-655f7f0b-zmmd7"
Sep 10 17:55:52.253074 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:55:52.253028    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-655f7f0b-zmmd7"]
Sep 10 17:55:52.255898 ip-10-0-141-180 kubenswrapper[2581]: W0910 17:55:52.255873    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod99cadc95_8d81_430c_8155_da1b362b65ca.slice/crio-9921edee07ae3242b0b99cee20fa72522010dd5b35fe7708576f8225e322bc60 WatchSource:0}: Error finding container 9921edee07ae3242b0b99cee20fa72522010dd5b35fe7708576f8225e322bc60: Status 404 returned error can't find the container with id 9921edee07ae3242b0b99cee20fa72522010dd5b35fe7708576f8225e322bc60
Sep 10 17:55:53.003587 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:55:53.003547    2581 generic.go:356] "Generic (PLEG): container finished" podID="99cadc95-8d81-430c-8155-da1b362b65ca" containerID="9f035bb916888bb336f1b712aedef76381ee59ca672c19a3b263f6e05c3a110a" exitCode=0
Sep 10 17:55:53.003993 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:55:53.003626    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-655f7f0b-zmmd7" event={"ID":"99cadc95-8d81-430c-8155-da1b362b65ca","Type":"ContainerDied","Data":"9f035bb916888bb336f1b712aedef76381ee59ca672c19a3b263f6e05c3a110a"}
Sep 10 17:55:53.003993 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:55:53.003688    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-655f7f0b-zmmd7" event={"ID":"99cadc95-8d81-430c-8155-da1b362b65ca","Type":"ContainerStarted","Data":"9921edee07ae3242b0b99cee20fa72522010dd5b35fe7708576f8225e322bc60"}
Sep 10 17:55:54.147900 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:55:54.147873    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-655f7f0b-zmmd7"
Sep 10 17:55:54.277324 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:55:54.277241    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/99cadc95-8d81-430c-8155-da1b362b65ca-maas-api-service-ca\") pod \"99cadc95-8d81-430c-8155-da1b362b65ca\" (UID: \"99cadc95-8d81-430c-8155-da1b362b65ca\") "
Sep 10 17:55:54.277555 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:55:54.277531    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/99cadc95-8d81-430c-8155-da1b362b65ca-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "99cadc95-8d81-430c-8155-da1b362b65ca" (UID: "99cadc95-8d81-430c-8155-da1b362b65ca"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 17:55:54.378528 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:55:54.378494    2581 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/99cadc95-8d81-430c-8155-da1b362b65ca-maas-api-service-ca\") on node \"ip-10-0-141-180.ec2.internal\" DevicePath \"\""
Sep 10 17:55:55.017923 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:55:55.017891    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-655f7f0b-zmmd7" event={"ID":"99cadc95-8d81-430c-8155-da1b362b65ca","Type":"ContainerDied","Data":"9921edee07ae3242b0b99cee20fa72522010dd5b35fe7708576f8225e322bc60"}
Sep 10 17:55:55.018087 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:55:55.017928    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-655f7f0b-zmmd7"
Sep 10 17:55:55.018087 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:55:55.017939    2581 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="9921edee07ae3242b0b99cee20fa72522010dd5b35fe7708576f8225e322bc60"
Sep 10 17:56:07.214058 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:56:07.214022    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-655f7f0b-7cddcdcbf7-cw82g"]
Sep 10 17:56:07.214496 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:56:07.214273    2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-mt-655f7f0b-7cddcdcbf7-cw82g" podUID="621424ca-033c-4500-93ec-14dd86344060" containerName="payload-processing" containerID="cri-o://a854153b5ece5ed426ffc12fad53084cccfd701097d6ba7fbcd3ef8b1ef7f065" gracePeriod=30
Sep 10 17:56:07.402319 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:56:07.402286    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-655f7f0b-zmmd7"]
Sep 10 17:56:07.406939 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:56:07.406910    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-655f7f0b-zmmd7"]
Sep 10 17:56:07.477750 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:56:07.477664    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-655f7f0b-7cddcdcbf7-cw82g"
Sep 10 17:56:07.503937 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:56:07.502821    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-dwrhd\" (UniqueName: \"kubernetes.io/projected/621424ca-033c-4500-93ec-14dd86344060-kube-api-access-dwrhd\") pod \"621424ca-033c-4500-93ec-14dd86344060\" (UID: \"621424ca-033c-4500-93ec-14dd86344060\") "
Sep 10 17:56:07.503937 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:56:07.502915    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/621424ca-033c-4500-93ec-14dd86344060-plugins-config-volume\") pod \"621424ca-033c-4500-93ec-14dd86344060\" (UID: \"621424ca-033c-4500-93ec-14dd86344060\") "
Sep 10 17:56:07.503937 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:56:07.503750    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/621424ca-033c-4500-93ec-14dd86344060-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "621424ca-033c-4500-93ec-14dd86344060" (UID: "621424ca-033c-4500-93ec-14dd86344060"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 17:56:07.507486 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:56:07.507454    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/621424ca-033c-4500-93ec-14dd86344060-kube-api-access-dwrhd" (OuterVolumeSpecName: "kube-api-access-dwrhd") pod "621424ca-033c-4500-93ec-14dd86344060" (UID: "621424ca-033c-4500-93ec-14dd86344060"). InnerVolumeSpecName "kube-api-access-dwrhd". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 17:56:07.604864 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:56:07.604826    2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-dwrhd\" (UniqueName: \"kubernetes.io/projected/621424ca-033c-4500-93ec-14dd86344060-kube-api-access-dwrhd\") on node \"ip-10-0-141-180.ec2.internal\" DevicePath \"\""
Sep 10 17:56:07.604864 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:56:07.604858    2581 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/621424ca-033c-4500-93ec-14dd86344060-plugins-config-volume\") on node \"ip-10-0-141-180.ec2.internal\" DevicePath \"\""
Sep 10 17:56:08.104689 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:56:08.104651    2581 generic.go:356] "Generic (PLEG): container finished" podID="621424ca-033c-4500-93ec-14dd86344060" containerID="a854153b5ece5ed426ffc12fad53084cccfd701097d6ba7fbcd3ef8b1ef7f065" exitCode=0
Sep 10 17:56:08.104885 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:56:08.104714    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-655f7f0b-7cddcdcbf7-cw82g"
Sep 10 17:56:08.104885 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:56:08.104734    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-655f7f0b-7cddcdcbf7-cw82g" event={"ID":"621424ca-033c-4500-93ec-14dd86344060","Type":"ContainerDied","Data":"a854153b5ece5ed426ffc12fad53084cccfd701097d6ba7fbcd3ef8b1ef7f065"}
Sep 10 17:56:08.104885 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:56:08.104789    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-655f7f0b-7cddcdcbf7-cw82g" event={"ID":"621424ca-033c-4500-93ec-14dd86344060","Type":"ContainerDied","Data":"a9f0b3fa361b9c2ebcbb9bb5f0387e64eba954983950e6dab2eb36917be60a35"}
Sep 10 17:56:08.104885 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:56:08.104816    2581 scope.go:117] "RemoveContainer" containerID="a854153b5ece5ed426ffc12fad53084cccfd701097d6ba7fbcd3ef8b1ef7f065"
Sep 10 17:56:08.115291 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:56:08.115275    2581 scope.go:117] "RemoveContainer" containerID="a854153b5ece5ed426ffc12fad53084cccfd701097d6ba7fbcd3ef8b1ef7f065"
Sep 10 17:56:08.115566 ip-10-0-141-180 kubenswrapper[2581]: E0910 17:56:08.115548    2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a854153b5ece5ed426ffc12fad53084cccfd701097d6ba7fbcd3ef8b1ef7f065\": container with ID starting with a854153b5ece5ed426ffc12fad53084cccfd701097d6ba7fbcd3ef8b1ef7f065 not found: ID does not exist" containerID="a854153b5ece5ed426ffc12fad53084cccfd701097d6ba7fbcd3ef8b1ef7f065"
Sep 10 17:56:08.115610 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:56:08.115579    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a854153b5ece5ed426ffc12fad53084cccfd701097d6ba7fbcd3ef8b1ef7f065"} err="failed to get container status \"a854153b5ece5ed426ffc12fad53084cccfd701097d6ba7fbcd3ef8b1ef7f065\": rpc error: code = NotFound desc = could not find container \"a854153b5ece5ed426ffc12fad53084cccfd701097d6ba7fbcd3ef8b1ef7f065\": container with ID starting with a854153b5ece5ed426ffc12fad53084cccfd701097d6ba7fbcd3ef8b1ef7f065 not found: ID does not exist"
Sep 10 17:56:08.127826 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:56:08.127804    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-655f7f0b-7cddcdcbf7-cw82g"]
Sep 10 17:56:08.132371 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:56:08.132348    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-655f7f0b-7cddcdcbf7-cw82g"]
Sep 10 17:56:08.864435 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:56:08.864403    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="621424ca-033c-4500-93ec-14dd86344060" path="/var/lib/kubelet/pods/621424ca-033c-4500-93ec-14dd86344060/volumes"
Sep 10 17:56:08.864830 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:56:08.864769    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="99cadc95-8d81-430c-8155-da1b362b65ca" path="/var/lib/kubelet/pods/99cadc95-8d81-430c-8155-da1b362b65ca/volumes"
Sep 10 17:56:20.996296 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:56:20.996267    2581 scope.go:117] "RemoveContainer" containerID="12982feb88d8846d024d793c7b2e542eb3004cb8d0d37a778939f56e9829f902"
Sep 10 17:56:21.005496 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:56:21.005472    2581 scope.go:117] "RemoveContainer" containerID="fce53936a17f1bed97660b5655f737a98f3a85f7ca469637e97e64d46d4d0c0d"
Sep 10 17:56:21.013681 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:56:21.013659    2581 scope.go:117] "RemoveContainer" containerID="e8b8a4a94ad0e94c71d9fc7fa65815c0d652113b37059323a8e84f329ef4a567"
Sep 10 17:57:21.048594 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:57:21.048521    2581 scope.go:117] "RemoveContainer" containerID="b8cc280ca38dbccbfae9bc816a156437a22151e0c52d44fc141209f39b941bca"
Sep 10 17:57:21.058171 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:57:21.058151    2581 scope.go:117] "RemoveContainer" containerID="5718f0eb8ee5c7d49c213e7feadd7770c478cba1b2e051aa55380f85f815c65f"
Sep 10 17:57:59.141118 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:57:59.141071    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-c5c751-56fcdcfbf5-wmxxb"]
Sep 10 17:57:59.141754 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:57:59.141737    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="99cadc95-8d81-430c-8155-da1b362b65ca" containerName="revoke-keys"
Sep 10 17:57:59.141754 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:57:59.141755    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="99cadc95-8d81-430c-8155-da1b362b65ca" containerName="revoke-keys"
Sep 10 17:57:59.141887 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:57:59.141780    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="621424ca-033c-4500-93ec-14dd86344060" containerName="payload-processing"
Sep 10 17:57:59.141887 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:57:59.141786    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="621424ca-033c-4500-93ec-14dd86344060" containerName="payload-processing"
Sep 10 17:57:59.141887 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:57:59.141856    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="621424ca-033c-4500-93ec-14dd86344060" containerName="payload-processing"
Sep 10 17:57:59.141887 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:57:59.141867    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="99cadc95-8d81-430c-8155-da1b362b65ca" containerName="revoke-keys"
Sep 10 17:57:59.144832 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:57:59.144815    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-a-c5c751-56fcdcfbf5-wmxxb"
Sep 10 17:57:59.147871 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:57:59.147844    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-api-a-c5c751\""
Sep 10 17:57:59.176403 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:57:59.176370    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-c5c751-56fcdcfbf5-wmxxb"]
Sep 10 17:57:59.209502 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:57:59.209472    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-f996v\" (UniqueName: \"kubernetes.io/projected/d9dda529-efe6-4866-ae87-54cde5aaea5d-kube-api-access-f996v\") pod \"payload-processing-e2e-api-a-c5c751-56fcdcfbf5-wmxxb\" (UID: \"d9dda529-efe6-4866-ae87-54cde5aaea5d\") " pod="openshift-ingress/payload-processing-e2e-api-a-c5c751-56fcdcfbf5-wmxxb"
Sep 10 17:57:59.209632 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:57:59.209614    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d9dda529-efe6-4866-ae87-54cde5aaea5d-plugins-config-volume\") pod \"payload-processing-e2e-api-a-c5c751-56fcdcfbf5-wmxxb\" (UID: \"d9dda529-efe6-4866-ae87-54cde5aaea5d\") " pod="openshift-ingress/payload-processing-e2e-api-a-c5c751-56fcdcfbf5-wmxxb"
Sep 10 17:57:59.311082 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:57:59.311042    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d9dda529-efe6-4866-ae87-54cde5aaea5d-plugins-config-volume\") pod \"payload-processing-e2e-api-a-c5c751-56fcdcfbf5-wmxxb\" (UID: \"d9dda529-efe6-4866-ae87-54cde5aaea5d\") " pod="openshift-ingress/payload-processing-e2e-api-a-c5c751-56fcdcfbf5-wmxxb"
Sep 10 17:57:59.311298 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:57:59.311144    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-f996v\" (UniqueName: \"kubernetes.io/projected/d9dda529-efe6-4866-ae87-54cde5aaea5d-kube-api-access-f996v\") pod \"payload-processing-e2e-api-a-c5c751-56fcdcfbf5-wmxxb\" (UID: \"d9dda529-efe6-4866-ae87-54cde5aaea5d\") " pod="openshift-ingress/payload-processing-e2e-api-a-c5c751-56fcdcfbf5-wmxxb"
Sep 10 17:57:59.312352 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:57:59.312321    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d9dda529-efe6-4866-ae87-54cde5aaea5d-plugins-config-volume\") pod \"payload-processing-e2e-api-a-c5c751-56fcdcfbf5-wmxxb\" (UID: \"d9dda529-efe6-4866-ae87-54cde5aaea5d\") " pod="openshift-ingress/payload-processing-e2e-api-a-c5c751-56fcdcfbf5-wmxxb"
Sep 10 17:57:59.325978 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:57:59.325947    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-f996v\" (UniqueName: \"kubernetes.io/projected/d9dda529-efe6-4866-ae87-54cde5aaea5d-kube-api-access-f996v\") pod \"payload-processing-e2e-api-a-c5c751-56fcdcfbf5-wmxxb\" (UID: \"d9dda529-efe6-4866-ae87-54cde5aaea5d\") " pod="openshift-ingress/payload-processing-e2e-api-a-c5c751-56fcdcfbf5-wmxxb"
Sep 10 17:57:59.454539 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:57:59.454461    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-a-c5c751-56fcdcfbf5-wmxxb"
Sep 10 17:57:59.759844 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:57:59.759818    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-c5c751-56fcdcfbf5-wmxxb"]
Sep 10 17:57:59.760287 ip-10-0-141-180 kubenswrapper[2581]: W0910 17:57:59.760261    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd9dda529_efe6_4866_ae87_54cde5aaea5d.slice/crio-6abb378310dcb910cac4c003cc08183cd6f18ea56c1ae94122eca3aa4bb56463 WatchSource:0}: Error finding container 6abb378310dcb910cac4c003cc08183cd6f18ea56c1ae94122eca3aa4bb56463: Status 404 returned error can't find the container with id 6abb378310dcb910cac4c003cc08183cd6f18ea56c1ae94122eca3aa4bb56463
Sep 10 17:57:59.842077 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:57:59.842035    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-a-c5c751-56fcdcfbf5-wmxxb" event={"ID":"d9dda529-efe6-4866-ae87-54cde5aaea5d","Type":"ContainerStarted","Data":"6abb378310dcb910cac4c003cc08183cd6f18ea56c1ae94122eca3aa4bb56463"}
Sep 10 17:58:00.850354 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:58:00.850313    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-a-c5c751-56fcdcfbf5-wmxxb" event={"ID":"d9dda529-efe6-4866-ae87-54cde5aaea5d","Type":"ContainerStarted","Data":"2ecdccc7a3ebe338fb8443d06bd1398c10be44da1d2fd24e4eeb0eee819280e9"}
Sep 10 17:58:00.850786 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:58:00.850448    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-api-a-c5c751-56fcdcfbf5-wmxxb"
Sep 10 17:58:00.869243 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:58:00.868995    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-api-a-c5c751-56fcdcfbf5-wmxxb" podStartSLOduration=1.868975917 podStartE2EDuration="1.868975917s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 17:57:59 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 17:58:00.868397699 +0000 UTC m=+1300.522471620" watchObservedRunningTime="2026-09-10 17:58:00.868975917 +0000 UTC m=+1300.523049821"
Sep 10 17:58:11.858316 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:58:11.858280    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-api-a-c5c751-56fcdcfbf5-wmxxb"
Sep 10 17:58:21.085463 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:58:21.085422    2581 scope.go:117] "RemoveContainer" containerID="d8976504ad86a25d1998bae7e03ca50d584c2410a8c5151eade93752dffdb563"
Sep 10 17:58:28.619107 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:58:28.619065    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-b-abad1a-768d8ff85d-c2xd2"]
Sep 10 17:58:28.624586 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:58:28.624563    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-b-abad1a-768d8ff85d-c2xd2"
Sep 10 17:58:28.627315 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:58:28.627291    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-api-b-abad1a\""
Sep 10 17:58:28.627732 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:58:28.627708    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-api-b-abad1a-dockercfg-jwwcv\""
Sep 10 17:58:28.642226 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:58:28.642184    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-b-abad1a-768d8ff85d-c2xd2"]
Sep 10 17:58:28.719532 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:58:28.719502    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/66559533-80d3-4598-a778-a507fa1938ec-plugins-config-volume\") pod \"payload-pre-processing-e2e-api-b-abad1a-768d8ff85d-c2xd2\" (UID: \"66559533-80d3-4598-a778-a507fa1938ec\") " pod="openshift-ingress/payload-pre-processing-e2e-api-b-abad1a-768d8ff85d-c2xd2"
Sep 10 17:58:28.719730 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:58:28.719663    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-z8bb6\" (UniqueName: \"kubernetes.io/projected/66559533-80d3-4598-a778-a507fa1938ec-kube-api-access-z8bb6\") pod \"payload-pre-processing-e2e-api-b-abad1a-768d8ff85d-c2xd2\" (UID: \"66559533-80d3-4598-a778-a507fa1938ec\") " pod="openshift-ingress/payload-pre-processing-e2e-api-b-abad1a-768d8ff85d-c2xd2"
Sep 10 17:58:28.820408 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:58:28.820377    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/66559533-80d3-4598-a778-a507fa1938ec-plugins-config-volume\") pod \"payload-pre-processing-e2e-api-b-abad1a-768d8ff85d-c2xd2\" (UID: \"66559533-80d3-4598-a778-a507fa1938ec\") " pod="openshift-ingress/payload-pre-processing-e2e-api-b-abad1a-768d8ff85d-c2xd2"
Sep 10 17:58:28.820575 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:58:28.820487    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-z8bb6\" (UniqueName: \"kubernetes.io/projected/66559533-80d3-4598-a778-a507fa1938ec-kube-api-access-z8bb6\") pod \"payload-pre-processing-e2e-api-b-abad1a-768d8ff85d-c2xd2\" (UID: \"66559533-80d3-4598-a778-a507fa1938ec\") " pod="openshift-ingress/payload-pre-processing-e2e-api-b-abad1a-768d8ff85d-c2xd2"
Sep 10 17:58:28.821112 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:58:28.821093    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/66559533-80d3-4598-a778-a507fa1938ec-plugins-config-volume\") pod \"payload-pre-processing-e2e-api-b-abad1a-768d8ff85d-c2xd2\" (UID: \"66559533-80d3-4598-a778-a507fa1938ec\") " pod="openshift-ingress/payload-pre-processing-e2e-api-b-abad1a-768d8ff85d-c2xd2"
Sep 10 17:58:28.829963 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:58:28.829938    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-z8bb6\" (UniqueName: \"kubernetes.io/projected/66559533-80d3-4598-a778-a507fa1938ec-kube-api-access-z8bb6\") pod \"payload-pre-processing-e2e-api-b-abad1a-768d8ff85d-c2xd2\" (UID: \"66559533-80d3-4598-a778-a507fa1938ec\") " pod="openshift-ingress/payload-pre-processing-e2e-api-b-abad1a-768d8ff85d-c2xd2"
Sep 10 17:58:28.936129 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:58:28.936043    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-b-abad1a-768d8ff85d-c2xd2"
Sep 10 17:58:29.217183 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:58:29.217152    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-b-abad1a-768d8ff85d-c2xd2"]
Sep 10 17:58:29.217873 ip-10-0-141-180 kubenswrapper[2581]: W0910 17:58:29.217850    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod66559533_80d3_4598_a778_a507fa1938ec.slice/crio-6ee6b6d659cdbcd14b5be431930939e6247af04ffae0a0b41506fd069f4e2474 WatchSource:0}: Error finding container 6ee6b6d659cdbcd14b5be431930939e6247af04ffae0a0b41506fd069f4e2474: Status 404 returned error can't find the container with id 6ee6b6d659cdbcd14b5be431930939e6247af04ffae0a0b41506fd069f4e2474
Sep 10 17:58:30.045047 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:58:30.045003    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-b-abad1a-768d8ff85d-c2xd2" event={"ID":"66559533-80d3-4598-a778-a507fa1938ec","Type":"ContainerStarted","Data":"16113b77a5581c31ecc0c4c228ae8f4497b6bbea172584dffb8d1e80cad9f688"}
Sep 10 17:58:30.045425 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:58:30.045161    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-api-b-abad1a-768d8ff85d-c2xd2"
Sep 10 17:58:30.045425 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:58:30.045193    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-b-abad1a-768d8ff85d-c2xd2" event={"ID":"66559533-80d3-4598-a778-a507fa1938ec","Type":"ContainerStarted","Data":"6ee6b6d659cdbcd14b5be431930939e6247af04ffae0a0b41506fd069f4e2474"}
Sep 10 17:58:30.116895 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:58:30.116814    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-api-b-abad1a-768d8ff85d-c2xd2" podStartSLOduration=2.116789132 podStartE2EDuration="2.116789132s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 17:58:28 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 17:58:30.111619969 +0000 UTC m=+1329.765693905" watchObservedRunningTime="2026-09-10 17:58:30.116789132 +0000 UTC m=+1329.770863058"
Sep 10 17:58:41.054044 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:58:41.054013    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-api-b-abad1a-768d8ff85d-c2xd2"
Sep 10 17:58:54.426300 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:58:54.426208    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-c5c751-kthqt"]
Sep 10 17:58:54.429918 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:58:54.429895    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-c5c751-kthqt"
Sep 10 17:58:54.432715 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:58:54.432683    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 10 17:58:54.432823 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:58:54.432690    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-k7zj6\""
Sep 10 17:58:54.439614 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:58:54.439593    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-c5c751-kthqt"]
Sep 10 17:58:54.484882 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:58:54.484850    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/dda1ba2e-b3e4-4024-a7d9-5176ce2aafed-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-a-c5c751-kthqt\" (UID: \"dda1ba2e-b3e4-4024-a7d9-5176ce2aafed\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-c5c751-kthqt"
Sep 10 17:58:54.585493 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:58:54.585461    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/dda1ba2e-b3e4-4024-a7d9-5176ce2aafed-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-a-c5c751-kthqt\" (UID: \"dda1ba2e-b3e4-4024-a7d9-5176ce2aafed\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-c5c751-kthqt"
Sep 10 17:58:54.586067 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:58:54.586047    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/dda1ba2e-b3e4-4024-a7d9-5176ce2aafed-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-a-c5c751-kthqt\" (UID: \"dda1ba2e-b3e4-4024-a7d9-5176ce2aafed\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-c5c751-kthqt"
Sep 10 17:58:54.742949 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:58:54.742916    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-c5c751-kthqt"
Sep 10 17:58:55.025002 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:58:55.024976    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-c5c751-kthqt"]
Sep 10 17:58:55.026910 ip-10-0-141-180 kubenswrapper[2581]: W0910 17:58:55.026874    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddda1ba2e_b3e4_4024_a7d9_5176ce2aafed.slice/crio-c5292f62cfe80f64710f72d1e26a259523c4562011ada732cb1b4f8ab89815ab WatchSource:0}: Error finding container c5292f62cfe80f64710f72d1e26a259523c4562011ada732cb1b4f8ab89815ab: Status 404 returned error can't find the container with id c5292f62cfe80f64710f72d1e26a259523c4562011ada732cb1b4f8ab89815ab
Sep 10 17:58:55.209855 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:58:55.209821    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-c5c751-kthqt" event={"ID":"dda1ba2e-b3e4-4024-a7d9-5176ce2aafed","Type":"ContainerStarted","Data":"c5292f62cfe80f64710f72d1e26a259523c4562011ada732cb1b4f8ab89815ab"}
Sep 10 17:58:56.218592 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:58:56.218559    2581 generic.go:356] "Generic (PLEG): container finished" podID="dda1ba2e-b3e4-4024-a7d9-5176ce2aafed" containerID="4d131001c54eb6693b85cc0fed208ef80ceefafc59e5d933152100fc36d64908" exitCode=0
Sep 10 17:58:56.218993 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:58:56.218605    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-c5c751-kthqt" event={"ID":"dda1ba2e-b3e4-4024-a7d9-5176ce2aafed","Type":"ContainerDied","Data":"4d131001c54eb6693b85cc0fed208ef80ceefafc59e5d933152100fc36d64908"}
Sep 10 17:58:57.367585 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:58:57.367555    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-c5c751-kthqt"
Sep 10 17:58:57.409601 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:58:57.409565    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/dda1ba2e-b3e4-4024-a7d9-5176ce2aafed-maas-api-service-ca\") pod \"dda1ba2e-b3e4-4024-a7d9-5176ce2aafed\" (UID: \"dda1ba2e-b3e4-4024-a7d9-5176ce2aafed\") "
Sep 10 17:58:57.409911 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:58:57.409885    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/dda1ba2e-b3e4-4024-a7d9-5176ce2aafed-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "dda1ba2e-b3e4-4024-a7d9-5176ce2aafed" (UID: "dda1ba2e-b3e4-4024-a7d9-5176ce2aafed"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 17:58:57.511060 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:58:57.510969    2581 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/dda1ba2e-b3e4-4024-a7d9-5176ce2aafed-maas-api-service-ca\") on node \"ip-10-0-141-180.ec2.internal\" DevicePath \"\""
Sep 10 17:58:58.233000 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:58:58.232965    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-c5c751-kthqt" event={"ID":"dda1ba2e-b3e4-4024-a7d9-5176ce2aafed","Type":"ContainerDied","Data":"c5292f62cfe80f64710f72d1e26a259523c4562011ada732cb1b4f8ab89815ab"}
Sep 10 17:58:58.233000 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:58:58.233004    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-c5c751-kthqt"
Sep 10 17:58:58.233207 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:58:58.233022    2581 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="c5292f62cfe80f64710f72d1e26a259523c4562011ada732cb1b4f8ab89815ab"
Sep 10 17:59:09.983214 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:59:09.983179    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-c5c751-56fcdcfbf5-wmxxb"]
Sep 10 17:59:09.983831 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:59:09.983478    2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-api-a-c5c751-56fcdcfbf5-wmxxb" podUID="d9dda529-efe6-4866-ae87-54cde5aaea5d" containerName="payload-processing" containerID="cri-o://2ecdccc7a3ebe338fb8443d06bd1398c10be44da1d2fd24e4eeb0eee819280e9" gracePeriod=30
Sep 10 17:59:10.210069 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:59:10.210033    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-c5c751-kthqt"]
Sep 10 17:59:10.214754 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:59:10.214728    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-c5c751-kthqt"]
Sep 10 17:59:10.242405 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:59:10.242351    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-a-c5c751-56fcdcfbf5-wmxxb"
Sep 10 17:59:10.319832 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:59:10.319793    2581 generic.go:356] "Generic (PLEG): container finished" podID="d9dda529-efe6-4866-ae87-54cde5aaea5d" containerID="2ecdccc7a3ebe338fb8443d06bd1398c10be44da1d2fd24e4eeb0eee819280e9" exitCode=0
Sep 10 17:59:10.320018 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:59:10.319847    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-a-c5c751-56fcdcfbf5-wmxxb"
Sep 10 17:59:10.320018 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:59:10.319845    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-a-c5c751-56fcdcfbf5-wmxxb" event={"ID":"d9dda529-efe6-4866-ae87-54cde5aaea5d","Type":"ContainerDied","Data":"2ecdccc7a3ebe338fb8443d06bd1398c10be44da1d2fd24e4eeb0eee819280e9"}
Sep 10 17:59:10.320018 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:59:10.319902    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-a-c5c751-56fcdcfbf5-wmxxb" event={"ID":"d9dda529-efe6-4866-ae87-54cde5aaea5d","Type":"ContainerDied","Data":"6abb378310dcb910cac4c003cc08183cd6f18ea56c1ae94122eca3aa4bb56463"}
Sep 10 17:59:10.320018 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:59:10.319925    2581 scope.go:117] "RemoveContainer" containerID="2ecdccc7a3ebe338fb8443d06bd1398c10be44da1d2fd24e4eeb0eee819280e9"
Sep 10 17:59:10.329976 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:59:10.329959    2581 scope.go:117] "RemoveContainer" containerID="2ecdccc7a3ebe338fb8443d06bd1398c10be44da1d2fd24e4eeb0eee819280e9"
Sep 10 17:59:10.330305 ip-10-0-141-180 kubenswrapper[2581]: E0910 17:59:10.330283    2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2ecdccc7a3ebe338fb8443d06bd1398c10be44da1d2fd24e4eeb0eee819280e9\": container with ID starting with 2ecdccc7a3ebe338fb8443d06bd1398c10be44da1d2fd24e4eeb0eee819280e9 not found: ID does not exist" containerID="2ecdccc7a3ebe338fb8443d06bd1398c10be44da1d2fd24e4eeb0eee819280e9"
Sep 10 17:59:10.330360 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:59:10.330313    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2ecdccc7a3ebe338fb8443d06bd1398c10be44da1d2fd24e4eeb0eee819280e9"} err="failed to get container status \"2ecdccc7a3ebe338fb8443d06bd1398c10be44da1d2fd24e4eeb0eee819280e9\": rpc error: code = NotFound desc = could not find container \"2ecdccc7a3ebe338fb8443d06bd1398c10be44da1d2fd24e4eeb0eee819280e9\": container with ID starting with 2ecdccc7a3ebe338fb8443d06bd1398c10be44da1d2fd24e4eeb0eee819280e9 not found: ID does not exist"
Sep 10 17:59:10.336697 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:59:10.336672    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d9dda529-efe6-4866-ae87-54cde5aaea5d-plugins-config-volume\") pod \"d9dda529-efe6-4866-ae87-54cde5aaea5d\" (UID: \"d9dda529-efe6-4866-ae87-54cde5aaea5d\") "
Sep 10 17:59:10.336828 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:59:10.336722    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-f996v\" (UniqueName: \"kubernetes.io/projected/d9dda529-efe6-4866-ae87-54cde5aaea5d-kube-api-access-f996v\") pod \"d9dda529-efe6-4866-ae87-54cde5aaea5d\" (UID: \"d9dda529-efe6-4866-ae87-54cde5aaea5d\") "
Sep 10 17:59:10.337071 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:59:10.337043    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d9dda529-efe6-4866-ae87-54cde5aaea5d-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "d9dda529-efe6-4866-ae87-54cde5aaea5d" (UID: "d9dda529-efe6-4866-ae87-54cde5aaea5d"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 17:59:10.337174 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:59:10.337115    2581 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d9dda529-efe6-4866-ae87-54cde5aaea5d-plugins-config-volume\") on node \"ip-10-0-141-180.ec2.internal\" DevicePath \"\""
Sep 10 17:59:10.338617 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:59:10.338588    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d9dda529-efe6-4866-ae87-54cde5aaea5d-kube-api-access-f996v" (OuterVolumeSpecName: "kube-api-access-f996v") pod "d9dda529-efe6-4866-ae87-54cde5aaea5d" (UID: "d9dda529-efe6-4866-ae87-54cde5aaea5d"). InnerVolumeSpecName "kube-api-access-f996v". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 17:59:10.438563 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:59:10.438528    2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-f996v\" (UniqueName: \"kubernetes.io/projected/d9dda529-efe6-4866-ae87-54cde5aaea5d-kube-api-access-f996v\") on node \"ip-10-0-141-180.ec2.internal\" DevicePath \"\""
Sep 10 17:59:10.662660 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:59:10.662613    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-c5c751-56fcdcfbf5-wmxxb"]
Sep 10 17:59:10.665370 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:59:10.665346    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-c5c751-56fcdcfbf5-wmxxb"]
Sep 10 17:59:10.862354 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:59:10.862317    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d9dda529-efe6-4866-ae87-54cde5aaea5d" path="/var/lib/kubelet/pods/d9dda529-efe6-4866-ae87-54cde5aaea5d/volumes"
Sep 10 17:59:10.862691 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:59:10.862678    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="dda1ba2e-b3e4-4024-a7d9-5176ce2aafed" path="/var/lib/kubelet/pods/dda1ba2e-b3e4-4024-a7d9-5176ce2aafed/volumes"
Sep 10 17:59:17.640683 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:59:17.640630    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-abad1a-g2zrl"]
Sep 10 17:59:17.641448 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:59:17.641423    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="dda1ba2e-b3e4-4024-a7d9-5176ce2aafed" containerName="revoke-keys"
Sep 10 17:59:17.641448 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:59:17.641447    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="dda1ba2e-b3e4-4024-a7d9-5176ce2aafed" containerName="revoke-keys"
Sep 10 17:59:17.641611 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:59:17.641477    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d9dda529-efe6-4866-ae87-54cde5aaea5d" containerName="payload-processing"
Sep 10 17:59:17.641611 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:59:17.641486    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="d9dda529-efe6-4866-ae87-54cde5aaea5d" containerName="payload-processing"
Sep 10 17:59:17.641611 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:59:17.641585    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="d9dda529-efe6-4866-ae87-54cde5aaea5d" containerName="payload-processing"
Sep 10 17:59:17.641611 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:59:17.641603    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="dda1ba2e-b3e4-4024-a7d9-5176ce2aafed" containerName="revoke-keys"
Sep 10 17:59:17.646661 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:59:17.646614    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-abad1a-g2zrl"
Sep 10 17:59:17.650113 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:59:17.650092    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-k7zj6\""
Sep 10 17:59:17.650246 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:59:17.650230    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 10 17:59:17.659519 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:59:17.659495    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-abad1a-g2zrl"]
Sep 10 17:59:17.712085 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:59:17.712048    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/43034ce3-0ea1-4aef-826d-b84c1e14675f-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-b-abad1a-g2zrl\" (UID: \"43034ce3-0ea1-4aef-826d-b84c1e14675f\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-abad1a-g2zrl"
Sep 10 17:59:17.813215 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:59:17.813176    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/43034ce3-0ea1-4aef-826d-b84c1e14675f-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-b-abad1a-g2zrl\" (UID: \"43034ce3-0ea1-4aef-826d-b84c1e14675f\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-abad1a-g2zrl"
Sep 10 17:59:17.813823 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:59:17.813797    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/43034ce3-0ea1-4aef-826d-b84c1e14675f-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-b-abad1a-g2zrl\" (UID: \"43034ce3-0ea1-4aef-826d-b84c1e14675f\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-abad1a-g2zrl"
Sep 10 17:59:17.959724 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:59:17.959695    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-abad1a-g2zrl"
Sep 10 17:59:18.276803 ip-10-0-141-180 kubenswrapper[2581]: W0910 17:59:18.276771    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod43034ce3_0ea1_4aef_826d_b84c1e14675f.slice/crio-06a9d600ff6efde02007f9bc5e8d3b556242035d8ffb9d87c818792013f99ff6 WatchSource:0}: Error finding container 06a9d600ff6efde02007f9bc5e8d3b556242035d8ffb9d87c818792013f99ff6: Status 404 returned error can't find the container with id 06a9d600ff6efde02007f9bc5e8d3b556242035d8ffb9d87c818792013f99ff6
Sep 10 17:59:18.283867 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:59:18.283838    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-abad1a-g2zrl"]
Sep 10 17:59:18.376259 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:59:18.376208    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-abad1a-g2zrl" event={"ID":"43034ce3-0ea1-4aef-826d-b84c1e14675f","Type":"ContainerStarted","Data":"06a9d600ff6efde02007f9bc5e8d3b556242035d8ffb9d87c818792013f99ff6"}
Sep 10 17:59:19.392944 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:59:19.392852    2581 generic.go:356] "Generic (PLEG): container finished" podID="43034ce3-0ea1-4aef-826d-b84c1e14675f" containerID="e0ed0c7ba945b8f7357d8ad597a1fcf2dee301fa6d4dcfda5c4da8234a1e1d13" exitCode=0
Sep 10 17:59:19.392944 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:59:19.392917    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-abad1a-g2zrl" event={"ID":"43034ce3-0ea1-4aef-826d-b84c1e14675f","Type":"ContainerDied","Data":"e0ed0c7ba945b8f7357d8ad597a1fcf2dee301fa6d4dcfda5c4da8234a1e1d13"}
Sep 10 17:59:20.538210 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:59:20.538182    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-abad1a-g2zrl"
Sep 10 17:59:20.642574 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:59:20.642545    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/43034ce3-0ea1-4aef-826d-b84c1e14675f-maas-api-service-ca\") pod \"43034ce3-0ea1-4aef-826d-b84c1e14675f\" (UID: \"43034ce3-0ea1-4aef-826d-b84c1e14675f\") "
Sep 10 17:59:20.642913 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:59:20.642890    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/43034ce3-0ea1-4aef-826d-b84c1e14675f-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "43034ce3-0ea1-4aef-826d-b84c1e14675f" (UID: "43034ce3-0ea1-4aef-826d-b84c1e14675f"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 17:59:20.744041 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:59:20.744013    2581 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/43034ce3-0ea1-4aef-826d-b84c1e14675f-maas-api-service-ca\") on node \"ip-10-0-141-180.ec2.internal\" DevicePath \"\""
Sep 10 17:59:21.109590 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:59:21.109501    2581 scope.go:117] "RemoveContainer" containerID="81e88b8cea6194560b7c059d7b328b127d40ac95277169faff5e475e9dba176c"
Sep 10 17:59:21.118691 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:59:21.118671    2581 scope.go:117] "RemoveContainer" containerID="2d74a641ab858634fdd44da7c4223c98256496a7ad9745e8889f52c4f600cba1"
Sep 10 17:59:21.126707 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:59:21.126685    2581 scope.go:117] "RemoveContainer" containerID="2dfa6189317915e2c311f129f578bcaf81268a7cc0931fe5cc8dfb4adf0fa714"
Sep 10 17:59:21.406845 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:59:21.406759    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-abad1a-g2zrl"
Sep 10 17:59:21.406845 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:59:21.406758    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-abad1a-g2zrl" event={"ID":"43034ce3-0ea1-4aef-826d-b84c1e14675f","Type":"ContainerDied","Data":"06a9d600ff6efde02007f9bc5e8d3b556242035d8ffb9d87c818792013f99ff6"}
Sep 10 17:59:21.407063 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:59:21.406848    2581 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="06a9d600ff6efde02007f9bc5e8d3b556242035d8ffb9d87c818792013f99ff6"
Sep 10 17:59:33.341628 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:59:33.341594    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-b-abad1a-768d8ff85d-c2xd2"]
Sep 10 17:59:33.344300 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:59:33.341860    2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-api-b-abad1a-768d8ff85d-c2xd2" podUID="66559533-80d3-4598-a778-a507fa1938ec" containerName="payload-pre-processing" containerID="cri-o://16113b77a5581c31ecc0c4c228ae8f4497b6bbea172584dffb8d1e80cad9f688" gracePeriod=30
Sep 10 17:59:33.492625 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:59:33.492585    2581 generic.go:356] "Generic (PLEG): container finished" podID="66559533-80d3-4598-a778-a507fa1938ec" containerID="16113b77a5581c31ecc0c4c228ae8f4497b6bbea172584dffb8d1e80cad9f688" exitCode=0
Sep 10 17:59:33.492804 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:59:33.492661    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-b-abad1a-768d8ff85d-c2xd2" event={"ID":"66559533-80d3-4598-a778-a507fa1938ec","Type":"ContainerDied","Data":"16113b77a5581c31ecc0c4c228ae8f4497b6bbea172584dffb8d1e80cad9f688"}
Sep 10 17:59:33.513735 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:59:33.513699    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-abad1a-g2zrl"]
Sep 10 17:59:33.521672 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:59:33.521626    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-abad1a-g2zrl"]
Sep 10 17:59:33.607422 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:59:33.607366    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-b-abad1a-768d8ff85d-c2xd2"
Sep 10 17:59:33.668109 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:59:33.668073    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-z8bb6\" (UniqueName: \"kubernetes.io/projected/66559533-80d3-4598-a778-a507fa1938ec-kube-api-access-z8bb6\") pod \"66559533-80d3-4598-a778-a507fa1938ec\" (UID: \"66559533-80d3-4598-a778-a507fa1938ec\") "
Sep 10 17:59:33.668306 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:59:33.668213    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/66559533-80d3-4598-a778-a507fa1938ec-plugins-config-volume\") pod \"66559533-80d3-4598-a778-a507fa1938ec\" (UID: \"66559533-80d3-4598-a778-a507fa1938ec\") "
Sep 10 17:59:33.669154 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:59:33.669121    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/66559533-80d3-4598-a778-a507fa1938ec-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "66559533-80d3-4598-a778-a507fa1938ec" (UID: "66559533-80d3-4598-a778-a507fa1938ec"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 17:59:33.677201 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:59:33.677149    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/66559533-80d3-4598-a778-a507fa1938ec-kube-api-access-z8bb6" (OuterVolumeSpecName: "kube-api-access-z8bb6") pod "66559533-80d3-4598-a778-a507fa1938ec" (UID: "66559533-80d3-4598-a778-a507fa1938ec"). InnerVolumeSpecName "kube-api-access-z8bb6". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 17:59:33.769049 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:59:33.769010    2581 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/66559533-80d3-4598-a778-a507fa1938ec-plugins-config-volume\") on node \"ip-10-0-141-180.ec2.internal\" DevicePath \"\""
Sep 10 17:59:33.769049 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:59:33.769041    2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-z8bb6\" (UniqueName: \"kubernetes.io/projected/66559533-80d3-4598-a778-a507fa1938ec-kube-api-access-z8bb6\") on node \"ip-10-0-141-180.ec2.internal\" DevicePath \"\""
Sep 10 17:59:34.499974 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:59:34.499940    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-b-abad1a-768d8ff85d-c2xd2" event={"ID":"66559533-80d3-4598-a778-a507fa1938ec","Type":"ContainerDied","Data":"6ee6b6d659cdbcd14b5be431930939e6247af04ffae0a0b41506fd069f4e2474"}
Sep 10 17:59:34.500435 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:59:34.499983    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-b-abad1a-768d8ff85d-c2xd2"
Sep 10 17:59:34.500435 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:59:34.500008    2581 scope.go:117] "RemoveContainer" containerID="16113b77a5581c31ecc0c4c228ae8f4497b6bbea172584dffb8d1e80cad9f688"
Sep 10 17:59:34.540231 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:59:34.540199    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-b-abad1a-768d8ff85d-c2xd2"]
Sep 10 17:59:34.550544 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:59:34.550509    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-b-abad1a-768d8ff85d-c2xd2"]
Sep 10 17:59:34.862383 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:59:34.862303    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="43034ce3-0ea1-4aef-826d-b84c1e14675f" path="/var/lib/kubelet/pods/43034ce3-0ea1-4aef-826d-b84c1e14675f/volumes"
Sep 10 17:59:34.862730 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:59:34.862716    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="66559533-80d3-4598-a778-a507fa1938ec" path="/var/lib/kubelet/pods/66559533-80d3-4598-a778-a507fa1938ec/volumes"
Sep 10 17:59:40.575633 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:59:40.575596    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-9391d2-n7fnq"]
Sep 10 17:59:40.576171 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:59:40.576097    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="66559533-80d3-4598-a778-a507fa1938ec" containerName="payload-pre-processing"
Sep 10 17:59:40.576171 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:59:40.576110    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="66559533-80d3-4598-a778-a507fa1938ec" containerName="payload-pre-processing"
Sep 10 17:59:40.576171 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:59:40.576131    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="43034ce3-0ea1-4aef-826d-b84c1e14675f" containerName="revoke-keys"
Sep 10 17:59:40.576171 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:59:40.576137    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="43034ce3-0ea1-4aef-826d-b84c1e14675f" containerName="revoke-keys"
Sep 10 17:59:40.576393 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:59:40.576204    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="66559533-80d3-4598-a778-a507fa1938ec" containerName="payload-pre-processing"
Sep 10 17:59:40.576393 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:59:40.576214    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="43034ce3-0ea1-4aef-826d-b84c1e14675f" containerName="revoke-keys"
Sep 10 17:59:40.580750 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:59:40.580724    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-9391d2-n7fnq"
Sep 10 17:59:40.583343 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:59:40.583307    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 10 17:59:40.583479 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:59:40.583348    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-k7zj6\""
Sep 10 17:59:40.604404 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:59:40.604376    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-9391d2-n7fnq"]
Sep 10 17:59:40.633449 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:59:40.633417    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/426735f0-0b2e-41c9-bb6b-a84c4697f8eb-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-a-w2-9391d2-n7fnq\" (UID: \"426735f0-0b2e-41c9-bb6b-a84c4697f8eb\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-9391d2-n7fnq"
Sep 10 17:59:40.734448 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:59:40.734411    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/426735f0-0b2e-41c9-bb6b-a84c4697f8eb-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-a-w2-9391d2-n7fnq\" (UID: \"426735f0-0b2e-41c9-bb6b-a84c4697f8eb\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-9391d2-n7fnq"
Sep 10 17:59:40.735109 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:59:40.735087    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/426735f0-0b2e-41c9-bb6b-a84c4697f8eb-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-a-w2-9391d2-n7fnq\" (UID: \"426735f0-0b2e-41c9-bb6b-a84c4697f8eb\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-9391d2-n7fnq"
Sep 10 17:59:40.898209 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:59:40.898103    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-9391d2-n7fnq"
Sep 10 17:59:41.175901 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:59:41.175869    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-9391d2-n7fnq"]
Sep 10 17:59:41.176335 ip-10-0-141-180 kubenswrapper[2581]: W0910 17:59:41.176305    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod426735f0_0b2e_41c9_bb6b_a84c4697f8eb.slice/crio-855232bd48c38bb1bf40272d0516d95b9132439929186fd7175b04e76b56a742 WatchSource:0}: Error finding container 855232bd48c38bb1bf40272d0516d95b9132439929186fd7175b04e76b56a742: Status 404 returned error can't find the container with id 855232bd48c38bb1bf40272d0516d95b9132439929186fd7175b04e76b56a742
Sep 10 17:59:41.545421 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:59:41.545379    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-9391d2-n7fnq" event={"ID":"426735f0-0b2e-41c9-bb6b-a84c4697f8eb","Type":"ContainerStarted","Data":"9c0bfb63cf340c784f4ed4cea2d6cb86d84eb485578449ef3f08cc3f4f795e7c"}
Sep 10 17:59:41.545585 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:59:41.545435    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-9391d2-n7fnq" event={"ID":"426735f0-0b2e-41c9-bb6b-a84c4697f8eb","Type":"ContainerStarted","Data":"855232bd48c38bb1bf40272d0516d95b9132439929186fd7175b04e76b56a742"}
Sep 10 17:59:41.572909 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:59:41.572867    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-9391d2-n7fnq" podStartSLOduration=1.572853459 podStartE2EDuration="1.572853459s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 17:59:40 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 17:59:41.571274166 +0000 UTC m=+1401.225348090" watchObservedRunningTime="2026-09-10 17:59:41.572853459 +0000 UTC m=+1401.226927393"
Sep 10 17:59:42.554779 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:59:42.554747    2581 generic.go:356] "Generic (PLEG): container finished" podID="426735f0-0b2e-41c9-bb6b-a84c4697f8eb" containerID="9c0bfb63cf340c784f4ed4cea2d6cb86d84eb485578449ef3f08cc3f4f795e7c" exitCode=0
Sep 10 17:59:42.555276 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:59:42.554831    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-9391d2-n7fnq" event={"ID":"426735f0-0b2e-41c9-bb6b-a84c4697f8eb","Type":"ContainerDied","Data":"9c0bfb63cf340c784f4ed4cea2d6cb86d84eb485578449ef3f08cc3f4f795e7c"}
Sep 10 17:59:43.691575 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:59:43.691554    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-9391d2-n7fnq"
Sep 10 17:59:43.765842 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:59:43.765807    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/426735f0-0b2e-41c9-bb6b-a84c4697f8eb-maas-api-service-ca\") pod \"426735f0-0b2e-41c9-bb6b-a84c4697f8eb\" (UID: \"426735f0-0b2e-41c9-bb6b-a84c4697f8eb\") "
Sep 10 17:59:43.766149 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:59:43.766128    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/426735f0-0b2e-41c9-bb6b-a84c4697f8eb-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "426735f0-0b2e-41c9-bb6b-a84c4697f8eb" (UID: "426735f0-0b2e-41c9-bb6b-a84c4697f8eb"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 17:59:43.866992 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:59:43.866904    2581 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/426735f0-0b2e-41c9-bb6b-a84c4697f8eb-maas-api-service-ca\") on node \"ip-10-0-141-180.ec2.internal\" DevicePath \"\""
Sep 10 17:59:44.568707 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:59:44.568659    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-9391d2-n7fnq" event={"ID":"426735f0-0b2e-41c9-bb6b-a84c4697f8eb","Type":"ContainerDied","Data":"855232bd48c38bb1bf40272d0516d95b9132439929186fd7175b04e76b56a742"}
Sep 10 17:59:44.568707 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:59:44.568698    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-9391d2-n7fnq"
Sep 10 17:59:44.568920 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:59:44.568759    2581 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="855232bd48c38bb1bf40272d0516d95b9132439929186fd7175b04e76b56a742"
Sep 10 17:59:48.881654 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:59:48.881607    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-a-904bbd-6b476dbd9f-bwbmg"]
Sep 10 17:59:48.882092 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:59:48.882078    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="426735f0-0b2e-41c9-bb6b-a84c4697f8eb" containerName="revoke-keys"
Sep 10 17:59:48.882092 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:59:48.882092    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="426735f0-0b2e-41c9-bb6b-a84c4697f8eb" containerName="revoke-keys"
Sep 10 17:59:48.882173 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:59:48.882159    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="426735f0-0b2e-41c9-bb6b-a84c4697f8eb" containerName="revoke-keys"
Sep 10 17:59:48.885321 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:59:48.885303    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-a-904bbd-6b476dbd9f-bwbmg"
Sep 10 17:59:48.887867 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:59:48.887845    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-ipp-a-904bbd\""
Sep 10 17:59:48.887946 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:59:48.887910    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-ipp-a-904bbd-dockercfg-b5n5r\""
Sep 10 17:59:48.897917 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:59:48.897896    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-a-904bbd-6b476dbd9f-bwbmg"]
Sep 10 17:59:49.025892 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:59:49.025860    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-b62h4\" (UniqueName: \"kubernetes.io/projected/26b1dfe3-1c5f-4387-80d6-5f9e583261d0-kube-api-access-b62h4\") pod \"payload-processing-e2e-ipp-a-904bbd-6b476dbd9f-bwbmg\" (UID: \"26b1dfe3-1c5f-4387-80d6-5f9e583261d0\") " pod="openshift-ingress/payload-processing-e2e-ipp-a-904bbd-6b476dbd9f-bwbmg"
Sep 10 17:59:49.026057 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:59:49.025994    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/26b1dfe3-1c5f-4387-80d6-5f9e583261d0-plugins-config-volume\") pod \"payload-processing-e2e-ipp-a-904bbd-6b476dbd9f-bwbmg\" (UID: \"26b1dfe3-1c5f-4387-80d6-5f9e583261d0\") " pod="openshift-ingress/payload-processing-e2e-ipp-a-904bbd-6b476dbd9f-bwbmg"
Sep 10 17:59:49.127170 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:59:49.127131    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/26b1dfe3-1c5f-4387-80d6-5f9e583261d0-plugins-config-volume\") pod \"payload-processing-e2e-ipp-a-904bbd-6b476dbd9f-bwbmg\" (UID: \"26b1dfe3-1c5f-4387-80d6-5f9e583261d0\") " pod="openshift-ingress/payload-processing-e2e-ipp-a-904bbd-6b476dbd9f-bwbmg"
Sep 10 17:59:49.127365 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:59:49.127185    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-b62h4\" (UniqueName: \"kubernetes.io/projected/26b1dfe3-1c5f-4387-80d6-5f9e583261d0-kube-api-access-b62h4\") pod \"payload-processing-e2e-ipp-a-904bbd-6b476dbd9f-bwbmg\" (UID: \"26b1dfe3-1c5f-4387-80d6-5f9e583261d0\") " pod="openshift-ingress/payload-processing-e2e-ipp-a-904bbd-6b476dbd9f-bwbmg"
Sep 10 17:59:49.127947 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:59:49.127923    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/26b1dfe3-1c5f-4387-80d6-5f9e583261d0-plugins-config-volume\") pod \"payload-processing-e2e-ipp-a-904bbd-6b476dbd9f-bwbmg\" (UID: \"26b1dfe3-1c5f-4387-80d6-5f9e583261d0\") " pod="openshift-ingress/payload-processing-e2e-ipp-a-904bbd-6b476dbd9f-bwbmg"
Sep 10 17:59:49.135847 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:59:49.135791    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-b62h4\" (UniqueName: \"kubernetes.io/projected/26b1dfe3-1c5f-4387-80d6-5f9e583261d0-kube-api-access-b62h4\") pod \"payload-processing-e2e-ipp-a-904bbd-6b476dbd9f-bwbmg\" (UID: \"26b1dfe3-1c5f-4387-80d6-5f9e583261d0\") " pod="openshift-ingress/payload-processing-e2e-ipp-a-904bbd-6b476dbd9f-bwbmg"
Sep 10 17:59:49.195729 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:59:49.195695    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-a-904bbd-6b476dbd9f-bwbmg"
Sep 10 17:59:49.518606 ip-10-0-141-180 kubenswrapper[2581]: W0910 17:59:49.518577    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod26b1dfe3_1c5f_4387_80d6_5f9e583261d0.slice/crio-007335568fcc8523d109d5041b2f072973b8cb20f22cb1f39fa7f5dd93adf976 WatchSource:0}: Error finding container 007335568fcc8523d109d5041b2f072973b8cb20f22cb1f39fa7f5dd93adf976: Status 404 returned error can't find the container with id 007335568fcc8523d109d5041b2f072973b8cb20f22cb1f39fa7f5dd93adf976
Sep 10 17:59:49.532587 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:59:49.531832    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-a-904bbd-6b476dbd9f-bwbmg"]
Sep 10 17:59:49.601482 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:59:49.601451    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-a-904bbd-6b476dbd9f-bwbmg" event={"ID":"26b1dfe3-1c5f-4387-80d6-5f9e583261d0","Type":"ContainerStarted","Data":"30d463fba5f97d07b1d48cc5211a39645887a9805e77c13dcdf3632be4e43575"}
Sep 10 17:59:49.601582 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:59:49.601508    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-a-904bbd-6b476dbd9f-bwbmg" event={"ID":"26b1dfe3-1c5f-4387-80d6-5f9e583261d0","Type":"ContainerStarted","Data":"007335568fcc8523d109d5041b2f072973b8cb20f22cb1f39fa7f5dd93adf976"}
Sep 10 17:59:49.601582 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:59:49.601560    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-ipp-a-904bbd-6b476dbd9f-bwbmg"
Sep 10 17:59:49.681676 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:59:49.681592    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-ipp-a-904bbd-6b476dbd9f-bwbmg" podStartSLOduration=1.681568237 podStartE2EDuration="1.681568237s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 17:59:48 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 17:59:49.678015937 +0000 UTC m=+1409.332089885" watchObservedRunningTime="2026-09-10 17:59:49.681568237 +0000 UTC m=+1409.335642142"
Sep 10 17:59:52.231342 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:59:52.231310    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w2-9391d2-569f5888bf-qqks8"]
Sep 10 17:59:52.231818 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:59:52.231507    2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-shared-a-w2-9391d2-569f5888bf-qqks8" podUID="58758048-0e64-4870-87cf-b95eb28c72e7" containerName="payload-processing" containerID="cri-o://bdfe2ffa86b505261d17f79563bda3d27e1266b081d7c3b574d8a49e4fde3863" gracePeriod=30
Sep 10 17:59:52.242209 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:59:52.242181    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w2-9391d2-66bd85c6c8-qwwg4"]
Sep 10 17:59:52.242470 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:59:52.242409    2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-9391d2-66bd85c6c8-qwwg4" podUID="b259c38c-d3fd-4f18-924c-fbbaf62d3624" containerName="payload-pre-processing" containerID="cri-o://6ab60620d47614239e36836204e6b9538acf9a2a2e7572ba2661c7dac6effd6c" gracePeriod=30
Sep 10 17:59:52.432457 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:59:52.432418    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-9391d2-n7fnq"]
Sep 10 17:59:52.439356 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:59:52.439330    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-9391d2-n7fnq"]
Sep 10 17:59:52.500755 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:59:52.500729    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-a-w2-9391d2-569f5888bf-qqks8"
Sep 10 17:59:52.536241 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:59:52.536217    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-9391d2-66bd85c6c8-qwwg4"
Sep 10 17:59:52.566810 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:59:52.564781    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-g7zsw\" (UniqueName: \"kubernetes.io/projected/58758048-0e64-4870-87cf-b95eb28c72e7-kube-api-access-g7zsw\") pod \"58758048-0e64-4870-87cf-b95eb28c72e7\" (UID: \"58758048-0e64-4870-87cf-b95eb28c72e7\") "
Sep 10 17:59:52.566810 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:59:52.564931    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/58758048-0e64-4870-87cf-b95eb28c72e7-plugins-config-volume\") pod \"58758048-0e64-4870-87cf-b95eb28c72e7\" (UID: \"58758048-0e64-4870-87cf-b95eb28c72e7\") "
Sep 10 17:59:52.566810 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:59:52.565841    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/58758048-0e64-4870-87cf-b95eb28c72e7-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "58758048-0e64-4870-87cf-b95eb28c72e7" (UID: "58758048-0e64-4870-87cf-b95eb28c72e7"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 17:59:52.580621 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:59:52.568784    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/58758048-0e64-4870-87cf-b95eb28c72e7-kube-api-access-g7zsw" (OuterVolumeSpecName: "kube-api-access-g7zsw") pod "58758048-0e64-4870-87cf-b95eb28c72e7" (UID: "58758048-0e64-4870-87cf-b95eb28c72e7"). InnerVolumeSpecName "kube-api-access-g7zsw". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 17:59:52.626437 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:59:52.626394    2581 generic.go:356] "Generic (PLEG): container finished" podID="b259c38c-d3fd-4f18-924c-fbbaf62d3624" containerID="6ab60620d47614239e36836204e6b9538acf9a2a2e7572ba2661c7dac6effd6c" exitCode=0
Sep 10 17:59:52.626629 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:59:52.626456    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-9391d2-66bd85c6c8-qwwg4"
Sep 10 17:59:52.626629 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:59:52.626488    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-9391d2-66bd85c6c8-qwwg4" event={"ID":"b259c38c-d3fd-4f18-924c-fbbaf62d3624","Type":"ContainerDied","Data":"6ab60620d47614239e36836204e6b9538acf9a2a2e7572ba2661c7dac6effd6c"}
Sep 10 17:59:52.626629 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:59:52.626591    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-9391d2-66bd85c6c8-qwwg4" event={"ID":"b259c38c-d3fd-4f18-924c-fbbaf62d3624","Type":"ContainerDied","Data":"abc1051c6b9c0629d25eb4abbe1dde23beb8ff8326fba18003c69145a2e14c4a"}
Sep 10 17:59:52.626629 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:59:52.626612    2581 scope.go:117] "RemoveContainer" containerID="6ab60620d47614239e36836204e6b9538acf9a2a2e7572ba2661c7dac6effd6c"
Sep 10 17:59:52.628026 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:59:52.627999    2581 generic.go:356] "Generic (PLEG): container finished" podID="58758048-0e64-4870-87cf-b95eb28c72e7" containerID="bdfe2ffa86b505261d17f79563bda3d27e1266b081d7c3b574d8a49e4fde3863" exitCode=0
Sep 10 17:59:52.628148 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:59:52.628053    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-a-w2-9391d2-569f5888bf-qqks8"
Sep 10 17:59:52.628148 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:59:52.628063    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-a-w2-9391d2-569f5888bf-qqks8" event={"ID":"58758048-0e64-4870-87cf-b95eb28c72e7","Type":"ContainerDied","Data":"bdfe2ffa86b505261d17f79563bda3d27e1266b081d7c3b574d8a49e4fde3863"}
Sep 10 17:59:52.628148 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:59:52.628091    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-a-w2-9391d2-569f5888bf-qqks8" event={"ID":"58758048-0e64-4870-87cf-b95eb28c72e7","Type":"ContainerDied","Data":"4b3dd3595ebf9ca0410ffcb7c37c641451dc9a98ca5a9aa3986b5639c47977e1"}
Sep 10 17:59:52.636275 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:59:52.636258    2581 scope.go:117] "RemoveContainer" containerID="6ab60620d47614239e36836204e6b9538acf9a2a2e7572ba2661c7dac6effd6c"
Sep 10 17:59:52.636555 ip-10-0-141-180 kubenswrapper[2581]: E0910 17:59:52.636534    2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6ab60620d47614239e36836204e6b9538acf9a2a2e7572ba2661c7dac6effd6c\": container with ID starting with 6ab60620d47614239e36836204e6b9538acf9a2a2e7572ba2661c7dac6effd6c not found: ID does not exist" containerID="6ab60620d47614239e36836204e6b9538acf9a2a2e7572ba2661c7dac6effd6c"
Sep 10 17:59:52.636612 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:59:52.636563    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6ab60620d47614239e36836204e6b9538acf9a2a2e7572ba2661c7dac6effd6c"} err="failed to get container status \"6ab60620d47614239e36836204e6b9538acf9a2a2e7572ba2661c7dac6effd6c\": rpc error: code = NotFound desc = could not find container \"6ab60620d47614239e36836204e6b9538acf9a2a2e7572ba2661c7dac6effd6c\": container with ID starting with 6ab60620d47614239e36836204e6b9538acf9a2a2e7572ba2661c7dac6effd6c not found: ID does not exist"
Sep 10 17:59:52.636612 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:59:52.636579    2581 scope.go:117] "RemoveContainer" containerID="bdfe2ffa86b505261d17f79563bda3d27e1266b081d7c3b574d8a49e4fde3863"
Sep 10 17:59:52.645336 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:59:52.645319    2581 scope.go:117] "RemoveContainer" containerID="bdfe2ffa86b505261d17f79563bda3d27e1266b081d7c3b574d8a49e4fde3863"
Sep 10 17:59:52.645584 ip-10-0-141-180 kubenswrapper[2581]: E0910 17:59:52.645567    2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"bdfe2ffa86b505261d17f79563bda3d27e1266b081d7c3b574d8a49e4fde3863\": container with ID starting with bdfe2ffa86b505261d17f79563bda3d27e1266b081d7c3b574d8a49e4fde3863 not found: ID does not exist" containerID="bdfe2ffa86b505261d17f79563bda3d27e1266b081d7c3b574d8a49e4fde3863"
Sep 10 17:59:52.645626 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:59:52.645590    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"bdfe2ffa86b505261d17f79563bda3d27e1266b081d7c3b574d8a49e4fde3863"} err="failed to get container status \"bdfe2ffa86b505261d17f79563bda3d27e1266b081d7c3b574d8a49e4fde3863\": rpc error: code = NotFound desc = could not find container \"bdfe2ffa86b505261d17f79563bda3d27e1266b081d7c3b574d8a49e4fde3863\": container with ID starting with bdfe2ffa86b505261d17f79563bda3d27e1266b081d7c3b574d8a49e4fde3863 not found: ID does not exist"
Sep 10 17:59:52.663152 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:59:52.663131    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w2-9391d2-569f5888bf-qqks8"]
Sep 10 17:59:52.666122 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:59:52.666104    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b259c38c-d3fd-4f18-924c-fbbaf62d3624-plugins-config-volume\") pod \"b259c38c-d3fd-4f18-924c-fbbaf62d3624\" (UID: \"b259c38c-d3fd-4f18-924c-fbbaf62d3624\") "
Sep 10 17:59:52.666193 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:59:52.666146    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-k2jdx\" (UniqueName: \"kubernetes.io/projected/b259c38c-d3fd-4f18-924c-fbbaf62d3624-kube-api-access-k2jdx\") pod \"b259c38c-d3fd-4f18-924c-fbbaf62d3624\" (UID: \"b259c38c-d3fd-4f18-924c-fbbaf62d3624\") "
Sep 10 17:59:52.666411 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:59:52.666391    2581 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/58758048-0e64-4870-87cf-b95eb28c72e7-plugins-config-volume\") on node \"ip-10-0-141-180.ec2.internal\" DevicePath \"\""
Sep 10 17:59:52.666411 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:59:52.666410    2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-g7zsw\" (UniqueName: \"kubernetes.io/projected/58758048-0e64-4870-87cf-b95eb28c72e7-kube-api-access-g7zsw\") on node \"ip-10-0-141-180.ec2.internal\" DevicePath \"\""
Sep 10 17:59:52.666549 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:59:52.666428    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b259c38c-d3fd-4f18-924c-fbbaf62d3624-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "b259c38c-d3fd-4f18-924c-fbbaf62d3624" (UID: "b259c38c-d3fd-4f18-924c-fbbaf62d3624"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 17:59:52.668049 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:59:52.668029    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b259c38c-d3fd-4f18-924c-fbbaf62d3624-kube-api-access-k2jdx" (OuterVolumeSpecName: "kube-api-access-k2jdx") pod "b259c38c-d3fd-4f18-924c-fbbaf62d3624" (UID: "b259c38c-d3fd-4f18-924c-fbbaf62d3624"). InnerVolumeSpecName "kube-api-access-k2jdx". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 17:59:52.675242 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:59:52.675218    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w2-9391d2-569f5888bf-qqks8"]
Sep 10 17:59:52.767727 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:59:52.767632    2581 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b259c38c-d3fd-4f18-924c-fbbaf62d3624-plugins-config-volume\") on node \"ip-10-0-141-180.ec2.internal\" DevicePath \"\""
Sep 10 17:59:52.767727 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:59:52.767683    2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-k2jdx\" (UniqueName: \"kubernetes.io/projected/b259c38c-d3fd-4f18-924c-fbbaf62d3624-kube-api-access-k2jdx\") on node \"ip-10-0-141-180.ec2.internal\" DevicePath \"\""
Sep 10 17:59:52.862492 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:59:52.862465    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="426735f0-0b2e-41c9-bb6b-a84c4697f8eb" path="/var/lib/kubelet/pods/426735f0-0b2e-41c9-bb6b-a84c4697f8eb/volumes"
Sep 10 17:59:52.862901 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:59:52.862885    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="58758048-0e64-4870-87cf-b95eb28c72e7" path="/var/lib/kubelet/pods/58758048-0e64-4870-87cf-b95eb28c72e7/volumes"
Sep 10 17:59:52.945146 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:59:52.945118    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w2-9391d2-66bd85c6c8-qwwg4"]
Sep 10 17:59:52.949936 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:59:52.949914    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w2-9391d2-66bd85c6c8-qwwg4"]
Sep 10 17:59:54.862543 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:59:54.862500    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b259c38c-d3fd-4f18-924c-fbbaf62d3624" path="/var/lib/kubelet/pods/b259c38c-d3fd-4f18-924c-fbbaf62d3624/volumes"
Sep 10 17:59:59.053311 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:59:59.053281    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-a-w2-9391d2-openshift-default-57c888f898-dwd2h"]
Sep 10 17:59:59.053757 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:59:59.053480    2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-shared-a-w2-9391d2-openshift-default-57c888f898-dwd2h" podUID="d406cdfe-2228-4c17-ba8f-01a86277b666" containerName="istio-proxy" containerID="cri-o://dbebc011dddf30407b12734364f786768c1eca83a4331bfa7362eda1f5f87a6a" gracePeriod=30
Sep 10 17:59:59.355010 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:59:59.354926    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-bdf47c-wvzvs"]
Sep 10 17:59:59.355686 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:59:59.355665    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b259c38c-d3fd-4f18-924c-fbbaf62d3624" containerName="payload-pre-processing"
Sep 10 17:59:59.355742 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:59:59.355689    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="b259c38c-d3fd-4f18-924c-fbbaf62d3624" containerName="payload-pre-processing"
Sep 10 17:59:59.355742 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:59:59.355699    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="58758048-0e64-4870-87cf-b95eb28c72e7" containerName="payload-processing"
Sep 10 17:59:59.355742 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:59:59.355708    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="58758048-0e64-4870-87cf-b95eb28c72e7" containerName="payload-processing"
Sep 10 17:59:59.355862 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:59:59.355850    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="b259c38c-d3fd-4f18-924c-fbbaf62d3624" containerName="payload-pre-processing"
Sep 10 17:59:59.355895 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:59:59.355868    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="58758048-0e64-4870-87cf-b95eb28c72e7" containerName="payload-processing"
Sep 10 17:59:59.361401 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:59:59.361379    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-bdf47c-wvzvs"
Sep 10 17:59:59.364865 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:59:59.364838    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 10 17:59:59.365036 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:59:59.364839    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-k7zj6\""
Sep 10 17:59:59.369095 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:59:59.369073    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-bdf47c-wvzvs"]
Sep 10 17:59:59.541824 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:59:59.541772    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/5bbc8900-14fd-4cfe-a2ed-bd8cf1c33a4b-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-b-w2-bdf47c-wvzvs\" (UID: \"5bbc8900-14fd-4cfe-a2ed-bd8cf1c33a4b\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-bdf47c-wvzvs"
Sep 10 17:59:59.642972 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:59:59.642869    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/5bbc8900-14fd-4cfe-a2ed-bd8cf1c33a4b-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-b-w2-bdf47c-wvzvs\" (UID: \"5bbc8900-14fd-4cfe-a2ed-bd8cf1c33a4b\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-bdf47c-wvzvs"
Sep 10 17:59:59.643479 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:59:59.643456    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/5bbc8900-14fd-4cfe-a2ed-bd8cf1c33a4b-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-b-w2-bdf47c-wvzvs\" (UID: \"5bbc8900-14fd-4cfe-a2ed-bd8cf1c33a4b\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-bdf47c-wvzvs"
Sep 10 17:59:59.673259 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:59:59.673232    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-bdf47c-wvzvs"
Sep 10 17:59:59.961970 ip-10-0-141-180 kubenswrapper[2581]: I0910 17:59:59.961928    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-bdf47c-wvzvs"]
Sep 10 17:59:59.966950 ip-10-0-141-180 kubenswrapper[2581]: W0910 17:59:59.966925    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5bbc8900_14fd_4cfe_a2ed_bd8cf1c33a4b.slice/crio-cb03c72e5b3a9a8cb605f47b3cfbc8a60bcccb2fa9667f91530d1450be6dca37 WatchSource:0}: Error finding container cb03c72e5b3a9a8cb605f47b3cfbc8a60bcccb2fa9667f91530d1450be6dca37: Status 404 returned error can't find the container with id cb03c72e5b3a9a8cb605f47b3cfbc8a60bcccb2fa9667f91530d1450be6dca37
Sep 10 18:00:00.151383 ip-10-0-141-180 kubenswrapper[2581]: I0910 18:00:00.151350    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w2-bdf47c-29817720-f6v44"]
Sep 10 18:00:00.156178 ip-10-0-141-180 kubenswrapper[2581]: I0910 18:00:00.156112    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w2-bdf47c-29817720-f6v44"
Sep 10 18:00:00.159616 ip-10-0-141-180 kubenswrapper[2581]: I0910 18:00:00.159597    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w3-57474d-29817720-69zzz"]
Sep 10 18:00:00.163058 ip-10-0-141-180 kubenswrapper[2581]: I0910 18:00:00.163037    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ipp-a-904bbd-29817720-vg6ts"]
Sep 10 18:00:00.163175 ip-10-0-141-180 kubenswrapper[2581]: I0910 18:00:00.163166    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w3-57474d-29817720-69zzz"
Sep 10 18:00:00.167162 ip-10-0-141-180 kubenswrapper[2581]: I0910 18:00:00.167138    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ipp-a-904bbd-29817720-vg6ts"
Sep 10 18:00:00.169010 ip-10-0-141-180 kubenswrapper[2581]: I0910 18:00:00.168991    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w2-bdf47c-29817720-f6v44"
Sep 10 18:00:00.169790 ip-10-0-141-180 kubenswrapper[2581]: I0910 18:00:00.169766    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w2-bdf47c-29817720-f6v44"]
Sep 10 18:00:00.181282 ip-10-0-141-180 kubenswrapper[2581]: I0910 18:00:00.181259    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ipp-a-904bbd-29817720-vg6ts"]
Sep 10 18:00:00.188861 ip-10-0-141-180 kubenswrapper[2581]: I0910 18:00:00.188839    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w3-57474d-29817720-69zzz"]
Sep 10 18:00:00.203470 ip-10-0-141-180 kubenswrapper[2581]: I0910 18:00:00.203445    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w3-57474d-29817720-69zzz"
Sep 10 18:00:00.212043 ip-10-0-141-180 kubenswrapper[2581]: I0910 18:00:00.211976    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ipp-a-904bbd-29817720-vg6ts"
Sep 10 18:00:00.491406 ip-10-0-141-180 kubenswrapper[2581]: W0910 18:00:00.491379    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podafac154b_2857_447a_a333_5844bb606210.slice/crio-b8ad734b3a7995b463981ee052a1a167fdca85b00ad67d3c0d3e4e1ae8764dfb WatchSource:0}: Error finding container b8ad734b3a7995b463981ee052a1a167fdca85b00ad67d3c0d3e4e1ae8764dfb: Status 404 returned error can't find the container with id b8ad734b3a7995b463981ee052a1a167fdca85b00ad67d3c0d3e4e1ae8764dfb
Sep 10 18:00:00.491863 ip-10-0-141-180 kubenswrapper[2581]: I0910 18:00:00.491842    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w2-bdf47c-29817720-f6v44"]
Sep 10 18:00:00.529314 ip-10-0-141-180 kubenswrapper[2581]: I0910 18:00:00.529293    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w3-57474d-29817720-69zzz"]
Sep 10 18:00:00.530487 ip-10-0-141-180 kubenswrapper[2581]: W0910 18:00:00.530463    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod81eeb254_adb2_42c9_9318_44a17603c3e3.slice/crio-74072576df037f739d3831287922912fc67e1146a9825ddd2f9a0ec5588ffcc0 WatchSource:0}: Error finding container 74072576df037f739d3831287922912fc67e1146a9825ddd2f9a0ec5588ffcc0: Status 404 returned error can't find the container with id 74072576df037f739d3831287922912fc67e1146a9825ddd2f9a0ec5588ffcc0
Sep 10 18:00:00.535862 ip-10-0-141-180 kubenswrapper[2581]: I0910 18:00:00.535838    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ipp-a-904bbd-29817720-vg6ts"]
Sep 10 18:00:00.536376 ip-10-0-141-180 kubenswrapper[2581]: W0910 18:00:00.536357    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod060db639_fc7f_43d0_9650_9414951bca3b.slice/crio-56e51d9064aaf7caf537d5330dac1625c92b3878b1bc671452431f4122fbc2e2 WatchSource:0}: Error finding container 56e51d9064aaf7caf537d5330dac1625c92b3878b1bc671452431f4122fbc2e2: Status 404 returned error can't find the container with id 56e51d9064aaf7caf537d5330dac1625c92b3878b1bc671452431f4122fbc2e2
Sep 10 18:00:00.613423 ip-10-0-141-180 kubenswrapper[2581]: I0910 18:00:00.613395    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-ipp-a-904bbd-6b476dbd9f-bwbmg"
Sep 10 18:00:00.685116 ip-10-0-141-180 kubenswrapper[2581]: I0910 18:00:00.685080    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-bdf47c-wvzvs" event={"ID":"5bbc8900-14fd-4cfe-a2ed-bd8cf1c33a4b","Type":"ContainerStarted","Data":"30bd7eb7b2e94aa25427bb3d2048f34f0fbd8812f3d5b95e03f163f666fa8023"}
Sep 10 18:00:00.685262 ip-10-0-141-180 kubenswrapper[2581]: I0910 18:00:00.685147    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-bdf47c-wvzvs" event={"ID":"5bbc8900-14fd-4cfe-a2ed-bd8cf1c33a4b","Type":"ContainerStarted","Data":"cb03c72e5b3a9a8cb605f47b3cfbc8a60bcccb2fa9667f91530d1450be6dca37"}
Sep 10 18:00:00.686453 ip-10-0-141-180 kubenswrapper[2581]: I0910 18:00:00.686421    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w3-57474d-29817720-69zzz" event={"ID":"81eeb254-adb2-42c9-9318-44a17603c3e3","Type":"ContainerStarted","Data":"74072576df037f739d3831287922912fc67e1146a9825ddd2f9a0ec5588ffcc0"}
Sep 10 18:00:00.687815 ip-10-0-141-180 kubenswrapper[2581]: I0910 18:00:00.687788    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ipp-a-904bbd-29817720-vg6ts" event={"ID":"060db639-fc7f-43d0-9650-9414951bca3b","Type":"ContainerStarted","Data":"56e51d9064aaf7caf537d5330dac1625c92b3878b1bc671452431f4122fbc2e2"}
Sep 10 18:00:00.689061 ip-10-0-141-180 kubenswrapper[2581]: I0910 18:00:00.689035    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w2-bdf47c-29817720-f6v44" event={"ID":"afac154b-2857-447a-a333-5844bb606210","Type":"ContainerStarted","Data":"b8ad734b3a7995b463981ee052a1a167fdca85b00ad67d3c0d3e4e1ae8764dfb"}
Sep 10 18:00:00.706908 ip-10-0-141-180 kubenswrapper[2581]: I0910 18:00:00.706865    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-bdf47c-wvzvs" podStartSLOduration=1.706845932 podStartE2EDuration="1.706845932s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 17:59:59 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 18:00:00.703818296 +0000 UTC m=+1420.357892219" watchObservedRunningTime="2026-09-10 18:00:00.706845932 +0000 UTC m=+1420.360919839"
Sep 10 18:00:01.698982 ip-10-0-141-180 kubenswrapper[2581]: I0910 18:00:01.698945    2581 generic.go:356] "Generic (PLEG): container finished" podID="5bbc8900-14fd-4cfe-a2ed-bd8cf1c33a4b" containerID="30bd7eb7b2e94aa25427bb3d2048f34f0fbd8812f3d5b95e03f163f666fa8023" exitCode=0
Sep 10 18:00:01.699368 ip-10-0-141-180 kubenswrapper[2581]: I0910 18:00:01.699020    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-bdf47c-wvzvs" event={"ID":"5bbc8900-14fd-4cfe-a2ed-bd8cf1c33a4b","Type":"ContainerDied","Data":"30bd7eb7b2e94aa25427bb3d2048f34f0fbd8812f3d5b95e03f163f666fa8023"}
Sep 10 18:00:01.700702 ip-10-0-141-180 kubenswrapper[2581]: I0910 18:00:01.700670    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w3-57474d-29817720-69zzz" event={"ID":"81eeb254-adb2-42c9-9318-44a17603c3e3","Type":"ContainerStarted","Data":"bf4e9bf289309393d915dc8ed94018600727b19b5302763317a1ca021106bf40"}
Sep 10 18:00:01.702260 ip-10-0-141-180 kubenswrapper[2581]: I0910 18:00:01.702242    2581 generic.go:356] "Generic (PLEG): container finished" podID="060db639-fc7f-43d0-9650-9414951bca3b" containerID="db8c6e70341359a3becce12285dc933df13e5816acc76d4a4e6772225d21c79e" exitCode=0
Sep 10 18:00:01.702357 ip-10-0-141-180 kubenswrapper[2581]: I0910 18:00:01.702330    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ipp-a-904bbd-29817720-vg6ts" event={"ID":"060db639-fc7f-43d0-9650-9414951bca3b","Type":"ContainerDied","Data":"db8c6e70341359a3becce12285dc933df13e5816acc76d4a4e6772225d21c79e"}
Sep 10 18:00:01.703883 ip-10-0-141-180 kubenswrapper[2581]: I0910 18:00:01.703860    2581 generic.go:356] "Generic (PLEG): container finished" podID="afac154b-2857-447a-a333-5844bb606210" containerID="f50dcd5f88dce06c174176e9e0870a345f3c4a213f4c73b600aba88618bd39b7" exitCode=0
Sep 10 18:00:01.703972 ip-10-0-141-180 kubenswrapper[2581]: I0910 18:00:01.703923    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w2-bdf47c-29817720-f6v44" event={"ID":"afac154b-2857-447a-a333-5844bb606210","Type":"ContainerDied","Data":"f50dcd5f88dce06c174176e9e0870a345f3c4a213f4c73b600aba88618bd39b7"}
Sep 10 18:00:02.712313 ip-10-0-141-180 kubenswrapper[2581]: I0910 18:00:02.712278    2581 generic.go:356] "Generic (PLEG): container finished" podID="81eeb254-adb2-42c9-9318-44a17603c3e3" containerID="bf4e9bf289309393d915dc8ed94018600727b19b5302763317a1ca021106bf40" exitCode=0
Sep 10 18:00:02.712894 ip-10-0-141-180 kubenswrapper[2581]: I0910 18:00:02.712370    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w3-57474d-29817720-69zzz" event={"ID":"81eeb254-adb2-42c9-9318-44a17603c3e3","Type":"ContainerDied","Data":"bf4e9bf289309393d915dc8ed94018600727b19b5302763317a1ca021106bf40"}
Sep 10 18:00:02.869022 ip-10-0-141-180 kubenswrapper[2581]: I0910 18:00:02.868992    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ipp-a-904bbd-29817720-vg6ts"
Sep 10 18:00:02.914298 ip-10-0-141-180 kubenswrapper[2581]: I0910 18:00:02.914267    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-bdf47c-wvzvs"
Sep 10 18:00:02.946485 ip-10-0-141-180 kubenswrapper[2581]: I0910 18:00:02.946460    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w2-bdf47c-29817720-f6v44"
Sep 10 18:00:02.950097 ip-10-0-141-180 kubenswrapper[2581]: I0910 18:00:02.950067    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w3-57474d-29817720-69zzz"
Sep 10 18:00:03.086692 ip-10-0-141-180 kubenswrapper[2581]: I0910 18:00:03.086556    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/5bbc8900-14fd-4cfe-a2ed-bd8cf1c33a4b-maas-api-service-ca\") pod \"5bbc8900-14fd-4cfe-a2ed-bd8cf1c33a4b\" (UID: \"5bbc8900-14fd-4cfe-a2ed-bd8cf1c33a4b\") "
Sep 10 18:00:03.087006 ip-10-0-141-180 kubenswrapper[2581]: I0910 18:00:03.086977    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5bbc8900-14fd-4cfe-a2ed-bd8cf1c33a4b-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "5bbc8900-14fd-4cfe-a2ed-bd8cf1c33a4b" (UID: "5bbc8900-14fd-4cfe-a2ed-bd8cf1c33a4b"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 18:00:03.188181 ip-10-0-141-180 kubenswrapper[2581]: I0910 18:00:03.188145    2581 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/5bbc8900-14fd-4cfe-a2ed-bd8cf1c33a4b-maas-api-service-ca\") on node \"ip-10-0-141-180.ec2.internal\" DevicePath \"\""
Sep 10 18:00:03.720380 ip-10-0-141-180 kubenswrapper[2581]: I0910 18:00:03.720352    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w2-bdf47c-29817720-f6v44"
Sep 10 18:00:03.720866 ip-10-0-141-180 kubenswrapper[2581]: I0910 18:00:03.720391    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w2-bdf47c-29817720-f6v44" event={"ID":"afac154b-2857-447a-a333-5844bb606210","Type":"ContainerDied","Data":"b8ad734b3a7995b463981ee052a1a167fdca85b00ad67d3c0d3e4e1ae8764dfb"}
Sep 10 18:00:03.720866 ip-10-0-141-180 kubenswrapper[2581]: I0910 18:00:03.720441    2581 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="b8ad734b3a7995b463981ee052a1a167fdca85b00ad67d3c0d3e4e1ae8764dfb"
Sep 10 18:00:03.721829 ip-10-0-141-180 kubenswrapper[2581]: I0910 18:00:03.721808    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-bdf47c-wvzvs"
Sep 10 18:00:03.721829 ip-10-0-141-180 kubenswrapper[2581]: I0910 18:00:03.721817    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-bdf47c-wvzvs" event={"ID":"5bbc8900-14fd-4cfe-a2ed-bd8cf1c33a4b","Type":"ContainerDied","Data":"cb03c72e5b3a9a8cb605f47b3cfbc8a60bcccb2fa9667f91530d1450be6dca37"}
Sep 10 18:00:03.722012 ip-10-0-141-180 kubenswrapper[2581]: I0910 18:00:03.721854    2581 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="cb03c72e5b3a9a8cb605f47b3cfbc8a60bcccb2fa9667f91530d1450be6dca37"
Sep 10 18:00:03.723296 ip-10-0-141-180 kubenswrapper[2581]: I0910 18:00:03.723279    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w3-57474d-29817720-69zzz"
Sep 10 18:00:03.723413 ip-10-0-141-180 kubenswrapper[2581]: I0910 18:00:03.723304    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w3-57474d-29817720-69zzz" event={"ID":"81eeb254-adb2-42c9-9318-44a17603c3e3","Type":"ContainerDied","Data":"74072576df037f739d3831287922912fc67e1146a9825ddd2f9a0ec5588ffcc0"}
Sep 10 18:00:03.723413 ip-10-0-141-180 kubenswrapper[2581]: I0910 18:00:03.723339    2581 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="74072576df037f739d3831287922912fc67e1146a9825ddd2f9a0ec5588ffcc0"
Sep 10 18:00:03.724753 ip-10-0-141-180 kubenswrapper[2581]: I0910 18:00:03.724726    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ipp-a-904bbd-29817720-vg6ts" event={"ID":"060db639-fc7f-43d0-9650-9414951bca3b","Type":"ContainerDied","Data":"56e51d9064aaf7caf537d5330dac1625c92b3878b1bc671452431f4122fbc2e2"}
Sep 10 18:00:03.724882 ip-10-0-141-180 kubenswrapper[2581]: I0910 18:00:03.724762    2581 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="56e51d9064aaf7caf537d5330dac1625c92b3878b1bc671452431f4122fbc2e2"
Sep 10 18:00:03.724882 ip-10-0-141-180 kubenswrapper[2581]: I0910 18:00:03.724783    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ipp-a-904bbd-29817720-vg6ts"
Sep 10 18:00:04.317358 ip-10-0-141-180 kubenswrapper[2581]: I0910 18:00:04.317294    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-w2-9391d2-openshift-default-57c888f898-dwd2h"
Sep 10 18:00:04.400268 ip-10-0-141-180 kubenswrapper[2581]: I0910 18:00:04.400230    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d406cdfe-2228-4c17-ba8f-01a86277b666-credential-socket\") pod \"d406cdfe-2228-4c17-ba8f-01a86277b666\" (UID: \"d406cdfe-2228-4c17-ba8f-01a86277b666\") "
Sep 10 18:00:04.400443 ip-10-0-141-180 kubenswrapper[2581]: I0910 18:00:04.400311    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d406cdfe-2228-4c17-ba8f-01a86277b666-istio-podinfo\") pod \"d406cdfe-2228-4c17-ba8f-01a86277b666\" (UID: \"d406cdfe-2228-4c17-ba8f-01a86277b666\") "
Sep 10 18:00:04.400443 ip-10-0-141-180 kubenswrapper[2581]: I0910 18:00:04.400338    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d406cdfe-2228-4c17-ba8f-01a86277b666-istio-data\") pod \"d406cdfe-2228-4c17-ba8f-01a86277b666\" (UID: \"d406cdfe-2228-4c17-ba8f-01a86277b666\") "
Sep 10 18:00:04.400443 ip-10-0-141-180 kubenswrapper[2581]: I0910 18:00:04.400361    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d406cdfe-2228-4c17-ba8f-01a86277b666-workload-certs\") pod \"d406cdfe-2228-4c17-ba8f-01a86277b666\" (UID: \"d406cdfe-2228-4c17-ba8f-01a86277b666\") "
Sep 10 18:00:04.400443 ip-10-0-141-180 kubenswrapper[2581]: I0910 18:00:04.400400    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d406cdfe-2228-4c17-ba8f-01a86277b666-istio-token\") pod \"d406cdfe-2228-4c17-ba8f-01a86277b666\" (UID: \"d406cdfe-2228-4c17-ba8f-01a86277b666\") "
Sep 10 18:00:04.400716 ip-10-0-141-180 kubenswrapper[2581]: I0910 18:00:04.400468    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d406cdfe-2228-4c17-ba8f-01a86277b666-istio-envoy\") pod \"d406cdfe-2228-4c17-ba8f-01a86277b666\" (UID: \"d406cdfe-2228-4c17-ba8f-01a86277b666\") "
Sep 10 18:00:04.400716 ip-10-0-141-180 kubenswrapper[2581]: I0910 18:00:04.400491    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d406cdfe-2228-4c17-ba8f-01a86277b666-istiod-ca-cert\") pod \"d406cdfe-2228-4c17-ba8f-01a86277b666\" (UID: \"d406cdfe-2228-4c17-ba8f-01a86277b666\") "
Sep 10 18:00:04.400716 ip-10-0-141-180 kubenswrapper[2581]: I0910 18:00:04.400521    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d406cdfe-2228-4c17-ba8f-01a86277b666-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "d406cdfe-2228-4c17-ba8f-01a86277b666" (UID: "d406cdfe-2228-4c17-ba8f-01a86277b666"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 18:00:04.400716 ip-10-0-141-180 kubenswrapper[2581]: I0910 18:00:04.400534    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-8qsxq\" (UniqueName: \"kubernetes.io/projected/d406cdfe-2228-4c17-ba8f-01a86277b666-kube-api-access-8qsxq\") pod \"d406cdfe-2228-4c17-ba8f-01a86277b666\" (UID: \"d406cdfe-2228-4c17-ba8f-01a86277b666\") "
Sep 10 18:00:04.400716 ip-10-0-141-180 kubenswrapper[2581]: I0910 18:00:04.400624    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d406cdfe-2228-4c17-ba8f-01a86277b666-istio-data" (OuterVolumeSpecName: "istio-data") pod "d406cdfe-2228-4c17-ba8f-01a86277b666" (UID: "d406cdfe-2228-4c17-ba8f-01a86277b666"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 18:00:04.400716 ip-10-0-141-180 kubenswrapper[2581]: I0910 18:00:04.400632    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d406cdfe-2228-4c17-ba8f-01a86277b666-workload-socket\") pod \"d406cdfe-2228-4c17-ba8f-01a86277b666\" (UID: \"d406cdfe-2228-4c17-ba8f-01a86277b666\") "
Sep 10 18:00:04.401014 ip-10-0-141-180 kubenswrapper[2581]: I0910 18:00:04.400846    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d406cdfe-2228-4c17-ba8f-01a86277b666-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "d406cdfe-2228-4c17-ba8f-01a86277b666" (UID: "d406cdfe-2228-4c17-ba8f-01a86277b666"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 18:00:04.401014 ip-10-0-141-180 kubenswrapper[2581]: I0910 18:00:04.400944    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d406cdfe-2228-4c17-ba8f-01a86277b666-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "d406cdfe-2228-4c17-ba8f-01a86277b666" (UID: "d406cdfe-2228-4c17-ba8f-01a86277b666"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 18:00:04.401014 ip-10-0-141-180 kubenswrapper[2581]: I0910 18:00:04.400973    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d406cdfe-2228-4c17-ba8f-01a86277b666-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "d406cdfe-2228-4c17-ba8f-01a86277b666" (UID: "d406cdfe-2228-4c17-ba8f-01a86277b666"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 18:00:04.401185 ip-10-0-141-180 kubenswrapper[2581]: I0910 18:00:04.401100    2581 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d406cdfe-2228-4c17-ba8f-01a86277b666-istiod-ca-cert\") on node \"ip-10-0-141-180.ec2.internal\" DevicePath \"\""
Sep 10 18:00:04.401185 ip-10-0-141-180 kubenswrapper[2581]: I0910 18:00:04.401120    2581 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d406cdfe-2228-4c17-ba8f-01a86277b666-workload-socket\") on node \"ip-10-0-141-180.ec2.internal\" DevicePath \"\""
Sep 10 18:00:04.401185 ip-10-0-141-180 kubenswrapper[2581]: I0910 18:00:04.401134    2581 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d406cdfe-2228-4c17-ba8f-01a86277b666-credential-socket\") on node \"ip-10-0-141-180.ec2.internal\" DevicePath \"\""
Sep 10 18:00:04.401185 ip-10-0-141-180 kubenswrapper[2581]: I0910 18:00:04.401149    2581 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d406cdfe-2228-4c17-ba8f-01a86277b666-istio-data\") on node \"ip-10-0-141-180.ec2.internal\" DevicePath \"\""
Sep 10 18:00:04.401185 ip-10-0-141-180 kubenswrapper[2581]: I0910 18:00:04.401162    2581 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d406cdfe-2228-4c17-ba8f-01a86277b666-workload-certs\") on node \"ip-10-0-141-180.ec2.internal\" DevicePath \"\""
Sep 10 18:00:04.402672 ip-10-0-141-180 kubenswrapper[2581]: I0910 18:00:04.402629    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d406cdfe-2228-4c17-ba8f-01a86277b666-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "d406cdfe-2228-4c17-ba8f-01a86277b666" (UID: "d406cdfe-2228-4c17-ba8f-01a86277b666"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 18:00:04.402779 ip-10-0-141-180 kubenswrapper[2581]: I0910 18:00:04.402745    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/d406cdfe-2228-4c17-ba8f-01a86277b666-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "d406cdfe-2228-4c17-ba8f-01a86277b666" (UID: "d406cdfe-2228-4c17-ba8f-01a86277b666"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 10 18:00:04.403021 ip-10-0-141-180 kubenswrapper[2581]: I0910 18:00:04.403006    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d406cdfe-2228-4c17-ba8f-01a86277b666-istio-token" (OuterVolumeSpecName: "istio-token") pod "d406cdfe-2228-4c17-ba8f-01a86277b666" (UID: "d406cdfe-2228-4c17-ba8f-01a86277b666"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 18:00:04.403157 ip-10-0-141-180 kubenswrapper[2581]: I0910 18:00:04.403135    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d406cdfe-2228-4c17-ba8f-01a86277b666-kube-api-access-8qsxq" (OuterVolumeSpecName: "kube-api-access-8qsxq") pod "d406cdfe-2228-4c17-ba8f-01a86277b666" (UID: "d406cdfe-2228-4c17-ba8f-01a86277b666"). InnerVolumeSpecName "kube-api-access-8qsxq". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 18:00:04.502428 ip-10-0-141-180 kubenswrapper[2581]: I0910 18:00:04.502391    2581 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d406cdfe-2228-4c17-ba8f-01a86277b666-istio-podinfo\") on node \"ip-10-0-141-180.ec2.internal\" DevicePath \"\""
Sep 10 18:00:04.502428 ip-10-0-141-180 kubenswrapper[2581]: I0910 18:00:04.502421    2581 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d406cdfe-2228-4c17-ba8f-01a86277b666-istio-token\") on node \"ip-10-0-141-180.ec2.internal\" DevicePath \"\""
Sep 10 18:00:04.502428 ip-10-0-141-180 kubenswrapper[2581]: I0910 18:00:04.502430    2581 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d406cdfe-2228-4c17-ba8f-01a86277b666-istio-envoy\") on node \"ip-10-0-141-180.ec2.internal\" DevicePath \"\""
Sep 10 18:00:04.502784 ip-10-0-141-180 kubenswrapper[2581]: I0910 18:00:04.502445    2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8qsxq\" (UniqueName: \"kubernetes.io/projected/d406cdfe-2228-4c17-ba8f-01a86277b666-kube-api-access-8qsxq\") on node \"ip-10-0-141-180.ec2.internal\" DevicePath \"\""
Sep 10 18:00:04.734794 ip-10-0-141-180 kubenswrapper[2581]: I0910 18:00:04.734759    2581 generic.go:356] "Generic (PLEG): container finished" podID="d406cdfe-2228-4c17-ba8f-01a86277b666" containerID="dbebc011dddf30407b12734364f786768c1eca83a4331bfa7362eda1f5f87a6a" exitCode=0
Sep 10 18:00:04.735218 ip-10-0-141-180 kubenswrapper[2581]: I0910 18:00:04.734818    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-w2-9391d2-openshift-default-57c888f898-dwd2h"
Sep 10 18:00:04.735218 ip-10-0-141-180 kubenswrapper[2581]: I0910 18:00:04.734834    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w2-9391d2-openshift-default-57c888f898-dwd2h" event={"ID":"d406cdfe-2228-4c17-ba8f-01a86277b666","Type":"ContainerDied","Data":"dbebc011dddf30407b12734364f786768c1eca83a4331bfa7362eda1f5f87a6a"}
Sep 10 18:00:04.735218 ip-10-0-141-180 kubenswrapper[2581]: I0910 18:00:04.734915    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w2-9391d2-openshift-default-57c888f898-dwd2h" event={"ID":"d406cdfe-2228-4c17-ba8f-01a86277b666","Type":"ContainerDied","Data":"5661824dc11979779607aea7d88155a991f4b2ce82d55e19b6f06216caa73540"}
Sep 10 18:00:04.735218 ip-10-0-141-180 kubenswrapper[2581]: I0910 18:00:04.734938    2581 scope.go:117] "RemoveContainer" containerID="dbebc011dddf30407b12734364f786768c1eca83a4331bfa7362eda1f5f87a6a"
Sep 10 18:00:04.745270 ip-10-0-141-180 kubenswrapper[2581]: I0910 18:00:04.745249    2581 scope.go:117] "RemoveContainer" containerID="691ba5c9e471b1f22f077c8147ee5334fa8b934e8b85b9b8a619ad60f1c1f2c2"
Sep 10 18:00:04.754243 ip-10-0-141-180 kubenswrapper[2581]: I0910 18:00:04.754224    2581 scope.go:117] "RemoveContainer" containerID="dbebc011dddf30407b12734364f786768c1eca83a4331bfa7362eda1f5f87a6a"
Sep 10 18:00:04.754510 ip-10-0-141-180 kubenswrapper[2581]: E0910 18:00:04.754490    2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"dbebc011dddf30407b12734364f786768c1eca83a4331bfa7362eda1f5f87a6a\": container with ID starting with dbebc011dddf30407b12734364f786768c1eca83a4331bfa7362eda1f5f87a6a not found: ID does not exist" containerID="dbebc011dddf30407b12734364f786768c1eca83a4331bfa7362eda1f5f87a6a"
Sep 10 18:00:04.754581 ip-10-0-141-180 kubenswrapper[2581]: I0910 18:00:04.754522    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"dbebc011dddf30407b12734364f786768c1eca83a4331bfa7362eda1f5f87a6a"} err="failed to get container status \"dbebc011dddf30407b12734364f786768c1eca83a4331bfa7362eda1f5f87a6a\": rpc error: code = NotFound desc = could not find container \"dbebc011dddf30407b12734364f786768c1eca83a4331bfa7362eda1f5f87a6a\": container with ID starting with dbebc011dddf30407b12734364f786768c1eca83a4331bfa7362eda1f5f87a6a not found: ID does not exist"
Sep 10 18:00:04.754581 ip-10-0-141-180 kubenswrapper[2581]: I0910 18:00:04.754548    2581 scope.go:117] "RemoveContainer" containerID="691ba5c9e471b1f22f077c8147ee5334fa8b934e8b85b9b8a619ad60f1c1f2c2"
Sep 10 18:00:04.754857 ip-10-0-141-180 kubenswrapper[2581]: E0910 18:00:04.754839    2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"691ba5c9e471b1f22f077c8147ee5334fa8b934e8b85b9b8a619ad60f1c1f2c2\": container with ID starting with 691ba5c9e471b1f22f077c8147ee5334fa8b934e8b85b9b8a619ad60f1c1f2c2 not found: ID does not exist" containerID="691ba5c9e471b1f22f077c8147ee5334fa8b934e8b85b9b8a619ad60f1c1f2c2"
Sep 10 18:00:04.754904 ip-10-0-141-180 kubenswrapper[2581]: I0910 18:00:04.754864    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"691ba5c9e471b1f22f077c8147ee5334fa8b934e8b85b9b8a619ad60f1c1f2c2"} err="failed to get container status \"691ba5c9e471b1f22f077c8147ee5334fa8b934e8b85b9b8a619ad60f1c1f2c2\": rpc error: code = NotFound desc = could not find container \"691ba5c9e471b1f22f077c8147ee5334fa8b934e8b85b9b8a619ad60f1c1f2c2\": container with ID starting with 691ba5c9e471b1f22f077c8147ee5334fa8b934e8b85b9b8a619ad60f1c1f2c2 not found: ID does not exist"
Sep 10 18:00:04.761744 ip-10-0-141-180 kubenswrapper[2581]: I0910 18:00:04.761719    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-a-w2-9391d2-openshift-default-57c888f898-dwd2h"]
Sep 10 18:00:04.764466 ip-10-0-141-180 kubenswrapper[2581]: I0910 18:00:04.764446    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-shared-a-w2-9391d2-openshift-default-57c888f898-dwd2h"]
Sep 10 18:00:04.862160 ip-10-0-141-180 kubenswrapper[2581]: I0910 18:00:04.862128    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d406cdfe-2228-4c17-ba8f-01a86277b666" path="/var/lib/kubelet/pods/d406cdfe-2228-4c17-ba8f-01a86277b666/volumes"
Sep 10 18:00:09.776425 ip-10-0-141-180 kubenswrapper[2581]: I0910 18:00:09.776385    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w2-bdf47c-29817720-f6v44"]
Sep 10 18:00:09.779430 ip-10-0-141-180 kubenswrapper[2581]: I0910 18:00:09.779396    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w2-bdf47c-29817720-f6v44"]
Sep 10 18:00:09.822738 ip-10-0-141-180 kubenswrapper[2581]: I0910 18:00:09.822700    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w2-bdf47c-6dfb5579ff-sgr2m"]
Sep 10 18:00:09.823094 ip-10-0-141-180 kubenswrapper[2581]: I0910 18:00:09.822929    2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-shared-b-w2-bdf47c-6dfb5579ff-sgr2m" podUID="4b2ac153-7018-4bfc-b150-079763c603e6" containerName="payload-processing" containerID="cri-o://dbf6b9714d4283c3203f1b344e8e5d474b5f34a262caaaee497271351a2d042c" gracePeriod=30
Sep 10 18:00:10.030108 ip-10-0-141-180 kubenswrapper[2581]: I0910 18:00:10.030023    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-bdf47c-wvzvs"]
Sep 10 18:00:10.037438 ip-10-0-141-180 kubenswrapper[2581]: I0910 18:00:10.037399    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-bdf47c-wvzvs"]
Sep 10 18:00:10.098796 ip-10-0-141-180 kubenswrapper[2581]: I0910 18:00:10.098762    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-b-w2-bdf47c-6dfb5579ff-sgr2m"
Sep 10 18:00:10.160062 ip-10-0-141-180 kubenswrapper[2581]: I0910 18:00:10.160029    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/4b2ac153-7018-4bfc-b150-079763c603e6-plugins-config-volume\") pod \"4b2ac153-7018-4bfc-b150-079763c603e6\" (UID: \"4b2ac153-7018-4bfc-b150-079763c603e6\") "
Sep 10 18:00:10.160240 ip-10-0-141-180 kubenswrapper[2581]: I0910 18:00:10.160083    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-fssxq\" (UniqueName: \"kubernetes.io/projected/4b2ac153-7018-4bfc-b150-079763c603e6-kube-api-access-fssxq\") pod \"4b2ac153-7018-4bfc-b150-079763c603e6\" (UID: \"4b2ac153-7018-4bfc-b150-079763c603e6\") "
Sep 10 18:00:10.160508 ip-10-0-141-180 kubenswrapper[2581]: I0910 18:00:10.160477    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4b2ac153-7018-4bfc-b150-079763c603e6-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "4b2ac153-7018-4bfc-b150-079763c603e6" (UID: "4b2ac153-7018-4bfc-b150-079763c603e6"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 18:00:10.162240 ip-10-0-141-180 kubenswrapper[2581]: I0910 18:00:10.162210    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4b2ac153-7018-4bfc-b150-079763c603e6-kube-api-access-fssxq" (OuterVolumeSpecName: "kube-api-access-fssxq") pod "4b2ac153-7018-4bfc-b150-079763c603e6" (UID: "4b2ac153-7018-4bfc-b150-079763c603e6"). InnerVolumeSpecName "kube-api-access-fssxq". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 18:00:10.261257 ip-10-0-141-180 kubenswrapper[2581]: I0910 18:00:10.261220    2581 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/4b2ac153-7018-4bfc-b150-079763c603e6-plugins-config-volume\") on node \"ip-10-0-141-180.ec2.internal\" DevicePath \"\""
Sep 10 18:00:10.261257 ip-10-0-141-180 kubenswrapper[2581]: I0910 18:00:10.261250    2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-fssxq\" (UniqueName: \"kubernetes.io/projected/4b2ac153-7018-4bfc-b150-079763c603e6-kube-api-access-fssxq\") on node \"ip-10-0-141-180.ec2.internal\" DevicePath \"\""
Sep 10 18:00:10.780272 ip-10-0-141-180 kubenswrapper[2581]: I0910 18:00:10.780232    2581 generic.go:356] "Generic (PLEG): container finished" podID="4b2ac153-7018-4bfc-b150-079763c603e6" containerID="dbf6b9714d4283c3203f1b344e8e5d474b5f34a262caaaee497271351a2d042c" exitCode=0
Sep 10 18:00:10.780752 ip-10-0-141-180 kubenswrapper[2581]: I0910 18:00:10.780277    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-b-w2-bdf47c-6dfb5579ff-sgr2m"
Sep 10 18:00:10.780752 ip-10-0-141-180 kubenswrapper[2581]: I0910 18:00:10.780310    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-b-w2-bdf47c-6dfb5579ff-sgr2m" event={"ID":"4b2ac153-7018-4bfc-b150-079763c603e6","Type":"ContainerDied","Data":"dbf6b9714d4283c3203f1b344e8e5d474b5f34a262caaaee497271351a2d042c"}
Sep 10 18:00:10.780752 ip-10-0-141-180 kubenswrapper[2581]: I0910 18:00:10.780385    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-b-w2-bdf47c-6dfb5579ff-sgr2m" event={"ID":"4b2ac153-7018-4bfc-b150-079763c603e6","Type":"ContainerDied","Data":"beb1e8f9f188880e78c55855065f9a9d4529c043489a9d7bc95e06c7891f9520"}
Sep 10 18:00:10.780752 ip-10-0-141-180 kubenswrapper[2581]: I0910 18:00:10.780408    2581 scope.go:117] "RemoveContainer" containerID="dbf6b9714d4283c3203f1b344e8e5d474b5f34a262caaaee497271351a2d042c"
Sep 10 18:00:10.790380 ip-10-0-141-180 kubenswrapper[2581]: I0910 18:00:10.790360    2581 scope.go:117] "RemoveContainer" containerID="dbf6b9714d4283c3203f1b344e8e5d474b5f34a262caaaee497271351a2d042c"
Sep 10 18:00:10.790752 ip-10-0-141-180 kubenswrapper[2581]: E0910 18:00:10.790731    2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"dbf6b9714d4283c3203f1b344e8e5d474b5f34a262caaaee497271351a2d042c\": container with ID starting with dbf6b9714d4283c3203f1b344e8e5d474b5f34a262caaaee497271351a2d042c not found: ID does not exist" containerID="dbf6b9714d4283c3203f1b344e8e5d474b5f34a262caaaee497271351a2d042c"
Sep 10 18:00:10.790826 ip-10-0-141-180 kubenswrapper[2581]: I0910 18:00:10.790762    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"dbf6b9714d4283c3203f1b344e8e5d474b5f34a262caaaee497271351a2d042c"} err="failed to get container status \"dbf6b9714d4283c3203f1b344e8e5d474b5f34a262caaaee497271351a2d042c\": rpc error: code = NotFound desc = could not find container \"dbf6b9714d4283c3203f1b344e8e5d474b5f34a262caaaee497271351a2d042c\": container with ID starting with dbf6b9714d4283c3203f1b344e8e5d474b5f34a262caaaee497271351a2d042c not found: ID does not exist"
Sep 10 18:00:10.802742 ip-10-0-141-180 kubenswrapper[2581]: I0910 18:00:10.802701    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w2-bdf47c-6dfb5579ff-sgr2m"]
Sep 10 18:00:10.809916 ip-10-0-141-180 kubenswrapper[2581]: I0910 18:00:10.809889    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w2-bdf47c-6dfb5579ff-sgr2m"]
Sep 10 18:00:10.862995 ip-10-0-141-180 kubenswrapper[2581]: I0910 18:00:10.862960    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4b2ac153-7018-4bfc-b150-079763c603e6" path="/var/lib/kubelet/pods/4b2ac153-7018-4bfc-b150-079763c603e6/volumes"
Sep 10 18:00:10.863367 ip-10-0-141-180 kubenswrapper[2581]: I0910 18:00:10.863352    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5bbc8900-14fd-4cfe-a2ed-bd8cf1c33a4b" path="/var/lib/kubelet/pods/5bbc8900-14fd-4cfe-a2ed-bd8cf1c33a4b/volumes"
Sep 10 18:00:10.863665 ip-10-0-141-180 kubenswrapper[2581]: I0910 18:00:10.863649    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="afac154b-2857-447a-a333-5844bb606210" path="/var/lib/kubelet/pods/afac154b-2857-447a-a333-5844bb606210/volumes"
Sep 10 18:00:12.818768 ip-10-0-141-180 kubenswrapper[2581]: I0910 18:00:12.818728    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ipp-b-db978e-openshift-default-6c97b9d7d6-t6j27"]
Sep 10 18:00:12.819411 ip-10-0-141-180 kubenswrapper[2581]: I0910 18:00:12.819388    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5bbc8900-14fd-4cfe-a2ed-bd8cf1c33a4b" containerName="revoke-keys"
Sep 10 18:00:12.819411 ip-10-0-141-180 kubenswrapper[2581]: I0910 18:00:12.819410    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="5bbc8900-14fd-4cfe-a2ed-bd8cf1c33a4b" containerName="revoke-keys"
Sep 10 18:00:12.819561 ip-10-0-141-180 kubenswrapper[2581]: I0910 18:00:12.819421    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="060db639-fc7f-43d0-9650-9414951bca3b" containerName="cleanup"
Sep 10 18:00:12.819561 ip-10-0-141-180 kubenswrapper[2581]: I0910 18:00:12.819427    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="060db639-fc7f-43d0-9650-9414951bca3b" containerName="cleanup"
Sep 10 18:00:12.819561 ip-10-0-141-180 kubenswrapper[2581]: I0910 18:00:12.819450    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4b2ac153-7018-4bfc-b150-079763c603e6" containerName="payload-processing"
Sep 10 18:00:12.819561 ip-10-0-141-180 kubenswrapper[2581]: I0910 18:00:12.819459    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="4b2ac153-7018-4bfc-b150-079763c603e6" containerName="payload-processing"
Sep 10 18:00:12.819561 ip-10-0-141-180 kubenswrapper[2581]: I0910 18:00:12.819469    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d406cdfe-2228-4c17-ba8f-01a86277b666" containerName="istio-proxy"
Sep 10 18:00:12.819561 ip-10-0-141-180 kubenswrapper[2581]: I0910 18:00:12.819475    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="d406cdfe-2228-4c17-ba8f-01a86277b666" containerName="istio-proxy"
Sep 10 18:00:12.819561 ip-10-0-141-180 kubenswrapper[2581]: I0910 18:00:12.819483    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="81eeb254-adb2-42c9-9318-44a17603c3e3" containerName="cleanup"
Sep 10 18:00:12.819561 ip-10-0-141-180 kubenswrapper[2581]: I0910 18:00:12.819488    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="81eeb254-adb2-42c9-9318-44a17603c3e3" containerName="cleanup"
Sep 10 18:00:12.819561 ip-10-0-141-180 kubenswrapper[2581]: I0910 18:00:12.819495    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="afac154b-2857-447a-a333-5844bb606210" containerName="cleanup"
Sep 10 18:00:12.819561 ip-10-0-141-180 kubenswrapper[2581]: I0910 18:00:12.819502    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="afac154b-2857-447a-a333-5844bb606210" containerName="cleanup"
Sep 10 18:00:12.820104 ip-10-0-141-180 kubenswrapper[2581]: I0910 18:00:12.819577    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="060db639-fc7f-43d0-9650-9414951bca3b" containerName="cleanup"
Sep 10 18:00:12.820104 ip-10-0-141-180 kubenswrapper[2581]: I0910 18:00:12.819588    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="81eeb254-adb2-42c9-9318-44a17603c3e3" containerName="cleanup"
Sep 10 18:00:12.820104 ip-10-0-141-180 kubenswrapper[2581]: I0910 18:00:12.819596    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="4b2ac153-7018-4bfc-b150-079763c603e6" containerName="payload-processing"
Sep 10 18:00:12.820104 ip-10-0-141-180 kubenswrapper[2581]: I0910 18:00:12.819603    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="d406cdfe-2228-4c17-ba8f-01a86277b666" containerName="istio-proxy"
Sep 10 18:00:12.820104 ip-10-0-141-180 kubenswrapper[2581]: I0910 18:00:12.819608    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="d406cdfe-2228-4c17-ba8f-01a86277b666" containerName="istio-proxy"
Sep 10 18:00:12.820104 ip-10-0-141-180 kubenswrapper[2581]: I0910 18:00:12.819613    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="5bbc8900-14fd-4cfe-a2ed-bd8cf1c33a4b" containerName="revoke-keys"
Sep 10 18:00:12.820104 ip-10-0-141-180 kubenswrapper[2581]: I0910 18:00:12.819618    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="afac154b-2857-447a-a333-5844bb606210" containerName="cleanup"
Sep 10 18:00:12.824563 ip-10-0-141-180 kubenswrapper[2581]: I0910 18:00:12.824541    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-b-db978e-openshift-default-6c97b9d7d6-t6j27"
Sep 10 18:00:12.827198 ip-10-0-141-180 kubenswrapper[2581]: I0910 18:00:12.827179    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ipp-b-db978e-openshift-default-dockercfg-257cz\""
Sep 10 18:00:12.838193 ip-10-0-141-180 kubenswrapper[2581]: I0910 18:00:12.838159    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ipp-b-db978e-openshift-default-6c97b9d7d6-t6j27"]
Sep 10 18:00:12.883270 ip-10-0-141-180 kubenswrapper[2581]: I0910 18:00:12.883226    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5d24e0bc-661e-41ca-9938-fd836289d1be-istio-podinfo\") pod \"e2e-ipp-b-db978e-openshift-default-6c97b9d7d6-t6j27\" (UID: \"5d24e0bc-661e-41ca-9938-fd836289d1be\") " pod="openshift-ingress/e2e-ipp-b-db978e-openshift-default-6c97b9d7d6-t6j27"
Sep 10 18:00:12.883457 ip-10-0-141-180 kubenswrapper[2581]: I0910 18:00:12.883280    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5d24e0bc-661e-41ca-9938-fd836289d1be-istio-envoy\") pod \"e2e-ipp-b-db978e-openshift-default-6c97b9d7d6-t6j27\" (UID: \"5d24e0bc-661e-41ca-9938-fd836289d1be\") " pod="openshift-ingress/e2e-ipp-b-db978e-openshift-default-6c97b9d7d6-t6j27"
Sep 10 18:00:12.883457 ip-10-0-141-180 kubenswrapper[2581]: I0910 18:00:12.883310    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5d24e0bc-661e-41ca-9938-fd836289d1be-istiod-ca-cert\") pod \"e2e-ipp-b-db978e-openshift-default-6c97b9d7d6-t6j27\" (UID: \"5d24e0bc-661e-41ca-9938-fd836289d1be\") " pod="openshift-ingress/e2e-ipp-b-db978e-openshift-default-6c97b9d7d6-t6j27"
Sep 10 18:00:12.883556 ip-10-0-141-180 kubenswrapper[2581]: I0910 18:00:12.883481    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5d24e0bc-661e-41ca-9938-fd836289d1be-workload-socket\") pod \"e2e-ipp-b-db978e-openshift-default-6c97b9d7d6-t6j27\" (UID: \"5d24e0bc-661e-41ca-9938-fd836289d1be\") " pod="openshift-ingress/e2e-ipp-b-db978e-openshift-default-6c97b9d7d6-t6j27"
Sep 10 18:00:12.883556 ip-10-0-141-180 kubenswrapper[2581]: I0910 18:00:12.883519    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5d24e0bc-661e-41ca-9938-fd836289d1be-istio-data\") pod \"e2e-ipp-b-db978e-openshift-default-6c97b9d7d6-t6j27\" (UID: \"5d24e0bc-661e-41ca-9938-fd836289d1be\") " pod="openshift-ingress/e2e-ipp-b-db978e-openshift-default-6c97b9d7d6-t6j27"
Sep 10 18:00:12.883789 ip-10-0-141-180 kubenswrapper[2581]: I0910 18:00:12.883756    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5d24e0bc-661e-41ca-9938-fd836289d1be-istio-token\") pod \"e2e-ipp-b-db978e-openshift-default-6c97b9d7d6-t6j27\" (UID: \"5d24e0bc-661e-41ca-9938-fd836289d1be\") " pod="openshift-ingress/e2e-ipp-b-db978e-openshift-default-6c97b9d7d6-t6j27"
Sep 10 18:00:12.883875 ip-10-0-141-180 kubenswrapper[2581]: I0910 18:00:12.883824    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5d24e0bc-661e-41ca-9938-fd836289d1be-workload-certs\") pod \"e2e-ipp-b-db978e-openshift-default-6c97b9d7d6-t6j27\" (UID: \"5d24e0bc-661e-41ca-9938-fd836289d1be\") " pod="openshift-ingress/e2e-ipp-b-db978e-openshift-default-6c97b9d7d6-t6j27"
Sep 10 18:00:12.883875 ip-10-0-141-180 kubenswrapper[2581]: I0910 18:00:12.883866    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5d24e0bc-661e-41ca-9938-fd836289d1be-credential-socket\") pod \"e2e-ipp-b-db978e-openshift-default-6c97b9d7d6-t6j27\" (UID: \"5d24e0bc-661e-41ca-9938-fd836289d1be\") " pod="openshift-ingress/e2e-ipp-b-db978e-openshift-default-6c97b9d7d6-t6j27"
Sep 10 18:00:12.883962 ip-10-0-141-180 kubenswrapper[2581]: I0910 18:00:12.883928    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bppwt\" (UniqueName: \"kubernetes.io/projected/5d24e0bc-661e-41ca-9938-fd836289d1be-kube-api-access-bppwt\") pod \"e2e-ipp-b-db978e-openshift-default-6c97b9d7d6-t6j27\" (UID: \"5d24e0bc-661e-41ca-9938-fd836289d1be\") " pod="openshift-ingress/e2e-ipp-b-db978e-openshift-default-6c97b9d7d6-t6j27"
Sep 10 18:00:12.985487 ip-10-0-141-180 kubenswrapper[2581]: I0910 18:00:12.985258    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5d24e0bc-661e-41ca-9938-fd836289d1be-workload-socket\") pod \"e2e-ipp-b-db978e-openshift-default-6c97b9d7d6-t6j27\" (UID: \"5d24e0bc-661e-41ca-9938-fd836289d1be\") " pod="openshift-ingress/e2e-ipp-b-db978e-openshift-default-6c97b9d7d6-t6j27"
Sep 10 18:00:12.985487 ip-10-0-141-180 kubenswrapper[2581]: I0910 18:00:12.985330    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5d24e0bc-661e-41ca-9938-fd836289d1be-istio-data\") pod \"e2e-ipp-b-db978e-openshift-default-6c97b9d7d6-t6j27\" (UID: \"5d24e0bc-661e-41ca-9938-fd836289d1be\") " pod="openshift-ingress/e2e-ipp-b-db978e-openshift-default-6c97b9d7d6-t6j27"
Sep 10 18:00:12.985487 ip-10-0-141-180 kubenswrapper[2581]: I0910 18:00:12.985399    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5d24e0bc-661e-41ca-9938-fd836289d1be-istio-token\") pod \"e2e-ipp-b-db978e-openshift-default-6c97b9d7d6-t6j27\" (UID: \"5d24e0bc-661e-41ca-9938-fd836289d1be\") " pod="openshift-ingress/e2e-ipp-b-db978e-openshift-default-6c97b9d7d6-t6j27"
Sep 10 18:00:12.985487 ip-10-0-141-180 kubenswrapper[2581]: I0910 18:00:12.985435    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5d24e0bc-661e-41ca-9938-fd836289d1be-workload-certs\") pod \"e2e-ipp-b-db978e-openshift-default-6c97b9d7d6-t6j27\" (UID: \"5d24e0bc-661e-41ca-9938-fd836289d1be\") " pod="openshift-ingress/e2e-ipp-b-db978e-openshift-default-6c97b9d7d6-t6j27"
Sep 10 18:00:12.985487 ip-10-0-141-180 kubenswrapper[2581]: I0910 18:00:12.985473    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5d24e0bc-661e-41ca-9938-fd836289d1be-credential-socket\") pod \"e2e-ipp-b-db978e-openshift-default-6c97b9d7d6-t6j27\" (UID: \"5d24e0bc-661e-41ca-9938-fd836289d1be\") " pod="openshift-ingress/e2e-ipp-b-db978e-openshift-default-6c97b9d7d6-t6j27"
Sep 10 18:00:12.985921 ip-10-0-141-180 kubenswrapper[2581]: I0910 18:00:12.985507    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bppwt\" (UniqueName: \"kubernetes.io/projected/5d24e0bc-661e-41ca-9938-fd836289d1be-kube-api-access-bppwt\") pod \"e2e-ipp-b-db978e-openshift-default-6c97b9d7d6-t6j27\" (UID: \"5d24e0bc-661e-41ca-9938-fd836289d1be\") " pod="openshift-ingress/e2e-ipp-b-db978e-openshift-default-6c97b9d7d6-t6j27"
Sep 10 18:00:12.985921 ip-10-0-141-180 kubenswrapper[2581]: I0910 18:00:12.985537    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5d24e0bc-661e-41ca-9938-fd836289d1be-istio-podinfo\") pod \"e2e-ipp-b-db978e-openshift-default-6c97b9d7d6-t6j27\" (UID: \"5d24e0bc-661e-41ca-9938-fd836289d1be\") " pod="openshift-ingress/e2e-ipp-b-db978e-openshift-default-6c97b9d7d6-t6j27"
Sep 10 18:00:12.985921 ip-10-0-141-180 kubenswrapper[2581]: I0910 18:00:12.985572    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5d24e0bc-661e-41ca-9938-fd836289d1be-istio-envoy\") pod \"e2e-ipp-b-db978e-openshift-default-6c97b9d7d6-t6j27\" (UID: \"5d24e0bc-661e-41ca-9938-fd836289d1be\") " pod="openshift-ingress/e2e-ipp-b-db978e-openshift-default-6c97b9d7d6-t6j27"
Sep 10 18:00:12.985921 ip-10-0-141-180 kubenswrapper[2581]: I0910 18:00:12.985601    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5d24e0bc-661e-41ca-9938-fd836289d1be-istiod-ca-cert\") pod \"e2e-ipp-b-db978e-openshift-default-6c97b9d7d6-t6j27\" (UID: \"5d24e0bc-661e-41ca-9938-fd836289d1be\") " pod="openshift-ingress/e2e-ipp-b-db978e-openshift-default-6c97b9d7d6-t6j27"
Sep 10 18:00:12.986492 ip-10-0-141-180 kubenswrapper[2581]: I0910 18:00:12.986461    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5d24e0bc-661e-41ca-9938-fd836289d1be-istiod-ca-cert\") pod \"e2e-ipp-b-db978e-openshift-default-6c97b9d7d6-t6j27\" (UID: \"5d24e0bc-661e-41ca-9938-fd836289d1be\") " pod="openshift-ingress/e2e-ipp-b-db978e-openshift-default-6c97b9d7d6-t6j27"
Sep 10 18:00:12.986794 ip-10-0-141-180 kubenswrapper[2581]: I0910 18:00:12.986770    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5d24e0bc-661e-41ca-9938-fd836289d1be-workload-socket\") pod \"e2e-ipp-b-db978e-openshift-default-6c97b9d7d6-t6j27\" (UID: \"5d24e0bc-661e-41ca-9938-fd836289d1be\") " pod="openshift-ingress/e2e-ipp-b-db978e-openshift-default-6c97b9d7d6-t6j27"
Sep 10 18:00:12.987035 ip-10-0-141-180 kubenswrapper[2581]: I0910 18:00:12.987011    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5d24e0bc-661e-41ca-9938-fd836289d1be-istio-data\") pod \"e2e-ipp-b-db978e-openshift-default-6c97b9d7d6-t6j27\" (UID: \"5d24e0bc-661e-41ca-9938-fd836289d1be\") " pod="openshift-ingress/e2e-ipp-b-db978e-openshift-default-6c97b9d7d6-t6j27"
Sep 10 18:00:12.987586 ip-10-0-141-180 kubenswrapper[2581]: I0910 18:00:12.987558    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5d24e0bc-661e-41ca-9938-fd836289d1be-workload-certs\") pod \"e2e-ipp-b-db978e-openshift-default-6c97b9d7d6-t6j27\" (UID: \"5d24e0bc-661e-41ca-9938-fd836289d1be\") " pod="openshift-ingress/e2e-ipp-b-db978e-openshift-default-6c97b9d7d6-t6j27"
Sep 10 18:00:12.987840 ip-10-0-141-180 kubenswrapper[2581]: I0910 18:00:12.987818    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5d24e0bc-661e-41ca-9938-fd836289d1be-credential-socket\") pod \"e2e-ipp-b-db978e-openshift-default-6c97b9d7d6-t6j27\" (UID: \"5d24e0bc-661e-41ca-9938-fd836289d1be\") " pod="openshift-ingress/e2e-ipp-b-db978e-openshift-default-6c97b9d7d6-t6j27"
Sep 10 18:00:12.996357 ip-10-0-141-180 kubenswrapper[2581]: I0910 18:00:12.996324    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5d24e0bc-661e-41ca-9938-fd836289d1be-istio-podinfo\") pod \"e2e-ipp-b-db978e-openshift-default-6c97b9d7d6-t6j27\" (UID: \"5d24e0bc-661e-41ca-9938-fd836289d1be\") " pod="openshift-ingress/e2e-ipp-b-db978e-openshift-default-6c97b9d7d6-t6j27"
Sep 10 18:00:13.003597 ip-10-0-141-180 kubenswrapper[2581]: I0910 18:00:13.003531    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5d24e0bc-661e-41ca-9938-fd836289d1be-istio-envoy\") pod \"e2e-ipp-b-db978e-openshift-default-6c97b9d7d6-t6j27\" (UID: \"5d24e0bc-661e-41ca-9938-fd836289d1be\") " pod="openshift-ingress/e2e-ipp-b-db978e-openshift-default-6c97b9d7d6-t6j27"
Sep 10 18:00:13.005402 ip-10-0-141-180 kubenswrapper[2581]: I0910 18:00:13.005069    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5d24e0bc-661e-41ca-9938-fd836289d1be-istio-token\") pod \"e2e-ipp-b-db978e-openshift-default-6c97b9d7d6-t6j27\" (UID: \"5d24e0bc-661e-41ca-9938-fd836289d1be\") " pod="openshift-ingress/e2e-ipp-b-db978e-openshift-default-6c97b9d7d6-t6j27"
Sep 10 18:00:13.007434 ip-10-0-141-180 kubenswrapper[2581]: I0910 18:00:13.007373    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bppwt\" (UniqueName: \"kubernetes.io/projected/5d24e0bc-661e-41ca-9938-fd836289d1be-kube-api-access-bppwt\") pod \"e2e-ipp-b-db978e-openshift-default-6c97b9d7d6-t6j27\" (UID: \"5d24e0bc-661e-41ca-9938-fd836289d1be\") " pod="openshift-ingress/e2e-ipp-b-db978e-openshift-default-6c97b9d7d6-t6j27"
Sep 10 18:00:13.136154 ip-10-0-141-180 kubenswrapper[2581]: I0910 18:00:13.136061    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-b-db978e-openshift-default-6c97b9d7d6-t6j27"
Sep 10 18:00:13.419215 ip-10-0-141-180 kubenswrapper[2581]: I0910 18:00:13.419188    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ipp-b-db978e-openshift-default-6c97b9d7d6-t6j27"]
Sep 10 18:00:13.420136 ip-10-0-141-180 kubenswrapper[2581]: W0910 18:00:13.420113    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5d24e0bc_661e_41ca_9938_fd836289d1be.slice/crio-6721c7358990a151072cc41b7324e500913f0558ba4beae3c9c72ed8532ee143 WatchSource:0}: Error finding container 6721c7358990a151072cc41b7324e500913f0558ba4beae3c9c72ed8532ee143: Status 404 returned error can't find the container with id 6721c7358990a151072cc41b7324e500913f0558ba4beae3c9c72ed8532ee143
Sep 10 18:00:13.422359 ip-10-0-141-180 kubenswrapper[2581]: I0910 18:00:13.422324    2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 10 18:00:13.422425 ip-10-0-141-180 kubenswrapper[2581]: I0910 18:00:13.422389    2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 10 18:00:13.422468 ip-10-0-141-180 kubenswrapper[2581]: I0910 18:00:13.422425    2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 10 18:00:13.804943 ip-10-0-141-180 kubenswrapper[2581]: I0910 18:00:13.804909    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-b-db978e-openshift-default-6c97b9d7d6-t6j27" event={"ID":"5d24e0bc-661e-41ca-9938-fd836289d1be","Type":"ContainerStarted","Data":"0c294711e6337bbc6b262ab11dcb0821767f039721f43f3f224ba9cb18ae8201"}
Sep 10 18:00:13.805120 ip-10-0-141-180 kubenswrapper[2581]: I0910 18:00:13.804959    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-b-db978e-openshift-default-6c97b9d7d6-t6j27" event={"ID":"5d24e0bc-661e-41ca-9938-fd836289d1be","Type":"ContainerStarted","Data":"6721c7358990a151072cc41b7324e500913f0558ba4beae3c9c72ed8532ee143"}
Sep 10 18:00:13.826829 ip-10-0-141-180 kubenswrapper[2581]: I0910 18:00:13.826775    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-ipp-b-db978e-openshift-default-6c97b9d7d6-t6j27" podStartSLOduration=1.82675612 podStartE2EDuration="1.82675612s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 18:00:12 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 18:00:13.825085614 +0000 UTC m=+1433.479159543" watchObservedRunningTime="2026-09-10 18:00:13.82675612 +0000 UTC m=+1433.480830018"
Sep 10 18:00:14.136921 ip-10-0-141-180 kubenswrapper[2581]: I0910 18:00:14.136827    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ipp-b-db978e-openshift-default-6c97b9d7d6-t6j27"
Sep 10 18:00:14.138566 ip-10-0-141-180 kubenswrapper[2581]: I0910 18:00:14.138522    2581 patch_prober.go:28] interesting pod/e2e-ipp-b-db978e-openshift-default-6c97b9d7d6-t6j27 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.86:15021/healthz/ready\": dial tcp 10.133.0.86:15021: connect: connection refused" start-of-body=
Sep 10 18:00:14.138734 ip-10-0-141-180 kubenswrapper[2581]: I0910 18:00:14.138621    2581 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-b-db978e-openshift-default-6c97b9d7d6-t6j27" podUID="5d24e0bc-661e-41ca-9938-fd836289d1be" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.86:15021/healthz/ready\": dial tcp 10.133.0.86:15021: connect: connection refused"
Sep 10 18:00:15.140395 ip-10-0-141-180 kubenswrapper[2581]: I0910 18:00:15.140362    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ipp-b-db978e-openshift-default-6c97b9d7d6-t6j27"
Sep 10 18:00:15.819929 ip-10-0-141-180 kubenswrapper[2581]: I0910 18:00:15.819892    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ipp-b-db978e-openshift-default-6c97b9d7d6-t6j27"
Sep 10 18:00:15.820763 ip-10-0-141-180 kubenswrapper[2581]: I0910 18:00:15.820744    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ipp-b-db978e-openshift-default-6c97b9d7d6-t6j27"
Sep 10 18:00:17.742592 ip-10-0-141-180 kubenswrapper[2581]: I0910 18:00:17.742557    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-b-db978e-645f9874df-sr74b"]
Sep 10 18:00:17.743024 ip-10-0-141-180 kubenswrapper[2581]: I0910 18:00:17.743010    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d406cdfe-2228-4c17-ba8f-01a86277b666" containerName="istio-proxy"
Sep 10 18:00:17.743024 ip-10-0-141-180 kubenswrapper[2581]: I0910 18:00:17.743024    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="d406cdfe-2228-4c17-ba8f-01a86277b666" containerName="istio-proxy"
Sep 10 18:00:17.746464 ip-10-0-141-180 kubenswrapper[2581]: I0910 18:00:17.746443    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-b-db978e-645f9874df-sr74b"
Sep 10 18:00:17.748898 ip-10-0-141-180 kubenswrapper[2581]: I0910 18:00:17.748880    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-ipp-b-db978e\""
Sep 10 18:00:17.758887 ip-10-0-141-180 kubenswrapper[2581]: I0910 18:00:17.758860    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-b-db978e-645f9874df-sr74b"]
Sep 10 18:00:17.819125 ip-10-0-141-180 kubenswrapper[2581]: I0910 18:00:17.819084    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-b-db978e-8545cd7bbb-fdrvn"]
Sep 10 18:00:17.822813 ip-10-0-141-180 kubenswrapper[2581]: I0910 18:00:17.822789    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-db978e-8545cd7bbb-fdrvn"
Sep 10 18:00:17.825778 ip-10-0-141-180 kubenswrapper[2581]: I0910 18:00:17.825755    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-ipp-b-db978e-dockercfg-7q9jr\""
Sep 10 18:00:17.834523 ip-10-0-141-180 kubenswrapper[2581]: I0910 18:00:17.834497    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-b-db978e-8545cd7bbb-fdrvn"]
Sep 10 18:00:17.835448 ip-10-0-141-180 kubenswrapper[2581]: I0910 18:00:17.835419    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-l89mb\" (UniqueName: \"kubernetes.io/projected/29ae9709-2d36-4edb-bb33-1bd286472160-kube-api-access-l89mb\") pod \"payload-processing-e2e-ipp-b-db978e-645f9874df-sr74b\" (UID: \"29ae9709-2d36-4edb-bb33-1bd286472160\") " pod="openshift-ingress/payload-processing-e2e-ipp-b-db978e-645f9874df-sr74b"
Sep 10 18:00:17.835574 ip-10-0-141-180 kubenswrapper[2581]: I0910 18:00:17.835485    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/29ae9709-2d36-4edb-bb33-1bd286472160-plugins-config-volume\") pod \"payload-processing-e2e-ipp-b-db978e-645f9874df-sr74b\" (UID: \"29ae9709-2d36-4edb-bb33-1bd286472160\") " pod="openshift-ingress/payload-processing-e2e-ipp-b-db978e-645f9874df-sr74b"
Sep 10 18:00:17.905712 ip-10-0-141-180 kubenswrapper[2581]: I0910 18:00:17.905625    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-b-w2-bdf47c-openshift-default-6d77688b7b-slw4g"]
Sep 10 18:00:17.905918 ip-10-0-141-180 kubenswrapper[2581]: I0910 18:00:17.905884    2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-shared-b-w2-bdf47c-openshift-default-6d77688b7b-slw4g" podUID="08571c40-a532-4fa2-a18b-c7e583a29609" containerName="istio-proxy" containerID="cri-o://5b7c9837102beba86605ce1da3791ab14f074b2719f75ca25e4779ef5659a90e" gracePeriod=30
Sep 10 18:00:17.936393 ip-10-0-141-180 kubenswrapper[2581]: I0910 18:00:17.936356    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2gwzk\" (UniqueName: \"kubernetes.io/projected/a27ba0b8-011f-4f02-b629-974dde21709e-kube-api-access-2gwzk\") pod \"payload-pre-processing-e2e-ipp-b-db978e-8545cd7bbb-fdrvn\" (UID: \"a27ba0b8-011f-4f02-b629-974dde21709e\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-db978e-8545cd7bbb-fdrvn"
Sep 10 18:00:17.936633 ip-10-0-141-180 kubenswrapper[2581]: I0910 18:00:17.936604    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-l89mb\" (UniqueName: \"kubernetes.io/projected/29ae9709-2d36-4edb-bb33-1bd286472160-kube-api-access-l89mb\") pod \"payload-processing-e2e-ipp-b-db978e-645f9874df-sr74b\" (UID: \"29ae9709-2d36-4edb-bb33-1bd286472160\") " pod="openshift-ingress/payload-processing-e2e-ipp-b-db978e-645f9874df-sr74b"
Sep 10 18:00:17.936744 ip-10-0-141-180 kubenswrapper[2581]: I0910 18:00:17.936666    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a27ba0b8-011f-4f02-b629-974dde21709e-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-b-db978e-8545cd7bbb-fdrvn\" (UID: \"a27ba0b8-011f-4f02-b629-974dde21709e\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-db978e-8545cd7bbb-fdrvn"
Sep 10 18:00:17.936822 ip-10-0-141-180 kubenswrapper[2581]: I0910 18:00:17.936738    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/29ae9709-2d36-4edb-bb33-1bd286472160-plugins-config-volume\") pod \"payload-processing-e2e-ipp-b-db978e-645f9874df-sr74b\" (UID: \"29ae9709-2d36-4edb-bb33-1bd286472160\") " pod="openshift-ingress/payload-processing-e2e-ipp-b-db978e-645f9874df-sr74b"
Sep 10 18:00:17.937549 ip-10-0-141-180 kubenswrapper[2581]: I0910 18:00:17.937525    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/29ae9709-2d36-4edb-bb33-1bd286472160-plugins-config-volume\") pod \"payload-processing-e2e-ipp-b-db978e-645f9874df-sr74b\" (UID: \"29ae9709-2d36-4edb-bb33-1bd286472160\") " pod="openshift-ingress/payload-processing-e2e-ipp-b-db978e-645f9874df-sr74b"
Sep 10 18:00:17.945124 ip-10-0-141-180 kubenswrapper[2581]: I0910 18:00:17.945101    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-l89mb\" (UniqueName: \"kubernetes.io/projected/29ae9709-2d36-4edb-bb33-1bd286472160-kube-api-access-l89mb\") pod \"payload-processing-e2e-ipp-b-db978e-645f9874df-sr74b\" (UID: \"29ae9709-2d36-4edb-bb33-1bd286472160\") " pod="openshift-ingress/payload-processing-e2e-ipp-b-db978e-645f9874df-sr74b"
Sep 10 18:00:18.037425 ip-10-0-141-180 kubenswrapper[2581]: I0910 18:00:18.037341    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2gwzk\" (UniqueName: \"kubernetes.io/projected/a27ba0b8-011f-4f02-b629-974dde21709e-kube-api-access-2gwzk\") pod \"payload-pre-processing-e2e-ipp-b-db978e-8545cd7bbb-fdrvn\" (UID: \"a27ba0b8-011f-4f02-b629-974dde21709e\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-db978e-8545cd7bbb-fdrvn"
Sep 10 18:00:18.037579 ip-10-0-141-180 kubenswrapper[2581]: I0910 18:00:18.037462    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a27ba0b8-011f-4f02-b629-974dde21709e-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-b-db978e-8545cd7bbb-fdrvn\" (UID: \"a27ba0b8-011f-4f02-b629-974dde21709e\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-db978e-8545cd7bbb-fdrvn"
Sep 10 18:00:18.038093 ip-10-0-141-180 kubenswrapper[2581]: I0910 18:00:18.038074    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a27ba0b8-011f-4f02-b629-974dde21709e-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-b-db978e-8545cd7bbb-fdrvn\" (UID: \"a27ba0b8-011f-4f02-b629-974dde21709e\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-db978e-8545cd7bbb-fdrvn"
Sep 10 18:00:18.046237 ip-10-0-141-180 kubenswrapper[2581]: I0910 18:00:18.046214    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2gwzk\" (UniqueName: \"kubernetes.io/projected/a27ba0b8-011f-4f02-b629-974dde21709e-kube-api-access-2gwzk\") pod \"payload-pre-processing-e2e-ipp-b-db978e-8545cd7bbb-fdrvn\" (UID: \"a27ba0b8-011f-4f02-b629-974dde21709e\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-db978e-8545cd7bbb-fdrvn"
Sep 10 18:00:18.058701 ip-10-0-141-180 kubenswrapper[2581]: I0910 18:00:18.058654    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-b-db978e-645f9874df-sr74b"
Sep 10 18:00:18.135018 ip-10-0-141-180 kubenswrapper[2581]: I0910 18:00:18.134988    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-db978e-8545cd7bbb-fdrvn"
Sep 10 18:00:18.337486 ip-10-0-141-180 kubenswrapper[2581]: I0910 18:00:18.337453    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-b-db978e-645f9874df-sr74b"]
Sep 10 18:00:18.420847 ip-10-0-141-180 kubenswrapper[2581]: I0910 18:00:18.420817    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-b-db978e-8545cd7bbb-fdrvn"]
Sep 10 18:00:18.421651 ip-10-0-141-180 kubenswrapper[2581]: W0910 18:00:18.421607    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda27ba0b8_011f_4f02_b629_974dde21709e.slice/crio-ce4d7c02f6a66e9e0f3841bf229d321bf5200aeb12331caed38cc9210cccbd0c WatchSource:0}: Error finding container ce4d7c02f6a66e9e0f3841bf229d321bf5200aeb12331caed38cc9210cccbd0c: Status 404 returned error can't find the container with id ce4d7c02f6a66e9e0f3841bf229d321bf5200aeb12331caed38cc9210cccbd0c
Sep 10 18:00:18.846051 ip-10-0-141-180 kubenswrapper[2581]: I0910 18:00:18.846007    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-b-db978e-645f9874df-sr74b" event={"ID":"29ae9709-2d36-4edb-bb33-1bd286472160","Type":"ContainerStarted","Data":"d204f4833d1ebf4535fa68219d08ac801e7bd6b0289c15af1d12ff32aff2ad7b"}
Sep 10 18:00:18.846528 ip-10-0-141-180 kubenswrapper[2581]: I0910 18:00:18.846079    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-b-db978e-645f9874df-sr74b" event={"ID":"29ae9709-2d36-4edb-bb33-1bd286472160","Type":"ContainerStarted","Data":"b5694695601b0e049660e485fa8021adb8acbffc15c1bedf7edaa33060fb4c7b"}
Sep 10 18:00:18.846528 ip-10-0-141-180 kubenswrapper[2581]: I0910 18:00:18.846140    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-ipp-b-db978e-645f9874df-sr74b"
Sep 10 18:00:18.848923 ip-10-0-141-180 kubenswrapper[2581]: I0910 18:00:18.848880    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-db978e-8545cd7bbb-fdrvn" event={"ID":"a27ba0b8-011f-4f02-b629-974dde21709e","Type":"ContainerStarted","Data":"6d6909e5ac1075bb09ded7296368dde0ea5970fe7b4d423d3054c44c46ecf74e"}
Sep 10 18:00:18.849207 ip-10-0-141-180 kubenswrapper[2581]: I0910 18:00:18.848938    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-db978e-8545cd7bbb-fdrvn" event={"ID":"a27ba0b8-011f-4f02-b629-974dde21709e","Type":"ContainerStarted","Data":"ce4d7c02f6a66e9e0f3841bf229d321bf5200aeb12331caed38cc9210cccbd0c"}
Sep 10 18:00:18.849521 ip-10-0-141-180 kubenswrapper[2581]: I0910 18:00:18.849497    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-db978e-8545cd7bbb-fdrvn"
Sep 10 18:00:18.866057 ip-10-0-141-180 kubenswrapper[2581]: I0910 18:00:18.866011    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-ipp-b-db978e-645f9874df-sr74b" podStartSLOduration=1.865983414 podStartE2EDuration="1.865983414s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 18:00:17 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 18:00:18.862722323 +0000 UTC m=+1438.516796253" watchObservedRunningTime="2026-09-10 18:00:18.865983414 +0000 UTC m=+1438.520057318"
Sep 10 18:00:18.882032 ip-10-0-141-180 kubenswrapper[2581]: I0910 18:00:18.881986    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-db978e-8545cd7bbb-fdrvn" podStartSLOduration=1.88196213 podStartE2EDuration="1.88196213s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 18:00:17 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 18:00:18.879005341 +0000 UTC m=+1438.533079264" watchObservedRunningTime="2026-09-10 18:00:18.88196213 +0000 UTC m=+1438.536036034"
Sep 10 18:00:23.173077 ip-10-0-141-180 kubenswrapper[2581]: I0910 18:00:23.173008    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-w2-bdf47c-openshift-default-6d77688b7b-slw4g"
Sep 10 18:00:23.297717 ip-10-0-141-180 kubenswrapper[2581]: I0910 18:00:23.297684    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/08571c40-a532-4fa2-a18b-c7e583a29609-workload-socket\") pod \"08571c40-a532-4fa2-a18b-c7e583a29609\" (UID: \"08571c40-a532-4fa2-a18b-c7e583a29609\") "
Sep 10 18:00:23.297894 ip-10-0-141-180 kubenswrapper[2581]: I0910 18:00:23.297729    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/08571c40-a532-4fa2-a18b-c7e583a29609-istio-podinfo\") pod \"08571c40-a532-4fa2-a18b-c7e583a29609\" (UID: \"08571c40-a532-4fa2-a18b-c7e583a29609\") "
Sep 10 18:00:23.297894 ip-10-0-141-180 kubenswrapper[2581]: I0910 18:00:23.297772    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/08571c40-a532-4fa2-a18b-c7e583a29609-istio-token\") pod \"08571c40-a532-4fa2-a18b-c7e583a29609\" (UID: \"08571c40-a532-4fa2-a18b-c7e583a29609\") "
Sep 10 18:00:23.297894 ip-10-0-141-180 kubenswrapper[2581]: I0910 18:00:23.297802    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-h5hlx\" (UniqueName: \"kubernetes.io/projected/08571c40-a532-4fa2-a18b-c7e583a29609-kube-api-access-h5hlx\") pod \"08571c40-a532-4fa2-a18b-c7e583a29609\" (UID: \"08571c40-a532-4fa2-a18b-c7e583a29609\") "
Sep 10 18:00:23.297894 ip-10-0-141-180 kubenswrapper[2581]: I0910 18:00:23.297823    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/08571c40-a532-4fa2-a18b-c7e583a29609-credential-socket\") pod \"08571c40-a532-4fa2-a18b-c7e583a29609\" (UID: \"08571c40-a532-4fa2-a18b-c7e583a29609\") "
Sep 10 18:00:23.297894 ip-10-0-141-180 kubenswrapper[2581]: I0910 18:00:23.297866    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/08571c40-a532-4fa2-a18b-c7e583a29609-istio-data\") pod \"08571c40-a532-4fa2-a18b-c7e583a29609\" (UID: \"08571c40-a532-4fa2-a18b-c7e583a29609\") "
Sep 10 18:00:23.298175 ip-10-0-141-180 kubenswrapper[2581]: I0910 18:00:23.297953    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/08571c40-a532-4fa2-a18b-c7e583a29609-workload-certs\") pod \"08571c40-a532-4fa2-a18b-c7e583a29609\" (UID: \"08571c40-a532-4fa2-a18b-c7e583a29609\") "
Sep 10 18:00:23.298175 ip-10-0-141-180 kubenswrapper[2581]: I0910 18:00:23.297960    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/08571c40-a532-4fa2-a18b-c7e583a29609-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "08571c40-a532-4fa2-a18b-c7e583a29609" (UID: "08571c40-a532-4fa2-a18b-c7e583a29609"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 18:00:23.298175 ip-10-0-141-180 kubenswrapper[2581]: I0910 18:00:23.297980    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/08571c40-a532-4fa2-a18b-c7e583a29609-istio-envoy\") pod \"08571c40-a532-4fa2-a18b-c7e583a29609\" (UID: \"08571c40-a532-4fa2-a18b-c7e583a29609\") "
Sep 10 18:00:23.298175 ip-10-0-141-180 kubenswrapper[2581]: I0910 18:00:23.298017    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/08571c40-a532-4fa2-a18b-c7e583a29609-istiod-ca-cert\") pod \"08571c40-a532-4fa2-a18b-c7e583a29609\" (UID: \"08571c40-a532-4fa2-a18b-c7e583a29609\") "
Sep 10 18:00:23.298392 ip-10-0-141-180 kubenswrapper[2581]: I0910 18:00:23.298160    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/08571c40-a532-4fa2-a18b-c7e583a29609-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "08571c40-a532-4fa2-a18b-c7e583a29609" (UID: "08571c40-a532-4fa2-a18b-c7e583a29609"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 18:00:23.298392 ip-10-0-141-180 kubenswrapper[2581]: I0910 18:00:23.298253    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/08571c40-a532-4fa2-a18b-c7e583a29609-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "08571c40-a532-4fa2-a18b-c7e583a29609" (UID: "08571c40-a532-4fa2-a18b-c7e583a29609"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 18:00:23.298505 ip-10-0-141-180 kubenswrapper[2581]: I0910 18:00:23.298425    2581 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/08571c40-a532-4fa2-a18b-c7e583a29609-workload-socket\") on node \"ip-10-0-141-180.ec2.internal\" DevicePath \"\""
Sep 10 18:00:23.298505 ip-10-0-141-180 kubenswrapper[2581]: I0910 18:00:23.298446    2581 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/08571c40-a532-4fa2-a18b-c7e583a29609-credential-socket\") on node \"ip-10-0-141-180.ec2.internal\" DevicePath \"\""
Sep 10 18:00:23.298505 ip-10-0-141-180 kubenswrapper[2581]: I0910 18:00:23.298449    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/08571c40-a532-4fa2-a18b-c7e583a29609-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "08571c40-a532-4fa2-a18b-c7e583a29609" (UID: "08571c40-a532-4fa2-a18b-c7e583a29609"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 18:00:23.298505 ip-10-0-141-180 kubenswrapper[2581]: I0910 18:00:23.298461    2581 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/08571c40-a532-4fa2-a18b-c7e583a29609-workload-certs\") on node \"ip-10-0-141-180.ec2.internal\" DevicePath \"\""
Sep 10 18:00:23.298750 ip-10-0-141-180 kubenswrapper[2581]: I0910 18:00:23.298547    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/08571c40-a532-4fa2-a18b-c7e583a29609-istio-data" (OuterVolumeSpecName: "istio-data") pod "08571c40-a532-4fa2-a18b-c7e583a29609" (UID: "08571c40-a532-4fa2-a18b-c7e583a29609"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 18:00:23.299981 ip-10-0-141-180 kubenswrapper[2581]: I0910 18:00:23.299955    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/08571c40-a532-4fa2-a18b-c7e583a29609-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "08571c40-a532-4fa2-a18b-c7e583a29609" (UID: "08571c40-a532-4fa2-a18b-c7e583a29609"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 10 18:00:23.300244 ip-10-0-141-180 kubenswrapper[2581]: I0910 18:00:23.300222    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/08571c40-a532-4fa2-a18b-c7e583a29609-istio-token" (OuterVolumeSpecName: "istio-token") pod "08571c40-a532-4fa2-a18b-c7e583a29609" (UID: "08571c40-a532-4fa2-a18b-c7e583a29609"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 18:00:23.300385 ip-10-0-141-180 kubenswrapper[2581]: I0910 18:00:23.300368    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/08571c40-a532-4fa2-a18b-c7e583a29609-kube-api-access-h5hlx" (OuterVolumeSpecName: "kube-api-access-h5hlx") pod "08571c40-a532-4fa2-a18b-c7e583a29609" (UID: "08571c40-a532-4fa2-a18b-c7e583a29609"). InnerVolumeSpecName "kube-api-access-h5hlx". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 18:00:23.300442 ip-10-0-141-180 kubenswrapper[2581]: I0910 18:00:23.300400    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/08571c40-a532-4fa2-a18b-c7e583a29609-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "08571c40-a532-4fa2-a18b-c7e583a29609" (UID: "08571c40-a532-4fa2-a18b-c7e583a29609"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 18:00:23.399121 ip-10-0-141-180 kubenswrapper[2581]: I0910 18:00:23.399089    2581 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/08571c40-a532-4fa2-a18b-c7e583a29609-istio-data\") on node \"ip-10-0-141-180.ec2.internal\" DevicePath \"\""
Sep 10 18:00:23.399121 ip-10-0-141-180 kubenswrapper[2581]: I0910 18:00:23.399113    2581 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/08571c40-a532-4fa2-a18b-c7e583a29609-istio-envoy\") on node \"ip-10-0-141-180.ec2.internal\" DevicePath \"\""
Sep 10 18:00:23.399121 ip-10-0-141-180 kubenswrapper[2581]: I0910 18:00:23.399124    2581 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/08571c40-a532-4fa2-a18b-c7e583a29609-istiod-ca-cert\") on node \"ip-10-0-141-180.ec2.internal\" DevicePath \"\""
Sep 10 18:00:23.399340 ip-10-0-141-180 kubenswrapper[2581]: I0910 18:00:23.399133    2581 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/08571c40-a532-4fa2-a18b-c7e583a29609-istio-podinfo\") on node \"ip-10-0-141-180.ec2.internal\" DevicePath \"\""
Sep 10 18:00:23.399340 ip-10-0-141-180 kubenswrapper[2581]: I0910 18:00:23.399140    2581 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/08571c40-a532-4fa2-a18b-c7e583a29609-istio-token\") on node \"ip-10-0-141-180.ec2.internal\" DevicePath \"\""
Sep 10 18:00:23.399340 ip-10-0-141-180 kubenswrapper[2581]: I0910 18:00:23.399149    2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-h5hlx\" (UniqueName: \"kubernetes.io/projected/08571c40-a532-4fa2-a18b-c7e583a29609-kube-api-access-h5hlx\") on node \"ip-10-0-141-180.ec2.internal\" DevicePath \"\""
Sep 10 18:00:23.882115 ip-10-0-141-180 kubenswrapper[2581]: I0910 18:00:23.882077    2581 generic.go:356] "Generic (PLEG): container finished" podID="08571c40-a532-4fa2-a18b-c7e583a29609" containerID="5b7c9837102beba86605ce1da3791ab14f074b2719f75ca25e4779ef5659a90e" exitCode=0
Sep 10 18:00:23.882293 ip-10-0-141-180 kubenswrapper[2581]: I0910 18:00:23.882123    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-w2-bdf47c-openshift-default-6d77688b7b-slw4g"
Sep 10 18:00:23.882293 ip-10-0-141-180 kubenswrapper[2581]: I0910 18:00:23.882136    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w2-bdf47c-openshift-default-6d77688b7b-slw4g" event={"ID":"08571c40-a532-4fa2-a18b-c7e583a29609","Type":"ContainerDied","Data":"5b7c9837102beba86605ce1da3791ab14f074b2719f75ca25e4779ef5659a90e"}
Sep 10 18:00:23.882293 ip-10-0-141-180 kubenswrapper[2581]: I0910 18:00:23.882183    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w2-bdf47c-openshift-default-6d77688b7b-slw4g" event={"ID":"08571c40-a532-4fa2-a18b-c7e583a29609","Type":"ContainerDied","Data":"289e603f76d0b83729f8ce35371a35d5e6207d1ee115e454f868f3f7ac575a56"}
Sep 10 18:00:23.882293 ip-10-0-141-180 kubenswrapper[2581]: I0910 18:00:23.882207    2581 scope.go:117] "RemoveContainer" containerID="5b7c9837102beba86605ce1da3791ab14f074b2719f75ca25e4779ef5659a90e"
Sep 10 18:00:23.892141 ip-10-0-141-180 kubenswrapper[2581]: I0910 18:00:23.892112    2581 scope.go:117] "RemoveContainer" containerID="c1b960f1ee5824b6bc7903fca91484dd280fc7bb381f4d793972530692cbcf2c"
Sep 10 18:00:23.900708 ip-10-0-141-180 kubenswrapper[2581]: I0910 18:00:23.900682    2581 scope.go:117] "RemoveContainer" containerID="5b7c9837102beba86605ce1da3791ab14f074b2719f75ca25e4779ef5659a90e"
Sep 10 18:00:23.900981 ip-10-0-141-180 kubenswrapper[2581]: E0910 18:00:23.900963    2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5b7c9837102beba86605ce1da3791ab14f074b2719f75ca25e4779ef5659a90e\": container with ID starting with 5b7c9837102beba86605ce1da3791ab14f074b2719f75ca25e4779ef5659a90e not found: ID does not exist" containerID="5b7c9837102beba86605ce1da3791ab14f074b2719f75ca25e4779ef5659a90e"
Sep 10 18:00:23.901037 ip-10-0-141-180 kubenswrapper[2581]: I0910 18:00:23.900991    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5b7c9837102beba86605ce1da3791ab14f074b2719f75ca25e4779ef5659a90e"} err="failed to get container status \"5b7c9837102beba86605ce1da3791ab14f074b2719f75ca25e4779ef5659a90e\": rpc error: code = NotFound desc = could not find container \"5b7c9837102beba86605ce1da3791ab14f074b2719f75ca25e4779ef5659a90e\": container with ID starting with 5b7c9837102beba86605ce1da3791ab14f074b2719f75ca25e4779ef5659a90e not found: ID does not exist"
Sep 10 18:00:23.901037 ip-10-0-141-180 kubenswrapper[2581]: I0910 18:00:23.901009    2581 scope.go:117] "RemoveContainer" containerID="c1b960f1ee5824b6bc7903fca91484dd280fc7bb381f4d793972530692cbcf2c"
Sep 10 18:00:23.901264 ip-10-0-141-180 kubenswrapper[2581]: E0910 18:00:23.901243    2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c1b960f1ee5824b6bc7903fca91484dd280fc7bb381f4d793972530692cbcf2c\": container with ID starting with c1b960f1ee5824b6bc7903fca91484dd280fc7bb381f4d793972530692cbcf2c not found: ID does not exist" containerID="c1b960f1ee5824b6bc7903fca91484dd280fc7bb381f4d793972530692cbcf2c"
Sep 10 18:00:23.901327 ip-10-0-141-180 kubenswrapper[2581]: I0910 18:00:23.901274    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c1b960f1ee5824b6bc7903fca91484dd280fc7bb381f4d793972530692cbcf2c"} err="failed to get container status \"c1b960f1ee5824b6bc7903fca91484dd280fc7bb381f4d793972530692cbcf2c\": rpc error: code = NotFound desc = could not find container \"c1b960f1ee5824b6bc7903fca91484dd280fc7bb381f4d793972530692cbcf2c\": container with ID starting with c1b960f1ee5824b6bc7903fca91484dd280fc7bb381f4d793972530692cbcf2c not found: ID does not exist"
Sep 10 18:00:23.926903 ip-10-0-141-180 kubenswrapper[2581]: I0910 18:00:23.926876    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-b-w2-bdf47c-openshift-default-6d77688b7b-slw4g"]
Sep 10 18:00:23.932768 ip-10-0-141-180 kubenswrapper[2581]: I0910 18:00:23.932747    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-shared-b-w2-bdf47c-openshift-default-6d77688b7b-slw4g"]
Sep 10 18:00:24.864401 ip-10-0-141-180 kubenswrapper[2581]: I0910 18:00:24.864370    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="08571c40-a532-4fa2-a18b-c7e583a29609" path="/var/lib/kubelet/pods/08571c40-a532-4fa2-a18b-c7e583a29609/volumes"
Sep 10 18:00:30.864941 ip-10-0-141-180 kubenswrapper[2581]: I0910 18:00:30.864906    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-db978e-8545cd7bbb-fdrvn"
Sep 10 18:00:30.865328 ip-10-0-141-180 kubenswrapper[2581]: I0910 18:00:30.864986    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-ipp-b-db978e-645f9874df-sr74b"
Sep 10 18:00:31.862812 ip-10-0-141-180 kubenswrapper[2581]: I0910 18:00:31.862774    2581 patch_prober.go:28] interesting pod/e2e-ipp-b-db978e-openshift-default-6c97b9d7d6-t6j27 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.86:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Sep 10 18:00:31.862981 ip-10-0-141-180 kubenswrapper[2581]: I0910 18:00:31.862832    2581 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-ipp-b-db978e-openshift-default-6c97b9d7d6-t6j27" podUID="5d24e0bc-661e-41ca-9938-fd836289d1be" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.86:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Sep 10 18:00:33.969942 ip-10-0-141-180 kubenswrapper[2581]: I0910 18:00:33.969903    2581 generic.go:356] "Generic (PLEG): container finished" podID="5d24e0bc-661e-41ca-9938-fd836289d1be" containerID="0c294711e6337bbc6b262ab11dcb0821767f039721f43f3f224ba9cb18ae8201" exitCode=0
Sep 10 18:00:33.970428 ip-10-0-141-180 kubenswrapper[2581]: I0910 18:00:33.969972    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-b-db978e-openshift-default-6c97b9d7d6-t6j27" event={"ID":"5d24e0bc-661e-41ca-9938-fd836289d1be","Type":"ContainerDied","Data":"0c294711e6337bbc6b262ab11dcb0821767f039721f43f3f224ba9cb18ae8201"}
Sep 10 18:00:33.970428 ip-10-0-141-180 kubenswrapper[2581]: I0910 18:00:33.970396    2581 scope.go:117] "RemoveContainer" containerID="0c294711e6337bbc6b262ab11dcb0821767f039721f43f3f224ba9cb18ae8201"
Sep 10 18:00:33.971503 ip-10-0-141-180 kubenswrapper[2581]: I0910 18:00:33.971474    2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 10 18:00:33.971633 ip-10-0-141-180 kubenswrapper[2581]: I0910 18:00:33.971550    2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 10 18:00:33.971633 ip-10-0-141-180 kubenswrapper[2581]: I0910 18:00:33.971596    2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 10 18:00:34.983801 ip-10-0-141-180 kubenswrapper[2581]: I0910 18:00:34.982836    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-b-db978e-openshift-default-6c97b9d7d6-t6j27" event={"ID":"5d24e0bc-661e-41ca-9938-fd836289d1be","Type":"ContainerStarted","Data":"7d05640564d831a0f9a151413d1c30f653fa955ee2f874965053c19d44ad4531"}
Sep 10 18:00:35.136991 ip-10-0-141-180 kubenswrapper[2581]: I0910 18:00:35.136957    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ipp-b-db978e-openshift-default-6c97b9d7d6-t6j27"
Sep 10 18:00:35.137191 ip-10-0-141-180 kubenswrapper[2581]: I0910 18:00:35.137094    2581 patch_prober.go:28] interesting pod/e2e-ipp-b-db978e-openshift-default-6c97b9d7d6-t6j27 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.86:15021/healthz/ready\": dial tcp 10.133.0.86:15021: connect: connection refused" start-of-body=
Sep 10 18:00:35.137191 ip-10-0-141-180 kubenswrapper[2581]: I0910 18:00:35.137150    2581 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-b-db978e-openshift-default-6c97b9d7d6-t6j27" podUID="5d24e0bc-661e-41ca-9938-fd836289d1be" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.86:15021/healthz/ready\": dial tcp 10.133.0.86:15021: connect: connection refused"
Sep 10 18:00:36.136961 ip-10-0-141-180 kubenswrapper[2581]: I0910 18:00:36.136930    2581 patch_prober.go:28] interesting pod/e2e-ipp-b-db978e-openshift-default-6c97b9d7d6-t6j27 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.86:15021/healthz/ready\": dial tcp 10.133.0.86:15021: connect: connection refused" start-of-body=
Sep 10 18:00:36.137339 ip-10-0-141-180 kubenswrapper[2581]: I0910 18:00:36.136984    2581 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-b-db978e-openshift-default-6c97b9d7d6-t6j27" podUID="5d24e0bc-661e-41ca-9938-fd836289d1be" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.86:15021/healthz/ready\": dial tcp 10.133.0.86:15021: connect: connection refused"
Sep 10 18:00:37.136786 ip-10-0-141-180 kubenswrapper[2581]: I0910 18:00:37.136750    2581 patch_prober.go:28] interesting pod/e2e-ipp-b-db978e-openshift-default-6c97b9d7d6-t6j27 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.86:15021/healthz/ready\": dial tcp 10.133.0.86:15021: connect: connection refused" start-of-body=
Sep 10 18:00:37.136976 ip-10-0-141-180 kubenswrapper[2581]: I0910 18:00:37.136801    2581 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-b-db978e-openshift-default-6c97b9d7d6-t6j27" podUID="5d24e0bc-661e-41ca-9938-fd836289d1be" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.86:15021/healthz/ready\": dial tcp 10.133.0.86:15021: connect: connection refused"
Sep 10 18:00:38.137077 ip-10-0-141-180 kubenswrapper[2581]: I0910 18:00:38.137040    2581 patch_prober.go:28] interesting pod/e2e-ipp-b-db978e-openshift-default-6c97b9d7d6-t6j27 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.86:15021/healthz/ready\": dial tcp 10.133.0.86:15021: connect: connection refused" start-of-body=
Sep 10 18:00:38.137463 ip-10-0-141-180 kubenswrapper[2581]: I0910 18:00:38.137098    2581 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-b-db978e-openshift-default-6c97b9d7d6-t6j27" podUID="5d24e0bc-661e-41ca-9938-fd836289d1be" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.86:15021/healthz/ready\": dial tcp 10.133.0.86:15021: connect: connection refused"
Sep 10 18:00:39.137413 ip-10-0-141-180 kubenswrapper[2581]: I0910 18:00:39.137373    2581 patch_prober.go:28] interesting pod/e2e-ipp-b-db978e-openshift-default-6c97b9d7d6-t6j27 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.86:15021/healthz/ready\": dial tcp 10.133.0.86:15021: connect: connection refused" start-of-body=
Sep 10 18:00:39.137840 ip-10-0-141-180 kubenswrapper[2581]: I0910 18:00:39.137431    2581 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-b-db978e-openshift-default-6c97b9d7d6-t6j27" podUID="5d24e0bc-661e-41ca-9938-fd836289d1be" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.86:15021/healthz/ready\": dial tcp 10.133.0.86:15021: connect: connection refused"
Sep 10 18:00:40.168400 ip-10-0-141-180 kubenswrapper[2581]: I0910 18:00:40.168361    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ipp-b-db978e-openshift-default-6c97b9d7d6-t6j27"
Sep 10 18:00:40.168900 ip-10-0-141-180 kubenswrapper[2581]: I0910 18:00:40.168876    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ipp-b-db978e-openshift-default-6c97b9d7d6-t6j27"
Sep 10 18:00:40.169622 ip-10-0-141-180 kubenswrapper[2581]: I0910 18:00:40.169600    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ipp-b-db978e-openshift-default-6c97b9d7d6-t6j27"
Sep 10 18:00:45.855931 ip-10-0-141-180 kubenswrapper[2581]: I0910 18:00:45.855902    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-e2e-ipp-b-db978e-645f9874df-sr74b_29ae9709-2d36-4edb-bb33-1bd286472160/payload-processing/0.log"
Sep 10 18:01:28.738031 ip-10-0-141-180 kubenswrapper[2581]: I0910 18:01:28.738001    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-e2e-ipp-a-904bbd-6b476dbd9f-bwbmg_26b1dfe3-1c5f-4387-80d6-5f9e583261d0/payload-processing/0.log"
Sep 10 18:01:28.977253 ip-10-0-141-180 kubenswrapper[2581]: I0910 18:01:28.977227    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-e2e-ipp-b-db978e-645f9874df-sr74b_29ae9709-2d36-4edb-bb33-1bd286472160/payload-processing/0.log"
Sep 10 18:01:34.248434 ip-10-0-141-180 kubenswrapper[2581]: I0910 18:01:34.248402    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-05a81f-d74d456b-lsxs6"]
Sep 10 18:01:34.248889 ip-10-0-141-180 kubenswrapper[2581]: I0910 18:01:34.248876    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="08571c40-a532-4fa2-a18b-c7e583a29609" containerName="istio-proxy"
Sep 10 18:01:34.248889 ip-10-0-141-180 kubenswrapper[2581]: I0910 18:01:34.248891    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="08571c40-a532-4fa2-a18b-c7e583a29609" containerName="istio-proxy"
Sep 10 18:01:34.248966 ip-10-0-141-180 kubenswrapper[2581]: I0910 18:01:34.248907    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="08571c40-a532-4fa2-a18b-c7e583a29609" containerName="istio-proxy"
Sep 10 18:01:34.248966 ip-10-0-141-180 kubenswrapper[2581]: I0910 18:01:34.248913    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="08571c40-a532-4fa2-a18b-c7e583a29609" containerName="istio-proxy"
Sep 10 18:01:34.249030 ip-10-0-141-180 kubenswrapper[2581]: I0910 18:01:34.248985    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="08571c40-a532-4fa2-a18b-c7e583a29609" containerName="istio-proxy"
Sep 10 18:01:34.251867 ip-10-0-141-180 kubenswrapper[2581]: I0910 18:01:34.251852    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-05a81f-d74d456b-lsxs6"
Sep 10 18:01:34.254506 ip-10-0-141-180 kubenswrapper[2581]: I0910 18:01:34.254481    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-ipp-cleanup-05a81f\""
Sep 10 18:01:34.254653 ip-10-0-141-180 kubenswrapper[2581]: I0910 18:01:34.254528    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-ipp-cleanup-05a81f-dockercfg-hmhh9\""
Sep 10 18:01:34.263748 ip-10-0-141-180 kubenswrapper[2581]: I0910 18:01:34.263722    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-05a81f-d74d456b-lsxs6"]
Sep 10 18:01:34.405622 ip-10-0-141-180 kubenswrapper[2581]: I0910 18:01:34.405573    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zkpsq\" (UniqueName: \"kubernetes.io/projected/614c63a0-cb6a-44ec-9f05-eba83d9dee89-kube-api-access-zkpsq\") pod \"payload-pre-processing-e2e-ipp-cleanup-05a81f-d74d456b-lsxs6\" (UID: \"614c63a0-cb6a-44ec-9f05-eba83d9dee89\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-05a81f-d74d456b-lsxs6"
Sep 10 18:01:34.405622 ip-10-0-141-180 kubenswrapper[2581]: I0910 18:01:34.405624    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/614c63a0-cb6a-44ec-9f05-eba83d9dee89-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-cleanup-05a81f-d74d456b-lsxs6\" (UID: \"614c63a0-cb6a-44ec-9f05-eba83d9dee89\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-05a81f-d74d456b-lsxs6"
Sep 10 18:01:34.506612 ip-10-0-141-180 kubenswrapper[2581]: I0910 18:01:34.506519    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zkpsq\" (UniqueName: \"kubernetes.io/projected/614c63a0-cb6a-44ec-9f05-eba83d9dee89-kube-api-access-zkpsq\") pod \"payload-pre-processing-e2e-ipp-cleanup-05a81f-d74d456b-lsxs6\" (UID: \"614c63a0-cb6a-44ec-9f05-eba83d9dee89\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-05a81f-d74d456b-lsxs6"
Sep 10 18:01:34.506612 ip-10-0-141-180 kubenswrapper[2581]: I0910 18:01:34.506570    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/614c63a0-cb6a-44ec-9f05-eba83d9dee89-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-cleanup-05a81f-d74d456b-lsxs6\" (UID: \"614c63a0-cb6a-44ec-9f05-eba83d9dee89\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-05a81f-d74d456b-lsxs6"
Sep 10 18:01:34.507287 ip-10-0-141-180 kubenswrapper[2581]: I0910 18:01:34.507266    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/614c63a0-cb6a-44ec-9f05-eba83d9dee89-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-cleanup-05a81f-d74d456b-lsxs6\" (UID: \"614c63a0-cb6a-44ec-9f05-eba83d9dee89\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-05a81f-d74d456b-lsxs6"
Sep 10 18:01:34.514810 ip-10-0-141-180 kubenswrapper[2581]: I0910 18:01:34.514775    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zkpsq\" (UniqueName: \"kubernetes.io/projected/614c63a0-cb6a-44ec-9f05-eba83d9dee89-kube-api-access-zkpsq\") pod \"payload-pre-processing-e2e-ipp-cleanup-05a81f-d74d456b-lsxs6\" (UID: \"614c63a0-cb6a-44ec-9f05-eba83d9dee89\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-05a81f-d74d456b-lsxs6"
Sep 10 18:01:34.562437 ip-10-0-141-180 kubenswrapper[2581]: I0910 18:01:34.562403    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-05a81f-d74d456b-lsxs6"
Sep 10 18:01:34.842548 ip-10-0-141-180 kubenswrapper[2581]: I0910 18:01:34.842510    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-05a81f-d74d456b-lsxs6"]
Sep 10 18:01:34.843146 ip-10-0-141-180 kubenswrapper[2581]: W0910 18:01:34.843118    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod614c63a0_cb6a_44ec_9f05_eba83d9dee89.slice/crio-97a116fe6b750b22e3eb902c76b52e45c7a58194bde705b9069b4ce5245d518b WatchSource:0}: Error finding container 97a116fe6b750b22e3eb902c76b52e45c7a58194bde705b9069b4ce5245d518b: Status 404 returned error can't find the container with id 97a116fe6b750b22e3eb902c76b52e45c7a58194bde705b9069b4ce5245d518b
Sep 10 18:01:35.367350 ip-10-0-141-180 kubenswrapper[2581]: I0910 18:01:35.367309    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-05a81f-d74d456b-lsxs6" event={"ID":"614c63a0-cb6a-44ec-9f05-eba83d9dee89","Type":"ContainerStarted","Data":"c9db7dca20b716298352e4c1af96546ee3a6c867ab95f8022917e110df2f8d70"}
Sep 10 18:01:35.367813 ip-10-0-141-180 kubenswrapper[2581]: I0910 18:01:35.367371    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-05a81f-d74d456b-lsxs6" event={"ID":"614c63a0-cb6a-44ec-9f05-eba83d9dee89","Type":"ContainerStarted","Data":"97a116fe6b750b22e3eb902c76b52e45c7a58194bde705b9069b4ce5245d518b"}
Sep 10 18:01:35.367813 ip-10-0-141-180 kubenswrapper[2581]: I0910 18:01:35.367392    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-05a81f-d74d456b-lsxs6"
Sep 10 18:01:35.383033 ip-10-0-141-180 kubenswrapper[2581]: I0910 18:01:35.382989    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-05a81f-d74d456b-lsxs6" podStartSLOduration=1.382975334 podStartE2EDuration="1.382975334s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 18:01:34 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 18:01:35.380966791 +0000 UTC m=+1515.035040696" watchObservedRunningTime="2026-09-10 18:01:35.382975334 +0000 UTC m=+1515.037049255"
Sep 10 18:01:46.377427 ip-10-0-141-180 kubenswrapper[2581]: I0910 18:01:46.377351    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-05a81f-d74d456b-lsxs6"
Sep 10 18:01:58.185998 ip-10-0-141-180 kubenswrapper[2581]: I0910 18:01:58.185955    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-05a81f-xcbt4"]
Sep 10 18:01:58.186545 ip-10-0-141-180 kubenswrapper[2581]: I0910 18:01:58.186529    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="08571c40-a532-4fa2-a18b-c7e583a29609" containerName="istio-proxy"
Sep 10 18:01:58.188703 ip-10-0-141-180 kubenswrapper[2581]: I0910 18:01:58.188665    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-05a81f-xcbt4"
Sep 10 18:01:58.191387 ip-10-0-141-180 kubenswrapper[2581]: I0910 18:01:58.191365    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 10 18:01:58.191488 ip-10-0-141-180 kubenswrapper[2581]: I0910 18:01:58.191366    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-k7zj6\""
Sep 10 18:01:58.199029 ip-10-0-141-180 kubenswrapper[2581]: I0910 18:01:58.199002    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-05a81f-xcbt4"]
Sep 10 18:01:58.246823 ip-10-0-141-180 kubenswrapper[2581]: I0910 18:01:58.246786    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/2ab550e0-1073-41c0-b76b-44a2297a2852-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-cleanup-05a81f-xcbt4\" (UID: \"2ab550e0-1073-41c0-b76b-44a2297a2852\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-05a81f-xcbt4"
Sep 10 18:01:58.348186 ip-10-0-141-180 kubenswrapper[2581]: I0910 18:01:58.348149    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/2ab550e0-1073-41c0-b76b-44a2297a2852-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-cleanup-05a81f-xcbt4\" (UID: \"2ab550e0-1073-41c0-b76b-44a2297a2852\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-05a81f-xcbt4"
Sep 10 18:01:58.348819 ip-10-0-141-180 kubenswrapper[2581]: I0910 18:01:58.348798    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/2ab550e0-1073-41c0-b76b-44a2297a2852-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-cleanup-05a81f-xcbt4\" (UID: \"2ab550e0-1073-41c0-b76b-44a2297a2852\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-05a81f-xcbt4"
Sep 10 18:01:58.501857 ip-10-0-141-180 kubenswrapper[2581]: I0910 18:01:58.501821    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-05a81f-xcbt4"
Sep 10 18:01:58.775755 ip-10-0-141-180 kubenswrapper[2581]: I0910 18:01:58.775723    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-05a81f-xcbt4"]
Sep 10 18:01:58.777332 ip-10-0-141-180 kubenswrapper[2581]: W0910 18:01:58.777305    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2ab550e0_1073_41c0_b76b_44a2297a2852.slice/crio-d62be05c5e52aab1145e821654e3af0d73222457b314cbce8c2cdb8ba2d99633 WatchSource:0}: Error finding container d62be05c5e52aab1145e821654e3af0d73222457b314cbce8c2cdb8ba2d99633: Status 404 returned error can't find the container with id d62be05c5e52aab1145e821654e3af0d73222457b314cbce8c2cdb8ba2d99633
Sep 10 18:01:59.532752 ip-10-0-141-180 kubenswrapper[2581]: I0910 18:01:59.532719    2581 generic.go:356] "Generic (PLEG): container finished" podID="2ab550e0-1073-41c0-b76b-44a2297a2852" containerID="c3cd6219273dfb56c9b3656063881f98042e28f357edc12cf49c2d8196d23912" exitCode=0
Sep 10 18:01:59.533186 ip-10-0-141-180 kubenswrapper[2581]: I0910 18:01:59.532808    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-05a81f-xcbt4" event={"ID":"2ab550e0-1073-41c0-b76b-44a2297a2852","Type":"ContainerDied","Data":"c3cd6219273dfb56c9b3656063881f98042e28f357edc12cf49c2d8196d23912"}
Sep 10 18:01:59.533186 ip-10-0-141-180 kubenswrapper[2581]: I0910 18:01:59.532861    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-05a81f-xcbt4" event={"ID":"2ab550e0-1073-41c0-b76b-44a2297a2852","Type":"ContainerStarted","Data":"d62be05c5e52aab1145e821654e3af0d73222457b314cbce8c2cdb8ba2d99633"}
Sep 10 18:02:00.669477 ip-10-0-141-180 kubenswrapper[2581]: I0910 18:02:00.669448    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-05a81f-xcbt4"
Sep 10 18:02:00.771155 ip-10-0-141-180 kubenswrapper[2581]: I0910 18:02:00.771123    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/2ab550e0-1073-41c0-b76b-44a2297a2852-maas-api-service-ca\") pod \"2ab550e0-1073-41c0-b76b-44a2297a2852\" (UID: \"2ab550e0-1073-41c0-b76b-44a2297a2852\") "
Sep 10 18:02:00.771454 ip-10-0-141-180 kubenswrapper[2581]: I0910 18:02:00.771432    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2ab550e0-1073-41c0-b76b-44a2297a2852-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "2ab550e0-1073-41c0-b76b-44a2297a2852" (UID: "2ab550e0-1073-41c0-b76b-44a2297a2852"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 18:02:00.872016 ip-10-0-141-180 kubenswrapper[2581]: I0910 18:02:00.871990    2581 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/2ab550e0-1073-41c0-b76b-44a2297a2852-maas-api-service-ca\") on node \"ip-10-0-141-180.ec2.internal\" DevicePath \"\""
Sep 10 18:02:01.546423 ip-10-0-141-180 kubenswrapper[2581]: I0910 18:02:01.546393    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-05a81f-xcbt4"
Sep 10 18:02:01.546595 ip-10-0-141-180 kubenswrapper[2581]: I0910 18:02:01.546428    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-05a81f-xcbt4" event={"ID":"2ab550e0-1073-41c0-b76b-44a2297a2852","Type":"ContainerDied","Data":"d62be05c5e52aab1145e821654e3af0d73222457b314cbce8c2cdb8ba2d99633"}
Sep 10 18:02:01.546595 ip-10-0-141-180 kubenswrapper[2581]: I0910 18:02:01.546467    2581 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="d62be05c5e52aab1145e821654e3af0d73222457b314cbce8c2cdb8ba2d99633"
Sep 10 18:02:04.151940 ip-10-0-141-180 kubenswrapper[2581]: I0910 18:02:04.151903    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-05a81f-d74d456b-lsxs6"]
Sep 10 18:02:04.152429 ip-10-0-141-180 kubenswrapper[2581]: I0910 18:02:04.152207    2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-05a81f-d74d456b-lsxs6" podUID="614c63a0-cb6a-44ec-9f05-eba83d9dee89" containerName="payload-pre-processing" containerID="cri-o://c9db7dca20b716298352e4c1af96546ee3a6c867ab95f8022917e110df2f8d70" gracePeriod=30
Sep 10 18:02:04.310962 ip-10-0-141-180 kubenswrapper[2581]: I0910 18:02:04.310912    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-05a81f-xcbt4"]
Sep 10 18:02:04.313776 ip-10-0-141-180 kubenswrapper[2581]: I0910 18:02:04.313749    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-05a81f-xcbt4"]
Sep 10 18:02:04.423687 ip-10-0-141-180 kubenswrapper[2581]: I0910 18:02:04.423624    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-05a81f-d74d456b-lsxs6"
Sep 10 18:02:04.508207 ip-10-0-141-180 kubenswrapper[2581]: I0910 18:02:04.508173    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-zkpsq\" (UniqueName: \"kubernetes.io/projected/614c63a0-cb6a-44ec-9f05-eba83d9dee89-kube-api-access-zkpsq\") pod \"614c63a0-cb6a-44ec-9f05-eba83d9dee89\" (UID: \"614c63a0-cb6a-44ec-9f05-eba83d9dee89\") "
Sep 10 18:02:04.508376 ip-10-0-141-180 kubenswrapper[2581]: I0910 18:02:04.508214    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/614c63a0-cb6a-44ec-9f05-eba83d9dee89-plugins-config-volume\") pod \"614c63a0-cb6a-44ec-9f05-eba83d9dee89\" (UID: \"614c63a0-cb6a-44ec-9f05-eba83d9dee89\") "
Sep 10 18:02:04.508679 ip-10-0-141-180 kubenswrapper[2581]: I0910 18:02:04.508627    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/614c63a0-cb6a-44ec-9f05-eba83d9dee89-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "614c63a0-cb6a-44ec-9f05-eba83d9dee89" (UID: "614c63a0-cb6a-44ec-9f05-eba83d9dee89"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 18:02:04.510331 ip-10-0-141-180 kubenswrapper[2581]: I0910 18:02:04.510306    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/614c63a0-cb6a-44ec-9f05-eba83d9dee89-kube-api-access-zkpsq" (OuterVolumeSpecName: "kube-api-access-zkpsq") pod "614c63a0-cb6a-44ec-9f05-eba83d9dee89" (UID: "614c63a0-cb6a-44ec-9f05-eba83d9dee89"). InnerVolumeSpecName "kube-api-access-zkpsq". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 18:02:04.566692 ip-10-0-141-180 kubenswrapper[2581]: I0910 18:02:04.566661    2581 generic.go:356] "Generic (PLEG): container finished" podID="614c63a0-cb6a-44ec-9f05-eba83d9dee89" containerID="c9db7dca20b716298352e4c1af96546ee3a6c867ab95f8022917e110df2f8d70" exitCode=0
Sep 10 18:02:04.566860 ip-10-0-141-180 kubenswrapper[2581]: I0910 18:02:04.566704    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-05a81f-d74d456b-lsxs6"
Sep 10 18:02:04.566860 ip-10-0-141-180 kubenswrapper[2581]: I0910 18:02:04.566754    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-05a81f-d74d456b-lsxs6" event={"ID":"614c63a0-cb6a-44ec-9f05-eba83d9dee89","Type":"ContainerDied","Data":"c9db7dca20b716298352e4c1af96546ee3a6c867ab95f8022917e110df2f8d70"}
Sep 10 18:02:04.566860 ip-10-0-141-180 kubenswrapper[2581]: I0910 18:02:04.566800    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-05a81f-d74d456b-lsxs6" event={"ID":"614c63a0-cb6a-44ec-9f05-eba83d9dee89","Type":"ContainerDied","Data":"97a116fe6b750b22e3eb902c76b52e45c7a58194bde705b9069b4ce5245d518b"}
Sep 10 18:02:04.566860 ip-10-0-141-180 kubenswrapper[2581]: I0910 18:02:04.566827    2581 scope.go:117] "RemoveContainer" containerID="c9db7dca20b716298352e4c1af96546ee3a6c867ab95f8022917e110df2f8d70"
Sep 10 18:02:04.576538 ip-10-0-141-180 kubenswrapper[2581]: I0910 18:02:04.576523    2581 scope.go:117] "RemoveContainer" containerID="c9db7dca20b716298352e4c1af96546ee3a6c867ab95f8022917e110df2f8d70"
Sep 10 18:02:04.576872 ip-10-0-141-180 kubenswrapper[2581]: E0910 18:02:04.576851    2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c9db7dca20b716298352e4c1af96546ee3a6c867ab95f8022917e110df2f8d70\": container with ID starting with c9db7dca20b716298352e4c1af96546ee3a6c867ab95f8022917e110df2f8d70 not found: ID does not exist" containerID="c9db7dca20b716298352e4c1af96546ee3a6c867ab95f8022917e110df2f8d70"
Sep 10 18:02:04.576934 ip-10-0-141-180 kubenswrapper[2581]: I0910 18:02:04.576879    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c9db7dca20b716298352e4c1af96546ee3a6c867ab95f8022917e110df2f8d70"} err="failed to get container status \"c9db7dca20b716298352e4c1af96546ee3a6c867ab95f8022917e110df2f8d70\": rpc error: code = NotFound desc = could not find container \"c9db7dca20b716298352e4c1af96546ee3a6c867ab95f8022917e110df2f8d70\": container with ID starting with c9db7dca20b716298352e4c1af96546ee3a6c867ab95f8022917e110df2f8d70 not found: ID does not exist"
Sep 10 18:02:04.604662 ip-10-0-141-180 kubenswrapper[2581]: I0910 18:02:04.604621    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-05a81f-d74d456b-lsxs6"]
Sep 10 18:02:04.609924 ip-10-0-141-180 kubenswrapper[2581]: I0910 18:02:04.609900    2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-zkpsq\" (UniqueName: \"kubernetes.io/projected/614c63a0-cb6a-44ec-9f05-eba83d9dee89-kube-api-access-zkpsq\") on node \"ip-10-0-141-180.ec2.internal\" DevicePath \"\""
Sep 10 18:02:04.610023 ip-10-0-141-180 kubenswrapper[2581]: I0910 18:02:04.609928    2581 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/614c63a0-cb6a-44ec-9f05-eba83d9dee89-plugins-config-volume\") on node \"ip-10-0-141-180.ec2.internal\" DevicePath \"\""
Sep 10 18:02:04.615117 ip-10-0-141-180 kubenswrapper[2581]: I0910 18:02:04.615093    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-05a81f-d74d456b-lsxs6"]
Sep 10 18:02:04.862555 ip-10-0-141-180 kubenswrapper[2581]: I0910 18:02:04.862519    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2ab550e0-1073-41c0-b76b-44a2297a2852" path="/var/lib/kubelet/pods/2ab550e0-1073-41c0-b76b-44a2297a2852/volumes"
Sep 10 18:02:04.862943 ip-10-0-141-180 kubenswrapper[2581]: I0910 18:02:04.862927    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="614c63a0-cb6a-44ec-9f05-eba83d9dee89" path="/var/lib/kubelet/pods/614c63a0-cb6a-44ec-9f05-eba83d9dee89/volumes"
Sep 10 18:02:14.723801 ip-10-0-141-180 kubenswrapper[2581]: I0910 18:02:14.723764    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-904bbd-9k7gg"]
Sep 10 18:02:14.724280 ip-10-0-141-180 kubenswrapper[2581]: I0910 18:02:14.724267    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="614c63a0-cb6a-44ec-9f05-eba83d9dee89" containerName="payload-pre-processing"
Sep 10 18:02:14.724329 ip-10-0-141-180 kubenswrapper[2581]: I0910 18:02:14.724283    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="614c63a0-cb6a-44ec-9f05-eba83d9dee89" containerName="payload-pre-processing"
Sep 10 18:02:14.724329 ip-10-0-141-180 kubenswrapper[2581]: I0910 18:02:14.724312    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2ab550e0-1073-41c0-b76b-44a2297a2852" containerName="revoke-keys"
Sep 10 18:02:14.724329 ip-10-0-141-180 kubenswrapper[2581]: I0910 18:02:14.724318    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="2ab550e0-1073-41c0-b76b-44a2297a2852" containerName="revoke-keys"
Sep 10 18:02:14.724447 ip-10-0-141-180 kubenswrapper[2581]: I0910 18:02:14.724400    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="2ab550e0-1073-41c0-b76b-44a2297a2852" containerName="revoke-keys"
Sep 10 18:02:14.724447 ip-10-0-141-180 kubenswrapper[2581]: I0910 18:02:14.724409    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="614c63a0-cb6a-44ec-9f05-eba83d9dee89" containerName="payload-pre-processing"
Sep 10 18:02:14.727206 ip-10-0-141-180 kubenswrapper[2581]: I0910 18:02:14.727187    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-904bbd-9k7gg"
Sep 10 18:02:14.729698 ip-10-0-141-180 kubenswrapper[2581]: I0910 18:02:14.729674    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 10 18:02:14.729798 ip-10-0-141-180 kubenswrapper[2581]: I0910 18:02:14.729684    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-k7zj6\""
Sep 10 18:02:14.735225 ip-10-0-141-180 kubenswrapper[2581]: I0910 18:02:14.734840    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-904bbd-9k7gg"]
Sep 10 18:02:14.810556 ip-10-0-141-180 kubenswrapper[2581]: I0910 18:02:14.810514    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/6de6f625-a67f-46aa-834d-3e53fb214552-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-a-904bbd-9k7gg\" (UID: \"6de6f625-a67f-46aa-834d-3e53fb214552\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-904bbd-9k7gg"
Sep 10 18:02:14.911524 ip-10-0-141-180 kubenswrapper[2581]: I0910 18:02:14.911489    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/6de6f625-a67f-46aa-834d-3e53fb214552-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-a-904bbd-9k7gg\" (UID: \"6de6f625-a67f-46aa-834d-3e53fb214552\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-904bbd-9k7gg"
Sep 10 18:02:14.912144 ip-10-0-141-180 kubenswrapper[2581]: I0910 18:02:14.912122    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/6de6f625-a67f-46aa-834d-3e53fb214552-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-a-904bbd-9k7gg\" (UID: \"6de6f625-a67f-46aa-834d-3e53fb214552\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-904bbd-9k7gg"
Sep 10 18:02:15.039506 ip-10-0-141-180 kubenswrapper[2581]: I0910 18:02:15.039414    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-904bbd-9k7gg"
Sep 10 18:02:15.314093 ip-10-0-141-180 kubenswrapper[2581]: I0910 18:02:15.314065    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-904bbd-9k7gg"]
Sep 10 18:02:15.316298 ip-10-0-141-180 kubenswrapper[2581]: W0910 18:02:15.316268    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6de6f625_a67f_46aa_834d_3e53fb214552.slice/crio-ea7a56151e3bb8119d4665489fb6eaa4fdd6f54746d357a5c37fdbe6d9e12536 WatchSource:0}: Error finding container ea7a56151e3bb8119d4665489fb6eaa4fdd6f54746d357a5c37fdbe6d9e12536: Status 404 returned error can't find the container with id ea7a56151e3bb8119d4665489fb6eaa4fdd6f54746d357a5c37fdbe6d9e12536
Sep 10 18:02:15.645418 ip-10-0-141-180 kubenswrapper[2581]: I0910 18:02:15.645378    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-904bbd-9k7gg" event={"ID":"6de6f625-a67f-46aa-834d-3e53fb214552","Type":"ContainerStarted","Data":"50b41bca4fa0202f192192da05d4e67da066071249c2605791e1d4c5e8fdda8d"}
Sep 10 18:02:15.645649 ip-10-0-141-180 kubenswrapper[2581]: I0910 18:02:15.645442    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-904bbd-9k7gg" event={"ID":"6de6f625-a67f-46aa-834d-3e53fb214552","Type":"ContainerStarted","Data":"ea7a56151e3bb8119d4665489fb6eaa4fdd6f54746d357a5c37fdbe6d9e12536"}
Sep 10 18:02:15.665918 ip-10-0-141-180 kubenswrapper[2581]: I0910 18:02:15.665862    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-904bbd-9k7gg" podStartSLOduration=1.665844353 podStartE2EDuration="1.665844353s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 18:02:14 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 18:02:15.662858501 +0000 UTC m=+1555.316932406" watchObservedRunningTime="2026-09-10 18:02:15.665844353 +0000 UTC m=+1555.319918579"
Sep 10 18:02:16.653152 ip-10-0-141-180 kubenswrapper[2581]: I0910 18:02:16.653116    2581 generic.go:356] "Generic (PLEG): container finished" podID="6de6f625-a67f-46aa-834d-3e53fb214552" containerID="50b41bca4fa0202f192192da05d4e67da066071249c2605791e1d4c5e8fdda8d" exitCode=0
Sep 10 18:02:16.653536 ip-10-0-141-180 kubenswrapper[2581]: I0910 18:02:16.653211    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-904bbd-9k7gg" event={"ID":"6de6f625-a67f-46aa-834d-3e53fb214552","Type":"ContainerDied","Data":"50b41bca4fa0202f192192da05d4e67da066071249c2605791e1d4c5e8fdda8d"}
Sep 10 18:02:17.796195 ip-10-0-141-180 kubenswrapper[2581]: I0910 18:02:17.796171    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-904bbd-9k7gg"
Sep 10 18:02:17.942399 ip-10-0-141-180 kubenswrapper[2581]: I0910 18:02:17.942313    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/6de6f625-a67f-46aa-834d-3e53fb214552-maas-api-service-ca\") pod \"6de6f625-a67f-46aa-834d-3e53fb214552\" (UID: \"6de6f625-a67f-46aa-834d-3e53fb214552\") "
Sep 10 18:02:17.942747 ip-10-0-141-180 kubenswrapper[2581]: I0910 18:02:17.942728    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6de6f625-a67f-46aa-834d-3e53fb214552-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "6de6f625-a67f-46aa-834d-3e53fb214552" (UID: "6de6f625-a67f-46aa-834d-3e53fb214552"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 18:02:18.043342 ip-10-0-141-180 kubenswrapper[2581]: I0910 18:02:18.043315    2581 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/6de6f625-a67f-46aa-834d-3e53fb214552-maas-api-service-ca\") on node \"ip-10-0-141-180.ec2.internal\" DevicePath \"\""
Sep 10 18:02:18.667792 ip-10-0-141-180 kubenswrapper[2581]: I0910 18:02:18.667766    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-904bbd-9k7gg"
Sep 10 18:02:18.667974 ip-10-0-141-180 kubenswrapper[2581]: I0910 18:02:18.667767    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-904bbd-9k7gg" event={"ID":"6de6f625-a67f-46aa-834d-3e53fb214552","Type":"ContainerDied","Data":"ea7a56151e3bb8119d4665489fb6eaa4fdd6f54746d357a5c37fdbe6d9e12536"}
Sep 10 18:02:18.667974 ip-10-0-141-180 kubenswrapper[2581]: I0910 18:02:18.667892    2581 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="ea7a56151e3bb8119d4665489fb6eaa4fdd6f54746d357a5c37fdbe6d9e12536"
Sep 10 18:02:21.214389 ip-10-0-141-180 kubenswrapper[2581]: I0910 18:02:21.214346    2581 scope.go:117] "RemoveContainer" containerID="9f035bb916888bb336f1b712aedef76381ee59ca672c19a3b263f6e05c3a110a"
Sep 10 18:02:30.473704 ip-10-0-141-180 kubenswrapper[2581]: I0910 18:02:30.473668    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ipp-a-904bbd-29817720-vg6ts"]
Sep 10 18:02:30.475541 ip-10-0-141-180 kubenswrapper[2581]: I0910 18:02:30.475513    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ipp-a-904bbd-29817720-vg6ts"]
Sep 10 18:02:30.489746 ip-10-0-141-180 kubenswrapper[2581]: I0910 18:02:30.489717    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-a-904bbd-6b476dbd9f-bwbmg"]
Sep 10 18:02:30.489991 ip-10-0-141-180 kubenswrapper[2581]: I0910 18:02:30.489965    2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-ipp-a-904bbd-6b476dbd9f-bwbmg" podUID="26b1dfe3-1c5f-4387-80d6-5f9e583261d0" containerName="payload-processing" containerID="cri-o://30d463fba5f97d07b1d48cc5211a39645887a9805e77c13dcdf3632be4e43575" gracePeriod=30
Sep 10 18:02:30.612110 ip-10-0-141-180 kubenswrapper[2581]: I0910 18:02:30.612078    2581 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-processing-e2e-ipp-a-904bbd-6b476dbd9f-bwbmg" podUID="26b1dfe3-1c5f-4387-80d6-5f9e583261d0" containerName="payload-processing" probeResult="failure" output="dial tcp 10.133.0.81:9004: connect: connection refused"
Sep 10 18:02:30.672614 ip-10-0-141-180 kubenswrapper[2581]: I0910 18:02:30.672570    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-904bbd-9k7gg"]
Sep 10 18:02:30.676818 ip-10-0-141-180 kubenswrapper[2581]: I0910 18:02:30.676790    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-904bbd-9k7gg"]
Sep 10 18:02:30.752135 ip-10-0-141-180 kubenswrapper[2581]: I0910 18:02:30.752107    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-a-904bbd-6b476dbd9f-bwbmg"
Sep 10 18:02:30.754632 ip-10-0-141-180 kubenswrapper[2581]: I0910 18:02:30.754600    2581 generic.go:356] "Generic (PLEG): container finished" podID="26b1dfe3-1c5f-4387-80d6-5f9e583261d0" containerID="30d463fba5f97d07b1d48cc5211a39645887a9805e77c13dcdf3632be4e43575" exitCode=0
Sep 10 18:02:30.754833 ip-10-0-141-180 kubenswrapper[2581]: I0910 18:02:30.754812    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-a-904bbd-6b476dbd9f-bwbmg" event={"ID":"26b1dfe3-1c5f-4387-80d6-5f9e583261d0","Type":"ContainerDied","Data":"30d463fba5f97d07b1d48cc5211a39645887a9805e77c13dcdf3632be4e43575"}
Sep 10 18:02:30.754929 ip-10-0-141-180 kubenswrapper[2581]: I0910 18:02:30.754870    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-a-904bbd-6b476dbd9f-bwbmg" event={"ID":"26b1dfe3-1c5f-4387-80d6-5f9e583261d0","Type":"ContainerDied","Data":"007335568fcc8523d109d5041b2f072973b8cb20f22cb1f39fa7f5dd93adf976"}
Sep 10 18:02:30.754929 ip-10-0-141-180 kubenswrapper[2581]: I0910 18:02:30.754904    2581 scope.go:117] "RemoveContainer" containerID="30d463fba5f97d07b1d48cc5211a39645887a9805e77c13dcdf3632be4e43575"
Sep 10 18:02:30.755086 ip-10-0-141-180 kubenswrapper[2581]: I0910 18:02:30.755068    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-a-904bbd-6b476dbd9f-bwbmg"
Sep 10 18:02:30.771345 ip-10-0-141-180 kubenswrapper[2581]: I0910 18:02:30.771212    2581 scope.go:117] "RemoveContainer" containerID="30d463fba5f97d07b1d48cc5211a39645887a9805e77c13dcdf3632be4e43575"
Sep 10 18:02:30.772369 ip-10-0-141-180 kubenswrapper[2581]: E0910 18:02:30.772334    2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"30d463fba5f97d07b1d48cc5211a39645887a9805e77c13dcdf3632be4e43575\": container with ID starting with 30d463fba5f97d07b1d48cc5211a39645887a9805e77c13dcdf3632be4e43575 not found: ID does not exist" containerID="30d463fba5f97d07b1d48cc5211a39645887a9805e77c13dcdf3632be4e43575"
Sep 10 18:02:30.772477 ip-10-0-141-180 kubenswrapper[2581]: I0910 18:02:30.772382    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"30d463fba5f97d07b1d48cc5211a39645887a9805e77c13dcdf3632be4e43575"} err="failed to get container status \"30d463fba5f97d07b1d48cc5211a39645887a9805e77c13dcdf3632be4e43575\": rpc error: code = NotFound desc = could not find container \"30d463fba5f97d07b1d48cc5211a39645887a9805e77c13dcdf3632be4e43575\": container with ID starting with 30d463fba5f97d07b1d48cc5211a39645887a9805e77c13dcdf3632be4e43575 not found: ID does not exist"
Sep 10 18:02:30.863430 ip-10-0-141-180 kubenswrapper[2581]: I0910 18:02:30.863393    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="060db639-fc7f-43d0-9650-9414951bca3b" path="/var/lib/kubelet/pods/060db639-fc7f-43d0-9650-9414951bca3b/volumes"
Sep 10 18:02:30.863878 ip-10-0-141-180 kubenswrapper[2581]: I0910 18:02:30.863860    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6de6f625-a67f-46aa-834d-3e53fb214552" path="/var/lib/kubelet/pods/6de6f625-a67f-46aa-834d-3e53fb214552/volumes"
Sep 10 18:02:30.869991 ip-10-0-141-180 kubenswrapper[2581]: I0910 18:02:30.869969    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-b62h4\" (UniqueName: \"kubernetes.io/projected/26b1dfe3-1c5f-4387-80d6-5f9e583261d0-kube-api-access-b62h4\") pod \"26b1dfe3-1c5f-4387-80d6-5f9e583261d0\" (UID: \"26b1dfe3-1c5f-4387-80d6-5f9e583261d0\") "
Sep 10 18:02:30.870118 ip-10-0-141-180 kubenswrapper[2581]: I0910 18:02:30.870001    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/26b1dfe3-1c5f-4387-80d6-5f9e583261d0-plugins-config-volume\") pod \"26b1dfe3-1c5f-4387-80d6-5f9e583261d0\" (UID: \"26b1dfe3-1c5f-4387-80d6-5f9e583261d0\") "
Sep 10 18:02:30.870386 ip-10-0-141-180 kubenswrapper[2581]: I0910 18:02:30.870361    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/26b1dfe3-1c5f-4387-80d6-5f9e583261d0-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "26b1dfe3-1c5f-4387-80d6-5f9e583261d0" (UID: "26b1dfe3-1c5f-4387-80d6-5f9e583261d0"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 18:02:30.871994 ip-10-0-141-180 kubenswrapper[2581]: I0910 18:02:30.871968    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/26b1dfe3-1c5f-4387-80d6-5f9e583261d0-kube-api-access-b62h4" (OuterVolumeSpecName: "kube-api-access-b62h4") pod "26b1dfe3-1c5f-4387-80d6-5f9e583261d0" (UID: "26b1dfe3-1c5f-4387-80d6-5f9e583261d0"). InnerVolumeSpecName "kube-api-access-b62h4". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 18:02:30.971398 ip-10-0-141-180 kubenswrapper[2581]: I0910 18:02:30.971365    2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-b62h4\" (UniqueName: \"kubernetes.io/projected/26b1dfe3-1c5f-4387-80d6-5f9e583261d0-kube-api-access-b62h4\") on node \"ip-10-0-141-180.ec2.internal\" DevicePath \"\""
Sep 10 18:02:30.971398 ip-10-0-141-180 kubenswrapper[2581]: I0910 18:02:30.971393    2581 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/26b1dfe3-1c5f-4387-80d6-5f9e583261d0-plugins-config-volume\") on node \"ip-10-0-141-180.ec2.internal\" DevicePath \"\""
Sep 10 18:02:31.078440 ip-10-0-141-180 kubenswrapper[2581]: I0910 18:02:31.078409    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-a-904bbd-6b476dbd9f-bwbmg"]
Sep 10 18:02:31.081907 ip-10-0-141-180 kubenswrapper[2581]: I0910 18:02:31.081883    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-a-904bbd-6b476dbd9f-bwbmg"]
Sep 10 18:02:32.862593 ip-10-0-141-180 kubenswrapper[2581]: I0910 18:02:32.862560    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="26b1dfe3-1c5f-4387-80d6-5f9e583261d0" path="/var/lib/kubelet/pods/26b1dfe3-1c5f-4387-80d6-5f9e583261d0/volumes"
Sep 10 18:03:34.445782 ip-10-0-141-180 kubenswrapper[2581]: I0910 18:03:34.445736    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-db978e-kzswd"]
Sep 10 18:03:34.446259 ip-10-0-141-180 kubenswrapper[2581]: I0910 18:03:34.446238    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6de6f625-a67f-46aa-834d-3e53fb214552" containerName="revoke-keys"
Sep 10 18:03:34.446259 ip-10-0-141-180 kubenswrapper[2581]: I0910 18:03:34.446250    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="6de6f625-a67f-46aa-834d-3e53fb214552" containerName="revoke-keys"
Sep 10 18:03:34.446333 ip-10-0-141-180 kubenswrapper[2581]: I0910 18:03:34.446264    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="26b1dfe3-1c5f-4387-80d6-5f9e583261d0" containerName="payload-processing"
Sep 10 18:03:34.446333 ip-10-0-141-180 kubenswrapper[2581]: I0910 18:03:34.446270    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="26b1dfe3-1c5f-4387-80d6-5f9e583261d0" containerName="payload-processing"
Sep 10 18:03:34.446402 ip-10-0-141-180 kubenswrapper[2581]: I0910 18:03:34.446348    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="6de6f625-a67f-46aa-834d-3e53fb214552" containerName="revoke-keys"
Sep 10 18:03:34.446402 ip-10-0-141-180 kubenswrapper[2581]: I0910 18:03:34.446358    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="26b1dfe3-1c5f-4387-80d6-5f9e583261d0" containerName="payload-processing"
Sep 10 18:03:34.449554 ip-10-0-141-180 kubenswrapper[2581]: I0910 18:03:34.449529    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-db978e-kzswd"
Sep 10 18:03:34.452231 ip-10-0-141-180 kubenswrapper[2581]: I0910 18:03:34.452207    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 10 18:03:34.452360 ip-10-0-141-180 kubenswrapper[2581]: I0910 18:03:34.452291    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-k7zj6\""
Sep 10 18:03:34.457301 ip-10-0-141-180 kubenswrapper[2581]: I0910 18:03:34.457276    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-db978e-kzswd"]
Sep 10 18:03:34.590433 ip-10-0-141-180 kubenswrapper[2581]: I0910 18:03:34.590387    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/fc95ce7b-bfb6-4e01-ba97-04d6b48458a1-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-b-db978e-kzswd\" (UID: \"fc95ce7b-bfb6-4e01-ba97-04d6b48458a1\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-db978e-kzswd"
Sep 10 18:03:34.691707 ip-10-0-141-180 kubenswrapper[2581]: I0910 18:03:34.691673    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/fc95ce7b-bfb6-4e01-ba97-04d6b48458a1-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-b-db978e-kzswd\" (UID: \"fc95ce7b-bfb6-4e01-ba97-04d6b48458a1\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-db978e-kzswd"
Sep 10 18:03:34.692287 ip-10-0-141-180 kubenswrapper[2581]: I0910 18:03:34.692267    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/fc95ce7b-bfb6-4e01-ba97-04d6b48458a1-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-b-db978e-kzswd\" (UID: \"fc95ce7b-bfb6-4e01-ba97-04d6b48458a1\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-db978e-kzswd"
Sep 10 18:03:34.764064 ip-10-0-141-180 kubenswrapper[2581]: I0910 18:03:34.764032    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-db978e-kzswd"
Sep 10 18:03:35.036779 ip-10-0-141-180 kubenswrapper[2581]: I0910 18:03:35.036746    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-db978e-kzswd"]
Sep 10 18:03:35.037962 ip-10-0-141-180 kubenswrapper[2581]: W0910 18:03:35.037935    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfc95ce7b_bfb6_4e01_ba97_04d6b48458a1.slice/crio-012840a1d41c883330cb6dd6911e90a782e407d47428586cf66078d312bb5cc1 WatchSource:0}: Error finding container 012840a1d41c883330cb6dd6911e90a782e407d47428586cf66078d312bb5cc1: Status 404 returned error can't find the container with id 012840a1d41c883330cb6dd6911e90a782e407d47428586cf66078d312bb5cc1
Sep 10 18:03:35.181366 ip-10-0-141-180 kubenswrapper[2581]: I0910 18:03:35.181327    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-db978e-kzswd" event={"ID":"fc95ce7b-bfb6-4e01-ba97-04d6b48458a1","Type":"ContainerStarted","Data":"012840a1d41c883330cb6dd6911e90a782e407d47428586cf66078d312bb5cc1"}
Sep 10 18:03:36.189215 ip-10-0-141-180 kubenswrapper[2581]: I0910 18:03:36.189127    2581 generic.go:356] "Generic (PLEG): container finished" podID="fc95ce7b-bfb6-4e01-ba97-04d6b48458a1" containerID="64f45c1825ae8ca368f92e4af610fa748b98ac6bae9d90c222c88d360f8de357" exitCode=0
Sep 10 18:03:36.189555 ip-10-0-141-180 kubenswrapper[2581]: I0910 18:03:36.189216    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-db978e-kzswd" event={"ID":"fc95ce7b-bfb6-4e01-ba97-04d6b48458a1","Type":"ContainerDied","Data":"64f45c1825ae8ca368f92e4af610fa748b98ac6bae9d90c222c88d360f8de357"}
Sep 10 18:03:37.333819 ip-10-0-141-180 kubenswrapper[2581]: I0910 18:03:37.333792    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-db978e-kzswd"
Sep 10 18:03:37.420383 ip-10-0-141-180 kubenswrapper[2581]: I0910 18:03:37.420347    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/fc95ce7b-bfb6-4e01-ba97-04d6b48458a1-maas-api-service-ca\") pod \"fc95ce7b-bfb6-4e01-ba97-04d6b48458a1\" (UID: \"fc95ce7b-bfb6-4e01-ba97-04d6b48458a1\") "
Sep 10 18:03:37.420754 ip-10-0-141-180 kubenswrapper[2581]: I0910 18:03:37.420730    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/fc95ce7b-bfb6-4e01-ba97-04d6b48458a1-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "fc95ce7b-bfb6-4e01-ba97-04d6b48458a1" (UID: "fc95ce7b-bfb6-4e01-ba97-04d6b48458a1"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 18:03:37.522050 ip-10-0-141-180 kubenswrapper[2581]: I0910 18:03:37.522013    2581 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/fc95ce7b-bfb6-4e01-ba97-04d6b48458a1-maas-api-service-ca\") on node \"ip-10-0-141-180.ec2.internal\" DevicePath \"\""
Sep 10 18:03:38.203141 ip-10-0-141-180 kubenswrapper[2581]: I0910 18:03:38.203112    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-db978e-kzswd"
Sep 10 18:03:38.203313 ip-10-0-141-180 kubenswrapper[2581]: I0910 18:03:38.203144    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-db978e-kzswd" event={"ID":"fc95ce7b-bfb6-4e01-ba97-04d6b48458a1","Type":"ContainerDied","Data":"012840a1d41c883330cb6dd6911e90a782e407d47428586cf66078d312bb5cc1"}
Sep 10 18:03:38.203313 ip-10-0-141-180 kubenswrapper[2581]: I0910 18:03:38.203194    2581 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="012840a1d41c883330cb6dd6911e90a782e407d47428586cf66078d312bb5cc1"
Sep 10 18:03:44.949004 ip-10-0-141-180 kubenswrapper[2581]: I0910 18:03:44.948971    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-b-db978e-645f9874df-sr74b"]
Sep 10 18:03:44.949583 ip-10-0-141-180 kubenswrapper[2581]: I0910 18:03:44.949218    2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-ipp-b-db978e-645f9874df-sr74b" podUID="29ae9709-2d36-4edb-bb33-1bd286472160" containerName="payload-processing" containerID="cri-o://d204f4833d1ebf4535fa68219d08ac801e7bd6b0289c15af1d12ff32aff2ad7b" gracePeriod=30
Sep 10 18:03:44.959736 ip-10-0-141-180 kubenswrapper[2581]: I0910 18:03:44.959655    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-b-db978e-8545cd7bbb-fdrvn"]
Sep 10 18:03:44.960207 ip-10-0-141-180 kubenswrapper[2581]: I0910 18:03:44.960145    2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-db978e-8545cd7bbb-fdrvn" podUID="a27ba0b8-011f-4f02-b629-974dde21709e" containerName="payload-pre-processing" containerID="cri-o://6d6909e5ac1075bb09ded7296368dde0ea5970fe7b4d423d3054c44c46ecf74e" gracePeriod=30
Sep 10 18:03:45.122754 ip-10-0-141-180 kubenswrapper[2581]: I0910 18:03:45.122717    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-db978e-kzswd"]
Sep 10 18:03:45.127836 ip-10-0-141-180 kubenswrapper[2581]: I0910 18:03:45.127806    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-db978e-kzswd"]
Sep 10 18:03:45.244955 ip-10-0-141-180 kubenswrapper[2581]: I0910 18:03:45.244932    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-b-db978e-645f9874df-sr74b"
Sep 10 18:03:45.247989 ip-10-0-141-180 kubenswrapper[2581]: I0910 18:03:45.247968    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-db978e-8545cd7bbb-fdrvn"
Sep 10 18:03:45.248958 ip-10-0-141-180 kubenswrapper[2581]: I0910 18:03:45.248930    2581 generic.go:356] "Generic (PLEG): container finished" podID="29ae9709-2d36-4edb-bb33-1bd286472160" containerID="d204f4833d1ebf4535fa68219d08ac801e7bd6b0289c15af1d12ff32aff2ad7b" exitCode=0
Sep 10 18:03:45.249075 ip-10-0-141-180 kubenswrapper[2581]: I0910 18:03:45.248996    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-b-db978e-645f9874df-sr74b"
Sep 10 18:03:45.249075 ip-10-0-141-180 kubenswrapper[2581]: I0910 18:03:45.249067    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-b-db978e-645f9874df-sr74b" event={"ID":"29ae9709-2d36-4edb-bb33-1bd286472160","Type":"ContainerDied","Data":"d204f4833d1ebf4535fa68219d08ac801e7bd6b0289c15af1d12ff32aff2ad7b"}
Sep 10 18:03:45.249194 ip-10-0-141-180 kubenswrapper[2581]: I0910 18:03:45.249114    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-b-db978e-645f9874df-sr74b" event={"ID":"29ae9709-2d36-4edb-bb33-1bd286472160","Type":"ContainerDied","Data":"b5694695601b0e049660e485fa8021adb8acbffc15c1bedf7edaa33060fb4c7b"}
Sep 10 18:03:45.249194 ip-10-0-141-180 kubenswrapper[2581]: I0910 18:03:45.249144    2581 scope.go:117] "RemoveContainer" containerID="d204f4833d1ebf4535fa68219d08ac801e7bd6b0289c15af1d12ff32aff2ad7b"
Sep 10 18:03:45.251213 ip-10-0-141-180 kubenswrapper[2581]: I0910 18:03:45.251187    2581 generic.go:356] "Generic (PLEG): container finished" podID="a27ba0b8-011f-4f02-b629-974dde21709e" containerID="6d6909e5ac1075bb09ded7296368dde0ea5970fe7b4d423d3054c44c46ecf74e" exitCode=0
Sep 10 18:03:45.251389 ip-10-0-141-180 kubenswrapper[2581]: I0910 18:03:45.251367    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-db978e-8545cd7bbb-fdrvn" event={"ID":"a27ba0b8-011f-4f02-b629-974dde21709e","Type":"ContainerDied","Data":"6d6909e5ac1075bb09ded7296368dde0ea5970fe7b4d423d3054c44c46ecf74e"}
Sep 10 18:03:45.251463 ip-10-0-141-180 kubenswrapper[2581]: I0910 18:03:45.251426    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-db978e-8545cd7bbb-fdrvn" event={"ID":"a27ba0b8-011f-4f02-b629-974dde21709e","Type":"ContainerDied","Data":"ce4d7c02f6a66e9e0f3841bf229d321bf5200aeb12331caed38cc9210cccbd0c"}
Sep 10 18:03:45.251525 ip-10-0-141-180 kubenswrapper[2581]: I0910 18:03:45.251493    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-db978e-8545cd7bbb-fdrvn"
Sep 10 18:03:45.263465 ip-10-0-141-180 kubenswrapper[2581]: I0910 18:03:45.263437    2581 scope.go:117] "RemoveContainer" containerID="d204f4833d1ebf4535fa68219d08ac801e7bd6b0289c15af1d12ff32aff2ad7b"
Sep 10 18:03:45.263849 ip-10-0-141-180 kubenswrapper[2581]: E0910 18:03:45.263822    2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d204f4833d1ebf4535fa68219d08ac801e7bd6b0289c15af1d12ff32aff2ad7b\": container with ID starting with d204f4833d1ebf4535fa68219d08ac801e7bd6b0289c15af1d12ff32aff2ad7b not found: ID does not exist" containerID="d204f4833d1ebf4535fa68219d08ac801e7bd6b0289c15af1d12ff32aff2ad7b"
Sep 10 18:03:45.263954 ip-10-0-141-180 kubenswrapper[2581]: I0910 18:03:45.263865    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d204f4833d1ebf4535fa68219d08ac801e7bd6b0289c15af1d12ff32aff2ad7b"} err="failed to get container status \"d204f4833d1ebf4535fa68219d08ac801e7bd6b0289c15af1d12ff32aff2ad7b\": rpc error: code = NotFound desc = could not find container \"d204f4833d1ebf4535fa68219d08ac801e7bd6b0289c15af1d12ff32aff2ad7b\": container with ID starting with d204f4833d1ebf4535fa68219d08ac801e7bd6b0289c15af1d12ff32aff2ad7b not found: ID does not exist"
Sep 10 18:03:45.263954 ip-10-0-141-180 kubenswrapper[2581]: I0910 18:03:45.263893    2581 scope.go:117] "RemoveContainer" containerID="6d6909e5ac1075bb09ded7296368dde0ea5970fe7b4d423d3054c44c46ecf74e"
Sep 10 18:03:45.285557 ip-10-0-141-180 kubenswrapper[2581]: I0910 18:03:45.285532    2581 scope.go:117] "RemoveContainer" containerID="6d6909e5ac1075bb09ded7296368dde0ea5970fe7b4d423d3054c44c46ecf74e"
Sep 10 18:03:45.286341 ip-10-0-141-180 kubenswrapper[2581]: E0910 18:03:45.286253    2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6d6909e5ac1075bb09ded7296368dde0ea5970fe7b4d423d3054c44c46ecf74e\": container with ID starting with 6d6909e5ac1075bb09ded7296368dde0ea5970fe7b4d423d3054c44c46ecf74e not found: ID does not exist" containerID="6d6909e5ac1075bb09ded7296368dde0ea5970fe7b4d423d3054c44c46ecf74e"
Sep 10 18:03:45.286341 ip-10-0-141-180 kubenswrapper[2581]: I0910 18:03:45.286308    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6d6909e5ac1075bb09ded7296368dde0ea5970fe7b4d423d3054c44c46ecf74e"} err="failed to get container status \"6d6909e5ac1075bb09ded7296368dde0ea5970fe7b4d423d3054c44c46ecf74e\": rpc error: code = NotFound desc = could not find container \"6d6909e5ac1075bb09ded7296368dde0ea5970fe7b4d423d3054c44c46ecf74e\": container with ID starting with 6d6909e5ac1075bb09ded7296368dde0ea5970fe7b4d423d3054c44c46ecf74e not found: ID does not exist"
Sep 10 18:03:45.294940 ip-10-0-141-180 kubenswrapper[2581]: I0910 18:03:45.294914    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a27ba0b8-011f-4f02-b629-974dde21709e-plugins-config-volume\") pod \"a27ba0b8-011f-4f02-b629-974dde21709e\" (UID: \"a27ba0b8-011f-4f02-b629-974dde21709e\") "
Sep 10 18:03:45.295066 ip-10-0-141-180 kubenswrapper[2581]: I0910 18:03:45.294988    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-l89mb\" (UniqueName: \"kubernetes.io/projected/29ae9709-2d36-4edb-bb33-1bd286472160-kube-api-access-l89mb\") pod \"29ae9709-2d36-4edb-bb33-1bd286472160\" (UID: \"29ae9709-2d36-4edb-bb33-1bd286472160\") "
Sep 10 18:03:45.295066 ip-10-0-141-180 kubenswrapper[2581]: I0910 18:03:45.295022    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/29ae9709-2d36-4edb-bb33-1bd286472160-plugins-config-volume\") pod \"29ae9709-2d36-4edb-bb33-1bd286472160\" (UID: \"29ae9709-2d36-4edb-bb33-1bd286472160\") "
Sep 10 18:03:45.295066 ip-10-0-141-180 kubenswrapper[2581]: I0910 18:03:45.295060    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-2gwzk\" (UniqueName: \"kubernetes.io/projected/a27ba0b8-011f-4f02-b629-974dde21709e-kube-api-access-2gwzk\") pod \"a27ba0b8-011f-4f02-b629-974dde21709e\" (UID: \"a27ba0b8-011f-4f02-b629-974dde21709e\") "
Sep 10 18:03:45.295333 ip-10-0-141-180 kubenswrapper[2581]: I0910 18:03:45.295310    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a27ba0b8-011f-4f02-b629-974dde21709e-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "a27ba0b8-011f-4f02-b629-974dde21709e" (UID: "a27ba0b8-011f-4f02-b629-974dde21709e"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 18:03:45.295497 ip-10-0-141-180 kubenswrapper[2581]: I0910 18:03:45.295483    2581 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a27ba0b8-011f-4f02-b629-974dde21709e-plugins-config-volume\") on node \"ip-10-0-141-180.ec2.internal\" DevicePath \"\""
Sep 10 18:03:45.295696 ip-10-0-141-180 kubenswrapper[2581]: I0910 18:03:45.295671    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/29ae9709-2d36-4edb-bb33-1bd286472160-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "29ae9709-2d36-4edb-bb33-1bd286472160" (UID: "29ae9709-2d36-4edb-bb33-1bd286472160"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 18:03:45.297353 ip-10-0-141-180 kubenswrapper[2581]: I0910 18:03:45.297325    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/29ae9709-2d36-4edb-bb33-1bd286472160-kube-api-access-l89mb" (OuterVolumeSpecName: "kube-api-access-l89mb") pod "29ae9709-2d36-4edb-bb33-1bd286472160" (UID: "29ae9709-2d36-4edb-bb33-1bd286472160"). InnerVolumeSpecName "kube-api-access-l89mb". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 18:03:45.297502 ip-10-0-141-180 kubenswrapper[2581]: I0910 18:03:45.297484    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a27ba0b8-011f-4f02-b629-974dde21709e-kube-api-access-2gwzk" (OuterVolumeSpecName: "kube-api-access-2gwzk") pod "a27ba0b8-011f-4f02-b629-974dde21709e" (UID: "a27ba0b8-011f-4f02-b629-974dde21709e"). InnerVolumeSpecName "kube-api-access-2gwzk". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 18:03:45.396802 ip-10-0-141-180 kubenswrapper[2581]: I0910 18:03:45.396771    2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-l89mb\" (UniqueName: \"kubernetes.io/projected/29ae9709-2d36-4edb-bb33-1bd286472160-kube-api-access-l89mb\") on node \"ip-10-0-141-180.ec2.internal\" DevicePath \"\""
Sep 10 18:03:45.396802 ip-10-0-141-180 kubenswrapper[2581]: I0910 18:03:45.396797    2581 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/29ae9709-2d36-4edb-bb33-1bd286472160-plugins-config-volume\") on node \"ip-10-0-141-180.ec2.internal\" DevicePath \"\""
Sep 10 18:03:45.396802 ip-10-0-141-180 kubenswrapper[2581]: I0910 18:03:45.396806    2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2gwzk\" (UniqueName: \"kubernetes.io/projected/a27ba0b8-011f-4f02-b629-974dde21709e-kube-api-access-2gwzk\") on node \"ip-10-0-141-180.ec2.internal\" DevicePath \"\""
Sep 10 18:03:45.575245 ip-10-0-141-180 kubenswrapper[2581]: I0910 18:03:45.575214    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-b-db978e-645f9874df-sr74b"]
Sep 10 18:03:45.581375 ip-10-0-141-180 kubenswrapper[2581]: I0910 18:03:45.581352    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-b-db978e-645f9874df-sr74b"]
Sep 10 18:03:45.591574 ip-10-0-141-180 kubenswrapper[2581]: I0910 18:03:45.591550    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-b-db978e-8545cd7bbb-fdrvn"]
Sep 10 18:03:45.593913 ip-10-0-141-180 kubenswrapper[2581]: I0910 18:03:45.593893    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-b-db978e-8545cd7bbb-fdrvn"]
Sep 10 18:03:46.862454 ip-10-0-141-180 kubenswrapper[2581]: I0910 18:03:46.862417    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="29ae9709-2d36-4edb-bb33-1bd286472160" path="/var/lib/kubelet/pods/29ae9709-2d36-4edb-bb33-1bd286472160/volumes"
Sep 10 18:03:46.862968 ip-10-0-141-180 kubenswrapper[2581]: I0910 18:03:46.862948    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a27ba0b8-011f-4f02-b629-974dde21709e" path="/var/lib/kubelet/pods/a27ba0b8-011f-4f02-b629-974dde21709e/volumes"
Sep 10 18:03:46.863322 ip-10-0-141-180 kubenswrapper[2581]: I0910 18:03:46.863303    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="fc95ce7b-bfb6-4e01-ba97-04d6b48458a1" path="/var/lib/kubelet/pods/fc95ce7b-bfb6-4e01-ba97-04d6b48458a1/volumes"
Sep 10 18:03:51.980812 ip-10-0-141-180 kubenswrapper[2581]: I0910 18:03:51.980772    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ipp-b-db978e-openshift-default-6c97b9d7d6-t6j27"]
Sep 10 18:03:51.981292 ip-10-0-141-180 kubenswrapper[2581]: I0910 18:03:51.981080    2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-ipp-b-db978e-openshift-default-6c97b9d7d6-t6j27" podUID="5d24e0bc-661e-41ca-9938-fd836289d1be" containerName="istio-proxy" containerID="cri-o://7d05640564d831a0f9a151413d1c30f653fa955ee2f874965053c19d44ad4531" gracePeriod=30
Sep 10 18:03:52.340060 ip-10-0-141-180 kubenswrapper[2581]: I0910 18:03:52.339984    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w3-18ecb2-z85jb"]
Sep 10 18:03:52.340417 ip-10-0-141-180 kubenswrapper[2581]: I0910 18:03:52.340404    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a27ba0b8-011f-4f02-b629-974dde21709e" containerName="payload-pre-processing"
Sep 10 18:03:52.340460 ip-10-0-141-180 kubenswrapper[2581]: I0910 18:03:52.340419    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="a27ba0b8-011f-4f02-b629-974dde21709e" containerName="payload-pre-processing"
Sep 10 18:03:52.340460 ip-10-0-141-180 kubenswrapper[2581]: I0910 18:03:52.340427    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="fc95ce7b-bfb6-4e01-ba97-04d6b48458a1" containerName="revoke-keys"
Sep 10 18:03:52.340460 ip-10-0-141-180 kubenswrapper[2581]: I0910 18:03:52.340433    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="fc95ce7b-bfb6-4e01-ba97-04d6b48458a1" containerName="revoke-keys"
Sep 10 18:03:52.340460 ip-10-0-141-180 kubenswrapper[2581]: I0910 18:03:52.340446    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="29ae9709-2d36-4edb-bb33-1bd286472160" containerName="payload-processing"
Sep 10 18:03:52.340460 ip-10-0-141-180 kubenswrapper[2581]: I0910 18:03:52.340451    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="29ae9709-2d36-4edb-bb33-1bd286472160" containerName="payload-processing"
Sep 10 18:03:52.340629 ip-10-0-141-180 kubenswrapper[2581]: I0910 18:03:52.340518    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="a27ba0b8-011f-4f02-b629-974dde21709e" containerName="payload-pre-processing"
Sep 10 18:03:52.340629 ip-10-0-141-180 kubenswrapper[2581]: I0910 18:03:52.340528    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="29ae9709-2d36-4edb-bb33-1bd286472160" containerName="payload-processing"
Sep 10 18:03:52.340629 ip-10-0-141-180 kubenswrapper[2581]: I0910 18:03:52.340536    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="fc95ce7b-bfb6-4e01-ba97-04d6b48458a1" containerName="revoke-keys"
Sep 10 18:03:52.344403 ip-10-0-141-180 kubenswrapper[2581]: I0910 18:03:52.344385    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w3-18ecb2-z85jb"
Sep 10 18:03:52.347139 ip-10-0-141-180 kubenswrapper[2581]: I0910 18:03:52.347115    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 10 18:03:52.347261 ip-10-0-141-180 kubenswrapper[2581]: I0910 18:03:52.347146    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-k7zj6\""
Sep 10 18:03:52.352235 ip-10-0-141-180 kubenswrapper[2581]: I0910 18:03:52.352210    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w3-18ecb2-z85jb"]
Sep 10 18:03:52.463018 ip-10-0-141-180 kubenswrapper[2581]: I0910 18:03:52.462989    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/e8f45ab9-1285-41f7-b2cd-a04ef56af3de-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-a-w3-18ecb2-z85jb\" (UID: \"e8f45ab9-1285-41f7-b2cd-a04ef56af3de\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w3-18ecb2-z85jb"
Sep 10 18:03:52.564382 ip-10-0-141-180 kubenswrapper[2581]: I0910 18:03:52.564349    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/e8f45ab9-1285-41f7-b2cd-a04ef56af3de-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-a-w3-18ecb2-z85jb\" (UID: \"e8f45ab9-1285-41f7-b2cd-a04ef56af3de\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w3-18ecb2-z85jb"
Sep 10 18:03:52.564986 ip-10-0-141-180 kubenswrapper[2581]: I0910 18:03:52.564966    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/e8f45ab9-1285-41f7-b2cd-a04ef56af3de-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-a-w3-18ecb2-z85jb\" (UID: \"e8f45ab9-1285-41f7-b2cd-a04ef56af3de\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w3-18ecb2-z85jb"
Sep 10 18:03:52.656804 ip-10-0-141-180 kubenswrapper[2581]: I0910 18:03:52.656726    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w3-18ecb2-z85jb"
Sep 10 18:03:52.933002 ip-10-0-141-180 kubenswrapper[2581]: I0910 18:03:52.932970    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w3-18ecb2-z85jb"]
Sep 10 18:03:52.933329 ip-10-0-141-180 kubenswrapper[2581]: W0910 18:03:52.933308    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode8f45ab9_1285_41f7_b2cd_a04ef56af3de.slice/crio-d705d3be2581c8dd6cba29533780aa3d921d4f94c0dd4178560288ba7c8fd466 WatchSource:0}: Error finding container d705d3be2581c8dd6cba29533780aa3d921d4f94c0dd4178560288ba7c8fd466: Status 404 returned error can't find the container with id d705d3be2581c8dd6cba29533780aa3d921d4f94c0dd4178560288ba7c8fd466
Sep 10 18:03:53.305502 ip-10-0-141-180 kubenswrapper[2581]: I0910 18:03:53.305467    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w3-18ecb2-z85jb" event={"ID":"e8f45ab9-1285-41f7-b2cd-a04ef56af3de","Type":"ContainerStarted","Data":"32eee832374e23e139e168c1ba3cfe3b03645401fcd316a7e93475fb3b8f7068"}
Sep 10 18:03:53.305836 ip-10-0-141-180 kubenswrapper[2581]: I0910 18:03:53.305535    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w3-18ecb2-z85jb" event={"ID":"e8f45ab9-1285-41f7-b2cd-a04ef56af3de","Type":"ContainerStarted","Data":"d705d3be2581c8dd6cba29533780aa3d921d4f94c0dd4178560288ba7c8fd466"}
Sep 10 18:03:53.323730 ip-10-0-141-180 kubenswrapper[2581]: I0910 18:03:53.323681    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w3-18ecb2-z85jb" podStartSLOduration=1.3236639700000001 podStartE2EDuration="1.32366397s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 18:03:52 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 18:03:53.321443031 +0000 UTC m=+1652.975516952" watchObservedRunningTime="2026-09-10 18:03:53.32366397 +0000 UTC m=+1652.977737876"
Sep 10 18:03:54.313535 ip-10-0-141-180 kubenswrapper[2581]: I0910 18:03:54.313495    2581 generic.go:356] "Generic (PLEG): container finished" podID="e8f45ab9-1285-41f7-b2cd-a04ef56af3de" containerID="32eee832374e23e139e168c1ba3cfe3b03645401fcd316a7e93475fb3b8f7068" exitCode=0
Sep 10 18:03:54.313934 ip-10-0-141-180 kubenswrapper[2581]: I0910 18:03:54.313588    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w3-18ecb2-z85jb" event={"ID":"e8f45ab9-1285-41f7-b2cd-a04ef56af3de","Type":"ContainerDied","Data":"32eee832374e23e139e168c1ba3cfe3b03645401fcd316a7e93475fb3b8f7068"}
Sep 10 18:03:55.171183 ip-10-0-141-180 kubenswrapper[2581]: I0910 18:03:55.171152    2581 patch_prober.go:28] interesting pod/e2e-ipp-b-db978e-openshift-default-6c97b9d7d6-t6j27 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Sep 10 18:03:55.171335 ip-10-0-141-180 kubenswrapper[2581]: I0910 18:03:55.171212    2581 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-ipp-b-db978e-openshift-default-6c97b9d7d6-t6j27" podUID="5d24e0bc-661e-41ca-9938-fd836289d1be" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Sep 10 18:03:55.452200 ip-10-0-141-180 kubenswrapper[2581]: I0910 18:03:55.452175    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w3-18ecb2-z85jb"
Sep 10 18:03:55.592767 ip-10-0-141-180 kubenswrapper[2581]: I0910 18:03:55.592735    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/e8f45ab9-1285-41f7-b2cd-a04ef56af3de-maas-api-service-ca\") pod \"e8f45ab9-1285-41f7-b2cd-a04ef56af3de\" (UID: \"e8f45ab9-1285-41f7-b2cd-a04ef56af3de\") "
Sep 10 18:03:55.593132 ip-10-0-141-180 kubenswrapper[2581]: I0910 18:03:55.593113    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e8f45ab9-1285-41f7-b2cd-a04ef56af3de-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "e8f45ab9-1285-41f7-b2cd-a04ef56af3de" (UID: "e8f45ab9-1285-41f7-b2cd-a04ef56af3de"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 18:03:55.694327 ip-10-0-141-180 kubenswrapper[2581]: I0910 18:03:55.694254    2581 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/e8f45ab9-1285-41f7-b2cd-a04ef56af3de-maas-api-service-ca\") on node \"ip-10-0-141-180.ec2.internal\" DevicePath \"\""
Sep 10 18:03:56.328134 ip-10-0-141-180 kubenswrapper[2581]: I0910 18:03:56.328105    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w3-18ecb2-z85jb"
Sep 10 18:03:56.328403 ip-10-0-141-180 kubenswrapper[2581]: I0910 18:03:56.328112    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w3-18ecb2-z85jb" event={"ID":"e8f45ab9-1285-41f7-b2cd-a04ef56af3de","Type":"ContainerDied","Data":"d705d3be2581c8dd6cba29533780aa3d921d4f94c0dd4178560288ba7c8fd466"}
Sep 10 18:03:56.328549 ip-10-0-141-180 kubenswrapper[2581]: I0910 18:03:56.328424    2581 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="d705d3be2581c8dd6cba29533780aa3d921d4f94c0dd4178560288ba7c8fd466"
Sep 10 18:03:57.243858 ip-10-0-141-180 kubenswrapper[2581]: I0910 18:03:57.243798    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-b-db978e-openshift-default-6c97b9d7d6-t6j27"
Sep 10 18:03:57.316021 ip-10-0-141-180 kubenswrapper[2581]: I0910 18:03:57.315984    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5d24e0bc-661e-41ca-9938-fd836289d1be-istio-envoy\") pod \"5d24e0bc-661e-41ca-9938-fd836289d1be\" (UID: \"5d24e0bc-661e-41ca-9938-fd836289d1be\") "
Sep 10 18:03:57.316187 ip-10-0-141-180 kubenswrapper[2581]: I0910 18:03:57.316039    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5d24e0bc-661e-41ca-9938-fd836289d1be-workload-socket\") pod \"5d24e0bc-661e-41ca-9938-fd836289d1be\" (UID: \"5d24e0bc-661e-41ca-9938-fd836289d1be\") "
Sep 10 18:03:57.316187 ip-10-0-141-180 kubenswrapper[2581]: I0910 18:03:57.316062    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5d24e0bc-661e-41ca-9938-fd836289d1be-istiod-ca-cert\") pod \"5d24e0bc-661e-41ca-9938-fd836289d1be\" (UID: \"5d24e0bc-661e-41ca-9938-fd836289d1be\") "
Sep 10 18:03:57.316187 ip-10-0-141-180 kubenswrapper[2581]: I0910 18:03:57.316081    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5d24e0bc-661e-41ca-9938-fd836289d1be-istio-data\") pod \"5d24e0bc-661e-41ca-9938-fd836289d1be\" (UID: \"5d24e0bc-661e-41ca-9938-fd836289d1be\") "
Sep 10 18:03:57.316187 ip-10-0-141-180 kubenswrapper[2581]: I0910 18:03:57.316120    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-bppwt\" (UniqueName: \"kubernetes.io/projected/5d24e0bc-661e-41ca-9938-fd836289d1be-kube-api-access-bppwt\") pod \"5d24e0bc-661e-41ca-9938-fd836289d1be\" (UID: \"5d24e0bc-661e-41ca-9938-fd836289d1be\") "
Sep 10 18:03:57.316187 ip-10-0-141-180 kubenswrapper[2581]: I0910 18:03:57.316140    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5d24e0bc-661e-41ca-9938-fd836289d1be-workload-certs\") pod \"5d24e0bc-661e-41ca-9938-fd836289d1be\" (UID: \"5d24e0bc-661e-41ca-9938-fd836289d1be\") "
Sep 10 18:03:57.316451 ip-10-0-141-180 kubenswrapper[2581]: I0910 18:03:57.316205    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5d24e0bc-661e-41ca-9938-fd836289d1be-istio-token\") pod \"5d24e0bc-661e-41ca-9938-fd836289d1be\" (UID: \"5d24e0bc-661e-41ca-9938-fd836289d1be\") "
Sep 10 18:03:57.316451 ip-10-0-141-180 kubenswrapper[2581]: I0910 18:03:57.316230    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5d24e0bc-661e-41ca-9938-fd836289d1be-istio-podinfo\") pod \"5d24e0bc-661e-41ca-9938-fd836289d1be\" (UID: \"5d24e0bc-661e-41ca-9938-fd836289d1be\") "
Sep 10 18:03:57.316451 ip-10-0-141-180 kubenswrapper[2581]: I0910 18:03:57.316294    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5d24e0bc-661e-41ca-9938-fd836289d1be-credential-socket\") pod \"5d24e0bc-661e-41ca-9938-fd836289d1be\" (UID: \"5d24e0bc-661e-41ca-9938-fd836289d1be\") "
Sep 10 18:03:57.316451 ip-10-0-141-180 kubenswrapper[2581]: I0910 18:03:57.316432    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5d24e0bc-661e-41ca-9938-fd836289d1be-istio-data" (OuterVolumeSpecName: "istio-data") pod "5d24e0bc-661e-41ca-9938-fd836289d1be" (UID: "5d24e0bc-661e-41ca-9938-fd836289d1be"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 18:03:57.316696 ip-10-0-141-180 kubenswrapper[2581]: I0910 18:03:57.316447    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5d24e0bc-661e-41ca-9938-fd836289d1be-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "5d24e0bc-661e-41ca-9938-fd836289d1be" (UID: "5d24e0bc-661e-41ca-9938-fd836289d1be"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 18:03:57.316696 ip-10-0-141-180 kubenswrapper[2581]: I0910 18:03:57.316521    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5d24e0bc-661e-41ca-9938-fd836289d1be-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "5d24e0bc-661e-41ca-9938-fd836289d1be" (UID: "5d24e0bc-661e-41ca-9938-fd836289d1be"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 18:03:57.316696 ip-10-0-141-180 kubenswrapper[2581]: I0910 18:03:57.316560    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5d24e0bc-661e-41ca-9938-fd836289d1be-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "5d24e0bc-661e-41ca-9938-fd836289d1be" (UID: "5d24e0bc-661e-41ca-9938-fd836289d1be"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 18:03:57.316696 ip-10-0-141-180 kubenswrapper[2581]: I0910 18:03:57.316678    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5d24e0bc-661e-41ca-9938-fd836289d1be-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "5d24e0bc-661e-41ca-9938-fd836289d1be" (UID: "5d24e0bc-661e-41ca-9938-fd836289d1be"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 18:03:57.316921 ip-10-0-141-180 kubenswrapper[2581]: I0910 18:03:57.316757    2581 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5d24e0bc-661e-41ca-9938-fd836289d1be-workload-socket\") on node \"ip-10-0-141-180.ec2.internal\" DevicePath \"\""
Sep 10 18:03:57.316921 ip-10-0-141-180 kubenswrapper[2581]: I0910 18:03:57.316774    2581 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5d24e0bc-661e-41ca-9938-fd836289d1be-istiod-ca-cert\") on node \"ip-10-0-141-180.ec2.internal\" DevicePath \"\""
Sep 10 18:03:57.316921 ip-10-0-141-180 kubenswrapper[2581]: I0910 18:03:57.316790    2581 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5d24e0bc-661e-41ca-9938-fd836289d1be-istio-data\") on node \"ip-10-0-141-180.ec2.internal\" DevicePath \"\""
Sep 10 18:03:57.316921 ip-10-0-141-180 kubenswrapper[2581]: I0910 18:03:57.316803    2581 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5d24e0bc-661e-41ca-9938-fd836289d1be-workload-certs\") on node \"ip-10-0-141-180.ec2.internal\" DevicePath \"\""
Sep 10 18:03:57.316921 ip-10-0-141-180 kubenswrapper[2581]: I0910 18:03:57.316814    2581 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5d24e0bc-661e-41ca-9938-fd836289d1be-credential-socket\") on node \"ip-10-0-141-180.ec2.internal\" DevicePath \"\""
Sep 10 18:03:57.318601 ip-10-0-141-180 kubenswrapper[2581]: I0910 18:03:57.318579    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5d24e0bc-661e-41ca-9938-fd836289d1be-istio-token" (OuterVolumeSpecName: "istio-token") pod "5d24e0bc-661e-41ca-9938-fd836289d1be" (UID: "5d24e0bc-661e-41ca-9938-fd836289d1be"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 18:03:57.318732 ip-10-0-141-180 kubenswrapper[2581]: I0910 18:03:57.318621    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/5d24e0bc-661e-41ca-9938-fd836289d1be-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "5d24e0bc-661e-41ca-9938-fd836289d1be" (UID: "5d24e0bc-661e-41ca-9938-fd836289d1be"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 10 18:03:57.318792 ip-10-0-141-180 kubenswrapper[2581]: I0910 18:03:57.318747    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5d24e0bc-661e-41ca-9938-fd836289d1be-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "5d24e0bc-661e-41ca-9938-fd836289d1be" (UID: "5d24e0bc-661e-41ca-9938-fd836289d1be"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 18:03:57.318792 ip-10-0-141-180 kubenswrapper[2581]: I0910 18:03:57.318761    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5d24e0bc-661e-41ca-9938-fd836289d1be-kube-api-access-bppwt" (OuterVolumeSpecName: "kube-api-access-bppwt") pod "5d24e0bc-661e-41ca-9938-fd836289d1be" (UID: "5d24e0bc-661e-41ca-9938-fd836289d1be"). InnerVolumeSpecName "kube-api-access-bppwt". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 18:03:57.337129 ip-10-0-141-180 kubenswrapper[2581]: I0910 18:03:57.337106    2581 generic.go:356] "Generic (PLEG): container finished" podID="5d24e0bc-661e-41ca-9938-fd836289d1be" containerID="7d05640564d831a0f9a151413d1c30f653fa955ee2f874965053c19d44ad4531" exitCode=0
Sep 10 18:03:57.337228 ip-10-0-141-180 kubenswrapper[2581]: I0910 18:03:57.337153    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-b-db978e-openshift-default-6c97b9d7d6-t6j27"
Sep 10 18:03:57.337228 ip-10-0-141-180 kubenswrapper[2581]: I0910 18:03:57.337188    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-b-db978e-openshift-default-6c97b9d7d6-t6j27" event={"ID":"5d24e0bc-661e-41ca-9938-fd836289d1be","Type":"ContainerDied","Data":"7d05640564d831a0f9a151413d1c30f653fa955ee2f874965053c19d44ad4531"}
Sep 10 18:03:57.337301 ip-10-0-141-180 kubenswrapper[2581]: I0910 18:03:57.337262    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-b-db978e-openshift-default-6c97b9d7d6-t6j27" event={"ID":"5d24e0bc-661e-41ca-9938-fd836289d1be","Type":"ContainerDied","Data":"6721c7358990a151072cc41b7324e500913f0558ba4beae3c9c72ed8532ee143"}
Sep 10 18:03:57.337301 ip-10-0-141-180 kubenswrapper[2581]: I0910 18:03:57.337285    2581 scope.go:117] "RemoveContainer" containerID="7d05640564d831a0f9a151413d1c30f653fa955ee2f874965053c19d44ad4531"
Sep 10 18:03:57.347126 ip-10-0-141-180 kubenswrapper[2581]: I0910 18:03:57.347111    2581 scope.go:117] "RemoveContainer" containerID="0c294711e6337bbc6b262ab11dcb0821767f039721f43f3f224ba9cb18ae8201"
Sep 10 18:03:57.355684 ip-10-0-141-180 kubenswrapper[2581]: I0910 18:03:57.355666    2581 scope.go:117] "RemoveContainer" containerID="7d05640564d831a0f9a151413d1c30f653fa955ee2f874965053c19d44ad4531"
Sep 10 18:03:57.355954 ip-10-0-141-180 kubenswrapper[2581]: E0910 18:03:57.355939    2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7d05640564d831a0f9a151413d1c30f653fa955ee2f874965053c19d44ad4531\": container with ID starting with 7d05640564d831a0f9a151413d1c30f653fa955ee2f874965053c19d44ad4531 not found: ID does not exist" containerID="7d05640564d831a0f9a151413d1c30f653fa955ee2f874965053c19d44ad4531"
Sep 10 18:03:57.356002 ip-10-0-141-180 kubenswrapper[2581]: I0910 18:03:57.355961    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7d05640564d831a0f9a151413d1c30f653fa955ee2f874965053c19d44ad4531"} err="failed to get container status \"7d05640564d831a0f9a151413d1c30f653fa955ee2f874965053c19d44ad4531\": rpc error: code = NotFound desc = could not find container \"7d05640564d831a0f9a151413d1c30f653fa955ee2f874965053c19d44ad4531\": container with ID starting with 7d05640564d831a0f9a151413d1c30f653fa955ee2f874965053c19d44ad4531 not found: ID does not exist"
Sep 10 18:03:57.356002 ip-10-0-141-180 kubenswrapper[2581]: I0910 18:03:57.355976    2581 scope.go:117] "RemoveContainer" containerID="0c294711e6337bbc6b262ab11dcb0821767f039721f43f3f224ba9cb18ae8201"
Sep 10 18:03:57.356189 ip-10-0-141-180 kubenswrapper[2581]: E0910 18:03:57.356168    2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0c294711e6337bbc6b262ab11dcb0821767f039721f43f3f224ba9cb18ae8201\": container with ID starting with 0c294711e6337bbc6b262ab11dcb0821767f039721f43f3f224ba9cb18ae8201 not found: ID does not exist" containerID="0c294711e6337bbc6b262ab11dcb0821767f039721f43f3f224ba9cb18ae8201"
Sep 10 18:03:57.356243 ip-10-0-141-180 kubenswrapper[2581]: I0910 18:03:57.356196    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0c294711e6337bbc6b262ab11dcb0821767f039721f43f3f224ba9cb18ae8201"} err="failed to get container status \"0c294711e6337bbc6b262ab11dcb0821767f039721f43f3f224ba9cb18ae8201\": rpc error: code = NotFound desc = could not find container \"0c294711e6337bbc6b262ab11dcb0821767f039721f43f3f224ba9cb18ae8201\": container with ID starting with 0c294711e6337bbc6b262ab11dcb0821767f039721f43f3f224ba9cb18ae8201 not found: ID does not exist"
Sep 10 18:03:57.360513 ip-10-0-141-180 kubenswrapper[2581]: I0910 18:03:57.360493    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ipp-b-db978e-openshift-default-6c97b9d7d6-t6j27"]
Sep 10 18:03:57.364855 ip-10-0-141-180 kubenswrapper[2581]: I0910 18:03:57.364833    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ipp-b-db978e-openshift-default-6c97b9d7d6-t6j27"]
Sep 10 18:03:57.417385 ip-10-0-141-180 kubenswrapper[2581]: I0910 18:03:57.417354    2581 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5d24e0bc-661e-41ca-9938-fd836289d1be-istio-token\") on node \"ip-10-0-141-180.ec2.internal\" DevicePath \"\""
Sep 10 18:03:57.417385 ip-10-0-141-180 kubenswrapper[2581]: I0910 18:03:57.417385    2581 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5d24e0bc-661e-41ca-9938-fd836289d1be-istio-podinfo\") on node \"ip-10-0-141-180.ec2.internal\" DevicePath \"\""
Sep 10 18:03:57.417555 ip-10-0-141-180 kubenswrapper[2581]: I0910 18:03:57.417395    2581 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5d24e0bc-661e-41ca-9938-fd836289d1be-istio-envoy\") on node \"ip-10-0-141-180.ec2.internal\" DevicePath \"\""
Sep 10 18:03:57.417555 ip-10-0-141-180 kubenswrapper[2581]: I0910 18:03:57.417404    2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-bppwt\" (UniqueName: \"kubernetes.io/projected/5d24e0bc-661e-41ca-9938-fd836289d1be-kube-api-access-bppwt\") on node \"ip-10-0-141-180.ec2.internal\" DevicePath \"\""
Sep 10 18:03:58.862755 ip-10-0-141-180 kubenswrapper[2581]: I0910 18:03:58.862725    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5d24e0bc-661e-41ca-9938-fd836289d1be" path="/var/lib/kubelet/pods/5d24e0bc-661e-41ca-9938-fd836289d1be/volumes"
Sep 10 18:04:02.820901 ip-10-0-141-180 kubenswrapper[2581]: I0910 18:04:02.820865    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w3-18ecb2-5d975ddfbf-7hcpz"]
Sep 10 18:04:02.821286 ip-10-0-141-180 kubenswrapper[2581]: I0910 18:04:02.821126    2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-shared-a-w3-18ecb2-5d975ddfbf-7hcpz" podUID="e50a465a-d483-413a-9dbe-d165d1b844b2" containerName="payload-processing" containerID="cri-o://d76f81c2b86d3fda1967b5abd9297e55c6e7f14a5577881d62dbb9f9dd120441" gracePeriod=30
Sep 10 18:04:02.830337 ip-10-0-141-180 kubenswrapper[2581]: I0910 18:04:02.830310    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w3-18ecb2-5cc65989c4-2v267"]
Sep 10 18:04:02.830709 ip-10-0-141-180 kubenswrapper[2581]: I0910 18:04:02.830683    2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-18ecb2-5cc65989c4-2v267" podUID="bb569e16-1fa4-4e68-abe0-6cb0983c966c" containerName="payload-pre-processing" containerID="cri-o://6c70e3e090bb1ec1008eca234e88d6ba28353ce3557e1c3591a32dbc5987b291" gracePeriod=30
Sep 10 18:04:03.089850 ip-10-0-141-180 kubenswrapper[2581]: I0910 18:04:03.089819    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w3-18ecb2-z85jb"]
Sep 10 18:04:03.096055 ip-10-0-141-180 kubenswrapper[2581]: I0910 18:04:03.096028    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w3-18ecb2-z85jb"]
Sep 10 18:04:03.097600 ip-10-0-141-180 kubenswrapper[2581]: I0910 18:04:03.097579    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-a-w3-18ecb2-5d975ddfbf-7hcpz"
Sep 10 18:04:03.119752 ip-10-0-141-180 kubenswrapper[2581]: I0910 18:04:03.119733    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-18ecb2-5cc65989c4-2v267"
Sep 10 18:04:03.274512 ip-10-0-141-180 kubenswrapper[2581]: I0910 18:04:03.274475    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e50a465a-d483-413a-9dbe-d165d1b844b2-plugins-config-volume\") pod \"e50a465a-d483-413a-9dbe-d165d1b844b2\" (UID: \"e50a465a-d483-413a-9dbe-d165d1b844b2\") "
Sep 10 18:04:03.274724 ip-10-0-141-180 kubenswrapper[2581]: I0910 18:04:03.274542    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-2jqcx\" (UniqueName: \"kubernetes.io/projected/bb569e16-1fa4-4e68-abe0-6cb0983c966c-kube-api-access-2jqcx\") pod \"bb569e16-1fa4-4e68-abe0-6cb0983c966c\" (UID: \"bb569e16-1fa4-4e68-abe0-6cb0983c966c\") "
Sep 10 18:04:03.274724 ip-10-0-141-180 kubenswrapper[2581]: I0910 18:04:03.274567    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-k9rhw\" (UniqueName: \"kubernetes.io/projected/e50a465a-d483-413a-9dbe-d165d1b844b2-kube-api-access-k9rhw\") pod \"e50a465a-d483-413a-9dbe-d165d1b844b2\" (UID: \"e50a465a-d483-413a-9dbe-d165d1b844b2\") "
Sep 10 18:04:03.274724 ip-10-0-141-180 kubenswrapper[2581]: I0910 18:04:03.274682    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/bb569e16-1fa4-4e68-abe0-6cb0983c966c-plugins-config-volume\") pod \"bb569e16-1fa4-4e68-abe0-6cb0983c966c\" (UID: \"bb569e16-1fa4-4e68-abe0-6cb0983c966c\") "
Sep 10 18:04:03.275078 ip-10-0-141-180 kubenswrapper[2581]: I0910 18:04:03.275043    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e50a465a-d483-413a-9dbe-d165d1b844b2-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "e50a465a-d483-413a-9dbe-d165d1b844b2" (UID: "e50a465a-d483-413a-9dbe-d165d1b844b2"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 18:04:03.275078 ip-10-0-141-180 kubenswrapper[2581]: I0910 18:04:03.275063    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/bb569e16-1fa4-4e68-abe0-6cb0983c966c-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "bb569e16-1fa4-4e68-abe0-6cb0983c966c" (UID: "bb569e16-1fa4-4e68-abe0-6cb0983c966c"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 18:04:03.276726 ip-10-0-141-180 kubenswrapper[2581]: I0910 18:04:03.276703    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/bb569e16-1fa4-4e68-abe0-6cb0983c966c-kube-api-access-2jqcx" (OuterVolumeSpecName: "kube-api-access-2jqcx") pod "bb569e16-1fa4-4e68-abe0-6cb0983c966c" (UID: "bb569e16-1fa4-4e68-abe0-6cb0983c966c"). InnerVolumeSpecName "kube-api-access-2jqcx". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 18:04:03.276806 ip-10-0-141-180 kubenswrapper[2581]: I0910 18:04:03.276760    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e50a465a-d483-413a-9dbe-d165d1b844b2-kube-api-access-k9rhw" (OuterVolumeSpecName: "kube-api-access-k9rhw") pod "e50a465a-d483-413a-9dbe-d165d1b844b2" (UID: "e50a465a-d483-413a-9dbe-d165d1b844b2"). InnerVolumeSpecName "kube-api-access-k9rhw". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 18:04:03.376003 ip-10-0-141-180 kubenswrapper[2581]: I0910 18:04:03.375913    2581 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/bb569e16-1fa4-4e68-abe0-6cb0983c966c-plugins-config-volume\") on node \"ip-10-0-141-180.ec2.internal\" DevicePath \"\""
Sep 10 18:04:03.376003 ip-10-0-141-180 kubenswrapper[2581]: I0910 18:04:03.375945    2581 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e50a465a-d483-413a-9dbe-d165d1b844b2-plugins-config-volume\") on node \"ip-10-0-141-180.ec2.internal\" DevicePath \"\""
Sep 10 18:04:03.376003 ip-10-0-141-180 kubenswrapper[2581]: I0910 18:04:03.375959    2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2jqcx\" (UniqueName: \"kubernetes.io/projected/bb569e16-1fa4-4e68-abe0-6cb0983c966c-kube-api-access-2jqcx\") on node \"ip-10-0-141-180.ec2.internal\" DevicePath \"\""
Sep 10 18:04:03.376003 ip-10-0-141-180 kubenswrapper[2581]: I0910 18:04:03.375970    2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-k9rhw\" (UniqueName: \"kubernetes.io/projected/e50a465a-d483-413a-9dbe-d165d1b844b2-kube-api-access-k9rhw\") on node \"ip-10-0-141-180.ec2.internal\" DevicePath \"\""
Sep 10 18:04:03.379781 ip-10-0-141-180 kubenswrapper[2581]: I0910 18:04:03.379749    2581 generic.go:356] "Generic (PLEG): container finished" podID="e50a465a-d483-413a-9dbe-d165d1b844b2" containerID="d76f81c2b86d3fda1967b5abd9297e55c6e7f14a5577881d62dbb9f9dd120441" exitCode=0
Sep 10 18:04:03.379934 ip-10-0-141-180 kubenswrapper[2581]: I0910 18:04:03.379801    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-a-w3-18ecb2-5d975ddfbf-7hcpz"
Sep 10 18:04:03.379934 ip-10-0-141-180 kubenswrapper[2581]: I0910 18:04:03.379839    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-a-w3-18ecb2-5d975ddfbf-7hcpz" event={"ID":"e50a465a-d483-413a-9dbe-d165d1b844b2","Type":"ContainerDied","Data":"d76f81c2b86d3fda1967b5abd9297e55c6e7f14a5577881d62dbb9f9dd120441"}
Sep 10 18:04:03.379934 ip-10-0-141-180 kubenswrapper[2581]: I0910 18:04:03.379905    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-a-w3-18ecb2-5d975ddfbf-7hcpz" event={"ID":"e50a465a-d483-413a-9dbe-d165d1b844b2","Type":"ContainerDied","Data":"e96d7bfcba5efd675c5e53874adc243ddd5c372dc61de6cd132c9374f16b5164"}
Sep 10 18:04:03.379934 ip-10-0-141-180 kubenswrapper[2581]: I0910 18:04:03.379934    2581 scope.go:117] "RemoveContainer" containerID="d76f81c2b86d3fda1967b5abd9297e55c6e7f14a5577881d62dbb9f9dd120441"
Sep 10 18:04:03.381302 ip-10-0-141-180 kubenswrapper[2581]: I0910 18:04:03.381282    2581 generic.go:356] "Generic (PLEG): container finished" podID="bb569e16-1fa4-4e68-abe0-6cb0983c966c" containerID="6c70e3e090bb1ec1008eca234e88d6ba28353ce3557e1c3591a32dbc5987b291" exitCode=0
Sep 10 18:04:03.381406 ip-10-0-141-180 kubenswrapper[2581]: I0910 18:04:03.381322    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-18ecb2-5cc65989c4-2v267"
Sep 10 18:04:03.381406 ip-10-0-141-180 kubenswrapper[2581]: I0910 18:04:03.381373    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-18ecb2-5cc65989c4-2v267" event={"ID":"bb569e16-1fa4-4e68-abe0-6cb0983c966c","Type":"ContainerDied","Data":"6c70e3e090bb1ec1008eca234e88d6ba28353ce3557e1c3591a32dbc5987b291"}
Sep 10 18:04:03.381491 ip-10-0-141-180 kubenswrapper[2581]: I0910 18:04:03.381425    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-18ecb2-5cc65989c4-2v267" event={"ID":"bb569e16-1fa4-4e68-abe0-6cb0983c966c","Type":"ContainerDied","Data":"1f491da253d18756d55d0ae5bd3e1551762a00934219158c47a88d62b37ad289"}
Sep 10 18:04:03.389585 ip-10-0-141-180 kubenswrapper[2581]: I0910 18:04:03.389569    2581 scope.go:117] "RemoveContainer" containerID="d76f81c2b86d3fda1967b5abd9297e55c6e7f14a5577881d62dbb9f9dd120441"
Sep 10 18:04:03.389964 ip-10-0-141-180 kubenswrapper[2581]: E0910 18:04:03.389944    2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d76f81c2b86d3fda1967b5abd9297e55c6e7f14a5577881d62dbb9f9dd120441\": container with ID starting with d76f81c2b86d3fda1967b5abd9297e55c6e7f14a5577881d62dbb9f9dd120441 not found: ID does not exist" containerID="d76f81c2b86d3fda1967b5abd9297e55c6e7f14a5577881d62dbb9f9dd120441"
Sep 10 18:04:03.390064 ip-10-0-141-180 kubenswrapper[2581]: I0910 18:04:03.389971    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d76f81c2b86d3fda1967b5abd9297e55c6e7f14a5577881d62dbb9f9dd120441"} err="failed to get container status \"d76f81c2b86d3fda1967b5abd9297e55c6e7f14a5577881d62dbb9f9dd120441\": rpc error: code = NotFound desc = could not find container \"d76f81c2b86d3fda1967b5abd9297e55c6e7f14a5577881d62dbb9f9dd120441\": container with ID starting with d76f81c2b86d3fda1967b5abd9297e55c6e7f14a5577881d62dbb9f9dd120441 not found: ID does not exist"
Sep 10 18:04:03.390064 ip-10-0-141-180 kubenswrapper[2581]: I0910 18:04:03.389991    2581 scope.go:117] "RemoveContainer" containerID="6c70e3e090bb1ec1008eca234e88d6ba28353ce3557e1c3591a32dbc5987b291"
Sep 10 18:04:03.400208 ip-10-0-141-180 kubenswrapper[2581]: I0910 18:04:03.400187    2581 scope.go:117] "RemoveContainer" containerID="6c70e3e090bb1ec1008eca234e88d6ba28353ce3557e1c3591a32dbc5987b291"
Sep 10 18:04:03.400519 ip-10-0-141-180 kubenswrapper[2581]: E0910 18:04:03.400502    2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6c70e3e090bb1ec1008eca234e88d6ba28353ce3557e1c3591a32dbc5987b291\": container with ID starting with 6c70e3e090bb1ec1008eca234e88d6ba28353ce3557e1c3591a32dbc5987b291 not found: ID does not exist" containerID="6c70e3e090bb1ec1008eca234e88d6ba28353ce3557e1c3591a32dbc5987b291"
Sep 10 18:04:03.400573 ip-10-0-141-180 kubenswrapper[2581]: I0910 18:04:03.400526    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6c70e3e090bb1ec1008eca234e88d6ba28353ce3557e1c3591a32dbc5987b291"} err="failed to get container status \"6c70e3e090bb1ec1008eca234e88d6ba28353ce3557e1c3591a32dbc5987b291\": rpc error: code = NotFound desc = could not find container \"6c70e3e090bb1ec1008eca234e88d6ba28353ce3557e1c3591a32dbc5987b291\": container with ID starting with 6c70e3e090bb1ec1008eca234e88d6ba28353ce3557e1c3591a32dbc5987b291 not found: ID does not exist"
Sep 10 18:04:03.408009 ip-10-0-141-180 kubenswrapper[2581]: I0910 18:04:03.407986    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w3-18ecb2-5cc65989c4-2v267"]
Sep 10 18:04:03.414883 ip-10-0-141-180 kubenswrapper[2581]: I0910 18:04:03.414860    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w3-18ecb2-5cc65989c4-2v267"]
Sep 10 18:04:03.425456 ip-10-0-141-180 kubenswrapper[2581]: I0910 18:04:03.425434    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w3-18ecb2-5d975ddfbf-7hcpz"]
Sep 10 18:04:03.429469 ip-10-0-141-180 kubenswrapper[2581]: I0910 18:04:03.429436    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w3-18ecb2-5d975ddfbf-7hcpz"]
Sep 10 18:04:04.862458 ip-10-0-141-180 kubenswrapper[2581]: I0910 18:04:04.862426    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="bb569e16-1fa4-4e68-abe0-6cb0983c966c" path="/var/lib/kubelet/pods/bb569e16-1fa4-4e68-abe0-6cb0983c966c/volumes"
Sep 10 18:04:04.862845 ip-10-0-141-180 kubenswrapper[2581]: I0910 18:04:04.862819    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e50a465a-d483-413a-9dbe-d165d1b844b2" path="/var/lib/kubelet/pods/e50a465a-d483-413a-9dbe-d165d1b844b2/volumes"
Sep 10 18:04:04.863106 ip-10-0-141-180 kubenswrapper[2581]: I0910 18:04:04.863093    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e8f45ab9-1285-41f7-b2cd-a04ef56af3de" path="/var/lib/kubelet/pods/e8f45ab9-1285-41f7-b2cd-a04ef56af3de/volumes"
Sep 10 18:04:10.065633 ip-10-0-141-180 kubenswrapper[2581]: I0910 18:04:10.065587    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w3-57474d-jx8pq"]
Sep 10 18:04:10.066120 ip-10-0-141-180 kubenswrapper[2581]: I0910 18:04:10.066052    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5d24e0bc-661e-41ca-9938-fd836289d1be" containerName="istio-proxy"
Sep 10 18:04:10.066120 ip-10-0-141-180 kubenswrapper[2581]: I0910 18:04:10.066066    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="5d24e0bc-661e-41ca-9938-fd836289d1be" containerName="istio-proxy"
Sep 10 18:04:10.066120 ip-10-0-141-180 kubenswrapper[2581]: I0910 18:04:10.066078    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5d24e0bc-661e-41ca-9938-fd836289d1be" containerName="istio-proxy"
Sep 10 18:04:10.066120 ip-10-0-141-180 kubenswrapper[2581]: I0910 18:04:10.066084    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="5d24e0bc-661e-41ca-9938-fd836289d1be" containerName="istio-proxy"
Sep 10 18:04:10.066120 ip-10-0-141-180 kubenswrapper[2581]: I0910 18:04:10.066092    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e8f45ab9-1285-41f7-b2cd-a04ef56af3de" containerName="revoke-keys"
Sep 10 18:04:10.066120 ip-10-0-141-180 kubenswrapper[2581]: I0910 18:04:10.066098    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="e8f45ab9-1285-41f7-b2cd-a04ef56af3de" containerName="revoke-keys"
Sep 10 18:04:10.066120 ip-10-0-141-180 kubenswrapper[2581]: I0910 18:04:10.066115    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e50a465a-d483-413a-9dbe-d165d1b844b2" containerName="payload-processing"
Sep 10 18:04:10.066120 ip-10-0-141-180 kubenswrapper[2581]: I0910 18:04:10.066121    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="e50a465a-d483-413a-9dbe-d165d1b844b2" containerName="payload-processing"
Sep 10 18:04:10.066539 ip-10-0-141-180 kubenswrapper[2581]: I0910 18:04:10.066131    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="bb569e16-1fa4-4e68-abe0-6cb0983c966c" containerName="payload-pre-processing"
Sep 10 18:04:10.066539 ip-10-0-141-180 kubenswrapper[2581]: I0910 18:04:10.066136    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="bb569e16-1fa4-4e68-abe0-6cb0983c966c" containerName="payload-pre-processing"
Sep 10 18:04:10.066539 ip-10-0-141-180 kubenswrapper[2581]: I0910 18:04:10.066205    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="5d24e0bc-661e-41ca-9938-fd836289d1be" containerName="istio-proxy"
Sep 10 18:04:10.066539 ip-10-0-141-180 kubenswrapper[2581]: I0910 18:04:10.066213    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="e50a465a-d483-413a-9dbe-d165d1b844b2" containerName="payload-processing"
Sep 10 18:04:10.066539 ip-10-0-141-180 kubenswrapper[2581]: I0910 18:04:10.066218    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="e8f45ab9-1285-41f7-b2cd-a04ef56af3de" containerName="revoke-keys"
Sep 10 18:04:10.066539 ip-10-0-141-180 kubenswrapper[2581]: I0910 18:04:10.066226    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="bb569e16-1fa4-4e68-abe0-6cb0983c966c" containerName="payload-pre-processing"
Sep 10 18:04:10.070813 ip-10-0-141-180 kubenswrapper[2581]: I0910 18:04:10.070791    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w3-57474d-jx8pq"
Sep 10 18:04:10.073822 ip-10-0-141-180 kubenswrapper[2581]: I0910 18:04:10.073797    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-k7zj6\""
Sep 10 18:04:10.073937 ip-10-0-141-180 kubenswrapper[2581]: I0910 18:04:10.073886    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 10 18:04:10.077350 ip-10-0-141-180 kubenswrapper[2581]: I0910 18:04:10.077327    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w3-57474d-jx8pq"]
Sep 10 18:04:10.138741 ip-10-0-141-180 kubenswrapper[2581]: I0910 18:04:10.138710    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/2e3759bc-65c4-4e78-abec-4fc1a2dfd23a-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-b-w3-57474d-jx8pq\" (UID: \"2e3759bc-65c4-4e78-abec-4fc1a2dfd23a\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w3-57474d-jx8pq"
Sep 10 18:04:10.239472 ip-10-0-141-180 kubenswrapper[2581]: I0910 18:04:10.239417    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/2e3759bc-65c4-4e78-abec-4fc1a2dfd23a-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-b-w3-57474d-jx8pq\" (UID: \"2e3759bc-65c4-4e78-abec-4fc1a2dfd23a\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w3-57474d-jx8pq"
Sep 10 18:04:10.240357 ip-10-0-141-180 kubenswrapper[2581]: I0910 18:04:10.240330    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/2e3759bc-65c4-4e78-abec-4fc1a2dfd23a-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-b-w3-57474d-jx8pq\" (UID: \"2e3759bc-65c4-4e78-abec-4fc1a2dfd23a\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w3-57474d-jx8pq"
Sep 10 18:04:10.384338 ip-10-0-141-180 kubenswrapper[2581]: I0910 18:04:10.384257    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w3-57474d-jx8pq"
Sep 10 18:04:10.660115 ip-10-0-141-180 kubenswrapper[2581]: I0910 18:04:10.660084    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w3-57474d-jx8pq"]
Sep 10 18:04:10.662029 ip-10-0-141-180 kubenswrapper[2581]: W0910 18:04:10.661996    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2e3759bc_65c4_4e78_abec_4fc1a2dfd23a.slice/crio-7fae1fb4e55b9927c495630fac1a4eb795a1de2c2440f4e1f71d7279cdbe714e WatchSource:0}: Error finding container 7fae1fb4e55b9927c495630fac1a4eb795a1de2c2440f4e1f71d7279cdbe714e: Status 404 returned error can't find the container with id 7fae1fb4e55b9927c495630fac1a4eb795a1de2c2440f4e1f71d7279cdbe714e
Sep 10 18:04:11.436123 ip-10-0-141-180 kubenswrapper[2581]: I0910 18:04:11.436087    2581 generic.go:356] "Generic (PLEG): container finished" podID="2e3759bc-65c4-4e78-abec-4fc1a2dfd23a" containerID="d0d0e6d4ad19859b181ac9985732416c04247ca4882b3fb37fffe02c8237d674" exitCode=0
Sep 10 18:04:11.436492 ip-10-0-141-180 kubenswrapper[2581]: I0910 18:04:11.436133    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w3-57474d-jx8pq" event={"ID":"2e3759bc-65c4-4e78-abec-4fc1a2dfd23a","Type":"ContainerDied","Data":"d0d0e6d4ad19859b181ac9985732416c04247ca4882b3fb37fffe02c8237d674"}
Sep 10 18:04:11.436492 ip-10-0-141-180 kubenswrapper[2581]: I0910 18:04:11.436199    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w3-57474d-jx8pq" event={"ID":"2e3759bc-65c4-4e78-abec-4fc1a2dfd23a","Type":"ContainerStarted","Data":"7fae1fb4e55b9927c495630fac1a4eb795a1de2c2440f4e1f71d7279cdbe714e"}
Sep 10 18:04:12.577551 ip-10-0-141-180 kubenswrapper[2581]: I0910 18:04:12.577529    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w3-57474d-jx8pq"
Sep 10 18:04:12.660923 ip-10-0-141-180 kubenswrapper[2581]: I0910 18:04:12.660894    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/2e3759bc-65c4-4e78-abec-4fc1a2dfd23a-maas-api-service-ca\") pod \"2e3759bc-65c4-4e78-abec-4fc1a2dfd23a\" (UID: \"2e3759bc-65c4-4e78-abec-4fc1a2dfd23a\") "
Sep 10 18:04:12.661270 ip-10-0-141-180 kubenswrapper[2581]: I0910 18:04:12.661246    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2e3759bc-65c4-4e78-abec-4fc1a2dfd23a-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "2e3759bc-65c4-4e78-abec-4fc1a2dfd23a" (UID: "2e3759bc-65c4-4e78-abec-4fc1a2dfd23a"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 18:04:12.762221 ip-10-0-141-180 kubenswrapper[2581]: I0910 18:04:12.762186    2581 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/2e3759bc-65c4-4e78-abec-4fc1a2dfd23a-maas-api-service-ca\") on node \"ip-10-0-141-180.ec2.internal\" DevicePath \"\""
Sep 10 18:04:13.450005 ip-10-0-141-180 kubenswrapper[2581]: I0910 18:04:13.449967    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w3-57474d-jx8pq" event={"ID":"2e3759bc-65c4-4e78-abec-4fc1a2dfd23a","Type":"ContainerDied","Data":"7fae1fb4e55b9927c495630fac1a4eb795a1de2c2440f4e1f71d7279cdbe714e"}
Sep 10 18:04:13.450005 ip-10-0-141-180 kubenswrapper[2581]: I0910 18:04:13.449995    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w3-57474d-jx8pq"
Sep 10 18:04:13.450211 ip-10-0-141-180 kubenswrapper[2581]: I0910 18:04:13.450044    2581 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="7fae1fb4e55b9927c495630fac1a4eb795a1de2c2440f4e1f71d7279cdbe714e"
Sep 10 18:04:20.769384 ip-10-0-141-180 kubenswrapper[2581]: I0910 18:04:20.769352    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w3-57474d-29817720-69zzz"]
Sep 10 18:04:20.771818 ip-10-0-141-180 kubenswrapper[2581]: I0910 18:04:20.771791    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w3-57474d-29817720-69zzz"]
Sep 10 18:04:20.862916 ip-10-0-141-180 kubenswrapper[2581]: I0910 18:04:20.862881    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="81eeb254-adb2-42c9-9318-44a17603c3e3" path="/var/lib/kubelet/pods/81eeb254-adb2-42c9-9318-44a17603c3e3/volumes"
Sep 10 18:04:20.984025 ip-10-0-141-180 kubenswrapper[2581]: I0910 18:04:20.983986    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w3-57474d-jx8pq"]
Sep 10 18:04:20.986485 ip-10-0-141-180 kubenswrapper[2581]: I0910 18:04:20.986460    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w3-57474d-jx8pq"]
Sep 10 18:04:22.861751 ip-10-0-141-180 kubenswrapper[2581]: I0910 18:04:22.861717    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2e3759bc-65c4-4e78-abec-4fc1a2dfd23a" path="/var/lib/kubelet/pods/2e3759bc-65c4-4e78-abec-4fc1a2dfd23a/volumes"
Sep 10 18:04:27.926394 ip-10-0-141-180 kubenswrapper[2581]: I0910 18:04:27.926362    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-b-w3-57474d-openshift-default-5c978cd6bd-d8phk"]
Sep 10 18:04:27.926844 ip-10-0-141-180 kubenswrapper[2581]: I0910 18:04:27.926576    2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-shared-b-w3-57474d-openshift-default-5c978cd6bd-d8phk" podUID="0a789c73-9891-4c17-bfa0-948af3f241eb" containerName="istio-proxy" containerID="cri-o://4947a863b2ef1bf41ef24ab6809a60274ea684fc5176358ab3e9e5c29dab3c6a" gracePeriod=30
Sep 10 18:04:29.881420 ip-10-0-141-180 kubenswrapper[2581]: I0910 18:04:29.881382    2581 patch_prober.go:28] interesting pod/e2e-shared-b-w3-57474d-openshift-default-5c978cd6bd-d8phk container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Sep 10 18:04:29.881815 ip-10-0-141-180 kubenswrapper[2581]: I0910 18:04:29.881447    2581 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-shared-b-w3-57474d-openshift-default-5c978cd6bd-d8phk" podUID="0a789c73-9891-4c17-bfa0-948af3f241eb" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Sep 10 18:04:30.578100 ip-10-0-141-180 kubenswrapper[2581]: I0910 18:04:30.578055    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-75895f5d8-524zp"]
Sep 10 18:04:30.578400 ip-10-0-141-180 kubenswrapper[2581]: I0910 18:04:30.578343    2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/maas-controller-75895f5d8-524zp" podUID="3f80838f-1a03-4dea-8e29-4aedb344ceb2" containerName="manager" containerID="cri-o://b374738c5642ddfad89ae1603ff593c2a92dfdbdd2313de1216553dcfc06b017" gracePeriod=30
Sep 10 18:04:30.819477 ip-10-0-141-180 kubenswrapper[2581]: I0910 18:04:30.819452    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-75895f5d8-524zp"
Sep 10 18:04:30.934794 ip-10-0-141-180 kubenswrapper[2581]: I0910 18:04:30.934715    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3f80838f-1a03-4dea-8e29-4aedb344ceb2-metrics-certs\") pod \"3f80838f-1a03-4dea-8e29-4aedb344ceb2\" (UID: \"3f80838f-1a03-4dea-8e29-4aedb344ceb2\") "
Sep 10 18:04:30.934794 ip-10-0-141-180 kubenswrapper[2581]: I0910 18:04:30.934767    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-8mqtl\" (UniqueName: \"kubernetes.io/projected/3f80838f-1a03-4dea-8e29-4aedb344ceb2-kube-api-access-8mqtl\") pod \"3f80838f-1a03-4dea-8e29-4aedb344ceb2\" (UID: \"3f80838f-1a03-4dea-8e29-4aedb344ceb2\") "
Sep 10 18:04:30.934794 ip-10-0-141-180 kubenswrapper[2581]: I0910 18:04:30.934795    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/3f80838f-1a03-4dea-8e29-4aedb344ceb2-webhook-cert\") pod \"3f80838f-1a03-4dea-8e29-4aedb344ceb2\" (UID: \"3f80838f-1a03-4dea-8e29-4aedb344ceb2\") "
Sep 10 18:04:30.937061 ip-10-0-141-180 kubenswrapper[2581]: I0910 18:04:30.937031    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3f80838f-1a03-4dea-8e29-4aedb344ceb2-webhook-cert" (OuterVolumeSpecName: "webhook-cert") pod "3f80838f-1a03-4dea-8e29-4aedb344ceb2" (UID: "3f80838f-1a03-4dea-8e29-4aedb344ceb2"). InnerVolumeSpecName "webhook-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 18:04:30.937318 ip-10-0-141-180 kubenswrapper[2581]: I0910 18:04:30.937291    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3f80838f-1a03-4dea-8e29-4aedb344ceb2-kube-api-access-8mqtl" (OuterVolumeSpecName: "kube-api-access-8mqtl") pod "3f80838f-1a03-4dea-8e29-4aedb344ceb2" (UID: "3f80838f-1a03-4dea-8e29-4aedb344ceb2"). InnerVolumeSpecName "kube-api-access-8mqtl". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 18:04:30.937318 ip-10-0-141-180 kubenswrapper[2581]: I0910 18:04:30.937309    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3f80838f-1a03-4dea-8e29-4aedb344ceb2-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "3f80838f-1a03-4dea-8e29-4aedb344ceb2" (UID: "3f80838f-1a03-4dea-8e29-4aedb344ceb2"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 18:04:31.036420 ip-10-0-141-180 kubenswrapper[2581]: I0910 18:04:31.036389    2581 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3f80838f-1a03-4dea-8e29-4aedb344ceb2-metrics-certs\") on node \"ip-10-0-141-180.ec2.internal\" DevicePath \"\""
Sep 10 18:04:31.036420 ip-10-0-141-180 kubenswrapper[2581]: I0910 18:04:31.036418    2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8mqtl\" (UniqueName: \"kubernetes.io/projected/3f80838f-1a03-4dea-8e29-4aedb344ceb2-kube-api-access-8mqtl\") on node \"ip-10-0-141-180.ec2.internal\" DevicePath \"\""
Sep 10 18:04:31.036614 ip-10-0-141-180 kubenswrapper[2581]: I0910 18:04:31.036430    2581 reconciler_common.go:299] "Volume detached for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/3f80838f-1a03-4dea-8e29-4aedb344ceb2-webhook-cert\") on node \"ip-10-0-141-180.ec2.internal\" DevicePath \"\""
Sep 10 18:04:31.576089 ip-10-0-141-180 kubenswrapper[2581]: I0910 18:04:31.576045    2581 generic.go:356] "Generic (PLEG): container finished" podID="3f80838f-1a03-4dea-8e29-4aedb344ceb2" containerID="b374738c5642ddfad89ae1603ff593c2a92dfdbdd2313de1216553dcfc06b017" exitCode=0
Sep 10 18:04:31.576270 ip-10-0-141-180 kubenswrapper[2581]: I0910 18:04:31.576092    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-75895f5d8-524zp"
Sep 10 18:04:31.576270 ip-10-0-141-180 kubenswrapper[2581]: I0910 18:04:31.576133    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-75895f5d8-524zp" event={"ID":"3f80838f-1a03-4dea-8e29-4aedb344ceb2","Type":"ContainerDied","Data":"b374738c5642ddfad89ae1603ff593c2a92dfdbdd2313de1216553dcfc06b017"}
Sep 10 18:04:31.576270 ip-10-0-141-180 kubenswrapper[2581]: I0910 18:04:31.576195    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-75895f5d8-524zp" event={"ID":"3f80838f-1a03-4dea-8e29-4aedb344ceb2","Type":"ContainerDied","Data":"a86821e2288032daa3c766f4972e7db93cb206c047324a0de16621f41d5bcb4e"}
Sep 10 18:04:31.576270 ip-10-0-141-180 kubenswrapper[2581]: I0910 18:04:31.576227    2581 scope.go:117] "RemoveContainer" containerID="b374738c5642ddfad89ae1603ff593c2a92dfdbdd2313de1216553dcfc06b017"
Sep 10 18:04:31.587414 ip-10-0-141-180 kubenswrapper[2581]: I0910 18:04:31.587277    2581 scope.go:117] "RemoveContainer" containerID="b374738c5642ddfad89ae1603ff593c2a92dfdbdd2313de1216553dcfc06b017"
Sep 10 18:04:31.587616 ip-10-0-141-180 kubenswrapper[2581]: E0910 18:04:31.587592    2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b374738c5642ddfad89ae1603ff593c2a92dfdbdd2313de1216553dcfc06b017\": container with ID starting with b374738c5642ddfad89ae1603ff593c2a92dfdbdd2313de1216553dcfc06b017 not found: ID does not exist" containerID="b374738c5642ddfad89ae1603ff593c2a92dfdbdd2313de1216553dcfc06b017"
Sep 10 18:04:31.587695 ip-10-0-141-180 kubenswrapper[2581]: I0910 18:04:31.587630    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b374738c5642ddfad89ae1603ff593c2a92dfdbdd2313de1216553dcfc06b017"} err="failed to get container status \"b374738c5642ddfad89ae1603ff593c2a92dfdbdd2313de1216553dcfc06b017\": rpc error: code = NotFound desc = could not find container \"b374738c5642ddfad89ae1603ff593c2a92dfdbdd2313de1216553dcfc06b017\": container with ID starting with b374738c5642ddfad89ae1603ff593c2a92dfdbdd2313de1216553dcfc06b017 not found: ID does not exist"
Sep 10 18:04:31.598896 ip-10-0-141-180 kubenswrapper[2581]: I0910 18:04:31.598876    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-75895f5d8-524zp"]
Sep 10 18:04:31.605400 ip-10-0-141-180 kubenswrapper[2581]: I0910 18:04:31.605378    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-75895f5d8-524zp"]
Sep 10 18:04:32.862859 ip-10-0-141-180 kubenswrapper[2581]: I0910 18:04:32.862826    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3f80838f-1a03-4dea-8e29-4aedb344ceb2" path="/var/lib/kubelet/pods/3f80838f-1a03-4dea-8e29-4aedb344ceb2/volumes"
Sep 10 18:04:33.192821 ip-10-0-141-180 kubenswrapper[2581]: I0910 18:04:33.192796    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-w3-57474d-openshift-default-5c978cd6bd-d8phk"
Sep 10 18:04:33.258402 ip-10-0-141-180 kubenswrapper[2581]: I0910 18:04:33.258371    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-c5s49\" (UniqueName: \"kubernetes.io/projected/0a789c73-9891-4c17-bfa0-948af3f241eb-kube-api-access-c5s49\") pod \"0a789c73-9891-4c17-bfa0-948af3f241eb\" (UID: \"0a789c73-9891-4c17-bfa0-948af3f241eb\") "
Sep 10 18:04:33.258402 ip-10-0-141-180 kubenswrapper[2581]: I0910 18:04:33.258403    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0a789c73-9891-4c17-bfa0-948af3f241eb-istio-token\") pod \"0a789c73-9891-4c17-bfa0-948af3f241eb\" (UID: \"0a789c73-9891-4c17-bfa0-948af3f241eb\") "
Sep 10 18:04:33.258633 ip-10-0-141-180 kubenswrapper[2581]: I0910 18:04:33.258426    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0a789c73-9891-4c17-bfa0-948af3f241eb-istio-podinfo\") pod \"0a789c73-9891-4c17-bfa0-948af3f241eb\" (UID: \"0a789c73-9891-4c17-bfa0-948af3f241eb\") "
Sep 10 18:04:33.258633 ip-10-0-141-180 kubenswrapper[2581]: I0910 18:04:33.258463    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0a789c73-9891-4c17-bfa0-948af3f241eb-workload-socket\") pod \"0a789c73-9891-4c17-bfa0-948af3f241eb\" (UID: \"0a789c73-9891-4c17-bfa0-948af3f241eb\") "
Sep 10 18:04:33.258633 ip-10-0-141-180 kubenswrapper[2581]: I0910 18:04:33.258491    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0a789c73-9891-4c17-bfa0-948af3f241eb-istio-envoy\") pod \"0a789c73-9891-4c17-bfa0-948af3f241eb\" (UID: \"0a789c73-9891-4c17-bfa0-948af3f241eb\") "
Sep 10 18:04:33.258633 ip-10-0-141-180 kubenswrapper[2581]: I0910 18:04:33.258516    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0a789c73-9891-4c17-bfa0-948af3f241eb-istiod-ca-cert\") pod \"0a789c73-9891-4c17-bfa0-948af3f241eb\" (UID: \"0a789c73-9891-4c17-bfa0-948af3f241eb\") "
Sep 10 18:04:33.258856 ip-10-0-141-180 kubenswrapper[2581]: I0910 18:04:33.258728    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0a789c73-9891-4c17-bfa0-948af3f241eb-workload-certs\") pod \"0a789c73-9891-4c17-bfa0-948af3f241eb\" (UID: \"0a789c73-9891-4c17-bfa0-948af3f241eb\") "
Sep 10 18:04:33.258856 ip-10-0-141-180 kubenswrapper[2581]: I0910 18:04:33.258762    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0a789c73-9891-4c17-bfa0-948af3f241eb-istio-data\") pod \"0a789c73-9891-4c17-bfa0-948af3f241eb\" (UID: \"0a789c73-9891-4c17-bfa0-948af3f241eb\") "
Sep 10 18:04:33.258856 ip-10-0-141-180 kubenswrapper[2581]: I0910 18:04:33.258840    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0a789c73-9891-4c17-bfa0-948af3f241eb-credential-socket\") pod \"0a789c73-9891-4c17-bfa0-948af3f241eb\" (UID: \"0a789c73-9891-4c17-bfa0-948af3f241eb\") "
Sep 10 18:04:33.259032 ip-10-0-141-180 kubenswrapper[2581]: I0910 18:04:33.258873    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0a789c73-9891-4c17-bfa0-948af3f241eb-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "0a789c73-9891-4c17-bfa0-948af3f241eb" (UID: "0a789c73-9891-4c17-bfa0-948af3f241eb"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 18:04:33.259032 ip-10-0-141-180 kubenswrapper[2581]: I0910 18:04:33.259013    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0a789c73-9891-4c17-bfa0-948af3f241eb-istio-data" (OuterVolumeSpecName: "istio-data") pod "0a789c73-9891-4c17-bfa0-948af3f241eb" (UID: "0a789c73-9891-4c17-bfa0-948af3f241eb"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 18:04:33.259137 ip-10-0-141-180 kubenswrapper[2581]: I0910 18:04:33.259035    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0a789c73-9891-4c17-bfa0-948af3f241eb-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "0a789c73-9891-4c17-bfa0-948af3f241eb" (UID: "0a789c73-9891-4c17-bfa0-948af3f241eb"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 18:04:33.259242 ip-10-0-141-180 kubenswrapper[2581]: I0910 18:04:33.259191    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0a789c73-9891-4c17-bfa0-948af3f241eb-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "0a789c73-9891-4c17-bfa0-948af3f241eb" (UID: "0a789c73-9891-4c17-bfa0-948af3f241eb"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 18:04:33.259242 ip-10-0-141-180 kubenswrapper[2581]: I0910 18:04:33.259223    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0a789c73-9891-4c17-bfa0-948af3f241eb-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "0a789c73-9891-4c17-bfa0-948af3f241eb" (UID: "0a789c73-9891-4c17-bfa0-948af3f241eb"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 18:04:33.259242 ip-10-0-141-180 kubenswrapper[2581]: I0910 18:04:33.259231    2581 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0a789c73-9891-4c17-bfa0-948af3f241eb-workload-socket\") on node \"ip-10-0-141-180.ec2.internal\" DevicePath \"\""
Sep 10 18:04:33.259407 ip-10-0-141-180 kubenswrapper[2581]: I0910 18:04:33.259250    2581 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0a789c73-9891-4c17-bfa0-948af3f241eb-workload-certs\") on node \"ip-10-0-141-180.ec2.internal\" DevicePath \"\""
Sep 10 18:04:33.259407 ip-10-0-141-180 kubenswrapper[2581]: I0910 18:04:33.259264    2581 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0a789c73-9891-4c17-bfa0-948af3f241eb-istio-data\") on node \"ip-10-0-141-180.ec2.internal\" DevicePath \"\""
Sep 10 18:04:33.260985 ip-10-0-141-180 kubenswrapper[2581]: I0910 18:04:33.260959    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0a789c73-9891-4c17-bfa0-948af3f241eb-istio-token" (OuterVolumeSpecName: "istio-token") pod "0a789c73-9891-4c17-bfa0-948af3f241eb" (UID: "0a789c73-9891-4c17-bfa0-948af3f241eb"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 18:04:33.261253 ip-10-0-141-180 kubenswrapper[2581]: I0910 18:04:33.261235    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0a789c73-9891-4c17-bfa0-948af3f241eb-kube-api-access-c5s49" (OuterVolumeSpecName: "kube-api-access-c5s49") pod "0a789c73-9891-4c17-bfa0-948af3f241eb" (UID: "0a789c73-9891-4c17-bfa0-948af3f241eb"). InnerVolumeSpecName "kube-api-access-c5s49". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 18:04:33.261253 ip-10-0-141-180 kubenswrapper[2581]: I0910 18:04:33.261240    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/0a789c73-9891-4c17-bfa0-948af3f241eb-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "0a789c73-9891-4c17-bfa0-948af3f241eb" (UID: "0a789c73-9891-4c17-bfa0-948af3f241eb"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 10 18:04:33.261342 ip-10-0-141-180 kubenswrapper[2581]: I0910 18:04:33.261256    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0a789c73-9891-4c17-bfa0-948af3f241eb-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "0a789c73-9891-4c17-bfa0-948af3f241eb" (UID: "0a789c73-9891-4c17-bfa0-948af3f241eb"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 18:04:33.360742 ip-10-0-141-180 kubenswrapper[2581]: I0910 18:04:33.360696    2581 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0a789c73-9891-4c17-bfa0-948af3f241eb-credential-socket\") on node \"ip-10-0-141-180.ec2.internal\" DevicePath \"\""
Sep 10 18:04:33.360742 ip-10-0-141-180 kubenswrapper[2581]: I0910 18:04:33.360731    2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-c5s49\" (UniqueName: \"kubernetes.io/projected/0a789c73-9891-4c17-bfa0-948af3f241eb-kube-api-access-c5s49\") on node \"ip-10-0-141-180.ec2.internal\" DevicePath \"\""
Sep 10 18:04:33.360742 ip-10-0-141-180 kubenswrapper[2581]: I0910 18:04:33.360740    2581 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0a789c73-9891-4c17-bfa0-948af3f241eb-istio-token\") on node \"ip-10-0-141-180.ec2.internal\" DevicePath \"\""
Sep 10 18:04:33.360742 ip-10-0-141-180 kubenswrapper[2581]: I0910 18:04:33.360749    2581 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0a789c73-9891-4c17-bfa0-948af3f241eb-istio-podinfo\") on node \"ip-10-0-141-180.ec2.internal\" DevicePath \"\""
Sep 10 18:04:33.360742 ip-10-0-141-180 kubenswrapper[2581]: I0910 18:04:33.360757    2581 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0a789c73-9891-4c17-bfa0-948af3f241eb-istio-envoy\") on node \"ip-10-0-141-180.ec2.internal\" DevicePath \"\""
Sep 10 18:04:33.361206 ip-10-0-141-180 kubenswrapper[2581]: I0910 18:04:33.360764    2581 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0a789c73-9891-4c17-bfa0-948af3f241eb-istiod-ca-cert\") on node \"ip-10-0-141-180.ec2.internal\" DevicePath \"\""
Sep 10 18:04:33.596612 ip-10-0-141-180 kubenswrapper[2581]: I0910 18:04:33.596574    2581 generic.go:356] "Generic (PLEG): container finished" podID="0a789c73-9891-4c17-bfa0-948af3f241eb" containerID="4947a863b2ef1bf41ef24ab6809a60274ea684fc5176358ab3e9e5c29dab3c6a" exitCode=0
Sep 10 18:04:33.596851 ip-10-0-141-180 kubenswrapper[2581]: I0910 18:04:33.596658    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-w3-57474d-openshift-default-5c978cd6bd-d8phk"
Sep 10 18:04:33.596851 ip-10-0-141-180 kubenswrapper[2581]: I0910 18:04:33.596683    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w3-57474d-openshift-default-5c978cd6bd-d8phk" event={"ID":"0a789c73-9891-4c17-bfa0-948af3f241eb","Type":"ContainerDied","Data":"4947a863b2ef1bf41ef24ab6809a60274ea684fc5176358ab3e9e5c29dab3c6a"}
Sep 10 18:04:33.596851 ip-10-0-141-180 kubenswrapper[2581]: I0910 18:04:33.596758    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w3-57474d-openshift-default-5c978cd6bd-d8phk" event={"ID":"0a789c73-9891-4c17-bfa0-948af3f241eb","Type":"ContainerDied","Data":"30eb3bf505b452d6e140d75ed2e7021d89d1f0bff84fba422d6ba399ecb7d966"}
Sep 10 18:04:33.596851 ip-10-0-141-180 kubenswrapper[2581]: I0910 18:04:33.596793    2581 scope.go:117] "RemoveContainer" containerID="4947a863b2ef1bf41ef24ab6809a60274ea684fc5176358ab3e9e5c29dab3c6a"
Sep 10 18:04:33.610432 ip-10-0-141-180 kubenswrapper[2581]: I0910 18:04:33.610400    2581 scope.go:117] "RemoveContainer" containerID="4bacf92809286f23b0dfc3f8c197832a99d190927f6517765596b6517f0b5232"
Sep 10 18:04:33.622006 ip-10-0-141-180 kubenswrapper[2581]: I0910 18:04:33.621978    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-b-w3-57474d-openshift-default-5c978cd6bd-d8phk"]
Sep 10 18:04:33.623000 ip-10-0-141-180 kubenswrapper[2581]: I0910 18:04:33.622976    2581 scope.go:117] "RemoveContainer" containerID="4947a863b2ef1bf41ef24ab6809a60274ea684fc5176358ab3e9e5c29dab3c6a"
Sep 10 18:04:33.623404 ip-10-0-141-180 kubenswrapper[2581]: E0910 18:04:33.623374    2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4947a863b2ef1bf41ef24ab6809a60274ea684fc5176358ab3e9e5c29dab3c6a\": container with ID starting with 4947a863b2ef1bf41ef24ab6809a60274ea684fc5176358ab3e9e5c29dab3c6a not found: ID does not exist" containerID="4947a863b2ef1bf41ef24ab6809a60274ea684fc5176358ab3e9e5c29dab3c6a"
Sep 10 18:04:33.623490 ip-10-0-141-180 kubenswrapper[2581]: I0910 18:04:33.623418    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4947a863b2ef1bf41ef24ab6809a60274ea684fc5176358ab3e9e5c29dab3c6a"} err="failed to get container status \"4947a863b2ef1bf41ef24ab6809a60274ea684fc5176358ab3e9e5c29dab3c6a\": rpc error: code = NotFound desc = could not find container \"4947a863b2ef1bf41ef24ab6809a60274ea684fc5176358ab3e9e5c29dab3c6a\": container with ID starting with 4947a863b2ef1bf41ef24ab6809a60274ea684fc5176358ab3e9e5c29dab3c6a not found: ID does not exist"
Sep 10 18:04:33.623490 ip-10-0-141-180 kubenswrapper[2581]: I0910 18:04:33.623446    2581 scope.go:117] "RemoveContainer" containerID="4bacf92809286f23b0dfc3f8c197832a99d190927f6517765596b6517f0b5232"
Sep 10 18:04:33.623815 ip-10-0-141-180 kubenswrapper[2581]: E0910 18:04:33.623788    2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4bacf92809286f23b0dfc3f8c197832a99d190927f6517765596b6517f0b5232\": container with ID starting with 4bacf92809286f23b0dfc3f8c197832a99d190927f6517765596b6517f0b5232 not found: ID does not exist" containerID="4bacf92809286f23b0dfc3f8c197832a99d190927f6517765596b6517f0b5232"
Sep 10 18:04:33.623926 ip-10-0-141-180 kubenswrapper[2581]: I0910 18:04:33.623820    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4bacf92809286f23b0dfc3f8c197832a99d190927f6517765596b6517f0b5232"} err="failed to get container status \"4bacf92809286f23b0dfc3f8c197832a99d190927f6517765596b6517f0b5232\": rpc error: code = NotFound desc = could not find container \"4bacf92809286f23b0dfc3f8c197832a99d190927f6517765596b6517f0b5232\": container with ID starting with 4bacf92809286f23b0dfc3f8c197832a99d190927f6517765596b6517f0b5232 not found: ID does not exist"
Sep 10 18:04:33.627586 ip-10-0-141-180 kubenswrapper[2581]: I0910 18:04:33.627562    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-shared-b-w3-57474d-openshift-default-5c978cd6bd-d8phk"]
Sep 10 18:04:34.861921 ip-10-0-141-180 kubenswrapper[2581]: I0910 18:04:34.861892    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0a789c73-9891-4c17-bfa0-948af3f241eb" path="/var/lib/kubelet/pods/0a789c73-9891-4c17-bfa0-948af3f241eb/volumes"
Sep 10 18:05:21.289052 ip-10-0-141-180 kubenswrapper[2581]: I0910 18:05:21.289021    2581 scope.go:117] "RemoveContainer" containerID="4d131001c54eb6693b85cc0fed208ef80ceefafc59e5d933152100fc36d64908"
Sep 10 18:05:21.305006 ip-10-0-141-180 kubenswrapper[2581]: I0910 18:05:21.304983    2581 scope.go:117] "RemoveContainer" containerID="e0ed0c7ba945b8f7357d8ad597a1fcf2dee301fa6d4dcfda5c4da8234a1e1d13"
Sep 10 18:05:55.689356 ip-10-0-141-180 kubenswrapper[2581]: I0910 18:05:55.689318    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-75895f5d8-khbqm"]
Sep 10 18:05:55.691772 ip-10-0-141-180 kubenswrapper[2581]: I0910 18:05:55.689753    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2e3759bc-65c4-4e78-abec-4fc1a2dfd23a" containerName="revoke-keys"
Sep 10 18:05:55.691772 ip-10-0-141-180 kubenswrapper[2581]: I0910 18:05:55.689769    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="2e3759bc-65c4-4e78-abec-4fc1a2dfd23a" containerName="revoke-keys"
Sep 10 18:05:55.691772 ip-10-0-141-180 kubenswrapper[2581]: I0910 18:05:55.689777    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0a789c73-9891-4c17-bfa0-948af3f241eb" containerName="istio-proxy"
Sep 10 18:05:55.691772 ip-10-0-141-180 kubenswrapper[2581]: I0910 18:05:55.689783    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="0a789c73-9891-4c17-bfa0-948af3f241eb" containerName="istio-proxy"
Sep 10 18:05:55.691772 ip-10-0-141-180 kubenswrapper[2581]: I0910 18:05:55.689804    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3f80838f-1a03-4dea-8e29-4aedb344ceb2" containerName="manager"
Sep 10 18:05:55.691772 ip-10-0-141-180 kubenswrapper[2581]: I0910 18:05:55.689810    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="3f80838f-1a03-4dea-8e29-4aedb344ceb2" containerName="manager"
Sep 10 18:05:55.691772 ip-10-0-141-180 kubenswrapper[2581]: I0910 18:05:55.689816    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0a789c73-9891-4c17-bfa0-948af3f241eb" containerName="istio-proxy"
Sep 10 18:05:55.691772 ip-10-0-141-180 kubenswrapper[2581]: I0910 18:05:55.689821    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="0a789c73-9891-4c17-bfa0-948af3f241eb" containerName="istio-proxy"
Sep 10 18:05:55.691772 ip-10-0-141-180 kubenswrapper[2581]: I0910 18:05:55.689901    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="0a789c73-9891-4c17-bfa0-948af3f241eb" containerName="istio-proxy"
Sep 10 18:05:55.691772 ip-10-0-141-180 kubenswrapper[2581]: I0910 18:05:55.689908    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="3f80838f-1a03-4dea-8e29-4aedb344ceb2" containerName="manager"
Sep 10 18:05:55.691772 ip-10-0-141-180 kubenswrapper[2581]: I0910 18:05:55.689916    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="5d24e0bc-661e-41ca-9938-fd836289d1be" containerName="istio-proxy"
Sep 10 18:05:55.691772 ip-10-0-141-180 kubenswrapper[2581]: I0910 18:05:55.689922    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="2e3759bc-65c4-4e78-abec-4fc1a2dfd23a" containerName="revoke-keys"
Sep 10 18:05:55.692877 ip-10-0-141-180 kubenswrapper[2581]: I0910 18:05:55.692861    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-75895f5d8-khbqm"
Sep 10 18:05:55.696708 ip-10-0-141-180 kubenswrapper[2581]: I0910 18:05:55.696683    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-metrics-tls\""
Sep 10 18:05:55.696708 ip-10-0-141-180 kubenswrapper[2581]: I0910 18:05:55.696699    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\""
Sep 10 18:05:55.696926 ip-10-0-141-180 kubenswrapper[2581]: I0910 18:05:55.696722    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\""
Sep 10 18:05:55.696926 ip-10-0-141-180 kubenswrapper[2581]: I0910 18:05:55.696723    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-bbp7v\""
Sep 10 18:05:55.703933 ip-10-0-141-180 kubenswrapper[2581]: I0910 18:05:55.703911    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-75895f5d8-khbqm"]
Sep 10 18:05:55.806350 ip-10-0-141-180 kubenswrapper[2581]: I0910 18:05:55.806314    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6bk56\" (UniqueName: \"kubernetes.io/projected/0960dca3-d5d6-4fd4-9605-20c9ba678bb4-kube-api-access-6bk56\") pod \"maas-controller-75895f5d8-khbqm\" (UID: \"0960dca3-d5d6-4fd4-9605-20c9ba678bb4\") " pod="opendatahub/maas-controller-75895f5d8-khbqm"
Sep 10 18:05:55.806512 ip-10-0-141-180 kubenswrapper[2581]: I0910 18:05:55.806379    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/0960dca3-d5d6-4fd4-9605-20c9ba678bb4-webhook-cert\") pod \"maas-controller-75895f5d8-khbqm\" (UID: \"0960dca3-d5d6-4fd4-9605-20c9ba678bb4\") " pod="opendatahub/maas-controller-75895f5d8-khbqm"
Sep 10 18:05:55.806512 ip-10-0-141-180 kubenswrapper[2581]: I0910 18:05:55.806444    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/0960dca3-d5d6-4fd4-9605-20c9ba678bb4-metrics-certs\") pod \"maas-controller-75895f5d8-khbqm\" (UID: \"0960dca3-d5d6-4fd4-9605-20c9ba678bb4\") " pod="opendatahub/maas-controller-75895f5d8-khbqm"
Sep 10 18:05:55.907377 ip-10-0-141-180 kubenswrapper[2581]: I0910 18:05:55.907329    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6bk56\" (UniqueName: \"kubernetes.io/projected/0960dca3-d5d6-4fd4-9605-20c9ba678bb4-kube-api-access-6bk56\") pod \"maas-controller-75895f5d8-khbqm\" (UID: \"0960dca3-d5d6-4fd4-9605-20c9ba678bb4\") " pod="opendatahub/maas-controller-75895f5d8-khbqm"
Sep 10 18:05:55.907542 ip-10-0-141-180 kubenswrapper[2581]: I0910 18:05:55.907404    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/0960dca3-d5d6-4fd4-9605-20c9ba678bb4-webhook-cert\") pod \"maas-controller-75895f5d8-khbqm\" (UID: \"0960dca3-d5d6-4fd4-9605-20c9ba678bb4\") " pod="opendatahub/maas-controller-75895f5d8-khbqm"
Sep 10 18:05:55.907542 ip-10-0-141-180 kubenswrapper[2581]: I0910 18:05:55.907430    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/0960dca3-d5d6-4fd4-9605-20c9ba678bb4-metrics-certs\") pod \"maas-controller-75895f5d8-khbqm\" (UID: \"0960dca3-d5d6-4fd4-9605-20c9ba678bb4\") " pod="opendatahub/maas-controller-75895f5d8-khbqm"
Sep 10 18:05:55.909858 ip-10-0-141-180 kubenswrapper[2581]: I0910 18:05:55.909834    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/0960dca3-d5d6-4fd4-9605-20c9ba678bb4-webhook-cert\") pod \"maas-controller-75895f5d8-khbqm\" (UID: \"0960dca3-d5d6-4fd4-9605-20c9ba678bb4\") " pod="opendatahub/maas-controller-75895f5d8-khbqm"
Sep 10 18:05:55.909967 ip-10-0-141-180 kubenswrapper[2581]: I0910 18:05:55.909857    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/0960dca3-d5d6-4fd4-9605-20c9ba678bb4-metrics-certs\") pod \"maas-controller-75895f5d8-khbqm\" (UID: \"0960dca3-d5d6-4fd4-9605-20c9ba678bb4\") " pod="opendatahub/maas-controller-75895f5d8-khbqm"
Sep 10 18:05:55.915495 ip-10-0-141-180 kubenswrapper[2581]: I0910 18:05:55.915471    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6bk56\" (UniqueName: \"kubernetes.io/projected/0960dca3-d5d6-4fd4-9605-20c9ba678bb4-kube-api-access-6bk56\") pod \"maas-controller-75895f5d8-khbqm\" (UID: \"0960dca3-d5d6-4fd4-9605-20c9ba678bb4\") " pod="opendatahub/maas-controller-75895f5d8-khbqm"
Sep 10 18:05:56.004510 ip-10-0-141-180 kubenswrapper[2581]: I0910 18:05:56.004485    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-75895f5d8-khbqm"
Sep 10 18:05:56.284355 ip-10-0-141-180 kubenswrapper[2581]: I0910 18:05:56.284327    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-75895f5d8-khbqm"]
Sep 10 18:05:56.285487 ip-10-0-141-180 kubenswrapper[2581]: W0910 18:05:56.285452    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0960dca3_d5d6_4fd4_9605_20c9ba678bb4.slice/crio-68fe4254402ebf28cc33b5aeb230f57c4f7b7e651e56d79806846ff61d3c6520 WatchSource:0}: Error finding container 68fe4254402ebf28cc33b5aeb230f57c4f7b7e651e56d79806846ff61d3c6520: Status 404 returned error can't find the container with id 68fe4254402ebf28cc33b5aeb230f57c4f7b7e651e56d79806846ff61d3c6520
Sep 10 18:05:56.286844 ip-10-0-141-180 kubenswrapper[2581]: I0910 18:05:56.286824    2581 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 10 18:05:57.104699 ip-10-0-141-180 kubenswrapper[2581]: I0910 18:05:57.104635    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-75895f5d8-khbqm" event={"ID":"0960dca3-d5d6-4fd4-9605-20c9ba678bb4","Type":"ContainerStarted","Data":"a2c3afb4fcb261ec42e4716eec0e6582cb693e68f9ee173332c156c443b747ca"}
Sep 10 18:05:57.105156 ip-10-0-141-180 kubenswrapper[2581]: I0910 18:05:57.104723    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-75895f5d8-khbqm"
Sep 10 18:05:57.105156 ip-10-0-141-180 kubenswrapper[2581]: I0910 18:05:57.104740    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-75895f5d8-khbqm" event={"ID":"0960dca3-d5d6-4fd4-9605-20c9ba678bb4","Type":"ContainerStarted","Data":"68fe4254402ebf28cc33b5aeb230f57c4f7b7e651e56d79806846ff61d3c6520"}
Sep 10 18:05:57.124142 ip-10-0-141-180 kubenswrapper[2581]: I0910 18:05:57.124097    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-75895f5d8-khbqm" podStartSLOduration=1.7232759020000001 podStartE2EDuration="2.124082658s" totalImagesPullingTime="400.806756ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 18:05:55 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 18:05:57.122431455 +0000 UTC m=+1776.776505378" watchObservedRunningTime="2026-09-10 18:05:57.124082658 +0000 UTC m=+1776.778156563"
Sep 10 18:06:08.113336 ip-10-0-141-180 kubenswrapper[2581]: I0910 18:06:08.113303    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-75895f5d8-khbqm"
Sep 10 18:06:21.343111 ip-10-0-141-180 kubenswrapper[2581]: I0910 18:06:21.343078    2581 scope.go:117] "RemoveContainer" containerID="30bd7eb7b2e94aa25427bb3d2048f34f0fbd8812f3d5b95e03f163f666fa8023"
Sep 10 18:06:21.352627 ip-10-0-141-180 kubenswrapper[2581]: I0910 18:06:21.352606    2581 scope.go:117] "RemoveContainer" containerID="bf4e9bf289309393d915dc8ed94018600727b19b5302763317a1ca021106bf40"
Sep 10 18:06:21.361495 ip-10-0-141-180 kubenswrapper[2581]: I0910 18:06:21.361469    2581 scope.go:117] "RemoveContainer" containerID="f50dcd5f88dce06c174176e9e0870a345f3c4a213f4c73b600aba88618bd39b7"
Sep 10 18:06:21.369529 ip-10-0-141-180 kubenswrapper[2581]: I0910 18:06:21.369511    2581 scope.go:117] "RemoveContainer" containerID="db8c6e70341359a3becce12285dc933df13e5816acc76d4a4e6772225d21c79e"
Sep 10 18:06:21.376961 ip-10-0-141-180 kubenswrapper[2581]: I0910 18:06:21.376941    2581 scope.go:117] "RemoveContainer" containerID="9c0bfb63cf340c784f4ed4cea2d6cb86d84eb485578449ef3f08cc3f4f795e7c"
Sep 10 18:08:21.427450 ip-10-0-141-180 kubenswrapper[2581]: I0910 18:08:21.427419    2581 scope.go:117] "RemoveContainer" containerID="50b41bca4fa0202f192192da05d4e67da066071249c2605791e1d4c5e8fdda8d"
Sep 10 18:08:21.436327 ip-10-0-141-180 kubenswrapper[2581]: I0910 18:08:21.436304    2581 scope.go:117] "RemoveContainer" containerID="c3cd6219273dfb56c9b3656063881f98042e28f357edc12cf49c2d8196d23912"
Sep 10 18:10:05.032427 ip-10-0-141-180 kubenswrapper[2581]: I0910 18:10:05.032394    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_kserve-controller-manager-d6f4b65b8-jpmlv_051c5f70-fb7c-4155-bc93-e6354409ac5e/manager/0.log"
Sep 10 18:10:05.168706 ip-10-0-141-180 kubenswrapper[2581]: I0910 18:10:05.168669    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_llmisvc-controller-manager-777c6bd7d5-8cr6w_726ce400-99e3-4b7b-a9ae-b8c3b0875f5f/manager/0.log"
Sep 10 18:10:05.304500 ip-10-0-141-180 kubenswrapper[2581]: I0910 18:10:05.304419    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-75895f5d8-khbqm_0960dca3-d5d6-4fd4-9605-20c9ba678bb4/manager/0.log"
Sep 10 18:10:05.435215 ip-10-0-141-180 kubenswrapper[2581]: I0910 18:10:05.435180    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_model-serving-api-54d89bd5fc-j5fcj_aa43a8cc-acb7-4845-bca6-e90b5848d533/server/0.log"
Sep 10 18:10:05.555914 ip-10-0-141-180 kubenswrapper[2581]: I0910 18:10:05.555837    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_odh-model-controller-7998c67dc4-g8jts_9b5d3298-2120-43a2-b2e5-50434143c3df/manager/0.log"
Sep 10 18:10:05.795173 ip-10-0-141-180 kubenswrapper[2581]: I0910 18:10:05.795145    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-dbf885bd4-qqm77_53a5b6c5-f6e4-455f-b06e-25b72432e1a6/copy-manifests/0.log"
Sep 10 18:10:05.807202 ip-10-0-141-180 kubenswrapper[2581]: I0910 18:10:05.807135    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-dbf885bd4-qqm77_53a5b6c5-f6e4-455f-b06e-25b72432e1a6/manager/0.log"
Sep 10 18:10:06.578781 ip-10-0-141-180 kubenswrapper[2581]: I0910 18:10:06.578734    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-key-cleanup-29817705-9x9dh_2cdabae7-bf8c-41c5-813a-fa8fc475eeeb/cleanup/0.log"
Sep 10 18:10:07.669392 ip-10-0-141-180 kubenswrapper[2581]: I0910 18:10:07.669364    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-operator-76b478bd78-hx2rv_bcaa2b00-1542-404e-87da-ce9345a99c3f/manager/0.log"
Sep 10 18:10:07.777253 ip-10-0-141-180 kubenswrapper[2581]: I0910 18:10:07.777220    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_dns-operator-controller-manager-7b84cd7c9c-dsd84_aa8daf81-2eee-4880-b323-fcbfa7420927/manager/0.log"
Sep 10 18:10:08.608732 ip-10-0-141-180 kubenswrapper[2581]: I0910 18:10:08.608700    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-d66cd8c7c-kb6rp_9097a0bb-c99e-4edb-b9a2-5c140c8c2773/istio-proxy/0.log"
Sep 10 18:10:08.742454 ip-10-0-141-180 kubenswrapper[2581]: I0910 18:10:08.742424    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-5677ccb49b-5jhrh_1cc585ba-5015-43e9-9752-1b4a288933bd/discovery/0.log"
Sep 10 18:10:08.972807 ip-10-0-141-180 kubenswrapper[2581]: I0910 18:10:08.972782    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-7cd5576685-rd9h7_bc13dda3-b684-423a-bf98-165fa717027c/kube-auth-proxy/0.log"
Sep 10 18:10:09.088050 ip-10-0-141-180 kubenswrapper[2581]: I0910 18:10:09.088020    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-78bb78ff78-nbnx9_a5425b39-1732-47b5-90b4-774966781b06/istio-proxy/1.log"
Sep 10 18:10:09.222141 ip-10-0-141-180 kubenswrapper[2581]: I0910 18:10:09.222113    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-59fbc47bf8-jdd25_cc45ca90-2458-4f31-aeb6-cef3e8a9cf2a/payload-pre-processing/0.log"
Sep 10 18:10:09.787614 ip-10-0-141-180 kubenswrapper[2581]: I0910 18:10:09.787581    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-distinct-2-simulated-kserve-67d8d5d6d-dmnd6_f3313d0c-b37f-4758-a78e-d28b750ffab5/main/0.log"
Sep 10 18:10:09.892626 ip-10-0-141-180 kubenswrapper[2581]: I0910 18:10:09.892596    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-distinct-simulated-kserve-59975d69f9-7m6ws_efe38876-314b-48ab-814a-e512a1dd0608/main/0.log"
Sep 10 18:10:19.097886 ip-10-0-141-180 kubenswrapper[2581]: I0910 18:10:19.097856    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-2t2rd_dee7aebb-2ba6-4bfe-8fb4-371bcc4e56b1/global-pull-secret-syncer/0.log"
Sep 10 18:10:19.262579 ip-10-0-141-180 kubenswrapper[2581]: I0910 18:10:19.262549    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-7p25j_459a65ca-0e07-47e2-aba9-6af10bb5387f/konnectivity-agent/0.log"
Sep 10 18:10:19.364525 ip-10-0-141-180 kubenswrapper[2581]: I0910 18:10:19.364435    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-141-180.ec2.internal_414c1afd223c55080845d6fd4dfe2409/haproxy/0.log"
Sep 10 18:10:21.469780 ip-10-0-141-180 kubenswrapper[2581]: I0910 18:10:21.469751    2581 scope.go:117] "RemoveContainer" containerID="32eee832374e23e139e168c1ba3cfe3b03645401fcd316a7e93475fb3b8f7068"
Sep 10 18:10:21.478438 ip-10-0-141-180 kubenswrapper[2581]: I0910 18:10:21.478418    2581 scope.go:117] "RemoveContainer" containerID="64f45c1825ae8ca368f92e4af610fa748b98ac6bae9d90c222c88d360f8de357"
Sep 10 18:10:21.487105 ip-10-0-141-180 kubenswrapper[2581]: I0910 18:10:21.487083    2581 scope.go:117] "RemoveContainer" containerID="d0d0e6d4ad19859b181ac9985732416c04247ca4882b3fb37fffe02c8237d674"
Sep 10 18:10:24.245342 ip-10-0-141-180 kubenswrapper[2581]: I0910 18:10:24.245306    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-operator-76b478bd78-hx2rv_bcaa2b00-1542-404e-87da-ce9345a99c3f/manager/0.log"
Sep 10 18:10:24.269409 ip-10-0-141-180 kubenswrapper[2581]: I0910 18:10:24.269386    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_dns-operator-controller-manager-7b84cd7c9c-dsd84_aa8daf81-2eee-4880-b323-fcbfa7420927/manager/0.log"
Sep 10 18:10:25.780008 ip-10-0-141-180 kubenswrapper[2581]: I0910 18:10:25.779984    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_cddd9715-a16e-4393-a78f-4ece06346420/alertmanager/0.log"
Sep 10 18:10:25.807881 ip-10-0-141-180 kubenswrapper[2581]: I0910 18:10:25.807858    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_cddd9715-a16e-4393-a78f-4ece06346420/config-reloader/0.log"
Sep 10 18:10:25.851932 ip-10-0-141-180 kubenswrapper[2581]: I0910 18:10:25.851913    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_cddd9715-a16e-4393-a78f-4ece06346420/kube-rbac-proxy-web/0.log"
Sep 10 18:10:25.872748 ip-10-0-141-180 kubenswrapper[2581]: I0910 18:10:25.872730    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_cddd9715-a16e-4393-a78f-4ece06346420/kube-rbac-proxy/0.log"
Sep 10 18:10:25.891421 ip-10-0-141-180 kubenswrapper[2581]: I0910 18:10:25.891398    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_cddd9715-a16e-4393-a78f-4ece06346420/kube-rbac-proxy-metric/0.log"
Sep 10 18:10:25.917621 ip-10-0-141-180 kubenswrapper[2581]: I0910 18:10:25.917601    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_cddd9715-a16e-4393-a78f-4ece06346420/prom-label-proxy/0.log"
Sep 10 18:10:25.937101 ip-10-0-141-180 kubenswrapper[2581]: I0910 18:10:25.937071    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_cddd9715-a16e-4393-a78f-4ece06346420/init-config-reloader/0.log"
Sep 10 18:10:26.005285 ip-10-0-141-180 kubenswrapper[2581]: I0910 18:10:26.005257    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-74cb57df7-9hv2h_df97b1f6-ca1d-481e-892c-68579c1dbfc4/kube-state-metrics/0.log"
Sep 10 18:10:26.024975 ip-10-0-141-180 kubenswrapper[2581]: I0910 18:10:26.024946    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-74cb57df7-9hv2h_df97b1f6-ca1d-481e-892c-68579c1dbfc4/kube-rbac-proxy-main/0.log"
Sep 10 18:10:26.049112 ip-10-0-141-180 kubenswrapper[2581]: I0910 18:10:26.049053    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-74cb57df7-9hv2h_df97b1f6-ca1d-481e-892c-68579c1dbfc4/kube-rbac-proxy-self/0.log"
Sep 10 18:10:26.329159 ip-10-0-141-180 kubenswrapper[2581]: I0910 18:10:26.329071    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-vz9b7_dcd0f9c6-c02f-437a-afe7-9dd75c827467/node-exporter/0.log"
Sep 10 18:10:26.355301 ip-10-0-141-180 kubenswrapper[2581]: I0910 18:10:26.355276    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-vz9b7_dcd0f9c6-c02f-437a-afe7-9dd75c827467/kube-rbac-proxy/0.log"
Sep 10 18:10:26.379046 ip-10-0-141-180 kubenswrapper[2581]: I0910 18:10:26.379018    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-vz9b7_dcd0f9c6-c02f-437a-afe7-9dd75c827467/init-textfile/0.log"
Sep 10 18:10:26.708986 ip-10-0-141-180 kubenswrapper[2581]: I0910 18:10:26.708953    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-6f4946db56-qxdfm_f32f968e-0732-4abd-a2ac-099f0cde2db6/prometheus-operator/0.log"
Sep 10 18:10:26.732550 ip-10-0-141-180 kubenswrapper[2581]: I0910 18:10:26.732520    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-6f4946db56-qxdfm_f32f968e-0732-4abd-a2ac-099f0cde2db6/kube-rbac-proxy/0.log"
Sep 10 18:10:26.816485 ip-10-0-141-180 kubenswrapper[2581]: I0910 18:10:26.816455    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-ddd5cd65c-rc8wq_fac9f077-8067-42f7-8930-0f28f6cb7ba6/telemeter-client/0.log"
Sep 10 18:10:26.854043 ip-10-0-141-180 kubenswrapper[2581]: I0910 18:10:26.853956    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-ddd5cd65c-rc8wq_fac9f077-8067-42f7-8930-0f28f6cb7ba6/reload/0.log"
Sep 10 18:10:26.879474 ip-10-0-141-180 kubenswrapper[2581]: I0910 18:10:26.879446    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-ddd5cd65c-rc8wq_fac9f077-8067-42f7-8930-0f28f6cb7ba6/kube-rbac-proxy/0.log"
Sep 10 18:10:26.915410 ip-10-0-141-180 kubenswrapper[2581]: I0910 18:10:26.915376    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-568869598d-ns9hc_6e66367a-0612-42be-948e-ebf346d9820c/thanos-query/0.log"
Sep 10 18:10:26.940605 ip-10-0-141-180 kubenswrapper[2581]: I0910 18:10:26.940575    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-568869598d-ns9hc_6e66367a-0612-42be-948e-ebf346d9820c/kube-rbac-proxy-web/0.log"
Sep 10 18:10:26.960348 ip-10-0-141-180 kubenswrapper[2581]: I0910 18:10:26.960319    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-568869598d-ns9hc_6e66367a-0612-42be-948e-ebf346d9820c/kube-rbac-proxy/0.log"
Sep 10 18:10:26.981810 ip-10-0-141-180 kubenswrapper[2581]: I0910 18:10:26.981783    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-568869598d-ns9hc_6e66367a-0612-42be-948e-ebf346d9820c/prom-label-proxy/0.log"
Sep 10 18:10:27.005300 ip-10-0-141-180 kubenswrapper[2581]: I0910 18:10:27.005272    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-568869598d-ns9hc_6e66367a-0612-42be-948e-ebf346d9820c/kube-rbac-proxy-rules/0.log"
Sep 10 18:10:27.024283 ip-10-0-141-180 kubenswrapper[2581]: I0910 18:10:27.024256    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-568869598d-ns9hc_6e66367a-0612-42be-948e-ebf346d9820c/kube-rbac-proxy-metrics/0.log"
Sep 10 18:10:29.302566 ip-10-0-141-180 kubenswrapper[2581]: I0910 18:10:29.302536    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-pqhnx/perf-node-gather-daemonset-qsqlj"]
Sep 10 18:10:29.303052 ip-10-0-141-180 kubenswrapper[2581]: I0910 18:10:29.303031    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="0a789c73-9891-4c17-bfa0-948af3f241eb" containerName="istio-proxy"
Sep 10 18:10:29.306286 ip-10-0-141-180 kubenswrapper[2581]: I0910 18:10:29.306267    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-pqhnx/perf-node-gather-daemonset-qsqlj"
Sep 10 18:10:29.308560 ip-10-0-141-180 kubenswrapper[2581]: I0910 18:10:29.308539    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-pqhnx\"/\"kube-root-ca.crt\""
Sep 10 18:10:29.309647 ip-10-0-141-180 kubenswrapper[2581]: I0910 18:10:29.309621    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-pqhnx\"/\"openshift-service-ca.crt\""
Sep 10 18:10:29.309740 ip-10-0-141-180 kubenswrapper[2581]: I0910 18:10:29.309721    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-pqhnx\"/\"default-dockercfg-6n2nd\""
Sep 10 18:10:29.316043 ip-10-0-141-180 kubenswrapper[2581]: I0910 18:10:29.316018    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-pqhnx/perf-node-gather-daemonset-qsqlj"]
Sep 10 18:10:29.323172 ip-10-0-141-180 kubenswrapper[2581]: I0910 18:10:29.323142    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7l4j4\" (UniqueName: \"kubernetes.io/projected/b200f67e-2b7b-4a6a-acb9-7fdc13eec824-kube-api-access-7l4j4\") pod \"perf-node-gather-daemonset-qsqlj\" (UID: \"b200f67e-2b7b-4a6a-acb9-7fdc13eec824\") " pod="openshift-must-gather-pqhnx/perf-node-gather-daemonset-qsqlj"
Sep 10 18:10:29.323290 ip-10-0-141-180 kubenswrapper[2581]: I0910 18:10:29.323220    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/b200f67e-2b7b-4a6a-acb9-7fdc13eec824-sys\") pod \"perf-node-gather-daemonset-qsqlj\" (UID: \"b200f67e-2b7b-4a6a-acb9-7fdc13eec824\") " pod="openshift-must-gather-pqhnx/perf-node-gather-daemonset-qsqlj"
Sep 10 18:10:29.323362 ip-10-0-141-180 kubenswrapper[2581]: I0910 18:10:29.323315    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/b200f67e-2b7b-4a6a-acb9-7fdc13eec824-lib-modules\") pod \"perf-node-gather-daemonset-qsqlj\" (UID: \"b200f67e-2b7b-4a6a-acb9-7fdc13eec824\") " pod="openshift-must-gather-pqhnx/perf-node-gather-daemonset-qsqlj"
Sep 10 18:10:29.323462 ip-10-0-141-180 kubenswrapper[2581]: I0910 18:10:29.323443    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/b200f67e-2b7b-4a6a-acb9-7fdc13eec824-proc\") pod \"perf-node-gather-daemonset-qsqlj\" (UID: \"b200f67e-2b7b-4a6a-acb9-7fdc13eec824\") " pod="openshift-must-gather-pqhnx/perf-node-gather-daemonset-qsqlj"
Sep 10 18:10:29.323530 ip-10-0-141-180 kubenswrapper[2581]: I0910 18:10:29.323503    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/b200f67e-2b7b-4a6a-acb9-7fdc13eec824-podres\") pod \"perf-node-gather-daemonset-qsqlj\" (UID: \"b200f67e-2b7b-4a6a-acb9-7fdc13eec824\") " pod="openshift-must-gather-pqhnx/perf-node-gather-daemonset-qsqlj"
Sep 10 18:10:29.424608 ip-10-0-141-180 kubenswrapper[2581]: I0910 18:10:29.424571    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7l4j4\" (UniqueName: \"kubernetes.io/projected/b200f67e-2b7b-4a6a-acb9-7fdc13eec824-kube-api-access-7l4j4\") pod \"perf-node-gather-daemonset-qsqlj\" (UID: \"b200f67e-2b7b-4a6a-acb9-7fdc13eec824\") " pod="openshift-must-gather-pqhnx/perf-node-gather-daemonset-qsqlj"
Sep 10 18:10:29.424838 ip-10-0-141-180 kubenswrapper[2581]: I0910 18:10:29.424618    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/b200f67e-2b7b-4a6a-acb9-7fdc13eec824-sys\") pod \"perf-node-gather-daemonset-qsqlj\" (UID: \"b200f67e-2b7b-4a6a-acb9-7fdc13eec824\") " pod="openshift-must-gather-pqhnx/perf-node-gather-daemonset-qsqlj"
Sep 10 18:10:29.424838 ip-10-0-141-180 kubenswrapper[2581]: I0910 18:10:29.424660    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/b200f67e-2b7b-4a6a-acb9-7fdc13eec824-lib-modules\") pod \"perf-node-gather-daemonset-qsqlj\" (UID: \"b200f67e-2b7b-4a6a-acb9-7fdc13eec824\") " pod="openshift-must-gather-pqhnx/perf-node-gather-daemonset-qsqlj"
Sep 10 18:10:29.424838 ip-10-0-141-180 kubenswrapper[2581]: I0910 18:10:29.424723    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/b200f67e-2b7b-4a6a-acb9-7fdc13eec824-proc\") pod \"perf-node-gather-daemonset-qsqlj\" (UID: \"b200f67e-2b7b-4a6a-acb9-7fdc13eec824\") " pod="openshift-must-gather-pqhnx/perf-node-gather-daemonset-qsqlj"
Sep 10 18:10:29.424838 ip-10-0-141-180 kubenswrapper[2581]: I0910 18:10:29.424745    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/b200f67e-2b7b-4a6a-acb9-7fdc13eec824-podres\") pod \"perf-node-gather-daemonset-qsqlj\" (UID: \"b200f67e-2b7b-4a6a-acb9-7fdc13eec824\") " pod="openshift-must-gather-pqhnx/perf-node-gather-daemonset-qsqlj"
Sep 10 18:10:29.424838 ip-10-0-141-180 kubenswrapper[2581]: I0910 18:10:29.424741    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/b200f67e-2b7b-4a6a-acb9-7fdc13eec824-sys\") pod \"perf-node-gather-daemonset-qsqlj\" (UID: \"b200f67e-2b7b-4a6a-acb9-7fdc13eec824\") " pod="openshift-must-gather-pqhnx/perf-node-gather-daemonset-qsqlj"
Sep 10 18:10:29.424838 ip-10-0-141-180 kubenswrapper[2581]: I0910 18:10:29.424825    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/b200f67e-2b7b-4a6a-acb9-7fdc13eec824-proc\") pod \"perf-node-gather-daemonset-qsqlj\" (UID: \"b200f67e-2b7b-4a6a-acb9-7fdc13eec824\") " pod="openshift-must-gather-pqhnx/perf-node-gather-daemonset-qsqlj"
Sep 10 18:10:29.425096 ip-10-0-141-180 kubenswrapper[2581]: I0910 18:10:29.424857    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/b200f67e-2b7b-4a6a-acb9-7fdc13eec824-podres\") pod \"perf-node-gather-daemonset-qsqlj\" (UID: \"b200f67e-2b7b-4a6a-acb9-7fdc13eec824\") " pod="openshift-must-gather-pqhnx/perf-node-gather-daemonset-qsqlj"
Sep 10 18:10:29.425096 ip-10-0-141-180 kubenswrapper[2581]: I0910 18:10:29.424866    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/b200f67e-2b7b-4a6a-acb9-7fdc13eec824-lib-modules\") pod \"perf-node-gather-daemonset-qsqlj\" (UID: \"b200f67e-2b7b-4a6a-acb9-7fdc13eec824\") " pod="openshift-must-gather-pqhnx/perf-node-gather-daemonset-qsqlj"
Sep 10 18:10:29.432760 ip-10-0-141-180 kubenswrapper[2581]: I0910 18:10:29.432738    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7l4j4\" (UniqueName: \"kubernetes.io/projected/b200f67e-2b7b-4a6a-acb9-7fdc13eec824-kube-api-access-7l4j4\") pod \"perf-node-gather-daemonset-qsqlj\" (UID: \"b200f67e-2b7b-4a6a-acb9-7fdc13eec824\") " pod="openshift-must-gather-pqhnx/perf-node-gather-daemonset-qsqlj"
Sep 10 18:10:29.617039 ip-10-0-141-180 kubenswrapper[2581]: I0910 18:10:29.616948    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-pqhnx/perf-node-gather-daemonset-qsqlj"
Sep 10 18:10:29.887935 ip-10-0-141-180 kubenswrapper[2581]: I0910 18:10:29.887909    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-pqhnx/perf-node-gather-daemonset-qsqlj"]
Sep 10 18:10:29.889566 ip-10-0-141-180 kubenswrapper[2581]: W0910 18:10:29.889545    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-podb200f67e_2b7b_4a6a_acb9_7fdc13eec824.slice/crio-99aaaf6963a06f08dd4b4b73b343f0f9dce275793556262e823455535b1cd4a7 WatchSource:0}: Error finding container 99aaaf6963a06f08dd4b4b73b343f0f9dce275793556262e823455535b1cd4a7: Status 404 returned error can't find the container with id 99aaaf6963a06f08dd4b4b73b343f0f9dce275793556262e823455535b1cd4a7
Sep 10 18:10:30.717310 ip-10-0-141-180 kubenswrapper[2581]: I0910 18:10:30.717272    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-pqhnx/perf-node-gather-daemonset-qsqlj" event={"ID":"b200f67e-2b7b-4a6a-acb9-7fdc13eec824","Type":"ContainerStarted","Data":"a9143b1632318269395826e86bde4837db336fb04e6c285386bdd459bb5b0bb7"}
Sep 10 18:10:30.717737 ip-10-0-141-180 kubenswrapper[2581]: I0910 18:10:30.717354    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-pqhnx/perf-node-gather-daemonset-qsqlj" event={"ID":"b200f67e-2b7b-4a6a-acb9-7fdc13eec824","Type":"ContainerStarted","Data":"99aaaf6963a06f08dd4b4b73b343f0f9dce275793556262e823455535b1cd4a7"}
Sep 10 18:10:30.717737 ip-10-0-141-180 kubenswrapper[2581]: I0910 18:10:30.717410    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-pqhnx/perf-node-gather-daemonset-qsqlj"
Sep 10 18:10:30.734340 ip-10-0-141-180 kubenswrapper[2581]: I0910 18:10:30.734291    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-pqhnx/perf-node-gather-daemonset-qsqlj" podStartSLOduration=1.7342772 podStartE2EDuration="1.7342772s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-10 18:10:29 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 18:10:30.732500856 +0000 UTC m=+2050.386574788" watchObservedRunningTime="2026-09-10 18:10:30.7342772 +0000 UTC m=+2050.388351111"
Sep 10 18:10:31.497965 ip-10-0-141-180 kubenswrapper[2581]: I0910 18:10:31.497935    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-99dqc_d73b14cf-7f49-458f-989d-3488354f3aa7/dns/0.log"
Sep 10 18:10:31.517146 ip-10-0-141-180 kubenswrapper[2581]: I0910 18:10:31.517115    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-99dqc_d73b14cf-7f49-458f-989d-3488354f3aa7/kube-rbac-proxy/0.log"
Sep 10 18:10:31.628931 ip-10-0-141-180 kubenswrapper[2581]: I0910 18:10:31.628895    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-jkplw_84e91010-05f8-4b38-801c-8e462cd4e38d/dns-node-resolver/0.log"
Sep 10 18:10:32.204332 ip-10-0-141-180 kubenswrapper[2581]: I0910 18:10:32.204297    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_image-registry-86f5bb8d74-5kxl4_93aaeb2d-3497-480f-b10e-9a535e3cba3c/registry/0.log"
Sep 10 18:10:32.250487 ip-10-0-141-180 kubenswrapper[2581]: I0910 18:10:32.250462    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-h6rpt_22cfca39-1a39-4266-9ee7-25dfc8723196/node-ca/0.log"
Sep 10 18:10:33.171648 ip-10-0-141-180 kubenswrapper[2581]: I0910 18:10:33.171604    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-d66cd8c7c-kb6rp_9097a0bb-c99e-4edb-b9a2-5c140c8c2773/istio-proxy/0.log"
Sep 10 18:10:33.603904 ip-10-0-141-180 kubenswrapper[2581]: I0910 18:10:33.603851    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-5677ccb49b-5jhrh_1cc585ba-5015-43e9-9752-1b4a288933bd/discovery/0.log"
Sep 10 18:10:33.650672 ip-10-0-141-180 kubenswrapper[2581]: I0910 18:10:33.650630    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-7cd5576685-rd9h7_bc13dda3-b684-423a-bf98-165fa717027c/kube-auth-proxy/0.log"
Sep 10 18:10:33.678848 ip-10-0-141-180 kubenswrapper[2581]: I0910 18:10:33.678818    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-78bb78ff78-nbnx9_a5425b39-1732-47b5-90b4-774966781b06/istio-proxy/0.log"
Sep 10 18:10:33.688603 ip-10-0-141-180 kubenswrapper[2581]: I0910 18:10:33.688582    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-78bb78ff78-nbnx9_a5425b39-1732-47b5-90b4-774966781b06/istio-proxy/1.log"
Sep 10 18:10:33.758528 ip-10-0-141-180 kubenswrapper[2581]: I0910 18:10:33.758503    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-59fbc47bf8-jdd25_cc45ca90-2458-4f31-aeb6-cef3e8a9cf2a/payload-pre-processing/0.log"
Sep 10 18:10:34.670532 ip-10-0-141-180 kubenswrapper[2581]: I0910 18:10:34.670502    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-hxnhw_e78eff7d-f4b5-4b89-b23b-fe0e70020758/serve-healthcheck-canary/0.log"
Sep 10 18:10:35.326261 ip-10-0-141-180 kubenswrapper[2581]: I0910 18:10:35.326223    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-wsx6q_dee8eef1-d418-42eb-9e73-e37e474775ad/kube-rbac-proxy/0.log"
Sep 10 18:10:35.345428 ip-10-0-141-180 kubenswrapper[2581]: I0910 18:10:35.345404    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-wsx6q_dee8eef1-d418-42eb-9e73-e37e474775ad/exporter/0.log"
Sep 10 18:10:35.364972 ip-10-0-141-180 kubenswrapper[2581]: I0910 18:10:35.364946    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-wsx6q_dee8eef1-d418-42eb-9e73-e37e474775ad/extractor/0.log"
Sep 10 18:10:36.732305 ip-10-0-141-180 kubenswrapper[2581]: I0910 18:10:36.732277    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-pqhnx/perf-node-gather-daemonset-qsqlj"
Sep 10 18:10:37.504840 ip-10-0-141-180 kubenswrapper[2581]: I0910 18:10:37.504805    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_kserve-controller-manager-d6f4b65b8-jpmlv_051c5f70-fb7c-4155-bc93-e6354409ac5e/manager/0.log"
Sep 10 18:10:37.808762 ip-10-0-141-180 kubenswrapper[2581]: I0910 18:10:37.808656    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_llmisvc-controller-manager-777c6bd7d5-8cr6w_726ce400-99e3-4b7b-a9ae-b8c3b0875f5f/manager/0.log"
Sep 10 18:10:37.883777 ip-10-0-141-180 kubenswrapper[2581]: I0910 18:10:37.883745    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-75895f5d8-khbqm_0960dca3-d5d6-4fd4-9605-20c9ba678bb4/manager/0.log"
Sep 10 18:10:37.904979 ip-10-0-141-180 kubenswrapper[2581]: I0910 18:10:37.904956    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_model-serving-api-54d89bd5fc-j5fcj_aa43a8cc-acb7-4845-bca6-e90b5848d533/server/0.log"
Sep 10 18:10:38.036180 ip-10-0-141-180 kubenswrapper[2581]: I0910 18:10:38.036155    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_odh-model-controller-7998c67dc4-g8jts_9b5d3298-2120-43a2-b2e5-50434143c3df/manager/0.log"
Sep 10 18:10:38.137843 ip-10-0-141-180 kubenswrapper[2581]: I0910 18:10:38.137768    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-dbf885bd4-qqm77_53a5b6c5-f6e4-455f-b06e-25b72432e1a6/manager/0.log"
Sep 10 18:10:38.156004 ip-10-0-141-180 kubenswrapper[2581]: I0910 18:10:38.155975    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-dbf885bd4-qqm77_53a5b6c5-f6e4-455f-b06e-25b72432e1a6/copy-manifests/0.log"
Sep 10 18:10:39.716511 ip-10-0-141-180 kubenswrapper[2581]: I0910 18:10:39.716482    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_openshift-lws-operator-fd8ccff4c-fgffh_9b90c56a-f146-4d31-be95-dd480b2ba2b8/openshift-lws-operator/0.log"
Sep 10 18:10:39.800634 ip-10-0-141-180 kubenswrapper[2581]: I0910 18:10:39.800589    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-d66cd8c7c-kb6rp_9097a0bb-c99e-4edb-b9a2-5c140c8c2773/istio-proxy/0.log"
Sep 10 18:10:40.215280 ip-10-0-141-180 kubenswrapper[2581]: I0910 18:10:40.215254    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-5677ccb49b-5jhrh_1cc585ba-5015-43e9-9752-1b4a288933bd/discovery/0.log"
Sep 10 18:10:40.256446 ip-10-0-141-180 kubenswrapper[2581]: I0910 18:10:40.256424    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-7cd5576685-rd9h7_bc13dda3-b684-423a-bf98-165fa717027c/kube-auth-proxy/0.log"
Sep 10 18:10:40.285092 ip-10-0-141-180 kubenswrapper[2581]: I0910 18:10:40.285064    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-78bb78ff78-nbnx9_a5425b39-1732-47b5-90b4-774966781b06/istio-proxy/0.log"
Sep 10 18:10:40.300075 ip-10-0-141-180 kubenswrapper[2581]: I0910 18:10:40.300049    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-78bb78ff78-nbnx9_a5425b39-1732-47b5-90b4-774966781b06/istio-proxy/1.log"
Sep 10 18:10:40.337897 ip-10-0-141-180 kubenswrapper[2581]: I0910 18:10:40.337864    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-59fbc47bf8-jdd25_cc45ca90-2458-4f31-aeb6-cef3e8a9cf2a/payload-pre-processing/0.log"
Sep 10 18:10:41.854203 ip-10-0-141-180 kubenswrapper[2581]: I0910 18:10:41.854165    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-d66cd8c7c-kb6rp_9097a0bb-c99e-4edb-b9a2-5c140c8c2773/istio-proxy/0.log"
Sep 10 18:10:42.299819 ip-10-0-141-180 kubenswrapper[2581]: I0910 18:10:42.299785    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-5677ccb49b-5jhrh_1cc585ba-5015-43e9-9752-1b4a288933bd/discovery/0.log"
Sep 10 18:10:42.331698 ip-10-0-141-180 kubenswrapper[2581]: I0910 18:10:42.331672    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-7cd5576685-rd9h7_bc13dda3-b684-423a-bf98-165fa717027c/kube-auth-proxy/0.log"
Sep 10 18:10:42.354589 ip-10-0-141-180 kubenswrapper[2581]: I0910 18:10:42.354565    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-78bb78ff78-nbnx9_a5425b39-1732-47b5-90b4-774966781b06/istio-proxy/0.log"
Sep 10 18:10:42.370805 ip-10-0-141-180 kubenswrapper[2581]: I0910 18:10:42.370777    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-78bb78ff78-nbnx9_a5425b39-1732-47b5-90b4-774966781b06/istio-proxy/1.log"
Sep 10 18:10:42.412332 ip-10-0-141-180 kubenswrapper[2581]: I0910 18:10:42.412298    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-59fbc47bf8-jdd25_cc45ca90-2458-4f31-aeb6-cef3e8a9cf2a/payload-pre-processing/0.log"
Sep 10 18:10:45.967814 ip-10-0-141-180 kubenswrapper[2581]: I0910 18:10:45.967781    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-bqj9g_099b7fa9-6908-43e3-b444-73b7f327290d/kube-multus-additional-cni-plugins/0.log"
Sep 10 18:10:45.986923 ip-10-0-141-180 kubenswrapper[2581]: I0910 18:10:45.986897    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-bqj9g_099b7fa9-6908-43e3-b444-73b7f327290d/egress-router-binary-copy/0.log"
Sep 10 18:10:46.005184 ip-10-0-141-180 kubenswrapper[2581]: I0910 18:10:46.005159    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-bqj9g_099b7fa9-6908-43e3-b444-73b7f327290d/cni-plugins/0.log"
Sep 10 18:10:46.023851 ip-10-0-141-180 kubenswrapper[2581]: I0910 18:10:46.023828    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-bqj9g_099b7fa9-6908-43e3-b444-73b7f327290d/bond-cni-plugin/0.log"
Sep 10 18:10:46.041977 ip-10-0-141-180 kubenswrapper[2581]: I0910 18:10:46.041958    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-bqj9g_099b7fa9-6908-43e3-b444-73b7f327290d/routeoverride-cni/0.log"
Sep 10 18:10:46.059967 ip-10-0-141-180 kubenswrapper[2581]: I0910 18:10:46.059949    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-bqj9g_099b7fa9-6908-43e3-b444-73b7f327290d/whereabouts-cni-bincopy/0.log"
Sep 10 18:10:46.077682 ip-10-0-141-180 kubenswrapper[2581]: I0910 18:10:46.077662    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-bqj9g_099b7fa9-6908-43e3-b444-73b7f327290d/whereabouts-cni/0.log"
Sep 10 18:10:46.516520 ip-10-0-141-180 kubenswrapper[2581]: I0910 18:10:46.516480    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-z5bpf_b1aa3439-0cd7-4496-b1e8-7b535c305002/kube-multus/0.log"
Sep 10 18:10:46.542581 ip-10-0-141-180 kubenswrapper[2581]: I0910 18:10:46.542555    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-b88bx_88d343e2-9756-438b-a19c-973b53948e68/network-metrics-daemon/0.log"
Sep 10 18:10:46.556541 ip-10-0-141-180 kubenswrapper[2581]: I0910 18:10:46.556513    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-b88bx_88d343e2-9756-438b-a19c-973b53948e68/kube-rbac-proxy/0.log"