Sep 12 00:14:28.120852 ip-10-0-141-128 systemd[1]: kubelet.service: Failed to load environment files: No such file or directory
Sep 12 00:14:28.120862 ip-10-0-141-128 systemd[1]: kubelet.service: Failed to run 'start-pre' task: No such file or directory
Sep 12 00:14:28.120869 ip-10-0-141-128 systemd[1]: kubelet.service: Failed with result 'resources'.
Sep 12 00:14:28.121092 ip-10-0-141-128 systemd[1]: Failed to start Kubernetes Kubelet.
Sep 12 00:14:38.353979 ip-10-0-141-128 systemd[1]: kubelet.service: Failed to schedule restart job: Unit crio.service not found.
Sep 12 00:14:38.353996 ip-10-0-141-128 systemd[1]: kubelet.service: Failed with result 'resources'.
-- Boot 8939ddfaee9e4650808c496d3d7205e1 --
Sep 12 00:16:28.622870 ip-10-0-141-128 systemd[1]: Starting Kubernetes Kubelet...
Sep 12 00:16:28.969878 ip-10-0-141-128 kubenswrapper[2571]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Sep 12 00:16:28.969878 ip-10-0-141-128 kubenswrapper[2571]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version.
Sep 12 00:16:28.969878 ip-10-0-141-128 kubenswrapper[2571]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Sep 12 00:16:28.969878 ip-10-0-141-128 kubenswrapper[2571]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI.
Sep 12 00:16:28.969878 ip-10-0-141-128 kubenswrapper[2571]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Sep 12 00:16:28.971671 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:16:28.971598    2571 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime"
Sep 12 00:16:28.979307 ip-10-0-141-128 kubenswrapper[2571]: W0912 00:16:28.979286    2571 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 12 00:16:28.979307 ip-10-0-141-128 kubenswrapper[2571]: W0912 00:16:28.979305    2571 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 12 00:16:28.979307 ip-10-0-141-128 kubenswrapper[2571]: W0912 00:16:28.979311    2571 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 12 00:16:28.979307 ip-10-0-141-128 kubenswrapper[2571]: W0912 00:16:28.979315    2571 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 12 00:16:28.979464 ip-10-0-141-128 kubenswrapper[2571]: W0912 00:16:28.979318    2571 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 12 00:16:28.979464 ip-10-0-141-128 kubenswrapper[2571]: W0912 00:16:28.979321    2571 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 12 00:16:28.979464 ip-10-0-141-128 kubenswrapper[2571]: W0912 00:16:28.979325    2571 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 12 00:16:28.979464 ip-10-0-141-128 kubenswrapper[2571]: W0912 00:16:28.979328    2571 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 12 00:16:28.979464 ip-10-0-141-128 kubenswrapper[2571]: W0912 00:16:28.979331    2571 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 12 00:16:28.979464 ip-10-0-141-128 kubenswrapper[2571]: W0912 00:16:28.979334    2571 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 12 00:16:28.979464 ip-10-0-141-128 kubenswrapper[2571]: W0912 00:16:28.979337    2571 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 12 00:16:28.979464 ip-10-0-141-128 kubenswrapper[2571]: W0912 00:16:28.979340    2571 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 12 00:16:28.979464 ip-10-0-141-128 kubenswrapper[2571]: W0912 00:16:28.979343    2571 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 12 00:16:28.979464 ip-10-0-141-128 kubenswrapper[2571]: W0912 00:16:28.979345    2571 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 12 00:16:28.979464 ip-10-0-141-128 kubenswrapper[2571]: W0912 00:16:28.979348    2571 feature_gate.go:328] unrecognized feature gate: Example
Sep 12 00:16:28.979464 ip-10-0-141-128 kubenswrapper[2571]: W0912 00:16:28.979350    2571 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 12 00:16:28.979464 ip-10-0-141-128 kubenswrapper[2571]: W0912 00:16:28.979353    2571 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 12 00:16:28.979464 ip-10-0-141-128 kubenswrapper[2571]: W0912 00:16:28.979356    2571 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 12 00:16:28.979464 ip-10-0-141-128 kubenswrapper[2571]: W0912 00:16:28.979358    2571 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 12 00:16:28.979464 ip-10-0-141-128 kubenswrapper[2571]: W0912 00:16:28.979361    2571 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 12 00:16:28.979464 ip-10-0-141-128 kubenswrapper[2571]: W0912 00:16:28.979363    2571 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 12 00:16:28.979464 ip-10-0-141-128 kubenswrapper[2571]: W0912 00:16:28.979366    2571 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 12 00:16:28.979464 ip-10-0-141-128 kubenswrapper[2571]: W0912 00:16:28.979369    2571 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 12 00:16:28.979464 ip-10-0-141-128 kubenswrapper[2571]: W0912 00:16:28.979372    2571 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 12 00:16:28.979971 ip-10-0-141-128 kubenswrapper[2571]: W0912 00:16:28.979374    2571 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 12 00:16:28.979971 ip-10-0-141-128 kubenswrapper[2571]: W0912 00:16:28.979377    2571 feature_gate.go:328] unrecognized feature gate: Example2
Sep 12 00:16:28.979971 ip-10-0-141-128 kubenswrapper[2571]: W0912 00:16:28.979379    2571 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 12 00:16:28.979971 ip-10-0-141-128 kubenswrapper[2571]: W0912 00:16:28.979382    2571 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 12 00:16:28.979971 ip-10-0-141-128 kubenswrapper[2571]: W0912 00:16:28.979387    2571 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 12 00:16:28.979971 ip-10-0-141-128 kubenswrapper[2571]: W0912 00:16:28.979390    2571 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 12 00:16:28.979971 ip-10-0-141-128 kubenswrapper[2571]: W0912 00:16:28.979393    2571 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 12 00:16:28.979971 ip-10-0-141-128 kubenswrapper[2571]: W0912 00:16:28.979396    2571 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 12 00:16:28.979971 ip-10-0-141-128 kubenswrapper[2571]: W0912 00:16:28.979405    2571 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 12 00:16:28.979971 ip-10-0-141-128 kubenswrapper[2571]: W0912 00:16:28.979407    2571 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 12 00:16:28.979971 ip-10-0-141-128 kubenswrapper[2571]: W0912 00:16:28.979410    2571 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 12 00:16:28.979971 ip-10-0-141-128 kubenswrapper[2571]: W0912 00:16:28.979412    2571 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 12 00:16:28.979971 ip-10-0-141-128 kubenswrapper[2571]: W0912 00:16:28.979415    2571 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 12 00:16:28.979971 ip-10-0-141-128 kubenswrapper[2571]: W0912 00:16:28.979418    2571 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 12 00:16:28.979971 ip-10-0-141-128 kubenswrapper[2571]: W0912 00:16:28.979421    2571 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 12 00:16:28.979971 ip-10-0-141-128 kubenswrapper[2571]: W0912 00:16:28.979423    2571 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 12 00:16:28.979971 ip-10-0-141-128 kubenswrapper[2571]: W0912 00:16:28.979426    2571 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 12 00:16:28.979971 ip-10-0-141-128 kubenswrapper[2571]: W0912 00:16:28.979430    2571 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 12 00:16:28.979971 ip-10-0-141-128 kubenswrapper[2571]: W0912 00:16:28.979433    2571 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 12 00:16:28.980540 ip-10-0-141-128 kubenswrapper[2571]: W0912 00:16:28.979436    2571 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 12 00:16:28.980540 ip-10-0-141-128 kubenswrapper[2571]: W0912 00:16:28.979439    2571 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 12 00:16:28.980540 ip-10-0-141-128 kubenswrapper[2571]: W0912 00:16:28.979443    2571 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 12 00:16:28.980540 ip-10-0-141-128 kubenswrapper[2571]: W0912 00:16:28.979445    2571 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 12 00:16:28.980540 ip-10-0-141-128 kubenswrapper[2571]: W0912 00:16:28.979448    2571 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 12 00:16:28.980540 ip-10-0-141-128 kubenswrapper[2571]: W0912 00:16:28.979451    2571 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 12 00:16:28.980540 ip-10-0-141-128 kubenswrapper[2571]: W0912 00:16:28.979454    2571 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 12 00:16:28.980540 ip-10-0-141-128 kubenswrapper[2571]: W0912 00:16:28.979457    2571 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 12 00:16:28.980540 ip-10-0-141-128 kubenswrapper[2571]: W0912 00:16:28.979459    2571 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 12 00:16:28.980540 ip-10-0-141-128 kubenswrapper[2571]: W0912 00:16:28.979462    2571 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 12 00:16:28.980540 ip-10-0-141-128 kubenswrapper[2571]: W0912 00:16:28.979464    2571 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 12 00:16:28.980540 ip-10-0-141-128 kubenswrapper[2571]: W0912 00:16:28.979467    2571 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 12 00:16:28.980540 ip-10-0-141-128 kubenswrapper[2571]: W0912 00:16:28.979470    2571 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 12 00:16:28.980540 ip-10-0-141-128 kubenswrapper[2571]: W0912 00:16:28.979473    2571 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 12 00:16:28.980540 ip-10-0-141-128 kubenswrapper[2571]: W0912 00:16:28.979476    2571 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 12 00:16:28.980540 ip-10-0-141-128 kubenswrapper[2571]: W0912 00:16:28.979479    2571 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 12 00:16:28.980540 ip-10-0-141-128 kubenswrapper[2571]: W0912 00:16:28.979481    2571 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 12 00:16:28.980540 ip-10-0-141-128 kubenswrapper[2571]: W0912 00:16:28.979483    2571 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 12 00:16:28.980540 ip-10-0-141-128 kubenswrapper[2571]: W0912 00:16:28.979486    2571 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 12 00:16:28.981024 ip-10-0-141-128 kubenswrapper[2571]: W0912 00:16:28.979489    2571 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 12 00:16:28.981024 ip-10-0-141-128 kubenswrapper[2571]: W0912 00:16:28.979492    2571 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 12 00:16:28.981024 ip-10-0-141-128 kubenswrapper[2571]: W0912 00:16:28.979495    2571 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 12 00:16:28.981024 ip-10-0-141-128 kubenswrapper[2571]: W0912 00:16:28.979497    2571 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 12 00:16:28.981024 ip-10-0-141-128 kubenswrapper[2571]: W0912 00:16:28.979500    2571 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 12 00:16:28.981024 ip-10-0-141-128 kubenswrapper[2571]: W0912 00:16:28.979502    2571 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 12 00:16:28.981024 ip-10-0-141-128 kubenswrapper[2571]: W0912 00:16:28.979505    2571 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 12 00:16:28.981024 ip-10-0-141-128 kubenswrapper[2571]: W0912 00:16:28.979507    2571 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 12 00:16:28.981024 ip-10-0-141-128 kubenswrapper[2571]: W0912 00:16:28.979510    2571 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 12 00:16:28.981024 ip-10-0-141-128 kubenswrapper[2571]: W0912 00:16:28.979512    2571 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 12 00:16:28.981024 ip-10-0-141-128 kubenswrapper[2571]: W0912 00:16:28.979515    2571 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 12 00:16:28.981024 ip-10-0-141-128 kubenswrapper[2571]: W0912 00:16:28.979518    2571 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 12 00:16:28.981024 ip-10-0-141-128 kubenswrapper[2571]: W0912 00:16:28.979520    2571 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 12 00:16:28.981024 ip-10-0-141-128 kubenswrapper[2571]: W0912 00:16:28.979523    2571 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 12 00:16:28.981024 ip-10-0-141-128 kubenswrapper[2571]: W0912 00:16:28.979526    2571 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 12 00:16:28.981024 ip-10-0-141-128 kubenswrapper[2571]: W0912 00:16:28.979528    2571 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 12 00:16:28.981024 ip-10-0-141-128 kubenswrapper[2571]: W0912 00:16:28.979530    2571 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 12 00:16:28.981024 ip-10-0-141-128 kubenswrapper[2571]: W0912 00:16:28.979533    2571 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 12 00:16:28.981024 ip-10-0-141-128 kubenswrapper[2571]: W0912 00:16:28.979535    2571 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 12 00:16:28.981024 ip-10-0-141-128 kubenswrapper[2571]: W0912 00:16:28.979538    2571 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 12 00:16:28.981513 ip-10-0-141-128 kubenswrapper[2571]: W0912 00:16:28.979540    2571 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 12 00:16:28.981513 ip-10-0-141-128 kubenswrapper[2571]: W0912 00:16:28.979543    2571 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 12 00:16:28.981513 ip-10-0-141-128 kubenswrapper[2571]: W0912 00:16:28.979546    2571 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 12 00:16:28.981513 ip-10-0-141-128 kubenswrapper[2571]: W0912 00:16:28.979548    2571 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 12 00:16:28.981513 ip-10-0-141-128 kubenswrapper[2571]: W0912 00:16:28.979551    2571 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 12 00:16:28.981513 ip-10-0-141-128 kubenswrapper[2571]: W0912 00:16:28.979553    2571 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 12 00:16:28.981513 ip-10-0-141-128 kubenswrapper[2571]: W0912 00:16:28.979556    2571 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 12 00:16:28.981513 ip-10-0-141-128 kubenswrapper[2571]: W0912 00:16:28.979558    2571 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 12 00:16:28.981513 ip-10-0-141-128 kubenswrapper[2571]: W0912 00:16:28.979561    2571 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 12 00:16:28.981513 ip-10-0-141-128 kubenswrapper[2571]: W0912 00:16:28.979563    2571 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 12 00:16:28.981513 ip-10-0-141-128 kubenswrapper[2571]: W0912 00:16:28.979566    2571 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 12 00:16:28.981513 ip-10-0-141-128 kubenswrapper[2571]: W0912 00:16:28.979568    2571 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 12 00:16:28.981513 ip-10-0-141-128 kubenswrapper[2571]: W0912 00:16:28.979570    2571 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 12 00:16:28.981513 ip-10-0-141-128 kubenswrapper[2571]: W0912 00:16:28.979573    2571 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 12 00:16:28.981513 ip-10-0-141-128 kubenswrapper[2571]: W0912 00:16:28.979575    2571 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 12 00:16:28.981513 ip-10-0-141-128 kubenswrapper[2571]: W0912 00:16:28.979578    2571 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 12 00:16:28.981513 ip-10-0-141-128 kubenswrapper[2571]: W0912 00:16:28.979580    2571 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 12 00:16:28.981513 ip-10-0-141-128 kubenswrapper[2571]: W0912 00:16:28.980251    2571 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 12 00:16:28.981513 ip-10-0-141-128 kubenswrapper[2571]: W0912 00:16:28.980258    2571 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 12 00:16:28.981987 ip-10-0-141-128 kubenswrapper[2571]: W0912 00:16:28.980262    2571 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 12 00:16:28.981987 ip-10-0-141-128 kubenswrapper[2571]: W0912 00:16:28.980265    2571 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 12 00:16:28.981987 ip-10-0-141-128 kubenswrapper[2571]: W0912 00:16:28.980268    2571 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 12 00:16:28.981987 ip-10-0-141-128 kubenswrapper[2571]: W0912 00:16:28.980270    2571 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 12 00:16:28.981987 ip-10-0-141-128 kubenswrapper[2571]: W0912 00:16:28.980273    2571 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 12 00:16:28.981987 ip-10-0-141-128 kubenswrapper[2571]: W0912 00:16:28.980275    2571 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 12 00:16:28.981987 ip-10-0-141-128 kubenswrapper[2571]: W0912 00:16:28.980278    2571 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 12 00:16:28.981987 ip-10-0-141-128 kubenswrapper[2571]: W0912 00:16:28.980281    2571 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 12 00:16:28.981987 ip-10-0-141-128 kubenswrapper[2571]: W0912 00:16:28.980283    2571 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 12 00:16:28.981987 ip-10-0-141-128 kubenswrapper[2571]: W0912 00:16:28.980286    2571 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 12 00:16:28.981987 ip-10-0-141-128 kubenswrapper[2571]: W0912 00:16:28.980288    2571 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 12 00:16:28.981987 ip-10-0-141-128 kubenswrapper[2571]: W0912 00:16:28.980291    2571 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 12 00:16:28.981987 ip-10-0-141-128 kubenswrapper[2571]: W0912 00:16:28.980293    2571 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 12 00:16:28.981987 ip-10-0-141-128 kubenswrapper[2571]: W0912 00:16:28.980296    2571 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 12 00:16:28.981987 ip-10-0-141-128 kubenswrapper[2571]: W0912 00:16:28.980299    2571 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 12 00:16:28.981987 ip-10-0-141-128 kubenswrapper[2571]: W0912 00:16:28.980302    2571 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 12 00:16:28.981987 ip-10-0-141-128 kubenswrapper[2571]: W0912 00:16:28.980304    2571 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 12 00:16:28.981987 ip-10-0-141-128 kubenswrapper[2571]: W0912 00:16:28.980307    2571 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 12 00:16:28.981987 ip-10-0-141-128 kubenswrapper[2571]: W0912 00:16:28.980310    2571 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 12 00:16:28.981987 ip-10-0-141-128 kubenswrapper[2571]: W0912 00:16:28.980313    2571 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 12 00:16:28.982478 ip-10-0-141-128 kubenswrapper[2571]: W0912 00:16:28.980315    2571 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 12 00:16:28.982478 ip-10-0-141-128 kubenswrapper[2571]: W0912 00:16:28.980318    2571 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 12 00:16:28.982478 ip-10-0-141-128 kubenswrapper[2571]: W0912 00:16:28.980321    2571 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 12 00:16:28.982478 ip-10-0-141-128 kubenswrapper[2571]: W0912 00:16:28.980323    2571 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 12 00:16:28.982478 ip-10-0-141-128 kubenswrapper[2571]: W0912 00:16:28.980326    2571 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 12 00:16:28.982478 ip-10-0-141-128 kubenswrapper[2571]: W0912 00:16:28.980328    2571 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 12 00:16:28.982478 ip-10-0-141-128 kubenswrapper[2571]: W0912 00:16:28.980331    2571 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 12 00:16:28.982478 ip-10-0-141-128 kubenswrapper[2571]: W0912 00:16:28.980333    2571 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 12 00:16:28.982478 ip-10-0-141-128 kubenswrapper[2571]: W0912 00:16:28.980336    2571 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 12 00:16:28.982478 ip-10-0-141-128 kubenswrapper[2571]: W0912 00:16:28.980338    2571 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 12 00:16:28.982478 ip-10-0-141-128 kubenswrapper[2571]: W0912 00:16:28.980340    2571 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 12 00:16:28.982478 ip-10-0-141-128 kubenswrapper[2571]: W0912 00:16:28.980343    2571 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 12 00:16:28.982478 ip-10-0-141-128 kubenswrapper[2571]: W0912 00:16:28.980346    2571 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 12 00:16:28.982478 ip-10-0-141-128 kubenswrapper[2571]: W0912 00:16:28.980349    2571 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 12 00:16:28.982478 ip-10-0-141-128 kubenswrapper[2571]: W0912 00:16:28.980351    2571 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 12 00:16:28.982478 ip-10-0-141-128 kubenswrapper[2571]: W0912 00:16:28.980354    2571 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 12 00:16:28.982478 ip-10-0-141-128 kubenswrapper[2571]: W0912 00:16:28.980356    2571 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 12 00:16:28.982478 ip-10-0-141-128 kubenswrapper[2571]: W0912 00:16:28.980358    2571 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 12 00:16:28.982478 ip-10-0-141-128 kubenswrapper[2571]: W0912 00:16:28.980361    2571 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 12 00:16:28.982478 ip-10-0-141-128 kubenswrapper[2571]: W0912 00:16:28.980364    2571 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 12 00:16:28.983013 ip-10-0-141-128 kubenswrapper[2571]: W0912 00:16:28.980367    2571 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 12 00:16:28.983013 ip-10-0-141-128 kubenswrapper[2571]: W0912 00:16:28.980369    2571 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 12 00:16:28.983013 ip-10-0-141-128 kubenswrapper[2571]: W0912 00:16:28.980372    2571 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 12 00:16:28.983013 ip-10-0-141-128 kubenswrapper[2571]: W0912 00:16:28.980375    2571 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 12 00:16:28.983013 ip-10-0-141-128 kubenswrapper[2571]: W0912 00:16:28.980379    2571 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 12 00:16:28.983013 ip-10-0-141-128 kubenswrapper[2571]: W0912 00:16:28.980383    2571 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 12 00:16:28.983013 ip-10-0-141-128 kubenswrapper[2571]: W0912 00:16:28.980386    2571 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 12 00:16:28.983013 ip-10-0-141-128 kubenswrapper[2571]: W0912 00:16:28.980389    2571 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 12 00:16:28.983013 ip-10-0-141-128 kubenswrapper[2571]: W0912 00:16:28.980393    2571 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 12 00:16:28.983013 ip-10-0-141-128 kubenswrapper[2571]: W0912 00:16:28.980396    2571 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 12 00:16:28.983013 ip-10-0-141-128 kubenswrapper[2571]: W0912 00:16:28.980398    2571 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 12 00:16:28.983013 ip-10-0-141-128 kubenswrapper[2571]: W0912 00:16:28.980401    2571 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 12 00:16:28.983013 ip-10-0-141-128 kubenswrapper[2571]: W0912 00:16:28.980403    2571 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 12 00:16:28.983013 ip-10-0-141-128 kubenswrapper[2571]: W0912 00:16:28.980406    2571 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 12 00:16:28.983013 ip-10-0-141-128 kubenswrapper[2571]: W0912 00:16:28.980408    2571 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 12 00:16:28.983013 ip-10-0-141-128 kubenswrapper[2571]: W0912 00:16:28.980411    2571 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 12 00:16:28.983013 ip-10-0-141-128 kubenswrapper[2571]: W0912 00:16:28.980413    2571 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 12 00:16:28.983013 ip-10-0-141-128 kubenswrapper[2571]: W0912 00:16:28.980416    2571 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 12 00:16:28.983013 ip-10-0-141-128 kubenswrapper[2571]: W0912 00:16:28.980418    2571 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 12 00:16:28.983475 ip-10-0-141-128 kubenswrapper[2571]: W0912 00:16:28.980421    2571 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 12 00:16:28.983475 ip-10-0-141-128 kubenswrapper[2571]: W0912 00:16:28.980425    2571 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 12 00:16:28.983475 ip-10-0-141-128 kubenswrapper[2571]: W0912 00:16:28.980428    2571 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 12 00:16:28.983475 ip-10-0-141-128 kubenswrapper[2571]: W0912 00:16:28.980431    2571 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 12 00:16:28.983475 ip-10-0-141-128 kubenswrapper[2571]: W0912 00:16:28.980434    2571 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 12 00:16:28.983475 ip-10-0-141-128 kubenswrapper[2571]: W0912 00:16:28.980437    2571 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 12 00:16:28.983475 ip-10-0-141-128 kubenswrapper[2571]: W0912 00:16:28.980441    2571 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 12 00:16:28.983475 ip-10-0-141-128 kubenswrapper[2571]: W0912 00:16:28.980444    2571 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 12 00:16:28.983475 ip-10-0-141-128 kubenswrapper[2571]: W0912 00:16:28.980446    2571 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 12 00:16:28.983475 ip-10-0-141-128 kubenswrapper[2571]: W0912 00:16:28.980449    2571 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 12 00:16:28.983475 ip-10-0-141-128 kubenswrapper[2571]: W0912 00:16:28.980451    2571 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 12 00:16:28.983475 ip-10-0-141-128 kubenswrapper[2571]: W0912 00:16:28.980454    2571 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 12 00:16:28.983475 ip-10-0-141-128 kubenswrapper[2571]: W0912 00:16:28.980456    2571 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 12 00:16:28.983475 ip-10-0-141-128 kubenswrapper[2571]: W0912 00:16:28.980459    2571 feature_gate.go:328] unrecognized feature gate: Example
Sep 12 00:16:28.983475 ip-10-0-141-128 kubenswrapper[2571]: W0912 00:16:28.980462    2571 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 12 00:16:28.983475 ip-10-0-141-128 kubenswrapper[2571]: W0912 00:16:28.980464    2571 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 12 00:16:28.983475 ip-10-0-141-128 kubenswrapper[2571]: W0912 00:16:28.980467    2571 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 12 00:16:28.983475 ip-10-0-141-128 kubenswrapper[2571]: W0912 00:16:28.980469    2571 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 12 00:16:28.983475 ip-10-0-141-128 kubenswrapper[2571]: W0912 00:16:28.980472    2571 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 12 00:16:28.983475 ip-10-0-141-128 kubenswrapper[2571]: W0912 00:16:28.980474    2571 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 12 00:16:28.983974 ip-10-0-141-128 kubenswrapper[2571]: W0912 00:16:28.980477    2571 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 12 00:16:28.983974 ip-10-0-141-128 kubenswrapper[2571]: W0912 00:16:28.980479    2571 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 12 00:16:28.983974 ip-10-0-141-128 kubenswrapper[2571]: W0912 00:16:28.980483    2571 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 12 00:16:28.983974 ip-10-0-141-128 kubenswrapper[2571]: W0912 00:16:28.980485    2571 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 12 00:16:28.983974 ip-10-0-141-128 kubenswrapper[2571]: W0912 00:16:28.980488    2571 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 12 00:16:28.983974 ip-10-0-141-128 kubenswrapper[2571]: W0912 00:16:28.980490    2571 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 12 00:16:28.983974 ip-10-0-141-128 kubenswrapper[2571]: W0912 00:16:28.980493    2571 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 12 00:16:28.983974 ip-10-0-141-128 kubenswrapper[2571]: W0912 00:16:28.980495    2571 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 12 00:16:28.983974 ip-10-0-141-128 kubenswrapper[2571]: W0912 00:16:28.980498    2571 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 12 00:16:28.983974 ip-10-0-141-128 kubenswrapper[2571]: W0912 00:16:28.980500    2571 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 12 00:16:28.983974 ip-10-0-141-128 kubenswrapper[2571]: W0912 00:16:28.980503    2571 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 12 00:16:28.983974 ip-10-0-141-128 kubenswrapper[2571]: W0912 00:16:28.980505    2571 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 12 00:16:28.983974 ip-10-0-141-128 kubenswrapper[2571]: W0912 00:16:28.980508    2571 feature_gate.go:328] unrecognized feature gate: Example2
Sep 12 00:16:28.983974 ip-10-0-141-128 kubenswrapper[2571]: W0912 00:16:28.980512    2571 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 12 00:16:28.983974 ip-10-0-141-128 kubenswrapper[2571]: W0912 00:16:28.980515    2571 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 12 00:16:28.983974 ip-10-0-141-128 kubenswrapper[2571]: W0912 00:16:28.980517    2571 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 12 00:16:28.983974 ip-10-0-141-128 kubenswrapper[2571]: W0912 00:16:28.980520    2571 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 12 00:16:28.983974 ip-10-0-141-128 kubenswrapper[2571]: W0912 00:16:28.980531    2571 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 12 00:16:28.983974 ip-10-0-141-128 kubenswrapper[2571]: W0912 00:16:28.980927    2571 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 12 00:16:28.984449 ip-10-0-141-128 kubenswrapper[2571]: W0912 00:16:28.980933    2571 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 12 00:16:28.984449 ip-10-0-141-128 kubenswrapper[2571]: W0912 00:16:28.980936    2571 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 12 00:16:28.984449 ip-10-0-141-128 kubenswrapper[2571]: W0912 00:16:28.980939    2571 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 12 00:16:28.984449 ip-10-0-141-128 kubenswrapper[2571]: W0912 00:16:28.980942    2571 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 12 00:16:28.984449 ip-10-0-141-128 kubenswrapper[2571]: W0912 00:16:28.980945    2571 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 12 00:16:28.984449 ip-10-0-141-128 kubenswrapper[2571]: W0912 00:16:28.980948    2571 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 12 00:16:28.984449 ip-10-0-141-128 kubenswrapper[2571]: W0912 00:16:28.980951    2571 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 12 00:16:28.984449 ip-10-0-141-128 kubenswrapper[2571]: W0912 00:16:28.980953    2571 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 12 00:16:28.984449 ip-10-0-141-128 kubenswrapper[2571]: W0912 00:16:28.980956    2571 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 12 00:16:28.984449 ip-10-0-141-128 kubenswrapper[2571]: W0912 00:16:28.980958    2571 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 12 00:16:28.984449 ip-10-0-141-128 kubenswrapper[2571]: W0912 00:16:28.980961    2571 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 12 00:16:28.984449 ip-10-0-141-128 kubenswrapper[2571]: W0912 00:16:28.980964    2571 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 12 00:16:28.984449 ip-10-0-141-128 kubenswrapper[2571]: W0912 00:16:28.980966    2571 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 12 00:16:28.984449 ip-10-0-141-128 kubenswrapper[2571]: W0912 00:16:28.980969    2571 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 12 00:16:28.984449 ip-10-0-141-128 kubenswrapper[2571]: W0912 00:16:28.980971    2571 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 12 00:16:28.984449 ip-10-0-141-128 kubenswrapper[2571]: W0912 00:16:28.980974    2571 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 12 00:16:28.984449 ip-10-0-141-128 kubenswrapper[2571]: W0912 00:16:28.980977    2571 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 12 00:16:28.984449 ip-10-0-141-128 kubenswrapper[2571]: W0912 00:16:28.980981    2571 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 12 00:16:28.984449 ip-10-0-141-128 kubenswrapper[2571]: W0912 00:16:28.980984    2571 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 12 00:16:28.984449 ip-10-0-141-128 kubenswrapper[2571]: W0912 00:16:28.980987    2571 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 12 00:16:28.984956 ip-10-0-141-128 kubenswrapper[2571]: W0912 00:16:28.980990    2571 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 12 00:16:28.984956 ip-10-0-141-128 kubenswrapper[2571]: W0912 00:16:28.980992    2571 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 12 00:16:28.984956 ip-10-0-141-128 kubenswrapper[2571]: W0912 00:16:28.980995    2571 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 12 00:16:28.984956 ip-10-0-141-128 kubenswrapper[2571]: W0912 00:16:28.980997    2571 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 12 00:16:28.984956 ip-10-0-141-128 kubenswrapper[2571]: W0912 00:16:28.981000    2571 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 12 00:16:28.984956 ip-10-0-141-128 kubenswrapper[2571]: W0912 00:16:28.981005    2571 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 12 00:16:28.984956 ip-10-0-141-128 kubenswrapper[2571]: W0912 00:16:28.981007    2571 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 12 00:16:28.984956 ip-10-0-141-128 kubenswrapper[2571]: W0912 00:16:28.981010    2571 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 12 00:16:28.984956 ip-10-0-141-128 kubenswrapper[2571]: W0912 00:16:28.981013    2571 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 12 00:16:28.984956 ip-10-0-141-128 kubenswrapper[2571]: W0912 00:16:28.981017    2571 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 12 00:16:28.984956 ip-10-0-141-128 kubenswrapper[2571]: W0912 00:16:28.981019    2571 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 12 00:16:28.984956 ip-10-0-141-128 kubenswrapper[2571]: W0912 00:16:28.981022    2571 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 12 00:16:28.984956 ip-10-0-141-128 kubenswrapper[2571]: W0912 00:16:28.981024    2571 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 12 00:16:28.984956 ip-10-0-141-128 kubenswrapper[2571]: W0912 00:16:28.981027    2571 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 12 00:16:28.984956 ip-10-0-141-128 kubenswrapper[2571]: W0912 00:16:28.981029    2571 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 12 00:16:28.984956 ip-10-0-141-128 kubenswrapper[2571]: W0912 00:16:28.981032    2571 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 12 00:16:28.984956 ip-10-0-141-128 kubenswrapper[2571]: W0912 00:16:28.981034    2571 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 12 00:16:28.984956 ip-10-0-141-128 kubenswrapper[2571]: W0912 00:16:28.981037    2571 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 12 00:16:28.984956 ip-10-0-141-128 kubenswrapper[2571]: W0912 00:16:28.981040    2571 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 12 00:16:28.985424 ip-10-0-141-128 kubenswrapper[2571]: W0912 00:16:28.981042    2571 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 12 00:16:28.985424 ip-10-0-141-128 kubenswrapper[2571]: W0912 00:16:28.981044    2571 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 12 00:16:28.985424 ip-10-0-141-128 kubenswrapper[2571]: W0912 00:16:28.981047    2571 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 12 00:16:28.985424 ip-10-0-141-128 kubenswrapper[2571]: W0912 00:16:28.981050    2571 feature_gate.go:328] unrecognized feature gate: Example2
Sep 12 00:16:28.985424 ip-10-0-141-128 kubenswrapper[2571]: W0912 00:16:28.981052    2571 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 12 00:16:28.985424 ip-10-0-141-128 kubenswrapper[2571]: W0912 00:16:28.981054    2571 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 12 00:16:28.985424 ip-10-0-141-128 kubenswrapper[2571]: W0912 00:16:28.981057    2571 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 12 00:16:28.985424 ip-10-0-141-128 kubenswrapper[2571]: W0912 00:16:28.981060    2571 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 12 00:16:28.985424 ip-10-0-141-128 kubenswrapper[2571]: W0912 00:16:28.981062    2571 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 12 00:16:28.985424 ip-10-0-141-128 kubenswrapper[2571]: W0912 00:16:28.981065    2571 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 12 00:16:28.985424 ip-10-0-141-128 kubenswrapper[2571]: W0912 00:16:28.981069    2571 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 12 00:16:28.985424 ip-10-0-141-128 kubenswrapper[2571]: W0912 00:16:28.981072    2571 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 12 00:16:28.985424 ip-10-0-141-128 kubenswrapper[2571]: W0912 00:16:28.981075    2571 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 12 00:16:28.985424 ip-10-0-141-128 kubenswrapper[2571]: W0912 00:16:28.981078    2571 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 12 00:16:28.985424 ip-10-0-141-128 kubenswrapper[2571]: W0912 00:16:28.981080    2571 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 12 00:16:28.985424 ip-10-0-141-128 kubenswrapper[2571]: W0912 00:16:28.981083    2571 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 12 00:16:28.985424 ip-10-0-141-128 kubenswrapper[2571]: W0912 00:16:28.981086    2571 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 12 00:16:28.985424 ip-10-0-141-128 kubenswrapper[2571]: W0912 00:16:28.981088    2571 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 12 00:16:28.985424 ip-10-0-141-128 kubenswrapper[2571]: W0912 00:16:28.981091    2571 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 12 00:16:28.985424 ip-10-0-141-128 kubenswrapper[2571]: W0912 00:16:28.981094    2571 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 12 00:16:28.985924 ip-10-0-141-128 kubenswrapper[2571]: W0912 00:16:28.981097    2571 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 12 00:16:28.985924 ip-10-0-141-128 kubenswrapper[2571]: W0912 00:16:28.981100    2571 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 12 00:16:28.985924 ip-10-0-141-128 kubenswrapper[2571]: W0912 00:16:28.981103    2571 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 12 00:16:28.985924 ip-10-0-141-128 kubenswrapper[2571]: W0912 00:16:28.981106    2571 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 12 00:16:28.985924 ip-10-0-141-128 kubenswrapper[2571]: W0912 00:16:28.981109    2571 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 12 00:16:28.985924 ip-10-0-141-128 kubenswrapper[2571]: W0912 00:16:28.981111    2571 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 12 00:16:28.985924 ip-10-0-141-128 kubenswrapper[2571]: W0912 00:16:28.981114    2571 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 12 00:16:28.985924 ip-10-0-141-128 kubenswrapper[2571]: W0912 00:16:28.981116    2571 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 12 00:16:28.985924 ip-10-0-141-128 kubenswrapper[2571]: W0912 00:16:28.981119    2571 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 12 00:16:28.985924 ip-10-0-141-128 kubenswrapper[2571]: W0912 00:16:28.981122    2571 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 12 00:16:28.985924 ip-10-0-141-128 kubenswrapper[2571]: W0912 00:16:28.981124    2571 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 12 00:16:28.985924 ip-10-0-141-128 kubenswrapper[2571]: W0912 00:16:28.981127    2571 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 12 00:16:28.985924 ip-10-0-141-128 kubenswrapper[2571]: W0912 00:16:28.981129    2571 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 12 00:16:28.985924 ip-10-0-141-128 kubenswrapper[2571]: W0912 00:16:28.981131    2571 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 12 00:16:28.985924 ip-10-0-141-128 kubenswrapper[2571]: W0912 00:16:28.981135    2571 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 12 00:16:28.985924 ip-10-0-141-128 kubenswrapper[2571]: W0912 00:16:28.981139    2571 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 12 00:16:28.985924 ip-10-0-141-128 kubenswrapper[2571]: W0912 00:16:28.981143    2571 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 12 00:16:28.985924 ip-10-0-141-128 kubenswrapper[2571]: W0912 00:16:28.981146    2571 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 12 00:16:28.985924 ip-10-0-141-128 kubenswrapper[2571]: W0912 00:16:28.981149    2571 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 12 00:16:28.986389 ip-10-0-141-128 kubenswrapper[2571]: W0912 00:16:28.981151    2571 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 12 00:16:28.986389 ip-10-0-141-128 kubenswrapper[2571]: W0912 00:16:28.981153    2571 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 12 00:16:28.986389 ip-10-0-141-128 kubenswrapper[2571]: W0912 00:16:28.981156    2571 feature_gate.go:328] unrecognized feature gate: Example
Sep 12 00:16:28.986389 ip-10-0-141-128 kubenswrapper[2571]: W0912 00:16:28.981159    2571 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 12 00:16:28.986389 ip-10-0-141-128 kubenswrapper[2571]: W0912 00:16:28.981162    2571 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 12 00:16:28.986389 ip-10-0-141-128 kubenswrapper[2571]: W0912 00:16:28.981164    2571 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 12 00:16:28.986389 ip-10-0-141-128 kubenswrapper[2571]: W0912 00:16:28.981167    2571 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 12 00:16:28.986389 ip-10-0-141-128 kubenswrapper[2571]: W0912 00:16:28.981169    2571 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 12 00:16:28.986389 ip-10-0-141-128 kubenswrapper[2571]: W0912 00:16:28.981172    2571 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 12 00:16:28.986389 ip-10-0-141-128 kubenswrapper[2571]: W0912 00:16:28.981175    2571 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 12 00:16:28.986389 ip-10-0-141-128 kubenswrapper[2571]: W0912 00:16:28.981177    2571 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 12 00:16:28.986389 ip-10-0-141-128 kubenswrapper[2571]: W0912 00:16:28.981181    2571 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 12 00:16:28.986389 ip-10-0-141-128 kubenswrapper[2571]: W0912 00:16:28.981184    2571 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 12 00:16:28.986389 ip-10-0-141-128 kubenswrapper[2571]: W0912 00:16:28.981186    2571 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 12 00:16:28.986389 ip-10-0-141-128 kubenswrapper[2571]: W0912 00:16:28.981189    2571 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 12 00:16:28.986389 ip-10-0-141-128 kubenswrapper[2571]: W0912 00:16:28.981192    2571 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 12 00:16:28.986389 ip-10-0-141-128 kubenswrapper[2571]: W0912 00:16:28.981195    2571 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 12 00:16:28.986389 ip-10-0-141-128 kubenswrapper[2571]: W0912 00:16:28.981197    2571 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 12 00:16:28.986389 ip-10-0-141-128 kubenswrapper[2571]: W0912 00:16:28.981200    2571 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 12 00:16:28.986389 ip-10-0-141-128 kubenswrapper[2571]: W0912 00:16:28.981202    2571 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 12 00:16:28.986389 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:16:28.981272    2571 flags.go:64] FLAG: --address="0.0.0.0"
Sep 12 00:16:28.986920 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:16:28.981314    2571 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]"
Sep 12 00:16:28.986920 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:16:28.981321    2571 flags.go:64] FLAG: --anonymous-auth="true"
Sep 12 00:16:28.986920 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:16:28.981325    2571 flags.go:64] FLAG: --application-metrics-count-limit="100"
Sep 12 00:16:28.986920 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:16:28.981329    2571 flags.go:64] FLAG: --authentication-token-webhook="false"
Sep 12 00:16:28.986920 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:16:28.981332    2571 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s"
Sep 12 00:16:28.986920 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:16:28.981336    2571 flags.go:64] FLAG: --authorization-mode="AlwaysAllow"
Sep 12 00:16:28.986920 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:16:28.981340    2571 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s"
Sep 12 00:16:28.986920 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:16:28.981343    2571 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s"
Sep 12 00:16:28.986920 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:16:28.981346    2571 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id"
Sep 12 00:16:28.986920 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:16:28.981349    2571 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig"
Sep 12 00:16:28.986920 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:16:28.981352    2571 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki"
Sep 12 00:16:28.986920 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:16:28.981356    2571 flags.go:64] FLAG: --cgroup-driver="cgroupfs"
Sep 12 00:16:28.986920 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:16:28.981359    2571 flags.go:64] FLAG: --cgroup-root=""
Sep 12 00:16:28.986920 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:16:28.981362    2571 flags.go:64] FLAG: --cgroups-per-qos="true"
Sep 12 00:16:28.986920 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:16:28.981365    2571 flags.go:64] FLAG: --client-ca-file=""
Sep 12 00:16:28.986920 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:16:28.981369    2571 flags.go:64] FLAG: --cloud-provider="external"
Sep 12 00:16:28.986920 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:16:28.981372    2571 flags.go:64] FLAG: --cluster-dns="[]"
Sep 12 00:16:28.986920 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:16:28.981375    2571 flags.go:64] FLAG: --cluster-domain=""
Sep 12 00:16:28.986920 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:16:28.981378    2571 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf"
Sep 12 00:16:28.986920 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:16:28.981381    2571 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d"
Sep 12 00:16:28.986920 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:16:28.981384    2571 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json"
Sep 12 00:16:28.986920 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:16:28.981387    2571 flags.go:64] FLAG: --container-log-max-files="5"
Sep 12 00:16:28.986920 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:16:28.981392    2571 flags.go:64] FLAG: --container-log-max-size="10Mi"
Sep 12 00:16:28.986920 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:16:28.981395    2571 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock"
Sep 12 00:16:28.987490 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:16:28.981398    2571 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock"
Sep 12 00:16:28.987490 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:16:28.981401    2571 flags.go:64] FLAG: --containerd-namespace="k8s.io"
Sep 12 00:16:28.987490 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:16:28.981405    2571 flags.go:64] FLAG: --contention-profiling="false"
Sep 12 00:16:28.987490 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:16:28.981408    2571 flags.go:64] FLAG: --cpu-cfs-quota="true"
Sep 12 00:16:28.987490 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:16:28.981411    2571 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms"
Sep 12 00:16:28.987490 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:16:28.981414    2571 flags.go:64] FLAG: --cpu-manager-policy="none"
Sep 12 00:16:28.987490 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:16:28.981417    2571 flags.go:64] FLAG: --cpu-manager-policy-options=""
Sep 12 00:16:28.987490 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:16:28.981421    2571 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s"
Sep 12 00:16:28.987490 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:16:28.981424    2571 flags.go:64] FLAG: --enable-controller-attach-detach="true"
Sep 12 00:16:28.987490 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:16:28.981426    2571 flags.go:64] FLAG: --enable-debugging-handlers="true"
Sep 12 00:16:28.987490 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:16:28.981429    2571 flags.go:64] FLAG: --enable-load-reader="false"
Sep 12 00:16:28.987490 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:16:28.981432    2571 flags.go:64] FLAG: --enable-server="true"
Sep 12 00:16:28.987490 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:16:28.981435    2571 flags.go:64] FLAG: --enforce-node-allocatable="[pods]"
Sep 12 00:16:28.987490 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:16:28.981440    2571 flags.go:64] FLAG: --event-burst="100"
Sep 12 00:16:28.987490 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:16:28.981443    2571 flags.go:64] FLAG: --event-qps="50"
Sep 12 00:16:28.987490 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:16:28.981446    2571 flags.go:64] FLAG: --event-storage-age-limit="default=0"
Sep 12 00:16:28.987490 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:16:28.981448    2571 flags.go:64] FLAG: --event-storage-event-limit="default=0"
Sep 12 00:16:28.987490 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:16:28.981451    2571 flags.go:64] FLAG: --eviction-hard=""
Sep 12 00:16:28.987490 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:16:28.981455    2571 flags.go:64] FLAG: --eviction-max-pod-grace-period="0"
Sep 12 00:16:28.987490 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:16:28.981458    2571 flags.go:64] FLAG: --eviction-minimum-reclaim=""
Sep 12 00:16:28.987490 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:16:28.981460    2571 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s"
Sep 12 00:16:28.987490 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:16:28.981463    2571 flags.go:64] FLAG: --eviction-soft=""
Sep 12 00:16:28.987490 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:16:28.981466    2571 flags.go:64] FLAG: --eviction-soft-grace-period=""
Sep 12 00:16:28.987490 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:16:28.981469    2571 flags.go:64] FLAG: --exit-on-lock-contention="false"
Sep 12 00:16:28.987490 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:16:28.981472    2571 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false"
Sep 12 00:16:28.988178 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:16:28.981476    2571 flags.go:64] FLAG: --experimental-mounter-path=""
Sep 12 00:16:28.988178 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:16:28.981478    2571 flags.go:64] FLAG: --fail-cgroupv1="false"
Sep 12 00:16:28.988178 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:16:28.981481    2571 flags.go:64] FLAG: --fail-swap-on="true"
Sep 12 00:16:28.988178 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:16:28.981484    2571 flags.go:64] FLAG: --feature-gates=""
Sep 12 00:16:28.988178 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:16:28.981487    2571 flags.go:64] FLAG: --file-check-frequency="20s"
Sep 12 00:16:28.988178 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:16:28.981492    2571 flags.go:64] FLAG: --global-housekeeping-interval="1m0s"
Sep 12 00:16:28.988178 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:16:28.981496    2571 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge"
Sep 12 00:16:28.988178 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:16:28.981499    2571 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1"
Sep 12 00:16:28.988178 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:16:28.981502    2571 flags.go:64] FLAG: --healthz-port="10248"
Sep 12 00:16:28.988178 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:16:28.981506    2571 flags.go:64] FLAG: --help="false"
Sep 12 00:16:28.988178 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:16:28.981509    2571 flags.go:64] FLAG: --hostname-override="ip-10-0-141-128.ec2.internal"
Sep 12 00:16:28.988178 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:16:28.981512    2571 flags.go:64] FLAG: --housekeeping-interval="10s"
Sep 12 00:16:28.988178 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:16:28.981514    2571 flags.go:64] FLAG: --http-check-frequency="20s"
Sep 12 00:16:28.988178 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:16:28.981517    2571 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins"
Sep 12 00:16:28.988178 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:16:28.981521    2571 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml"
Sep 12 00:16:28.988178 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:16:28.981524    2571 flags.go:64] FLAG: --image-gc-high-threshold="85"
Sep 12 00:16:28.988178 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:16:28.981527    2571 flags.go:64] FLAG: --image-gc-low-threshold="80"
Sep 12 00:16:28.988178 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:16:28.981530    2571 flags.go:64] FLAG: --image-service-endpoint=""
Sep 12 00:16:28.988178 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:16:28.981533    2571 flags.go:64] FLAG: --kernel-memcg-notification="false"
Sep 12 00:16:28.988178 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:16:28.981536    2571 flags.go:64] FLAG: --kube-api-burst="100"
Sep 12 00:16:28.988178 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:16:28.981539    2571 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf"
Sep 12 00:16:28.988178 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:16:28.981542    2571 flags.go:64] FLAG: --kube-api-qps="50"
Sep 12 00:16:28.988178 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:16:28.981545    2571 flags.go:64] FLAG: --kube-reserved=""
Sep 12 00:16:28.988178 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:16:28.981548    2571 flags.go:64] FLAG: --kube-reserved-cgroup=""
Sep 12 00:16:28.988800 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:16:28.981551    2571 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig"
Sep 12 00:16:28.988800 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:16:28.981554    2571 flags.go:64] FLAG: --kubelet-cgroups=""
Sep 12 00:16:28.988800 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:16:28.981556    2571 flags.go:64] FLAG: --local-storage-capacity-isolation="true"
Sep 12 00:16:28.988800 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:16:28.981559    2571 flags.go:64] FLAG: --lock-file=""
Sep 12 00:16:28.988800 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:16:28.981562    2571 flags.go:64] FLAG: --log-cadvisor-usage="false"
Sep 12 00:16:28.988800 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:16:28.981565    2571 flags.go:64] FLAG: --log-flush-frequency="5s"
Sep 12 00:16:28.988800 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:16:28.981568    2571 flags.go:64] FLAG: --log-json-info-buffer-size="0"
Sep 12 00:16:28.988800 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:16:28.981573    2571 flags.go:64] FLAG: --log-json-split-stream="false"
Sep 12 00:16:28.988800 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:16:28.981576    2571 flags.go:64] FLAG: --log-text-info-buffer-size="0"
Sep 12 00:16:28.988800 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:16:28.981579    2571 flags.go:64] FLAG: --log-text-split-stream="false"
Sep 12 00:16:28.988800 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:16:28.981582    2571 flags.go:64] FLAG: --logging-format="text"
Sep 12 00:16:28.988800 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:16:28.981596    2571 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id"
Sep 12 00:16:28.988800 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:16:28.981600    2571 flags.go:64] FLAG: --make-iptables-util-chains="true"
Sep 12 00:16:28.988800 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:16:28.981603    2571 flags.go:64] FLAG: --manifest-url=""
Sep 12 00:16:28.988800 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:16:28.981606    2571 flags.go:64] FLAG: --manifest-url-header=""
Sep 12 00:16:28.988800 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:16:28.981610    2571 flags.go:64] FLAG: --max-housekeeping-interval="15s"
Sep 12 00:16:28.988800 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:16:28.981613    2571 flags.go:64] FLAG: --max-open-files="1000000"
Sep 12 00:16:28.988800 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:16:28.981618    2571 flags.go:64] FLAG: --max-pods="110"
Sep 12 00:16:28.988800 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:16:28.981621    2571 flags.go:64] FLAG: --maximum-dead-containers="-1"
Sep 12 00:16:28.988800 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:16:28.981624    2571 flags.go:64] FLAG: --maximum-dead-containers-per-container="1"
Sep 12 00:16:28.988800 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:16:28.981631    2571 flags.go:64] FLAG: --memory-manager-policy="None"
Sep 12 00:16:28.988800 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:16:28.981634    2571 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s"
Sep 12 00:16:28.988800 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:16:28.981637    2571 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s"
Sep 12 00:16:28.988800 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:16:28.981640    2571 flags.go:64] FLAG: --node-ip="0.0.0.0"
Sep 12 00:16:28.988800 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:16:28.981643    2571 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel"
Sep 12 00:16:28.989407 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:16:28.981650    2571 flags.go:64] FLAG: --node-status-max-images="50"
Sep 12 00:16:28.989407 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:16:28.981653    2571 flags.go:64] FLAG: --node-status-update-frequency="10s"
Sep 12 00:16:28.989407 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:16:28.981656    2571 flags.go:64] FLAG: --oom-score-adj="-999"
Sep 12 00:16:28.989407 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:16:28.981659    2571 flags.go:64] FLAG: --pod-cidr=""
Sep 12 00:16:28.989407 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:16:28.981662    2571 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:eee392dbe3ac2ed4a38bca29ae3bd486f5b7780406c62c600154cd1906871b10"
Sep 12 00:16:28.989407 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:16:28.981667    2571 flags.go:64] FLAG: --pod-manifest-path=""
Sep 12 00:16:28.989407 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:16:28.981670    2571 flags.go:64] FLAG: --pod-max-pids="-1"
Sep 12 00:16:28.989407 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:16:28.981673    2571 flags.go:64] FLAG: --pods-per-core="0"
Sep 12 00:16:28.989407 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:16:28.981675    2571 flags.go:64] FLAG: --port="10250"
Sep 12 00:16:28.989407 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:16:28.981678    2571 flags.go:64] FLAG: --protect-kernel-defaults="false"
Sep 12 00:16:28.989407 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:16:28.981681    2571 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-05c73d3a3d4f0ffae"
Sep 12 00:16:28.989407 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:16:28.981685    2571 flags.go:64] FLAG: --qos-reserved=""
Sep 12 00:16:28.989407 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:16:28.981688    2571 flags.go:64] FLAG: --read-only-port="10255"
Sep 12 00:16:28.989407 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:16:28.981690    2571 flags.go:64] FLAG: --register-node="true"
Sep 12 00:16:28.989407 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:16:28.981693    2571 flags.go:64] FLAG: --register-with-taints=""
Sep 12 00:16:28.989407 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:16:28.981696    2571 flags.go:64] FLAG: --registry-burst="10"
Sep 12 00:16:28.989407 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:16:28.981700    2571 flags.go:64] FLAG: --registry-qps="5"
Sep 12 00:16:28.989407 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:16:28.981703    2571 flags.go:64] FLAG: --reserved-cpus=""
Sep 12 00:16:28.989407 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:16:28.981706    2571 flags.go:64] FLAG: --reserved-memory=""
Sep 12 00:16:28.989407 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:16:28.981709    2571 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf"
Sep 12 00:16:28.989407 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:16:28.981712    2571 flags.go:64] FLAG: --root-dir="/var/lib/kubelet"
Sep 12 00:16:28.989407 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:16:28.981715    2571 flags.go:64] FLAG: --rotate-certificates="false"
Sep 12 00:16:28.989407 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:16:28.981719    2571 flags.go:64] FLAG: --rotate-server-certificates="false"
Sep 12 00:16:28.989407 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:16:28.981722    2571 flags.go:64] FLAG: --runonce="false"
Sep 12 00:16:28.989407 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:16:28.981725    2571 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service"
Sep 12 00:16:28.990027 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:16:28.981728    2571 flags.go:64] FLAG: --runtime-request-timeout="2m0s"
Sep 12 00:16:28.990027 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:16:28.981731    2571 flags.go:64] FLAG: --seccomp-default="false"
Sep 12 00:16:28.990027 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:16:28.981734    2571 flags.go:64] FLAG: --serialize-image-pulls="true"
Sep 12 00:16:28.990027 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:16:28.981737    2571 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s"
Sep 12 00:16:28.990027 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:16:28.981741    2571 flags.go:64] FLAG: --storage-driver-db="cadvisor"
Sep 12 00:16:28.990027 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:16:28.981743    2571 flags.go:64] FLAG: --storage-driver-host="localhost:8086"
Sep 12 00:16:28.990027 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:16:28.981746    2571 flags.go:64] FLAG: --storage-driver-password="root"
Sep 12 00:16:28.990027 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:16:28.981749    2571 flags.go:64] FLAG: --storage-driver-secure="false"
Sep 12 00:16:28.990027 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:16:28.981752    2571 flags.go:64] FLAG: --storage-driver-table="stats"
Sep 12 00:16:28.990027 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:16:28.981755    2571 flags.go:64] FLAG: --storage-driver-user="root"
Sep 12 00:16:28.990027 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:16:28.981758    2571 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s"
Sep 12 00:16:28.990027 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:16:28.981761    2571 flags.go:64] FLAG: --sync-frequency="1m0s"
Sep 12 00:16:28.990027 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:16:28.981764    2571 flags.go:64] FLAG: --system-cgroups=""
Sep 12 00:16:28.990027 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:16:28.981766    2571 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi"
Sep 12 00:16:28.990027 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:16:28.981771    2571 flags.go:64] FLAG: --system-reserved-cgroup=""
Sep 12 00:16:28.990027 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:16:28.981774    2571 flags.go:64] FLAG: --tls-cert-file=""
Sep 12 00:16:28.990027 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:16:28.981777    2571 flags.go:64] FLAG: --tls-cipher-suites="[]"
Sep 12 00:16:28.990027 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:16:28.981780    2571 flags.go:64] FLAG: --tls-min-version=""
Sep 12 00:16:28.990027 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:16:28.981783    2571 flags.go:64] FLAG: --tls-private-key-file=""
Sep 12 00:16:28.990027 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:16:28.981786    2571 flags.go:64] FLAG: --topology-manager-policy="none"
Sep 12 00:16:28.990027 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:16:28.981789    2571 flags.go:64] FLAG: --topology-manager-policy-options=""
Sep 12 00:16:28.990027 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:16:28.981792    2571 flags.go:64] FLAG: --topology-manager-scope="container"
Sep 12 00:16:28.990027 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:16:28.981795    2571 flags.go:64] FLAG: --v="2"
Sep 12 00:16:28.990027 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:16:28.981798    2571 flags.go:64] FLAG: --version="false"
Sep 12 00:16:28.990027 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:16:28.981803    2571 flags.go:64] FLAG: --vmodule=""
Sep 12 00:16:28.990648 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:16:28.981806    2571 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec"
Sep 12 00:16:28.990648 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:16:28.981809    2571 flags.go:64] FLAG: --volume-stats-agg-period="1m0s"
Sep 12 00:16:28.990648 ip-10-0-141-128 kubenswrapper[2571]: W0912 00:16:28.981915    2571 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 12 00:16:28.990648 ip-10-0-141-128 kubenswrapper[2571]: W0912 00:16:28.981918    2571 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 12 00:16:28.990648 ip-10-0-141-128 kubenswrapper[2571]: W0912 00:16:28.981921    2571 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 12 00:16:28.990648 ip-10-0-141-128 kubenswrapper[2571]: W0912 00:16:28.981924    2571 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 12 00:16:28.990648 ip-10-0-141-128 kubenswrapper[2571]: W0912 00:16:28.981929    2571 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 12 00:16:28.990648 ip-10-0-141-128 kubenswrapper[2571]: W0912 00:16:28.981932    2571 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 12 00:16:28.990648 ip-10-0-141-128 kubenswrapper[2571]: W0912 00:16:28.981934    2571 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 12 00:16:28.990648 ip-10-0-141-128 kubenswrapper[2571]: W0912 00:16:28.981937    2571 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 12 00:16:28.990648 ip-10-0-141-128 kubenswrapper[2571]: W0912 00:16:28.981940    2571 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 12 00:16:28.990648 ip-10-0-141-128 kubenswrapper[2571]: W0912 00:16:28.981944    2571 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 12 00:16:28.990648 ip-10-0-141-128 kubenswrapper[2571]: W0912 00:16:28.981946    2571 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 12 00:16:28.990648 ip-10-0-141-128 kubenswrapper[2571]: W0912 00:16:28.981949    2571 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 12 00:16:28.990648 ip-10-0-141-128 kubenswrapper[2571]: W0912 00:16:28.981952    2571 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 12 00:16:28.990648 ip-10-0-141-128 kubenswrapper[2571]: W0912 00:16:28.981954    2571 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 12 00:16:28.990648 ip-10-0-141-128 kubenswrapper[2571]: W0912 00:16:28.981957    2571 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 12 00:16:28.990648 ip-10-0-141-128 kubenswrapper[2571]: W0912 00:16:28.981959    2571 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 12 00:16:28.990648 ip-10-0-141-128 kubenswrapper[2571]: W0912 00:16:28.981962    2571 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 12 00:16:28.991110 ip-10-0-141-128 kubenswrapper[2571]: W0912 00:16:28.981964    2571 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 12 00:16:28.991110 ip-10-0-141-128 kubenswrapper[2571]: W0912 00:16:28.981967    2571 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 12 00:16:28.991110 ip-10-0-141-128 kubenswrapper[2571]: W0912 00:16:28.981970    2571 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 12 00:16:28.991110 ip-10-0-141-128 kubenswrapper[2571]: W0912 00:16:28.981972    2571 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 12 00:16:28.991110 ip-10-0-141-128 kubenswrapper[2571]: W0912 00:16:28.981975    2571 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 12 00:16:28.991110 ip-10-0-141-128 kubenswrapper[2571]: W0912 00:16:28.981977    2571 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 12 00:16:28.991110 ip-10-0-141-128 kubenswrapper[2571]: W0912 00:16:28.981980    2571 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 12 00:16:28.991110 ip-10-0-141-128 kubenswrapper[2571]: W0912 00:16:28.981982    2571 feature_gate.go:328] unrecognized feature gate: Example
Sep 12 00:16:28.991110 ip-10-0-141-128 kubenswrapper[2571]: W0912 00:16:28.981985    2571 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 12 00:16:28.991110 ip-10-0-141-128 kubenswrapper[2571]: W0912 00:16:28.981987    2571 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 12 00:16:28.991110 ip-10-0-141-128 kubenswrapper[2571]: W0912 00:16:28.981990    2571 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 12 00:16:28.991110 ip-10-0-141-128 kubenswrapper[2571]: W0912 00:16:28.981993    2571 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 12 00:16:28.991110 ip-10-0-141-128 kubenswrapper[2571]: W0912 00:16:28.981995    2571 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 12 00:16:28.991110 ip-10-0-141-128 kubenswrapper[2571]: W0912 00:16:28.981999    2571 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 12 00:16:28.991110 ip-10-0-141-128 kubenswrapper[2571]: W0912 00:16:28.982002    2571 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 12 00:16:28.991110 ip-10-0-141-128 kubenswrapper[2571]: W0912 00:16:28.982005    2571 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 12 00:16:28.991110 ip-10-0-141-128 kubenswrapper[2571]: W0912 00:16:28.982007    2571 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 12 00:16:28.991110 ip-10-0-141-128 kubenswrapper[2571]: W0912 00:16:28.982009    2571 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 12 00:16:28.991110 ip-10-0-141-128 kubenswrapper[2571]: W0912 00:16:28.982012    2571 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 12 00:16:28.991110 ip-10-0-141-128 kubenswrapper[2571]: W0912 00:16:28.982015    2571 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 12 00:16:28.991627 ip-10-0-141-128 kubenswrapper[2571]: W0912 00:16:28.982018    2571 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 12 00:16:28.991627 ip-10-0-141-128 kubenswrapper[2571]: W0912 00:16:28.982022    2571 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 12 00:16:28.991627 ip-10-0-141-128 kubenswrapper[2571]: W0912 00:16:28.982026    2571 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 12 00:16:28.991627 ip-10-0-141-128 kubenswrapper[2571]: W0912 00:16:28.982029    2571 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 12 00:16:28.991627 ip-10-0-141-128 kubenswrapper[2571]: W0912 00:16:28.982031    2571 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 12 00:16:28.991627 ip-10-0-141-128 kubenswrapper[2571]: W0912 00:16:28.982034    2571 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 12 00:16:28.991627 ip-10-0-141-128 kubenswrapper[2571]: W0912 00:16:28.982037    2571 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 12 00:16:28.991627 ip-10-0-141-128 kubenswrapper[2571]: W0912 00:16:28.982040    2571 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 12 00:16:28.991627 ip-10-0-141-128 kubenswrapper[2571]: W0912 00:16:28.982043    2571 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 12 00:16:28.991627 ip-10-0-141-128 kubenswrapper[2571]: W0912 00:16:28.982046    2571 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 12 00:16:28.991627 ip-10-0-141-128 kubenswrapper[2571]: W0912 00:16:28.982049    2571 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 12 00:16:28.991627 ip-10-0-141-128 kubenswrapper[2571]: W0912 00:16:28.982052    2571 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 12 00:16:28.991627 ip-10-0-141-128 kubenswrapper[2571]: W0912 00:16:28.982054    2571 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 12 00:16:28.991627 ip-10-0-141-128 kubenswrapper[2571]: W0912 00:16:28.982057    2571 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 12 00:16:28.991627 ip-10-0-141-128 kubenswrapper[2571]: W0912 00:16:28.982059    2571 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 12 00:16:28.991627 ip-10-0-141-128 kubenswrapper[2571]: W0912 00:16:28.982062    2571 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 12 00:16:28.991627 ip-10-0-141-128 kubenswrapper[2571]: W0912 00:16:28.982064    2571 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 12 00:16:28.991627 ip-10-0-141-128 kubenswrapper[2571]: W0912 00:16:28.982067    2571 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 12 00:16:28.991627 ip-10-0-141-128 kubenswrapper[2571]: W0912 00:16:28.982069    2571 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 12 00:16:28.992115 ip-10-0-141-128 kubenswrapper[2571]: W0912 00:16:28.982072    2571 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 12 00:16:28.992115 ip-10-0-141-128 kubenswrapper[2571]: W0912 00:16:28.982076    2571 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 12 00:16:28.992115 ip-10-0-141-128 kubenswrapper[2571]: W0912 00:16:28.982079    2571 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 12 00:16:28.992115 ip-10-0-141-128 kubenswrapper[2571]: W0912 00:16:28.982081    2571 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 12 00:16:28.992115 ip-10-0-141-128 kubenswrapper[2571]: W0912 00:16:28.982084    2571 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 12 00:16:28.992115 ip-10-0-141-128 kubenswrapper[2571]: W0912 00:16:28.982086    2571 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 12 00:16:28.992115 ip-10-0-141-128 kubenswrapper[2571]: W0912 00:16:28.982089    2571 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 12 00:16:28.992115 ip-10-0-141-128 kubenswrapper[2571]: W0912 00:16:28.982093    2571 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 12 00:16:28.992115 ip-10-0-141-128 kubenswrapper[2571]: W0912 00:16:28.982109    2571 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 12 00:16:28.992115 ip-10-0-141-128 kubenswrapper[2571]: W0912 00:16:28.982113    2571 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 12 00:16:28.992115 ip-10-0-141-128 kubenswrapper[2571]: W0912 00:16:28.982116    2571 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 12 00:16:28.992115 ip-10-0-141-128 kubenswrapper[2571]: W0912 00:16:28.982119    2571 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 12 00:16:28.992115 ip-10-0-141-128 kubenswrapper[2571]: W0912 00:16:28.982122    2571 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 12 00:16:28.992115 ip-10-0-141-128 kubenswrapper[2571]: W0912 00:16:28.982125    2571 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 12 00:16:28.992115 ip-10-0-141-128 kubenswrapper[2571]: W0912 00:16:28.982128    2571 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 12 00:16:28.992115 ip-10-0-141-128 kubenswrapper[2571]: W0912 00:16:28.982131    2571 feature_gate.go:328] unrecognized feature gate: Example2
Sep 12 00:16:28.992115 ip-10-0-141-128 kubenswrapper[2571]: W0912 00:16:28.982134    2571 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 12 00:16:28.992115 ip-10-0-141-128 kubenswrapper[2571]: W0912 00:16:28.982136    2571 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 12 00:16:28.992115 ip-10-0-141-128 kubenswrapper[2571]: W0912 00:16:28.982140    2571 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 12 00:16:28.992115 ip-10-0-141-128 kubenswrapper[2571]: W0912 00:16:28.982143    2571 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 12 00:16:28.992627 ip-10-0-141-128 kubenswrapper[2571]: W0912 00:16:28.982147    2571 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 12 00:16:28.992627 ip-10-0-141-128 kubenswrapper[2571]: W0912 00:16:28.982149    2571 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 12 00:16:28.992627 ip-10-0-141-128 kubenswrapper[2571]: W0912 00:16:28.982152    2571 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 12 00:16:28.992627 ip-10-0-141-128 kubenswrapper[2571]: W0912 00:16:28.982155    2571 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 12 00:16:28.992627 ip-10-0-141-128 kubenswrapper[2571]: W0912 00:16:28.982158    2571 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 12 00:16:28.992627 ip-10-0-141-128 kubenswrapper[2571]: W0912 00:16:28.982161    2571 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 12 00:16:28.992627 ip-10-0-141-128 kubenswrapper[2571]: W0912 00:16:28.982163    2571 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 12 00:16:28.992627 ip-10-0-141-128 kubenswrapper[2571]: W0912 00:16:28.982165    2571 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 12 00:16:28.992627 ip-10-0-141-128 kubenswrapper[2571]: W0912 00:16:28.982168    2571 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 12 00:16:28.992627 ip-10-0-141-128 kubenswrapper[2571]: W0912 00:16:28.982171    2571 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 12 00:16:28.992627 ip-10-0-141-128 kubenswrapper[2571]: W0912 00:16:28.982173    2571 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 12 00:16:28.992627 ip-10-0-141-128 kubenswrapper[2571]: W0912 00:16:28.982176    2571 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 12 00:16:28.992627 ip-10-0-141-128 kubenswrapper[2571]: W0912 00:16:28.982179    2571 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 12 00:16:28.992627 ip-10-0-141-128 kubenswrapper[2571]: W0912 00:16:28.982182    2571 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 12 00:16:28.992627 ip-10-0-141-128 kubenswrapper[2571]: W0912 00:16:28.982185    2571 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 12 00:16:28.992627 ip-10-0-141-128 kubenswrapper[2571]: W0912 00:16:28.982188    2571 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 12 00:16:28.992627 ip-10-0-141-128 kubenswrapper[2571]: W0912 00:16:28.982190    2571 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 12 00:16:28.992627 ip-10-0-141-128 kubenswrapper[2571]: W0912 00:16:28.982193    2571 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 12 00:16:28.992627 ip-10-0-141-128 kubenswrapper[2571]: W0912 00:16:28.982196    2571 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 12 00:16:28.992627 ip-10-0-141-128 kubenswrapper[2571]: W0912 00:16:28.982198    2571 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 12 00:16:28.993124 ip-10-0-141-128 kubenswrapper[2571]: W0912 00:16:28.982201    2571 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 12 00:16:28.993124 ip-10-0-141-128 kubenswrapper[2571]: W0912 00:16:28.982204    2571 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 12 00:16:28.993124 ip-10-0-141-128 kubenswrapper[2571]: W0912 00:16:28.982207    2571 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 12 00:16:28.993124 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:16:28.982670    2571 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Sep 12 00:16:28.993124 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:16:28.989646    2571 server.go:529] "Kubelet version" kubeletVersion="v1.34.9"
Sep 12 00:16:28.993124 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:16:28.989660    2571 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK=""
Sep 12 00:16:28.993124 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:16:28.990439    2571 watchdog_linux.go:95] "Systemd watchdog is not enabled"
Sep 12 00:16:28.993124 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:16:28.990450    2571 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started."
Sep 12 00:16:28.993124 ip-10-0-141-128 kubenswrapper[2571]: W0912 00:16:28.990505    2571 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 12 00:16:28.993124 ip-10-0-141-128 kubenswrapper[2571]: W0912 00:16:28.990510    2571 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 12 00:16:28.993124 ip-10-0-141-128 kubenswrapper[2571]: W0912 00:16:28.990513    2571 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 12 00:16:28.993124 ip-10-0-141-128 kubenswrapper[2571]: W0912 00:16:28.990516    2571 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 12 00:16:28.993124 ip-10-0-141-128 kubenswrapper[2571]: W0912 00:16:28.990520    2571 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 12 00:16:28.993124 ip-10-0-141-128 kubenswrapper[2571]: W0912 00:16:28.990523    2571 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 12 00:16:28.993124 ip-10-0-141-128 kubenswrapper[2571]: W0912 00:16:28.990526    2571 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 12 00:16:28.993501 ip-10-0-141-128 kubenswrapper[2571]: W0912 00:16:28.990529    2571 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 12 00:16:28.993501 ip-10-0-141-128 kubenswrapper[2571]: W0912 00:16:28.990532    2571 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 12 00:16:28.993501 ip-10-0-141-128 kubenswrapper[2571]: W0912 00:16:28.990535    2571 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 12 00:16:28.993501 ip-10-0-141-128 kubenswrapper[2571]: W0912 00:16:28.990538    2571 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 12 00:16:28.993501 ip-10-0-141-128 kubenswrapper[2571]: W0912 00:16:28.990540    2571 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 12 00:16:28.993501 ip-10-0-141-128 kubenswrapper[2571]: W0912 00:16:28.990543    2571 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 12 00:16:28.993501 ip-10-0-141-128 kubenswrapper[2571]: W0912 00:16:28.990546    2571 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 12 00:16:28.993501 ip-10-0-141-128 kubenswrapper[2571]: W0912 00:16:28.990548    2571 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 12 00:16:28.993501 ip-10-0-141-128 kubenswrapper[2571]: W0912 00:16:28.990551    2571 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 12 00:16:28.993501 ip-10-0-141-128 kubenswrapper[2571]: W0912 00:16:28.990553    2571 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 12 00:16:28.993501 ip-10-0-141-128 kubenswrapper[2571]: W0912 00:16:28.990556    2571 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 12 00:16:28.993501 ip-10-0-141-128 kubenswrapper[2571]: W0912 00:16:28.990559    2571 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 12 00:16:28.993501 ip-10-0-141-128 kubenswrapper[2571]: W0912 00:16:28.990562    2571 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 12 00:16:28.993501 ip-10-0-141-128 kubenswrapper[2571]: W0912 00:16:28.990565    2571 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 12 00:16:28.993501 ip-10-0-141-128 kubenswrapper[2571]: W0912 00:16:28.990568    2571 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 12 00:16:28.993501 ip-10-0-141-128 kubenswrapper[2571]: W0912 00:16:28.990570    2571 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 12 00:16:28.993501 ip-10-0-141-128 kubenswrapper[2571]: W0912 00:16:28.990573    2571 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 12 00:16:28.993501 ip-10-0-141-128 kubenswrapper[2571]: W0912 00:16:28.990575    2571 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 12 00:16:28.993501 ip-10-0-141-128 kubenswrapper[2571]: W0912 00:16:28.990578    2571 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 12 00:16:28.993501 ip-10-0-141-128 kubenswrapper[2571]: W0912 00:16:28.990581    2571 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 12 00:16:28.994051 ip-10-0-141-128 kubenswrapper[2571]: W0912 00:16:28.990594    2571 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 12 00:16:28.994051 ip-10-0-141-128 kubenswrapper[2571]: W0912 00:16:28.990597    2571 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 12 00:16:28.994051 ip-10-0-141-128 kubenswrapper[2571]: W0912 00:16:28.990602    2571 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 12 00:16:28.994051 ip-10-0-141-128 kubenswrapper[2571]: W0912 00:16:28.990605    2571 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 12 00:16:28.994051 ip-10-0-141-128 kubenswrapper[2571]: W0912 00:16:28.990608    2571 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 12 00:16:28.994051 ip-10-0-141-128 kubenswrapper[2571]: W0912 00:16:28.990612    2571 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 12 00:16:28.994051 ip-10-0-141-128 kubenswrapper[2571]: W0912 00:16:28.990616    2571 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 12 00:16:28.994051 ip-10-0-141-128 kubenswrapper[2571]: W0912 00:16:28.990619    2571 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 12 00:16:28.994051 ip-10-0-141-128 kubenswrapper[2571]: W0912 00:16:28.990622    2571 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 12 00:16:28.994051 ip-10-0-141-128 kubenswrapper[2571]: W0912 00:16:28.990626    2571 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 12 00:16:28.994051 ip-10-0-141-128 kubenswrapper[2571]: W0912 00:16:28.990628    2571 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 12 00:16:28.994051 ip-10-0-141-128 kubenswrapper[2571]: W0912 00:16:28.990631    2571 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 12 00:16:28.994051 ip-10-0-141-128 kubenswrapper[2571]: W0912 00:16:28.990634    2571 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 12 00:16:28.994051 ip-10-0-141-128 kubenswrapper[2571]: W0912 00:16:28.990637    2571 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 12 00:16:28.994051 ip-10-0-141-128 kubenswrapper[2571]: W0912 00:16:28.990640    2571 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 12 00:16:28.994051 ip-10-0-141-128 kubenswrapper[2571]: W0912 00:16:28.990642    2571 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 12 00:16:28.994051 ip-10-0-141-128 kubenswrapper[2571]: W0912 00:16:28.990645    2571 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 12 00:16:28.994051 ip-10-0-141-128 kubenswrapper[2571]: W0912 00:16:28.990647    2571 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 12 00:16:28.994051 ip-10-0-141-128 kubenswrapper[2571]: W0912 00:16:28.990651    2571 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 12 00:16:28.994051 ip-10-0-141-128 kubenswrapper[2571]: W0912 00:16:28.990653    2571 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 12 00:16:28.994552 ip-10-0-141-128 kubenswrapper[2571]: W0912 00:16:28.990656    2571 feature_gate.go:328] unrecognized feature gate: Example2
Sep 12 00:16:28.994552 ip-10-0-141-128 kubenswrapper[2571]: W0912 00:16:28.990659    2571 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 12 00:16:28.994552 ip-10-0-141-128 kubenswrapper[2571]: W0912 00:16:28.990661    2571 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 12 00:16:28.994552 ip-10-0-141-128 kubenswrapper[2571]: W0912 00:16:28.990664    2571 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 12 00:16:28.994552 ip-10-0-141-128 kubenswrapper[2571]: W0912 00:16:28.990667    2571 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 12 00:16:28.994552 ip-10-0-141-128 kubenswrapper[2571]: W0912 00:16:28.990669    2571 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 12 00:16:28.994552 ip-10-0-141-128 kubenswrapper[2571]: W0912 00:16:28.990672    2571 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 12 00:16:28.994552 ip-10-0-141-128 kubenswrapper[2571]: W0912 00:16:28.990675    2571 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 12 00:16:28.994552 ip-10-0-141-128 kubenswrapper[2571]: W0912 00:16:28.990677    2571 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 12 00:16:28.994552 ip-10-0-141-128 kubenswrapper[2571]: W0912 00:16:28.990680    2571 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 12 00:16:28.994552 ip-10-0-141-128 kubenswrapper[2571]: W0912 00:16:28.990683    2571 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 12 00:16:28.994552 ip-10-0-141-128 kubenswrapper[2571]: W0912 00:16:28.990685    2571 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 12 00:16:28.994552 ip-10-0-141-128 kubenswrapper[2571]: W0912 00:16:28.990688    2571 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 12 00:16:28.994552 ip-10-0-141-128 kubenswrapper[2571]: W0912 00:16:28.990691    2571 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 12 00:16:28.994552 ip-10-0-141-128 kubenswrapper[2571]: W0912 00:16:28.990694    2571 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 12 00:16:28.994552 ip-10-0-141-128 kubenswrapper[2571]: W0912 00:16:28.990697    2571 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 12 00:16:28.994552 ip-10-0-141-128 kubenswrapper[2571]: W0912 00:16:28.990700    2571 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 12 00:16:28.994552 ip-10-0-141-128 kubenswrapper[2571]: W0912 00:16:28.990703    2571 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 12 00:16:28.994552 ip-10-0-141-128 kubenswrapper[2571]: W0912 00:16:28.990707    2571 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 12 00:16:28.994552 ip-10-0-141-128 kubenswrapper[2571]: W0912 00:16:28.990710    2571 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 12 00:16:28.995070 ip-10-0-141-128 kubenswrapper[2571]: W0912 00:16:28.990712    2571 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 12 00:16:28.995070 ip-10-0-141-128 kubenswrapper[2571]: W0912 00:16:28.990715    2571 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 12 00:16:28.995070 ip-10-0-141-128 kubenswrapper[2571]: W0912 00:16:28.990718    2571 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 12 00:16:28.995070 ip-10-0-141-128 kubenswrapper[2571]: W0912 00:16:28.990720    2571 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 12 00:16:28.995070 ip-10-0-141-128 kubenswrapper[2571]: W0912 00:16:28.990723    2571 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 12 00:16:28.995070 ip-10-0-141-128 kubenswrapper[2571]: W0912 00:16:28.990726    2571 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 12 00:16:28.995070 ip-10-0-141-128 kubenswrapper[2571]: W0912 00:16:28.990728    2571 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 12 00:16:28.995070 ip-10-0-141-128 kubenswrapper[2571]: W0912 00:16:28.990731    2571 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 12 00:16:28.995070 ip-10-0-141-128 kubenswrapper[2571]: W0912 00:16:28.990733    2571 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 12 00:16:28.995070 ip-10-0-141-128 kubenswrapper[2571]: W0912 00:16:28.990736    2571 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 12 00:16:28.995070 ip-10-0-141-128 kubenswrapper[2571]: W0912 00:16:28.990738    2571 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 12 00:16:28.995070 ip-10-0-141-128 kubenswrapper[2571]: W0912 00:16:28.990741    2571 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 12 00:16:28.995070 ip-10-0-141-128 kubenswrapper[2571]: W0912 00:16:28.990744    2571 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 12 00:16:28.995070 ip-10-0-141-128 kubenswrapper[2571]: W0912 00:16:28.990746    2571 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 12 00:16:28.995070 ip-10-0-141-128 kubenswrapper[2571]: W0912 00:16:28.990749    2571 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 12 00:16:28.995070 ip-10-0-141-128 kubenswrapper[2571]: W0912 00:16:28.990751    2571 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 12 00:16:28.995070 ip-10-0-141-128 kubenswrapper[2571]: W0912 00:16:28.990754    2571 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 12 00:16:28.995070 ip-10-0-141-128 kubenswrapper[2571]: W0912 00:16:28.990757    2571 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 12 00:16:28.995070 ip-10-0-141-128 kubenswrapper[2571]: W0912 00:16:28.990760    2571 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 12 00:16:28.995557 ip-10-0-141-128 kubenswrapper[2571]: W0912 00:16:28.990762    2571 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 12 00:16:28.995557 ip-10-0-141-128 kubenswrapper[2571]: W0912 00:16:28.990767    2571 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 12 00:16:28.995557 ip-10-0-141-128 kubenswrapper[2571]: W0912 00:16:28.990770    2571 feature_gate.go:328] unrecognized feature gate: Example
Sep 12 00:16:28.995557 ip-10-0-141-128 kubenswrapper[2571]: W0912 00:16:28.990773    2571 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 12 00:16:28.995557 ip-10-0-141-128 kubenswrapper[2571]: W0912 00:16:28.990776    2571 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 12 00:16:28.995557 ip-10-0-141-128 kubenswrapper[2571]: W0912 00:16:28.990779    2571 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 12 00:16:28.995557 ip-10-0-141-128 kubenswrapper[2571]: W0912 00:16:28.990781    2571 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 12 00:16:28.995557 ip-10-0-141-128 kubenswrapper[2571]: W0912 00:16:28.990784    2571 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 12 00:16:28.995557 ip-10-0-141-128 kubenswrapper[2571]: W0912 00:16:28.990786    2571 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 12 00:16:28.995557 ip-10-0-141-128 kubenswrapper[2571]: W0912 00:16:28.990790    2571 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 12 00:16:28.995557 ip-10-0-141-128 kubenswrapper[2571]: W0912 00:16:28.990792    2571 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 12 00:16:28.995557 ip-10-0-141-128 kubenswrapper[2571]: W0912 00:16:28.990795    2571 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 12 00:16:28.995557 ip-10-0-141-128 kubenswrapper[2571]: W0912 00:16:28.990797    2571 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 12 00:16:28.995557 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:16:28.990802    2571 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Sep 12 00:16:28.995926 ip-10-0-141-128 kubenswrapper[2571]: W0912 00:16:28.990911    2571 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 12 00:16:28.995926 ip-10-0-141-128 kubenswrapper[2571]: W0912 00:16:28.990916    2571 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 12 00:16:28.995926 ip-10-0-141-128 kubenswrapper[2571]: W0912 00:16:28.990918    2571 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 12 00:16:28.995926 ip-10-0-141-128 kubenswrapper[2571]: W0912 00:16:28.990921    2571 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 12 00:16:28.995926 ip-10-0-141-128 kubenswrapper[2571]: W0912 00:16:28.990924    2571 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 12 00:16:28.995926 ip-10-0-141-128 kubenswrapper[2571]: W0912 00:16:28.990926    2571 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 12 00:16:28.995926 ip-10-0-141-128 kubenswrapper[2571]: W0912 00:16:28.990929    2571 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 12 00:16:28.995926 ip-10-0-141-128 kubenswrapper[2571]: W0912 00:16:28.990932    2571 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 12 00:16:28.995926 ip-10-0-141-128 kubenswrapper[2571]: W0912 00:16:28.990935    2571 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 12 00:16:28.995926 ip-10-0-141-128 kubenswrapper[2571]: W0912 00:16:28.990937    2571 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 12 00:16:28.995926 ip-10-0-141-128 kubenswrapper[2571]: W0912 00:16:28.990940    2571 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 12 00:16:28.995926 ip-10-0-141-128 kubenswrapper[2571]: W0912 00:16:28.990942    2571 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 12 00:16:28.995926 ip-10-0-141-128 kubenswrapper[2571]: W0912 00:16:28.990945    2571 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 12 00:16:28.995926 ip-10-0-141-128 kubenswrapper[2571]: W0912 00:16:28.990947    2571 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 12 00:16:28.995926 ip-10-0-141-128 kubenswrapper[2571]: W0912 00:16:28.990950    2571 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 12 00:16:28.995926 ip-10-0-141-128 kubenswrapper[2571]: W0912 00:16:28.990952    2571 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 12 00:16:28.995926 ip-10-0-141-128 kubenswrapper[2571]: W0912 00:16:28.990955    2571 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 12 00:16:28.995926 ip-10-0-141-128 kubenswrapper[2571]: W0912 00:16:28.990958    2571 feature_gate.go:328] unrecognized feature gate: Example
Sep 12 00:16:28.995926 ip-10-0-141-128 kubenswrapper[2571]: W0912 00:16:28.990961    2571 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 12 00:16:28.995926 ip-10-0-141-128 kubenswrapper[2571]: W0912 00:16:28.990963    2571 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 12 00:16:28.996417 ip-10-0-141-128 kubenswrapper[2571]: W0912 00:16:28.990966    2571 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 12 00:16:28.996417 ip-10-0-141-128 kubenswrapper[2571]: W0912 00:16:28.990970    2571 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 12 00:16:28.996417 ip-10-0-141-128 kubenswrapper[2571]: W0912 00:16:28.990973    2571 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 12 00:16:28.996417 ip-10-0-141-128 kubenswrapper[2571]: W0912 00:16:28.990976    2571 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 12 00:16:28.996417 ip-10-0-141-128 kubenswrapper[2571]: W0912 00:16:28.990979    2571 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 12 00:16:28.996417 ip-10-0-141-128 kubenswrapper[2571]: W0912 00:16:28.990982    2571 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 12 00:16:28.996417 ip-10-0-141-128 kubenswrapper[2571]: W0912 00:16:28.990984    2571 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 12 00:16:28.996417 ip-10-0-141-128 kubenswrapper[2571]: W0912 00:16:28.990987    2571 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 12 00:16:28.996417 ip-10-0-141-128 kubenswrapper[2571]: W0912 00:16:28.990990    2571 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 12 00:16:28.996417 ip-10-0-141-128 kubenswrapper[2571]: W0912 00:16:28.990992    2571 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 12 00:16:28.996417 ip-10-0-141-128 kubenswrapper[2571]: W0912 00:16:28.990995    2571 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 12 00:16:28.996417 ip-10-0-141-128 kubenswrapper[2571]: W0912 00:16:28.990997    2571 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 12 00:16:28.996417 ip-10-0-141-128 kubenswrapper[2571]: W0912 00:16:28.991000    2571 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 12 00:16:28.996417 ip-10-0-141-128 kubenswrapper[2571]: W0912 00:16:28.991002    2571 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 12 00:16:28.996417 ip-10-0-141-128 kubenswrapper[2571]: W0912 00:16:28.991005    2571 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 12 00:16:28.996417 ip-10-0-141-128 kubenswrapper[2571]: W0912 00:16:28.991007    2571 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 12 00:16:28.996417 ip-10-0-141-128 kubenswrapper[2571]: W0912 00:16:28.991010    2571 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 12 00:16:28.996417 ip-10-0-141-128 kubenswrapper[2571]: W0912 00:16:28.991012    2571 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 12 00:16:28.996417 ip-10-0-141-128 kubenswrapper[2571]: W0912 00:16:28.991015    2571 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 12 00:16:28.996417 ip-10-0-141-128 kubenswrapper[2571]: W0912 00:16:28.991018    2571 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 12 00:16:28.996928 ip-10-0-141-128 kubenswrapper[2571]: W0912 00:16:28.991020    2571 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 12 00:16:28.996928 ip-10-0-141-128 kubenswrapper[2571]: W0912 00:16:28.991023    2571 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 12 00:16:28.996928 ip-10-0-141-128 kubenswrapper[2571]: W0912 00:16:28.991025    2571 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 12 00:16:28.996928 ip-10-0-141-128 kubenswrapper[2571]: W0912 00:16:28.991028    2571 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 12 00:16:28.996928 ip-10-0-141-128 kubenswrapper[2571]: W0912 00:16:28.991030    2571 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 12 00:16:28.996928 ip-10-0-141-128 kubenswrapper[2571]: W0912 00:16:28.991033    2571 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 12 00:16:28.996928 ip-10-0-141-128 kubenswrapper[2571]: W0912 00:16:28.991035    2571 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 12 00:16:28.996928 ip-10-0-141-128 kubenswrapper[2571]: W0912 00:16:28.991038    2571 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 12 00:16:28.996928 ip-10-0-141-128 kubenswrapper[2571]: W0912 00:16:28.991040    2571 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 12 00:16:28.996928 ip-10-0-141-128 kubenswrapper[2571]: W0912 00:16:28.991043    2571 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 12 00:16:28.996928 ip-10-0-141-128 kubenswrapper[2571]: W0912 00:16:28.991045    2571 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 12 00:16:28.996928 ip-10-0-141-128 kubenswrapper[2571]: W0912 00:16:28.991048    2571 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 12 00:16:28.996928 ip-10-0-141-128 kubenswrapper[2571]: W0912 00:16:28.991051    2571 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 12 00:16:28.996928 ip-10-0-141-128 kubenswrapper[2571]: W0912 00:16:28.991054    2571 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 12 00:16:28.996928 ip-10-0-141-128 kubenswrapper[2571]: W0912 00:16:28.991056    2571 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 12 00:16:28.996928 ip-10-0-141-128 kubenswrapper[2571]: W0912 00:16:28.991059    2571 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 12 00:16:28.996928 ip-10-0-141-128 kubenswrapper[2571]: W0912 00:16:28.991062    2571 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 12 00:16:28.996928 ip-10-0-141-128 kubenswrapper[2571]: W0912 00:16:28.991066    2571 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 12 00:16:28.996928 ip-10-0-141-128 kubenswrapper[2571]: W0912 00:16:28.991069    2571 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 12 00:16:28.997436 ip-10-0-141-128 kubenswrapper[2571]: W0912 00:16:28.991072    2571 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 12 00:16:28.997436 ip-10-0-141-128 kubenswrapper[2571]: W0912 00:16:28.991074    2571 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 12 00:16:28.997436 ip-10-0-141-128 kubenswrapper[2571]: W0912 00:16:28.991077    2571 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 12 00:16:28.997436 ip-10-0-141-128 kubenswrapper[2571]: W0912 00:16:28.991080    2571 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 12 00:16:28.997436 ip-10-0-141-128 kubenswrapper[2571]: W0912 00:16:28.991083    2571 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 12 00:16:28.997436 ip-10-0-141-128 kubenswrapper[2571]: W0912 00:16:28.991085    2571 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 12 00:16:28.997436 ip-10-0-141-128 kubenswrapper[2571]: W0912 00:16:28.991088    2571 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 12 00:16:28.997436 ip-10-0-141-128 kubenswrapper[2571]: W0912 00:16:28.991091    2571 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 12 00:16:28.997436 ip-10-0-141-128 kubenswrapper[2571]: W0912 00:16:28.991093    2571 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 12 00:16:28.997436 ip-10-0-141-128 kubenswrapper[2571]: W0912 00:16:28.991096    2571 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 12 00:16:28.997436 ip-10-0-141-128 kubenswrapper[2571]: W0912 00:16:28.991099    2571 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 12 00:16:28.997436 ip-10-0-141-128 kubenswrapper[2571]: W0912 00:16:28.991104    2571 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 12 00:16:28.997436 ip-10-0-141-128 kubenswrapper[2571]: W0912 00:16:28.991106    2571 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 12 00:16:28.997436 ip-10-0-141-128 kubenswrapper[2571]: W0912 00:16:28.991109    2571 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 12 00:16:28.997436 ip-10-0-141-128 kubenswrapper[2571]: W0912 00:16:28.991112    2571 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 12 00:16:28.997436 ip-10-0-141-128 kubenswrapper[2571]: W0912 00:16:28.991114    2571 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 12 00:16:28.997436 ip-10-0-141-128 kubenswrapper[2571]: W0912 00:16:28.991117    2571 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 12 00:16:28.997436 ip-10-0-141-128 kubenswrapper[2571]: W0912 00:16:28.991120    2571 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 12 00:16:28.997436 ip-10-0-141-128 kubenswrapper[2571]: W0912 00:16:28.991123    2571 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 12 00:16:28.997436 ip-10-0-141-128 kubenswrapper[2571]: W0912 00:16:28.991125    2571 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 12 00:16:28.997930 ip-10-0-141-128 kubenswrapper[2571]: W0912 00:16:28.991128    2571 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 12 00:16:28.997930 ip-10-0-141-128 kubenswrapper[2571]: W0912 00:16:28.991130    2571 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 12 00:16:28.997930 ip-10-0-141-128 kubenswrapper[2571]: W0912 00:16:28.991133    2571 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 12 00:16:28.997930 ip-10-0-141-128 kubenswrapper[2571]: W0912 00:16:28.991135    2571 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 12 00:16:28.997930 ip-10-0-141-128 kubenswrapper[2571]: W0912 00:16:28.991138    2571 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 12 00:16:28.997930 ip-10-0-141-128 kubenswrapper[2571]: W0912 00:16:28.991141    2571 feature_gate.go:328] unrecognized feature gate: Example2
Sep 12 00:16:28.997930 ip-10-0-141-128 kubenswrapper[2571]: W0912 00:16:28.991143    2571 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 12 00:16:28.997930 ip-10-0-141-128 kubenswrapper[2571]: W0912 00:16:28.991146    2571 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 12 00:16:28.997930 ip-10-0-141-128 kubenswrapper[2571]: W0912 00:16:28.991149    2571 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 12 00:16:28.997930 ip-10-0-141-128 kubenswrapper[2571]: W0912 00:16:28.991151    2571 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 12 00:16:28.997930 ip-10-0-141-128 kubenswrapper[2571]: W0912 00:16:28.991154    2571 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 12 00:16:28.997930 ip-10-0-141-128 kubenswrapper[2571]: W0912 00:16:28.991156    2571 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 12 00:16:28.997930 ip-10-0-141-128 kubenswrapper[2571]: W0912 00:16:28.991159    2571 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 12 00:16:28.997930 ip-10-0-141-128 kubenswrapper[2571]: W0912 00:16:28.991161    2571 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 12 00:16:28.997930 ip-10-0-141-128 kubenswrapper[2571]: W0912 00:16:28.991164    2571 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 12 00:16:28.997930 ip-10-0-141-128 kubenswrapper[2571]: W0912 00:16:28.991168    2571 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 12 00:16:28.997930 ip-10-0-141-128 kubenswrapper[2571]: W0912 00:16:28.991171    2571 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 12 00:16:28.997930 ip-10-0-141-128 kubenswrapper[2571]: W0912 00:16:28.991173    2571 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 12 00:16:28.997930 ip-10-0-141-128 kubenswrapper[2571]: W0912 00:16:28.991176    2571 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 12 00:16:28.997930 ip-10-0-141-128 kubenswrapper[2571]: W0912 00:16:28.991178    2571 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 12 00:16:28.998413 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:16:28.991183    2571 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Sep 12 00:16:28.998413 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:16:28.991274    2571 server.go:962] "Client rotation is on, will bootstrap in background"
Sep 12 00:16:28.998413 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:16:28.993509    2571 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir"
Sep 12 00:16:28.998413 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:16:28.994311    2571 server.go:1019] "Starting client certificate rotation"
Sep 12 00:16:28.998413 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:16:28.994404    2571 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet"
Sep 12 00:16:28.998413 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:16:28.994440    2571 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet"
Sep 12 00:16:28.998413 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:16:28.997438    2571 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Sep 12 00:16:29.000746 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:16:29.000713    2571 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Sep 12 00:16:29.012793 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:16:29.012770    2571 log.go:25] "Validated CRI v1 runtime API"
Sep 12 00:16:29.017004 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:16:29.016988    2571 log.go:25] "Validated CRI v1 image API"
Sep 12 00:16:29.017840 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:16:29.017826    2571 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd"
Sep 12 00:16:29.021820 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:16:29.021798    2571 fs.go:135] Filesystem UUIDs: map[10622fda-f65d-4bc7-ad0d-871d6a1a584a:/dev/nvme0n1p4 69162038-c2f5-4a6d-89b5-f0d1f3bcfb25:/dev/nvme0n1p3 7B77-95E7:/dev/nvme0n1p2]
Sep 12 00:16:29.021905 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:16:29.021818    2571 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}]
Sep 12 00:16:29.026719 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:16:29.026610    2571 manager.go:216] Machine: {Timestamp:2026-09-12 00:16:29.025352693 +0000 UTC m=+0.296993192 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3203625 MemoryCapacity:33164464128 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec2d86e32e1556150d95b0a889bacb29 SystemUUID:ec2d86e3-2e15-5615-0d95-b0a889bacb29 BootID:8939ddfa-ee9e-4650-808c-496d3d7205e1 Filesystems:[{Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true} {Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16582230016 Type:vfs Inodes:4048396 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6632894464 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6111232 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16582234112 Type:vfs Inodes:1048576 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:be:84:df:59:3b Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:be:84:df:59:3b Speed:0 Mtu:9001} {Name:ovs-system MacAddress:62:0b:74:d8:b8:7a Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:33164464128 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:37486592 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None}
Sep 12 00:16:29.026719 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:16:29.026711    2571 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available.
Sep 12 00:16:29.026879 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:16:29.026804    2571 manager.go:232] Version: {KernelVersion:5.14.0-570.140.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260908-1 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:}
Sep 12 00:16:29.029496 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:16:29.029475    2571 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[]
Sep 12 00:16:29.029690 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:16:29.029500    2571 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-141-128.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2}
Sep 12 00:16:29.031005 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:16:29.030993    2571 topology_manager.go:138] "Creating topology manager with none policy"
Sep 12 00:16:29.031073 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:16:29.031011    2571 container_manager_linux.go:309] "Creating device plugin manager"
Sep 12 00:16:29.031073 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:16:29.031029    2571 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock"
Sep 12 00:16:29.031694 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:16:29.031682    2571 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock"
Sep 12 00:16:29.032706 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:16:29.032690    2571 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Sep 12 00:16:29.032706 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:16:29.032702    2571 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager"
Sep 12 00:16:29.036891 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:16:29.036878    2571 state_mem.go:36] "Initialized new in-memory state store"
Sep 12 00:16:29.037019 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:16:29.037008    2571 server.go:1240] "Using root directory" path="/var/lib/kubelet"
Sep 12 00:16:29.037074 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:16:29.037043    2571 kubelet.go:486] "Attempting to sync node with API server"
Sep 12 00:16:29.037074 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:16:29.037058    2571 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests"
Sep 12 00:16:29.037154 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:16:29.037076    2571 file.go:69] "Watching path" path="/etc/kubernetes/manifests"
Sep 12 00:16:29.037154 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:16:29.037088    2571 kubelet.go:398] "Adding apiserver pod source"
Sep 12 00:16:29.037154 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:16:29.037101    2571 apiserver.go:42] "Waiting for node sync before watching apiserver pods"
Sep 12 00:16:29.037459 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:16:29.037445    2571 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Sep 12 00:16:29.037530 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:16:29.037468    2571 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Sep 12 00:16:29.039477 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:16:29.039459    2571 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.12-5.rhaos4.21.gitd1ec63c.el9" apiVersion="v1"
Sep 12 00:16:29.042061 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:16:29.042042    2571 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled"
Sep 12 00:16:29.042144 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:16:29.042083    2571 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled"
Sep 12 00:16:29.043919 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:16:29.043905    2571 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume"
Sep 12 00:16:29.043989 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:16:29.043925    2571 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir"
Sep 12 00:16:29.043989 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:16:29.043934    2571 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo"
Sep 12 00:16:29.043989 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:16:29.043941    2571 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path"
Sep 12 00:16:29.043989 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:16:29.043954    2571 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs"
Sep 12 00:16:29.043989 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:16:29.043962    2571 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret"
Sep 12 00:16:29.043989 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:16:29.043984    2571 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi"
Sep 12 00:16:29.043989 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:16:29.043992    2571 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api"
Sep 12 00:16:29.044221 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:16:29.044010    2571 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc"
Sep 12 00:16:29.044221 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:16:29.044020    2571 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap"
Sep 12 00:16:29.044221 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:16:29.044034    2571 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected"
Sep 12 00:16:29.044221 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:16:29.044042    2571 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume"
Sep 12 00:16:29.044635 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:16:29.044624    2571 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi"
Sep 12 00:16:29.044696 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:16:29.044637    2571 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image"
Sep 12 00:16:29.048125 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:16:29.048110    2571 server.go:1268] "Started kubelet"
Sep 12 00:16:29.050728 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:16:29.048164    2571 server.go:180] "Starting to listen" address="0.0.0.0" port=10250
Sep 12 00:16:29.051315 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:16:29.050944    2571 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-141-128.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Sep 12 00:16:29.051315 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:16:29.050966    2571 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10
Sep 12 00:16:29.051315 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:16:29.051041    2571 server_v1.go:49] "podresources" method="list" useActivePods=true
Sep 12 00:16:29.051245 ip-10-0-141-128 systemd[1]: Started Kubernetes Kubelet.
Sep 12 00:16:29.053700 ip-10-0-141-128 kubenswrapper[2571]: E0912 00:16:29.052881    2571 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-141-128.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node"
Sep 12 00:16:29.054008 ip-10-0-141-128 kubenswrapper[2571]: E0912 00:16:29.053978    2571 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service"
Sep 12 00:16:29.054500 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:16:29.054303    2571 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock"
Sep 12 00:16:29.055046 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:16:29.054884    2571 server.go:310] "Adding debug handlers to kubelet server"
Sep 12 00:16:29.057861 ip-10-0-141-128 kubenswrapper[2571]: E0912 00:16:29.057173    2571 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-141-128.ec2.internal.18d46aa06237ec26  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-141-128.ec2.internal,UID:ip-10-0-141-128.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-141-128.ec2.internal,},FirstTimestamp:2026-09-12 00:16:29.048073254 +0000 UTC m=+0.319713754,LastTimestamp:2026-09-12 00:16:29.048073254 +0000 UTC m=+0.319713754,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-141-128.ec2.internal,}"
Sep 12 00:16:29.059955 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:16:29.059931    2571 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving"
Sep 12 00:16:29.060570 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:16:29.060551    2571 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer"
Sep 12 00:16:29.060804 ip-10-0-141-128 kubenswrapper[2571]: E0912 00:16:29.060777    2571 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-141-128.ec2.internal\" not found"
Sep 12 00:16:29.061071 ip-10-0-141-128 kubenswrapper[2571]: E0912 00:16:29.061045    2571 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache"
Sep 12 00:16:29.061348 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:16:29.061333    2571 desired_state_of_world_populator.go:146] "Desired state populator starts to run"
Sep 12 00:16:29.061416 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:16:29.061372    2571 factory.go:55] Registering systemd factory
Sep 12 00:16:29.061416 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:16:29.061387    2571 factory.go:223] Registration of the systemd container factory successfully
Sep 12 00:16:29.061521 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:16:29.061342    2571 volume_manager.go:311] "The desired_state_of_world populator starts"
Sep 12 00:16:29.061521 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:16:29.061431    2571 volume_manager.go:313] "Starting Kubelet Volume Manager"
Sep 12 00:16:29.061642 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:16:29.061551    2571 reconstruct.go:97] "Volume reconstruction finished"
Sep 12 00:16:29.061642 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:16:29.061558    2571 reconciler.go:29] "Reconciler: start to sync state"
Sep 12 00:16:29.061642 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:16:29.061618    2571 factory.go:153] Registering CRI-O factory
Sep 12 00:16:29.061642 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:16:29.061633    2571 factory.go:223] Registration of the crio container factory successfully
Sep 12 00:16:29.061833 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:16:29.061676    2571 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory
Sep 12 00:16:29.061833 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:16:29.061691    2571 factory.go:103] Registering Raw factory
Sep 12 00:16:29.061833 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:16:29.061703    2571 manager.go:1196] Started watching for new ooms in manager
Sep 12 00:16:29.062021 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:16:29.062010    2571 manager.go:349] Starting recovery of all containers
Sep 12 00:16:29.071214 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:16:29.071196    2571 manager.go:354] Recovery completed
Sep 12 00:16:29.071434 ip-10-0-141-128 kubenswrapper[2571]: E0912 00:16:29.071406    2571 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-141-128.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms"
Sep 12 00:16:29.071568 ip-10-0-141-128 kubenswrapper[2571]: E0912 00:16:29.071547    2571 reflector.go:205] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver"
Sep 12 00:16:29.075488 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:16:29.075476    2571 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 12 00:16:29.077784 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:16:29.077770    2571 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-128.ec2.internal" event="NodeHasSufficientMemory"
Sep 12 00:16:29.077857 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:16:29.077795    2571 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-128.ec2.internal" event="NodeHasNoDiskPressure"
Sep 12 00:16:29.077857 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:16:29.077807    2571 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-128.ec2.internal" event="NodeHasSufficientPID"
Sep 12 00:16:29.078465 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:16:29.078306    2571 cpu_manager.go:222] "Starting CPU manager" policy="none"
Sep 12 00:16:29.078465 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:16:29.078457    2571 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s"
Sep 12 00:16:29.078578 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:16:29.078477    2571 state_mem.go:36] "Initialized new in-memory state store"
Sep 12 00:16:29.078643 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:16:29.078343    2571 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 12 00:16:29.079408 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:16:29.079395    2571 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-128.ec2.internal" event="NodeHasSufficientMemory"
Sep 12 00:16:29.079476 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:16:29.079416    2571 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-128.ec2.internal" event="NodeHasNoDiskPressure"
Sep 12 00:16:29.079476 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:16:29.079426    2571 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-128.ec2.internal" event="NodeHasSufficientPID"
Sep 12 00:16:29.080469 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:16:29.080455    2571 policy_none.go:49] "None policy: Start"
Sep 12 00:16:29.080544 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:16:29.080474    2571 memory_manager.go:187] "Starting memorymanager" policy="None"
Sep 12 00:16:29.080544 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:16:29.080485    2571 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint"
Sep 12 00:16:29.082270 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:16:29.082258    2571 policy_none.go:47] "Start"
Sep 12 00:16:29.090495 ip-10-0-141-128 kubenswrapper[2571]: E0912 00:16:29.090434    2571 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-141-128.ec2.internal.18d46aa063fd439a  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-141-128.ec2.internal,UID:ip-10-0-141-128.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientMemory,Message:Node ip-10-0-141-128.ec2.internal status is now: NodeHasSufficientMemory,Source:EventSource{Component:kubelet,Host:ip-10-0-141-128.ec2.internal,},FirstTimestamp:2026-09-12 00:16:29.07778345 +0000 UTC m=+0.349423953,LastTimestamp:2026-09-12 00:16:29.07778345 +0000 UTC m=+0.349423953,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-141-128.ec2.internal,}"
Sep 12 00:16:29.093223 ip-10-0-141-128 kubenswrapper[2571]: E0912 00:16:29.093171    2571 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-141-128.ec2.internal.18d46aa063fd81ed  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-141-128.ec2.internal,UID:ip-10-0-141-128.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasNoDiskPressure,Message:Node ip-10-0-141-128.ec2.internal status is now: NodeHasNoDiskPressure,Source:EventSource{Component:kubelet,Host:ip-10-0-141-128.ec2.internal,},FirstTimestamp:2026-09-12 00:16:29.077799405 +0000 UTC m=+0.349439908,LastTimestamp:2026-09-12 00:16:29.077799405 +0000 UTC m=+0.349439908,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-141-128.ec2.internal,}"
Sep 12 00:16:29.101648 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:16:29.101634    2571 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-wl72v"
Sep 12 00:16:29.103247 ip-10-0-141-128 kubenswrapper[2571]: E0912 00:16:29.103192    2571 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-141-128.ec2.internal.18d46aa063fdb78a  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-141-128.ec2.internal,UID:ip-10-0-141-128.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientPID,Message:Node ip-10-0-141-128.ec2.internal status is now: NodeHasSufficientPID,Source:EventSource{Component:kubelet,Host:ip-10-0-141-128.ec2.internal,},FirstTimestamp:2026-09-12 00:16:29.07781313 +0000 UTC m=+0.349453632,LastTimestamp:2026-09-12 00:16:29.07781313 +0000 UTC m=+0.349453632,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-141-128.ec2.internal,}"
Sep 12 00:16:29.111400 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:16:29.111376    2571 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-wl72v"
Sep 12 00:16:29.113387 ip-10-0-141-128 kubenswrapper[2571]: E0912 00:16:29.113316    2571 event.go:359] "Server rejected event (will not retry!)" err="events \"ip-10-0-141-128.ec2.internal.18d46aa063fd439a\" is forbidden: User \"system:anonymous\" cannot patch resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-141-128.ec2.internal.18d46aa063fd439a  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-141-128.ec2.internal,UID:ip-10-0-141-128.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientMemory,Message:Node ip-10-0-141-128.ec2.internal status is now: NodeHasSufficientMemory,Source:EventSource{Component:kubelet,Host:ip-10-0-141-128.ec2.internal,},FirstTimestamp:2026-09-12 00:16:29.07778345 +0000 UTC m=+0.349423953,LastTimestamp:2026-09-12 00:16:29.07940636 +0000 UTC m=+0.351046860,Count:2,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-141-128.ec2.internal,}"
Sep 12 00:16:29.132946 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:16:29.132925    2571 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4"
Sep 12 00:16:29.144315 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:16:29.133983    2571 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6"
Sep 12 00:16:29.144315 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:16:29.134021    2571 status_manager.go:244] "Starting to sync pod status with apiserver"
Sep 12 00:16:29.144315 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:16:29.134038    2571 kubelet.go:2448] "Starting kubelet main sync loop"
Sep 12 00:16:29.144315 ip-10-0-141-128 kubenswrapper[2571]: E0912 00:16:29.134090    2571 kubelet.go:2472] "Skipping pod synchronization" err="[container runtime status check may not have completed yet, PLEG is not healthy: pleg has yet to be successful]"
Sep 12 00:16:29.144315 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:16:29.136340    2571 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 12 00:16:29.161633 ip-10-0-141-128 kubenswrapper[2571]: E0912 00:16:29.161615    2571 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-141-128.ec2.internal\" not found"
Sep 12 00:16:29.234332 ip-10-0-141-128 kubenswrapper[2571]: E0912 00:16:29.234315    2571 kubelet.go:2472] "Skipping pod synchronization" err="container runtime status check may not have completed yet"
Sep 12 00:16:29.261119 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:16:29.261101    2571 manager.go:337] "Starting Device Plugin manager"
Sep 12 00:16:29.261200 ip-10-0-141-128 kubenswrapper[2571]: E0912 00:16:29.261143    2571 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint"
Sep 12 00:16:29.261200 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:16:29.261157    2571 server.go:87] "Starting device plugin registration server"
Sep 12 00:16:29.261483 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:16:29.261471    2571 eviction_manager.go:189] "Eviction manager: starting control loop"
Sep 12 00:16:29.261539 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:16:29.261484    2571 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s"
Sep 12 00:16:29.261598 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:16:29.261558    2571 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry"
Sep 12 00:16:29.261652 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:16:29.261646    2571 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts"
Sep 12 00:16:29.261708 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:16:29.261655    2571 plugin_manager.go:118] "Starting Kubelet Plugin Manager"
Sep 12 00:16:29.262168 ip-10-0-141-128 kubenswrapper[2571]: E0912 00:16:29.262149    2571 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\""
Sep 12 00:16:29.262245 ip-10-0-141-128 kubenswrapper[2571]: E0912 00:16:29.262196    2571 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-141-128.ec2.internal\" not found"
Sep 12 00:16:29.278942 ip-10-0-141-128 kubenswrapper[2571]: E0912 00:16:29.278923    2571 nodelease.go:49] "Failed to get node when trying to set owner ref to the node lease" err="nodes \"ip-10-0-141-128.ec2.internal\" not found" node="ip-10-0-141-128.ec2.internal"
Sep 12 00:16:29.362191 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:16:29.362175    2571 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 12 00:16:29.363089 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:16:29.363072    2571 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-128.ec2.internal" event="NodeHasSufficientMemory"
Sep 12 00:16:29.363170 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:16:29.363104    2571 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-128.ec2.internal" event="NodeHasNoDiskPressure"
Sep 12 00:16:29.363170 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:16:29.363115    2571 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-128.ec2.internal" event="NodeHasSufficientPID"
Sep 12 00:16:29.363170 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:16:29.363137    2571 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-141-128.ec2.internal"
Sep 12 00:16:29.384284 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:16:29.384268    2571 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-141-128.ec2.internal"
Sep 12 00:16:29.384331 ip-10-0-141-128 kubenswrapper[2571]: E0912 00:16:29.384287    2571 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-141-128.ec2.internal\": node \"ip-10-0-141-128.ec2.internal\" not found"
Sep 12 00:16:29.430644 ip-10-0-141-128 kubenswrapper[2571]: E0912 00:16:29.430622    2571 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-141-128.ec2.internal\" not found"
Sep 12 00:16:29.434698 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:16:29.434674    2571 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-128.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-141-128.ec2.internal"]
Sep 12 00:16:29.434797 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:16:29.434738    2571 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 12 00:16:29.436252 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:16:29.436238    2571 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-128.ec2.internal" event="NodeHasSufficientMemory"
Sep 12 00:16:29.436350 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:16:29.436268    2571 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-128.ec2.internal" event="NodeHasNoDiskPressure"
Sep 12 00:16:29.436350 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:16:29.436282    2571 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-128.ec2.internal" event="NodeHasSufficientPID"
Sep 12 00:16:29.438562 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:16:29.438547    2571 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 12 00:16:29.438725 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:16:29.438708    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-128.ec2.internal"
Sep 12 00:16:29.438778 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:16:29.438740    2571 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 12 00:16:29.439309 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:16:29.439295    2571 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-128.ec2.internal" event="NodeHasSufficientMemory"
Sep 12 00:16:29.439396 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:16:29.439313    2571 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-128.ec2.internal" event="NodeHasSufficientMemory"
Sep 12 00:16:29.439396 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:16:29.439322    2571 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-128.ec2.internal" event="NodeHasNoDiskPressure"
Sep 12 00:16:29.439396 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:16:29.439337    2571 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-128.ec2.internal" event="NodeHasSufficientPID"
Sep 12 00:16:29.439396 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:16:29.439348    2571 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-128.ec2.internal" event="NodeHasNoDiskPressure"
Sep 12 00:16:29.439396 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:16:29.439364    2571 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-128.ec2.internal" event="NodeHasSufficientPID"
Sep 12 00:16:29.441555 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:16:29.441540    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-141-128.ec2.internal"
Sep 12 00:16:29.441646 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:16:29.441567    2571 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 12 00:16:29.442252 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:16:29.442236    2571 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-128.ec2.internal" event="NodeHasSufficientMemory"
Sep 12 00:16:29.442321 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:16:29.442259    2571 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-128.ec2.internal" event="NodeHasNoDiskPressure"
Sep 12 00:16:29.442321 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:16:29.442271    2571 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-128.ec2.internal" event="NodeHasSufficientPID"
Sep 12 00:16:29.462132 ip-10-0-141-128 kubenswrapper[2571]: E0912 00:16:29.462114    2571 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-141-128.ec2.internal\" not found" node="ip-10-0-141-128.ec2.internal"
Sep 12 00:16:29.462759 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:16:29.462742    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/a0171d806087a75aaabfcaff6900d5a8-config\") pod \"kube-apiserver-proxy-ip-10-0-141-128.ec2.internal\" (UID: \"a0171d806087a75aaabfcaff6900d5a8\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-141-128.ec2.internal"
Sep 12 00:16:29.462844 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:16:29.462792    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/1e7d3039174fd351468a1c716f20fcb9-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-141-128.ec2.internal\" (UID: \"1e7d3039174fd351468a1c716f20fcb9\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-128.ec2.internal"
Sep 12 00:16:29.462844 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:16:29.462815    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/1e7d3039174fd351468a1c716f20fcb9-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-141-128.ec2.internal\" (UID: \"1e7d3039174fd351468a1c716f20fcb9\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-128.ec2.internal"
Sep 12 00:16:29.466286 ip-10-0-141-128 kubenswrapper[2571]: E0912 00:16:29.466269    2571 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-141-128.ec2.internal\" not found" node="ip-10-0-141-128.ec2.internal"
Sep 12 00:16:29.531152 ip-10-0-141-128 kubenswrapper[2571]: E0912 00:16:29.531115    2571 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-141-128.ec2.internal\" not found"
Sep 12 00:16:29.562947 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:16:29.562932    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/1e7d3039174fd351468a1c716f20fcb9-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-141-128.ec2.internal\" (UID: \"1e7d3039174fd351468a1c716f20fcb9\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-128.ec2.internal"
Sep 12 00:16:29.563011 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:16:29.562954    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/1e7d3039174fd351468a1c716f20fcb9-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-141-128.ec2.internal\" (UID: \"1e7d3039174fd351468a1c716f20fcb9\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-128.ec2.internal"
Sep 12 00:16:29.563011 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:16:29.562972    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/a0171d806087a75aaabfcaff6900d5a8-config\") pod \"kube-apiserver-proxy-ip-10-0-141-128.ec2.internal\" (UID: \"a0171d806087a75aaabfcaff6900d5a8\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-141-128.ec2.internal"
Sep 12 00:16:29.563083 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:16:29.563033    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/a0171d806087a75aaabfcaff6900d5a8-config\") pod \"kube-apiserver-proxy-ip-10-0-141-128.ec2.internal\" (UID: \"a0171d806087a75aaabfcaff6900d5a8\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-141-128.ec2.internal"
Sep 12 00:16:29.563083 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:16:29.563040    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/1e7d3039174fd351468a1c716f20fcb9-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-141-128.ec2.internal\" (UID: \"1e7d3039174fd351468a1c716f20fcb9\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-128.ec2.internal"
Sep 12 00:16:29.563083 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:16:29.563048    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/1e7d3039174fd351468a1c716f20fcb9-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-141-128.ec2.internal\" (UID: \"1e7d3039174fd351468a1c716f20fcb9\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-128.ec2.internal"
Sep 12 00:16:29.632035 ip-10-0-141-128 kubenswrapper[2571]: E0912 00:16:29.632009    2571 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-141-128.ec2.internal\" not found"
Sep 12 00:16:29.732942 ip-10-0-141-128 kubenswrapper[2571]: E0912 00:16:29.732920    2571 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-141-128.ec2.internal\" not found"
Sep 12 00:16:29.765391 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:16:29.765371    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-128.ec2.internal"
Sep 12 00:16:29.770657 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:16:29.770637    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-141-128.ec2.internal"
Sep 12 00:16:29.833723 ip-10-0-141-128 kubenswrapper[2571]: E0912 00:16:29.833668    2571 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-141-128.ec2.internal\" not found"
Sep 12 00:16:29.913710 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:16:29.913692    2571 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 12 00:16:29.961300 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:16:29.961284    2571 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-128.ec2.internal"
Sep 12 00:16:29.985728 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:16:29.985709    2571 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Sep 12 00:16:29.987272 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:16:29.987259    2571 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-141-128.ec2.internal"
Sep 12 00:16:29.994831 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:16:29.994818    2571 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials"
Sep 12 00:16:29.994960 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:16:29.994929    2571 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received"
Sep 12 00:16:29.994998 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:16:29.994980    2571 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received"
Sep 12 00:16:29.994998 ip-10-0-141-128 kubenswrapper[2571]: E0912 00:16:29.994977    2571 kubelet.go:3250] "Failed creating a mirror pod" err="Post \"https://a53fa64530fdb49ff97ff0882fe22dc6-cac38b439bc9e9a9.elb.us-east-1.amazonaws.com:6443/api/v1/namespaces/kube-system/pods\": read tcp 10.0.141.128:40112->52.73.154.167:6443: use of closed network connection" pod="kube-system/kube-apiserver-proxy-ip-10-0-141-128.ec2.internal"
Sep 12 00:16:30.037818 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:16:30.037801    2571 apiserver.go:52] "Watching apiserver"
Sep 12 00:16:30.060805 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:16:30.060785    2571 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving"
Sep 12 00:16:30.062527 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:16:30.062499    2571 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66"
Sep 12 00:16:30.063424 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:16:30.063369    2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-multus/multus-additional-cni-plugins-62v9r","openshift-network-diagnostics/network-check-target-b6792","openshift-network-operator/iptables-alerter-jnfzr","kube-system/konnectivity-agent-r92pg","kube-system/kube-apiserver-proxy-ip-10-0-141-128.ec2.internal","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-nxczt","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-128.ec2.internal","openshift-multus/network-metrics-daemon-bdp25","openshift-ovn-kubernetes/ovnkube-node-j6bgr","kube-system/global-pull-secret-syncer-nxrg5","openshift-cluster-node-tuning-operator/tuned-zjwjs","openshift-dns/node-resolver-dnnhn","openshift-image-registry/node-ca-qm6vt","openshift-multus/multus-57l7n"]
Sep 12 00:16:30.066195 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:16:30.066178    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-62v9r"
Sep 12 00:16:30.068224 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:16:30.068210    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-b6792"
Sep 12 00:16:30.068300 ip-10-0-141-128 kubenswrapper[2571]: E0912 00:16:30.068285    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-b6792" podUID="dd7e1ff1-8f0d-4f40-9a32-2a6cd30cb239"
Sep 12 00:16:30.070260 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:16:30.070245    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-jnfzr"
Sep 12 00:16:30.071842 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:16:30.071825    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\""
Sep 12 00:16:30.071936 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:16:30.071831    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\""
Sep 12 00:16:30.072345 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:16:30.072333    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-r92pg"
Sep 12 00:16:30.074154 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:16:30.074136    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\""
Sep 12 00:16:30.074247 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:16:30.074139    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\""
Sep 12 00:16:30.074247 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:16:30.074220    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-th2hh\""
Sep 12 00:16:30.074432 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:16:30.074414    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-nxczt"
Sep 12 00:16:30.074755 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:16:30.074740    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\""
Sep 12 00:16:30.076495 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:16:30.076484    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-bdp25"
Sep 12 00:16:30.076683 ip-10-0-141-128 kubenswrapper[2571]: E0912 00:16:30.076576    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-bdp25" podUID="6d479776-cc30-4bea-bb3a-8b4c259d59c7"
Sep 12 00:16:30.076834 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:16:30.076820    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\""
Sep 12 00:16:30.076895 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:16:30.076880    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\""
Sep 12 00:16:30.076895 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:16:30.076884    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\""
Sep 12 00:16:30.076988 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:16:30.076905    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-ltmqg\""
Sep 12 00:16:30.078758 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:16:30.078745    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-j6bgr"
Sep 12 00:16:30.079429 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:16:30.079404    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\""
Sep 12 00:16:30.079519 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:16:30.079382    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\""
Sep 12 00:16:30.079519 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:16:30.079456    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\""
Sep 12 00:16:30.079756 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:16:30.079725    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\""
Sep 12 00:16:30.079852 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:16:30.079815    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-djnt9\""
Sep 12 00:16:30.081049 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:16:30.081029    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-nxrg5"
Sep 12 00:16:30.081116 ip-10-0-141-128 kubenswrapper[2571]: E0912 00:16:30.081087    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-nxrg5" podUID="09b83899-89b7-4f4a-b485-c102295f188e"
Sep 12 00:16:30.081732 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:16:30.081715    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-b7wkj\""
Sep 12 00:16:30.081732 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:16:30.081729    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\""
Sep 12 00:16:30.083338 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:16:30.083323    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-zjwjs"
Sep 12 00:16:30.085688 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:16:30.085645    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-dnnhn"
Sep 12 00:16:30.087560 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:16:30.087535    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\""
Sep 12 00:16:30.087667 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:16:30.087580    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\""
Sep 12 00:16:30.087667 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:16:30.087601    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\""
Sep 12 00:16:30.087667 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:16:30.087640    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\""
Sep 12 00:16:30.087823 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:16:30.087672    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\""
Sep 12 00:16:30.087823 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:16:30.087704    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\""
Sep 12 00:16:30.087823 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:16:30.087707    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\""
Sep 12 00:16:30.087823 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:16:30.087821    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-jg8qq\""
Sep 12 00:16:30.087964 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:16:30.087907    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\""
Sep 12 00:16:30.087964 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:16:30.087947    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-q69j2\""
Sep 12 00:16:30.088115 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:16:30.088101    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-qm6vt"
Sep 12 00:16:30.090273 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:16:30.090260    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-57l7n"
Sep 12 00:16:30.102152 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:16:30.101704    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\""
Sep 12 00:16:30.102152 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:16:30.101762    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\""
Sep 12 00:16:30.102152 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:16:30.101945    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\""
Sep 12 00:16:30.102152 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:16:30.102020    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\""
Sep 12 00:16:30.117021 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:16:30.116965    2571 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-09-11 00:11:29 +0000 UTC" deadline="2028-04-27 13:06:58.382670102 +0000 UTC"
Sep 12 00:16:30.117084 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:16:30.117021    2571 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="14244h50m28.265654053s"
Sep 12 00:16:30.135784 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:16:30.135768    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\""
Sep 12 00:16:30.139754 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:16:30.139737    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-8zxtt\""
Sep 12 00:16:30.143850 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:16:30.143833    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-tp4kk\""
Sep 12 00:16:30.144016 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:16:30.144004    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\""
Sep 12 00:16:30.155275 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:16:30.155257    2571 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Sep 12 00:16:30.156759 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:16:30.156742    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-dmmn8\""
Sep 12 00:16:30.162436 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:16:30.162420    2571 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world"
Sep 12 00:16:30.167072 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:16:30.167052    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/0a3e3c8c-cafa-4254-a972-fcf494ae4e28-cnibin\") pod \"multus-additional-cni-plugins-62v9r\" (UID: \"0a3e3c8c-cafa-4254-a972-fcf494ae4e28\") " pod="openshift-multus/multus-additional-cni-plugins-62v9r"
Sep 12 00:16:30.167165 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:16:30.167077    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/0a3e3c8c-cafa-4254-a972-fcf494ae4e28-tuning-conf-dir\") pod \"multus-additional-cni-plugins-62v9r\" (UID: \"0a3e3c8c-cafa-4254-a972-fcf494ae4e28\") " pod="openshift-multus/multus-additional-cni-plugins-62v9r"
Sep 12 00:16:30.167165 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:16:30.167096    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/a9b84142-f0ea-4b17-91b9-cff39fb1d4cf-etc-sysctl-conf\") pod \"tuned-zjwjs\" (UID: \"a9b84142-f0ea-4b17-91b9-cff39fb1d4cf\") " pod="openshift-cluster-node-tuning-operator/tuned-zjwjs"
Sep 12 00:16:30.167165 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:16:30.167111    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/a9b84142-f0ea-4b17-91b9-cff39fb1d4cf-tmp\") pod \"tuned-zjwjs\" (UID: \"a9b84142-f0ea-4b17-91b9-cff39fb1d4cf\") " pod="openshift-cluster-node-tuning-operator/tuned-zjwjs"
Sep 12 00:16:30.167165 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:16:30.167127    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9r2pn\" (UniqueName: \"kubernetes.io/projected/a9b84142-f0ea-4b17-91b9-cff39fb1d4cf-kube-api-access-9r2pn\") pod \"tuned-zjwjs\" (UID: \"a9b84142-f0ea-4b17-91b9-cff39fb1d4cf\") " pod="openshift-cluster-node-tuning-operator/tuned-zjwjs"
Sep 12 00:16:30.167165 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:16:30.167145    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/40698f9e-0e26-4d86-ac05-ffcd30e172d4-systemd-units\") pod \"ovnkube-node-j6bgr\" (UID: \"40698f9e-0e26-4d86-ac05-ffcd30e172d4\") " pod="openshift-ovn-kubernetes/ovnkube-node-j6bgr"
Sep 12 00:16:30.167412 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:16:30.167169    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/40698f9e-0e26-4d86-ac05-ffcd30e172d4-ovnkube-config\") pod \"ovnkube-node-j6bgr\" (UID: \"40698f9e-0e26-4d86-ac05-ffcd30e172d4\") " pod="openshift-ovn-kubernetes/ovnkube-node-j6bgr"
Sep 12 00:16:30.167412 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:16:30.167213    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/76f830f1-1f3e-4dd4-9deb-023f6ad1e2e4-host-run-multus-certs\") pod \"multus-57l7n\" (UID: \"76f830f1-1f3e-4dd4-9deb-023f6ad1e2e4\") " pod="openshift-multus/multus-57l7n"
Sep 12 00:16:30.167412 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:16:30.167244    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/76f830f1-1f3e-4dd4-9deb-023f6ad1e2e4-etc-kubernetes\") pod \"multus-57l7n\" (UID: \"76f830f1-1f3e-4dd4-9deb-023f6ad1e2e4\") " pod="openshift-multus/multus-57l7n"
Sep 12 00:16:30.167412 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:16:30.167262    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/ef505089-5ca0-44f6-a9c1-c7c951c30faf-kubelet-dir\") pod \"aws-ebs-csi-driver-node-nxczt\" (UID: \"ef505089-5ca0-44f6-a9c1-c7c951c30faf\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-nxczt"
Sep 12 00:16:30.167412 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:16:30.167306    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/76f830f1-1f3e-4dd4-9deb-023f6ad1e2e4-host-var-lib-cni-bin\") pod \"multus-57l7n\" (UID: \"76f830f1-1f3e-4dd4-9deb-023f6ad1e2e4\") " pod="openshift-multus/multus-57l7n"
Sep 12 00:16:30.167412 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:16:30.167332    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/71d215ed-e9d2-4a8b-bc73-f0ee2e0d83c5-agent-certs\") pod \"konnectivity-agent-r92pg\" (UID: \"71d215ed-e9d2-4a8b-bc73-f0ee2e0d83c5\") " pod="kube-system/konnectivity-agent-r92pg"
Sep 12 00:16:30.167412 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:16:30.167353    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/40698f9e-0e26-4d86-ac05-ffcd30e172d4-var-lib-openvswitch\") pod \"ovnkube-node-j6bgr\" (UID: \"40698f9e-0e26-4d86-ac05-ffcd30e172d4\") " pod="openshift-ovn-kubernetes/ovnkube-node-j6bgr"
Sep 12 00:16:30.167412 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:16:30.167375    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nqsrv\" (UniqueName: \"kubernetes.io/projected/ef505089-5ca0-44f6-a9c1-c7c951c30faf-kube-api-access-nqsrv\") pod \"aws-ebs-csi-driver-node-nxczt\" (UID: \"ef505089-5ca0-44f6-a9c1-c7c951c30faf\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-nxczt"
Sep 12 00:16:30.167412 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:16:30.167400    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gz7sz\" (UniqueName: \"kubernetes.io/projected/6d479776-cc30-4bea-bb3a-8b4c259d59c7-kube-api-access-gz7sz\") pod \"network-metrics-daemon-bdp25\" (UID: \"6d479776-cc30-4bea-bb3a-8b4c259d59c7\") " pod="openshift-multus/network-metrics-daemon-bdp25"
Sep 12 00:16:30.167888 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:16:30.167423    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/40698f9e-0e26-4d86-ac05-ffcd30e172d4-node-log\") pod \"ovnkube-node-j6bgr\" (UID: \"40698f9e-0e26-4d86-ac05-ffcd30e172d4\") " pod="openshift-ovn-kubernetes/ovnkube-node-j6bgr"
Sep 12 00:16:30.167888 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:16:30.167451    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/0a3e3c8c-cafa-4254-a972-fcf494ae4e28-system-cni-dir\") pod \"multus-additional-cni-plugins-62v9r\" (UID: \"0a3e3c8c-cafa-4254-a972-fcf494ae4e28\") " pod="openshift-multus/multus-additional-cni-plugins-62v9r"
Sep 12 00:16:30.167888 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:16:30.167467    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/76f830f1-1f3e-4dd4-9deb-023f6ad1e2e4-system-cni-dir\") pod \"multus-57l7n\" (UID: \"76f830f1-1f3e-4dd4-9deb-023f6ad1e2e4\") " pod="openshift-multus/multus-57l7n"
Sep 12 00:16:30.167888 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:16:30.167488    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/76f830f1-1f3e-4dd4-9deb-023f6ad1e2e4-multus-socket-dir-parent\") pod \"multus-57l7n\" (UID: \"76f830f1-1f3e-4dd4-9deb-023f6ad1e2e4\") " pod="openshift-multus/multus-57l7n"
Sep 12 00:16:30.167888 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:16:30.167503    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/76f830f1-1f3e-4dd4-9deb-023f6ad1e2e4-host-run-netns\") pod \"multus-57l7n\" (UID: \"76f830f1-1f3e-4dd4-9deb-023f6ad1e2e4\") " pod="openshift-multus/multus-57l7n"
Sep 12 00:16:30.167888 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:16:30.167518    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/40698f9e-0e26-4d86-ac05-ffcd30e172d4-log-socket\") pod \"ovnkube-node-j6bgr\" (UID: \"40698f9e-0e26-4d86-ac05-ffcd30e172d4\") " pod="openshift-ovn-kubernetes/ovnkube-node-j6bgr"
Sep 12 00:16:30.167888 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:16:30.167542    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/40698f9e-0e26-4d86-ac05-ffcd30e172d4-ovn-node-metrics-cert\") pod \"ovnkube-node-j6bgr\" (UID: \"40698f9e-0e26-4d86-ac05-ffcd30e172d4\") " pod="openshift-ovn-kubernetes/ovnkube-node-j6bgr"
Sep 12 00:16:30.167888 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:16:30.167562    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/a9b84142-f0ea-4b17-91b9-cff39fb1d4cf-etc-systemd\") pod \"tuned-zjwjs\" (UID: \"a9b84142-f0ea-4b17-91b9-cff39fb1d4cf\") " pod="openshift-cluster-node-tuning-operator/tuned-zjwjs"
Sep 12 00:16:30.167888 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:16:30.167578    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/a9b84142-f0ea-4b17-91b9-cff39fb1d4cf-run\") pod \"tuned-zjwjs\" (UID: \"a9b84142-f0ea-4b17-91b9-cff39fb1d4cf\") " pod="openshift-cluster-node-tuning-operator/tuned-zjwjs"
Sep 12 00:16:30.167888 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:16:30.167595    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/19b621f9-8fcc-4a57-b13d-862aad3d75ce-tmp-dir\") pod \"node-resolver-dnnhn\" (UID: \"19b621f9-8fcc-4a57-b13d-862aad3d75ce\") " pod="openshift-dns/node-resolver-dnnhn"
Sep 12 00:16:30.167888 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:16:30.167628    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/76f830f1-1f3e-4dd4-9deb-023f6ad1e2e4-host-var-lib-cni-multus\") pod \"multus-57l7n\" (UID: \"76f830f1-1f3e-4dd4-9deb-023f6ad1e2e4\") " pod="openshift-multus/multus-57l7n"
Sep 12 00:16:30.167888 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:16:30.167647    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7h4vm\" (UniqueName: \"kubernetes.io/projected/dd7e1ff1-8f0d-4f40-9a32-2a6cd30cb239-kube-api-access-7h4vm\") pod \"network-check-target-b6792\" (UID: \"dd7e1ff1-8f0d-4f40-9a32-2a6cd30cb239\") " pod="openshift-network-diagnostics/network-check-target-b6792"
Sep 12 00:16:30.167888 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:16:30.167661    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xhc74\" (UniqueName: \"kubernetes.io/projected/76f830f1-1f3e-4dd4-9deb-023f6ad1e2e4-kube-api-access-xhc74\") pod \"multus-57l7n\" (UID: \"76f830f1-1f3e-4dd4-9deb-023f6ad1e2e4\") " pod="openshift-multus/multus-57l7n"
Sep 12 00:16:30.167888 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:16:30.167677    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/6d479776-cc30-4bea-bb3a-8b4c259d59c7-metrics-certs\") pod \"network-metrics-daemon-bdp25\" (UID: \"6d479776-cc30-4bea-bb3a-8b4c259d59c7\") " pod="openshift-multus/network-metrics-daemon-bdp25"
Sep 12 00:16:30.167888 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:16:30.167701    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/40698f9e-0e26-4d86-ac05-ffcd30e172d4-host-run-ovn-kubernetes\") pod \"ovnkube-node-j6bgr\" (UID: \"40698f9e-0e26-4d86-ac05-ffcd30e172d4\") " pod="openshift-ovn-kubernetes/ovnkube-node-j6bgr"
Sep 12 00:16:30.167888 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:16:30.167726    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/a9b84142-f0ea-4b17-91b9-cff39fb1d4cf-sys\") pod \"tuned-zjwjs\" (UID: \"a9b84142-f0ea-4b17-91b9-cff39fb1d4cf\") " pod="openshift-cluster-node-tuning-operator/tuned-zjwjs"
Sep 12 00:16:30.167888 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:16:30.167750    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/76f830f1-1f3e-4dd4-9deb-023f6ad1e2e4-multus-cni-dir\") pod \"multus-57l7n\" (UID: \"76f830f1-1f3e-4dd4-9deb-023f6ad1e2e4\") " pod="openshift-multus/multus-57l7n"
Sep 12 00:16:30.168770 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:16:30.167769    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/76f830f1-1f3e-4dd4-9deb-023f6ad1e2e4-cnibin\") pod \"multus-57l7n\" (UID: \"76f830f1-1f3e-4dd4-9deb-023f6ad1e2e4\") " pod="openshift-multus/multus-57l7n"
Sep 12 00:16:30.168770 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:16:30.167804    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/a8819f1a-9bcc-495c-91fc-adfc4e4d87fd-host-slash\") pod \"iptables-alerter-jnfzr\" (UID: \"a8819f1a-9bcc-495c-91fc-adfc4e4d87fd\") " pod="openshift-network-operator/iptables-alerter-jnfzr"
Sep 12 00:16:30.168770 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:16:30.167826    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/40698f9e-0e26-4d86-ac05-ffcd30e172d4-host-cni-netd\") pod \"ovnkube-node-j6bgr\" (UID: \"40698f9e-0e26-4d86-ac05-ffcd30e172d4\") " pod="openshift-ovn-kubernetes/ovnkube-node-j6bgr"
Sep 12 00:16:30.168770 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:16:30.167843    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/40698f9e-0e26-4d86-ac05-ffcd30e172d4-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-j6bgr\" (UID: \"40698f9e-0e26-4d86-ac05-ffcd30e172d4\") " pod="openshift-ovn-kubernetes/ovnkube-node-j6bgr"
Sep 12 00:16:30.168770 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:16:30.167866    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/ef505089-5ca0-44f6-a9c1-c7c951c30faf-socket-dir\") pod \"aws-ebs-csi-driver-node-nxczt\" (UID: \"ef505089-5ca0-44f6-a9c1-c7c951c30faf\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-nxczt"
Sep 12 00:16:30.168770 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:16:30.167887    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/ef505089-5ca0-44f6-a9c1-c7c951c30faf-etc-selinux\") pod \"aws-ebs-csi-driver-node-nxczt\" (UID: \"ef505089-5ca0-44f6-a9c1-c7c951c30faf\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-nxczt"
Sep 12 00:16:30.168770 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:16:30.167903    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/76f830f1-1f3e-4dd4-9deb-023f6ad1e2e4-multus-daemon-config\") pod \"multus-57l7n\" (UID: \"76f830f1-1f3e-4dd4-9deb-023f6ad1e2e4\") " pod="openshift-multus/multus-57l7n"
Sep 12 00:16:30.168770 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:16:30.167918    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bljm6\" (UniqueName: \"kubernetes.io/projected/a8819f1a-9bcc-495c-91fc-adfc4e4d87fd-kube-api-access-bljm6\") pod \"iptables-alerter-jnfzr\" (UID: \"a8819f1a-9bcc-495c-91fc-adfc4e4d87fd\") " pod="openshift-network-operator/iptables-alerter-jnfzr"
Sep 12 00:16:30.168770 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:16:30.167934    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/71d215ed-e9d2-4a8b-bc73-f0ee2e0d83c5-konnectivity-ca\") pod \"konnectivity-agent-r92pg\" (UID: \"71d215ed-e9d2-4a8b-bc73-f0ee2e0d83c5\") " pod="kube-system/konnectivity-agent-r92pg"
Sep 12 00:16:30.168770 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:16:30.167954    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/7a6f512a-ad51-4ec7-a9a5-8b6a34cb5bb8-host\") pod \"node-ca-qm6vt\" (UID: \"7a6f512a-ad51-4ec7-a9a5-8b6a34cb5bb8\") " pod="openshift-image-registry/node-ca-qm6vt"
Sep 12 00:16:30.168770 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:16:30.167989    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/40698f9e-0e26-4d86-ac05-ffcd30e172d4-run-openvswitch\") pod \"ovnkube-node-j6bgr\" (UID: \"40698f9e-0e26-4d86-ac05-ffcd30e172d4\") " pod="openshift-ovn-kubernetes/ovnkube-node-j6bgr"
Sep 12 00:16:30.168770 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:16:30.168010    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/76f830f1-1f3e-4dd4-9deb-023f6ad1e2e4-hostroot\") pod \"multus-57l7n\" (UID: \"76f830f1-1f3e-4dd4-9deb-023f6ad1e2e4\") " pod="openshift-multus/multus-57l7n"
Sep 12 00:16:30.168770 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:16:30.168026    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/0a3e3c8c-cafa-4254-a972-fcf494ae4e28-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-62v9r\" (UID: \"0a3e3c8c-cafa-4254-a972-fcf494ae4e28\") " pod="openshift-multus/multus-additional-cni-plugins-62v9r"
Sep 12 00:16:30.168770 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:16:30.168048    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/ef505089-5ca0-44f6-a9c1-c7c951c30faf-sys-fs\") pod \"aws-ebs-csi-driver-node-nxczt\" (UID: \"ef505089-5ca0-44f6-a9c1-c7c951c30faf\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-nxczt"
Sep 12 00:16:30.168770 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:16:30.168075    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/a9b84142-f0ea-4b17-91b9-cff39fb1d4cf-etc-sysctl-d\") pod \"tuned-zjwjs\" (UID: \"a9b84142-f0ea-4b17-91b9-cff39fb1d4cf\") " pod="openshift-cluster-node-tuning-operator/tuned-zjwjs"
Sep 12 00:16:30.168770 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:16:30.168091    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-m6rhb\" (UniqueName: \"kubernetes.io/projected/19b621f9-8fcc-4a57-b13d-862aad3d75ce-kube-api-access-m6rhb\") pod \"node-resolver-dnnhn\" (UID: \"19b621f9-8fcc-4a57-b13d-862aad3d75ce\") " pod="openshift-dns/node-resolver-dnnhn"
Sep 12 00:16:30.169274 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:16:30.168133    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/7a6f512a-ad51-4ec7-a9a5-8b6a34cb5bb8-serviceca\") pod \"node-ca-qm6vt\" (UID: \"7a6f512a-ad51-4ec7-a9a5-8b6a34cb5bb8\") " pod="openshift-image-registry/node-ca-qm6vt"
Sep 12 00:16:30.169274 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:16:30.168161    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/76f830f1-1f3e-4dd4-9deb-023f6ad1e2e4-host-run-k8s-cni-cncf-io\") pod \"multus-57l7n\" (UID: \"76f830f1-1f3e-4dd4-9deb-023f6ad1e2e4\") " pod="openshift-multus/multus-57l7n"
Sep 12 00:16:30.169274 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:16:30.168180    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/40698f9e-0e26-4d86-ac05-ffcd30e172d4-host-slash\") pod \"ovnkube-node-j6bgr\" (UID: \"40698f9e-0e26-4d86-ac05-ffcd30e172d4\") " pod="openshift-ovn-kubernetes/ovnkube-node-j6bgr"
Sep 12 00:16:30.169274 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:16:30.168205    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/0a3e3c8c-cafa-4254-a972-fcf494ae4e28-cni-binary-copy\") pod \"multus-additional-cni-plugins-62v9r\" (UID: \"0a3e3c8c-cafa-4254-a972-fcf494ae4e28\") " pod="openshift-multus/multus-additional-cni-plugins-62v9r"
Sep 12 00:16:30.169274 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:16:30.168238    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-d79n8\" (UniqueName: \"kubernetes.io/projected/0a3e3c8c-cafa-4254-a972-fcf494ae4e28-kube-api-access-d79n8\") pod \"multus-additional-cni-plugins-62v9r\" (UID: \"0a3e3c8c-cafa-4254-a972-fcf494ae4e28\") " pod="openshift-multus/multus-additional-cni-plugins-62v9r"
Sep 12 00:16:30.169274 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:16:30.168262    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/ef505089-5ca0-44f6-a9c1-c7c951c30faf-device-dir\") pod \"aws-ebs-csi-driver-node-nxczt\" (UID: \"ef505089-5ca0-44f6-a9c1-c7c951c30faf\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-nxczt"
Sep 12 00:16:30.169274 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:16:30.168278    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/a9b84142-f0ea-4b17-91b9-cff39fb1d4cf-lib-modules\") pod \"tuned-zjwjs\" (UID: \"a9b84142-f0ea-4b17-91b9-cff39fb1d4cf\") " pod="openshift-cluster-node-tuning-operator/tuned-zjwjs"
Sep 12 00:16:30.169274 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:16:30.168307    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/0a3e3c8c-cafa-4254-a972-fcf494ae4e28-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-62v9r\" (UID: \"0a3e3c8c-cafa-4254-a972-fcf494ae4e28\") " pod="openshift-multus/multus-additional-cni-plugins-62v9r"
Sep 12 00:16:30.169274 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:16:30.168324    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/40698f9e-0e26-4d86-ac05-ffcd30e172d4-host-kubelet\") pod \"ovnkube-node-j6bgr\" (UID: \"40698f9e-0e26-4d86-ac05-ffcd30e172d4\") " pod="openshift-ovn-kubernetes/ovnkube-node-j6bgr"
Sep 12 00:16:30.169274 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:16:30.168358    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/40698f9e-0e26-4d86-ac05-ffcd30e172d4-host-cni-bin\") pod \"ovnkube-node-j6bgr\" (UID: \"40698f9e-0e26-4d86-ac05-ffcd30e172d4\") " pod="openshift-ovn-kubernetes/ovnkube-node-j6bgr"
Sep 12 00:16:30.169274 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:16:30.168383    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/40698f9e-0e26-4d86-ac05-ffcd30e172d4-env-overrides\") pod \"ovnkube-node-j6bgr\" (UID: \"40698f9e-0e26-4d86-ac05-ffcd30e172d4\") " pod="openshift-ovn-kubernetes/ovnkube-node-j6bgr"
Sep 12 00:16:30.169274 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:16:30.168399    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/40698f9e-0e26-4d86-ac05-ffcd30e172d4-ovnkube-script-lib\") pod \"ovnkube-node-j6bgr\" (UID: \"40698f9e-0e26-4d86-ac05-ffcd30e172d4\") " pod="openshift-ovn-kubernetes/ovnkube-node-j6bgr"
Sep 12 00:16:30.169274 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:16:30.168426    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/09b83899-89b7-4f4a-b485-c102295f188e-original-pull-secret\") pod \"global-pull-secret-syncer-nxrg5\" (UID: \"09b83899-89b7-4f4a-b485-c102295f188e\") " pod="kube-system/global-pull-secret-syncer-nxrg5"
Sep 12 00:16:30.169274 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:16:30.168457    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/a9b84142-f0ea-4b17-91b9-cff39fb1d4cf-etc-modprobe-d\") pod \"tuned-zjwjs\" (UID: \"a9b84142-f0ea-4b17-91b9-cff39fb1d4cf\") " pod="openshift-cluster-node-tuning-operator/tuned-zjwjs"
Sep 12 00:16:30.169274 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:16:30.168477    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/a9b84142-f0ea-4b17-91b9-cff39fb1d4cf-var-lib-kubelet\") pod \"tuned-zjwjs\" (UID: \"a9b84142-f0ea-4b17-91b9-cff39fb1d4cf\") " pod="openshift-cluster-node-tuning-operator/tuned-zjwjs"
Sep 12 00:16:30.169274 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:16:30.168500    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/a9b84142-f0ea-4b17-91b9-cff39fb1d4cf-etc-tuned\") pod \"tuned-zjwjs\" (UID: \"a9b84142-f0ea-4b17-91b9-cff39fb1d4cf\") " pod="openshift-cluster-node-tuning-operator/tuned-zjwjs"
Sep 12 00:16:30.169769 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:16:30.168523    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mn2n7\" (UniqueName: \"kubernetes.io/projected/7a6f512a-ad51-4ec7-a9a5-8b6a34cb5bb8-kube-api-access-mn2n7\") pod \"node-ca-qm6vt\" (UID: \"7a6f512a-ad51-4ec7-a9a5-8b6a34cb5bb8\") " pod="openshift-image-registry/node-ca-qm6vt"
Sep 12 00:16:30.169769 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:16:30.168538    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/40698f9e-0e26-4d86-ac05-ffcd30e172d4-run-ovn\") pod \"ovnkube-node-j6bgr\" (UID: \"40698f9e-0e26-4d86-ac05-ffcd30e172d4\") " pod="openshift-ovn-kubernetes/ovnkube-node-j6bgr"
Sep 12 00:16:30.169769 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:16:30.168574    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/ef505089-5ca0-44f6-a9c1-c7c951c30faf-registration-dir\") pod \"aws-ebs-csi-driver-node-nxczt\" (UID: \"ef505089-5ca0-44f6-a9c1-c7c951c30faf\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-nxczt"
Sep 12 00:16:30.169769 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:16:30.168633    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/19b621f9-8fcc-4a57-b13d-862aad3d75ce-hosts-file\") pod \"node-resolver-dnnhn\" (UID: \"19b621f9-8fcc-4a57-b13d-862aad3d75ce\") " pod="openshift-dns/node-resolver-dnnhn"
Sep 12 00:16:30.169769 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:16:30.168663    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/76f830f1-1f3e-4dd4-9deb-023f6ad1e2e4-cni-binary-copy\") pod \"multus-57l7n\" (UID: \"76f830f1-1f3e-4dd4-9deb-023f6ad1e2e4\") " pod="openshift-multus/multus-57l7n"
Sep 12 00:16:30.169769 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:16:30.168695    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/a8819f1a-9bcc-495c-91fc-adfc4e4d87fd-iptables-alerter-script\") pod \"iptables-alerter-jnfzr\" (UID: \"a8819f1a-9bcc-495c-91fc-adfc4e4d87fd\") " pod="openshift-network-operator/iptables-alerter-jnfzr"
Sep 12 00:16:30.169769 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:16:30.168720    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/40698f9e-0e26-4d86-ac05-ffcd30e172d4-host-run-netns\") pod \"ovnkube-node-j6bgr\" (UID: \"40698f9e-0e26-4d86-ac05-ffcd30e172d4\") " pod="openshift-ovn-kubernetes/ovnkube-node-j6bgr"
Sep 12 00:16:30.169769 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:16:30.168747    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/09b83899-89b7-4f4a-b485-c102295f188e-kubelet-config\") pod \"global-pull-secret-syncer-nxrg5\" (UID: \"09b83899-89b7-4f4a-b485-c102295f188e\") " pod="kube-system/global-pull-secret-syncer-nxrg5"
Sep 12 00:16:30.169769 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:16:30.168784    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-96pjd\" (UniqueName: \"kubernetes.io/projected/40698f9e-0e26-4d86-ac05-ffcd30e172d4-kube-api-access-96pjd\") pod \"ovnkube-node-j6bgr\" (UID: \"40698f9e-0e26-4d86-ac05-ffcd30e172d4\") " pod="openshift-ovn-kubernetes/ovnkube-node-j6bgr"
Sep 12 00:16:30.169769 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:16:30.168821    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/09b83899-89b7-4f4a-b485-c102295f188e-dbus\") pod \"global-pull-secret-syncer-nxrg5\" (UID: \"09b83899-89b7-4f4a-b485-c102295f188e\") " pod="kube-system/global-pull-secret-syncer-nxrg5"
Sep 12 00:16:30.169769 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:16:30.168855    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/a9b84142-f0ea-4b17-91b9-cff39fb1d4cf-etc-sysconfig\") pod \"tuned-zjwjs\" (UID: \"a9b84142-f0ea-4b17-91b9-cff39fb1d4cf\") " pod="openshift-cluster-node-tuning-operator/tuned-zjwjs"
Sep 12 00:16:30.169769 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:16:30.168884    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/a9b84142-f0ea-4b17-91b9-cff39fb1d4cf-etc-kubernetes\") pod \"tuned-zjwjs\" (UID: \"a9b84142-f0ea-4b17-91b9-cff39fb1d4cf\") " pod="openshift-cluster-node-tuning-operator/tuned-zjwjs"
Sep 12 00:16:30.169769 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:16:30.168906    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/a9b84142-f0ea-4b17-91b9-cff39fb1d4cf-host\") pod \"tuned-zjwjs\" (UID: \"a9b84142-f0ea-4b17-91b9-cff39fb1d4cf\") " pod="openshift-cluster-node-tuning-operator/tuned-zjwjs"
Sep 12 00:16:30.169769 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:16:30.168929    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/76f830f1-1f3e-4dd4-9deb-023f6ad1e2e4-host-var-lib-kubelet\") pod \"multus-57l7n\" (UID: \"76f830f1-1f3e-4dd4-9deb-023f6ad1e2e4\") " pod="openshift-multus/multus-57l7n"
Sep 12 00:16:30.169769 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:16:30.168946    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/76f830f1-1f3e-4dd4-9deb-023f6ad1e2e4-multus-conf-dir\") pod \"multus-57l7n\" (UID: \"76f830f1-1f3e-4dd4-9deb-023f6ad1e2e4\") " pod="openshift-multus/multus-57l7n"
Sep 12 00:16:30.169769 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:16:30.168966    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/40698f9e-0e26-4d86-ac05-ffcd30e172d4-run-systemd\") pod \"ovnkube-node-j6bgr\" (UID: \"40698f9e-0e26-4d86-ac05-ffcd30e172d4\") " pod="openshift-ovn-kubernetes/ovnkube-node-j6bgr"
Sep 12 00:16:30.170265 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:16:30.168990    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/40698f9e-0e26-4d86-ac05-ffcd30e172d4-etc-openvswitch\") pod \"ovnkube-node-j6bgr\" (UID: \"40698f9e-0e26-4d86-ac05-ffcd30e172d4\") " pod="openshift-ovn-kubernetes/ovnkube-node-j6bgr"
Sep 12 00:16:30.210891 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:16:30.210846    2571 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 12 00:16:30.240218 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:16:30.240195    2571 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-lqrr2"
Sep 12 00:16:30.269516 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:16:30.269493    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/ef505089-5ca0-44f6-a9c1-c7c951c30faf-socket-dir\") pod \"aws-ebs-csi-driver-node-nxczt\" (UID: \"ef505089-5ca0-44f6-a9c1-c7c951c30faf\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-nxczt"
Sep 12 00:16:30.269516 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:16:30.269510    2571 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-lqrr2"
Sep 12 00:16:30.269740 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:16:30.269522    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/ef505089-5ca0-44f6-a9c1-c7c951c30faf-etc-selinux\") pod \"aws-ebs-csi-driver-node-nxczt\" (UID: \"ef505089-5ca0-44f6-a9c1-c7c951c30faf\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-nxczt"
Sep 12 00:16:30.269740 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:16:30.269541    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/76f830f1-1f3e-4dd4-9deb-023f6ad1e2e4-multus-daemon-config\") pod \"multus-57l7n\" (UID: \"76f830f1-1f3e-4dd4-9deb-023f6ad1e2e4\") " pod="openshift-multus/multus-57l7n"
Sep 12 00:16:30.269740 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:16:30.269566    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bljm6\" (UniqueName: \"kubernetes.io/projected/a8819f1a-9bcc-495c-91fc-adfc4e4d87fd-kube-api-access-bljm6\") pod \"iptables-alerter-jnfzr\" (UID: \"a8819f1a-9bcc-495c-91fc-adfc4e4d87fd\") " pod="openshift-network-operator/iptables-alerter-jnfzr"
Sep 12 00:16:30.269740 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:16:30.269594    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/71d215ed-e9d2-4a8b-bc73-f0ee2e0d83c5-konnectivity-ca\") pod \"konnectivity-agent-r92pg\" (UID: \"71d215ed-e9d2-4a8b-bc73-f0ee2e0d83c5\") " pod="kube-system/konnectivity-agent-r92pg"
Sep 12 00:16:30.269740 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:16:30.269627    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/7a6f512a-ad51-4ec7-a9a5-8b6a34cb5bb8-host\") pod \"node-ca-qm6vt\" (UID: \"7a6f512a-ad51-4ec7-a9a5-8b6a34cb5bb8\") " pod="openshift-image-registry/node-ca-qm6vt"
Sep 12 00:16:30.269740 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:16:30.269633    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/ef505089-5ca0-44f6-a9c1-c7c951c30faf-etc-selinux\") pod \"aws-ebs-csi-driver-node-nxczt\" (UID: \"ef505089-5ca0-44f6-a9c1-c7c951c30faf\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-nxczt"
Sep 12 00:16:30.269740 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:16:30.269652    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/40698f9e-0e26-4d86-ac05-ffcd30e172d4-run-openvswitch\") pod \"ovnkube-node-j6bgr\" (UID: \"40698f9e-0e26-4d86-ac05-ffcd30e172d4\") " pod="openshift-ovn-kubernetes/ovnkube-node-j6bgr"
Sep 12 00:16:30.269740 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:16:30.269684    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/ef505089-5ca0-44f6-a9c1-c7c951c30faf-socket-dir\") pod \"aws-ebs-csi-driver-node-nxczt\" (UID: \"ef505089-5ca0-44f6-a9c1-c7c951c30faf\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-nxczt"
Sep 12 00:16:30.269740 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:16:30.269705    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/76f830f1-1f3e-4dd4-9deb-023f6ad1e2e4-hostroot\") pod \"multus-57l7n\" (UID: \"76f830f1-1f3e-4dd4-9deb-023f6ad1e2e4\") " pod="openshift-multus/multus-57l7n"
Sep 12 00:16:30.269740 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:16:30.269734    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/0a3e3c8c-cafa-4254-a972-fcf494ae4e28-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-62v9r\" (UID: \"0a3e3c8c-cafa-4254-a972-fcf494ae4e28\") " pod="openshift-multus/multus-additional-cni-plugins-62v9r"
Sep 12 00:16:30.270270 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:16:30.269761    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/ef505089-5ca0-44f6-a9c1-c7c951c30faf-sys-fs\") pod \"aws-ebs-csi-driver-node-nxczt\" (UID: \"ef505089-5ca0-44f6-a9c1-c7c951c30faf\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-nxczt"
Sep 12 00:16:30.270270 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:16:30.269773    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/7a6f512a-ad51-4ec7-a9a5-8b6a34cb5bb8-host\") pod \"node-ca-qm6vt\" (UID: \"7a6f512a-ad51-4ec7-a9a5-8b6a34cb5bb8\") " pod="openshift-image-registry/node-ca-qm6vt"
Sep 12 00:16:30.270270 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:16:30.269785    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/a9b84142-f0ea-4b17-91b9-cff39fb1d4cf-etc-sysctl-d\") pod \"tuned-zjwjs\" (UID: \"a9b84142-f0ea-4b17-91b9-cff39fb1d4cf\") " pod="openshift-cluster-node-tuning-operator/tuned-zjwjs"
Sep 12 00:16:30.270270 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:16:30.269794    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/40698f9e-0e26-4d86-ac05-ffcd30e172d4-run-openvswitch\") pod \"ovnkube-node-j6bgr\" (UID: \"40698f9e-0e26-4d86-ac05-ffcd30e172d4\") " pod="openshift-ovn-kubernetes/ovnkube-node-j6bgr"
Sep 12 00:16:30.270270 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:16:30.269850    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-m6rhb\" (UniqueName: \"kubernetes.io/projected/19b621f9-8fcc-4a57-b13d-862aad3d75ce-kube-api-access-m6rhb\") pod \"node-resolver-dnnhn\" (UID: \"19b621f9-8fcc-4a57-b13d-862aad3d75ce\") " pod="openshift-dns/node-resolver-dnnhn"
Sep 12 00:16:30.270270 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:16:30.269860    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/76f830f1-1f3e-4dd4-9deb-023f6ad1e2e4-hostroot\") pod \"multus-57l7n\" (UID: \"76f830f1-1f3e-4dd4-9deb-023f6ad1e2e4\") " pod="openshift-multus/multus-57l7n"
Sep 12 00:16:30.270270 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:16:30.269876    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/7a6f512a-ad51-4ec7-a9a5-8b6a34cb5bb8-serviceca\") pod \"node-ca-qm6vt\" (UID: \"7a6f512a-ad51-4ec7-a9a5-8b6a34cb5bb8\") " pod="openshift-image-registry/node-ca-qm6vt"
Sep 12 00:16:30.270270 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:16:30.269905    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/76f830f1-1f3e-4dd4-9deb-023f6ad1e2e4-host-run-k8s-cni-cncf-io\") pod \"multus-57l7n\" (UID: \"76f830f1-1f3e-4dd4-9deb-023f6ad1e2e4\") " pod="openshift-multus/multus-57l7n"
Sep 12 00:16:30.270270 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:16:30.269948    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/40698f9e-0e26-4d86-ac05-ffcd30e172d4-host-slash\") pod \"ovnkube-node-j6bgr\" (UID: \"40698f9e-0e26-4d86-ac05-ffcd30e172d4\") " pod="openshift-ovn-kubernetes/ovnkube-node-j6bgr"
Sep 12 00:16:30.270270 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:16:30.269971    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/ef505089-5ca0-44f6-a9c1-c7c951c30faf-sys-fs\") pod \"aws-ebs-csi-driver-node-nxczt\" (UID: \"ef505089-5ca0-44f6-a9c1-c7c951c30faf\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-nxczt"
Sep 12 00:16:30.270270 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:16:30.269975    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/0a3e3c8c-cafa-4254-a972-fcf494ae4e28-cni-binary-copy\") pod \"multus-additional-cni-plugins-62v9r\" (UID: \"0a3e3c8c-cafa-4254-a972-fcf494ae4e28\") " pod="openshift-multus/multus-additional-cni-plugins-62v9r"
Sep 12 00:16:30.270270 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:16:30.269980    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/a9b84142-f0ea-4b17-91b9-cff39fb1d4cf-etc-sysctl-d\") pod \"tuned-zjwjs\" (UID: \"a9b84142-f0ea-4b17-91b9-cff39fb1d4cf\") " pod="openshift-cluster-node-tuning-operator/tuned-zjwjs"
Sep 12 00:16:30.270270 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:16:30.270004    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-d79n8\" (UniqueName: \"kubernetes.io/projected/0a3e3c8c-cafa-4254-a972-fcf494ae4e28-kube-api-access-d79n8\") pod \"multus-additional-cni-plugins-62v9r\" (UID: \"0a3e3c8c-cafa-4254-a972-fcf494ae4e28\") " pod="openshift-multus/multus-additional-cni-plugins-62v9r"
Sep 12 00:16:30.270270 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:16:30.270031    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/40698f9e-0e26-4d86-ac05-ffcd30e172d4-host-slash\") pod \"ovnkube-node-j6bgr\" (UID: \"40698f9e-0e26-4d86-ac05-ffcd30e172d4\") " pod="openshift-ovn-kubernetes/ovnkube-node-j6bgr"
Sep 12 00:16:30.270270 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:16:30.270031    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/ef505089-5ca0-44f6-a9c1-c7c951c30faf-device-dir\") pod \"aws-ebs-csi-driver-node-nxczt\" (UID: \"ef505089-5ca0-44f6-a9c1-c7c951c30faf\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-nxczt"
Sep 12 00:16:30.270270 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:16:30.270073    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/a9b84142-f0ea-4b17-91b9-cff39fb1d4cf-lib-modules\") pod \"tuned-zjwjs\" (UID: \"a9b84142-f0ea-4b17-91b9-cff39fb1d4cf\") " pod="openshift-cluster-node-tuning-operator/tuned-zjwjs"
Sep 12 00:16:30.270270 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:16:30.270101    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/0a3e3c8c-cafa-4254-a972-fcf494ae4e28-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-62v9r\" (UID: \"0a3e3c8c-cafa-4254-a972-fcf494ae4e28\") " pod="openshift-multus/multus-additional-cni-plugins-62v9r"
Sep 12 00:16:30.271161 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:16:30.270129    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/40698f9e-0e26-4d86-ac05-ffcd30e172d4-host-kubelet\") pod \"ovnkube-node-j6bgr\" (UID: \"40698f9e-0e26-4d86-ac05-ffcd30e172d4\") " pod="openshift-ovn-kubernetes/ovnkube-node-j6bgr"
Sep 12 00:16:30.271161 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:16:30.270143    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/ef505089-5ca0-44f6-a9c1-c7c951c30faf-device-dir\") pod \"aws-ebs-csi-driver-node-nxczt\" (UID: \"ef505089-5ca0-44f6-a9c1-c7c951c30faf\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-nxczt"
Sep 12 00:16:30.271161 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:16:30.270154    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/40698f9e-0e26-4d86-ac05-ffcd30e172d4-host-cni-bin\") pod \"ovnkube-node-j6bgr\" (UID: \"40698f9e-0e26-4d86-ac05-ffcd30e172d4\") " pod="openshift-ovn-kubernetes/ovnkube-node-j6bgr"
Sep 12 00:16:30.271161 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:16:30.270167    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/76f830f1-1f3e-4dd4-9deb-023f6ad1e2e4-multus-daemon-config\") pod \"multus-57l7n\" (UID: \"76f830f1-1f3e-4dd4-9deb-023f6ad1e2e4\") " pod="openshift-multus/multus-57l7n"
Sep 12 00:16:30.271161 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:16:30.270179    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/40698f9e-0e26-4d86-ac05-ffcd30e172d4-env-overrides\") pod \"ovnkube-node-j6bgr\" (UID: \"40698f9e-0e26-4d86-ac05-ffcd30e172d4\") " pod="openshift-ovn-kubernetes/ovnkube-node-j6bgr"
Sep 12 00:16:30.271161 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:16:30.270204    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/40698f9e-0e26-4d86-ac05-ffcd30e172d4-ovnkube-script-lib\") pod \"ovnkube-node-j6bgr\" (UID: \"40698f9e-0e26-4d86-ac05-ffcd30e172d4\") " pod="openshift-ovn-kubernetes/ovnkube-node-j6bgr"
Sep 12 00:16:30.271161 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:16:30.270230    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/09b83899-89b7-4f4a-b485-c102295f188e-original-pull-secret\") pod \"global-pull-secret-syncer-nxrg5\" (UID: \"09b83899-89b7-4f4a-b485-c102295f188e\") " pod="kube-system/global-pull-secret-syncer-nxrg5"
Sep 12 00:16:30.271161 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:16:30.270237    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/71d215ed-e9d2-4a8b-bc73-f0ee2e0d83c5-konnectivity-ca\") pod \"konnectivity-agent-r92pg\" (UID: \"71d215ed-e9d2-4a8b-bc73-f0ee2e0d83c5\") " pod="kube-system/konnectivity-agent-r92pg"
Sep 12 00:16:30.271161 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:16:30.270252    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/a9b84142-f0ea-4b17-91b9-cff39fb1d4cf-lib-modules\") pod \"tuned-zjwjs\" (UID: \"a9b84142-f0ea-4b17-91b9-cff39fb1d4cf\") " pod="openshift-cluster-node-tuning-operator/tuned-zjwjs"
Sep 12 00:16:30.271161 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:16:30.270261    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/a9b84142-f0ea-4b17-91b9-cff39fb1d4cf-etc-modprobe-d\") pod \"tuned-zjwjs\" (UID: \"a9b84142-f0ea-4b17-91b9-cff39fb1d4cf\") " pod="openshift-cluster-node-tuning-operator/tuned-zjwjs"
Sep 12 00:16:30.271161 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:16:30.270299    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/a9b84142-f0ea-4b17-91b9-cff39fb1d4cf-var-lib-kubelet\") pod \"tuned-zjwjs\" (UID: \"a9b84142-f0ea-4b17-91b9-cff39fb1d4cf\") " pod="openshift-cluster-node-tuning-operator/tuned-zjwjs"
Sep 12 00:16:30.271161 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:16:30.270302    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/40698f9e-0e26-4d86-ac05-ffcd30e172d4-host-cni-bin\") pod \"ovnkube-node-j6bgr\" (UID: \"40698f9e-0e26-4d86-ac05-ffcd30e172d4\") " pod="openshift-ovn-kubernetes/ovnkube-node-j6bgr"
Sep 12 00:16:30.271161 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:16:30.270328    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/a9b84142-f0ea-4b17-91b9-cff39fb1d4cf-etc-tuned\") pod \"tuned-zjwjs\" (UID: \"a9b84142-f0ea-4b17-91b9-cff39fb1d4cf\") " pod="openshift-cluster-node-tuning-operator/tuned-zjwjs"
Sep 12 00:16:30.271161 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:16:30.270353    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mn2n7\" (UniqueName: \"kubernetes.io/projected/7a6f512a-ad51-4ec7-a9a5-8b6a34cb5bb8-kube-api-access-mn2n7\") pod \"node-ca-qm6vt\" (UID: \"7a6f512a-ad51-4ec7-a9a5-8b6a34cb5bb8\") " pod="openshift-image-registry/node-ca-qm6vt"
Sep 12 00:16:30.271161 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:16:30.270381    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/40698f9e-0e26-4d86-ac05-ffcd30e172d4-run-ovn\") pod \"ovnkube-node-j6bgr\" (UID: \"40698f9e-0e26-4d86-ac05-ffcd30e172d4\") " pod="openshift-ovn-kubernetes/ovnkube-node-j6bgr"
Sep 12 00:16:30.271161 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:16:30.270406    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/ef505089-5ca0-44f6-a9c1-c7c951c30faf-registration-dir\") pod \"aws-ebs-csi-driver-node-nxczt\" (UID: \"ef505089-5ca0-44f6-a9c1-c7c951c30faf\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-nxczt"
Sep 12 00:16:30.271161 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:16:30.270410    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/0a3e3c8c-cafa-4254-a972-fcf494ae4e28-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-62v9r\" (UID: \"0a3e3c8c-cafa-4254-a972-fcf494ae4e28\") " pod="openshift-multus/multus-additional-cni-plugins-62v9r"
Sep 12 00:16:30.272097 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:16:30.270432    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/19b621f9-8fcc-4a57-b13d-862aad3d75ce-hosts-file\") pod \"node-resolver-dnnhn\" (UID: \"19b621f9-8fcc-4a57-b13d-862aad3d75ce\") " pod="openshift-dns/node-resolver-dnnhn"
Sep 12 00:16:30.272097 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:16:30.270483    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/40698f9e-0e26-4d86-ac05-ffcd30e172d4-host-kubelet\") pod \"ovnkube-node-j6bgr\" (UID: \"40698f9e-0e26-4d86-ac05-ffcd30e172d4\") " pod="openshift-ovn-kubernetes/ovnkube-node-j6bgr"
Sep 12 00:16:30.272097 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:16:30.270488    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/19b621f9-8fcc-4a57-b13d-862aad3d75ce-hosts-file\") pod \"node-resolver-dnnhn\" (UID: \"19b621f9-8fcc-4a57-b13d-862aad3d75ce\") " pod="openshift-dns/node-resolver-dnnhn"
Sep 12 00:16:30.272097 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:16:30.270518    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/76f830f1-1f3e-4dd4-9deb-023f6ad1e2e4-cni-binary-copy\") pod \"multus-57l7n\" (UID: \"76f830f1-1f3e-4dd4-9deb-023f6ad1e2e4\") " pod="openshift-multus/multus-57l7n"
Sep 12 00:16:30.272097 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:16:30.270547    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/a8819f1a-9bcc-495c-91fc-adfc4e4d87fd-iptables-alerter-script\") pod \"iptables-alerter-jnfzr\" (UID: \"a8819f1a-9bcc-495c-91fc-adfc4e4d87fd\") " pod="openshift-network-operator/iptables-alerter-jnfzr"
Sep 12 00:16:30.272097 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:16:30.270577    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/40698f9e-0e26-4d86-ac05-ffcd30e172d4-host-run-netns\") pod \"ovnkube-node-j6bgr\" (UID: \"40698f9e-0e26-4d86-ac05-ffcd30e172d4\") " pod="openshift-ovn-kubernetes/ovnkube-node-j6bgr"
Sep 12 00:16:30.272097 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:16:30.270605    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/09b83899-89b7-4f4a-b485-c102295f188e-kubelet-config\") pod \"global-pull-secret-syncer-nxrg5\" (UID: \"09b83899-89b7-4f4a-b485-c102295f188e\") " pod="kube-system/global-pull-secret-syncer-nxrg5"
Sep 12 00:16:30.272097 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:16:30.270638    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-96pjd\" (UniqueName: \"kubernetes.io/projected/40698f9e-0e26-4d86-ac05-ffcd30e172d4-kube-api-access-96pjd\") pod \"ovnkube-node-j6bgr\" (UID: \"40698f9e-0e26-4d86-ac05-ffcd30e172d4\") " pod="openshift-ovn-kubernetes/ovnkube-node-j6bgr"
Sep 12 00:16:30.272097 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:16:30.270663    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/09b83899-89b7-4f4a-b485-c102295f188e-dbus\") pod \"global-pull-secret-syncer-nxrg5\" (UID: \"09b83899-89b7-4f4a-b485-c102295f188e\") " pod="kube-system/global-pull-secret-syncer-nxrg5"
Sep 12 00:16:30.272097 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:16:30.270704    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/a9b84142-f0ea-4b17-91b9-cff39fb1d4cf-etc-sysconfig\") pod \"tuned-zjwjs\" (UID: \"a9b84142-f0ea-4b17-91b9-cff39fb1d4cf\") " pod="openshift-cluster-node-tuning-operator/tuned-zjwjs"
Sep 12 00:16:30.272097 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:16:30.270730    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/a9b84142-f0ea-4b17-91b9-cff39fb1d4cf-etc-kubernetes\") pod \"tuned-zjwjs\" (UID: \"a9b84142-f0ea-4b17-91b9-cff39fb1d4cf\") " pod="openshift-cluster-node-tuning-operator/tuned-zjwjs"
Sep 12 00:16:30.272097 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:16:30.270753    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/7a6f512a-ad51-4ec7-a9a5-8b6a34cb5bb8-serviceca\") pod \"node-ca-qm6vt\" (UID: \"7a6f512a-ad51-4ec7-a9a5-8b6a34cb5bb8\") " pod="openshift-image-registry/node-ca-qm6vt"
Sep 12 00:16:30.272097 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:16:30.270779    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/40698f9e-0e26-4d86-ac05-ffcd30e172d4-env-overrides\") pod \"ovnkube-node-j6bgr\" (UID: \"40698f9e-0e26-4d86-ac05-ffcd30e172d4\") " pod="openshift-ovn-kubernetes/ovnkube-node-j6bgr"
Sep 12 00:16:30.272097 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:16:30.270794    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/a9b84142-f0ea-4b17-91b9-cff39fb1d4cf-host\") pod \"tuned-zjwjs\" (UID: \"a9b84142-f0ea-4b17-91b9-cff39fb1d4cf\") " pod="openshift-cluster-node-tuning-operator/tuned-zjwjs"
Sep 12 00:16:30.272097 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:16:30.270837    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/76f830f1-1f3e-4dd4-9deb-023f6ad1e2e4-host-run-k8s-cni-cncf-io\") pod \"multus-57l7n\" (UID: \"76f830f1-1f3e-4dd4-9deb-023f6ad1e2e4\") " pod="openshift-multus/multus-57l7n"
Sep 12 00:16:30.272097 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:16:30.270852    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/a9b84142-f0ea-4b17-91b9-cff39fb1d4cf-etc-modprobe-d\") pod \"tuned-zjwjs\" (UID: \"a9b84142-f0ea-4b17-91b9-cff39fb1d4cf\") " pod="openshift-cluster-node-tuning-operator/tuned-zjwjs"
Sep 12 00:16:30.272097 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:16:30.270823    2571 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory"
Sep 12 00:16:30.272097 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:16:30.270865    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/76f830f1-1f3e-4dd4-9deb-023f6ad1e2e4-host-var-lib-kubelet\") pod \"multus-57l7n\" (UID: \"76f830f1-1f3e-4dd4-9deb-023f6ad1e2e4\") " pod="openshift-multus/multus-57l7n"
Sep 12 00:16:30.273026 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:16:30.270893    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/76f830f1-1f3e-4dd4-9deb-023f6ad1e2e4-multus-conf-dir\") pod \"multus-57l7n\" (UID: \"76f830f1-1f3e-4dd4-9deb-023f6ad1e2e4\") " pod="openshift-multus/multus-57l7n"
Sep 12 00:16:30.273026 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:16:30.270900    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/40698f9e-0e26-4d86-ac05-ffcd30e172d4-run-ovn\") pod \"ovnkube-node-j6bgr\" (UID: \"40698f9e-0e26-4d86-ac05-ffcd30e172d4\") " pod="openshift-ovn-kubernetes/ovnkube-node-j6bgr"
Sep 12 00:16:30.273026 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:16:30.270929    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/40698f9e-0e26-4d86-ac05-ffcd30e172d4-run-systemd\") pod \"ovnkube-node-j6bgr\" (UID: \"40698f9e-0e26-4d86-ac05-ffcd30e172d4\") " pod="openshift-ovn-kubernetes/ovnkube-node-j6bgr"
Sep 12 00:16:30.273026 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:16:30.270959    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/40698f9e-0e26-4d86-ac05-ffcd30e172d4-etc-openvswitch\") pod \"ovnkube-node-j6bgr\" (UID: \"40698f9e-0e26-4d86-ac05-ffcd30e172d4\") " pod="openshift-ovn-kubernetes/ovnkube-node-j6bgr"
Sep 12 00:16:30.273026 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:16:30.270961    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/ef505089-5ca0-44f6-a9c1-c7c951c30faf-registration-dir\") pod \"aws-ebs-csi-driver-node-nxczt\" (UID: \"ef505089-5ca0-44f6-a9c1-c7c951c30faf\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-nxczt"
Sep 12 00:16:30.273026 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:16:30.270986    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/0a3e3c8c-cafa-4254-a972-fcf494ae4e28-cnibin\") pod \"multus-additional-cni-plugins-62v9r\" (UID: \"0a3e3c8c-cafa-4254-a972-fcf494ae4e28\") " pod="openshift-multus/multus-additional-cni-plugins-62v9r"
Sep 12 00:16:30.273026 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:16:30.271019    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/0a3e3c8c-cafa-4254-a972-fcf494ae4e28-tuning-conf-dir\") pod \"multus-additional-cni-plugins-62v9r\" (UID: \"0a3e3c8c-cafa-4254-a972-fcf494ae4e28\") " pod="openshift-multus/multus-additional-cni-plugins-62v9r"
Sep 12 00:16:30.273026 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:16:30.271048    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/a9b84142-f0ea-4b17-91b9-cff39fb1d4cf-etc-sysctl-conf\") pod \"tuned-zjwjs\" (UID: \"a9b84142-f0ea-4b17-91b9-cff39fb1d4cf\") " pod="openshift-cluster-node-tuning-operator/tuned-zjwjs"
Sep 12 00:16:30.273026 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:16:30.271072    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/a9b84142-f0ea-4b17-91b9-cff39fb1d4cf-tmp\") pod \"tuned-zjwjs\" (UID: \"a9b84142-f0ea-4b17-91b9-cff39fb1d4cf\") " pod="openshift-cluster-node-tuning-operator/tuned-zjwjs"
Sep 12 00:16:30.273026 ip-10-0-141-128 kubenswrapper[2571]: E0912 00:16:30.271077    2571 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 12 00:16:30.273026 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:16:30.271098    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9r2pn\" (UniqueName: \"kubernetes.io/projected/a9b84142-f0ea-4b17-91b9-cff39fb1d4cf-kube-api-access-9r2pn\") pod \"tuned-zjwjs\" (UID: \"a9b84142-f0ea-4b17-91b9-cff39fb1d4cf\") " pod="openshift-cluster-node-tuning-operator/tuned-zjwjs"
Sep 12 00:16:30.273026 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:16:30.271123    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/40698f9e-0e26-4d86-ac05-ffcd30e172d4-systemd-units\") pod \"ovnkube-node-j6bgr\" (UID: \"40698f9e-0e26-4d86-ac05-ffcd30e172d4\") " pod="openshift-ovn-kubernetes/ovnkube-node-j6bgr"
Sep 12 00:16:30.273026 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:16:30.271154    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/40698f9e-0e26-4d86-ac05-ffcd30e172d4-ovnkube-config\") pod \"ovnkube-node-j6bgr\" (UID: \"40698f9e-0e26-4d86-ac05-ffcd30e172d4\") " pod="openshift-ovn-kubernetes/ovnkube-node-j6bgr"
Sep 12 00:16:30.273026 ip-10-0-141-128 kubenswrapper[2571]: E0912 00:16:30.271171    2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/09b83899-89b7-4f4a-b485-c102295f188e-original-pull-secret podName:09b83899-89b7-4f4a-b485-c102295f188e nodeName:}" failed. No retries permitted until 2026-09-12 00:16:30.771148131 +0000 UTC m=+2.042788644 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/09b83899-89b7-4f4a-b485-c102295f188e-original-pull-secret") pod "global-pull-secret-syncer-nxrg5" (UID: "09b83899-89b7-4f4a-b485-c102295f188e") : object "kube-system"/"original-pull-secret" not registered
Sep 12 00:16:30.273026 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:16:30.271208    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/76f830f1-1f3e-4dd4-9deb-023f6ad1e2e4-host-run-multus-certs\") pod \"multus-57l7n\" (UID: \"76f830f1-1f3e-4dd4-9deb-023f6ad1e2e4\") " pod="openshift-multus/multus-57l7n"
Sep 12 00:16:30.273026 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:16:30.271238    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/76f830f1-1f3e-4dd4-9deb-023f6ad1e2e4-etc-kubernetes\") pod \"multus-57l7n\" (UID: \"76f830f1-1f3e-4dd4-9deb-023f6ad1e2e4\") " pod="openshift-multus/multus-57l7n"
Sep 12 00:16:30.273026 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:16:30.271267    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/ef505089-5ca0-44f6-a9c1-c7c951c30faf-kubelet-dir\") pod \"aws-ebs-csi-driver-node-nxczt\" (UID: \"ef505089-5ca0-44f6-a9c1-c7c951c30faf\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-nxczt"
Sep 12 00:16:30.273818 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:16:30.271298    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/76f830f1-1f3e-4dd4-9deb-023f6ad1e2e4-host-var-lib-cni-bin\") pod \"multus-57l7n\" (UID: \"76f830f1-1f3e-4dd4-9deb-023f6ad1e2e4\") " pod="openshift-multus/multus-57l7n"
Sep 12 00:16:30.273818 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:16:30.271323    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/71d215ed-e9d2-4a8b-bc73-f0ee2e0d83c5-agent-certs\") pod \"konnectivity-agent-r92pg\" (UID: \"71d215ed-e9d2-4a8b-bc73-f0ee2e0d83c5\") " pod="kube-system/konnectivity-agent-r92pg"
Sep 12 00:16:30.273818 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:16:30.271342    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/40698f9e-0e26-4d86-ac05-ffcd30e172d4-ovnkube-script-lib\") pod \"ovnkube-node-j6bgr\" (UID: \"40698f9e-0e26-4d86-ac05-ffcd30e172d4\") " pod="openshift-ovn-kubernetes/ovnkube-node-j6bgr"
Sep 12 00:16:30.273818 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:16:30.271351    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/40698f9e-0e26-4d86-ac05-ffcd30e172d4-var-lib-openvswitch\") pod \"ovnkube-node-j6bgr\" (UID: \"40698f9e-0e26-4d86-ac05-ffcd30e172d4\") " pod="openshift-ovn-kubernetes/ovnkube-node-j6bgr"
Sep 12 00:16:30.273818 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:16:30.271402    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/a9b84142-f0ea-4b17-91b9-cff39fb1d4cf-var-lib-kubelet\") pod \"tuned-zjwjs\" (UID: \"a9b84142-f0ea-4b17-91b9-cff39fb1d4cf\") " pod="openshift-cluster-node-tuning-operator/tuned-zjwjs"
Sep 12 00:16:30.273818 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:16:30.271406    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/40698f9e-0e26-4d86-ac05-ffcd30e172d4-var-lib-openvswitch\") pod \"ovnkube-node-j6bgr\" (UID: \"40698f9e-0e26-4d86-ac05-ffcd30e172d4\") " pod="openshift-ovn-kubernetes/ovnkube-node-j6bgr"
Sep 12 00:16:30.273818 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:16:30.271419    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/ef505089-5ca0-44f6-a9c1-c7c951c30faf-kubelet-dir\") pod \"aws-ebs-csi-driver-node-nxczt\" (UID: \"ef505089-5ca0-44f6-a9c1-c7c951c30faf\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-nxczt"
Sep 12 00:16:30.273818 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:16:30.271434    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nqsrv\" (UniqueName: \"kubernetes.io/projected/ef505089-5ca0-44f6-a9c1-c7c951c30faf-kube-api-access-nqsrv\") pod \"aws-ebs-csi-driver-node-nxczt\" (UID: \"ef505089-5ca0-44f6-a9c1-c7c951c30faf\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-nxczt"
Sep 12 00:16:30.273818 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:16:30.271453    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/76f830f1-1f3e-4dd4-9deb-023f6ad1e2e4-host-var-lib-cni-bin\") pod \"multus-57l7n\" (UID: \"76f830f1-1f3e-4dd4-9deb-023f6ad1e2e4\") " pod="openshift-multus/multus-57l7n"
Sep 12 00:16:30.273818 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:16:30.271460    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/76f830f1-1f3e-4dd4-9deb-023f6ad1e2e4-host-run-multus-certs\") pod \"multus-57l7n\" (UID: \"76f830f1-1f3e-4dd4-9deb-023f6ad1e2e4\") " pod="openshift-multus/multus-57l7n"
Sep 12 00:16:30.273818 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:16:30.271468    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gz7sz\" (UniqueName: \"kubernetes.io/projected/6d479776-cc30-4bea-bb3a-8b4c259d59c7-kube-api-access-gz7sz\") pod \"network-metrics-daemon-bdp25\" (UID: \"6d479776-cc30-4bea-bb3a-8b4c259d59c7\") " pod="openshift-multus/network-metrics-daemon-bdp25"
Sep 12 00:16:30.273818 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:16:30.271486    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/76f830f1-1f3e-4dd4-9deb-023f6ad1e2e4-etc-kubernetes\") pod \"multus-57l7n\" (UID: \"76f830f1-1f3e-4dd4-9deb-023f6ad1e2e4\") " pod="openshift-multus/multus-57l7n"
Sep 12 00:16:30.273818 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:16:30.271498    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/40698f9e-0e26-4d86-ac05-ffcd30e172d4-node-log\") pod \"ovnkube-node-j6bgr\" (UID: \"40698f9e-0e26-4d86-ac05-ffcd30e172d4\") " pod="openshift-ovn-kubernetes/ovnkube-node-j6bgr"
Sep 12 00:16:30.273818 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:16:30.271525    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/76f830f1-1f3e-4dd4-9deb-023f6ad1e2e4-multus-conf-dir\") pod \"multus-57l7n\" (UID: \"76f830f1-1f3e-4dd4-9deb-023f6ad1e2e4\") " pod="openshift-multus/multus-57l7n"
Sep 12 00:16:30.273818 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:16:30.271523    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/0a3e3c8c-cafa-4254-a972-fcf494ae4e28-system-cni-dir\") pod \"multus-additional-cni-plugins-62v9r\" (UID: \"0a3e3c8c-cafa-4254-a972-fcf494ae4e28\") " pod="openshift-multus/multus-additional-cni-plugins-62v9r"
Sep 12 00:16:30.273818 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:16:30.271560    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/76f830f1-1f3e-4dd4-9deb-023f6ad1e2e4-system-cni-dir\") pod \"multus-57l7n\" (UID: \"76f830f1-1f3e-4dd4-9deb-023f6ad1e2e4\") " pod="openshift-multus/multus-57l7n"
Sep 12 00:16:30.273818 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:16:30.271559    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/0a3e3c8c-cafa-4254-a972-fcf494ae4e28-system-cni-dir\") pod \"multus-additional-cni-plugins-62v9r\" (UID: \"0a3e3c8c-cafa-4254-a972-fcf494ae4e28\") " pod="openshift-multus/multus-additional-cni-plugins-62v9r"
Sep 12 00:16:30.274403 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:16:30.270549    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/0a3e3c8c-cafa-4254-a972-fcf494ae4e28-cni-binary-copy\") pod \"multus-additional-cni-plugins-62v9r\" (UID: \"0a3e3c8c-cafa-4254-a972-fcf494ae4e28\") " pod="openshift-multus/multus-additional-cni-plugins-62v9r"
Sep 12 00:16:30.274403 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:16:30.271733    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/a9b84142-f0ea-4b17-91b9-cff39fb1d4cf-etc-sysconfig\") pod \"tuned-zjwjs\" (UID: \"a9b84142-f0ea-4b17-91b9-cff39fb1d4cf\") " pod="openshift-cluster-node-tuning-operator/tuned-zjwjs"
Sep 12 00:16:30.274403 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:16:30.271779    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/40698f9e-0e26-4d86-ac05-ffcd30e172d4-host-run-netns\") pod \"ovnkube-node-j6bgr\" (UID: \"40698f9e-0e26-4d86-ac05-ffcd30e172d4\") " pod="openshift-ovn-kubernetes/ovnkube-node-j6bgr"
Sep 12 00:16:30.274403 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:16:30.271782    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/a8819f1a-9bcc-495c-91fc-adfc4e4d87fd-iptables-alerter-script\") pod \"iptables-alerter-jnfzr\" (UID: \"a8819f1a-9bcc-495c-91fc-adfc4e4d87fd\") " pod="openshift-network-operator/iptables-alerter-jnfzr"
Sep 12 00:16:30.274403 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:16:30.271826    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/09b83899-89b7-4f4a-b485-c102295f188e-kubelet-config\") pod \"global-pull-secret-syncer-nxrg5\" (UID: \"09b83899-89b7-4f4a-b485-c102295f188e\") " pod="kube-system/global-pull-secret-syncer-nxrg5"
Sep 12 00:16:30.274403 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:16:30.271852    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/40698f9e-0e26-4d86-ac05-ffcd30e172d4-run-systemd\") pod \"ovnkube-node-j6bgr\" (UID: \"40698f9e-0e26-4d86-ac05-ffcd30e172d4\") " pod="openshift-ovn-kubernetes/ovnkube-node-j6bgr"
Sep 12 00:16:30.274403 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:16:30.271881    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/09b83899-89b7-4f4a-b485-c102295f188e-dbus\") pod \"global-pull-secret-syncer-nxrg5\" (UID: \"09b83899-89b7-4f4a-b485-c102295f188e\") " pod="kube-system/global-pull-secret-syncer-nxrg5"
Sep 12 00:16:30.274403 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:16:30.271925    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/a9b84142-f0ea-4b17-91b9-cff39fb1d4cf-host\") pod \"tuned-zjwjs\" (UID: \"a9b84142-f0ea-4b17-91b9-cff39fb1d4cf\") " pod="openshift-cluster-node-tuning-operator/tuned-zjwjs"
Sep 12 00:16:30.274403 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:16:30.271957    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/40698f9e-0e26-4d86-ac05-ffcd30e172d4-node-log\") pod \"ovnkube-node-j6bgr\" (UID: \"40698f9e-0e26-4d86-ac05-ffcd30e172d4\") " pod="openshift-ovn-kubernetes/ovnkube-node-j6bgr"
Sep 12 00:16:30.274403 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:16:30.271994    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/76f830f1-1f3e-4dd4-9deb-023f6ad1e2e4-multus-socket-dir-parent\") pod \"multus-57l7n\" (UID: \"76f830f1-1f3e-4dd4-9deb-023f6ad1e2e4\") " pod="openshift-multus/multus-57l7n"
Sep 12 00:16:30.274403 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:16:30.272006    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/76f830f1-1f3e-4dd4-9deb-023f6ad1e2e4-cni-binary-copy\") pod \"multus-57l7n\" (UID: \"76f830f1-1f3e-4dd4-9deb-023f6ad1e2e4\") " pod="openshift-multus/multus-57l7n"
Sep 12 00:16:30.274403 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:16:30.272023    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/76f830f1-1f3e-4dd4-9deb-023f6ad1e2e4-host-run-netns\") pod \"multus-57l7n\" (UID: \"76f830f1-1f3e-4dd4-9deb-023f6ad1e2e4\") " pod="openshift-multus/multus-57l7n"
Sep 12 00:16:30.274403 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:16:30.272050    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/40698f9e-0e26-4d86-ac05-ffcd30e172d4-log-socket\") pod \"ovnkube-node-j6bgr\" (UID: \"40698f9e-0e26-4d86-ac05-ffcd30e172d4\") " pod="openshift-ovn-kubernetes/ovnkube-node-j6bgr"
Sep 12 00:16:30.274403 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:16:30.272076    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/76f830f1-1f3e-4dd4-9deb-023f6ad1e2e4-system-cni-dir\") pod \"multus-57l7n\" (UID: \"76f830f1-1f3e-4dd4-9deb-023f6ad1e2e4\") " pod="openshift-multus/multus-57l7n"
Sep 12 00:16:30.274403 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:16:30.272081    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/a9b84142-f0ea-4b17-91b9-cff39fb1d4cf-etc-kubernetes\") pod \"tuned-zjwjs\" (UID: \"a9b84142-f0ea-4b17-91b9-cff39fb1d4cf\") " pod="openshift-cluster-node-tuning-operator/tuned-zjwjs"
Sep 12 00:16:30.274403 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:16:30.272078    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/40698f9e-0e26-4d86-ac05-ffcd30e172d4-ovn-node-metrics-cert\") pod \"ovnkube-node-j6bgr\" (UID: \"40698f9e-0e26-4d86-ac05-ffcd30e172d4\") " pod="openshift-ovn-kubernetes/ovnkube-node-j6bgr"
Sep 12 00:16:30.274403 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:16:30.272119    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/76f830f1-1f3e-4dd4-9deb-023f6ad1e2e4-host-run-netns\") pod \"multus-57l7n\" (UID: \"76f830f1-1f3e-4dd4-9deb-023f6ad1e2e4\") " pod="openshift-multus/multus-57l7n"
Sep 12 00:16:30.274403 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:16:30.272128    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/a9b84142-f0ea-4b17-91b9-cff39fb1d4cf-etc-systemd\") pod \"tuned-zjwjs\" (UID: \"a9b84142-f0ea-4b17-91b9-cff39fb1d4cf\") " pod="openshift-cluster-node-tuning-operator/tuned-zjwjs"
Sep 12 00:16:30.275167 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:16:30.272156    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/a9b84142-f0ea-4b17-91b9-cff39fb1d4cf-run\") pod \"tuned-zjwjs\" (UID: \"a9b84142-f0ea-4b17-91b9-cff39fb1d4cf\") " pod="openshift-cluster-node-tuning-operator/tuned-zjwjs"
Sep 12 00:16:30.275167 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:16:30.272164    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/76f830f1-1f3e-4dd4-9deb-023f6ad1e2e4-multus-socket-dir-parent\") pod \"multus-57l7n\" (UID: \"76f830f1-1f3e-4dd4-9deb-023f6ad1e2e4\") " pod="openshift-multus/multus-57l7n"
Sep 12 00:16:30.275167 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:16:30.272191    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/19b621f9-8fcc-4a57-b13d-862aad3d75ce-tmp-dir\") pod \"node-resolver-dnnhn\" (UID: \"19b621f9-8fcc-4a57-b13d-862aad3d75ce\") " pod="openshift-dns/node-resolver-dnnhn"
Sep 12 00:16:30.275167 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:16:30.272212    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/a9b84142-f0ea-4b17-91b9-cff39fb1d4cf-run\") pod \"tuned-zjwjs\" (UID: \"a9b84142-f0ea-4b17-91b9-cff39fb1d4cf\") " pod="openshift-cluster-node-tuning-operator/tuned-zjwjs"
Sep 12 00:16:30.275167 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:16:30.272218    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/76f830f1-1f3e-4dd4-9deb-023f6ad1e2e4-host-var-lib-cni-multus\") pod \"multus-57l7n\" (UID: \"76f830f1-1f3e-4dd4-9deb-023f6ad1e2e4\") " pod="openshift-multus/multus-57l7n"
Sep 12 00:16:30.275167 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:16:30.272247    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7h4vm\" (UniqueName: \"kubernetes.io/projected/dd7e1ff1-8f0d-4f40-9a32-2a6cd30cb239-kube-api-access-7h4vm\") pod \"network-check-target-b6792\" (UID: \"dd7e1ff1-8f0d-4f40-9a32-2a6cd30cb239\") " pod="openshift-network-diagnostics/network-check-target-b6792"
Sep 12 00:16:30.275167 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:16:30.272256    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/a9b84142-f0ea-4b17-91b9-cff39fb1d4cf-etc-systemd\") pod \"tuned-zjwjs\" (UID: \"a9b84142-f0ea-4b17-91b9-cff39fb1d4cf\") " pod="openshift-cluster-node-tuning-operator/tuned-zjwjs"
Sep 12 00:16:30.275167 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:16:30.272275    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xhc74\" (UniqueName: \"kubernetes.io/projected/76f830f1-1f3e-4dd4-9deb-023f6ad1e2e4-kube-api-access-xhc74\") pod \"multus-57l7n\" (UID: \"76f830f1-1f3e-4dd4-9deb-023f6ad1e2e4\") " pod="openshift-multus/multus-57l7n"
Sep 12 00:16:30.275167 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:16:30.272298    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/40698f9e-0e26-4d86-ac05-ffcd30e172d4-ovnkube-config\") pod \"ovnkube-node-j6bgr\" (UID: \"40698f9e-0e26-4d86-ac05-ffcd30e172d4\") " pod="openshift-ovn-kubernetes/ovnkube-node-j6bgr"
Sep 12 00:16:30.275167 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:16:30.272303    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/6d479776-cc30-4bea-bb3a-8b4c259d59c7-metrics-certs\") pod \"network-metrics-daemon-bdp25\" (UID: \"6d479776-cc30-4bea-bb3a-8b4c259d59c7\") " pod="openshift-multus/network-metrics-daemon-bdp25"
Sep 12 00:16:30.275167 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:16:30.272362    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/40698f9e-0e26-4d86-ac05-ffcd30e172d4-host-run-ovn-kubernetes\") pod \"ovnkube-node-j6bgr\" (UID: \"40698f9e-0e26-4d86-ac05-ffcd30e172d4\") " pod="openshift-ovn-kubernetes/ovnkube-node-j6bgr"
Sep 12 00:16:30.275167 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:16:30.272373    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/a9b84142-f0ea-4b17-91b9-cff39fb1d4cf-etc-sysctl-conf\") pod \"tuned-zjwjs\" (UID: \"a9b84142-f0ea-4b17-91b9-cff39fb1d4cf\") " pod="openshift-cluster-node-tuning-operator/tuned-zjwjs"
Sep 12 00:16:30.275167 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:16:30.270959    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/0a3e3c8c-cafa-4254-a972-fcf494ae4e28-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-62v9r\" (UID: \"0a3e3c8c-cafa-4254-a972-fcf494ae4e28\") " pod="openshift-multus/multus-additional-cni-plugins-62v9r"
Sep 12 00:16:30.275167 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:16:30.272388    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/a9b84142-f0ea-4b17-91b9-cff39fb1d4cf-sys\") pod \"tuned-zjwjs\" (UID: \"a9b84142-f0ea-4b17-91b9-cff39fb1d4cf\") " pod="openshift-cluster-node-tuning-operator/tuned-zjwjs"
Sep 12 00:16:30.275167 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:16:30.272414    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/76f830f1-1f3e-4dd4-9deb-023f6ad1e2e4-multus-cni-dir\") pod \"multus-57l7n\" (UID: \"76f830f1-1f3e-4dd4-9deb-023f6ad1e2e4\") " pod="openshift-multus/multus-57l7n"
Sep 12 00:16:30.275167 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:16:30.272439    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/76f830f1-1f3e-4dd4-9deb-023f6ad1e2e4-cnibin\") pod \"multus-57l7n\" (UID: \"76f830f1-1f3e-4dd4-9deb-023f6ad1e2e4\") " pod="openshift-multus/multus-57l7n"
Sep 12 00:16:30.275167 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:16:30.272446    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/0a3e3c8c-cafa-4254-a972-fcf494ae4e28-cnibin\") pod \"multus-additional-cni-plugins-62v9r\" (UID: \"0a3e3c8c-cafa-4254-a972-fcf494ae4e28\") " pod="openshift-multus/multus-additional-cni-plugins-62v9r"
Sep 12 00:16:30.275167 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:16:30.272468    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/a8819f1a-9bcc-495c-91fc-adfc4e4d87fd-host-slash\") pod \"iptables-alerter-jnfzr\" (UID: \"a8819f1a-9bcc-495c-91fc-adfc4e4d87fd\") " pod="openshift-network-operator/iptables-alerter-jnfzr"
Sep 12 00:16:30.275768 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:16:30.272503    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/40698f9e-0e26-4d86-ac05-ffcd30e172d4-host-cni-netd\") pod \"ovnkube-node-j6bgr\" (UID: \"40698f9e-0e26-4d86-ac05-ffcd30e172d4\") " pod="openshift-ovn-kubernetes/ovnkube-node-j6bgr"
Sep 12 00:16:30.275768 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:16:30.272530    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/76f830f1-1f3e-4dd4-9deb-023f6ad1e2e4-host-var-lib-kubelet\") pod \"multus-57l7n\" (UID: \"76f830f1-1f3e-4dd4-9deb-023f6ad1e2e4\") " pod="openshift-multus/multus-57l7n"
Sep 12 00:16:30.275768 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:16:30.272533    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/40698f9e-0e26-4d86-ac05-ffcd30e172d4-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-j6bgr\" (UID: \"40698f9e-0e26-4d86-ac05-ffcd30e172d4\") " pod="openshift-ovn-kubernetes/ovnkube-node-j6bgr"
Sep 12 00:16:30.275768 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:16:30.272580    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/40698f9e-0e26-4d86-ac05-ffcd30e172d4-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-j6bgr\" (UID: \"40698f9e-0e26-4d86-ac05-ffcd30e172d4\") " pod="openshift-ovn-kubernetes/ovnkube-node-j6bgr"
Sep 12 00:16:30.275768 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:16:30.272600    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/0a3e3c8c-cafa-4254-a972-fcf494ae4e28-tuning-conf-dir\") pod \"multus-additional-cni-plugins-62v9r\" (UID: \"0a3e3c8c-cafa-4254-a972-fcf494ae4e28\") " pod="openshift-multus/multus-additional-cni-plugins-62v9r"
Sep 12 00:16:30.275768 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:16:30.272627    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/76f830f1-1f3e-4dd4-9deb-023f6ad1e2e4-host-var-lib-cni-multus\") pod \"multus-57l7n\" (UID: \"76f830f1-1f3e-4dd4-9deb-023f6ad1e2e4\") " pod="openshift-multus/multus-57l7n"
Sep 12 00:16:30.275768 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:16:30.272635    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/19b621f9-8fcc-4a57-b13d-862aad3d75ce-tmp-dir\") pod \"node-resolver-dnnhn\" (UID: \"19b621f9-8fcc-4a57-b13d-862aad3d75ce\") " pod="openshift-dns/node-resolver-dnnhn"
Sep 12 00:16:30.275768 ip-10-0-141-128 kubenswrapper[2571]: E0912 00:16:30.272740    2571 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 12 00:16:30.275768 ip-10-0-141-128 kubenswrapper[2571]: E0912 00:16:30.272792    2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6d479776-cc30-4bea-bb3a-8b4c259d59c7-metrics-certs podName:6d479776-cc30-4bea-bb3a-8b4c259d59c7 nodeName:}" failed. No retries permitted until 2026-09-12 00:16:30.772775086 +0000 UTC m=+2.044415581 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/6d479776-cc30-4bea-bb3a-8b4c259d59c7-metrics-certs") pod "network-metrics-daemon-bdp25" (UID: "6d479776-cc30-4bea-bb3a-8b4c259d59c7") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 12 00:16:30.275768 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:16:30.272828    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/40698f9e-0e26-4d86-ac05-ffcd30e172d4-log-socket\") pod \"ovnkube-node-j6bgr\" (UID: \"40698f9e-0e26-4d86-ac05-ffcd30e172d4\") " pod="openshift-ovn-kubernetes/ovnkube-node-j6bgr"
Sep 12 00:16:30.275768 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:16:30.272850    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/40698f9e-0e26-4d86-ac05-ffcd30e172d4-systemd-units\") pod \"ovnkube-node-j6bgr\" (UID: \"40698f9e-0e26-4d86-ac05-ffcd30e172d4\") " pod="openshift-ovn-kubernetes/ovnkube-node-j6bgr"
Sep 12 00:16:30.275768 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:16:30.272887    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/40698f9e-0e26-4d86-ac05-ffcd30e172d4-host-run-ovn-kubernetes\") pod \"ovnkube-node-j6bgr\" (UID: \"40698f9e-0e26-4d86-ac05-ffcd30e172d4\") " pod="openshift-ovn-kubernetes/ovnkube-node-j6bgr"
Sep 12 00:16:30.275768 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:16:30.272901    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/40698f9e-0e26-4d86-ac05-ffcd30e172d4-host-cni-netd\") pod \"ovnkube-node-j6bgr\" (UID: \"40698f9e-0e26-4d86-ac05-ffcd30e172d4\") " pod="openshift-ovn-kubernetes/ovnkube-node-j6bgr"
Sep 12 00:16:30.275768 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:16:30.272906    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/76f830f1-1f3e-4dd4-9deb-023f6ad1e2e4-multus-cni-dir\") pod \"multus-57l7n\" (UID: \"76f830f1-1f3e-4dd4-9deb-023f6ad1e2e4\") " pod="openshift-multus/multus-57l7n"
Sep 12 00:16:30.275768 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:16:30.272956    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/76f830f1-1f3e-4dd4-9deb-023f6ad1e2e4-cnibin\") pod \"multus-57l7n\" (UID: \"76f830f1-1f3e-4dd4-9deb-023f6ad1e2e4\") " pod="openshift-multus/multus-57l7n"
Sep 12 00:16:30.275768 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:16:30.272976    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/a8819f1a-9bcc-495c-91fc-adfc4e4d87fd-host-slash\") pod \"iptables-alerter-jnfzr\" (UID: \"a8819f1a-9bcc-495c-91fc-adfc4e4d87fd\") " pod="openshift-network-operator/iptables-alerter-jnfzr"
Sep 12 00:16:30.275768 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:16:30.272988    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/a9b84142-f0ea-4b17-91b9-cff39fb1d4cf-sys\") pod \"tuned-zjwjs\" (UID: \"a9b84142-f0ea-4b17-91b9-cff39fb1d4cf\") " pod="openshift-cluster-node-tuning-operator/tuned-zjwjs"
Sep 12 00:16:30.276280 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:16:30.273001    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/40698f9e-0e26-4d86-ac05-ffcd30e172d4-etc-openvswitch\") pod \"ovnkube-node-j6bgr\" (UID: \"40698f9e-0e26-4d86-ac05-ffcd30e172d4\") " pod="openshift-ovn-kubernetes/ovnkube-node-j6bgr"
Sep 12 00:16:30.276280 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:16:30.274694    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/a9b84142-f0ea-4b17-91b9-cff39fb1d4cf-etc-tuned\") pod \"tuned-zjwjs\" (UID: \"a9b84142-f0ea-4b17-91b9-cff39fb1d4cf\") " pod="openshift-cluster-node-tuning-operator/tuned-zjwjs"
Sep 12 00:16:30.276280 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:16:30.274885    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/40698f9e-0e26-4d86-ac05-ffcd30e172d4-ovn-node-metrics-cert\") pod \"ovnkube-node-j6bgr\" (UID: \"40698f9e-0e26-4d86-ac05-ffcd30e172d4\") " pod="openshift-ovn-kubernetes/ovnkube-node-j6bgr"
Sep 12 00:16:30.276280 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:16:30.274997    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/71d215ed-e9d2-4a8b-bc73-f0ee2e0d83c5-agent-certs\") pod \"konnectivity-agent-r92pg\" (UID: \"71d215ed-e9d2-4a8b-bc73-f0ee2e0d83c5\") " pod="kube-system/konnectivity-agent-r92pg"
Sep 12 00:16:30.276280 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:16:30.275007    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/a9b84142-f0ea-4b17-91b9-cff39fb1d4cf-tmp\") pod \"tuned-zjwjs\" (UID: \"a9b84142-f0ea-4b17-91b9-cff39fb1d4cf\") " pod="openshift-cluster-node-tuning-operator/tuned-zjwjs"
Sep 12 00:16:30.290386 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:16:30.290366    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bljm6\" (UniqueName: \"kubernetes.io/projected/a8819f1a-9bcc-495c-91fc-adfc4e4d87fd-kube-api-access-bljm6\") pod \"iptables-alerter-jnfzr\" (UID: \"a8819f1a-9bcc-495c-91fc-adfc4e4d87fd\") " pod="openshift-network-operator/iptables-alerter-jnfzr"
Sep 12 00:16:30.291795 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:16:30.291773    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gz7sz\" (UniqueName: \"kubernetes.io/projected/6d479776-cc30-4bea-bb3a-8b4c259d59c7-kube-api-access-gz7sz\") pod \"network-metrics-daemon-bdp25\" (UID: \"6d479776-cc30-4bea-bb3a-8b4c259d59c7\") " pod="openshift-multus/network-metrics-daemon-bdp25"
Sep 12 00:16:30.291877 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:16:30.291806    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xhc74\" (UniqueName: \"kubernetes.io/projected/76f830f1-1f3e-4dd4-9deb-023f6ad1e2e4-kube-api-access-xhc74\") pod \"multus-57l7n\" (UID: \"76f830f1-1f3e-4dd4-9deb-023f6ad1e2e4\") " pod="openshift-multus/multus-57l7n"
Sep 12 00:16:30.292854 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:16:30.292827    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9r2pn\" (UniqueName: \"kubernetes.io/projected/a9b84142-f0ea-4b17-91b9-cff39fb1d4cf-kube-api-access-9r2pn\") pod \"tuned-zjwjs\" (UID: \"a9b84142-f0ea-4b17-91b9-cff39fb1d4cf\") " pod="openshift-cluster-node-tuning-operator/tuned-zjwjs"
Sep 12 00:16:30.294067 ip-10-0-141-128 kubenswrapper[2571]: E0912 00:16:30.293867    2571 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 12 00:16:30.294067 ip-10-0-141-128 kubenswrapper[2571]: E0912 00:16:30.293895    2571 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 12 00:16:30.294067 ip-10-0-141-128 kubenswrapper[2571]: E0912 00:16:30.293909    2571 projected.go:196] Error preparing data for projected volume kube-api-access-7h4vm for pod openshift-network-diagnostics/network-check-target-b6792: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 12 00:16:30.294067 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:16:30.293941    2571 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 12 00:16:30.294067 ip-10-0-141-128 kubenswrapper[2571]: E0912 00:16:30.293968    2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/dd7e1ff1-8f0d-4f40-9a32-2a6cd30cb239-kube-api-access-7h4vm podName:dd7e1ff1-8f0d-4f40-9a32-2a6cd30cb239 nodeName:}" failed. No retries permitted until 2026-09-12 00:16:30.793952337 +0000 UTC m=+2.065592847 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-7h4vm" (UniqueName: "kubernetes.io/projected/dd7e1ff1-8f0d-4f40-9a32-2a6cd30cb239-kube-api-access-7h4vm") pod "network-check-target-b6792" (UID: "dd7e1ff1-8f0d-4f40-9a32-2a6cd30cb239") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 12 00:16:30.297605 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:16:30.296851    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mn2n7\" (UniqueName: \"kubernetes.io/projected/7a6f512a-ad51-4ec7-a9a5-8b6a34cb5bb8-kube-api-access-mn2n7\") pod \"node-ca-qm6vt\" (UID: \"7a6f512a-ad51-4ec7-a9a5-8b6a34cb5bb8\") " pod="openshift-image-registry/node-ca-qm6vt"
Sep 12 00:16:30.298560 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:16:30.298540    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-d79n8\" (UniqueName: \"kubernetes.io/projected/0a3e3c8c-cafa-4254-a972-fcf494ae4e28-kube-api-access-d79n8\") pod \"multus-additional-cni-plugins-62v9r\" (UID: \"0a3e3c8c-cafa-4254-a972-fcf494ae4e28\") " pod="openshift-multus/multus-additional-cni-plugins-62v9r"
Sep 12 00:16:30.298560 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:16:30.298553    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nqsrv\" (UniqueName: \"kubernetes.io/projected/ef505089-5ca0-44f6-a9c1-c7c951c30faf-kube-api-access-nqsrv\") pod \"aws-ebs-csi-driver-node-nxczt\" (UID: \"ef505089-5ca0-44f6-a9c1-c7c951c30faf\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-nxczt"
Sep 12 00:16:30.300137 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:16:30.300117    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-96pjd\" (UniqueName: \"kubernetes.io/projected/40698f9e-0e26-4d86-ac05-ffcd30e172d4-kube-api-access-96pjd\") pod \"ovnkube-node-j6bgr\" (UID: \"40698f9e-0e26-4d86-ac05-ffcd30e172d4\") " pod="openshift-ovn-kubernetes/ovnkube-node-j6bgr"
Sep 12 00:16:30.300450 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:16:30.300429    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-m6rhb\" (UniqueName: \"kubernetes.io/projected/19b621f9-8fcc-4a57-b13d-862aad3d75ce-kube-api-access-m6rhb\") pod \"node-resolver-dnnhn\" (UID: \"19b621f9-8fcc-4a57-b13d-862aad3d75ce\") " pod="openshift-dns/node-resolver-dnnhn"
Sep 12 00:16:30.357123 ip-10-0-141-128 kubenswrapper[2571]: W0912 00:16:30.357092    2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda0171d806087a75aaabfcaff6900d5a8.slice/crio-e486c16fc4d554c83f9fe2a14d407b99a44b3042fa149cc6f8152d9338582f1a WatchSource:0}: Error finding container e486c16fc4d554c83f9fe2a14d407b99a44b3042fa149cc6f8152d9338582f1a: Status 404 returned error can't find the container with id e486c16fc4d554c83f9fe2a14d407b99a44b3042fa149cc6f8152d9338582f1a
Sep 12 00:16:30.357567 ip-10-0-141-128 kubenswrapper[2571]: W0912 00:16:30.357551    2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1e7d3039174fd351468a1c716f20fcb9.slice/crio-1b87702e289991efa6a5051bd311bd33b3c440148f11f8ac2a3653a8efe09aa7 WatchSource:0}: Error finding container 1b87702e289991efa6a5051bd311bd33b3c440148f11f8ac2a3653a8efe09aa7: Status 404 returned error can't find the container with id 1b87702e289991efa6a5051bd311bd33b3c440148f11f8ac2a3653a8efe09aa7
Sep 12 00:16:30.363524 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:16:30.363508    2571 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 12 00:16:30.387005 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:16:30.386977    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-62v9r"
Sep 12 00:16:30.396865 ip-10-0-141-128 kubenswrapper[2571]: W0912 00:16:30.396845    2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0a3e3c8c_cafa_4254_a972_fcf494ae4e28.slice/crio-6bba8e764701646399ebe7ac703ea03f45241c76a3ef9ba160b72cb773373a95 WatchSource:0}: Error finding container 6bba8e764701646399ebe7ac703ea03f45241c76a3ef9ba160b72cb773373a95: Status 404 returned error can't find the container with id 6bba8e764701646399ebe7ac703ea03f45241c76a3ef9ba160b72cb773373a95
Sep 12 00:16:30.403306 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:16:30.403287    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-jnfzr"
Sep 12 00:16:30.411142 ip-10-0-141-128 kubenswrapper[2571]: W0912 00:16:30.411118    2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda8819f1a_9bcc_495c_91fc_adfc4e4d87fd.slice/crio-e41ac3af1e6a88e83a905b0090c329dfe6cf891ad667b362a245bce390961309 WatchSource:0}: Error finding container e41ac3af1e6a88e83a905b0090c329dfe6cf891ad667b362a245bce390961309: Status 404 returned error can't find the container with id e41ac3af1e6a88e83a905b0090c329dfe6cf891ad667b362a245bce390961309
Sep 12 00:16:30.413050 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:16:30.413032    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-r92pg"
Sep 12 00:16:30.420548 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:16:30.420529    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-nxczt"
Sep 12 00:16:30.420950 ip-10-0-141-128 kubenswrapper[2571]: W0912 00:16:30.420930    2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod71d215ed_e9d2_4a8b_bc73_f0ee2e0d83c5.slice/crio-059d4cd097d0de948695c2159d76b5cd62e6caa297d636b99eeefd5fb9fb207a WatchSource:0}: Error finding container 059d4cd097d0de948695c2159d76b5cd62e6caa297d636b99eeefd5fb9fb207a: Status 404 returned error can't find the container with id 059d4cd097d0de948695c2159d76b5cd62e6caa297d636b99eeefd5fb9fb207a
Sep 12 00:16:30.429066 ip-10-0-141-128 kubenswrapper[2571]: W0912 00:16:30.429047    2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podef505089_5ca0_44f6_a9c1_c7c951c30faf.slice/crio-c37fec7cf10ab01f34e4c36c6807b9f9ff81df56ca6523cc8f3f6df823ec45ad WatchSource:0}: Error finding container c37fec7cf10ab01f34e4c36c6807b9f9ff81df56ca6523cc8f3f6df823ec45ad: Status 404 returned error can't find the container with id c37fec7cf10ab01f34e4c36c6807b9f9ff81df56ca6523cc8f3f6df823ec45ad
Sep 12 00:16:30.437152 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:16:30.437131    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-j6bgr"
Sep 12 00:16:30.444857 ip-10-0-141-128 kubenswrapper[2571]: W0912 00:16:30.444829    2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod40698f9e_0e26_4d86_ac05_ffcd30e172d4.slice/crio-1b87ad0c7924624c2867a2bcfda150843d5b0d1a7add1df11852a78e8585aa16 WatchSource:0}: Error finding container 1b87ad0c7924624c2867a2bcfda150843d5b0d1a7add1df11852a78e8585aa16: Status 404 returned error can't find the container with id 1b87ad0c7924624c2867a2bcfda150843d5b0d1a7add1df11852a78e8585aa16
Sep 12 00:16:30.453042 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:16:30.453026    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-zjwjs"
Sep 12 00:16:30.461020 ip-10-0-141-128 kubenswrapper[2571]: W0912 00:16:30.460999    2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda9b84142_f0ea_4b17_91b9_cff39fb1d4cf.slice/crio-cfe14092d535a551b349d16ed27a8dbd912c576767aa967457c3dca2ed84b7e4 WatchSource:0}: Error finding container cfe14092d535a551b349d16ed27a8dbd912c576767aa967457c3dca2ed84b7e4: Status 404 returned error can't find the container with id cfe14092d535a551b349d16ed27a8dbd912c576767aa967457c3dca2ed84b7e4
Sep 12 00:16:30.469320 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:16:30.469301    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-dnnhn"
Sep 12 00:16:30.476573 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:16:30.476426    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-qm6vt"
Sep 12 00:16:30.477421 ip-10-0-141-128 kubenswrapper[2571]: W0912 00:16:30.477402    2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod19b621f9_8fcc_4a57_b13d_862aad3d75ce.slice/crio-4edb64706f5084cfd936ab1763cbfbd3f6a7f88ae309a1b7a8b33e1f9289da89 WatchSource:0}: Error finding container 4edb64706f5084cfd936ab1763cbfbd3f6a7f88ae309a1b7a8b33e1f9289da89: Status 404 returned error can't find the container with id 4edb64706f5084cfd936ab1763cbfbd3f6a7f88ae309a1b7a8b33e1f9289da89
Sep 12 00:16:30.484064 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:16:30.484048    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-57l7n"
Sep 12 00:16:30.485013 ip-10-0-141-128 kubenswrapper[2571]: W0912 00:16:30.484992    2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7a6f512a_ad51_4ec7_a9a5_8b6a34cb5bb8.slice/crio-fb1b65b20e027dd2dbd4fdccd49769cccc7c5b853036633aa2740cfc252c0716 WatchSource:0}: Error finding container fb1b65b20e027dd2dbd4fdccd49769cccc7c5b853036633aa2740cfc252c0716: Status 404 returned error can't find the container with id fb1b65b20e027dd2dbd4fdccd49769cccc7c5b853036633aa2740cfc252c0716
Sep 12 00:16:30.494229 ip-10-0-141-128 kubenswrapper[2571]: W0912 00:16:30.494206    2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod76f830f1_1f3e_4dd4_9deb_023f6ad1e2e4.slice/crio-c527cf7f6b3b8784e559b0df347257892b9a58d004830ae57c5433daf20b434c WatchSource:0}: Error finding container c527cf7f6b3b8784e559b0df347257892b9a58d004830ae57c5433daf20b434c: Status 404 returned error can't find the container with id c527cf7f6b3b8784e559b0df347257892b9a58d004830ae57c5433daf20b434c
Sep 12 00:16:30.495786 ip-10-0-141-128 kubenswrapper[2571]: E0912 00:16:30.495762    2571 kuberuntime_manager.go:1459] "Unhandled Error" err="container kube-multus start failed in pod multus-57l7n_openshift-multus(76f830f1-1f3e-4dd4-9deb-023f6ad1e2e4): ErrImagePull: pull QPS exceeded" logger="UnhandledError"
Sep 12 00:16:30.495856 ip-10-0-141-128 kubenswrapper[2571]: E0912 00:16:30.495793    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"kube-multus\" with ErrImagePull: \"pull QPS exceeded\"" pod="openshift-multus/multus-57l7n" podUID="76f830f1-1f3e-4dd4-9deb-023f6ad1e2e4"
Sep 12 00:16:30.778144 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:16:30.778063    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/09b83899-89b7-4f4a-b485-c102295f188e-original-pull-secret\") pod \"global-pull-secret-syncer-nxrg5\" (UID: \"09b83899-89b7-4f4a-b485-c102295f188e\") " pod="kube-system/global-pull-secret-syncer-nxrg5"
Sep 12 00:16:30.778144 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:16:30.778127    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/6d479776-cc30-4bea-bb3a-8b4c259d59c7-metrics-certs\") pod \"network-metrics-daemon-bdp25\" (UID: \"6d479776-cc30-4bea-bb3a-8b4c259d59c7\") " pod="openshift-multus/network-metrics-daemon-bdp25"
Sep 12 00:16:30.778307 ip-10-0-141-128 kubenswrapper[2571]: E0912 00:16:30.778219    2571 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 12 00:16:30.778307 ip-10-0-141-128 kubenswrapper[2571]: E0912 00:16:30.778269    2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6d479776-cc30-4bea-bb3a-8b4c259d59c7-metrics-certs podName:6d479776-cc30-4bea-bb3a-8b4c259d59c7 nodeName:}" failed. No retries permitted until 2026-09-12 00:16:31.778255627 +0000 UTC m=+3.049896116 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/6d479776-cc30-4bea-bb3a-8b4c259d59c7-metrics-certs") pod "network-metrics-daemon-bdp25" (UID: "6d479776-cc30-4bea-bb3a-8b4c259d59c7") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 12 00:16:30.778382 ip-10-0-141-128 kubenswrapper[2571]: E0912 00:16:30.778219    2571 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 12 00:16:30.778382 ip-10-0-141-128 kubenswrapper[2571]: E0912 00:16:30.778351    2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/09b83899-89b7-4f4a-b485-c102295f188e-original-pull-secret podName:09b83899-89b7-4f4a-b485-c102295f188e nodeName:}" failed. No retries permitted until 2026-09-12 00:16:31.77833747 +0000 UTC m=+3.049977964 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/09b83899-89b7-4f4a-b485-c102295f188e-original-pull-secret") pod "global-pull-secret-syncer-nxrg5" (UID: "09b83899-89b7-4f4a-b485-c102295f188e") : object "kube-system"/"original-pull-secret" not registered
Sep 12 00:16:30.879441 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:16:30.879406    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7h4vm\" (UniqueName: \"kubernetes.io/projected/dd7e1ff1-8f0d-4f40-9a32-2a6cd30cb239-kube-api-access-7h4vm\") pod \"network-check-target-b6792\" (UID: \"dd7e1ff1-8f0d-4f40-9a32-2a6cd30cb239\") " pod="openshift-network-diagnostics/network-check-target-b6792"
Sep 12 00:16:30.879636 ip-10-0-141-128 kubenswrapper[2571]: E0912 00:16:30.879615    2571 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 12 00:16:30.879720 ip-10-0-141-128 kubenswrapper[2571]: E0912 00:16:30.879644    2571 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 12 00:16:30.879720 ip-10-0-141-128 kubenswrapper[2571]: E0912 00:16:30.879661    2571 projected.go:196] Error preparing data for projected volume kube-api-access-7h4vm for pod openshift-network-diagnostics/network-check-target-b6792: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 12 00:16:30.879807 ip-10-0-141-128 kubenswrapper[2571]: E0912 00:16:30.879725    2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/dd7e1ff1-8f0d-4f40-9a32-2a6cd30cb239-kube-api-access-7h4vm podName:dd7e1ff1-8f0d-4f40-9a32-2a6cd30cb239 nodeName:}" failed. No retries permitted until 2026-09-12 00:16:31.879706091 +0000 UTC m=+3.151346594 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-7h4vm" (UniqueName: "kubernetes.io/projected/dd7e1ff1-8f0d-4f40-9a32-2a6cd30cb239-kube-api-access-7h4vm") pod "network-check-target-b6792" (UID: "dd7e1ff1-8f0d-4f40-9a32-2a6cd30cb239") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 12 00:16:31.135423 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:16:31.135344    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-nxrg5"
Sep 12 00:16:31.135981 ip-10-0-141-128 kubenswrapper[2571]: E0912 00:16:31.135490    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-nxrg5" podUID="09b83899-89b7-4f4a-b485-c102295f188e"
Sep 12 00:16:31.142703 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:16:31.142632    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-nxczt" event={"ID":"ef505089-5ca0-44f6-a9c1-c7c951c30faf","Type":"ContainerStarted","Data":"c37fec7cf10ab01f34e4c36c6807b9f9ff81df56ca6523cc8f3f6df823ec45ad"}
Sep 12 00:16:31.146954 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:16:31.146860    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-r92pg" event={"ID":"71d215ed-e9d2-4a8b-bc73-f0ee2e0d83c5","Type":"ContainerStarted","Data":"059d4cd097d0de948695c2159d76b5cd62e6caa297d636b99eeefd5fb9fb207a"}
Sep 12 00:16:31.159167 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:16:31.159131    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-141-128.ec2.internal" event={"ID":"a0171d806087a75aaabfcaff6900d5a8","Type":"ContainerStarted","Data":"e486c16fc4d554c83f9fe2a14d407b99a44b3042fa149cc6f8152d9338582f1a"}
Sep 12 00:16:31.169098 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:16:31.169067    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-128.ec2.internal" event={"ID":"1e7d3039174fd351468a1c716f20fcb9","Type":"ContainerStarted","Data":"1b87702e289991efa6a5051bd311bd33b3c440148f11f8ac2a3653a8efe09aa7"}
Sep 12 00:16:31.178627 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:16:31.178594    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-j6bgr" event={"ID":"40698f9e-0e26-4d86-ac05-ffcd30e172d4","Type":"ContainerStarted","Data":"1b87ad0c7924624c2867a2bcfda150843d5b0d1a7add1df11852a78e8585aa16"}
Sep 12 00:16:31.180333 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:16:31.180308    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-jnfzr" event={"ID":"a8819f1a-9bcc-495c-91fc-adfc4e4d87fd","Type":"ContainerStarted","Data":"e41ac3af1e6a88e83a905b0090c329dfe6cf891ad667b362a245bce390961309"}
Sep 12 00:16:31.191094 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:16:31.191067    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-62v9r" event={"ID":"0a3e3c8c-cafa-4254-a972-fcf494ae4e28","Type":"ContainerStarted","Data":"6bba8e764701646399ebe7ac703ea03f45241c76a3ef9ba160b72cb773373a95"}
Sep 12 00:16:31.196657 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:16:31.196630    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-57l7n" event={"ID":"76f830f1-1f3e-4dd4-9deb-023f6ad1e2e4","Type":"ContainerStarted","Data":"c527cf7f6b3b8784e559b0df347257892b9a58d004830ae57c5433daf20b434c"}
Sep 12 00:16:31.201981 ip-10-0-141-128 kubenswrapper[2571]: E0912 00:16:31.201951    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"kube-multus\" with ImagePullBackOff: \"Back-off pulling image \\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:53f547a13d43a4e4789407df692a7707f1a00f37794bf7afc92db6fdc85929a6\\\": ErrImagePull: pull QPS exceeded\"" pod="openshift-multus/multus-57l7n" podUID="76f830f1-1f3e-4dd4-9deb-023f6ad1e2e4"
Sep 12 00:16:31.202632 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:16:31.202606    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-qm6vt" event={"ID":"7a6f512a-ad51-4ec7-a9a5-8b6a34cb5bb8","Type":"ContainerStarted","Data":"fb1b65b20e027dd2dbd4fdccd49769cccc7c5b853036633aa2740cfc252c0716"}
Sep 12 00:16:31.205960 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:16:31.205932    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-dnnhn" event={"ID":"19b621f9-8fcc-4a57-b13d-862aad3d75ce","Type":"ContainerStarted","Data":"4edb64706f5084cfd936ab1763cbfbd3f6a7f88ae309a1b7a8b33e1f9289da89"}
Sep 12 00:16:31.220895 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:16:31.220866    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-zjwjs" event={"ID":"a9b84142-f0ea-4b17-91b9-cff39fb1d4cf","Type":"ContainerStarted","Data":"cfe14092d535a551b349d16ed27a8dbd912c576767aa967457c3dca2ed84b7e4"}
Sep 12 00:16:31.284415 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:16:31.284324    2571 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-09-11 00:11:30 +0000 UTC" deadline="2028-02-16 15:03:07.33007745 +0000 UTC"
Sep 12 00:16:31.284415 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:16:31.284368    2571 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="12542h46m36.045712069s"
Sep 12 00:16:31.492746 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:16:31.492650    2571 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 12 00:16:31.788816 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:16:31.788673    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/6d479776-cc30-4bea-bb3a-8b4c259d59c7-metrics-certs\") pod \"network-metrics-daemon-bdp25\" (UID: \"6d479776-cc30-4bea-bb3a-8b4c259d59c7\") " pod="openshift-multus/network-metrics-daemon-bdp25"
Sep 12 00:16:31.788816 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:16:31.788750    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/09b83899-89b7-4f4a-b485-c102295f188e-original-pull-secret\") pod \"global-pull-secret-syncer-nxrg5\" (UID: \"09b83899-89b7-4f4a-b485-c102295f188e\") " pod="kube-system/global-pull-secret-syncer-nxrg5"
Sep 12 00:16:31.789055 ip-10-0-141-128 kubenswrapper[2571]: E0912 00:16:31.788889    2571 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 12 00:16:31.789055 ip-10-0-141-128 kubenswrapper[2571]: E0912 00:16:31.788957    2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/09b83899-89b7-4f4a-b485-c102295f188e-original-pull-secret podName:09b83899-89b7-4f4a-b485-c102295f188e nodeName:}" failed. No retries permitted until 2026-09-12 00:16:33.788934835 +0000 UTC m=+5.060575342 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/09b83899-89b7-4f4a-b485-c102295f188e-original-pull-secret") pod "global-pull-secret-syncer-nxrg5" (UID: "09b83899-89b7-4f4a-b485-c102295f188e") : object "kube-system"/"original-pull-secret" not registered
Sep 12 00:16:31.789490 ip-10-0-141-128 kubenswrapper[2571]: E0912 00:16:31.789466    2571 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 12 00:16:31.789585 ip-10-0-141-128 kubenswrapper[2571]: E0912 00:16:31.789526    2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6d479776-cc30-4bea-bb3a-8b4c259d59c7-metrics-certs podName:6d479776-cc30-4bea-bb3a-8b4c259d59c7 nodeName:}" failed. No retries permitted until 2026-09-12 00:16:33.789510315 +0000 UTC m=+5.061150804 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/6d479776-cc30-4bea-bb3a-8b4c259d59c7-metrics-certs") pod "network-metrics-daemon-bdp25" (UID: "6d479776-cc30-4bea-bb3a-8b4c259d59c7") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 12 00:16:31.889842 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:16:31.889800    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7h4vm\" (UniqueName: \"kubernetes.io/projected/dd7e1ff1-8f0d-4f40-9a32-2a6cd30cb239-kube-api-access-7h4vm\") pod \"network-check-target-b6792\" (UID: \"dd7e1ff1-8f0d-4f40-9a32-2a6cd30cb239\") " pod="openshift-network-diagnostics/network-check-target-b6792"
Sep 12 00:16:31.890090 ip-10-0-141-128 kubenswrapper[2571]: E0912 00:16:31.890013    2571 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 12 00:16:31.890090 ip-10-0-141-128 kubenswrapper[2571]: E0912 00:16:31.890034    2571 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 12 00:16:31.890090 ip-10-0-141-128 kubenswrapper[2571]: E0912 00:16:31.890047    2571 projected.go:196] Error preparing data for projected volume kube-api-access-7h4vm for pod openshift-network-diagnostics/network-check-target-b6792: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 12 00:16:31.890276 ip-10-0-141-128 kubenswrapper[2571]: E0912 00:16:31.890109    2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/dd7e1ff1-8f0d-4f40-9a32-2a6cd30cb239-kube-api-access-7h4vm podName:dd7e1ff1-8f0d-4f40-9a32-2a6cd30cb239 nodeName:}" failed. No retries permitted until 2026-09-12 00:16:33.890090343 +0000 UTC m=+5.161730848 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-7h4vm" (UniqueName: "kubernetes.io/projected/dd7e1ff1-8f0d-4f40-9a32-2a6cd30cb239-kube-api-access-7h4vm") pod "network-check-target-b6792" (UID: "dd7e1ff1-8f0d-4f40-9a32-2a6cd30cb239") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 12 00:16:32.134777 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:16:32.134699    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-bdp25"
Sep 12 00:16:32.134944 ip-10-0-141-128 kubenswrapper[2571]: E0912 00:16:32.134837    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-bdp25" podUID="6d479776-cc30-4bea-bb3a-8b4c259d59c7"
Sep 12 00:16:32.135277 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:16:32.135255    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-b6792"
Sep 12 00:16:32.135381 ip-10-0-141-128 kubenswrapper[2571]: E0912 00:16:32.135354    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-b6792" podUID="dd7e1ff1-8f0d-4f40-9a32-2a6cd30cb239"
Sep 12 00:16:32.285133 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:16:32.285084    2571 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-09-11 00:11:30 +0000 UTC" deadline="2028-06-23 04:09:16.702244598 +0000 UTC"
Sep 12 00:16:32.285133 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:16:32.285126    2571 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="15603h52m44.417124189s"
Sep 12 00:16:33.126796 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:16:33.126765    2571 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 12 00:16:33.139397 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:16:33.139374    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-nxrg5"
Sep 12 00:16:33.139548 ip-10-0-141-128 kubenswrapper[2571]: E0912 00:16:33.139508    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-nxrg5" podUID="09b83899-89b7-4f4a-b485-c102295f188e"
Sep 12 00:16:33.808578 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:16:33.808540    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/09b83899-89b7-4f4a-b485-c102295f188e-original-pull-secret\") pod \"global-pull-secret-syncer-nxrg5\" (UID: \"09b83899-89b7-4f4a-b485-c102295f188e\") " pod="kube-system/global-pull-secret-syncer-nxrg5"
Sep 12 00:16:33.809163 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:16:33.808641    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/6d479776-cc30-4bea-bb3a-8b4c259d59c7-metrics-certs\") pod \"network-metrics-daemon-bdp25\" (UID: \"6d479776-cc30-4bea-bb3a-8b4c259d59c7\") " pod="openshift-multus/network-metrics-daemon-bdp25"
Sep 12 00:16:33.809163 ip-10-0-141-128 kubenswrapper[2571]: E0912 00:16:33.808772    2571 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 12 00:16:33.809163 ip-10-0-141-128 kubenswrapper[2571]: E0912 00:16:33.808830    2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6d479776-cc30-4bea-bb3a-8b4c259d59c7-metrics-certs podName:6d479776-cc30-4bea-bb3a-8b4c259d59c7 nodeName:}" failed. No retries permitted until 2026-09-12 00:16:37.808811112 +0000 UTC m=+9.080451609 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/6d479776-cc30-4bea-bb3a-8b4c259d59c7-metrics-certs") pod "network-metrics-daemon-bdp25" (UID: "6d479776-cc30-4bea-bb3a-8b4c259d59c7") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 12 00:16:33.809547 ip-10-0-141-128 kubenswrapper[2571]: E0912 00:16:33.809402    2571 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 12 00:16:33.809547 ip-10-0-141-128 kubenswrapper[2571]: E0912 00:16:33.809464    2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/09b83899-89b7-4f4a-b485-c102295f188e-original-pull-secret podName:09b83899-89b7-4f4a-b485-c102295f188e nodeName:}" failed. No retries permitted until 2026-09-12 00:16:37.809444409 +0000 UTC m=+9.081084916 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/09b83899-89b7-4f4a-b485-c102295f188e-original-pull-secret") pod "global-pull-secret-syncer-nxrg5" (UID: "09b83899-89b7-4f4a-b485-c102295f188e") : object "kube-system"/"original-pull-secret" not registered
Sep 12 00:16:33.909038 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:16:33.908995    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7h4vm\" (UniqueName: \"kubernetes.io/projected/dd7e1ff1-8f0d-4f40-9a32-2a6cd30cb239-kube-api-access-7h4vm\") pod \"network-check-target-b6792\" (UID: \"dd7e1ff1-8f0d-4f40-9a32-2a6cd30cb239\") " pod="openshift-network-diagnostics/network-check-target-b6792"
Sep 12 00:16:33.909217 ip-10-0-141-128 kubenswrapper[2571]: E0912 00:16:33.909194    2571 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 12 00:16:33.909274 ip-10-0-141-128 kubenswrapper[2571]: E0912 00:16:33.909228    2571 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 12 00:16:33.909274 ip-10-0-141-128 kubenswrapper[2571]: E0912 00:16:33.909245    2571 projected.go:196] Error preparing data for projected volume kube-api-access-7h4vm for pod openshift-network-diagnostics/network-check-target-b6792: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 12 00:16:33.909370 ip-10-0-141-128 kubenswrapper[2571]: E0912 00:16:33.909316    2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/dd7e1ff1-8f0d-4f40-9a32-2a6cd30cb239-kube-api-access-7h4vm podName:dd7e1ff1-8f0d-4f40-9a32-2a6cd30cb239 nodeName:}" failed. No retries permitted until 2026-09-12 00:16:37.909296536 +0000 UTC m=+9.180937042 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-7h4vm" (UniqueName: "kubernetes.io/projected/dd7e1ff1-8f0d-4f40-9a32-2a6cd30cb239-kube-api-access-7h4vm") pod "network-check-target-b6792" (UID: "dd7e1ff1-8f0d-4f40-9a32-2a6cd30cb239") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 12 00:16:34.135083 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:16:34.135005    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-bdp25"
Sep 12 00:16:34.135252 ip-10-0-141-128 kubenswrapper[2571]: E0912 00:16:34.135184    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-bdp25" podUID="6d479776-cc30-4bea-bb3a-8b4c259d59c7"
Sep 12 00:16:34.135459 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:16:34.135004    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-b6792"
Sep 12 00:16:34.135619 ip-10-0-141-128 kubenswrapper[2571]: E0912 00:16:34.135452    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-b6792" podUID="dd7e1ff1-8f0d-4f40-9a32-2a6cd30cb239"
Sep 12 00:16:35.135345 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:16:35.135309    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-nxrg5"
Sep 12 00:16:35.135807 ip-10-0-141-128 kubenswrapper[2571]: E0912 00:16:35.135450    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-nxrg5" podUID="09b83899-89b7-4f4a-b485-c102295f188e"
Sep 12 00:16:36.135062 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:16:36.135024    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-bdp25"
Sep 12 00:16:36.135062 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:16:36.135046    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-b6792"
Sep 12 00:16:36.135333 ip-10-0-141-128 kubenswrapper[2571]: E0912 00:16:36.135173    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-bdp25" podUID="6d479776-cc30-4bea-bb3a-8b4c259d59c7"
Sep 12 00:16:36.135386 ip-10-0-141-128 kubenswrapper[2571]: E0912 00:16:36.135336    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-b6792" podUID="dd7e1ff1-8f0d-4f40-9a32-2a6cd30cb239"
Sep 12 00:16:37.137598 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:16:37.137549    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-nxrg5"
Sep 12 00:16:37.138023 ip-10-0-141-128 kubenswrapper[2571]: E0912 00:16:37.137689    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-nxrg5" podUID="09b83899-89b7-4f4a-b485-c102295f188e"
Sep 12 00:16:37.843290 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:16:37.843239    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/09b83899-89b7-4f4a-b485-c102295f188e-original-pull-secret\") pod \"global-pull-secret-syncer-nxrg5\" (UID: \"09b83899-89b7-4f4a-b485-c102295f188e\") " pod="kube-system/global-pull-secret-syncer-nxrg5"
Sep 12 00:16:37.843492 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:16:37.843328    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/6d479776-cc30-4bea-bb3a-8b4c259d59c7-metrics-certs\") pod \"network-metrics-daemon-bdp25\" (UID: \"6d479776-cc30-4bea-bb3a-8b4c259d59c7\") " pod="openshift-multus/network-metrics-daemon-bdp25"
Sep 12 00:16:37.843492 ip-10-0-141-128 kubenswrapper[2571]: E0912 00:16:37.843399    2571 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 12 00:16:37.843492 ip-10-0-141-128 kubenswrapper[2571]: E0912 00:16:37.843443    2571 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 12 00:16:37.843492 ip-10-0-141-128 kubenswrapper[2571]: E0912 00:16:37.843469    2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/09b83899-89b7-4f4a-b485-c102295f188e-original-pull-secret podName:09b83899-89b7-4f4a-b485-c102295f188e nodeName:}" failed. No retries permitted until 2026-09-12 00:16:45.843450741 +0000 UTC m=+17.115091248 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/09b83899-89b7-4f4a-b485-c102295f188e-original-pull-secret") pod "global-pull-secret-syncer-nxrg5" (UID: "09b83899-89b7-4f4a-b485-c102295f188e") : object "kube-system"/"original-pull-secret" not registered
Sep 12 00:16:37.843492 ip-10-0-141-128 kubenswrapper[2571]: E0912 00:16:37.843493    2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6d479776-cc30-4bea-bb3a-8b4c259d59c7-metrics-certs podName:6d479776-cc30-4bea-bb3a-8b4c259d59c7 nodeName:}" failed. No retries permitted until 2026-09-12 00:16:45.843476752 +0000 UTC m=+17.115117241 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/6d479776-cc30-4bea-bb3a-8b4c259d59c7-metrics-certs") pod "network-metrics-daemon-bdp25" (UID: "6d479776-cc30-4bea-bb3a-8b4c259d59c7") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 12 00:16:37.944702 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:16:37.944661    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7h4vm\" (UniqueName: \"kubernetes.io/projected/dd7e1ff1-8f0d-4f40-9a32-2a6cd30cb239-kube-api-access-7h4vm\") pod \"network-check-target-b6792\" (UID: \"dd7e1ff1-8f0d-4f40-9a32-2a6cd30cb239\") " pod="openshift-network-diagnostics/network-check-target-b6792"
Sep 12 00:16:37.944887 ip-10-0-141-128 kubenswrapper[2571]: E0912 00:16:37.944863    2571 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 12 00:16:37.944955 ip-10-0-141-128 kubenswrapper[2571]: E0912 00:16:37.944898    2571 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 12 00:16:37.944955 ip-10-0-141-128 kubenswrapper[2571]: E0912 00:16:37.944912    2571 projected.go:196] Error preparing data for projected volume kube-api-access-7h4vm for pod openshift-network-diagnostics/network-check-target-b6792: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 12 00:16:37.945071 ip-10-0-141-128 kubenswrapper[2571]: E0912 00:16:37.944977    2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/dd7e1ff1-8f0d-4f40-9a32-2a6cd30cb239-kube-api-access-7h4vm podName:dd7e1ff1-8f0d-4f40-9a32-2a6cd30cb239 nodeName:}" failed. No retries permitted until 2026-09-12 00:16:45.944956953 +0000 UTC m=+17.216597458 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-7h4vm" (UniqueName: "kubernetes.io/projected/dd7e1ff1-8f0d-4f40-9a32-2a6cd30cb239-kube-api-access-7h4vm") pod "network-check-target-b6792" (UID: "dd7e1ff1-8f0d-4f40-9a32-2a6cd30cb239") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 12 00:16:38.135183 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:16:38.135089    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-bdp25"
Sep 12 00:16:38.135183 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:16:38.135126    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-b6792"
Sep 12 00:16:38.135408 ip-10-0-141-128 kubenswrapper[2571]: E0912 00:16:38.135245    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-bdp25" podUID="6d479776-cc30-4bea-bb3a-8b4c259d59c7"
Sep 12 00:16:38.135408 ip-10-0-141-128 kubenswrapper[2571]: E0912 00:16:38.135375    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-b6792" podUID="dd7e1ff1-8f0d-4f40-9a32-2a6cd30cb239"
Sep 12 00:16:39.136624 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:16:39.135874    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-nxrg5"
Sep 12 00:16:39.136624 ip-10-0-141-128 kubenswrapper[2571]: E0912 00:16:39.135980    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-nxrg5" podUID="09b83899-89b7-4f4a-b485-c102295f188e"
Sep 12 00:16:40.134694 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:16:40.134660    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-bdp25"
Sep 12 00:16:40.134878 ip-10-0-141-128 kubenswrapper[2571]: E0912 00:16:40.134793    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-bdp25" podUID="6d479776-cc30-4bea-bb3a-8b4c259d59c7"
Sep 12 00:16:40.134878 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:16:40.134830    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-b6792"
Sep 12 00:16:40.134993 ip-10-0-141-128 kubenswrapper[2571]: E0912 00:16:40.134884    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-b6792" podUID="dd7e1ff1-8f0d-4f40-9a32-2a6cd30cb239"
Sep 12 00:16:41.135045 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:16:41.135007    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-nxrg5"
Sep 12 00:16:41.135613 ip-10-0-141-128 kubenswrapper[2571]: E0912 00:16:41.135139    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-nxrg5" podUID="09b83899-89b7-4f4a-b485-c102295f188e"
Sep 12 00:16:42.134449 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:16:42.134413    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-bdp25"
Sep 12 00:16:42.134652 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:16:42.134462    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-b6792"
Sep 12 00:16:42.134652 ip-10-0-141-128 kubenswrapper[2571]: E0912 00:16:42.134556    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-bdp25" podUID="6d479776-cc30-4bea-bb3a-8b4c259d59c7"
Sep 12 00:16:42.134767 ip-10-0-141-128 kubenswrapper[2571]: E0912 00:16:42.134686    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-b6792" podUID="dd7e1ff1-8f0d-4f40-9a32-2a6cd30cb239"
Sep 12 00:16:43.134549 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:16:43.134512    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-nxrg5"
Sep 12 00:16:43.135036 ip-10-0-141-128 kubenswrapper[2571]: E0912 00:16:43.134661    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-nxrg5" podUID="09b83899-89b7-4f4a-b485-c102295f188e"
Sep 12 00:16:44.135147 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:16:44.135102    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-b6792"
Sep 12 00:16:44.135626 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:16:44.135150    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-bdp25"
Sep 12 00:16:44.135626 ip-10-0-141-128 kubenswrapper[2571]: E0912 00:16:44.135239    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-b6792" podUID="dd7e1ff1-8f0d-4f40-9a32-2a6cd30cb239"
Sep 12 00:16:44.135626 ip-10-0-141-128 kubenswrapper[2571]: E0912 00:16:44.135364    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-bdp25" podUID="6d479776-cc30-4bea-bb3a-8b4c259d59c7"
Sep 12 00:16:45.136220 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:16:45.135843    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-nxrg5"
Sep 12 00:16:45.136220 ip-10-0-141-128 kubenswrapper[2571]: E0912 00:16:45.136049    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-nxrg5" podUID="09b83899-89b7-4f4a-b485-c102295f188e"
Sep 12 00:16:45.282970 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:16:45.282727    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-zjwjs" event={"ID":"a9b84142-f0ea-4b17-91b9-cff39fb1d4cf","Type":"ContainerStarted","Data":"45bfeef029c3cc61164b6e6da08502406c1fab9fc85ba282ea4de7db94946b6f"}
Sep 12 00:16:45.291433 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:16:45.291405    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-r92pg" event={"ID":"71d215ed-e9d2-4a8b-bc73-f0ee2e0d83c5","Type":"ContainerStarted","Data":"a84006c3f8dde45128e978f959b7cb4ef7a28e5570ca3faf6ce2cee8cc8b96fd"}
Sep 12 00:16:45.293126 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:16:45.293068    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-141-128.ec2.internal" event={"ID":"a0171d806087a75aaabfcaff6900d5a8","Type":"ContainerStarted","Data":"7cc25d5de71bf3b53f248844c3fb69fcb00c4828605a39c1cdb2da1c2ef5e58e"}
Sep 12 00:16:45.297738 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:16:45.297710    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-j6bgr" event={"ID":"40698f9e-0e26-4d86-ac05-ffcd30e172d4","Type":"ContainerStarted","Data":"0f43072980235502341ff4ac26b9d583ddf61d9e35c31364fb12416714a78658"}
Sep 12 00:16:45.297839 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:16:45.297789    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-j6bgr" event={"ID":"40698f9e-0e26-4d86-ac05-ffcd30e172d4","Type":"ContainerStarted","Data":"ce8199279a742f52d108f1334f01fe684646ba7a358b625c5883273427f45694"}
Sep 12 00:16:45.297839 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:16:45.297834    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-j6bgr" event={"ID":"40698f9e-0e26-4d86-ac05-ffcd30e172d4","Type":"ContainerStarted","Data":"9186effb5558f951373eadcc3d249615f35fa6ac7924033399c01daae0569207"}
Sep 12 00:16:45.297914 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:16:45.297877    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-j6bgr" event={"ID":"40698f9e-0e26-4d86-ac05-ffcd30e172d4","Type":"ContainerStarted","Data":"d635bbe13caa7f0a3a268f7d85e712f329b0e8da09f0d0dfc4ec73e9e0059484"}
Sep 12 00:16:45.299423 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:16:45.299401    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-62v9r" event={"ID":"0a3e3c8c-cafa-4254-a972-fcf494ae4e28","Type":"ContainerStarted","Data":"803ba0c4fdb15eaeec79a4a0b3f87e0d8b802c173c9eede2371a67cd9e78b416"}
Sep 12 00:16:45.301378 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:16:45.301355    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-qm6vt" event={"ID":"7a6f512a-ad51-4ec7-a9a5-8b6a34cb5bb8","Type":"ContainerStarted","Data":"1d71fb6d2387fc01cf4bdf1df33ec75698ca4622afc09c4bb2ac96ba2be88ac4"}
Sep 12 00:16:45.319970 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:16:45.319920    2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-zjwjs" podStartSLOduration=0.978106724 podStartE2EDuration="15.319905727s" totalImagesPullingTime="14.341799003s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-12 00:16:30 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-12 00:16:45.304827083 +0000 UTC m=+16.576467591" watchObservedRunningTime="2026-09-12 00:16:45.319905727 +0000 UTC m=+16.591546314"
Sep 12 00:16:45.320099 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:16:45.320079    2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-qm6vt" podStartSLOduration=1.003038092 podStartE2EDuration="15.320070717s" totalImagesPullingTime="14.317032625s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-12 00:16:30 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-12 00:16:45.319666511 +0000 UTC m=+16.591307021" watchObservedRunningTime="2026-09-12 00:16:45.320070717 +0000 UTC m=+16.591711236"
Sep 12 00:16:45.374843 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:16:45.374793    2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-141-128.ec2.internal" podStartSLOduration=15.374777495 podStartE2EDuration="15.374777495s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-12 00:16:30 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-12 00:16:45.335970186 +0000 UTC m=+16.607610708" watchObservedRunningTime="2026-09-12 00:16:45.374777495 +0000 UTC m=+16.646418075"
Sep 12 00:16:45.393520 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:16:45.393479    2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-r92pg" podStartSLOduration=1.039936976 podStartE2EDuration="15.393462816s" totalImagesPullingTime="14.35352584s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-12 00:16:30 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-12 00:16:45.39311463 +0000 UTC m=+16.664755140" watchObservedRunningTime="2026-09-12 00:16:45.393462816 +0000 UTC m=+16.665103325"
Sep 12 00:16:45.905063 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:16:45.905025    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/6d479776-cc30-4bea-bb3a-8b4c259d59c7-metrics-certs\") pod \"network-metrics-daemon-bdp25\" (UID: \"6d479776-cc30-4bea-bb3a-8b4c259d59c7\") " pod="openshift-multus/network-metrics-daemon-bdp25"
Sep 12 00:16:45.905271 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:16:45.905095    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/09b83899-89b7-4f4a-b485-c102295f188e-original-pull-secret\") pod \"global-pull-secret-syncer-nxrg5\" (UID: \"09b83899-89b7-4f4a-b485-c102295f188e\") " pod="kube-system/global-pull-secret-syncer-nxrg5"
Sep 12 00:16:45.905271 ip-10-0-141-128 kubenswrapper[2571]: E0912 00:16:45.905160    2571 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 12 00:16:45.905271 ip-10-0-141-128 kubenswrapper[2571]: E0912 00:16:45.905219    2571 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 12 00:16:45.905271 ip-10-0-141-128 kubenswrapper[2571]: E0912 00:16:45.905232    2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6d479776-cc30-4bea-bb3a-8b4c259d59c7-metrics-certs podName:6d479776-cc30-4bea-bb3a-8b4c259d59c7 nodeName:}" failed. No retries permitted until 2026-09-12 00:17:01.905212277 +0000 UTC m=+33.176852765 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/6d479776-cc30-4bea-bb3a-8b4c259d59c7-metrics-certs") pod "network-metrics-daemon-bdp25" (UID: "6d479776-cc30-4bea-bb3a-8b4c259d59c7") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 12 00:16:45.905271 ip-10-0-141-128 kubenswrapper[2571]: E0912 00:16:45.905270    2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/09b83899-89b7-4f4a-b485-c102295f188e-original-pull-secret podName:09b83899-89b7-4f4a-b485-c102295f188e nodeName:}" failed. No retries permitted until 2026-09-12 00:17:01.90525489 +0000 UTC m=+33.176895387 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/09b83899-89b7-4f4a-b485-c102295f188e-original-pull-secret") pod "global-pull-secret-syncer-nxrg5" (UID: "09b83899-89b7-4f4a-b485-c102295f188e") : object "kube-system"/"original-pull-secret" not registered
Sep 12 00:16:46.006244 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:16:46.006209    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7h4vm\" (UniqueName: \"kubernetes.io/projected/dd7e1ff1-8f0d-4f40-9a32-2a6cd30cb239-kube-api-access-7h4vm\") pod \"network-check-target-b6792\" (UID: \"dd7e1ff1-8f0d-4f40-9a32-2a6cd30cb239\") " pod="openshift-network-diagnostics/network-check-target-b6792"
Sep 12 00:16:46.006425 ip-10-0-141-128 kubenswrapper[2571]: E0912 00:16:46.006331    2571 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 12 00:16:46.006425 ip-10-0-141-128 kubenswrapper[2571]: E0912 00:16:46.006350    2571 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 12 00:16:46.006425 ip-10-0-141-128 kubenswrapper[2571]: E0912 00:16:46.006363    2571 projected.go:196] Error preparing data for projected volume kube-api-access-7h4vm for pod openshift-network-diagnostics/network-check-target-b6792: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 12 00:16:46.006425 ip-10-0-141-128 kubenswrapper[2571]: E0912 00:16:46.006422    2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/dd7e1ff1-8f0d-4f40-9a32-2a6cd30cb239-kube-api-access-7h4vm podName:dd7e1ff1-8f0d-4f40-9a32-2a6cd30cb239 nodeName:}" failed. No retries permitted until 2026-09-12 00:17:02.006407443 +0000 UTC m=+33.278047939 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-7h4vm" (UniqueName: "kubernetes.io/projected/dd7e1ff1-8f0d-4f40-9a32-2a6cd30cb239-kube-api-access-7h4vm") pod "network-check-target-b6792" (UID: "dd7e1ff1-8f0d-4f40-9a32-2a6cd30cb239") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 12 00:16:46.134941 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:16:46.134863    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-bdp25"
Sep 12 00:16:46.135074 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:16:46.134882    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-b6792"
Sep 12 00:16:46.135074 ip-10-0-141-128 kubenswrapper[2571]: E0912 00:16:46.135000    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-bdp25" podUID="6d479776-cc30-4bea-bb3a-8b4c259d59c7"
Sep 12 00:16:46.135074 ip-10-0-141-128 kubenswrapper[2571]: E0912 00:16:46.135025    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-b6792" podUID="dd7e1ff1-8f0d-4f40-9a32-2a6cd30cb239"
Sep 12 00:16:46.305381 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:16:46.305350    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-nxczt" event={"ID":"ef505089-5ca0-44f6-a9c1-c7c951c30faf","Type":"ContainerStarted","Data":"8035b70eb39cc5d5491a91cbe4d2a54d2ff84cfe97fa5885a2b758aac10ffea8"}
Sep 12 00:16:46.307134 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:16:46.307098    2571 generic.go:356] "Generic (PLEG): container finished" podID="1e7d3039174fd351468a1c716f20fcb9" containerID="0c217ce035e63fb2a5b2f3e781a99f1a2711a838705197e53b82e98333b6f235" exitCode=0
Sep 12 00:16:46.307255 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:16:46.307176    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-128.ec2.internal" event={"ID":"1e7d3039174fd351468a1c716f20fcb9","Type":"ContainerDied","Data":"0c217ce035e63fb2a5b2f3e781a99f1a2711a838705197e53b82e98333b6f235"}
Sep 12 00:16:46.310676 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:16:46.310652    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-j6bgr" event={"ID":"40698f9e-0e26-4d86-ac05-ffcd30e172d4","Type":"ContainerStarted","Data":"eb97920906561fb1369b82a7507d2f5aac44be88bcb06bcd39fbf2c261575287"}
Sep 12 00:16:46.310796 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:16:46.310730    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-j6bgr" event={"ID":"40698f9e-0e26-4d86-ac05-ffcd30e172d4","Type":"ContainerStarted","Data":"2f7607340b8d6345d7ba3e6807979bf7be8d19718a0da5e687ef0eb517f63421"}
Sep 12 00:16:46.314549 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:16:46.312988    2571 generic.go:356] "Generic (PLEG): container finished" podID="0a3e3c8c-cafa-4254-a972-fcf494ae4e28" containerID="803ba0c4fdb15eaeec79a4a0b3f87e0d8b802c173c9eede2371a67cd9e78b416" exitCode=0
Sep 12 00:16:46.314549 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:16:46.313085    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-62v9r" event={"ID":"0a3e3c8c-cafa-4254-a972-fcf494ae4e28","Type":"ContainerDied","Data":"803ba0c4fdb15eaeec79a4a0b3f87e0d8b802c173c9eede2371a67cd9e78b416"}
Sep 12 00:16:46.317115 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:16:46.317093    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-dnnhn" event={"ID":"19b621f9-8fcc-4a57-b13d-862aad3d75ce","Type":"ContainerStarted","Data":"8a4465cc20de21cccf560e9920a99c16cd858473770a8ff6ac155f58c5e2d1d5"}
Sep 12 00:16:46.446165 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:16:46.446137    2571 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock"
Sep 12 00:16:47.135722 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:16:47.135219    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-nxrg5"
Sep 12 00:16:47.135722 ip-10-0-141-128 kubenswrapper[2571]: E0912 00:16:47.135357    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-nxrg5" podUID="09b83899-89b7-4f4a-b485-c102295f188e"
Sep 12 00:16:47.269789 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:16:47.269659    2571 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-09-12T00:16:46.446160697Z","UUID":"e7a27414-ab8d-44d4-a3cc-6d80f7f79acb","Handler":null,"Name":"","Endpoint":""}
Sep 12 00:16:47.271404 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:16:47.271367    2571 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0
Sep 12 00:16:47.271404 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:16:47.271405    2571 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock
Sep 12 00:16:47.322165 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:16:47.322121    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-nxczt" event={"ID":"ef505089-5ca0-44f6-a9c1-c7c951c30faf","Type":"ContainerStarted","Data":"08eadfd0b9005bd8cc50678986dbbeaa5ffb2f6f1d67e5f0cfce59bc4b2ead5b"}
Sep 12 00:16:47.324421 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:16:47.324395    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-128.ec2.internal" event={"ID":"1e7d3039174fd351468a1c716f20fcb9","Type":"ContainerStarted","Data":"4242e7404a7210044b419c8b818d6f96e7082082c91ac02a249a420fbe8f1272"}
Sep 12 00:16:47.325995 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:16:47.325903    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-jnfzr" event={"ID":"a8819f1a-9bcc-495c-91fc-adfc4e4d87fd","Type":"ContainerStarted","Data":"dda8ebd20466f151c491d5d93f5ad6e808a215b9bb23dfb9f7e204d3facc0dd3"}
Sep 12 00:16:47.338659 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:16:47.338530    2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-128.ec2.internal" podStartSLOduration=17.338511341 podStartE2EDuration="17.338511341s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-12 00:16:30 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-12 00:16:47.338460987 +0000 UTC m=+18.610101500" watchObservedRunningTime="2026-09-12 00:16:47.338511341 +0000 UTC m=+18.610151851"
Sep 12 00:16:47.338960 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:16:47.338922    2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-dnnhn" podStartSLOduration=3.042117568 podStartE2EDuration="17.338913931s" totalImagesPullingTime="14.296796363s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-12 00:16:30 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-12 00:16:46.354190857 +0000 UTC m=+17.625831367" watchObservedRunningTime="2026-09-12 00:16:47.338913931 +0000 UTC m=+18.610554450"
Sep 12 00:16:47.352826 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:16:47.352732    2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-jnfzr" podStartSLOduration=6.548738313 podStartE2EDuration="17.352713011s" totalImagesPullingTime="10.803974698s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-12 00:16:30 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-12 00:16:47.352012109 +0000 UTC m=+18.623652620" watchObservedRunningTime="2026-09-12 00:16:47.352713011 +0000 UTC m=+18.624353521"
Sep 12 00:16:48.134848 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:16:48.134807    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-bdp25"
Sep 12 00:16:48.134848 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:16:48.134843    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-b6792"
Sep 12 00:16:48.135070 ip-10-0-141-128 kubenswrapper[2571]: E0912 00:16:48.134946    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-bdp25" podUID="6d479776-cc30-4bea-bb3a-8b4c259d59c7"
Sep 12 00:16:48.135138 ip-10-0-141-128 kubenswrapper[2571]: E0912 00:16:48.135071    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-b6792" podUID="dd7e1ff1-8f0d-4f40-9a32-2a6cd30cb239"
Sep 12 00:16:48.332527 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:16:48.332489    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-j6bgr" event={"ID":"40698f9e-0e26-4d86-ac05-ffcd30e172d4","Type":"ContainerStarted","Data":"dca013e3dc9e58efda81dc88b645f034e67e90992724359b8992019291e11658"}
Sep 12 00:16:48.334875 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:16:48.334842    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-nxczt" event={"ID":"ef505089-5ca0-44f6-a9c1-c7c951c30faf","Type":"ContainerStarted","Data":"7cb7a1c7e8201fb21044ae3cf08015ef5acab28f7147ea1dcdcac0df949e2438"}
Sep 12 00:16:48.352581 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:16:48.352353    2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-nxczt" podStartSLOduration=1.9294853189999999 podStartE2EDuration="18.352339721s" totalImagesPullingTime="16.422854402s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-12 00:16:30 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-12 00:16:48.352201026 +0000 UTC m=+19.623841537" watchObservedRunningTime="2026-09-12 00:16:48.352339721 +0000 UTC m=+19.623980230"
Sep 12 00:16:48.554152 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:16:48.554071    2571 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-r92pg"
Sep 12 00:16:48.554736 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:16:48.554715    2571 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-r92pg"
Sep 12 00:16:49.135034 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:16:49.134999    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-nxrg5"
Sep 12 00:16:49.135201 ip-10-0-141-128 kubenswrapper[2571]: E0912 00:16:49.135084    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-nxrg5" podUID="09b83899-89b7-4f4a-b485-c102295f188e"
Sep 12 00:16:50.135178 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:16:50.135090    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-bdp25"
Sep 12 00:16:50.135178 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:16:50.135119    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-b6792"
Sep 12 00:16:50.135638 ip-10-0-141-128 kubenswrapper[2571]: E0912 00:16:50.135238    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-bdp25" podUID="6d479776-cc30-4bea-bb3a-8b4c259d59c7"
Sep 12 00:16:50.135638 ip-10-0-141-128 kubenswrapper[2571]: E0912 00:16:50.135313    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-b6792" podUID="dd7e1ff1-8f0d-4f40-9a32-2a6cd30cb239"
Sep 12 00:16:50.344207 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:16:50.344161    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-j6bgr" event={"ID":"40698f9e-0e26-4d86-ac05-ffcd30e172d4","Type":"ContainerStarted","Data":"2a32ed5048bc3c95f924a02fe38e8f949f8b13d9852060402fc3c306849071b8"}
Sep 12 00:16:50.347965 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:16:50.345553    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-j6bgr"
Sep 12 00:16:50.362912 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:16:50.362886    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-j6bgr"
Sep 12 00:16:50.380191 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:16:50.380134    2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-j6bgr" podStartSLOduration=5.933990258 podStartE2EDuration="20.380108444s" totalImagesPullingTime="14.446118186s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-12 00:16:30 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-12 00:16:50.379415426 +0000 UTC m=+21.651055956" watchObservedRunningTime="2026-09-12 00:16:50.380108444 +0000 UTC m=+21.651748956"
Sep 12 00:16:51.134778 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:16:51.134540    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-nxrg5"
Sep 12 00:16:51.134943 ip-10-0-141-128 kubenswrapper[2571]: E0912 00:16:51.134876    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-nxrg5" podUID="09b83899-89b7-4f4a-b485-c102295f188e"
Sep 12 00:16:51.346622 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:16:51.346450    2571 prober_manager.go:312] "Failed to trigger a manual run" probe="Readiness"
Sep 12 00:16:51.347045 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:16:51.346985    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-j6bgr"
Sep 12 00:16:51.367682 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:16:51.367541    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-j6bgr"
Sep 12 00:16:51.653008 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:16:51.652973    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-nxrg5"]
Sep 12 00:16:51.653184 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:16:51.653085    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-nxrg5"
Sep 12 00:16:51.653257 ip-10-0-141-128 kubenswrapper[2571]: E0912 00:16:51.653183    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-nxrg5" podUID="09b83899-89b7-4f4a-b485-c102295f188e"
Sep 12 00:16:51.655965 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:16:51.655912    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-b6792"]
Sep 12 00:16:51.656296 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:16:51.656278    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-b6792"
Sep 12 00:16:51.656400 ip-10-0-141-128 kubenswrapper[2571]: E0912 00:16:51.656376    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-b6792" podUID="dd7e1ff1-8f0d-4f40-9a32-2a6cd30cb239"
Sep 12 00:16:51.656687 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:16:51.656664    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-bdp25"]
Sep 12 00:16:51.656783 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:16:51.656767    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-bdp25"
Sep 12 00:16:51.656906 ip-10-0-141-128 kubenswrapper[2571]: E0912 00:16:51.656864    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-bdp25" podUID="6d479776-cc30-4bea-bb3a-8b4c259d59c7"
Sep 12 00:16:52.349154 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:16:52.349124    2571 prober_manager.go:312] "Failed to trigger a manual run" probe="Readiness"
Sep 12 00:16:52.816046 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:16:52.816005    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-j6bgr"
Sep 12 00:16:53.134809 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:16:53.134731    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-b6792"
Sep 12 00:16:53.135067 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:16:53.134731    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-nxrg5"
Sep 12 00:16:53.135067 ip-10-0-141-128 kubenswrapper[2571]: E0912 00:16:53.134857    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-b6792" podUID="dd7e1ff1-8f0d-4f40-9a32-2a6cd30cb239"
Sep 12 00:16:53.135067 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:16:53.134741    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-bdp25"
Sep 12 00:16:53.135067 ip-10-0-141-128 kubenswrapper[2571]: E0912 00:16:53.134967    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-nxrg5" podUID="09b83899-89b7-4f4a-b485-c102295f188e"
Sep 12 00:16:53.135273 ip-10-0-141-128 kubenswrapper[2571]: E0912 00:16:53.135115    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-bdp25" podUID="6d479776-cc30-4bea-bb3a-8b4c259d59c7"
Sep 12 00:16:53.352924 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:16:53.352890    2571 generic.go:356] "Generic (PLEG): container finished" podID="0a3e3c8c-cafa-4254-a972-fcf494ae4e28" containerID="eeeb433a1f0ab518d72a69c2e3dc42b4e0e253f94c8359ef3422cc0f2b38a185" exitCode=0
Sep 12 00:16:53.353474 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:16:53.352973    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-62v9r" event={"ID":"0a3e3c8c-cafa-4254-a972-fcf494ae4e28","Type":"ContainerDied","Data":"eeeb433a1f0ab518d72a69c2e3dc42b4e0e253f94c8359ef3422cc0f2b38a185"}
Sep 12 00:16:54.370538 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:16:54.370495    2571 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ovn-kubernetes/ovnkube-node-j6bgr" podUID="40698f9e-0e26-4d86-ac05-ffcd30e172d4" containerName="ovnkube-controller" probeResult="failure" output=""
Sep 12 00:16:55.135096 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:16:55.135059    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-b6792"
Sep 12 00:16:55.135096 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:16:55.135076    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-nxrg5"
Sep 12 00:16:55.135321 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:16:55.135059    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-bdp25"
Sep 12 00:16:55.135321 ip-10-0-141-128 kubenswrapper[2571]: E0912 00:16:55.135182    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-b6792" podUID="dd7e1ff1-8f0d-4f40-9a32-2a6cd30cb239"
Sep 12 00:16:55.135321 ip-10-0-141-128 kubenswrapper[2571]: E0912 00:16:55.135249    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-nxrg5" podUID="09b83899-89b7-4f4a-b485-c102295f188e"
Sep 12 00:16:55.135470 ip-10-0-141-128 kubenswrapper[2571]: E0912 00:16:55.135353    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-bdp25" podUID="6d479776-cc30-4bea-bb3a-8b4c259d59c7"
Sep 12 00:16:57.134912 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:16:57.134775    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-b6792"
Sep 12 00:16:57.135292 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:16:57.134775    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-nxrg5"
Sep 12 00:16:57.135292 ip-10-0-141-128 kubenswrapper[2571]: E0912 00:16:57.134979    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-b6792" podUID="dd7e1ff1-8f0d-4f40-9a32-2a6cd30cb239"
Sep 12 00:16:57.135292 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:16:57.134794    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-bdp25"
Sep 12 00:16:57.135292 ip-10-0-141-128 kubenswrapper[2571]: E0912 00:16:57.135044    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-nxrg5" podUID="09b83899-89b7-4f4a-b485-c102295f188e"
Sep 12 00:16:57.135292 ip-10-0-141-128 kubenswrapper[2571]: E0912 00:16:57.135150    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-bdp25" podUID="6d479776-cc30-4bea-bb3a-8b4c259d59c7"
Sep 12 00:16:57.365909 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:16:57.365809    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-57l7n" event={"ID":"76f830f1-1f3e-4dd4-9deb-023f6ad1e2e4","Type":"ContainerStarted","Data":"11bfe336e21b072269ba20ca2ecd632bdf032165bfb854e1b6ceefb2979f7351"}
Sep 12 00:16:57.382994 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:16:57.382946    2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-57l7n" podStartSLOduration=0.777744026 podStartE2EDuration="27.382927204s" totalImagesPullingTime="26.605183178s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-12 00:16:30 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=1 observedRunningTime="2026-09-12 00:16:57.38237725 +0000 UTC m=+28.654017762" watchObservedRunningTime="2026-09-12 00:16:57.382927204 +0000 UTC m=+28.654567728"
Sep 12 00:16:58.047034 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:16:58.047007    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-r92pg"
Sep 12 00:16:58.047124 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:16:58.047112    2571 prober_manager.go:312] "Failed to trigger a manual run" probe="Readiness"
Sep 12 00:16:58.047563 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:16:58.047546    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-r92pg"
Sep 12 00:16:58.369656 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:16:58.369577    2571 generic.go:356] "Generic (PLEG): container finished" podID="0a3e3c8c-cafa-4254-a972-fcf494ae4e28" containerID="7dad3bcd27925ed18ffcebf59b26311cae841d2ec57de428e37945dd205f17ac" exitCode=0
Sep 12 00:16:58.369656 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:16:58.369616    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-62v9r" event={"ID":"0a3e3c8c-cafa-4254-a972-fcf494ae4e28","Type":"ContainerDied","Data":"7dad3bcd27925ed18ffcebf59b26311cae841d2ec57de428e37945dd205f17ac"}
Sep 12 00:16:59.030819 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:16:59.030791    2571 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-128.ec2.internal" event="NodeReady"
Sep 12 00:16:59.030962 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:16:59.030875    2571 kubelet_node_status.go:489] "Fast updating node status as it just became ready"
Sep 12 00:16:59.062595 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:16:59.062568    2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-744fcccbf-5vhvb"]
Sep 12 00:16:59.105810 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:16:59.105786    2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-fr2kt"]
Sep 12 00:16:59.105940 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:16:59.105914    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-744fcccbf-5vhvb"
Sep 12 00:16:59.108265 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:16:59.108243    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\""
Sep 12 00:16:59.108517 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:16:59.108499    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\""
Sep 12 00:16:59.108624 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:16:59.108499    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\""
Sep 12 00:16:59.108681 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:16:59.108640    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-tp2x6\""
Sep 12 00:16:59.114211 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:16:59.114190    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\""
Sep 12 00:16:59.115496 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:16:59.115480    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-744fcccbf-5vhvb"]
Sep 12 00:16:59.115568 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:16:59.115508    2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-rlr5m"]
Sep 12 00:16:59.115621 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:16:59.115604    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-fr2kt"
Sep 12 00:16:59.117927 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:16:59.117870    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\""
Sep 12 00:16:59.117927 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:16:59.117884    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-knhwr\""
Sep 12 00:16:59.117927 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:16:59.117887    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\""
Sep 12 00:16:59.123804 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:16:59.123786    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-fr2kt"]
Sep 12 00:16:59.123905 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:16:59.123816    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-rlr5m"]
Sep 12 00:16:59.123905 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:16:59.123898    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-rlr5m"
Sep 12 00:16:59.126337 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:16:59.126315    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\""
Sep 12 00:16:59.126440 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:16:59.126385    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\""
Sep 12 00:16:59.126440 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:16:59.126392    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-ns4k9\""
Sep 12 00:16:59.126549 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:16:59.126471    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\""
Sep 12 00:16:59.135115 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:16:59.135089    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-nxrg5"
Sep 12 00:16:59.135200 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:16:59.135173    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-b6792"
Sep 12 00:16:59.135256 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:16:59.135099    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-bdp25"
Sep 12 00:16:59.137221 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:16:59.137203    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Sep 12 00:16:59.137474 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:16:59.137446    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Sep 12 00:16:59.137690 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:16:59.137664    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-tdrkx\""
Sep 12 00:16:59.137772 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:16:59.137668    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Sep 12 00:16:59.137772 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:16:59.137732    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-vv7j9\""
Sep 12 00:16:59.138008 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:16:59.137991    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Sep 12 00:16:59.199946 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:16:59.199924    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-z67b4\" (UniqueName: \"kubernetes.io/projected/80cbfa96-879b-48ea-8732-d50ebebea182-kube-api-access-z67b4\") pod \"image-registry-744fcccbf-5vhvb\" (UID: \"80cbfa96-879b-48ea-8732-d50ebebea182\") " pod="openshift-image-registry/image-registry-744fcccbf-5vhvb"
Sep 12 00:16:59.200050 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:16:59.199955    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/80cbfa96-879b-48ea-8732-d50ebebea182-registry-tls\") pod \"image-registry-744fcccbf-5vhvb\" (UID: \"80cbfa96-879b-48ea-8732-d50ebebea182\") " pod="openshift-image-registry/image-registry-744fcccbf-5vhvb"
Sep 12 00:16:59.200050 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:16:59.199983    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/3a2114c7-d34a-4249-97a4-65f5375453ce-config-volume\") pod \"dns-default-fr2kt\" (UID: \"3a2114c7-d34a-4249-97a4-65f5375453ce\") " pod="openshift-dns/dns-default-fr2kt"
Sep 12 00:16:59.200200 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:16:59.200060    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/80cbfa96-879b-48ea-8732-d50ebebea182-registry-certificates\") pod \"image-registry-744fcccbf-5vhvb\" (UID: \"80cbfa96-879b-48ea-8732-d50ebebea182\") " pod="openshift-image-registry/image-registry-744fcccbf-5vhvb"
Sep 12 00:16:59.200200 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:16:59.200082    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/80cbfa96-879b-48ea-8732-d50ebebea182-bound-sa-token\") pod \"image-registry-744fcccbf-5vhvb\" (UID: \"80cbfa96-879b-48ea-8732-d50ebebea182\") " pod="openshift-image-registry/image-registry-744fcccbf-5vhvb"
Sep 12 00:16:59.200200 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:16:59.200102    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/80cbfa96-879b-48ea-8732-d50ebebea182-installation-pull-secrets\") pod \"image-registry-744fcccbf-5vhvb\" (UID: \"80cbfa96-879b-48ea-8732-d50ebebea182\") " pod="openshift-image-registry/image-registry-744fcccbf-5vhvb"
Sep 12 00:16:59.200200 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:16:59.200127    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/80cbfa96-879b-48ea-8732-d50ebebea182-image-registry-private-configuration\") pod \"image-registry-744fcccbf-5vhvb\" (UID: \"80cbfa96-879b-48ea-8732-d50ebebea182\") " pod="openshift-image-registry/image-registry-744fcccbf-5vhvb"
Sep 12 00:16:59.200200 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:16:59.200149    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/80cbfa96-879b-48ea-8732-d50ebebea182-trusted-ca\") pod \"image-registry-744fcccbf-5vhvb\" (UID: \"80cbfa96-879b-48ea-8732-d50ebebea182\") " pod="openshift-image-registry/image-registry-744fcccbf-5vhvb"
Sep 12 00:16:59.200200 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:16:59.200176    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wjgvv\" (UniqueName: \"kubernetes.io/projected/ba88d840-64fd-419f-8444-ebde7a94c354-kube-api-access-wjgvv\") pod \"ingress-canary-rlr5m\" (UID: \"ba88d840-64fd-419f-8444-ebde7a94c354\") " pod="openshift-ingress-canary/ingress-canary-rlr5m"
Sep 12 00:16:59.200391 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:16:59.200217    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/3a2114c7-d34a-4249-97a4-65f5375453ce-metrics-tls\") pod \"dns-default-fr2kt\" (UID: \"3a2114c7-d34a-4249-97a4-65f5375453ce\") " pod="openshift-dns/dns-default-fr2kt"
Sep 12 00:16:59.200391 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:16:59.200235    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3a2114c7-d34a-4249-97a4-65f5375453ce-tmp-dir\") pod \"dns-default-fr2kt\" (UID: \"3a2114c7-d34a-4249-97a4-65f5375453ce\") " pod="openshift-dns/dns-default-fr2kt"
Sep 12 00:16:59.200391 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:16:59.200253    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/80cbfa96-879b-48ea-8732-d50ebebea182-ca-trust-extracted\") pod \"image-registry-744fcccbf-5vhvb\" (UID: \"80cbfa96-879b-48ea-8732-d50ebebea182\") " pod="openshift-image-registry/image-registry-744fcccbf-5vhvb"
Sep 12 00:16:59.200391 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:16:59.200270    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/ba88d840-64fd-419f-8444-ebde7a94c354-cert\") pod \"ingress-canary-rlr5m\" (UID: \"ba88d840-64fd-419f-8444-ebde7a94c354\") " pod="openshift-ingress-canary/ingress-canary-rlr5m"
Sep 12 00:16:59.200391 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:16:59.200287    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mnv4w\" (UniqueName: \"kubernetes.io/projected/3a2114c7-d34a-4249-97a4-65f5375453ce-kube-api-access-mnv4w\") pod \"dns-default-fr2kt\" (UID: \"3a2114c7-d34a-4249-97a4-65f5375453ce\") " pod="openshift-dns/dns-default-fr2kt"
Sep 12 00:16:59.207337 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:16:59.207315    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-r92pg_71d215ed-e9d2-4a8b-bc73-f0ee2e0d83c5/konnectivity-agent/0.log" limit=1024
Sep 12 00:16:59.300594 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:16:59.300516    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/3a2114c7-d34a-4249-97a4-65f5375453ce-metrics-tls\") pod \"dns-default-fr2kt\" (UID: \"3a2114c7-d34a-4249-97a4-65f5375453ce\") " pod="openshift-dns/dns-default-fr2kt"
Sep 12 00:16:59.300594 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:16:59.300553    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3a2114c7-d34a-4249-97a4-65f5375453ce-tmp-dir\") pod \"dns-default-fr2kt\" (UID: \"3a2114c7-d34a-4249-97a4-65f5375453ce\") " pod="openshift-dns/dns-default-fr2kt"
Sep 12 00:16:59.300782 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:16:59.300582    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/80cbfa96-879b-48ea-8732-d50ebebea182-ca-trust-extracted\") pod \"image-registry-744fcccbf-5vhvb\" (UID: \"80cbfa96-879b-48ea-8732-d50ebebea182\") " pod="openshift-image-registry/image-registry-744fcccbf-5vhvb"
Sep 12 00:16:59.300782 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:16:59.300628    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/ba88d840-64fd-419f-8444-ebde7a94c354-cert\") pod \"ingress-canary-rlr5m\" (UID: \"ba88d840-64fd-419f-8444-ebde7a94c354\") " pod="openshift-ingress-canary/ingress-canary-rlr5m"
Sep 12 00:16:59.300782 ip-10-0-141-128 kubenswrapper[2571]: E0912 00:16:59.300645    2571 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 12 00:16:59.300782 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:16:59.300654    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mnv4w\" (UniqueName: \"kubernetes.io/projected/3a2114c7-d34a-4249-97a4-65f5375453ce-kube-api-access-mnv4w\") pod \"dns-default-fr2kt\" (UID: \"3a2114c7-d34a-4249-97a4-65f5375453ce\") " pod="openshift-dns/dns-default-fr2kt"
Sep 12 00:16:59.300782 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:16:59.300681    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-z67b4\" (UniqueName: \"kubernetes.io/projected/80cbfa96-879b-48ea-8732-d50ebebea182-kube-api-access-z67b4\") pod \"image-registry-744fcccbf-5vhvb\" (UID: \"80cbfa96-879b-48ea-8732-d50ebebea182\") " pod="openshift-image-registry/image-registry-744fcccbf-5vhvb"
Sep 12 00:16:59.300782 ip-10-0-141-128 kubenswrapper[2571]: E0912 00:16:59.300713    2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3a2114c7-d34a-4249-97a4-65f5375453ce-metrics-tls podName:3a2114c7-d34a-4249-97a4-65f5375453ce nodeName:}" failed. No retries permitted until 2026-09-12 00:16:59.80068929 +0000 UTC m=+31.072329791 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/3a2114c7-d34a-4249-97a4-65f5375453ce-metrics-tls") pod "dns-default-fr2kt" (UID: "3a2114c7-d34a-4249-97a4-65f5375453ce") : secret "dns-default-metrics-tls" not found
Sep 12 00:16:59.300782 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:16:59.300770    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/80cbfa96-879b-48ea-8732-d50ebebea182-registry-tls\") pod \"image-registry-744fcccbf-5vhvb\" (UID: \"80cbfa96-879b-48ea-8732-d50ebebea182\") " pod="openshift-image-registry/image-registry-744fcccbf-5vhvb"
Sep 12 00:16:59.301123 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:16:59.300806    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/3a2114c7-d34a-4249-97a4-65f5375453ce-config-volume\") pod \"dns-default-fr2kt\" (UID: \"3a2114c7-d34a-4249-97a4-65f5375453ce\") " pod="openshift-dns/dns-default-fr2kt"
Sep 12 00:16:59.301123 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:16:59.300866    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/80cbfa96-879b-48ea-8732-d50ebebea182-registry-certificates\") pod \"image-registry-744fcccbf-5vhvb\" (UID: \"80cbfa96-879b-48ea-8732-d50ebebea182\") " pod="openshift-image-registry/image-registry-744fcccbf-5vhvb"
Sep 12 00:16:59.301123 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:16:59.300894    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/80cbfa96-879b-48ea-8732-d50ebebea182-bound-sa-token\") pod \"image-registry-744fcccbf-5vhvb\" (UID: \"80cbfa96-879b-48ea-8732-d50ebebea182\") " pod="openshift-image-registry/image-registry-744fcccbf-5vhvb"
Sep 12 00:16:59.301123 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:16:59.300925    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/80cbfa96-879b-48ea-8732-d50ebebea182-installation-pull-secrets\") pod \"image-registry-744fcccbf-5vhvb\" (UID: \"80cbfa96-879b-48ea-8732-d50ebebea182\") " pod="openshift-image-registry/image-registry-744fcccbf-5vhvb"
Sep 12 00:16:59.301123 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:16:59.300954    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3a2114c7-d34a-4249-97a4-65f5375453ce-tmp-dir\") pod \"dns-default-fr2kt\" (UID: \"3a2114c7-d34a-4249-97a4-65f5375453ce\") " pod="openshift-dns/dns-default-fr2kt"
Sep 12 00:16:59.301123 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:16:59.300967    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/80cbfa96-879b-48ea-8732-d50ebebea182-ca-trust-extracted\") pod \"image-registry-744fcccbf-5vhvb\" (UID: \"80cbfa96-879b-48ea-8732-d50ebebea182\") " pod="openshift-image-registry/image-registry-744fcccbf-5vhvb"
Sep 12 00:16:59.301401 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:16:59.301183    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/80cbfa96-879b-48ea-8732-d50ebebea182-image-registry-private-configuration\") pod \"image-registry-744fcccbf-5vhvb\" (UID: \"80cbfa96-879b-48ea-8732-d50ebebea182\") " pod="openshift-image-registry/image-registry-744fcccbf-5vhvb"
Sep 12 00:16:59.301401 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:16:59.301227    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/80cbfa96-879b-48ea-8732-d50ebebea182-trusted-ca\") pod \"image-registry-744fcccbf-5vhvb\" (UID: \"80cbfa96-879b-48ea-8732-d50ebebea182\") " pod="openshift-image-registry/image-registry-744fcccbf-5vhvb"
Sep 12 00:16:59.301401 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:16:59.301252    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wjgvv\" (UniqueName: \"kubernetes.io/projected/ba88d840-64fd-419f-8444-ebde7a94c354-kube-api-access-wjgvv\") pod \"ingress-canary-rlr5m\" (UID: \"ba88d840-64fd-419f-8444-ebde7a94c354\") " pod="openshift-ingress-canary/ingress-canary-rlr5m"
Sep 12 00:16:59.307528 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:16:59.303456    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/80cbfa96-879b-48ea-8732-d50ebebea182-registry-certificates\") pod \"image-registry-744fcccbf-5vhvb\" (UID: \"80cbfa96-879b-48ea-8732-d50ebebea182\") " pod="openshift-image-registry/image-registry-744fcccbf-5vhvb"
Sep 12 00:16:59.307528 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:16:59.303703    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/3a2114c7-d34a-4249-97a4-65f5375453ce-config-volume\") pod \"dns-default-fr2kt\" (UID: \"3a2114c7-d34a-4249-97a4-65f5375453ce\") " pod="openshift-dns/dns-default-fr2kt"
Sep 12 00:16:59.307528 ip-10-0-141-128 kubenswrapper[2571]: E0912 00:16:59.304979    2571 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Sep 12 00:16:59.307528 ip-10-0-141-128 kubenswrapper[2571]: E0912 00:16:59.305003    2571 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-744fcccbf-5vhvb: secret "image-registry-tls" not found
Sep 12 00:16:59.307528 ip-10-0-141-128 kubenswrapper[2571]: E0912 00:16:59.305562    2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/80cbfa96-879b-48ea-8732-d50ebebea182-registry-tls podName:80cbfa96-879b-48ea-8732-d50ebebea182 nodeName:}" failed. No retries permitted until 2026-09-12 00:16:59.805541858 +0000 UTC m=+31.077182348 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/80cbfa96-879b-48ea-8732-d50ebebea182-registry-tls") pod "image-registry-744fcccbf-5vhvb" (UID: "80cbfa96-879b-48ea-8732-d50ebebea182") : secret "image-registry-tls" not found
Sep 12 00:16:59.307528 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:16:59.305806    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/ba88d840-64fd-419f-8444-ebde7a94c354-cert\") pod \"ingress-canary-rlr5m\" (UID: \"ba88d840-64fd-419f-8444-ebde7a94c354\") " pod="openshift-ingress-canary/ingress-canary-rlr5m"
Sep 12 00:16:59.307528 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:16:59.306428    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/80cbfa96-879b-48ea-8732-d50ebebea182-installation-pull-secrets\") pod \"image-registry-744fcccbf-5vhvb\" (UID: \"80cbfa96-879b-48ea-8732-d50ebebea182\") " pod="openshift-image-registry/image-registry-744fcccbf-5vhvb"
Sep 12 00:16:59.307528 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:16:59.306619    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/80cbfa96-879b-48ea-8732-d50ebebea182-image-registry-private-configuration\") pod \"image-registry-744fcccbf-5vhvb\" (UID: \"80cbfa96-879b-48ea-8732-d50ebebea182\") " pod="openshift-image-registry/image-registry-744fcccbf-5vhvb"
Sep 12 00:16:59.307528 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:16:59.306734    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/80cbfa96-879b-48ea-8732-d50ebebea182-trusted-ca\") pod \"image-registry-744fcccbf-5vhvb\" (UID: \"80cbfa96-879b-48ea-8732-d50ebebea182\") " pod="openshift-image-registry/image-registry-744fcccbf-5vhvb"
Sep 12 00:16:59.309499 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:16:59.309330    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-r92pg_71d215ed-e9d2-4a8b-bc73-f0ee2e0d83c5/konnectivity-agent/0.log" limit=1024
Sep 12 00:16:59.312760 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:16:59.312710    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-z67b4\" (UniqueName: \"kubernetes.io/projected/80cbfa96-879b-48ea-8732-d50ebebea182-kube-api-access-z67b4\") pod \"image-registry-744fcccbf-5vhvb\" (UID: \"80cbfa96-879b-48ea-8732-d50ebebea182\") " pod="openshift-image-registry/image-registry-744fcccbf-5vhvb"
Sep 12 00:16:59.317307 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:16:59.317283    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/80cbfa96-879b-48ea-8732-d50ebebea182-bound-sa-token\") pod \"image-registry-744fcccbf-5vhvb\" (UID: \"80cbfa96-879b-48ea-8732-d50ebebea182\") " pod="openshift-image-registry/image-registry-744fcccbf-5vhvb"
Sep 12 00:16:59.318137 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:16:59.318115    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mnv4w\" (UniqueName: \"kubernetes.io/projected/3a2114c7-d34a-4249-97a4-65f5375453ce-kube-api-access-mnv4w\") pod \"dns-default-fr2kt\" (UID: \"3a2114c7-d34a-4249-97a4-65f5375453ce\") " pod="openshift-dns/dns-default-fr2kt"
Sep 12 00:16:59.318643 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:16:59.318619    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wjgvv\" (UniqueName: \"kubernetes.io/projected/ba88d840-64fd-419f-8444-ebde7a94c354-kube-api-access-wjgvv\") pod \"ingress-canary-rlr5m\" (UID: \"ba88d840-64fd-419f-8444-ebde7a94c354\") " pod="openshift-ingress-canary/ingress-canary-rlr5m"
Sep 12 00:16:59.434047 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:16:59.434021    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-rlr5m"
Sep 12 00:16:59.741568 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:16:59.741427    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-rlr5m"]
Sep 12 00:16:59.744376 ip-10-0-141-128 kubenswrapper[2571]: W0912 00:16:59.744348    2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podba88d840_64fd_419f_8444_ebde7a94c354.slice/crio-9a7cb6cf8aa463cb4ecd005da2c3386669ffce325e05043b9fca2f1c3a9e91b5 WatchSource:0}: Error finding container 9a7cb6cf8aa463cb4ecd005da2c3386669ffce325e05043b9fca2f1c3a9e91b5: Status 404 returned error can't find the container with id 9a7cb6cf8aa463cb4ecd005da2c3386669ffce325e05043b9fca2f1c3a9e91b5
Sep 12 00:16:59.808040 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:16:59.808014    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/3a2114c7-d34a-4249-97a4-65f5375453ce-metrics-tls\") pod \"dns-default-fr2kt\" (UID: \"3a2114c7-d34a-4249-97a4-65f5375453ce\") " pod="openshift-dns/dns-default-fr2kt"
Sep 12 00:16:59.808144 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:16:59.808066    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/80cbfa96-879b-48ea-8732-d50ebebea182-registry-tls\") pod \"image-registry-744fcccbf-5vhvb\" (UID: \"80cbfa96-879b-48ea-8732-d50ebebea182\") " pod="openshift-image-registry/image-registry-744fcccbf-5vhvb"
Sep 12 00:16:59.808197 ip-10-0-141-128 kubenswrapper[2571]: E0912 00:16:59.808156    2571 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 12 00:16:59.808235 ip-10-0-141-128 kubenswrapper[2571]: E0912 00:16:59.808214    2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3a2114c7-d34a-4249-97a4-65f5375453ce-metrics-tls podName:3a2114c7-d34a-4249-97a4-65f5375453ce nodeName:}" failed. No retries permitted until 2026-09-12 00:17:00.808197707 +0000 UTC m=+32.079838207 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/3a2114c7-d34a-4249-97a4-65f5375453ce-metrics-tls") pod "dns-default-fr2kt" (UID: "3a2114c7-d34a-4249-97a4-65f5375453ce") : secret "dns-default-metrics-tls" not found
Sep 12 00:16:59.808283 ip-10-0-141-128 kubenswrapper[2571]: E0912 00:16:59.808159    2571 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Sep 12 00:16:59.808283 ip-10-0-141-128 kubenswrapper[2571]: E0912 00:16:59.808257    2571 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-744fcccbf-5vhvb: secret "image-registry-tls" not found
Sep 12 00:16:59.808344 ip-10-0-141-128 kubenswrapper[2571]: E0912 00:16:59.808299    2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/80cbfa96-879b-48ea-8732-d50ebebea182-registry-tls podName:80cbfa96-879b-48ea-8732-d50ebebea182 nodeName:}" failed. No retries permitted until 2026-09-12 00:17:00.808287505 +0000 UTC m=+32.079927997 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/80cbfa96-879b-48ea-8732-d50ebebea182-registry-tls") pod "image-registry-744fcccbf-5vhvb" (UID: "80cbfa96-879b-48ea-8732-d50ebebea182") : secret "image-registry-tls" not found
Sep 12 00:17:00.376020 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:17:00.375990    2571 generic.go:356] "Generic (PLEG): container finished" podID="0a3e3c8c-cafa-4254-a972-fcf494ae4e28" containerID="af20819d7304131b44280a40a36483f23d6650b60a082628437db651a0b5ccd1" exitCode=0
Sep 12 00:17:00.376194 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:17:00.376059    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-62v9r" event={"ID":"0a3e3c8c-cafa-4254-a972-fcf494ae4e28","Type":"ContainerDied","Data":"af20819d7304131b44280a40a36483f23d6650b60a082628437db651a0b5ccd1"}
Sep 12 00:17:00.377258 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:17:00.377233    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-rlr5m" event={"ID":"ba88d840-64fd-419f-8444-ebde7a94c354","Type":"ContainerStarted","Data":"9a7cb6cf8aa463cb4ecd005da2c3386669ffce325e05043b9fca2f1c3a9e91b5"}
Sep 12 00:17:00.816233 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:17:00.816146    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/3a2114c7-d34a-4249-97a4-65f5375453ce-metrics-tls\") pod \"dns-default-fr2kt\" (UID: \"3a2114c7-d34a-4249-97a4-65f5375453ce\") " pod="openshift-dns/dns-default-fr2kt"
Sep 12 00:17:00.816233 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:17:00.816209    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/80cbfa96-879b-48ea-8732-d50ebebea182-registry-tls\") pod \"image-registry-744fcccbf-5vhvb\" (UID: \"80cbfa96-879b-48ea-8732-d50ebebea182\") " pod="openshift-image-registry/image-registry-744fcccbf-5vhvb"
Sep 12 00:17:00.817190 ip-10-0-141-128 kubenswrapper[2571]: E0912 00:17:00.816324    2571 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Sep 12 00:17:00.817190 ip-10-0-141-128 kubenswrapper[2571]: E0912 00:17:00.816340    2571 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-744fcccbf-5vhvb: secret "image-registry-tls" not found
Sep 12 00:17:00.817190 ip-10-0-141-128 kubenswrapper[2571]: E0912 00:17:00.816347    2571 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 12 00:17:00.817190 ip-10-0-141-128 kubenswrapper[2571]: E0912 00:17:00.816400    2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/80cbfa96-879b-48ea-8732-d50ebebea182-registry-tls podName:80cbfa96-879b-48ea-8732-d50ebebea182 nodeName:}" failed. No retries permitted until 2026-09-12 00:17:02.816383965 +0000 UTC m=+34.088024466 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/80cbfa96-879b-48ea-8732-d50ebebea182-registry-tls") pod "image-registry-744fcccbf-5vhvb" (UID: "80cbfa96-879b-48ea-8732-d50ebebea182") : secret "image-registry-tls" not found
Sep 12 00:17:00.817190 ip-10-0-141-128 kubenswrapper[2571]: E0912 00:17:00.816418    2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3a2114c7-d34a-4249-97a4-65f5375453ce-metrics-tls podName:3a2114c7-d34a-4249-97a4-65f5375453ce nodeName:}" failed. No retries permitted until 2026-09-12 00:17:02.816410991 +0000 UTC m=+34.088051482 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/3a2114c7-d34a-4249-97a4-65f5375453ce-metrics-tls") pod "dns-default-fr2kt" (UID: "3a2114c7-d34a-4249-97a4-65f5375453ce") : secret "dns-default-metrics-tls" not found
Sep 12 00:17:01.924455 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:17:01.924413    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/09b83899-89b7-4f4a-b485-c102295f188e-original-pull-secret\") pod \"global-pull-secret-syncer-nxrg5\" (UID: \"09b83899-89b7-4f4a-b485-c102295f188e\") " pod="kube-system/global-pull-secret-syncer-nxrg5"
Sep 12 00:17:01.924964 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:17:01.924534    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/6d479776-cc30-4bea-bb3a-8b4c259d59c7-metrics-certs\") pod \"network-metrics-daemon-bdp25\" (UID: \"6d479776-cc30-4bea-bb3a-8b4c259d59c7\") " pod="openshift-multus/network-metrics-daemon-bdp25"
Sep 12 00:17:01.924964 ip-10-0-141-128 kubenswrapper[2571]: E0912 00:17:01.924655    2571 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found
Sep 12 00:17:01.924964 ip-10-0-141-128 kubenswrapper[2571]: E0912 00:17:01.924720    2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6d479776-cc30-4bea-bb3a-8b4c259d59c7-metrics-certs podName:6d479776-cc30-4bea-bb3a-8b4c259d59c7 nodeName:}" failed. No retries permitted until 2026-09-12 00:17:33.92470206 +0000 UTC m=+65.196342564 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/6d479776-cc30-4bea-bb3a-8b4c259d59c7-metrics-certs") pod "network-metrics-daemon-bdp25" (UID: "6d479776-cc30-4bea-bb3a-8b4c259d59c7") : secret "metrics-daemon-secret" not found
Sep 12 00:17:01.928107 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:17:01.928078    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/09b83899-89b7-4f4a-b485-c102295f188e-original-pull-secret\") pod \"global-pull-secret-syncer-nxrg5\" (UID: \"09b83899-89b7-4f4a-b485-c102295f188e\") " pod="kube-system/global-pull-secret-syncer-nxrg5"
Sep 12 00:17:02.025555 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:17:02.025483    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7h4vm\" (UniqueName: \"kubernetes.io/projected/dd7e1ff1-8f0d-4f40-9a32-2a6cd30cb239-kube-api-access-7h4vm\") pod \"network-check-target-b6792\" (UID: \"dd7e1ff1-8f0d-4f40-9a32-2a6cd30cb239\") " pod="openshift-network-diagnostics/network-check-target-b6792"
Sep 12 00:17:02.028044 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:17:02.028020    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7h4vm\" (UniqueName: \"kubernetes.io/projected/dd7e1ff1-8f0d-4f40-9a32-2a6cd30cb239-kube-api-access-7h4vm\") pod \"network-check-target-b6792\" (UID: \"dd7e1ff1-8f0d-4f40-9a32-2a6cd30cb239\") " pod="openshift-network-diagnostics/network-check-target-b6792"
Sep 12 00:17:02.147393 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:17:02.147164    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-nxrg5"
Sep 12 00:17:02.155635 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:17:02.155608    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-b6792"
Sep 12 00:17:02.833255 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:17:02.833048    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/80cbfa96-879b-48ea-8732-d50ebebea182-registry-tls\") pod \"image-registry-744fcccbf-5vhvb\" (UID: \"80cbfa96-879b-48ea-8732-d50ebebea182\") " pod="openshift-image-registry/image-registry-744fcccbf-5vhvb"
Sep 12 00:17:02.833255 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:17:02.833142    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/3a2114c7-d34a-4249-97a4-65f5375453ce-metrics-tls\") pod \"dns-default-fr2kt\" (UID: \"3a2114c7-d34a-4249-97a4-65f5375453ce\") " pod="openshift-dns/dns-default-fr2kt"
Sep 12 00:17:02.833375 ip-10-0-141-128 kubenswrapper[2571]: E0912 00:17:02.833273    2571 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 12 00:17:02.833375 ip-10-0-141-128 kubenswrapper[2571]: E0912 00:17:02.833316    2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3a2114c7-d34a-4249-97a4-65f5375453ce-metrics-tls podName:3a2114c7-d34a-4249-97a4-65f5375453ce nodeName:}" failed. No retries permitted until 2026-09-12 00:17:06.833304327 +0000 UTC m=+38.104944814 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/3a2114c7-d34a-4249-97a4-65f5375453ce-metrics-tls") pod "dns-default-fr2kt" (UID: "3a2114c7-d34a-4249-97a4-65f5375453ce") : secret "dns-default-metrics-tls" not found
Sep 12 00:17:02.833375 ip-10-0-141-128 kubenswrapper[2571]: E0912 00:17:02.833359    2571 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Sep 12 00:17:02.833375 ip-10-0-141-128 kubenswrapper[2571]: E0912 00:17:02.833366    2571 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-744fcccbf-5vhvb: secret "image-registry-tls" not found
Sep 12 00:17:02.833574 ip-10-0-141-128 kubenswrapper[2571]: E0912 00:17:02.833387    2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/80cbfa96-879b-48ea-8732-d50ebebea182-registry-tls podName:80cbfa96-879b-48ea-8732-d50ebebea182 nodeName:}" failed. No retries permitted until 2026-09-12 00:17:06.833380096 +0000 UTC m=+38.105020582 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/80cbfa96-879b-48ea-8732-d50ebebea182-registry-tls") pod "image-registry-744fcccbf-5vhvb" (UID: "80cbfa96-879b-48ea-8732-d50ebebea182") : secret "image-registry-tls" not found
Sep 12 00:17:03.049226 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:17:03.049151    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-nxrg5"]
Sep 12 00:17:03.053488 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:17:03.053386    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-b6792"]
Sep 12 00:17:03.057627 ip-10-0-141-128 kubenswrapper[2571]: W0912 00:17:03.057573    2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddd7e1ff1_8f0d_4f40_9a32_2a6cd30cb239.slice/crio-2cd90b72330685df9aacdc4015f6e73ee81249a205a0065fa042dd35d4c6f566 WatchSource:0}: Error finding container 2cd90b72330685df9aacdc4015f6e73ee81249a205a0065fa042dd35d4c6f566: Status 404 returned error can't find the container with id 2cd90b72330685df9aacdc4015f6e73ee81249a205a0065fa042dd35d4c6f566
Sep 12 00:17:03.386980 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:17:03.386894    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-rlr5m" event={"ID":"ba88d840-64fd-419f-8444-ebde7a94c354","Type":"ContainerStarted","Data":"450ed7781767f4cc84d1dce14df9c7401a3619c14b62c9c143afd04e96ee4086"}
Sep 12 00:17:03.388172 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:17:03.388149    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-nxrg5" event={"ID":"09b83899-89b7-4f4a-b485-c102295f188e","Type":"ContainerStarted","Data":"975fa44cc8cfc0f765422dc25724a9c75cf8bbdf154bf235e9505096aa0258e8"}
Sep 12 00:17:03.389348 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:17:03.389326    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-b6792" event={"ID":"dd7e1ff1-8f0d-4f40-9a32-2a6cd30cb239","Type":"ContainerStarted","Data":"2cd90b72330685df9aacdc4015f6e73ee81249a205a0065fa042dd35d4c6f566"}
Sep 12 00:17:03.401088 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:17:03.401043    2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-rlr5m" podStartSLOduration=1.314495078 podStartE2EDuration="4.401030761s" totalImagesPullingTime="3.086535683s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-12 00:16:59 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-12 00:17:03.400896752 +0000 UTC m=+34.672537252" watchObservedRunningTime="2026-09-12 00:17:03.401030761 +0000 UTC m=+34.672671251"
Sep 12 00:17:06.866527 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:17:06.866480    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/80cbfa96-879b-48ea-8732-d50ebebea182-registry-tls\") pod \"image-registry-744fcccbf-5vhvb\" (UID: \"80cbfa96-879b-48ea-8732-d50ebebea182\") " pod="openshift-image-registry/image-registry-744fcccbf-5vhvb"
Sep 12 00:17:06.866967 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:17:06.866570    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/3a2114c7-d34a-4249-97a4-65f5375453ce-metrics-tls\") pod \"dns-default-fr2kt\" (UID: \"3a2114c7-d34a-4249-97a4-65f5375453ce\") " pod="openshift-dns/dns-default-fr2kt"
Sep 12 00:17:06.866967 ip-10-0-141-128 kubenswrapper[2571]: E0912 00:17:06.866666    2571 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Sep 12 00:17:06.866967 ip-10-0-141-128 kubenswrapper[2571]: E0912 00:17:06.866685    2571 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-744fcccbf-5vhvb: secret "image-registry-tls" not found
Sep 12 00:17:06.866967 ip-10-0-141-128 kubenswrapper[2571]: E0912 00:17:06.866704    2571 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 12 00:17:06.866967 ip-10-0-141-128 kubenswrapper[2571]: E0912 00:17:06.866755    2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/80cbfa96-879b-48ea-8732-d50ebebea182-registry-tls podName:80cbfa96-879b-48ea-8732-d50ebebea182 nodeName:}" failed. No retries permitted until 2026-09-12 00:17:14.866734484 +0000 UTC m=+46.138374972 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/80cbfa96-879b-48ea-8732-d50ebebea182-registry-tls") pod "image-registry-744fcccbf-5vhvb" (UID: "80cbfa96-879b-48ea-8732-d50ebebea182") : secret "image-registry-tls" not found
Sep 12 00:17:06.866967 ip-10-0-141-128 kubenswrapper[2571]: E0912 00:17:06.866795    2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3a2114c7-d34a-4249-97a4-65f5375453ce-metrics-tls podName:3a2114c7-d34a-4249-97a4-65f5375453ce nodeName:}" failed. No retries permitted until 2026-09-12 00:17:14.866775701 +0000 UTC m=+46.138416192 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/3a2114c7-d34a-4249-97a4-65f5375453ce-metrics-tls") pod "dns-default-fr2kt" (UID: "3a2114c7-d34a-4249-97a4-65f5375453ce") : secret "dns-default-metrics-tls" not found
Sep 12 00:17:10.410115 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:17:10.410081    2571 generic.go:356] "Generic (PLEG): container finished" podID="0a3e3c8c-cafa-4254-a972-fcf494ae4e28" containerID="1d81373be4b1db76d241f590a9aff65c99302093b827fcf5350323261fbf5c2c" exitCode=0
Sep 12 00:17:10.410532 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:17:10.410158    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-62v9r" event={"ID":"0a3e3c8c-cafa-4254-a972-fcf494ae4e28","Type":"ContainerDied","Data":"1d81373be4b1db76d241f590a9aff65c99302093b827fcf5350323261fbf5c2c"}
Sep 12 00:17:10.411752 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:17:10.411727    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-nxrg5" event={"ID":"09b83899-89b7-4f4a-b485-c102295f188e","Type":"ContainerStarted","Data":"9b0733537dd857e29f6047f98664bfeeee531b280c0f5c7350722e781e539155"}
Sep 12 00:17:10.413220 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:17:10.413199    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-b6792" event={"ID":"dd7e1ff1-8f0d-4f40-9a32-2a6cd30cb239","Type":"ContainerStarted","Data":"f25ed81614930faf4e19b4fb6d5665b95170536e4ca00cf7ccf7ef08d534d9e8"}
Sep 12 00:17:10.413328 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:17:10.413317    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-b6792"
Sep 12 00:17:10.451308 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:17:10.451210    2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-nxrg5" podStartSLOduration=33.491268529 podStartE2EDuration="40.451193623s" totalImagesPullingTime="6.959925094s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-12 00:16:30 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-12 00:17:10.450895362 +0000 UTC m=+41.722535872" watchObservedRunningTime="2026-09-12 00:17:10.451193623 +0000 UTC m=+41.722834133"
Sep 12 00:17:10.469026 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:17:10.468985    2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-b6792" podStartSLOduration=33.523852199 podStartE2EDuration="40.468972168s" totalImagesPullingTime="6.945119969s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-12 00:16:30 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-12 00:17:10.468430533 +0000 UTC m=+41.740071042" watchObservedRunningTime="2026-09-12 00:17:10.468972168 +0000 UTC m=+41.740612687"
Sep 12 00:17:11.419300 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:17:11.419219    2571 generic.go:356] "Generic (PLEG): container finished" podID="0a3e3c8c-cafa-4254-a972-fcf494ae4e28" containerID="6a30d51a4de0b225152b5bfb6c50e351affb52d27d0ca26d349fe27988258559" exitCode=0
Sep 12 00:17:11.419704 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:17:11.419303    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-62v9r" event={"ID":"0a3e3c8c-cafa-4254-a972-fcf494ae4e28","Type":"ContainerDied","Data":"6a30d51a4de0b225152b5bfb6c50e351affb52d27d0ca26d349fe27988258559"}
Sep 12 00:17:12.425001 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:17:12.424959    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-62v9r" event={"ID":"0a3e3c8c-cafa-4254-a972-fcf494ae4e28","Type":"ContainerStarted","Data":"eca9280b99b1f4944fc76121ef65457c56923cdb6f30d368208a9e9c9ee6f761"}
Sep 12 00:17:12.445548 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:17:12.445501    2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-62v9r" podStartSLOduration=6.705363382 podStartE2EDuration="42.445485971s" totalImagesPullingTime="35.740122589s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-12 00:16:30 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-12 00:17:12.444616021 +0000 UTC m=+43.716256530" watchObservedRunningTime="2026-09-12 00:17:12.445485971 +0000 UTC m=+43.717126461"
Sep 12 00:17:14.923151 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:17:14.923115    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/3a2114c7-d34a-4249-97a4-65f5375453ce-metrics-tls\") pod \"dns-default-fr2kt\" (UID: \"3a2114c7-d34a-4249-97a4-65f5375453ce\") " pod="openshift-dns/dns-default-fr2kt"
Sep 12 00:17:14.923507 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:17:14.923160    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/80cbfa96-879b-48ea-8732-d50ebebea182-registry-tls\") pod \"image-registry-744fcccbf-5vhvb\" (UID: \"80cbfa96-879b-48ea-8732-d50ebebea182\") " pod="openshift-image-registry/image-registry-744fcccbf-5vhvb"
Sep 12 00:17:14.923507 ip-10-0-141-128 kubenswrapper[2571]: E0912 00:17:14.923258    2571 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 12 00:17:14.923507 ip-10-0-141-128 kubenswrapper[2571]: E0912 00:17:14.923270    2571 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Sep 12 00:17:14.923507 ip-10-0-141-128 kubenswrapper[2571]: E0912 00:17:14.923283    2571 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-744fcccbf-5vhvb: secret "image-registry-tls" not found
Sep 12 00:17:14.923507 ip-10-0-141-128 kubenswrapper[2571]: E0912 00:17:14.923320    2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3a2114c7-d34a-4249-97a4-65f5375453ce-metrics-tls podName:3a2114c7-d34a-4249-97a4-65f5375453ce nodeName:}" failed. No retries permitted until 2026-09-12 00:17:30.923306549 +0000 UTC m=+62.194947035 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/3a2114c7-d34a-4249-97a4-65f5375453ce-metrics-tls") pod "dns-default-fr2kt" (UID: "3a2114c7-d34a-4249-97a4-65f5375453ce") : secret "dns-default-metrics-tls" not found
Sep 12 00:17:14.923507 ip-10-0-141-128 kubenswrapper[2571]: E0912 00:17:14.923335    2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/80cbfa96-879b-48ea-8732-d50ebebea182-registry-tls podName:80cbfa96-879b-48ea-8732-d50ebebea182 nodeName:}" failed. No retries permitted until 2026-09-12 00:17:30.923328999 +0000 UTC m=+62.194969485 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/80cbfa96-879b-48ea-8732-d50ebebea182-registry-tls") pod "image-registry-744fcccbf-5vhvb" (UID: "80cbfa96-879b-48ea-8732-d50ebebea182") : secret "image-registry-tls" not found
Sep 12 00:17:24.365447 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:17:24.365418    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-j6bgr"
Sep 12 00:17:31.019666 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:17:31.019628    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/3a2114c7-d34a-4249-97a4-65f5375453ce-metrics-tls\") pod \"dns-default-fr2kt\" (UID: \"3a2114c7-d34a-4249-97a4-65f5375453ce\") " pod="openshift-dns/dns-default-fr2kt"
Sep 12 00:17:31.020152 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:17:31.019676    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/80cbfa96-879b-48ea-8732-d50ebebea182-registry-tls\") pod \"image-registry-744fcccbf-5vhvb\" (UID: \"80cbfa96-879b-48ea-8732-d50ebebea182\") " pod="openshift-image-registry/image-registry-744fcccbf-5vhvb"
Sep 12 00:17:31.020152 ip-10-0-141-128 kubenswrapper[2571]: E0912 00:17:31.019780    2571 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Sep 12 00:17:31.020152 ip-10-0-141-128 kubenswrapper[2571]: E0912 00:17:31.019791    2571 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-744fcccbf-5vhvb: secret "image-registry-tls" not found
Sep 12 00:17:31.020152 ip-10-0-141-128 kubenswrapper[2571]: E0912 00:17:31.019788    2571 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 12 00:17:31.020152 ip-10-0-141-128 kubenswrapper[2571]: E0912 00:17:31.019855    2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/80cbfa96-879b-48ea-8732-d50ebebea182-registry-tls podName:80cbfa96-879b-48ea-8732-d50ebebea182 nodeName:}" failed. No retries permitted until 2026-09-12 00:18:03.019840828 +0000 UTC m=+94.291481315 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/80cbfa96-879b-48ea-8732-d50ebebea182-registry-tls") pod "image-registry-744fcccbf-5vhvb" (UID: "80cbfa96-879b-48ea-8732-d50ebebea182") : secret "image-registry-tls" not found
Sep 12 00:17:31.020152 ip-10-0-141-128 kubenswrapper[2571]: E0912 00:17:31.019883    2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3a2114c7-d34a-4249-97a4-65f5375453ce-metrics-tls podName:3a2114c7-d34a-4249-97a4-65f5375453ce nodeName:}" failed. No retries permitted until 2026-09-12 00:18:03.019864117 +0000 UTC m=+94.291504609 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/3a2114c7-d34a-4249-97a4-65f5375453ce-metrics-tls") pod "dns-default-fr2kt" (UID: "3a2114c7-d34a-4249-97a4-65f5375453ce") : secret "dns-default-metrics-tls" not found
Sep 12 00:17:33.939725 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:17:33.939680    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/6d479776-cc30-4bea-bb3a-8b4c259d59c7-metrics-certs\") pod \"network-metrics-daemon-bdp25\" (UID: \"6d479776-cc30-4bea-bb3a-8b4c259d59c7\") " pod="openshift-multus/network-metrics-daemon-bdp25"
Sep 12 00:17:33.940101 ip-10-0-141-128 kubenswrapper[2571]: E0912 00:17:33.939805    2571 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found
Sep 12 00:17:33.940101 ip-10-0-141-128 kubenswrapper[2571]: E0912 00:17:33.939873    2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6d479776-cc30-4bea-bb3a-8b4c259d59c7-metrics-certs podName:6d479776-cc30-4bea-bb3a-8b4c259d59c7 nodeName:}" failed. No retries permitted until 2026-09-12 00:18:37.939857808 +0000 UTC m=+129.211498310 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/6d479776-cc30-4bea-bb3a-8b4c259d59c7-metrics-certs") pod "network-metrics-daemon-bdp25" (UID: "6d479776-cc30-4bea-bb3a-8b4c259d59c7") : secret "metrics-daemon-secret" not found
Sep 12 00:17:41.422033 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:17:41.422003    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-b6792"
Sep 12 00:18:03.027724 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:18:03.027686    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/3a2114c7-d34a-4249-97a4-65f5375453ce-metrics-tls\") pod \"dns-default-fr2kt\" (UID: \"3a2114c7-d34a-4249-97a4-65f5375453ce\") " pod="openshift-dns/dns-default-fr2kt"
Sep 12 00:18:03.028084 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:18:03.027733    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/80cbfa96-879b-48ea-8732-d50ebebea182-registry-tls\") pod \"image-registry-744fcccbf-5vhvb\" (UID: \"80cbfa96-879b-48ea-8732-d50ebebea182\") " pod="openshift-image-registry/image-registry-744fcccbf-5vhvb"
Sep 12 00:18:03.028084 ip-10-0-141-128 kubenswrapper[2571]: E0912 00:18:03.027831    2571 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 12 00:18:03.028084 ip-10-0-141-128 kubenswrapper[2571]: E0912 00:18:03.027842    2571 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Sep 12 00:18:03.028084 ip-10-0-141-128 kubenswrapper[2571]: E0912 00:18:03.027854    2571 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-744fcccbf-5vhvb: secret "image-registry-tls" not found
Sep 12 00:18:03.028084 ip-10-0-141-128 kubenswrapper[2571]: E0912 00:18:03.027899    2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3a2114c7-d34a-4249-97a4-65f5375453ce-metrics-tls podName:3a2114c7-d34a-4249-97a4-65f5375453ce nodeName:}" failed. No retries permitted until 2026-09-12 00:19:07.027885199 +0000 UTC m=+158.299525689 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/3a2114c7-d34a-4249-97a4-65f5375453ce-metrics-tls") pod "dns-default-fr2kt" (UID: "3a2114c7-d34a-4249-97a4-65f5375453ce") : secret "dns-default-metrics-tls" not found
Sep 12 00:18:03.028084 ip-10-0-141-128 kubenswrapper[2571]: E0912 00:18:03.027915    2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/80cbfa96-879b-48ea-8732-d50ebebea182-registry-tls podName:80cbfa96-879b-48ea-8732-d50ebebea182 nodeName:}" failed. No retries permitted until 2026-09-12 00:19:07.0279079 +0000 UTC m=+158.299548387 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/80cbfa96-879b-48ea-8732-d50ebebea182-registry-tls") pod "image-registry-744fcccbf-5vhvb" (UID: "80cbfa96-879b-48ea-8732-d50ebebea182") : secret "image-registry-tls" not found
Sep 12 00:18:08.770264 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:18:08.770231    2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-6cbf6fc9d-jk8ql"]
Sep 12 00:18:08.771860 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:18:08.771844    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-6cbf6fc9d-jk8ql"
Sep 12 00:18:08.775771 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:18:08.775748    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"work-manager-hub-kubeconfig\""
Sep 12 00:18:08.775881 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:18:08.775756    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"kube-root-ca.crt\""
Sep 12 00:18:08.775881 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:18:08.775834    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"open-cluster-management-image-pull-credentials\""
Sep 12 00:18:08.775881 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:18:08.775850    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"openshift-service-ca.crt\""
Sep 12 00:18:08.786149 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:18:08.786117    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-6cbf6fc9d-jk8ql"]
Sep 12 00:18:08.864679 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:18:08.864650    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-627h4\" (UniqueName: \"kubernetes.io/projected/194068ef-2475-4f30-a7eb-85f33a16aae1-kube-api-access-627h4\") pod \"klusterlet-addon-workmgr-6cbf6fc9d-jk8ql\" (UID: \"194068ef-2475-4f30-a7eb-85f33a16aae1\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-6cbf6fc9d-jk8ql"
Sep 12 00:18:08.864801 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:18:08.864745    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/194068ef-2475-4f30-a7eb-85f33a16aae1-tmp\") pod \"klusterlet-addon-workmgr-6cbf6fc9d-jk8ql\" (UID: \"194068ef-2475-4f30-a7eb-85f33a16aae1\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-6cbf6fc9d-jk8ql"
Sep 12 00:18:08.864801 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:18:08.864772    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/194068ef-2475-4f30-a7eb-85f33a16aae1-klusterlet-config\") pod \"klusterlet-addon-workmgr-6cbf6fc9d-jk8ql\" (UID: \"194068ef-2475-4f30-a7eb-85f33a16aae1\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-6cbf6fc9d-jk8ql"
Sep 12 00:18:08.965986 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:18:08.965963    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/194068ef-2475-4f30-a7eb-85f33a16aae1-tmp\") pod \"klusterlet-addon-workmgr-6cbf6fc9d-jk8ql\" (UID: \"194068ef-2475-4f30-a7eb-85f33a16aae1\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-6cbf6fc9d-jk8ql"
Sep 12 00:18:08.966078 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:18:08.965998    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/194068ef-2475-4f30-a7eb-85f33a16aae1-klusterlet-config\") pod \"klusterlet-addon-workmgr-6cbf6fc9d-jk8ql\" (UID: \"194068ef-2475-4f30-a7eb-85f33a16aae1\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-6cbf6fc9d-jk8ql"
Sep 12 00:18:08.966078 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:18:08.966044    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-627h4\" (UniqueName: \"kubernetes.io/projected/194068ef-2475-4f30-a7eb-85f33a16aae1-kube-api-access-627h4\") pod \"klusterlet-addon-workmgr-6cbf6fc9d-jk8ql\" (UID: \"194068ef-2475-4f30-a7eb-85f33a16aae1\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-6cbf6fc9d-jk8ql"
Sep 12 00:18:08.966336 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:18:08.966315    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/194068ef-2475-4f30-a7eb-85f33a16aae1-tmp\") pod \"klusterlet-addon-workmgr-6cbf6fc9d-jk8ql\" (UID: \"194068ef-2475-4f30-a7eb-85f33a16aae1\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-6cbf6fc9d-jk8ql"
Sep 12 00:18:08.968474 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:18:08.968456    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/194068ef-2475-4f30-a7eb-85f33a16aae1-klusterlet-config\") pod \"klusterlet-addon-workmgr-6cbf6fc9d-jk8ql\" (UID: \"194068ef-2475-4f30-a7eb-85f33a16aae1\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-6cbf6fc9d-jk8ql"
Sep 12 00:18:08.973687 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:18:08.973664    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-627h4\" (UniqueName: \"kubernetes.io/projected/194068ef-2475-4f30-a7eb-85f33a16aae1-kube-api-access-627h4\") pod \"klusterlet-addon-workmgr-6cbf6fc9d-jk8ql\" (UID: \"194068ef-2475-4f30-a7eb-85f33a16aae1\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-6cbf6fc9d-jk8ql"
Sep 12 00:18:09.080681 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:18:09.080622    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-6cbf6fc9d-jk8ql"
Sep 12 00:18:09.343287 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:18:09.343215    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-6cbf6fc9d-jk8ql"]
Sep 12 00:18:09.345972 ip-10-0-141-128 kubenswrapper[2571]: W0912 00:18:09.345939    2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod194068ef_2475_4f30_a7eb_85f33a16aae1.slice/crio-91b269b8ae1730f1f97c5f75129c163a0d8ba6702e97ba3834d66b704b7ce581 WatchSource:0}: Error finding container 91b269b8ae1730f1f97c5f75129c163a0d8ba6702e97ba3834d66b704b7ce581: Status 404 returned error can't find the container with id 91b269b8ae1730f1f97c5f75129c163a0d8ba6702e97ba3834d66b704b7ce581
Sep 12 00:18:09.559563 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:18:09.559532    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-6cbf6fc9d-jk8ql" event={"ID":"194068ef-2475-4f30-a7eb-85f33a16aae1","Type":"ContainerStarted","Data":"91b269b8ae1730f1f97c5f75129c163a0d8ba6702e97ba3834d66b704b7ce581"}
Sep 12 00:18:13.573708 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:18:13.573670    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-6cbf6fc9d-jk8ql" event={"ID":"194068ef-2475-4f30-a7eb-85f33a16aae1","Type":"ContainerStarted","Data":"e3b9a2939af2598bf9cacaf154c9a3576fef3f4a47b1bf26033df2869a7a8eb1"}
Sep 12 00:18:13.574098 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:18:13.573885    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-6cbf6fc9d-jk8ql"
Sep 12 00:18:13.575396 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:18:13.575376    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-6cbf6fc9d-jk8ql"
Sep 12 00:18:13.590093 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:18:13.590053    2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-6cbf6fc9d-jk8ql" podStartSLOduration=1.866619488 podStartE2EDuration="5.590040111s" totalImagesPullingTime="3.723420623s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-12 00:18:08 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-12 00:18:13.588796767 +0000 UTC m=+104.860437276" watchObservedRunningTime="2026-09-12 00:18:13.590040111 +0000 UTC m=+104.861680619"
Sep 12 00:18:37.956553 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:18:37.956505    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/6d479776-cc30-4bea-bb3a-8b4c259d59c7-metrics-certs\") pod \"network-metrics-daemon-bdp25\" (UID: \"6d479776-cc30-4bea-bb3a-8b4c259d59c7\") " pod="openshift-multus/network-metrics-daemon-bdp25"
Sep 12 00:18:37.957074 ip-10-0-141-128 kubenswrapper[2571]: E0912 00:18:37.956670    2571 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found
Sep 12 00:18:37.957074 ip-10-0-141-128 kubenswrapper[2571]: E0912 00:18:37.956747    2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6d479776-cc30-4bea-bb3a-8b4c259d59c7-metrics-certs podName:6d479776-cc30-4bea-bb3a-8b4c259d59c7 nodeName:}" failed. No retries permitted until 2026-09-12 00:20:39.956731041 +0000 UTC m=+251.228371527 (durationBeforeRetry 2m2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/6d479776-cc30-4bea-bb3a-8b4c259d59c7-metrics-certs") pod "network-metrics-daemon-bdp25" (UID: "6d479776-cc30-4bea-bb3a-8b4c259d59c7") : secret "metrics-daemon-secret" not found
Sep 12 00:19:02.116358 ip-10-0-141-128 kubenswrapper[2571]: E0912 00:19:02.116303    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[registry-tls], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-image-registry/image-registry-744fcccbf-5vhvb" podUID="80cbfa96-879b-48ea-8732-d50ebebea182"
Sep 12 00:19:02.124465 ip-10-0-141-128 kubenswrapper[2571]: E0912 00:19:02.124431    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[metrics-tls], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-dns/dns-default-fr2kt" podUID="3a2114c7-d34a-4249-97a4-65f5375453ce"
Sep 12 00:19:02.161229 ip-10-0-141-128 kubenswrapper[2571]: E0912 00:19:02.161205    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[metrics-certs], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-multus/network-metrics-daemon-bdp25" podUID="6d479776-cc30-4bea-bb3a-8b4c259d59c7"
Sep 12 00:19:02.690029 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:19:02.690003    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-744fcccbf-5vhvb"
Sep 12 00:19:02.690185 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:19:02.690007    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-fr2kt"
Sep 12 00:19:07.038575 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:19:07.038533    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/3a2114c7-d34a-4249-97a4-65f5375453ce-metrics-tls\") pod \"dns-default-fr2kt\" (UID: \"3a2114c7-d34a-4249-97a4-65f5375453ce\") " pod="openshift-dns/dns-default-fr2kt"
Sep 12 00:19:07.038956 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:19:07.038607    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/80cbfa96-879b-48ea-8732-d50ebebea182-registry-tls\") pod \"image-registry-744fcccbf-5vhvb\" (UID: \"80cbfa96-879b-48ea-8732-d50ebebea182\") " pod="openshift-image-registry/image-registry-744fcccbf-5vhvb"
Sep 12 00:19:07.038956 ip-10-0-141-128 kubenswrapper[2571]: E0912 00:19:07.038708    2571 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 12 00:19:07.038956 ip-10-0-141-128 kubenswrapper[2571]: E0912 00:19:07.038782    2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3a2114c7-d34a-4249-97a4-65f5375453ce-metrics-tls podName:3a2114c7-d34a-4249-97a4-65f5375453ce nodeName:}" failed. No retries permitted until 2026-09-12 00:21:09.038767201 +0000 UTC m=+280.310407693 (durationBeforeRetry 2m2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/3a2114c7-d34a-4249-97a4-65f5375453ce-metrics-tls") pod "dns-default-fr2kt" (UID: "3a2114c7-d34a-4249-97a4-65f5375453ce") : secret "dns-default-metrics-tls" not found
Sep 12 00:19:07.038956 ip-10-0-141-128 kubenswrapper[2571]: E0912 00:19:07.038711    2571 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Sep 12 00:19:07.038956 ip-10-0-141-128 kubenswrapper[2571]: E0912 00:19:07.038814    2571 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-744fcccbf-5vhvb: secret "image-registry-tls" not found
Sep 12 00:19:07.038956 ip-10-0-141-128 kubenswrapper[2571]: E0912 00:19:07.038861    2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/80cbfa96-879b-48ea-8732-d50ebebea182-registry-tls podName:80cbfa96-879b-48ea-8732-d50ebebea182 nodeName:}" failed. No retries permitted until 2026-09-12 00:21:09.038850477 +0000 UTC m=+280.310490978 (durationBeforeRetry 2m2s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/80cbfa96-879b-48ea-8732-d50ebebea182-registry-tls") pod "image-registry-744fcccbf-5vhvb" (UID: "80cbfa96-879b-48ea-8732-d50ebebea182") : secret "image-registry-tls" not found
Sep 12 00:19:14.134754 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:19:14.134722    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-bdp25"
Sep 12 00:20:13.574347 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:20:13.574297    2571 prober.go:120] "Probe failed" probeType="Readiness" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-6cbf6fc9d-jk8ql" podUID="194068ef-2475-4f30-a7eb-85f33a16aae1" containerName="acm-agent" probeResult="failure" output="Get \"http://10.132.0.9:8000/readyz\": dial tcp 10.132.0.9:8000: connect: connection refused"
Sep 12 00:20:13.856626 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:20:13.856527    2571 generic.go:356] "Generic (PLEG): container finished" podID="194068ef-2475-4f30-a7eb-85f33a16aae1" containerID="e3b9a2939af2598bf9cacaf154c9a3576fef3f4a47b1bf26033df2869a7a8eb1" exitCode=1
Sep 12 00:20:13.856626 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:20:13.856604    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-6cbf6fc9d-jk8ql" event={"ID":"194068ef-2475-4f30-a7eb-85f33a16aae1","Type":"ContainerDied","Data":"e3b9a2939af2598bf9cacaf154c9a3576fef3f4a47b1bf26033df2869a7a8eb1"}
Sep 12 00:20:13.856888 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:20:13.856876    2571 scope.go:117] "RemoveContainer" containerID="e3b9a2939af2598bf9cacaf154c9a3576fef3f4a47b1bf26033df2869a7a8eb1"
Sep 12 00:20:14.862038 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:20:14.862002    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-6cbf6fc9d-jk8ql" event={"ID":"194068ef-2475-4f30-a7eb-85f33a16aae1","Type":"ContainerStarted","Data":"8de2098a3a96af95d9a4cecf8f353f81feb1f3536efe08517ceeddbdfa81e554"}
Sep 12 00:20:14.862488 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:20:14.862258    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-6cbf6fc9d-jk8ql"
Sep 12 00:20:14.862945 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:20:14.862929    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-6cbf6fc9d-jk8ql"
Sep 12 00:20:29.749626 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:20:29.749573    2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/router-default-57b84c8474-8xjql"]
Sep 12 00:20:29.752964 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:20:29.752942    2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-86695b7cc7-b9jtz"]
Sep 12 00:20:29.753083 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:20:29.753050    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/router-default-57b84c8474-8xjql"
Sep 12 00:20:29.754680 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:20:29.754659    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-storage-operator/volume-data-source-validator-86695b7cc7-b9jtz"
Sep 12 00:20:29.756142 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:20:29.756111    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-metrics-certs-default\""
Sep 12 00:20:29.756142 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:20:29.756127    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-dockercfg-j6cbg\""
Sep 12 00:20:29.756142 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:20:29.756133    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-stats-default\""
Sep 12 00:20:29.756438 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:20:29.756422    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"default-ingress-cert\""
Sep 12 00:20:29.756492 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:20:29.756438    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\""
Sep 12 00:20:29.756492 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:20:29.756465    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\""
Sep 12 00:20:29.756787 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:20:29.756772    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"service-ca-bundle\""
Sep 12 00:20:29.757416 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:20:29.757400    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-storage-operator\"/\"volume-data-source-validator-dockercfg-gcv7x\""
Sep 12 00:20:29.757416 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:20:29.757406    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-storage-operator\"/\"openshift-service-ca.crt\""
Sep 12 00:20:29.757545 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:20:29.757442    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-storage-operator\"/\"kube-root-ca.crt\""
Sep 12 00:20:29.769607 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:20:29.767171    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-86695b7cc7-b9jtz"]
Sep 12 00:20:29.772741 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:20:29.772719    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/router-default-57b84c8474-8xjql"]
Sep 12 00:20:29.872051 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:20:29.872025    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/c2123986-4ae4-40ce-a423-b5e9a4accda2-stats-auth\") pod \"router-default-57b84c8474-8xjql\" (UID: \"c2123986-4ae4-40ce-a423-b5e9a4accda2\") " pod="openshift-ingress/router-default-57b84c8474-8xjql"
Sep 12 00:20:29.872159 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:20:29.872069    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xmwh5\" (UniqueName: \"kubernetes.io/projected/c2123986-4ae4-40ce-a423-b5e9a4accda2-kube-api-access-xmwh5\") pod \"router-default-57b84c8474-8xjql\" (UID: \"c2123986-4ae4-40ce-a423-b5e9a4accda2\") " pod="openshift-ingress/router-default-57b84c8474-8xjql"
Sep 12 00:20:29.872159 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:20:29.872103    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c2123986-4ae4-40ce-a423-b5e9a4accda2-service-ca-bundle\") pod \"router-default-57b84c8474-8xjql\" (UID: \"c2123986-4ae4-40ce-a423-b5e9a4accda2\") " pod="openshift-ingress/router-default-57b84c8474-8xjql"
Sep 12 00:20:29.872159 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:20:29.872119    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c2123986-4ae4-40ce-a423-b5e9a4accda2-metrics-certs\") pod \"router-default-57b84c8474-8xjql\" (UID: \"c2123986-4ae4-40ce-a423-b5e9a4accda2\") " pod="openshift-ingress/router-default-57b84c8474-8xjql"
Sep 12 00:20:29.872159 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:20:29.872154    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/c2123986-4ae4-40ce-a423-b5e9a4accda2-default-certificate\") pod \"router-default-57b84c8474-8xjql\" (UID: \"c2123986-4ae4-40ce-a423-b5e9a4accda2\") " pod="openshift-ingress/router-default-57b84c8474-8xjql"
Sep 12 00:20:29.872288 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:20:29.872170    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8c9nx\" (UniqueName: \"kubernetes.io/projected/eea0700d-afa8-412b-b099-4f6953e54eee-kube-api-access-8c9nx\") pod \"volume-data-source-validator-86695b7cc7-b9jtz\" (UID: \"eea0700d-afa8-412b-b099-4f6953e54eee\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-86695b7cc7-b9jtz"
Sep 12 00:20:29.939205 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:20:29.939182    2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-service-ca-operator/service-ca-operator-cf7c776b7-k2ftn"]
Sep 12 00:20:29.940776 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:20:29.940762    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca-operator/service-ca-operator-cf7c776b7-k2ftn"
Sep 12 00:20:29.943162 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:20:29.943137    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca-operator\"/\"serving-cert\""
Sep 12 00:20:29.943162 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:20:29.943152    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca-operator\"/\"service-ca-operator-dockercfg-qv8fc\""
Sep 12 00:20:29.943313 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:20:29.943199    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"openshift-service-ca.crt\""
Sep 12 00:20:29.943435 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:20:29.943422    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"service-ca-operator-config\""
Sep 12 00:20:29.943471 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:20:29.943422    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"kube-root-ca.crt\""
Sep 12 00:20:29.951519 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:20:29.951495    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca-operator/service-ca-operator-cf7c776b7-k2ftn"]
Sep 12 00:20:29.972667 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:20:29.972643    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/c2123986-4ae4-40ce-a423-b5e9a4accda2-default-certificate\") pod \"router-default-57b84c8474-8xjql\" (UID: \"c2123986-4ae4-40ce-a423-b5e9a4accda2\") " pod="openshift-ingress/router-default-57b84c8474-8xjql"
Sep 12 00:20:29.972757 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:20:29.972670    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8c9nx\" (UniqueName: \"kubernetes.io/projected/eea0700d-afa8-412b-b099-4f6953e54eee-kube-api-access-8c9nx\") pod \"volume-data-source-validator-86695b7cc7-b9jtz\" (UID: \"eea0700d-afa8-412b-b099-4f6953e54eee\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-86695b7cc7-b9jtz"
Sep 12 00:20:29.972757 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:20:29.972689    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pkfrb\" (UniqueName: \"kubernetes.io/projected/bfc37701-bd22-49af-8189-77a984f3a57d-kube-api-access-pkfrb\") pod \"service-ca-operator-cf7c776b7-k2ftn\" (UID: \"bfc37701-bd22-49af-8189-77a984f3a57d\") " pod="openshift-service-ca-operator/service-ca-operator-cf7c776b7-k2ftn"
Sep 12 00:20:29.972757 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:20:29.972712    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/bfc37701-bd22-49af-8189-77a984f3a57d-serving-cert\") pod \"service-ca-operator-cf7c776b7-k2ftn\" (UID: \"bfc37701-bd22-49af-8189-77a984f3a57d\") " pod="openshift-service-ca-operator/service-ca-operator-cf7c776b7-k2ftn"
Sep 12 00:20:29.972757 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:20:29.972729    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/c2123986-4ae4-40ce-a423-b5e9a4accda2-stats-auth\") pod \"router-default-57b84c8474-8xjql\" (UID: \"c2123986-4ae4-40ce-a423-b5e9a4accda2\") " pod="openshift-ingress/router-default-57b84c8474-8xjql"
Sep 12 00:20:29.972911 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:20:29.972857    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xmwh5\" (UniqueName: \"kubernetes.io/projected/c2123986-4ae4-40ce-a423-b5e9a4accda2-kube-api-access-xmwh5\") pod \"router-default-57b84c8474-8xjql\" (UID: \"c2123986-4ae4-40ce-a423-b5e9a4accda2\") " pod="openshift-ingress/router-default-57b84c8474-8xjql"
Sep 12 00:20:29.972911 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:20:29.972898    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/bfc37701-bd22-49af-8189-77a984f3a57d-config\") pod \"service-ca-operator-cf7c776b7-k2ftn\" (UID: \"bfc37701-bd22-49af-8189-77a984f3a57d\") " pod="openshift-service-ca-operator/service-ca-operator-cf7c776b7-k2ftn"
Sep 12 00:20:29.972986 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:20:29.972935    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c2123986-4ae4-40ce-a423-b5e9a4accda2-service-ca-bundle\") pod \"router-default-57b84c8474-8xjql\" (UID: \"c2123986-4ae4-40ce-a423-b5e9a4accda2\") " pod="openshift-ingress/router-default-57b84c8474-8xjql"
Sep 12 00:20:29.972986 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:20:29.972959    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c2123986-4ae4-40ce-a423-b5e9a4accda2-metrics-certs\") pod \"router-default-57b84c8474-8xjql\" (UID: \"c2123986-4ae4-40ce-a423-b5e9a4accda2\") " pod="openshift-ingress/router-default-57b84c8474-8xjql"
Sep 12 00:20:29.973083 ip-10-0-141-128 kubenswrapper[2571]: E0912 00:20:29.973052    2571 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Sep 12 00:20:29.973132 ip-10-0-141-128 kubenswrapper[2571]: E0912 00:20:29.973102    2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/c2123986-4ae4-40ce-a423-b5e9a4accda2-service-ca-bundle podName:c2123986-4ae4-40ce-a423-b5e9a4accda2 nodeName:}" failed. No retries permitted until 2026-09-12 00:20:30.47308122 +0000 UTC m=+241.744721706 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/c2123986-4ae4-40ce-a423-b5e9a4accda2-service-ca-bundle") pod "router-default-57b84c8474-8xjql" (UID: "c2123986-4ae4-40ce-a423-b5e9a4accda2") : configmap references non-existent config key: service-ca.crt
Sep 12 00:20:29.973132 ip-10-0-141-128 kubenswrapper[2571]: E0912 00:20:29.973128    2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c2123986-4ae4-40ce-a423-b5e9a4accda2-metrics-certs podName:c2123986-4ae4-40ce-a423-b5e9a4accda2 nodeName:}" failed. No retries permitted until 2026-09-12 00:20:30.473116986 +0000 UTC m=+241.744757479 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/c2123986-4ae4-40ce-a423-b5e9a4accda2-metrics-certs") pod "router-default-57b84c8474-8xjql" (UID: "c2123986-4ae4-40ce-a423-b5e9a4accda2") : secret "router-metrics-certs-default" not found
Sep 12 00:20:29.975009 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:20:29.974992    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/c2123986-4ae4-40ce-a423-b5e9a4accda2-default-certificate\") pod \"router-default-57b84c8474-8xjql\" (UID: \"c2123986-4ae4-40ce-a423-b5e9a4accda2\") " pod="openshift-ingress/router-default-57b84c8474-8xjql"
Sep 12 00:20:29.975096 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:20:29.975060    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/c2123986-4ae4-40ce-a423-b5e9a4accda2-stats-auth\") pod \"router-default-57b84c8474-8xjql\" (UID: \"c2123986-4ae4-40ce-a423-b5e9a4accda2\") " pod="openshift-ingress/router-default-57b84c8474-8xjql"
Sep 12 00:20:29.980845 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:20:29.980820    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8c9nx\" (UniqueName: \"kubernetes.io/projected/eea0700d-afa8-412b-b099-4f6953e54eee-kube-api-access-8c9nx\") pod \"volume-data-source-validator-86695b7cc7-b9jtz\" (UID: \"eea0700d-afa8-412b-b099-4f6953e54eee\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-86695b7cc7-b9jtz"
Sep 12 00:20:29.980924 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:20:29.980873    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xmwh5\" (UniqueName: \"kubernetes.io/projected/c2123986-4ae4-40ce-a423-b5e9a4accda2-kube-api-access-xmwh5\") pod \"router-default-57b84c8474-8xjql\" (UID: \"c2123986-4ae4-40ce-a423-b5e9a4accda2\") " pod="openshift-ingress/router-default-57b84c8474-8xjql"
Sep 12 00:20:30.069303 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:20:30.069231    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-storage-operator/volume-data-source-validator-86695b7cc7-b9jtz"
Sep 12 00:20:30.074027 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:20:30.074008    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pkfrb\" (UniqueName: \"kubernetes.io/projected/bfc37701-bd22-49af-8189-77a984f3a57d-kube-api-access-pkfrb\") pod \"service-ca-operator-cf7c776b7-k2ftn\" (UID: \"bfc37701-bd22-49af-8189-77a984f3a57d\") " pod="openshift-service-ca-operator/service-ca-operator-cf7c776b7-k2ftn"
Sep 12 00:20:30.074117 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:20:30.074046    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/bfc37701-bd22-49af-8189-77a984f3a57d-serving-cert\") pod \"service-ca-operator-cf7c776b7-k2ftn\" (UID: \"bfc37701-bd22-49af-8189-77a984f3a57d\") " pod="openshift-service-ca-operator/service-ca-operator-cf7c776b7-k2ftn"
Sep 12 00:20:30.074117 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:20:30.074109    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/bfc37701-bd22-49af-8189-77a984f3a57d-config\") pod \"service-ca-operator-cf7c776b7-k2ftn\" (UID: \"bfc37701-bd22-49af-8189-77a984f3a57d\") " pod="openshift-service-ca-operator/service-ca-operator-cf7c776b7-k2ftn"
Sep 12 00:20:30.074644 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:20:30.074620    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/bfc37701-bd22-49af-8189-77a984f3a57d-config\") pod \"service-ca-operator-cf7c776b7-k2ftn\" (UID: \"bfc37701-bd22-49af-8189-77a984f3a57d\") " pod="openshift-service-ca-operator/service-ca-operator-cf7c776b7-k2ftn"
Sep 12 00:20:30.076108 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:20:30.076089    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/bfc37701-bd22-49af-8189-77a984f3a57d-serving-cert\") pod \"service-ca-operator-cf7c776b7-k2ftn\" (UID: \"bfc37701-bd22-49af-8189-77a984f3a57d\") " pod="openshift-service-ca-operator/service-ca-operator-cf7c776b7-k2ftn"
Sep 12 00:20:30.081482 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:20:30.081457    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pkfrb\" (UniqueName: \"kubernetes.io/projected/bfc37701-bd22-49af-8189-77a984f3a57d-kube-api-access-pkfrb\") pod \"service-ca-operator-cf7c776b7-k2ftn\" (UID: \"bfc37701-bd22-49af-8189-77a984f3a57d\") " pod="openshift-service-ca-operator/service-ca-operator-cf7c776b7-k2ftn"
Sep 12 00:20:30.249313 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:20:30.249275    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca-operator/service-ca-operator-cf7c776b7-k2ftn"
Sep 12 00:20:30.330666 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:20:30.330612    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-86695b7cc7-b9jtz"]
Sep 12 00:20:30.333385 ip-10-0-141-128 kubenswrapper[2571]: W0912 00:20:30.333361    2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podeea0700d_afa8_412b_b099_4f6953e54eee.slice/crio-426129f3e05ecc5ce9ac2cc66d3e0b4997dd27778e29727a5240af4f4d791caf WatchSource:0}: Error finding container 426129f3e05ecc5ce9ac2cc66d3e0b4997dd27778e29727a5240af4f4d791caf: Status 404 returned error can't find the container with id 426129f3e05ecc5ce9ac2cc66d3e0b4997dd27778e29727a5240af4f4d791caf
Sep 12 00:20:30.477239 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:20:30.477213    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c2123986-4ae4-40ce-a423-b5e9a4accda2-service-ca-bundle\") pod \"router-default-57b84c8474-8xjql\" (UID: \"c2123986-4ae4-40ce-a423-b5e9a4accda2\") " pod="openshift-ingress/router-default-57b84c8474-8xjql"
Sep 12 00:20:30.477239 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:20:30.477240    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c2123986-4ae4-40ce-a423-b5e9a4accda2-metrics-certs\") pod \"router-default-57b84c8474-8xjql\" (UID: \"c2123986-4ae4-40ce-a423-b5e9a4accda2\") " pod="openshift-ingress/router-default-57b84c8474-8xjql"
Sep 12 00:20:30.477385 ip-10-0-141-128 kubenswrapper[2571]: E0912 00:20:30.477344    2571 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Sep 12 00:20:30.477385 ip-10-0-141-128 kubenswrapper[2571]: E0912 00:20:30.477369    2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/c2123986-4ae4-40ce-a423-b5e9a4accda2-service-ca-bundle podName:c2123986-4ae4-40ce-a423-b5e9a4accda2 nodeName:}" failed. No retries permitted until 2026-09-12 00:20:31.477349829 +0000 UTC m=+242.748990338 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/c2123986-4ae4-40ce-a423-b5e9a4accda2-service-ca-bundle") pod "router-default-57b84c8474-8xjql" (UID: "c2123986-4ae4-40ce-a423-b5e9a4accda2") : configmap references non-existent config key: service-ca.crt
Sep 12 00:20:30.477455 ip-10-0-141-128 kubenswrapper[2571]: E0912 00:20:30.477431    2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c2123986-4ae4-40ce-a423-b5e9a4accda2-metrics-certs podName:c2123986-4ae4-40ce-a423-b5e9a4accda2 nodeName:}" failed. No retries permitted until 2026-09-12 00:20:31.477423717 +0000 UTC m=+242.749064204 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/c2123986-4ae4-40ce-a423-b5e9a4accda2-metrics-certs") pod "router-default-57b84c8474-8xjql" (UID: "c2123986-4ae4-40ce-a423-b5e9a4accda2") : secret "router-metrics-certs-default" not found
Sep 12 00:20:30.512563 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:20:30.512536    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca-operator/service-ca-operator-cf7c776b7-k2ftn"]
Sep 12 00:20:30.516360 ip-10-0-141-128 kubenswrapper[2571]: W0912 00:20:30.516337    2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbfc37701_bd22_49af_8189_77a984f3a57d.slice/crio-b905d7fdab406b4b1e5e469c6845713b4e1360eb7e238bc84edfbf2eca32067a WatchSource:0}: Error finding container b905d7fdab406b4b1e5e469c6845713b4e1360eb7e238bc84edfbf2eca32067a: Status 404 returned error can't find the container with id b905d7fdab406b4b1e5e469c6845713b4e1360eb7e238bc84edfbf2eca32067a
Sep 12 00:20:30.902815 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:20:30.902780    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-cf7c776b7-k2ftn" event={"ID":"bfc37701-bd22-49af-8189-77a984f3a57d","Type":"ContainerStarted","Data":"b905d7fdab406b4b1e5e469c6845713b4e1360eb7e238bc84edfbf2eca32067a"}
Sep 12 00:20:30.904082 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:20:30.904059    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-storage-operator/volume-data-source-validator-86695b7cc7-b9jtz" event={"ID":"eea0700d-afa8-412b-b099-4f6953e54eee","Type":"ContainerStarted","Data":"426129f3e05ecc5ce9ac2cc66d3e0b4997dd27778e29727a5240af4f4d791caf"}
Sep 12 00:20:31.485911 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:20:31.485882    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c2123986-4ae4-40ce-a423-b5e9a4accda2-service-ca-bundle\") pod \"router-default-57b84c8474-8xjql\" (UID: \"c2123986-4ae4-40ce-a423-b5e9a4accda2\") " pod="openshift-ingress/router-default-57b84c8474-8xjql"
Sep 12 00:20:31.486023 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:20:31.485925    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c2123986-4ae4-40ce-a423-b5e9a4accda2-metrics-certs\") pod \"router-default-57b84c8474-8xjql\" (UID: \"c2123986-4ae4-40ce-a423-b5e9a4accda2\") " pod="openshift-ingress/router-default-57b84c8474-8xjql"
Sep 12 00:20:31.486090 ip-10-0-141-128 kubenswrapper[2571]: E0912 00:20:31.486038    2571 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Sep 12 00:20:31.486090 ip-10-0-141-128 kubenswrapper[2571]: E0912 00:20:31.486043    2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/c2123986-4ae4-40ce-a423-b5e9a4accda2-service-ca-bundle podName:c2123986-4ae4-40ce-a423-b5e9a4accda2 nodeName:}" failed. No retries permitted until 2026-09-12 00:20:33.48602754 +0000 UTC m=+244.757668028 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/c2123986-4ae4-40ce-a423-b5e9a4accda2-service-ca-bundle") pod "router-default-57b84c8474-8xjql" (UID: "c2123986-4ae4-40ce-a423-b5e9a4accda2") : configmap references non-existent config key: service-ca.crt
Sep 12 00:20:31.486198 ip-10-0-141-128 kubenswrapper[2571]: E0912 00:20:31.486120    2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c2123986-4ae4-40ce-a423-b5e9a4accda2-metrics-certs podName:c2123986-4ae4-40ce-a423-b5e9a4accda2 nodeName:}" failed. No retries permitted until 2026-09-12 00:20:33.486109317 +0000 UTC m=+244.757749808 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/c2123986-4ae4-40ce-a423-b5e9a4accda2-metrics-certs") pod "router-default-57b84c8474-8xjql" (UID: "c2123986-4ae4-40ce-a423-b5e9a4accda2") : secret "router-metrics-certs-default" not found
Sep 12 00:20:31.909635 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:20:31.909546    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-storage-operator/volume-data-source-validator-86695b7cc7-b9jtz" event={"ID":"eea0700d-afa8-412b-b099-4f6953e54eee","Type":"ContainerStarted","Data":"a359cc1608bbf25b7a641fe4311cbcdf6f303476fcb666776b9b38f4a7bca41c"}
Sep 12 00:20:31.924494 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:20:31.924447    2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-storage-operator/volume-data-source-validator-86695b7cc7-b9jtz" podStartSLOduration=1.8246211799999998 podStartE2EDuration="2.924432022s" totalImagesPullingTime="1.099810842s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-12 00:20:29 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-12 00:20:31.923858528 +0000 UTC m=+243.195499040" watchObservedRunningTime="2026-09-12 00:20:31.924432022 +0000 UTC m=+243.196072530"
Sep 12 00:20:32.914830 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:20:32.914796    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-cf7c776b7-k2ftn" event={"ID":"bfc37701-bd22-49af-8189-77a984f3a57d","Type":"ContainerStarted","Data":"2a4af38f75b0d4466b8dd096ba2696694cdcf312b5d7c34b7d3917c3820b4625"}
Sep 12 00:20:32.929010 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:20:32.928959    2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-service-ca-operator/service-ca-operator-cf7c776b7-k2ftn" podStartSLOduration=2.058360958 podStartE2EDuration="3.928943719s" totalImagesPullingTime="1.870582761s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-12 00:20:29 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-12 00:20:32.928922524 +0000 UTC m=+244.200563033" watchObservedRunningTime="2026-09-12 00:20:32.928943719 +0000 UTC m=+244.200584229"
Sep 12 00:20:33.501376 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:20:33.501338    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c2123986-4ae4-40ce-a423-b5e9a4accda2-service-ca-bundle\") pod \"router-default-57b84c8474-8xjql\" (UID: \"c2123986-4ae4-40ce-a423-b5e9a4accda2\") " pod="openshift-ingress/router-default-57b84c8474-8xjql"
Sep 12 00:20:33.501376 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:20:33.501373    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c2123986-4ae4-40ce-a423-b5e9a4accda2-metrics-certs\") pod \"router-default-57b84c8474-8xjql\" (UID: \"c2123986-4ae4-40ce-a423-b5e9a4accda2\") " pod="openshift-ingress/router-default-57b84c8474-8xjql"
Sep 12 00:20:33.501566 ip-10-0-141-128 kubenswrapper[2571]: E0912 00:20:33.501476    2571 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Sep 12 00:20:33.501566 ip-10-0-141-128 kubenswrapper[2571]: E0912 00:20:33.501493    2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/c2123986-4ae4-40ce-a423-b5e9a4accda2-service-ca-bundle podName:c2123986-4ae4-40ce-a423-b5e9a4accda2 nodeName:}" failed. No retries permitted until 2026-09-12 00:20:37.501476497 +0000 UTC m=+248.773116991 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/c2123986-4ae4-40ce-a423-b5e9a4accda2-service-ca-bundle") pod "router-default-57b84c8474-8xjql" (UID: "c2123986-4ae4-40ce-a423-b5e9a4accda2") : configmap references non-existent config key: service-ca.crt
Sep 12 00:20:33.501566 ip-10-0-141-128 kubenswrapper[2571]: E0912 00:20:33.501516    2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c2123986-4ae4-40ce-a423-b5e9a4accda2-metrics-certs podName:c2123986-4ae4-40ce-a423-b5e9a4accda2 nodeName:}" failed. No retries permitted until 2026-09-12 00:20:37.501510035 +0000 UTC m=+248.773150521 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/c2123986-4ae4-40ce-a423-b5e9a4accda2-metrics-certs") pod "router-default-57b84c8474-8xjql" (UID: "c2123986-4ae4-40ce-a423-b5e9a4accda2") : secret "router-metrics-certs-default" not found
Sep 12 00:20:34.997623 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:20:34.997573    2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-kkm5j"]
Sep 12 00:20:34.999654 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:20:34.999637    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-kkm5j"
Sep 12 00:20:35.001902 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:20:35.001880    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\""
Sep 12 00:20:35.002031 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:20:35.002010    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\""
Sep 12 00:20:35.002088 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:20:35.002035    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\""
Sep 12 00:20:35.002147 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:20:35.002101    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\""
Sep 12 00:20:35.002205 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:20:35.002145    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-pwzwl\""
Sep 12 00:20:35.012134 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:20:35.012115    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-kkm5j"]
Sep 12 00:20:35.113271 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:20:35.113248    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/531e7c03-cde5-4559-9d03-403158fb7168-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-kkm5j\" (UID: \"531e7c03-cde5-4559-9d03-403158fb7168\") " pod="openshift-insights/insights-runtime-extractor-kkm5j"
Sep 12 00:20:35.113438 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:20:35.113276    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/531e7c03-cde5-4559-9d03-403158fb7168-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-kkm5j\" (UID: \"531e7c03-cde5-4559-9d03-403158fb7168\") " pod="openshift-insights/insights-runtime-extractor-kkm5j"
Sep 12 00:20:35.113438 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:20:35.113312    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/531e7c03-cde5-4559-9d03-403158fb7168-crio-socket\") pod \"insights-runtime-extractor-kkm5j\" (UID: \"531e7c03-cde5-4559-9d03-403158fb7168\") " pod="openshift-insights/insights-runtime-extractor-kkm5j"
Sep 12 00:20:35.113438 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:20:35.113408    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/531e7c03-cde5-4559-9d03-403158fb7168-data-volume\") pod \"insights-runtime-extractor-kkm5j\" (UID: \"531e7c03-cde5-4559-9d03-403158fb7168\") " pod="openshift-insights/insights-runtime-extractor-kkm5j"
Sep 12 00:20:35.113534 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:20:35.113439    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-n6x6n\" (UniqueName: \"kubernetes.io/projected/531e7c03-cde5-4559-9d03-403158fb7168-kube-api-access-n6x6n\") pod \"insights-runtime-extractor-kkm5j\" (UID: \"531e7c03-cde5-4559-9d03-403158fb7168\") " pod="openshift-insights/insights-runtime-extractor-kkm5j"
Sep 12 00:20:35.213933 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:20:35.213899    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/531e7c03-cde5-4559-9d03-403158fb7168-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-kkm5j\" (UID: \"531e7c03-cde5-4559-9d03-403158fb7168\") " pod="openshift-insights/insights-runtime-extractor-kkm5j"
Sep 12 00:20:35.214073 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:20:35.213944    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/531e7c03-cde5-4559-9d03-403158fb7168-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-kkm5j\" (UID: \"531e7c03-cde5-4559-9d03-403158fb7168\") " pod="openshift-insights/insights-runtime-extractor-kkm5j"
Sep 12 00:20:35.214073 ip-10-0-141-128 kubenswrapper[2571]: E0912 00:20:35.214006    2571 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Sep 12 00:20:35.214073 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:20:35.214006    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/531e7c03-cde5-4559-9d03-403158fb7168-crio-socket\") pod \"insights-runtime-extractor-kkm5j\" (UID: \"531e7c03-cde5-4559-9d03-403158fb7168\") " pod="openshift-insights/insights-runtime-extractor-kkm5j"
Sep 12 00:20:35.214073 ip-10-0-141-128 kubenswrapper[2571]: E0912 00:20:35.214055    2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/531e7c03-cde5-4559-9d03-403158fb7168-kube-rbac-proxy-cm podName:531e7c03-cde5-4559-9d03-403158fb7168 nodeName:}" failed. No retries permitted until 2026-09-12 00:20:35.714040783 +0000 UTC m=+246.985681270 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/531e7c03-cde5-4559-9d03-403158fb7168-kube-rbac-proxy-cm") pod "insights-runtime-extractor-kkm5j" (UID: "531e7c03-cde5-4559-9d03-403158fb7168") : configmap "kube-rbac-proxy" not found
Sep 12 00:20:35.214073 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:20:35.214061    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/531e7c03-cde5-4559-9d03-403158fb7168-crio-socket\") pod \"insights-runtime-extractor-kkm5j\" (UID: \"531e7c03-cde5-4559-9d03-403158fb7168\") " pod="openshift-insights/insights-runtime-extractor-kkm5j"
Sep 12 00:20:35.214277 ip-10-0-141-128 kubenswrapper[2571]: E0912 00:20:35.214112    2571 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Sep 12 00:20:35.214277 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:20:35.214151    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/531e7c03-cde5-4559-9d03-403158fb7168-data-volume\") pod \"insights-runtime-extractor-kkm5j\" (UID: \"531e7c03-cde5-4559-9d03-403158fb7168\") " pod="openshift-insights/insights-runtime-extractor-kkm5j"
Sep 12 00:20:35.214277 ip-10-0-141-128 kubenswrapper[2571]: E0912 00:20:35.214164    2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/531e7c03-cde5-4559-9d03-403158fb7168-insights-runtime-extractor-tls podName:531e7c03-cde5-4559-9d03-403158fb7168 nodeName:}" failed. No retries permitted until 2026-09-12 00:20:35.714148015 +0000 UTC m=+246.985788507 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/531e7c03-cde5-4559-9d03-403158fb7168-insights-runtime-extractor-tls") pod "insights-runtime-extractor-kkm5j" (UID: "531e7c03-cde5-4559-9d03-403158fb7168") : secret "insights-runtime-extractor-tls" not found
Sep 12 00:20:35.214277 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:20:35.214201    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-n6x6n\" (UniqueName: \"kubernetes.io/projected/531e7c03-cde5-4559-9d03-403158fb7168-kube-api-access-n6x6n\") pod \"insights-runtime-extractor-kkm5j\" (UID: \"531e7c03-cde5-4559-9d03-403158fb7168\") " pod="openshift-insights/insights-runtime-extractor-kkm5j"
Sep 12 00:20:35.214410 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:20:35.214394    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/531e7c03-cde5-4559-9d03-403158fb7168-data-volume\") pod \"insights-runtime-extractor-kkm5j\" (UID: \"531e7c03-cde5-4559-9d03-403158fb7168\") " pod="openshift-insights/insights-runtime-extractor-kkm5j"
Sep 12 00:20:35.223509 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:20:35.223483    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-n6x6n\" (UniqueName: \"kubernetes.io/projected/531e7c03-cde5-4559-9d03-403158fb7168-kube-api-access-n6x6n\") pod \"insights-runtime-extractor-kkm5j\" (UID: \"531e7c03-cde5-4559-9d03-403158fb7168\") " pod="openshift-insights/insights-runtime-extractor-kkm5j"
Sep 12 00:20:35.696406 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:20:35.696373    2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-service-ca/service-ca-568869fdc5-bz947"]
Sep 12 00:20:35.698071 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:20:35.698054    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca/service-ca-568869fdc5-bz947"
Sep 12 00:20:35.700299 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:20:35.700277    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca\"/\"signing-key\""
Sep 12 00:20:35.700417 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:20:35.700278    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"openshift-service-ca.crt\""
Sep 12 00:20:35.701160 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:20:35.701143    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"kube-root-ca.crt\""
Sep 12 00:20:35.701255 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:20:35.701184    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca\"/\"service-ca-dockercfg-4pb7j\""
Sep 12 00:20:35.701255 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:20:35.701210    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"signing-cabundle\""
Sep 12 00:20:35.705864 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:20:35.705841    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca/service-ca-568869fdc5-bz947"]
Sep 12 00:20:35.718676 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:20:35.718650    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/531e7c03-cde5-4559-9d03-403158fb7168-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-kkm5j\" (UID: \"531e7c03-cde5-4559-9d03-403158fb7168\") " pod="openshift-insights/insights-runtime-extractor-kkm5j"
Sep 12 00:20:35.718766 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:20:35.718684    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/531e7c03-cde5-4559-9d03-403158fb7168-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-kkm5j\" (UID: \"531e7c03-cde5-4559-9d03-403158fb7168\") " pod="openshift-insights/insights-runtime-extractor-kkm5j"
Sep 12 00:20:35.718766 ip-10-0-141-128 kubenswrapper[2571]: E0912 00:20:35.718761    2571 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Sep 12 00:20:35.718835 ip-10-0-141-128 kubenswrapper[2571]: E0912 00:20:35.718794    2571 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Sep 12 00:20:35.718835 ip-10-0-141-128 kubenswrapper[2571]: E0912 00:20:35.718821    2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/531e7c03-cde5-4559-9d03-403158fb7168-kube-rbac-proxy-cm podName:531e7c03-cde5-4559-9d03-403158fb7168 nodeName:}" failed. No retries permitted until 2026-09-12 00:20:36.718803276 +0000 UTC m=+247.990443772 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/531e7c03-cde5-4559-9d03-403158fb7168-kube-rbac-proxy-cm") pod "insights-runtime-extractor-kkm5j" (UID: "531e7c03-cde5-4559-9d03-403158fb7168") : configmap "kube-rbac-proxy" not found
Sep 12 00:20:35.718903 ip-10-0-141-128 kubenswrapper[2571]: E0912 00:20:35.718835    2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/531e7c03-cde5-4559-9d03-403158fb7168-insights-runtime-extractor-tls podName:531e7c03-cde5-4559-9d03-403158fb7168 nodeName:}" failed. No retries permitted until 2026-09-12 00:20:36.718829232 +0000 UTC m=+247.990469719 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/531e7c03-cde5-4559-9d03-403158fb7168-insights-runtime-extractor-tls") pod "insights-runtime-extractor-kkm5j" (UID: "531e7c03-cde5-4559-9d03-403158fb7168") : secret "insights-runtime-extractor-tls" not found
Sep 12 00:20:35.819763 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:20:35.819738    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/2042a5c0-7fb9-4236-997f-c6533ed439cf-signing-key\") pod \"service-ca-568869fdc5-bz947\" (UID: \"2042a5c0-7fb9-4236-997f-c6533ed439cf\") " pod="openshift-service-ca/service-ca-568869fdc5-bz947"
Sep 12 00:20:35.819843 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:20:35.819791    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6mwqx\" (UniqueName: \"kubernetes.io/projected/2042a5c0-7fb9-4236-997f-c6533ed439cf-kube-api-access-6mwqx\") pod \"service-ca-568869fdc5-bz947\" (UID: \"2042a5c0-7fb9-4236-997f-c6533ed439cf\") " pod="openshift-service-ca/service-ca-568869fdc5-bz947"
Sep 12 00:20:35.819881 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:20:35.819861    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/2042a5c0-7fb9-4236-997f-c6533ed439cf-signing-cabundle\") pod \"service-ca-568869fdc5-bz947\" (UID: \"2042a5c0-7fb9-4236-997f-c6533ed439cf\") " pod="openshift-service-ca/service-ca-568869fdc5-bz947"
Sep 12 00:20:35.921022 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:20:35.921000    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/2042a5c0-7fb9-4236-997f-c6533ed439cf-signing-key\") pod \"service-ca-568869fdc5-bz947\" (UID: \"2042a5c0-7fb9-4236-997f-c6533ed439cf\") " pod="openshift-service-ca/service-ca-568869fdc5-bz947"
Sep 12 00:20:35.921105 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:20:35.921046    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6mwqx\" (UniqueName: \"kubernetes.io/projected/2042a5c0-7fb9-4236-997f-c6533ed439cf-kube-api-access-6mwqx\") pod \"service-ca-568869fdc5-bz947\" (UID: \"2042a5c0-7fb9-4236-997f-c6533ed439cf\") " pod="openshift-service-ca/service-ca-568869fdc5-bz947"
Sep 12 00:20:35.921105 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:20:35.921093    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/2042a5c0-7fb9-4236-997f-c6533ed439cf-signing-cabundle\") pod \"service-ca-568869fdc5-bz947\" (UID: \"2042a5c0-7fb9-4236-997f-c6533ed439cf\") " pod="openshift-service-ca/service-ca-568869fdc5-bz947"
Sep 12 00:20:35.921749 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:20:35.921733    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/2042a5c0-7fb9-4236-997f-c6533ed439cf-signing-cabundle\") pod \"service-ca-568869fdc5-bz947\" (UID: \"2042a5c0-7fb9-4236-997f-c6533ed439cf\") " pod="openshift-service-ca/service-ca-568869fdc5-bz947"
Sep 12 00:20:35.923286 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:20:35.923265    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/2042a5c0-7fb9-4236-997f-c6533ed439cf-signing-key\") pod \"service-ca-568869fdc5-bz947\" (UID: \"2042a5c0-7fb9-4236-997f-c6533ed439cf\") " pod="openshift-service-ca/service-ca-568869fdc5-bz947"
Sep 12 00:20:35.929003 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:20:35.928985    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6mwqx\" (UniqueName: \"kubernetes.io/projected/2042a5c0-7fb9-4236-997f-c6533ed439cf-kube-api-access-6mwqx\") pod \"service-ca-568869fdc5-bz947\" (UID: \"2042a5c0-7fb9-4236-997f-c6533ed439cf\") " pod="openshift-service-ca/service-ca-568869fdc5-bz947"
Sep 12 00:20:36.006491 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:20:36.006442    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca/service-ca-568869fdc5-bz947"
Sep 12 00:20:36.271708 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:20:36.271634    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca/service-ca-568869fdc5-bz947"]
Sep 12 00:20:36.275098 ip-10-0-141-128 kubenswrapper[2571]: W0912 00:20:36.275067    2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2042a5c0_7fb9_4236_997f_c6533ed439cf.slice/crio-892f0958ba8b87cb78961b81be3c47a6358af002a392447b238aec0fef68ef9c WatchSource:0}: Error finding container 892f0958ba8b87cb78961b81be3c47a6358af002a392447b238aec0fef68ef9c: Status 404 returned error can't find the container with id 892f0958ba8b87cb78961b81be3c47a6358af002a392447b238aec0fef68ef9c
Sep 12 00:20:36.727769 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:20:36.727744    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/531e7c03-cde5-4559-9d03-403158fb7168-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-kkm5j\" (UID: \"531e7c03-cde5-4559-9d03-403158fb7168\") " pod="openshift-insights/insights-runtime-extractor-kkm5j"
Sep 12 00:20:36.727930 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:20:36.727773    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/531e7c03-cde5-4559-9d03-403158fb7168-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-kkm5j\" (UID: \"531e7c03-cde5-4559-9d03-403158fb7168\") " pod="openshift-insights/insights-runtime-extractor-kkm5j"
Sep 12 00:20:36.727930 ip-10-0-141-128 kubenswrapper[2571]: E0912 00:20:36.727883    2571 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Sep 12 00:20:36.728048 ip-10-0-141-128 kubenswrapper[2571]: E0912 00:20:36.727937    2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/531e7c03-cde5-4559-9d03-403158fb7168-insights-runtime-extractor-tls podName:531e7c03-cde5-4559-9d03-403158fb7168 nodeName:}" failed. No retries permitted until 2026-09-12 00:20:38.727923531 +0000 UTC m=+249.999564018 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/531e7c03-cde5-4559-9d03-403158fb7168-insights-runtime-extractor-tls") pod "insights-runtime-extractor-kkm5j" (UID: "531e7c03-cde5-4559-9d03-403158fb7168") : secret "insights-runtime-extractor-tls" not found
Sep 12 00:20:36.728048 ip-10-0-141-128 kubenswrapper[2571]: E0912 00:20:36.727882    2571 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Sep 12 00:20:36.728048 ip-10-0-141-128 kubenswrapper[2571]: E0912 00:20:36.728031    2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/531e7c03-cde5-4559-9d03-403158fb7168-kube-rbac-proxy-cm podName:531e7c03-cde5-4559-9d03-403158fb7168 nodeName:}" failed. No retries permitted until 2026-09-12 00:20:38.72801293 +0000 UTC m=+249.999653420 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/531e7c03-cde5-4559-9d03-403158fb7168-kube-rbac-proxy-cm") pod "insights-runtime-extractor-kkm5j" (UID: "531e7c03-cde5-4559-9d03-403158fb7168") : configmap "kube-rbac-proxy" not found
Sep 12 00:20:36.930932 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:20:36.930897    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca/service-ca-568869fdc5-bz947" event={"ID":"2042a5c0-7fb9-4236-997f-c6533ed439cf","Type":"ContainerStarted","Data":"8d9b845a8ea7d6a8ed6c826e0c0c443a95c4b7e9d51815d253ebd750d8e00650"}
Sep 12 00:20:36.931073 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:20:36.930954    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca/service-ca-568869fdc5-bz947" event={"ID":"2042a5c0-7fb9-4236-997f-c6533ed439cf","Type":"ContainerStarted","Data":"892f0958ba8b87cb78961b81be3c47a6358af002a392447b238aec0fef68ef9c"}
Sep 12 00:20:36.952299 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:20:36.952254    2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-service-ca/service-ca-568869fdc5-bz947" podStartSLOduration=1.952239973 podStartE2EDuration="1.952239973s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-12 00:20:35 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-12 00:20:36.951398279 +0000 UTC m=+248.223038788" watchObservedRunningTime="2026-09-12 00:20:36.952239973 +0000 UTC m=+248.223880482"
Sep 12 00:20:36.978502 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:20:36.978449    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-dnnhn_19b621f9-8fcc-4a57-b13d-862aad3d75ce/dns-node-resolver/0.log"
Sep 12 00:20:37.534166 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:20:37.534139    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c2123986-4ae4-40ce-a423-b5e9a4accda2-metrics-certs\") pod \"router-default-57b84c8474-8xjql\" (UID: \"c2123986-4ae4-40ce-a423-b5e9a4accda2\") " pod="openshift-ingress/router-default-57b84c8474-8xjql"
Sep 12 00:20:37.534531 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:20:37.534185    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c2123986-4ae4-40ce-a423-b5e9a4accda2-service-ca-bundle\") pod \"router-default-57b84c8474-8xjql\" (UID: \"c2123986-4ae4-40ce-a423-b5e9a4accda2\") " pod="openshift-ingress/router-default-57b84c8474-8xjql"
Sep 12 00:20:37.534531 ip-10-0-141-128 kubenswrapper[2571]: E0912 00:20:37.534277    2571 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Sep 12 00:20:37.534531 ip-10-0-141-128 kubenswrapper[2571]: E0912 00:20:37.534308    2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/c2123986-4ae4-40ce-a423-b5e9a4accda2-service-ca-bundle podName:c2123986-4ae4-40ce-a423-b5e9a4accda2 nodeName:}" failed. No retries permitted until 2026-09-12 00:20:45.534295489 +0000 UTC m=+256.805935982 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/c2123986-4ae4-40ce-a423-b5e9a4accda2-service-ca-bundle") pod "router-default-57b84c8474-8xjql" (UID: "c2123986-4ae4-40ce-a423-b5e9a4accda2") : configmap references non-existent config key: service-ca.crt
Sep 12 00:20:37.534531 ip-10-0-141-128 kubenswrapper[2571]: E0912 00:20:37.534322    2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c2123986-4ae4-40ce-a423-b5e9a4accda2-metrics-certs podName:c2123986-4ae4-40ce-a423-b5e9a4accda2 nodeName:}" failed. No retries permitted until 2026-09-12 00:20:45.534316107 +0000 UTC m=+256.805956594 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/c2123986-4ae4-40ce-a423-b5e9a4accda2-metrics-certs") pod "router-default-57b84c8474-8xjql" (UID: "c2123986-4ae4-40ce-a423-b5e9a4accda2") : secret "router-metrics-certs-default" not found
Sep 12 00:20:38.179016 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:20:38.178988    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-qm6vt_7a6f512a-ad51-4ec7-a9a5-8b6a34cb5bb8/node-ca/0.log"
Sep 12 00:20:38.743123 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:20:38.743079    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/531e7c03-cde5-4559-9d03-403158fb7168-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-kkm5j\" (UID: \"531e7c03-cde5-4559-9d03-403158fb7168\") " pod="openshift-insights/insights-runtime-extractor-kkm5j"
Sep 12 00:20:38.743123 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:20:38.743123    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/531e7c03-cde5-4559-9d03-403158fb7168-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-kkm5j\" (UID: \"531e7c03-cde5-4559-9d03-403158fb7168\") " pod="openshift-insights/insights-runtime-extractor-kkm5j"
Sep 12 00:20:38.743530 ip-10-0-141-128 kubenswrapper[2571]: E0912 00:20:38.743194    2571 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Sep 12 00:20:38.743530 ip-10-0-141-128 kubenswrapper[2571]: E0912 00:20:38.743213    2571 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Sep 12 00:20:38.743530 ip-10-0-141-128 kubenswrapper[2571]: E0912 00:20:38.743254    2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/531e7c03-cde5-4559-9d03-403158fb7168-kube-rbac-proxy-cm podName:531e7c03-cde5-4559-9d03-403158fb7168 nodeName:}" failed. No retries permitted until 2026-09-12 00:20:42.743239028 +0000 UTC m=+254.014879520 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/531e7c03-cde5-4559-9d03-403158fb7168-kube-rbac-proxy-cm") pod "insights-runtime-extractor-kkm5j" (UID: "531e7c03-cde5-4559-9d03-403158fb7168") : configmap "kube-rbac-proxy" not found
Sep 12 00:20:38.743530 ip-10-0-141-128 kubenswrapper[2571]: E0912 00:20:38.743268    2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/531e7c03-cde5-4559-9d03-403158fb7168-insights-runtime-extractor-tls podName:531e7c03-cde5-4559-9d03-403158fb7168 nodeName:}" failed. No retries permitted until 2026-09-12 00:20:42.743261978 +0000 UTC m=+254.014902465 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/531e7c03-cde5-4559-9d03-403158fb7168-insights-runtime-extractor-tls") pod "insights-runtime-extractor-kkm5j" (UID: "531e7c03-cde5-4559-9d03-403158fb7168") : secret "insights-runtime-extractor-tls" not found
Sep 12 00:20:38.778317 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:20:38.778290    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-rlr5m_ba88d840-64fd-419f-8444-ebde7a94c354/serve-healthcheck-canary/0.log"
Sep 12 00:20:40.052605 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:20:40.052506    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/6d479776-cc30-4bea-bb3a-8b4c259d59c7-metrics-certs\") pod \"network-metrics-daemon-bdp25\" (UID: \"6d479776-cc30-4bea-bb3a-8b4c259d59c7\") " pod="openshift-multus/network-metrics-daemon-bdp25"
Sep 12 00:20:40.053042 ip-10-0-141-128 kubenswrapper[2571]: E0912 00:20:40.052661    2571 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found
Sep 12 00:20:40.053042 ip-10-0-141-128 kubenswrapper[2571]: E0912 00:20:40.052729    2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6d479776-cc30-4bea-bb3a-8b4c259d59c7-metrics-certs podName:6d479776-cc30-4bea-bb3a-8b4c259d59c7 nodeName:}" failed. No retries permitted until 2026-09-12 00:22:42.052714142 +0000 UTC m=+373.324354629 (durationBeforeRetry 2m2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/6d479776-cc30-4bea-bb3a-8b4c259d59c7-metrics-certs") pod "network-metrics-daemon-bdp25" (UID: "6d479776-cc30-4bea-bb3a-8b4c259d59c7") : secret "metrics-daemon-secret" not found
Sep 12 00:20:42.776365 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:20:42.776325    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/531e7c03-cde5-4559-9d03-403158fb7168-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-kkm5j\" (UID: \"531e7c03-cde5-4559-9d03-403158fb7168\") " pod="openshift-insights/insights-runtime-extractor-kkm5j"
Sep 12 00:20:42.776365 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:20:42.776368    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/531e7c03-cde5-4559-9d03-403158fb7168-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-kkm5j\" (UID: \"531e7c03-cde5-4559-9d03-403158fb7168\") " pod="openshift-insights/insights-runtime-extractor-kkm5j"
Sep 12 00:20:42.776845 ip-10-0-141-128 kubenswrapper[2571]: E0912 00:20:42.776442    2571 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Sep 12 00:20:42.776845 ip-10-0-141-128 kubenswrapper[2571]: E0912 00:20:42.776501    2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/531e7c03-cde5-4559-9d03-403158fb7168-kube-rbac-proxy-cm podName:531e7c03-cde5-4559-9d03-403158fb7168 nodeName:}" failed. No retries permitted until 2026-09-12 00:20:50.776486888 +0000 UTC m=+262.048127384 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/531e7c03-cde5-4559-9d03-403158fb7168-kube-rbac-proxy-cm") pod "insights-runtime-extractor-kkm5j" (UID: "531e7c03-cde5-4559-9d03-403158fb7168") : configmap "kube-rbac-proxy" not found
Sep 12 00:20:42.776845 ip-10-0-141-128 kubenswrapper[2571]: E0912 00:20:42.776555    2571 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Sep 12 00:20:42.776845 ip-10-0-141-128 kubenswrapper[2571]: E0912 00:20:42.776649    2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/531e7c03-cde5-4559-9d03-403158fb7168-insights-runtime-extractor-tls podName:531e7c03-cde5-4559-9d03-403158fb7168 nodeName:}" failed. No retries permitted until 2026-09-12 00:20:50.77663405 +0000 UTC m=+262.048274541 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/531e7c03-cde5-4559-9d03-403158fb7168-insights-runtime-extractor-tls") pod "insights-runtime-extractor-kkm5j" (UID: "531e7c03-cde5-4559-9d03-403158fb7168") : secret "insights-runtime-extractor-tls" not found
Sep 12 00:20:45.599755 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:20:45.599723    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c2123986-4ae4-40ce-a423-b5e9a4accda2-service-ca-bundle\") pod \"router-default-57b84c8474-8xjql\" (UID: \"c2123986-4ae4-40ce-a423-b5e9a4accda2\") " pod="openshift-ingress/router-default-57b84c8474-8xjql"
Sep 12 00:20:45.600147 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:20:45.599904    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c2123986-4ae4-40ce-a423-b5e9a4accda2-metrics-certs\") pod \"router-default-57b84c8474-8xjql\" (UID: \"c2123986-4ae4-40ce-a423-b5e9a4accda2\") " pod="openshift-ingress/router-default-57b84c8474-8xjql"
Sep 12 00:20:45.600573 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:20:45.600546    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c2123986-4ae4-40ce-a423-b5e9a4accda2-service-ca-bundle\") pod \"router-default-57b84c8474-8xjql\" (UID: \"c2123986-4ae4-40ce-a423-b5e9a4accda2\") " pod="openshift-ingress/router-default-57b84c8474-8xjql"
Sep 12 00:20:45.602191 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:20:45.602165    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c2123986-4ae4-40ce-a423-b5e9a4accda2-metrics-certs\") pod \"router-default-57b84c8474-8xjql\" (UID: \"c2123986-4ae4-40ce-a423-b5e9a4accda2\") " pod="openshift-ingress/router-default-57b84c8474-8xjql"
Sep 12 00:20:45.663433 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:20:45.663389    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/router-default-57b84c8474-8xjql"
Sep 12 00:20:45.931504 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:20:45.931479    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/router-default-57b84c8474-8xjql"]
Sep 12 00:20:45.933861 ip-10-0-141-128 kubenswrapper[2571]: W0912 00:20:45.933824    2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc2123986_4ae4_40ce_a423_b5e9a4accda2.slice/crio-566d7ad0ae5ac3e259e4ce791c88c107b7446e393397c944173be85660e49899 WatchSource:0}: Error finding container 566d7ad0ae5ac3e259e4ce791c88c107b7446e393397c944173be85660e49899: Status 404 returned error can't find the container with id 566d7ad0ae5ac3e259e4ce791c88c107b7446e393397c944173be85660e49899
Sep 12 00:20:45.965642 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:20:45.965607    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/router-default-57b84c8474-8xjql" event={"ID":"c2123986-4ae4-40ce-a423-b5e9a4accda2","Type":"ContainerStarted","Data":"566d7ad0ae5ac3e259e4ce791c88c107b7446e393397c944173be85660e49899"}
Sep 12 00:20:46.970125 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:20:46.970092    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/router-default-57b84c8474-8xjql" event={"ID":"c2123986-4ae4-40ce-a423-b5e9a4accda2","Type":"ContainerStarted","Data":"ac5e2ab30da88e47fd1f403d9ed11959ff9c50faaac417b62d35170e62182f2e"}
Sep 12 00:20:46.987493 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:20:46.987440    2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/router-default-57b84c8474-8xjql" podStartSLOduration=17.987422641 podStartE2EDuration="17.987422641s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-12 00:20:29 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-12 00:20:46.986902536 +0000 UTC m=+258.258543046" watchObservedRunningTime="2026-09-12 00:20:46.987422641 +0000 UTC m=+258.259063167"
Sep 12 00:20:47.664418 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:20:47.664386    2571 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/router-default-57b84c8474-8xjql"
Sep 12 00:20:47.667038 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:20:47.667015    2571 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/router-default-57b84c8474-8xjql"
Sep 12 00:20:47.973350 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:20:47.973321    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/router-default-57b84c8474-8xjql"
Sep 12 00:20:47.974600 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:20:47.974568    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/router-default-57b84c8474-8xjql"
Sep 12 00:20:50.839641 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:20:50.839603    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/531e7c03-cde5-4559-9d03-403158fb7168-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-kkm5j\" (UID: \"531e7c03-cde5-4559-9d03-403158fb7168\") " pod="openshift-insights/insights-runtime-extractor-kkm5j"
Sep 12 00:20:50.839641 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:20:50.839645    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/531e7c03-cde5-4559-9d03-403158fb7168-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-kkm5j\" (UID: \"531e7c03-cde5-4559-9d03-403158fb7168\") " pod="openshift-insights/insights-runtime-extractor-kkm5j"
Sep 12 00:20:50.840277 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:20:50.840257    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/531e7c03-cde5-4559-9d03-403158fb7168-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-kkm5j\" (UID: \"531e7c03-cde5-4559-9d03-403158fb7168\") " pod="openshift-insights/insights-runtime-extractor-kkm5j"
Sep 12 00:20:50.841987 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:20:50.841969    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/531e7c03-cde5-4559-9d03-403158fb7168-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-kkm5j\" (UID: \"531e7c03-cde5-4559-9d03-403158fb7168\") " pod="openshift-insights/insights-runtime-extractor-kkm5j"
Sep 12 00:20:50.908670 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:20:50.908629    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-kkm5j"
Sep 12 00:20:51.172236 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:20:51.172168    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-kkm5j"]
Sep 12 00:20:51.175771 ip-10-0-141-128 kubenswrapper[2571]: W0912 00:20:51.175732    2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod531e7c03_cde5_4559_9d03_403158fb7168.slice/crio-5377eb4d4d8f822e0a8e392fe365bd5b1f8ddc98a7ed2bac84c2a35f02cd1c86 WatchSource:0}: Error finding container 5377eb4d4d8f822e0a8e392fe365bd5b1f8ddc98a7ed2bac84c2a35f02cd1c86: Status 404 returned error can't find the container with id 5377eb4d4d8f822e0a8e392fe365bd5b1f8ddc98a7ed2bac84c2a35f02cd1c86
Sep 12 00:20:51.986850 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:20:51.986778    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-kkm5j" event={"ID":"531e7c03-cde5-4559-9d03-403158fb7168","Type":"ContainerStarted","Data":"9f9c203a728983e7af44713721638dadabb77378655bdba81dc14557b6e383a9"}
Sep 12 00:20:51.987182 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:20:51.986858    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-kkm5j" event={"ID":"531e7c03-cde5-4559-9d03-403158fb7168","Type":"ContainerStarted","Data":"5dfc7d05c53d921f46efa6d30f0e3de2007ba18ee76cfe5ba7ee2b57c614c6bc"}
Sep 12 00:20:51.987182 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:20:51.986882    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-kkm5j" event={"ID":"531e7c03-cde5-4559-9d03-403158fb7168","Type":"ContainerStarted","Data":"5377eb4d4d8f822e0a8e392fe365bd5b1f8ddc98a7ed2bac84c2a35f02cd1c86"}
Sep 12 00:20:53.994666 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:20:53.994631    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-kkm5j" event={"ID":"531e7c03-cde5-4559-9d03-403158fb7168","Type":"ContainerStarted","Data":"a641ee343e92e51b92333a662e56b90c70e58daa38f910b47256ad058945a548"}
Sep 12 00:20:54.013893 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:20:54.013848    2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-kkm5j" podStartSLOduration=17.940614029 podStartE2EDuration="20.0138361s" totalImagesPullingTime="2.073222071s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-12 00:20:34 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-12 00:20:54.012518114 +0000 UTC m=+265.284158624" watchObservedRunningTime="2026-09-12 00:20:54.0138361 +0000 UTC m=+265.285476608"
Sep 12 00:20:54.743761 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:20:54.743735    2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/downloads-76f99584f4-wqhtj"]
Sep 12 00:20:54.745399 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:20:54.745384    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/downloads-76f99584f4-wqhtj"
Sep 12 00:20:54.748288 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:20:54.748262    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\""
Sep 12 00:20:54.748417 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:20:54.748269    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"default-dockercfg-w46gz\""
Sep 12 00:20:54.748417 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:20:54.748324    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\""
Sep 12 00:20:54.768541 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:20:54.768517    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/downloads-76f99584f4-wqhtj"]
Sep 12 00:20:54.870580 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:20:54.870552    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gxr4c\" (UniqueName: \"kubernetes.io/projected/25e2de93-94e2-4231-97ff-208a1e9694a7-kube-api-access-gxr4c\") pod \"downloads-76f99584f4-wqhtj\" (UID: \"25e2de93-94e2-4231-97ff-208a1e9694a7\") " pod="openshift-console/downloads-76f99584f4-wqhtj"
Sep 12 00:20:54.971368 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:20:54.971342    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gxr4c\" (UniqueName: \"kubernetes.io/projected/25e2de93-94e2-4231-97ff-208a1e9694a7-kube-api-access-gxr4c\") pod \"downloads-76f99584f4-wqhtj\" (UID: \"25e2de93-94e2-4231-97ff-208a1e9694a7\") " pod="openshift-console/downloads-76f99584f4-wqhtj"
Sep 12 00:20:54.980056 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:20:54.980033    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gxr4c\" (UniqueName: \"kubernetes.io/projected/25e2de93-94e2-4231-97ff-208a1e9694a7-kube-api-access-gxr4c\") pod \"downloads-76f99584f4-wqhtj\" (UID: \"25e2de93-94e2-4231-97ff-208a1e9694a7\") " pod="openshift-console/downloads-76f99584f4-wqhtj"
Sep 12 00:20:55.054375 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:20:55.054317    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/downloads-76f99584f4-wqhtj"
Sep 12 00:20:55.319108 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:20:55.319033    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/downloads-76f99584f4-wqhtj"]
Sep 12 00:20:55.322444 ip-10-0-141-128 kubenswrapper[2571]: W0912 00:20:55.322412    2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod25e2de93_94e2_4231_97ff_208a1e9694a7.slice/crio-faecb60ae9e800612728c4a75164107c79341a4cb737a7bd7c8f119ed3170804 WatchSource:0}: Error finding container faecb60ae9e800612728c4a75164107c79341a4cb737a7bd7c8f119ed3170804: Status 404 returned error can't find the container with id faecb60ae9e800612728c4a75164107c79341a4cb737a7bd7c8f119ed3170804
Sep 12 00:20:56.001746 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:20:56.001706    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/downloads-76f99584f4-wqhtj" event={"ID":"25e2de93-94e2-4231-97ff-208a1e9694a7","Type":"ContainerStarted","Data":"faecb60ae9e800612728c4a75164107c79341a4cb737a7bd7c8f119ed3170804"}
Sep 12 00:21:01.637655 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:21:01.637326    2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/openshift-state-metrics-67dcf8c979-n4njq"]
Sep 12 00:21:01.640833 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:21:01.640810    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/openshift-state-metrics-67dcf8c979-n4njq"
Sep 12 00:21:01.644379 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:21:01.643715    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-tls\""
Sep 12 00:21:01.644379 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:21:01.643946    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-dockercfg-g6q2c\""
Sep 12 00:21:01.644379 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:21:01.644118    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\""
Sep 12 00:21:01.644379 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:21:01.644182    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-kube-rbac-proxy-config\""
Sep 12 00:21:01.645649 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:21:01.645484    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\""
Sep 12 00:21:01.645649 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:21:01.645522    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\""
Sep 12 00:21:01.657040 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:21:01.656375    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/openshift-state-metrics-67dcf8c979-n4njq"]
Sep 12 00:21:01.658374 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:21:01.657176    2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/kube-state-metrics-59fbdd77d-fc26r"]
Sep 12 00:21:01.661008 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:21:01.660970    2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-qgr6s"]
Sep 12 00:21:01.661236 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:21:01.661198    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/kube-state-metrics-59fbdd77d-fc26r"
Sep 12 00:21:01.664227 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:21:01.664075    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-custom-resource-state-configmap\""
Sep 12 00:21:01.664227 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:21:01.664083    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-kube-rbac-proxy-config\""
Sep 12 00:21:01.664427 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:21:01.664311    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-tls\""
Sep 12 00:21:01.664626 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:21:01.664579    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-dockercfg-pwwp7\""
Sep 12 00:21:01.667131 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:21:01.667108    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-qgr6s"
Sep 12 00:21:01.672176 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:21:01.672002    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/kube-state-metrics-59fbdd77d-fc26r"]
Sep 12 00:21:01.672422 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:21:01.672400    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\""
Sep 12 00:21:01.672954 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:21:01.672919    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-j7djq\""
Sep 12 00:21:01.673640 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:21:01.673340    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\""
Sep 12 00:21:01.673640 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:21:01.673499    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\""
Sep 12 00:21:01.725599 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:21:01.725567    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xg4rw\" (UniqueName: \"kubernetes.io/projected/f0b346be-c2c0-43da-9858-f2e93a091d75-kube-api-access-xg4rw\") pod \"openshift-state-metrics-67dcf8c979-n4njq\" (UID: \"f0b346be-c2c0-43da-9858-f2e93a091d75\") " pod="openshift-monitoring/openshift-state-metrics-67dcf8c979-n4njq"
Sep 12 00:21:01.725708 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:21:01.725686    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/f0b346be-c2c0-43da-9858-f2e93a091d75-metrics-client-ca\") pod \"openshift-state-metrics-67dcf8c979-n4njq\" (UID: \"f0b346be-c2c0-43da-9858-f2e93a091d75\") " pod="openshift-monitoring/openshift-state-metrics-67dcf8c979-n4njq"
Sep 12 00:21:01.725820 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:21:01.725793    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/f0b346be-c2c0-43da-9858-f2e93a091d75-openshift-state-metrics-tls\") pod \"openshift-state-metrics-67dcf8c979-n4njq\" (UID: \"f0b346be-c2c0-43da-9858-f2e93a091d75\") " pod="openshift-monitoring/openshift-state-metrics-67dcf8c979-n4njq"
Sep 12 00:21:01.725875 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:21:01.725838    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/f0b346be-c2c0-43da-9858-f2e93a091d75-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-67dcf8c979-n4njq\" (UID: \"f0b346be-c2c0-43da-9858-f2e93a091d75\") " pod="openshift-monitoring/openshift-state-metrics-67dcf8c979-n4njq"
Sep 12 00:21:01.826937 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:21:01.826909    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/951506d4-b1be-4ee7-b72d-1b5dc1d556f2-kube-state-metrics-tls\") pod \"kube-state-metrics-59fbdd77d-fc26r\" (UID: \"951506d4-b1be-4ee7-b72d-1b5dc1d556f2\") " pod="openshift-monitoring/kube-state-metrics-59fbdd77d-fc26r"
Sep 12 00:21:01.827079 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:21:01.826959    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/951506d4-b1be-4ee7-b72d-1b5dc1d556f2-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-59fbdd77d-fc26r\" (UID: \"951506d4-b1be-4ee7-b72d-1b5dc1d556f2\") " pod="openshift-monitoring/kube-state-metrics-59fbdd77d-fc26r"
Sep 12 00:21:01.827079 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:21:01.826991    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/951506d4-b1be-4ee7-b72d-1b5dc1d556f2-volume-directive-shadow\") pod \"kube-state-metrics-59fbdd77d-fc26r\" (UID: \"951506d4-b1be-4ee7-b72d-1b5dc1d556f2\") " pod="openshift-monitoring/kube-state-metrics-59fbdd77d-fc26r"
Sep 12 00:21:01.827079 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:21:01.827019    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/17f775e6-4bfa-4d7f-aeac-4e66cb551e46-node-exporter-tls\") pod \"node-exporter-qgr6s\" (UID: \"17f775e6-4bfa-4d7f-aeac-4e66cb551e46\") " pod="openshift-monitoring/node-exporter-qgr6s"
Sep 12 00:21:01.827079 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:21:01.827047    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/17f775e6-4bfa-4d7f-aeac-4e66cb551e46-metrics-client-ca\") pod \"node-exporter-qgr6s\" (UID: \"17f775e6-4bfa-4d7f-aeac-4e66cb551e46\") " pod="openshift-monitoring/node-exporter-qgr6s"
Sep 12 00:21:01.827287 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:21:01.827093    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xg4rw\" (UniqueName: \"kubernetes.io/projected/f0b346be-c2c0-43da-9858-f2e93a091d75-kube-api-access-xg4rw\") pod \"openshift-state-metrics-67dcf8c979-n4njq\" (UID: \"f0b346be-c2c0-43da-9858-f2e93a091d75\") " pod="openshift-monitoring/openshift-state-metrics-67dcf8c979-n4njq"
Sep 12 00:21:01.827287 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:21:01.827118    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/17f775e6-4bfa-4d7f-aeac-4e66cb551e46-node-exporter-textfile\") pod \"node-exporter-qgr6s\" (UID: \"17f775e6-4bfa-4d7f-aeac-4e66cb551e46\") " pod="openshift-monitoring/node-exporter-qgr6s"
Sep 12 00:21:01.827287 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:21:01.827154    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/f0b346be-c2c0-43da-9858-f2e93a091d75-metrics-client-ca\") pod \"openshift-state-metrics-67dcf8c979-n4njq\" (UID: \"f0b346be-c2c0-43da-9858-f2e93a091d75\") " pod="openshift-monitoring/openshift-state-metrics-67dcf8c979-n4njq"
Sep 12 00:21:01.827287 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:21:01.827188    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/17f775e6-4bfa-4d7f-aeac-4e66cb551e46-sys\") pod \"node-exporter-qgr6s\" (UID: \"17f775e6-4bfa-4d7f-aeac-4e66cb551e46\") " pod="openshift-monitoring/node-exporter-qgr6s"
Sep 12 00:21:01.827287 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:21:01.827231    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/17f775e6-4bfa-4d7f-aeac-4e66cb551e46-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-qgr6s\" (UID: \"17f775e6-4bfa-4d7f-aeac-4e66cb551e46\") " pod="openshift-monitoring/node-exporter-qgr6s"
Sep 12 00:21:01.827287 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:21:01.827258    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/951506d4-b1be-4ee7-b72d-1b5dc1d556f2-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-59fbdd77d-fc26r\" (UID: \"951506d4-b1be-4ee7-b72d-1b5dc1d556f2\") " pod="openshift-monitoring/kube-state-metrics-59fbdd77d-fc26r"
Sep 12 00:21:01.827287 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:21:01.827280    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-g64x5\" (UniqueName: \"kubernetes.io/projected/951506d4-b1be-4ee7-b72d-1b5dc1d556f2-kube-api-access-g64x5\") pod \"kube-state-metrics-59fbdd77d-fc26r\" (UID: \"951506d4-b1be-4ee7-b72d-1b5dc1d556f2\") " pod="openshift-monitoring/kube-state-metrics-59fbdd77d-fc26r"
Sep 12 00:21:01.827621 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:21:01.827299    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/17f775e6-4bfa-4d7f-aeac-4e66cb551e46-root\") pod \"node-exporter-qgr6s\" (UID: \"17f775e6-4bfa-4d7f-aeac-4e66cb551e46\") " pod="openshift-monitoring/node-exporter-qgr6s"
Sep 12 00:21:01.827621 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:21:01.827316    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/f0b346be-c2c0-43da-9858-f2e93a091d75-openshift-state-metrics-tls\") pod \"openshift-state-metrics-67dcf8c979-n4njq\" (UID: \"f0b346be-c2c0-43da-9858-f2e93a091d75\") " pod="openshift-monitoring/openshift-state-metrics-67dcf8c979-n4njq"
Sep 12 00:21:01.827743 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:21:01.827678    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/f0b346be-c2c0-43da-9858-f2e93a091d75-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-67dcf8c979-n4njq\" (UID: \"f0b346be-c2c0-43da-9858-f2e93a091d75\") " pod="openshift-monitoring/openshift-state-metrics-67dcf8c979-n4njq"
Sep 12 00:21:01.827743 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:21:01.827734    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/17f775e6-4bfa-4d7f-aeac-4e66cb551e46-node-exporter-wtmp\") pod \"node-exporter-qgr6s\" (UID: \"17f775e6-4bfa-4d7f-aeac-4e66cb551e46\") " pod="openshift-monitoring/node-exporter-qgr6s"
Sep 12 00:21:01.827912 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:21:01.827872    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/951506d4-b1be-4ee7-b72d-1b5dc1d556f2-metrics-client-ca\") pod \"kube-state-metrics-59fbdd77d-fc26r\" (UID: \"951506d4-b1be-4ee7-b72d-1b5dc1d556f2\") " pod="openshift-monitoring/kube-state-metrics-59fbdd77d-fc26r"
Sep 12 00:21:01.827979 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:21:01.827945    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/17f775e6-4bfa-4d7f-aeac-4e66cb551e46-node-exporter-accelerators-collector-config\") pod \"node-exporter-qgr6s\" (UID: \"17f775e6-4bfa-4d7f-aeac-4e66cb551e46\") " pod="openshift-monitoring/node-exporter-qgr6s"
Sep 12 00:21:01.828044 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:21:01.827976    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5g9jx\" (UniqueName: \"kubernetes.io/projected/17f775e6-4bfa-4d7f-aeac-4e66cb551e46-kube-api-access-5g9jx\") pod \"node-exporter-qgr6s\" (UID: \"17f775e6-4bfa-4d7f-aeac-4e66cb551e46\") " pod="openshift-monitoring/node-exporter-qgr6s"
Sep 12 00:21:01.828241 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:21:01.828219    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/f0b346be-c2c0-43da-9858-f2e93a091d75-metrics-client-ca\") pod \"openshift-state-metrics-67dcf8c979-n4njq\" (UID: \"f0b346be-c2c0-43da-9858-f2e93a091d75\") " pod="openshift-monitoring/openshift-state-metrics-67dcf8c979-n4njq"
Sep 12 00:21:01.830726 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:21:01.830700    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/f0b346be-c2c0-43da-9858-f2e93a091d75-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-67dcf8c979-n4njq\" (UID: \"f0b346be-c2c0-43da-9858-f2e93a091d75\") " pod="openshift-monitoring/openshift-state-metrics-67dcf8c979-n4njq"
Sep 12 00:21:01.830726 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:21:01.830722    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/f0b346be-c2c0-43da-9858-f2e93a091d75-openshift-state-metrics-tls\") pod \"openshift-state-metrics-67dcf8c979-n4njq\" (UID: \"f0b346be-c2c0-43da-9858-f2e93a091d75\") " pod="openshift-monitoring/openshift-state-metrics-67dcf8c979-n4njq"
Sep 12 00:21:01.837776 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:21:01.837735    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xg4rw\" (UniqueName: \"kubernetes.io/projected/f0b346be-c2c0-43da-9858-f2e93a091d75-kube-api-access-xg4rw\") pod \"openshift-state-metrics-67dcf8c979-n4njq\" (UID: \"f0b346be-c2c0-43da-9858-f2e93a091d75\") " pod="openshift-monitoring/openshift-state-metrics-67dcf8c979-n4njq"
Sep 12 00:21:01.928999 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:21:01.928916    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/17f775e6-4bfa-4d7f-aeac-4e66cb551e46-sys\") pod \"node-exporter-qgr6s\" (UID: \"17f775e6-4bfa-4d7f-aeac-4e66cb551e46\") " pod="openshift-monitoring/node-exporter-qgr6s"
Sep 12 00:21:01.928999 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:21:01.928975    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/17f775e6-4bfa-4d7f-aeac-4e66cb551e46-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-qgr6s\" (UID: \"17f775e6-4bfa-4d7f-aeac-4e66cb551e46\") " pod="openshift-monitoring/node-exporter-qgr6s"
Sep 12 00:21:01.929191 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:21:01.929043    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/17f775e6-4bfa-4d7f-aeac-4e66cb551e46-sys\") pod \"node-exporter-qgr6s\" (UID: \"17f775e6-4bfa-4d7f-aeac-4e66cb551e46\") " pod="openshift-monitoring/node-exporter-qgr6s"
Sep 12 00:21:01.929191 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:21:01.929167    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/951506d4-b1be-4ee7-b72d-1b5dc1d556f2-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-59fbdd77d-fc26r\" (UID: \"951506d4-b1be-4ee7-b72d-1b5dc1d556f2\") " pod="openshift-monitoring/kube-state-metrics-59fbdd77d-fc26r"
Sep 12 00:21:01.929298 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:21:01.929198    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-g64x5\" (UniqueName: \"kubernetes.io/projected/951506d4-b1be-4ee7-b72d-1b5dc1d556f2-kube-api-access-g64x5\") pod \"kube-state-metrics-59fbdd77d-fc26r\" (UID: \"951506d4-b1be-4ee7-b72d-1b5dc1d556f2\") " pod="openshift-monitoring/kube-state-metrics-59fbdd77d-fc26r"
Sep 12 00:21:01.929298 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:21:01.929235    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/17f775e6-4bfa-4d7f-aeac-4e66cb551e46-root\") pod \"node-exporter-qgr6s\" (UID: \"17f775e6-4bfa-4d7f-aeac-4e66cb551e46\") " pod="openshift-monitoring/node-exporter-qgr6s"
Sep 12 00:21:01.929298 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:21:01.929275    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/17f775e6-4bfa-4d7f-aeac-4e66cb551e46-node-exporter-wtmp\") pod \"node-exporter-qgr6s\" (UID: \"17f775e6-4bfa-4d7f-aeac-4e66cb551e46\") " pod="openshift-monitoring/node-exporter-qgr6s"
Sep 12 00:21:01.929433 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:21:01.929304    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/951506d4-b1be-4ee7-b72d-1b5dc1d556f2-metrics-client-ca\") pod \"kube-state-metrics-59fbdd77d-fc26r\" (UID: \"951506d4-b1be-4ee7-b72d-1b5dc1d556f2\") " pod="openshift-monitoring/kube-state-metrics-59fbdd77d-fc26r"
Sep 12 00:21:01.929433 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:21:01.929337    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/17f775e6-4bfa-4d7f-aeac-4e66cb551e46-node-exporter-accelerators-collector-config\") pod \"node-exporter-qgr6s\" (UID: \"17f775e6-4bfa-4d7f-aeac-4e66cb551e46\") " pod="openshift-monitoring/node-exporter-qgr6s"
Sep 12 00:21:01.929433 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:21:01.929367    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5g9jx\" (UniqueName: \"kubernetes.io/projected/17f775e6-4bfa-4d7f-aeac-4e66cb551e46-kube-api-access-5g9jx\") pod \"node-exporter-qgr6s\" (UID: \"17f775e6-4bfa-4d7f-aeac-4e66cb551e46\") " pod="openshift-monitoring/node-exporter-qgr6s"
Sep 12 00:21:01.929711 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:21:01.929408    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/951506d4-b1be-4ee7-b72d-1b5dc1d556f2-kube-state-metrics-tls\") pod \"kube-state-metrics-59fbdd77d-fc26r\" (UID: \"951506d4-b1be-4ee7-b72d-1b5dc1d556f2\") " pod="openshift-monitoring/kube-state-metrics-59fbdd77d-fc26r"
Sep 12 00:21:01.929711 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:21:01.929469    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/951506d4-b1be-4ee7-b72d-1b5dc1d556f2-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-59fbdd77d-fc26r\" (UID: \"951506d4-b1be-4ee7-b72d-1b5dc1d556f2\") " pod="openshift-monitoring/kube-state-metrics-59fbdd77d-fc26r"
Sep 12 00:21:01.929711 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:21:01.929497    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/951506d4-b1be-4ee7-b72d-1b5dc1d556f2-volume-directive-shadow\") pod \"kube-state-metrics-59fbdd77d-fc26r\" (UID: \"951506d4-b1be-4ee7-b72d-1b5dc1d556f2\") " pod="openshift-monitoring/kube-state-metrics-59fbdd77d-fc26r"
Sep 12 00:21:01.929711 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:21:01.929525    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/17f775e6-4bfa-4d7f-aeac-4e66cb551e46-node-exporter-tls\") pod \"node-exporter-qgr6s\" (UID: \"17f775e6-4bfa-4d7f-aeac-4e66cb551e46\") " pod="openshift-monitoring/node-exporter-qgr6s"
Sep 12 00:21:01.929711 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:21:01.929552    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/17f775e6-4bfa-4d7f-aeac-4e66cb551e46-metrics-client-ca\") pod \"node-exporter-qgr6s\" (UID: \"17f775e6-4bfa-4d7f-aeac-4e66cb551e46\") " pod="openshift-monitoring/node-exporter-qgr6s"
Sep 12 00:21:01.929711 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:21:01.929562    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/17f775e6-4bfa-4d7f-aeac-4e66cb551e46-node-exporter-wtmp\") pod \"node-exporter-qgr6s\" (UID: \"17f775e6-4bfa-4d7f-aeac-4e66cb551e46\") " pod="openshift-monitoring/node-exporter-qgr6s"
Sep 12 00:21:01.929711 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:21:01.929615    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/17f775e6-4bfa-4d7f-aeac-4e66cb551e46-node-exporter-textfile\") pod \"node-exporter-qgr6s\" (UID: \"17f775e6-4bfa-4d7f-aeac-4e66cb551e46\") " pod="openshift-monitoring/node-exporter-qgr6s"
Sep 12 00:21:01.930042 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:21:01.929933    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/17f775e6-4bfa-4d7f-aeac-4e66cb551e46-node-exporter-textfile\") pod \"node-exporter-qgr6s\" (UID: \"17f775e6-4bfa-4d7f-aeac-4e66cb551e46\") " pod="openshift-monitoring/node-exporter-qgr6s"
Sep 12 00:21:01.930354 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:21:01.930322    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/951506d4-b1be-4ee7-b72d-1b5dc1d556f2-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-59fbdd77d-fc26r\" (UID: \"951506d4-b1be-4ee7-b72d-1b5dc1d556f2\") " pod="openshift-monitoring/kube-state-metrics-59fbdd77d-fc26r"
Sep 12 00:21:01.931797 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:21:01.930453    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/951506d4-b1be-4ee7-b72d-1b5dc1d556f2-metrics-client-ca\") pod \"kube-state-metrics-59fbdd77d-fc26r\" (UID: \"951506d4-b1be-4ee7-b72d-1b5dc1d556f2\") " pod="openshift-monitoring/kube-state-metrics-59fbdd77d-fc26r"
Sep 12 00:21:01.931797 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:21:01.930668    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/17f775e6-4bfa-4d7f-aeac-4e66cb551e46-root\") pod \"node-exporter-qgr6s\" (UID: \"17f775e6-4bfa-4d7f-aeac-4e66cb551e46\") " pod="openshift-monitoring/node-exporter-qgr6s"
Sep 12 00:21:01.931797 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:21:01.930906    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/17f775e6-4bfa-4d7f-aeac-4e66cb551e46-node-exporter-accelerators-collector-config\") pod \"node-exporter-qgr6s\" (UID: \"17f775e6-4bfa-4d7f-aeac-4e66cb551e46\") " pod="openshift-monitoring/node-exporter-qgr6s"
Sep 12 00:21:01.931797 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:21:01.931145    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/951506d4-b1be-4ee7-b72d-1b5dc1d556f2-volume-directive-shadow\") pod \"kube-state-metrics-59fbdd77d-fc26r\" (UID: \"951506d4-b1be-4ee7-b72d-1b5dc1d556f2\") " pod="openshift-monitoring/kube-state-metrics-59fbdd77d-fc26r"
Sep 12 00:21:01.931797 ip-10-0-141-128 kubenswrapper[2571]: E0912 00:21:01.931225    2571 secret.go:189] Couldn't get secret openshift-monitoring/node-exporter-tls: secret "node-exporter-tls" not found
Sep 12 00:21:01.931797 ip-10-0-141-128 kubenswrapper[2571]: E0912 00:21:01.931307    2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/17f775e6-4bfa-4d7f-aeac-4e66cb551e46-node-exporter-tls podName:17f775e6-4bfa-4d7f-aeac-4e66cb551e46 nodeName:}" failed. No retries permitted until 2026-09-12 00:21:02.431289214 +0000 UTC m=+273.702929710 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "node-exporter-tls" (UniqueName: "kubernetes.io/secret/17f775e6-4bfa-4d7f-aeac-4e66cb551e46-node-exporter-tls") pod "node-exporter-qgr6s" (UID: "17f775e6-4bfa-4d7f-aeac-4e66cb551e46") : secret "node-exporter-tls" not found
Sep 12 00:21:01.931797 ip-10-0-141-128 kubenswrapper[2571]: E0912 00:21:01.931322    2571 secret.go:189] Couldn't get secret openshift-monitoring/kube-state-metrics-tls: secret "kube-state-metrics-tls" not found
Sep 12 00:21:01.931797 ip-10-0-141-128 kubenswrapper[2571]: E0912 00:21:01.931398    2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/951506d4-b1be-4ee7-b72d-1b5dc1d556f2-kube-state-metrics-tls podName:951506d4-b1be-4ee7-b72d-1b5dc1d556f2 nodeName:}" failed. No retries permitted until 2026-09-12 00:21:02.431381676 +0000 UTC m=+273.703022167 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-state-metrics-tls" (UniqueName: "kubernetes.io/secret/951506d4-b1be-4ee7-b72d-1b5dc1d556f2-kube-state-metrics-tls") pod "kube-state-metrics-59fbdd77d-fc26r" (UID: "951506d4-b1be-4ee7-b72d-1b5dc1d556f2") : secret "kube-state-metrics-tls" not found
Sep 12 00:21:01.931797 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:21:01.931717    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/17f775e6-4bfa-4d7f-aeac-4e66cb551e46-metrics-client-ca\") pod \"node-exporter-qgr6s\" (UID: \"17f775e6-4bfa-4d7f-aeac-4e66cb551e46\") " pod="openshift-monitoring/node-exporter-qgr6s"
Sep 12 00:21:01.939344 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:21:01.939306    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/17f775e6-4bfa-4d7f-aeac-4e66cb551e46-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-qgr6s\" (UID: \"17f775e6-4bfa-4d7f-aeac-4e66cb551e46\") " pod="openshift-monitoring/node-exporter-qgr6s"
Sep 12 00:21:01.940083 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:21:01.940059    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/951506d4-b1be-4ee7-b72d-1b5dc1d556f2-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-59fbdd77d-fc26r\" (UID: \"951506d4-b1be-4ee7-b72d-1b5dc1d556f2\") " pod="openshift-monitoring/kube-state-metrics-59fbdd77d-fc26r"
Sep 12 00:21:01.940835 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:21:01.940507    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5g9jx\" (UniqueName: \"kubernetes.io/projected/17f775e6-4bfa-4d7f-aeac-4e66cb551e46-kube-api-access-5g9jx\") pod \"node-exporter-qgr6s\" (UID: \"17f775e6-4bfa-4d7f-aeac-4e66cb551e46\") " pod="openshift-monitoring/node-exporter-qgr6s"
Sep 12 00:21:01.944156 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:21:01.944107    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-g64x5\" (UniqueName: \"kubernetes.io/projected/951506d4-b1be-4ee7-b72d-1b5dc1d556f2-kube-api-access-g64x5\") pod \"kube-state-metrics-59fbdd77d-fc26r\" (UID: \"951506d4-b1be-4ee7-b72d-1b5dc1d556f2\") " pod="openshift-monitoring/kube-state-metrics-59fbdd77d-fc26r"
Sep 12 00:21:01.957129 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:21:01.956941    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/openshift-state-metrics-67dcf8c979-n4njq"
Sep 12 00:21:02.248345 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:21:02.248226    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/openshift-state-metrics-67dcf8c979-n4njq"]
Sep 12 00:21:02.251842 ip-10-0-141-128 kubenswrapper[2571]: W0912 00:21:02.251815    2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf0b346be_c2c0_43da_9858_f2e93a091d75.slice/crio-14bbe4b01d0d5974824e42da1e23d64bbe4b3865b7745ae0e373a086790f892a WatchSource:0}: Error finding container 14bbe4b01d0d5974824e42da1e23d64bbe4b3865b7745ae0e373a086790f892a: Status 404 returned error can't find the container with id 14bbe4b01d0d5974824e42da1e23d64bbe4b3865b7745ae0e373a086790f892a
Sep 12 00:21:02.434324 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:21:02.434290    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/17f775e6-4bfa-4d7f-aeac-4e66cb551e46-node-exporter-tls\") pod \"node-exporter-qgr6s\" (UID: \"17f775e6-4bfa-4d7f-aeac-4e66cb551e46\") " pod="openshift-monitoring/node-exporter-qgr6s"
Sep 12 00:21:02.434475 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:21:02.434437    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/951506d4-b1be-4ee7-b72d-1b5dc1d556f2-kube-state-metrics-tls\") pod \"kube-state-metrics-59fbdd77d-fc26r\" (UID: \"951506d4-b1be-4ee7-b72d-1b5dc1d556f2\") " pod="openshift-monitoring/kube-state-metrics-59fbdd77d-fc26r"
Sep 12 00:21:02.436865 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:21:02.436838    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/17f775e6-4bfa-4d7f-aeac-4e66cb551e46-node-exporter-tls\") pod \"node-exporter-qgr6s\" (UID: \"17f775e6-4bfa-4d7f-aeac-4e66cb551e46\") " pod="openshift-monitoring/node-exporter-qgr6s"
Sep 12 00:21:02.436993 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:21:02.436971    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/951506d4-b1be-4ee7-b72d-1b5dc1d556f2-kube-state-metrics-tls\") pod \"kube-state-metrics-59fbdd77d-fc26r\" (UID: \"951506d4-b1be-4ee7-b72d-1b5dc1d556f2\") " pod="openshift-monitoring/kube-state-metrics-59fbdd77d-fc26r"
Sep 12 00:21:02.579873 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:21:02.579804    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/kube-state-metrics-59fbdd77d-fc26r"
Sep 12 00:21:02.595731 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:21:02.595704    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-qgr6s"
Sep 12 00:21:02.608927 ip-10-0-141-128 kubenswrapper[2571]: W0912 00:21:02.608898    2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod17f775e6_4bfa_4d7f_aeac_4e66cb551e46.slice/crio-3181584b41be654d82cf7ec6a4d1b31861dc060ab4528dea3541b077af9ed962 WatchSource:0}: Error finding container 3181584b41be654d82cf7ec6a4d1b31861dc060ab4528dea3541b077af9ed962: Status 404 returned error can't find the container with id 3181584b41be654d82cf7ec6a4d1b31861dc060ab4528dea3541b077af9ed962
Sep 12 00:21:02.860751 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:21:02.860727    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/kube-state-metrics-59fbdd77d-fc26r"]
Sep 12 00:21:02.862912 ip-10-0-141-128 kubenswrapper[2571]: W0912 00:21:02.862884    2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod951506d4_b1be_4ee7_b72d_1b5dc1d556f2.slice/crio-8fb62707035536b9c19b0ae13db3a3119c7feef5ffb4c1aee9a72f2a5c5e8bd1 WatchSource:0}: Error finding container 8fb62707035536b9c19b0ae13db3a3119c7feef5ffb4c1aee9a72f2a5c5e8bd1: Status 404 returned error can't find the container with id 8fb62707035536b9c19b0ae13db3a3119c7feef5ffb4c1aee9a72f2a5c5e8bd1
Sep 12 00:21:03.032779 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:21:03.032739    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-59fbdd77d-fc26r" event={"ID":"951506d4-b1be-4ee7-b72d-1b5dc1d556f2","Type":"ContainerStarted","Data":"8fb62707035536b9c19b0ae13db3a3119c7feef5ffb4c1aee9a72f2a5c5e8bd1"}
Sep 12 00:21:03.034844 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:21:03.034810    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-67dcf8c979-n4njq" event={"ID":"f0b346be-c2c0-43da-9858-f2e93a091d75","Type":"ContainerStarted","Data":"aab46c0627e7319d960e5b6300051575e522a86e88c78092fbd783f60fc58d35"}
Sep 12 00:21:03.034971 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:21:03.034866    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-67dcf8c979-n4njq" event={"ID":"f0b346be-c2c0-43da-9858-f2e93a091d75","Type":"ContainerStarted","Data":"60a87707bbfcf8e614b3de41fd6446627fb932e104371d1f107d206073627a65"}
Sep 12 00:21:03.034971 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:21:03.034894    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-67dcf8c979-n4njq" event={"ID":"f0b346be-c2c0-43da-9858-f2e93a091d75","Type":"ContainerStarted","Data":"14bbe4b01d0d5974824e42da1e23d64bbe4b3865b7745ae0e373a086790f892a"}
Sep 12 00:21:03.036235 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:21:03.036208    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-qgr6s" event={"ID":"17f775e6-4bfa-4d7f-aeac-4e66cb551e46","Type":"ContainerStarted","Data":"3181584b41be654d82cf7ec6a4d1b31861dc060ab4528dea3541b077af9ed962"}
Sep 12 00:21:05.046645 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:21:05.046546    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-59fbdd77d-fc26r" event={"ID":"951506d4-b1be-4ee7-b72d-1b5dc1d556f2","Type":"ContainerStarted","Data":"532f3d60d33840f6b203517c821cfb109fe1fa7155d20ff7b1feb2bdfe79622d"}
Sep 12 00:21:05.046645 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:21:05.046641    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-59fbdd77d-fc26r" event={"ID":"951506d4-b1be-4ee7-b72d-1b5dc1d556f2","Type":"ContainerStarted","Data":"57bba9fd7e3e54816a4fa9bdee4ff899287b5ac931d0e92c7dcd7ea3f9627555"}
Sep 12 00:21:05.047074 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:21:05.046671    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-59fbdd77d-fc26r" event={"ID":"951506d4-b1be-4ee7-b72d-1b5dc1d556f2","Type":"ContainerStarted","Data":"180e1eccb42a7514a0f192dd52b6968891210175b0b19eded77be408537fbba6"}
Sep 12 00:21:05.049633 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:21:05.049604    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-67dcf8c979-n4njq" event={"ID":"f0b346be-c2c0-43da-9858-f2e93a091d75","Type":"ContainerStarted","Data":"e527e78eb502e22fa4768612bde8b995ee58a14bdcc593186a5a49c778c2804a"}
Sep 12 00:21:05.052144 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:21:05.052117    2571 generic.go:356] "Generic (PLEG): container finished" podID="17f775e6-4bfa-4d7f-aeac-4e66cb551e46" containerID="43adabf69a17726b8092979afd6355c7c4fcc352438cd7f39158374bbef476ed" exitCode=0
Sep 12 00:21:05.052265 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:21:05.052192    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-qgr6s" event={"ID":"17f775e6-4bfa-4d7f-aeac-4e66cb551e46","Type":"ContainerDied","Data":"43adabf69a17726b8092979afd6355c7c4fcc352438cd7f39158374bbef476ed"}
Sep 12 00:21:05.064598 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:21:05.064537    2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/kube-state-metrics-59fbdd77d-fc26r" podStartSLOduration=2.428702145 podStartE2EDuration="4.06451571s" totalImagesPullingTime="1.635813565s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-12 00:21:01 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-12 00:21:05.06314694 +0000 UTC m=+276.334787476" watchObservedRunningTime="2026-09-12 00:21:05.06451571 +0000 UTC m=+276.336156223"
Sep 12 00:21:05.101209 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:21:05.101167    2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/openshift-state-metrics-67dcf8c979-n4njq" podStartSLOduration=1.980411838 podStartE2EDuration="4.101152893s" totalImagesPullingTime="2.120741055s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-12 00:21:01 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-12 00:21:05.07905911 +0000 UTC m=+276.350699620" watchObservedRunningTime="2026-09-12 00:21:05.101152893 +0000 UTC m=+276.372793402"
Sep 12 00:21:05.690784 ip-10-0-141-128 kubenswrapper[2571]: E0912 00:21:05.690732    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[registry-tls], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-image-registry/image-registry-744fcccbf-5vhvb" podUID="80cbfa96-879b-48ea-8732-d50ebebea182"
Sep 12 00:21:05.690784 ip-10-0-141-128 kubenswrapper[2571]: E0912 00:21:05.690732    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[metrics-tls], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-dns/dns-default-fr2kt" podUID="3a2114c7-d34a-4249-97a4-65f5375453ce"
Sep 12 00:21:05.949632 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:21:05.949539    2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/metrics-server-6fc7858648-mvh4h"]
Sep 12 00:21:05.953862 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:21:05.953837    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/metrics-server-6fc7858648-mvh4h"
Sep 12 00:21:05.957408 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:21:05.957383    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-dqd3sgm56rugb\""
Sep 12 00:21:05.957537 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:21:05.957422    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-server-audit-profiles\""
Sep 12 00:21:05.957537 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:21:05.957431    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kubelet-serving-ca-bundle\""
Sep 12 00:21:05.957537 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:21:05.957444    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-tls\""
Sep 12 00:21:05.957537 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:21:05.957488    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-client-certs\""
Sep 12 00:21:05.957537 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:21:05.957436    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-dockercfg-q6gn2\""
Sep 12 00:21:05.960450 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:21:05.960429    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/metrics-server-6fc7858648-mvh4h"]
Sep 12 00:21:06.056273 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:21:06.056241    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-744fcccbf-5vhvb"
Sep 12 00:21:06.056735 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:21:06.056544    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-fr2kt"
Sep 12 00:21:06.069467 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:21:06.069437    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/980eacbb-2f36-403a-a2b8-bce355264f9f-secret-metrics-server-tls\") pod \"metrics-server-6fc7858648-mvh4h\" (UID: \"980eacbb-2f36-403a-a2b8-bce355264f9f\") " pod="openshift-monitoring/metrics-server-6fc7858648-mvh4h"
Sep 12 00:21:06.069580 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:21:06.069479    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/980eacbb-2f36-403a-a2b8-bce355264f9f-audit-log\") pod \"metrics-server-6fc7858648-mvh4h\" (UID: \"980eacbb-2f36-403a-a2b8-bce355264f9f\") " pod="openshift-monitoring/metrics-server-6fc7858648-mvh4h"
Sep 12 00:21:06.069580 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:21:06.069544    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/980eacbb-2f36-403a-a2b8-bce355264f9f-client-ca-bundle\") pod \"metrics-server-6fc7858648-mvh4h\" (UID: \"980eacbb-2f36-403a-a2b8-bce355264f9f\") " pod="openshift-monitoring/metrics-server-6fc7858648-mvh4h"
Sep 12 00:21:06.069709 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:21:06.069623    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/980eacbb-2f36-403a-a2b8-bce355264f9f-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-6fc7858648-mvh4h\" (UID: \"980eacbb-2f36-403a-a2b8-bce355264f9f\") " pod="openshift-monitoring/metrics-server-6fc7858648-mvh4h"
Sep 12 00:21:06.069709 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:21:06.069651    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cfh9g\" (UniqueName: \"kubernetes.io/projected/980eacbb-2f36-403a-a2b8-bce355264f9f-kube-api-access-cfh9g\") pod \"metrics-server-6fc7858648-mvh4h\" (UID: \"980eacbb-2f36-403a-a2b8-bce355264f9f\") " pod="openshift-monitoring/metrics-server-6fc7858648-mvh4h"
Sep 12 00:21:06.069709 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:21:06.069684    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/980eacbb-2f36-403a-a2b8-bce355264f9f-metrics-server-audit-profiles\") pod \"metrics-server-6fc7858648-mvh4h\" (UID: \"980eacbb-2f36-403a-a2b8-bce355264f9f\") " pod="openshift-monitoring/metrics-server-6fc7858648-mvh4h"
Sep 12 00:21:06.069837 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:21:06.069710    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/980eacbb-2f36-403a-a2b8-bce355264f9f-secret-metrics-server-client-certs\") pod \"metrics-server-6fc7858648-mvh4h\" (UID: \"980eacbb-2f36-403a-a2b8-bce355264f9f\") " pod="openshift-monitoring/metrics-server-6fc7858648-mvh4h"
Sep 12 00:21:06.171182 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:21:06.171136    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/980eacbb-2f36-403a-a2b8-bce355264f9f-secret-metrics-server-tls\") pod \"metrics-server-6fc7858648-mvh4h\" (UID: \"980eacbb-2f36-403a-a2b8-bce355264f9f\") " pod="openshift-monitoring/metrics-server-6fc7858648-mvh4h"
Sep 12 00:21:06.171182 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:21:06.171182    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/980eacbb-2f36-403a-a2b8-bce355264f9f-audit-log\") pod \"metrics-server-6fc7858648-mvh4h\" (UID: \"980eacbb-2f36-403a-a2b8-bce355264f9f\") " pod="openshift-monitoring/metrics-server-6fc7858648-mvh4h"
Sep 12 00:21:06.171413 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:21:06.171266    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/980eacbb-2f36-403a-a2b8-bce355264f9f-client-ca-bundle\") pod \"metrics-server-6fc7858648-mvh4h\" (UID: \"980eacbb-2f36-403a-a2b8-bce355264f9f\") " pod="openshift-monitoring/metrics-server-6fc7858648-mvh4h"
Sep 12 00:21:06.171413 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:21:06.171368    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/980eacbb-2f36-403a-a2b8-bce355264f9f-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-6fc7858648-mvh4h\" (UID: \"980eacbb-2f36-403a-a2b8-bce355264f9f\") " pod="openshift-monitoring/metrics-server-6fc7858648-mvh4h"
Sep 12 00:21:06.171413 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:21:06.171402    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cfh9g\" (UniqueName: \"kubernetes.io/projected/980eacbb-2f36-403a-a2b8-bce355264f9f-kube-api-access-cfh9g\") pod \"metrics-server-6fc7858648-mvh4h\" (UID: \"980eacbb-2f36-403a-a2b8-bce355264f9f\") " pod="openshift-monitoring/metrics-server-6fc7858648-mvh4h"
Sep 12 00:21:06.171572 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:21:06.171507    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/980eacbb-2f36-403a-a2b8-bce355264f9f-audit-log\") pod \"metrics-server-6fc7858648-mvh4h\" (UID: \"980eacbb-2f36-403a-a2b8-bce355264f9f\") " pod="openshift-monitoring/metrics-server-6fc7858648-mvh4h"
Sep 12 00:21:06.172304 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:21:06.172006    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/980eacbb-2f36-403a-a2b8-bce355264f9f-metrics-server-audit-profiles\") pod \"metrics-server-6fc7858648-mvh4h\" (UID: \"980eacbb-2f36-403a-a2b8-bce355264f9f\") " pod="openshift-monitoring/metrics-server-6fc7858648-mvh4h"
Sep 12 00:21:06.172304 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:21:06.172047    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/980eacbb-2f36-403a-a2b8-bce355264f9f-secret-metrics-server-client-certs\") pod \"metrics-server-6fc7858648-mvh4h\" (UID: \"980eacbb-2f36-403a-a2b8-bce355264f9f\") " pod="openshift-monitoring/metrics-server-6fc7858648-mvh4h"
Sep 12 00:21:06.172304 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:21:06.172128    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/980eacbb-2f36-403a-a2b8-bce355264f9f-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-6fc7858648-mvh4h\" (UID: \"980eacbb-2f36-403a-a2b8-bce355264f9f\") " pod="openshift-monitoring/metrics-server-6fc7858648-mvh4h"
Sep 12 00:21:06.173003 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:21:06.172974    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/980eacbb-2f36-403a-a2b8-bce355264f9f-metrics-server-audit-profiles\") pod \"metrics-server-6fc7858648-mvh4h\" (UID: \"980eacbb-2f36-403a-a2b8-bce355264f9f\") " pod="openshift-monitoring/metrics-server-6fc7858648-mvh4h"
Sep 12 00:21:06.174140 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:21:06.174118    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/980eacbb-2f36-403a-a2b8-bce355264f9f-client-ca-bundle\") pod \"metrics-server-6fc7858648-mvh4h\" (UID: \"980eacbb-2f36-403a-a2b8-bce355264f9f\") " pod="openshift-monitoring/metrics-server-6fc7858648-mvh4h"
Sep 12 00:21:06.174236 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:21:06.174186    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/980eacbb-2f36-403a-a2b8-bce355264f9f-secret-metrics-server-tls\") pod \"metrics-server-6fc7858648-mvh4h\" (UID: \"980eacbb-2f36-403a-a2b8-bce355264f9f\") " pod="openshift-monitoring/metrics-server-6fc7858648-mvh4h"
Sep 12 00:21:06.175009 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:21:06.174989    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/980eacbb-2f36-403a-a2b8-bce355264f9f-secret-metrics-server-client-certs\") pod \"metrics-server-6fc7858648-mvh4h\" (UID: \"980eacbb-2f36-403a-a2b8-bce355264f9f\") " pod="openshift-monitoring/metrics-server-6fc7858648-mvh4h"
Sep 12 00:21:06.178974 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:21:06.178954    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cfh9g\" (UniqueName: \"kubernetes.io/projected/980eacbb-2f36-403a-a2b8-bce355264f9f-kube-api-access-cfh9g\") pod \"metrics-server-6fc7858648-mvh4h\" (UID: \"980eacbb-2f36-403a-a2b8-bce355264f9f\") " pod="openshift-monitoring/metrics-server-6fc7858648-mvh4h"
Sep 12 00:21:06.266060 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:21:06.265977    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/metrics-server-6fc7858648-mvh4h"
Sep 12 00:21:06.417481 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:21:06.417447    2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/monitoring-plugin-5d85778b9d-tsthv"]
Sep 12 00:21:06.420832 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:21:06.420808    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/monitoring-plugin-5d85778b9d-tsthv"
Sep 12 00:21:06.423042 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:21:06.423017    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"monitoring-plugin-dockercfg-v77gk\""
Sep 12 00:21:06.423167 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:21:06.423060    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"monitoring-plugin-cert\""
Sep 12 00:21:06.427803 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:21:06.427751    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/monitoring-plugin-5d85778b9d-tsthv"]
Sep 12 00:21:06.575420 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:21:06.575341    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/3f2f3cbe-620b-4bcb-ba33-109bd0b35b1b-monitoring-plugin-cert\") pod \"monitoring-plugin-5d85778b9d-tsthv\" (UID: \"3f2f3cbe-620b-4bcb-ba33-109bd0b35b1b\") " pod="openshift-monitoring/monitoring-plugin-5d85778b9d-tsthv"
Sep 12 00:21:06.575576 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:21:06.575469    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gwh4z\" (UniqueName: \"kubernetes.io/projected/3f2f3cbe-620b-4bcb-ba33-109bd0b35b1b-kube-api-access-gwh4z\") pod \"monitoring-plugin-5d85778b9d-tsthv\" (UID: \"3f2f3cbe-620b-4bcb-ba33-109bd0b35b1b\") " pod="openshift-monitoring/monitoring-plugin-5d85778b9d-tsthv"
Sep 12 00:21:06.676801 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:21:06.676765    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/3f2f3cbe-620b-4bcb-ba33-109bd0b35b1b-monitoring-plugin-cert\") pod \"monitoring-plugin-5d85778b9d-tsthv\" (UID: \"3f2f3cbe-620b-4bcb-ba33-109bd0b35b1b\") " pod="openshift-monitoring/monitoring-plugin-5d85778b9d-tsthv"
Sep 12 00:21:06.676972 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:21:06.676857    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gwh4z\" (UniqueName: \"kubernetes.io/projected/3f2f3cbe-620b-4bcb-ba33-109bd0b35b1b-kube-api-access-gwh4z\") pod \"monitoring-plugin-5d85778b9d-tsthv\" (UID: \"3f2f3cbe-620b-4bcb-ba33-109bd0b35b1b\") " pod="openshift-monitoring/monitoring-plugin-5d85778b9d-tsthv"
Sep 12 00:21:06.676972 ip-10-0-141-128 kubenswrapper[2571]: E0912 00:21:06.676926    2571 secret.go:189] Couldn't get secret openshift-monitoring/monitoring-plugin-cert: secret "monitoring-plugin-cert" not found
Sep 12 00:21:06.677058 ip-10-0-141-128 kubenswrapper[2571]: E0912 00:21:06.677004    2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3f2f3cbe-620b-4bcb-ba33-109bd0b35b1b-monitoring-plugin-cert podName:3f2f3cbe-620b-4bcb-ba33-109bd0b35b1b nodeName:}" failed. No retries permitted until 2026-09-12 00:21:07.176986097 +0000 UTC m=+278.448626601 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "monitoring-plugin-cert" (UniqueName: "kubernetes.io/secret/3f2f3cbe-620b-4bcb-ba33-109bd0b35b1b-monitoring-plugin-cert") pod "monitoring-plugin-5d85778b9d-tsthv" (UID: "3f2f3cbe-620b-4bcb-ba33-109bd0b35b1b") : secret "monitoring-plugin-cert" not found
Sep 12 00:21:06.687188 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:21:06.687164    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gwh4z\" (UniqueName: \"kubernetes.io/projected/3f2f3cbe-620b-4bcb-ba33-109bd0b35b1b-kube-api-access-gwh4z\") pod \"monitoring-plugin-5d85778b9d-tsthv\" (UID: \"3f2f3cbe-620b-4bcb-ba33-109bd0b35b1b\") " pod="openshift-monitoring/monitoring-plugin-5d85778b9d-tsthv"
Sep 12 00:21:07.182369 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:21:07.182336    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/3f2f3cbe-620b-4bcb-ba33-109bd0b35b1b-monitoring-plugin-cert\") pod \"monitoring-plugin-5d85778b9d-tsthv\" (UID: \"3f2f3cbe-620b-4bcb-ba33-109bd0b35b1b\") " pod="openshift-monitoring/monitoring-plugin-5d85778b9d-tsthv"
Sep 12 00:21:07.185074 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:21:07.185049    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/3f2f3cbe-620b-4bcb-ba33-109bd0b35b1b-monitoring-plugin-cert\") pod \"monitoring-plugin-5d85778b9d-tsthv\" (UID: \"3f2f3cbe-620b-4bcb-ba33-109bd0b35b1b\") " pod="openshift-monitoring/monitoring-plugin-5d85778b9d-tsthv"
Sep 12 00:21:07.334197 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:21:07.334152    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/monitoring-plugin-5d85778b9d-tsthv"
Sep 12 00:21:07.882698 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:21:07.882667    2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Sep 12 00:21:07.887531 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:21:07.887506    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Sep 12 00:21:07.890604 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:21:07.890147    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls-assets-0\""
Sep 12 00:21:07.890604 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:21:07.890196    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-web-config\""
Sep 12 00:21:07.890604 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:21:07.890431    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-sidecar-tls\""
Sep 12 00:21:07.890604 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:21:07.890452    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls\""
Sep 12 00:21:07.890883 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:21:07.890715    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-dockercfg-r8wkk\""
Sep 12 00:21:07.890883 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:21:07.890769    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s\""
Sep 12 00:21:07.891445 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:21:07.891252    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-prometheus-http-client-file\""
Sep 12 00:21:07.891445 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:21:07.891310    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-rbac-proxy\""
Sep 12 00:21:07.891445 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:21:07.891349    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-grpc-tls-2dt13jv4hi1jn\""
Sep 12 00:21:07.891445 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:21:07.891357    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-1\""
Sep 12 00:21:07.891445 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:21:07.891351    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"serving-certs-ca-bundle\""
Sep 12 00:21:07.891978 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:21:07.891523    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-kube-rbac-proxy-web\""
Sep 12 00:21:07.891978 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:21:07.891650    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-2\""
Sep 12 00:21:07.891978 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:21:07.891667    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-client-certs\""
Sep 12 00:21:07.894367 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:21:07.894330    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-0\""
Sep 12 00:21:07.899213 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:21:07.899191    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-trusted-ca-bundle\""
Sep 12 00:21:07.904447 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:21:07.904172    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Sep 12 00:21:07.989129 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:21:07.989094    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/c65dfaaf-3dd5-451f-98c1-7ae44786fb2a-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"c65dfaaf-3dd5-451f-98c1-7ae44786fb2a\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 12 00:21:07.989288 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:21:07.989144    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/c65dfaaf-3dd5-451f-98c1-7ae44786fb2a-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"c65dfaaf-3dd5-451f-98c1-7ae44786fb2a\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 12 00:21:07.989288 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:21:07.989174    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/c65dfaaf-3dd5-451f-98c1-7ae44786fb2a-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"c65dfaaf-3dd5-451f-98c1-7ae44786fb2a\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 12 00:21:07.989288 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:21:07.989230    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/c65dfaaf-3dd5-451f-98c1-7ae44786fb2a-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"c65dfaaf-3dd5-451f-98c1-7ae44786fb2a\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 12 00:21:07.989460 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:21:07.989288    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/c65dfaaf-3dd5-451f-98c1-7ae44786fb2a-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"c65dfaaf-3dd5-451f-98c1-7ae44786fb2a\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 12 00:21:07.989460 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:21:07.989313    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/c65dfaaf-3dd5-451f-98c1-7ae44786fb2a-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"c65dfaaf-3dd5-451f-98c1-7ae44786fb2a\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 12 00:21:07.989460 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:21:07.989345    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/c65dfaaf-3dd5-451f-98c1-7ae44786fb2a-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"c65dfaaf-3dd5-451f-98c1-7ae44786fb2a\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 12 00:21:07.989460 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:21:07.989385    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/c65dfaaf-3dd5-451f-98c1-7ae44786fb2a-config-out\") pod \"prometheus-k8s-0\" (UID: \"c65dfaaf-3dd5-451f-98c1-7ae44786fb2a\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 12 00:21:07.989460 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:21:07.989409    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/c65dfaaf-3dd5-451f-98c1-7ae44786fb2a-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"c65dfaaf-3dd5-451f-98c1-7ae44786fb2a\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 12 00:21:07.989460 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:21:07.989438    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c65dfaaf-3dd5-451f-98c1-7ae44786fb2a-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"c65dfaaf-3dd5-451f-98c1-7ae44786fb2a\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 12 00:21:07.989742 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:21:07.989513    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c65dfaaf-3dd5-451f-98c1-7ae44786fb2a-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"c65dfaaf-3dd5-451f-98c1-7ae44786fb2a\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 12 00:21:07.989742 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:21:07.989538    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/c65dfaaf-3dd5-451f-98c1-7ae44786fb2a-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"c65dfaaf-3dd5-451f-98c1-7ae44786fb2a\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 12 00:21:07.989742 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:21:07.989569    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/c65dfaaf-3dd5-451f-98c1-7ae44786fb2a-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"c65dfaaf-3dd5-451f-98c1-7ae44786fb2a\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 12 00:21:07.989742 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:21:07.989630    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/c65dfaaf-3dd5-451f-98c1-7ae44786fb2a-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"c65dfaaf-3dd5-451f-98c1-7ae44786fb2a\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 12 00:21:07.989742 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:21:07.989655    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9lhpr\" (UniqueName: \"kubernetes.io/projected/c65dfaaf-3dd5-451f-98c1-7ae44786fb2a-kube-api-access-9lhpr\") pod \"prometheus-k8s-0\" (UID: \"c65dfaaf-3dd5-451f-98c1-7ae44786fb2a\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 12 00:21:07.989742 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:21:07.989705    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/c65dfaaf-3dd5-451f-98c1-7ae44786fb2a-config\") pod \"prometheus-k8s-0\" (UID: \"c65dfaaf-3dd5-451f-98c1-7ae44786fb2a\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 12 00:21:07.989965 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:21:07.989765    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c65dfaaf-3dd5-451f-98c1-7ae44786fb2a-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"c65dfaaf-3dd5-451f-98c1-7ae44786fb2a\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 12 00:21:07.989965 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:21:07.989795    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/c65dfaaf-3dd5-451f-98c1-7ae44786fb2a-web-config\") pod \"prometheus-k8s-0\" (UID: \"c65dfaaf-3dd5-451f-98c1-7ae44786fb2a\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 12 00:21:07.989965 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:21:07.989840    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/c65dfaaf-3dd5-451f-98c1-7ae44786fb2a-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"c65dfaaf-3dd5-451f-98c1-7ae44786fb2a\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 12 00:21:07.989965 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:21:07.989884    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/c65dfaaf-3dd5-451f-98c1-7ae44786fb2a-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"c65dfaaf-3dd5-451f-98c1-7ae44786fb2a\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 12 00:21:08.090931 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:21:08.090885    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/c65dfaaf-3dd5-451f-98c1-7ae44786fb2a-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"c65dfaaf-3dd5-451f-98c1-7ae44786fb2a\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 12 00:21:08.091097 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:21:08.090936    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/c65dfaaf-3dd5-451f-98c1-7ae44786fb2a-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"c65dfaaf-3dd5-451f-98c1-7ae44786fb2a\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 12 00:21:08.091171 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:21:08.091088    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/c65dfaaf-3dd5-451f-98c1-7ae44786fb2a-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"c65dfaaf-3dd5-451f-98c1-7ae44786fb2a\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 12 00:21:08.091171 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:21:08.091142    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/c65dfaaf-3dd5-451f-98c1-7ae44786fb2a-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"c65dfaaf-3dd5-451f-98c1-7ae44786fb2a\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 12 00:21:08.091277 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:21:08.091184    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/c65dfaaf-3dd5-451f-98c1-7ae44786fb2a-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"c65dfaaf-3dd5-451f-98c1-7ae44786fb2a\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 12 00:21:08.091277 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:21:08.091211    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/c65dfaaf-3dd5-451f-98c1-7ae44786fb2a-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"c65dfaaf-3dd5-451f-98c1-7ae44786fb2a\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 12 00:21:08.091277 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:21:08.091266    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/c65dfaaf-3dd5-451f-98c1-7ae44786fb2a-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"c65dfaaf-3dd5-451f-98c1-7ae44786fb2a\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 12 00:21:08.091428 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:21:08.091308    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/c65dfaaf-3dd5-451f-98c1-7ae44786fb2a-config-out\") pod \"prometheus-k8s-0\" (UID: \"c65dfaaf-3dd5-451f-98c1-7ae44786fb2a\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 12 00:21:08.091428 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:21:08.091333    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/c65dfaaf-3dd5-451f-98c1-7ae44786fb2a-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"c65dfaaf-3dd5-451f-98c1-7ae44786fb2a\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 12 00:21:08.091428 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:21:08.091359    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c65dfaaf-3dd5-451f-98c1-7ae44786fb2a-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"c65dfaaf-3dd5-451f-98c1-7ae44786fb2a\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 12 00:21:08.091428 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:21:08.091396    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c65dfaaf-3dd5-451f-98c1-7ae44786fb2a-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"c65dfaaf-3dd5-451f-98c1-7ae44786fb2a\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 12 00:21:08.091428 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:21:08.091422    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/c65dfaaf-3dd5-451f-98c1-7ae44786fb2a-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"c65dfaaf-3dd5-451f-98c1-7ae44786fb2a\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 12 00:21:08.091699 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:21:08.091458    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/c65dfaaf-3dd5-451f-98c1-7ae44786fb2a-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"c65dfaaf-3dd5-451f-98c1-7ae44786fb2a\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 12 00:21:08.091699 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:21:08.091487    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/c65dfaaf-3dd5-451f-98c1-7ae44786fb2a-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"c65dfaaf-3dd5-451f-98c1-7ae44786fb2a\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 12 00:21:08.091699 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:21:08.091510    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9lhpr\" (UniqueName: \"kubernetes.io/projected/c65dfaaf-3dd5-451f-98c1-7ae44786fb2a-kube-api-access-9lhpr\") pod \"prometheus-k8s-0\" (UID: \"c65dfaaf-3dd5-451f-98c1-7ae44786fb2a\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 12 00:21:08.091699 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:21:08.091550    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/c65dfaaf-3dd5-451f-98c1-7ae44786fb2a-config\") pod \"prometheus-k8s-0\" (UID: \"c65dfaaf-3dd5-451f-98c1-7ae44786fb2a\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 12 00:21:08.091699 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:21:08.091636    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c65dfaaf-3dd5-451f-98c1-7ae44786fb2a-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"c65dfaaf-3dd5-451f-98c1-7ae44786fb2a\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 12 00:21:08.091699 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:21:08.091667    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/c65dfaaf-3dd5-451f-98c1-7ae44786fb2a-web-config\") pod \"prometheus-k8s-0\" (UID: \"c65dfaaf-3dd5-451f-98c1-7ae44786fb2a\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 12 00:21:08.092013 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:21:08.091728    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/c65dfaaf-3dd5-451f-98c1-7ae44786fb2a-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"c65dfaaf-3dd5-451f-98c1-7ae44786fb2a\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 12 00:21:08.092013 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:21:08.091783    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/c65dfaaf-3dd5-451f-98c1-7ae44786fb2a-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"c65dfaaf-3dd5-451f-98c1-7ae44786fb2a\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 12 00:21:08.092573 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:21:08.092541    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/c65dfaaf-3dd5-451f-98c1-7ae44786fb2a-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"c65dfaaf-3dd5-451f-98c1-7ae44786fb2a\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 12 00:21:08.093225 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:21:08.093196    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/c65dfaaf-3dd5-451f-98c1-7ae44786fb2a-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"c65dfaaf-3dd5-451f-98c1-7ae44786fb2a\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 12 00:21:08.095041 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:21:08.094075    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/c65dfaaf-3dd5-451f-98c1-7ae44786fb2a-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"c65dfaaf-3dd5-451f-98c1-7ae44786fb2a\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 12 00:21:08.095041 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:21:08.094343    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c65dfaaf-3dd5-451f-98c1-7ae44786fb2a-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"c65dfaaf-3dd5-451f-98c1-7ae44786fb2a\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 12 00:21:08.095041 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:21:08.094571    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/c65dfaaf-3dd5-451f-98c1-7ae44786fb2a-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"c65dfaaf-3dd5-451f-98c1-7ae44786fb2a\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 12 00:21:08.095041 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:21:08.094678    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/c65dfaaf-3dd5-451f-98c1-7ae44786fb2a-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"c65dfaaf-3dd5-451f-98c1-7ae44786fb2a\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 12 00:21:08.095041 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:21:08.094825    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/c65dfaaf-3dd5-451f-98c1-7ae44786fb2a-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"c65dfaaf-3dd5-451f-98c1-7ae44786fb2a\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 12 00:21:08.095041 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:21:08.094997    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/c65dfaaf-3dd5-451f-98c1-7ae44786fb2a-config-out\") pod \"prometheus-k8s-0\" (UID: \"c65dfaaf-3dd5-451f-98c1-7ae44786fb2a\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 12 00:21:08.095396 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:21:08.095207    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/c65dfaaf-3dd5-451f-98c1-7ae44786fb2a-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"c65dfaaf-3dd5-451f-98c1-7ae44786fb2a\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 12 00:21:08.095396 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:21:08.095227    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c65dfaaf-3dd5-451f-98c1-7ae44786fb2a-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"c65dfaaf-3dd5-451f-98c1-7ae44786fb2a\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 12 00:21:08.095507 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:21:08.095473    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c65dfaaf-3dd5-451f-98c1-7ae44786fb2a-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"c65dfaaf-3dd5-451f-98c1-7ae44786fb2a\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 12 00:21:08.096624 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:21:08.096578    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/c65dfaaf-3dd5-451f-98c1-7ae44786fb2a-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"c65dfaaf-3dd5-451f-98c1-7ae44786fb2a\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 12 00:21:08.096762 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:21:08.096726    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/c65dfaaf-3dd5-451f-98c1-7ae44786fb2a-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"c65dfaaf-3dd5-451f-98c1-7ae44786fb2a\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 12 00:21:08.097747 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:21:08.097473    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/c65dfaaf-3dd5-451f-98c1-7ae44786fb2a-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"c65dfaaf-3dd5-451f-98c1-7ae44786fb2a\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 12 00:21:08.097843 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:21:08.097743    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/c65dfaaf-3dd5-451f-98c1-7ae44786fb2a-web-config\") pod \"prometheus-k8s-0\" (UID: \"c65dfaaf-3dd5-451f-98c1-7ae44786fb2a\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 12 00:21:08.097843 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:21:08.097818    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/c65dfaaf-3dd5-451f-98c1-7ae44786fb2a-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"c65dfaaf-3dd5-451f-98c1-7ae44786fb2a\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 12 00:21:08.097956 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:21:08.097887    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/c65dfaaf-3dd5-451f-98c1-7ae44786fb2a-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"c65dfaaf-3dd5-451f-98c1-7ae44786fb2a\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 12 00:21:08.098453 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:21:08.098416    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/c65dfaaf-3dd5-451f-98c1-7ae44786fb2a-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"c65dfaaf-3dd5-451f-98c1-7ae44786fb2a\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 12 00:21:08.098849 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:21:08.098830    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/secret/c65dfaaf-3dd5-451f-98c1-7ae44786fb2a-config\") pod \"prometheus-k8s-0\" (UID: \"c65dfaaf-3dd5-451f-98c1-7ae44786fb2a\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 12 00:21:08.102966 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:21:08.102946    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9lhpr\" (UniqueName: \"kubernetes.io/projected/c65dfaaf-3dd5-451f-98c1-7ae44786fb2a-kube-api-access-9lhpr\") pod \"prometheus-k8s-0\" (UID: \"c65dfaaf-3dd5-451f-98c1-7ae44786fb2a\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 12 00:21:08.201569 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:21:08.201534    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Sep 12 00:21:09.101468 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:21:09.101438    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/3a2114c7-d34a-4249-97a4-65f5375453ce-metrics-tls\") pod \"dns-default-fr2kt\" (UID: \"3a2114c7-d34a-4249-97a4-65f5375453ce\") " pod="openshift-dns/dns-default-fr2kt"
Sep 12 00:21:09.101657 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:21:09.101496    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/80cbfa96-879b-48ea-8732-d50ebebea182-registry-tls\") pod \"image-registry-744fcccbf-5vhvb\" (UID: \"80cbfa96-879b-48ea-8732-d50ebebea182\") " pod="openshift-image-registry/image-registry-744fcccbf-5vhvb"
Sep 12 00:21:09.104110 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:21:09.104084    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/3a2114c7-d34a-4249-97a4-65f5375453ce-metrics-tls\") pod \"dns-default-fr2kt\" (UID: \"3a2114c7-d34a-4249-97a4-65f5375453ce\") " pod="openshift-dns/dns-default-fr2kt"
Sep 12 00:21:09.104219 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:21:09.104114    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/80cbfa96-879b-48ea-8732-d50ebebea182-registry-tls\") pod \"image-registry-744fcccbf-5vhvb\" (UID: \"80cbfa96-879b-48ea-8732-d50ebebea182\") " pod="openshift-image-registry/image-registry-744fcccbf-5vhvb"
Sep 12 00:21:09.360574 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:21:09.360500    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-knhwr\""
Sep 12 00:21:09.360574 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:21:09.360501    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-tp2x6\""
Sep 12 00:21:09.368513 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:21:09.368494    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-744fcccbf-5vhvb"
Sep 12 00:21:09.368660 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:21:09.368626    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-fr2kt"
Sep 12 00:21:12.086444 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:21:12.086406    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-qgr6s" event={"ID":"17f775e6-4bfa-4d7f-aeac-4e66cb551e46","Type":"ContainerStarted","Data":"229ac8f9d98182f4963fe490523fe142b27a1a68f6b6f3bc05022168c4481003"}
Sep 12 00:21:12.086876 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:21:12.086499    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-qgr6s" event={"ID":"17f775e6-4bfa-4d7f-aeac-4e66cb551e46","Type":"ContainerStarted","Data":"6be3d54be8100644dd2d21cbb4e627a7f44b25f2f9c498470dcaaa3cf1d7ccc0"}
Sep 12 00:21:12.087957 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:21:12.087931    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/downloads-76f99584f4-wqhtj" event={"ID":"25e2de93-94e2-4231-97ff-208a1e9694a7","Type":"ContainerStarted","Data":"a1f8fc74ef0ce642ba214bdd5839df480fd7893fb187bb7ff77ce35a3f98fe67"}
Sep 12 00:21:12.088076 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:21:12.087986    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/downloads-76f99584f4-wqhtj"
Sep 12 00:21:12.089500 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:21:12.089475    2571 patch_prober.go:28] interesting pod/downloads-76f99584f4-wqhtj container/download-server namespace/openshift-console: Readiness probe status=failure output="Get \"http://10.132.0.15:8080/\": dial tcp 10.132.0.15:8080: connect: connection refused" start-of-body=
Sep 12 00:21:12.089642 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:21:12.089535    2571 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-console/downloads-76f99584f4-wqhtj" podUID="25e2de93-94e2-4231-97ff-208a1e9694a7" containerName="download-server" probeResult="failure" output="Get \"http://10.132.0.15:8080/\": dial tcp 10.132.0.15:8080: connect: connection refused"
Sep 12 00:21:12.104042 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:21:12.103965    2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-qgr6s" podStartSLOduration=9.129257208 podStartE2EDuration="11.103951746s" totalImagesPullingTime="1.885544082s" totalInitContainerRuntime="89.150456ms" isStatefulPod=true podCreationTimestamp="2026-09-12 00:21:01 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-12 00:21:12.102654894 +0000 UTC m=+283.374295411" watchObservedRunningTime="2026-09-12 00:21:12.103951746 +0000 UTC m=+283.375592261"
Sep 12 00:21:12.119093 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:21:12.119048    2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/downloads-76f99584f4-wqhtj" podStartSLOduration=1.479875156 podStartE2EDuration="18.119033929s" totalImagesPullingTime="16.639158773s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-12 00:20:54 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-12 00:21:12.118822477 +0000 UTC m=+283.390462987" watchObservedRunningTime="2026-09-12 00:21:12.119033929 +0000 UTC m=+283.390674439"
Sep 12 00:21:12.147252 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:21:12.147059    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/monitoring-plugin-5d85778b9d-tsthv"]
Sep 12 00:21:12.149367 ip-10-0-141-128 kubenswrapper[2571]: W0912 00:21:12.149341    2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3f2f3cbe_620b_4bcb_ba33_109bd0b35b1b.slice/crio-b791738fb53ae506629e8924ddda637051b81952517f5b952f3f16f27e35926c WatchSource:0}: Error finding container b791738fb53ae506629e8924ddda637051b81952517f5b952f3f16f27e35926c: Status 404 returned error can't find the container with id b791738fb53ae506629e8924ddda637051b81952517f5b952f3f16f27e35926c
Sep 12 00:21:12.154842 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:21:12.153249    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-fr2kt"]
Sep 12 00:21:12.154842 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:21:12.154819    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/metrics-server-6fc7858648-mvh4h"]
Sep 12 00:21:12.155297 ip-10-0-141-128 kubenswrapper[2571]: W0912 00:21:12.155269    2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod980eacbb_2f36_403a_a2b8_bce355264f9f.slice/crio-2db9c29f9393cc88b955e08724be985a57732b26b53427bce67c65e640828e82 WatchSource:0}: Error finding container 2db9c29f9393cc88b955e08724be985a57732b26b53427bce67c65e640828e82: Status 404 returned error can't find the container with id 2db9c29f9393cc88b955e08724be985a57732b26b53427bce67c65e640828e82
Sep 12 00:21:12.156277 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:21:12.156253    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-744fcccbf-5vhvb"]
Sep 12 00:21:12.157090 ip-10-0-141-128 kubenswrapper[2571]: W0912 00:21:12.157068    2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3a2114c7_d34a_4249_97a4_65f5375453ce.slice/crio-d145e6219307f00c08e22fc09cb574ba7615e5e304fef01e1f39306e70dd14fd WatchSource:0}: Error finding container d145e6219307f00c08e22fc09cb574ba7615e5e304fef01e1f39306e70dd14fd: Status 404 returned error can't find the container with id d145e6219307f00c08e22fc09cb574ba7615e5e304fef01e1f39306e70dd14fd
Sep 12 00:21:12.157256 ip-10-0-141-128 kubenswrapper[2571]: W0912 00:21:12.157231    2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod80cbfa96_879b_48ea_8732_d50ebebea182.slice/crio-6e3c502d3fa8cd2fbf23b30d68728207705ea6401bcff9d7aa8a87df3d0db7de WatchSource:0}: Error finding container 6e3c502d3fa8cd2fbf23b30d68728207705ea6401bcff9d7aa8a87df3d0db7de: Status 404 returned error can't find the container with id 6e3c502d3fa8cd2fbf23b30d68728207705ea6401bcff9d7aa8a87df3d0db7de
Sep 12 00:21:12.161890 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:21:12.161082    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Sep 12 00:21:12.163162 ip-10-0-141-128 kubenswrapper[2571]: W0912 00:21:12.163115    2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc65dfaaf_3dd5_451f_98c1_7ae44786fb2a.slice/crio-519e7ec966928fb94c3fb1e1d0b7b4a2f8036fb262dd1bc0d6e7211c3a991795 WatchSource:0}: Error finding container 519e7ec966928fb94c3fb1e1d0b7b4a2f8036fb262dd1bc0d6e7211c3a991795: Status 404 returned error can't find the container with id 519e7ec966928fb94c3fb1e1d0b7b4a2f8036fb262dd1bc0d6e7211c3a991795
Sep 12 00:21:13.097856 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:21:13.097809    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"c65dfaaf-3dd5-451f-98c1-7ae44786fb2a","Type":"ContainerStarted","Data":"519e7ec966928fb94c3fb1e1d0b7b4a2f8036fb262dd1bc0d6e7211c3a991795"}
Sep 12 00:21:13.100992 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:21:13.100057    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-744fcccbf-5vhvb" event={"ID":"80cbfa96-879b-48ea-8732-d50ebebea182","Type":"ContainerStarted","Data":"cd9b431a5120d48ed4ad8117e03c153087687bcab8e9304912b036c5a6e1649c"}
Sep 12 00:21:13.100992 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:21:13.100106    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-744fcccbf-5vhvb" event={"ID":"80cbfa96-879b-48ea-8732-d50ebebea182","Type":"ContainerStarted","Data":"6e3c502d3fa8cd2fbf23b30d68728207705ea6401bcff9d7aa8a87df3d0db7de"}
Sep 12 00:21:13.100992 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:21:13.100925    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-744fcccbf-5vhvb"
Sep 12 00:21:13.105106 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:21:13.105031    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/metrics-server-6fc7858648-mvh4h" event={"ID":"980eacbb-2f36-403a-a2b8-bce355264f9f","Type":"ContainerStarted","Data":"2db9c29f9393cc88b955e08724be985a57732b26b53427bce67c65e640828e82"}
Sep 12 00:21:13.109239 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:21:13.109210    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-fr2kt" event={"ID":"3a2114c7-d34a-4249-97a4-65f5375453ce","Type":"ContainerStarted","Data":"d145e6219307f00c08e22fc09cb574ba7615e5e304fef01e1f39306e70dd14fd"}
Sep 12 00:21:13.113369 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:21:13.111630    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/monitoring-plugin-5d85778b9d-tsthv" event={"ID":"3f2f3cbe-620b-4bcb-ba33-109bd0b35b1b","Type":"ContainerStarted","Data":"b791738fb53ae506629e8924ddda637051b81952517f5b952f3f16f27e35926c"}
Sep 12 00:21:13.119350 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:21:13.119300    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/downloads-76f99584f4-wqhtj"
Sep 12 00:21:13.142651 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:21:13.142524    2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/image-registry-744fcccbf-5vhvb" podStartSLOduration=283.142505642 podStartE2EDuration="4m43.142505642s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-12 00:16:30 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-12 00:21:13.123457196 +0000 UTC m=+284.395097707" watchObservedRunningTime="2026-09-12 00:21:13.142505642 +0000 UTC m=+284.414146155"
Sep 12 00:21:17.132397 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:21:17.132358    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-fr2kt" event={"ID":"3a2114c7-d34a-4249-97a4-65f5375453ce","Type":"ContainerStarted","Data":"47606f1ed6830a6e0362a3775aab877de120ac9f8fca6984503b79a0de1d9fac"}
Sep 12 00:21:17.132926 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:21:17.132455    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-fr2kt" event={"ID":"3a2114c7-d34a-4249-97a4-65f5375453ce","Type":"ContainerStarted","Data":"5a104c0e3fcadb49e6924d366af389e9804245105c18e43609ad2c336191f1fa"}
Sep 12 00:21:17.132926 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:21:17.132488    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-fr2kt"
Sep 12 00:21:17.136007 ip-10-0-141-128 kubenswrapper[2571]: E0912 00:21:17.135363    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[metrics-certs], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-multus/network-metrics-daemon-bdp25" podUID="6d479776-cc30-4bea-bb3a-8b4c259d59c7"
Sep 12 00:21:17.137861 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:21:17.136918    2571 generic.go:356] "Generic (PLEG): container finished" podID="c65dfaaf-3dd5-451f-98c1-7ae44786fb2a" containerID="5aaa27351c7ca308a5a2217bf3e199ec076d7da50d9e83e73d2e6ea94d7d2c9d" exitCode=0
Sep 12 00:21:17.140926 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:21:17.140909    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/monitoring-plugin-5d85778b9d-tsthv"
Sep 12 00:21:17.141017 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:21:17.140975    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/monitoring-plugin-5d85778b9d-tsthv"
Sep 12 00:21:17.141017 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:21:17.140992    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/monitoring-plugin-5d85778b9d-tsthv" event={"ID":"3f2f3cbe-620b-4bcb-ba33-109bd0b35b1b","Type":"ContainerStarted","Data":"1739ede926324e361cef30b7f3f41743ae4b2796e4cfacf0263163b93fcfa123"}
Sep 12 00:21:17.141017 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:21:17.141013    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"c65dfaaf-3dd5-451f-98c1-7ae44786fb2a","Type":"ContainerDied","Data":"5aaa27351c7ca308a5a2217bf3e199ec076d7da50d9e83e73d2e6ea94d7d2c9d"}
Sep 12 00:21:17.141186 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:21:17.141066    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/metrics-server-6fc7858648-mvh4h" event={"ID":"980eacbb-2f36-403a-a2b8-bce355264f9f","Type":"ContainerStarted","Data":"a3cefc1dc0891c982a058c7227eacd00afbb8e6c6dbb013d8d8451a9b5f6e30d"}
Sep 12 00:21:17.149063 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:21:17.149021    2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-fr2kt" podStartSLOduration=253.912320275 podStartE2EDuration="4m18.149003257s" totalImagesPullingTime="4.236682982s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-12 00:16:59 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-12 00:21:17.148346479 +0000 UTC m=+288.419986990" watchObservedRunningTime="2026-09-12 00:21:17.149003257 +0000 UTC m=+288.420643782"
Sep 12 00:21:17.189483 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:21:17.189439    2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/monitoring-plugin-5d85778b9d-tsthv" podStartSLOduration=6.954145944 podStartE2EDuration="11.189419441s" totalImagesPullingTime="4.235273497s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-12 00:21:06 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-12 00:21:17.188107669 +0000 UTC m=+288.459748178" watchObservedRunningTime="2026-09-12 00:21:17.189419441 +0000 UTC m=+288.461059966"
Sep 12 00:21:17.205887 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:21:17.205814    2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/metrics-server-6fc7858648-mvh4h" podStartSLOduration=7.975110352 podStartE2EDuration="12.205796618s" totalImagesPullingTime="4.230686266s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-12 00:21:05 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-12 00:21:17.205055662 +0000 UTC m=+288.476696198" watchObservedRunningTime="2026-09-12 00:21:17.205796618 +0000 UTC m=+288.477437132"
Sep 12 00:21:22.167086 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:21:22.167045    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"c65dfaaf-3dd5-451f-98c1-7ae44786fb2a","Type":"ContainerStarted","Data":"d56861000d33e8db1c5e5200802d1330b5c6ad7cf8aee53e003279cdd4cf0cfc"}
Sep 12 00:21:22.167638 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:21:22.167130    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"c65dfaaf-3dd5-451f-98c1-7ae44786fb2a","Type":"ContainerStarted","Data":"34cc2f3b6cddb3895459b0c9a20c2cd6e5aa267a5e1cb8e2e5bc79da8e09a1cc"}
Sep 12 00:21:24.176897 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:21:24.176870    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"c65dfaaf-3dd5-451f-98c1-7ae44786fb2a","Type":"ContainerStarted","Data":"d2be2428757c823ea80d8e3553e6d444104f6fa3e9c1aa371d9725c5a8bd5a8a"}
Sep 12 00:21:24.177196 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:21:24.176945    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"c65dfaaf-3dd5-451f-98c1-7ae44786fb2a","Type":"ContainerStarted","Data":"2f2255ca86826463228a6ec8aba88c8c9798277a74652b98297046fe6560f416"}
Sep 12 00:21:25.183199 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:21:25.183160    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"c65dfaaf-3dd5-451f-98c1-7ae44786fb2a","Type":"ContainerStarted","Data":"04268be4110e6b64ebe94e5e690d28565b1895e70929b873987f0e0ae5e6d448"}
Sep 12 00:21:25.183697 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:21:25.183225    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"c65dfaaf-3dd5-451f-98c1-7ae44786fb2a","Type":"ContainerStarted","Data":"005838029f7431e414e24e713fb4656d8153e62831d3e5f83f2f51e00bb9753a"}
Sep 12 00:21:25.211541 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:21:25.211495    2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-k8s-0" podStartSLOduration=7.417623006 podStartE2EDuration="18.211479438s" totalImagesPullingTime="10.753691609s" totalInitContainerRuntime="40.164823ms" isStatefulPod=false podCreationTimestamp="2026-09-12 00:21:07 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-12 00:21:25.209763989 +0000 UTC m=+296.481404497" watchObservedRunningTime="2026-09-12 00:21:25.211479438 +0000 UTC m=+296.483119948"
Sep 12 00:21:26.266967 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:21:26.266925    2571 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/metrics-server-6fc7858648-mvh4h"
Sep 12 00:21:26.267344 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:21:26.266986    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/metrics-server-6fc7858648-mvh4h"
Sep 12 00:21:27.147905 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:21:27.147877    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-fr2kt"
Sep 12 00:21:28.201772 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:21:28.201742    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-k8s-0"
Sep 12 00:21:29.106105 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:21:29.106072    2571 kubelet.go:1648] "Image garbage collection succeeded"
Sep 12 00:21:29.373065 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:21:29.373002    2571 patch_prober.go:28] interesting pod/image-registry-744fcccbf-5vhvb container/registry namespace/openshift-image-registry: Liveness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]}
Sep 12 00:21:29.373065 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:21:29.373051    2571 prober.go:120] "Probe failed" probeType="Liveness" pod="openshift-image-registry/image-registry-744fcccbf-5vhvb" podUID="80cbfa96-879b-48ea-8732-d50ebebea182" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Sep 12 00:21:32.134296 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:21:32.134261    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-bdp25"
Sep 12 00:21:35.125235 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:21:35.125208    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-744fcccbf-5vhvb"
Sep 12 00:21:38.231253 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:21:38.231173    2571 generic.go:356] "Generic (PLEG): container finished" podID="bfc37701-bd22-49af-8189-77a984f3a57d" containerID="2a4af38f75b0d4466b8dd096ba2696694cdcf312b5d7c34b7d3917c3820b4625" exitCode=0
Sep 12 00:21:38.231611 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:21:38.231254    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-cf7c776b7-k2ftn" event={"ID":"bfc37701-bd22-49af-8189-77a984f3a57d","Type":"ContainerDied","Data":"2a4af38f75b0d4466b8dd096ba2696694cdcf312b5d7c34b7d3917c3820b4625"}
Sep 12 00:21:38.231611 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:21:38.231563    2571 scope.go:117] "RemoveContainer" containerID="2a4af38f75b0d4466b8dd096ba2696694cdcf312b5d7c34b7d3917c3820b4625"
Sep 12 00:21:39.236093 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:21:39.236060    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-cf7c776b7-k2ftn" event={"ID":"bfc37701-bd22-49af-8189-77a984f3a57d","Type":"ContainerStarted","Data":"80ac6a6662cce379706b583bc2e053acc18bb311f4688b4d5f8407efb8d70c46"}
Sep 12 00:21:45.167299 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:21:45.167258    2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-744fcccbf-5vhvb"]
Sep 12 00:21:46.271343 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:21:46.271315    2571 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/metrics-server-6fc7858648-mvh4h"
Sep 12 00:21:46.275057 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:21:46.275037    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/metrics-server-6fc7858648-mvh4h"
Sep 12 00:22:00.592603 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:22:00.592555    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-57b84c8474-8xjql_c2123986-4ae4-40ce-a423-b5e9a4accda2/router/0.log"
Sep 12 00:22:00.606411 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:22:00.606390    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-rlr5m_ba88d840-64fd-419f-8444-ebde7a94c354/serve-healthcheck-canary/0.log"
Sep 12 00:22:08.202487 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:22:08.202452    2571 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/prometheus-k8s-0"
Sep 12 00:22:08.217733 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:22:08.217707    2571 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/prometheus-k8s-0"
Sep 12 00:22:08.358199 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:22:08.358175    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-k8s-0"
Sep 12 00:22:10.185661 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:22:10.185611    2571 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-image-registry/image-registry-744fcccbf-5vhvb" podUID="80cbfa96-879b-48ea-8732-d50ebebea182" containerName="registry" containerID="cri-o://cd9b431a5120d48ed4ad8117e03c153087687bcab8e9304912b036c5a6e1649c" gracePeriod=30
Sep 12 00:22:11.357814 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:22:11.357767    2571 generic.go:356] "Generic (PLEG): container finished" podID="80cbfa96-879b-48ea-8732-d50ebebea182" containerID="cd9b431a5120d48ed4ad8117e03c153087687bcab8e9304912b036c5a6e1649c" exitCode=0
Sep 12 00:22:11.358121 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:22:11.357818    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-744fcccbf-5vhvb" event={"ID":"80cbfa96-879b-48ea-8732-d50ebebea182","Type":"ContainerDied","Data":"cd9b431a5120d48ed4ad8117e03c153087687bcab8e9304912b036c5a6e1649c"}
Sep 12 00:22:11.438984 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:22:11.438961    2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-744fcccbf-5vhvb"
Sep 12 00:22:11.506002 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:22:11.505896    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/80cbfa96-879b-48ea-8732-d50ebebea182-image-registry-private-configuration\") pod \"80cbfa96-879b-48ea-8732-d50ebebea182\" (UID: \"80cbfa96-879b-48ea-8732-d50ebebea182\") "
Sep 12 00:22:11.506002 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:22:11.505951    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/80cbfa96-879b-48ea-8732-d50ebebea182-ca-trust-extracted\") pod \"80cbfa96-879b-48ea-8732-d50ebebea182\" (UID: \"80cbfa96-879b-48ea-8732-d50ebebea182\") "
Sep 12 00:22:11.506002 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:22:11.505986    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/80cbfa96-879b-48ea-8732-d50ebebea182-registry-tls\") pod \"80cbfa96-879b-48ea-8732-d50ebebea182\" (UID: \"80cbfa96-879b-48ea-8732-d50ebebea182\") "
Sep 12 00:22:11.506248 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:22:11.506152    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/80cbfa96-879b-48ea-8732-d50ebebea182-bound-sa-token\") pod \"80cbfa96-879b-48ea-8732-d50ebebea182\" (UID: \"80cbfa96-879b-48ea-8732-d50ebebea182\") "
Sep 12 00:22:11.506248 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:22:11.506189    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/80cbfa96-879b-48ea-8732-d50ebebea182-registry-certificates\") pod \"80cbfa96-879b-48ea-8732-d50ebebea182\" (UID: \"80cbfa96-879b-48ea-8732-d50ebebea182\") "
Sep 12 00:22:11.506248 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:22:11.506223    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/80cbfa96-879b-48ea-8732-d50ebebea182-installation-pull-secrets\") pod \"80cbfa96-879b-48ea-8732-d50ebebea182\" (UID: \"80cbfa96-879b-48ea-8732-d50ebebea182\") "
Sep 12 00:22:11.506399 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:22:11.506253    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/80cbfa96-879b-48ea-8732-d50ebebea182-trusted-ca\") pod \"80cbfa96-879b-48ea-8732-d50ebebea182\" (UID: \"80cbfa96-879b-48ea-8732-d50ebebea182\") "
Sep 12 00:22:11.506399 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:22:11.506364    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-z67b4\" (UniqueName: \"kubernetes.io/projected/80cbfa96-879b-48ea-8732-d50ebebea182-kube-api-access-z67b4\") pod \"80cbfa96-879b-48ea-8732-d50ebebea182\" (UID: \"80cbfa96-879b-48ea-8732-d50ebebea182\") "
Sep 12 00:22:11.507872 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:22:11.507457    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/80cbfa96-879b-48ea-8732-d50ebebea182-registry-certificates" (OuterVolumeSpecName: "registry-certificates") pod "80cbfa96-879b-48ea-8732-d50ebebea182" (UID: "80cbfa96-879b-48ea-8732-d50ebebea182"). InnerVolumeSpecName "registry-certificates". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 12 00:22:11.507872 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:22:11.507757    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/80cbfa96-879b-48ea-8732-d50ebebea182-trusted-ca" (OuterVolumeSpecName: "trusted-ca") pod "80cbfa96-879b-48ea-8732-d50ebebea182" (UID: "80cbfa96-879b-48ea-8732-d50ebebea182"). InnerVolumeSpecName "trusted-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 12 00:22:11.508707 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:22:11.508668    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/80cbfa96-879b-48ea-8732-d50ebebea182-bound-sa-token" (OuterVolumeSpecName: "bound-sa-token") pod "80cbfa96-879b-48ea-8732-d50ebebea182" (UID: "80cbfa96-879b-48ea-8732-d50ebebea182"). InnerVolumeSpecName "bound-sa-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 12 00:22:11.508804 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:22:11.508749    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/80cbfa96-879b-48ea-8732-d50ebebea182-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "80cbfa96-879b-48ea-8732-d50ebebea182" (UID: "80cbfa96-879b-48ea-8732-d50ebebea182"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 12 00:22:11.509108 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:22:11.509055    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/80cbfa96-879b-48ea-8732-d50ebebea182-image-registry-private-configuration" (OuterVolumeSpecName: "image-registry-private-configuration") pod "80cbfa96-879b-48ea-8732-d50ebebea182" (UID: "80cbfa96-879b-48ea-8732-d50ebebea182"). InnerVolumeSpecName "image-registry-private-configuration". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 12 00:22:11.509200 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:22:11.509145    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/80cbfa96-879b-48ea-8732-d50ebebea182-kube-api-access-z67b4" (OuterVolumeSpecName: "kube-api-access-z67b4") pod "80cbfa96-879b-48ea-8732-d50ebebea182" (UID: "80cbfa96-879b-48ea-8732-d50ebebea182"). InnerVolumeSpecName "kube-api-access-z67b4". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 12 00:22:11.510398 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:22:11.510369    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/80cbfa96-879b-48ea-8732-d50ebebea182-installation-pull-secrets" (OuterVolumeSpecName: "installation-pull-secrets") pod "80cbfa96-879b-48ea-8732-d50ebebea182" (UID: "80cbfa96-879b-48ea-8732-d50ebebea182"). InnerVolumeSpecName "installation-pull-secrets". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 12 00:22:11.518384 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:22:11.518361    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/80cbfa96-879b-48ea-8732-d50ebebea182-ca-trust-extracted" (OuterVolumeSpecName: "ca-trust-extracted") pod "80cbfa96-879b-48ea-8732-d50ebebea182" (UID: "80cbfa96-879b-48ea-8732-d50ebebea182"). InnerVolumeSpecName "ca-trust-extracted". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 12 00:22:11.607004 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:22:11.606974    2571 reconciler_common.go:299] "Volume detached for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/80cbfa96-879b-48ea-8732-d50ebebea182-trusted-ca\") on node \"ip-10-0-141-128.ec2.internal\" DevicePath \"\""
Sep 12 00:22:11.607004 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:22:11.607003    2571 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-z67b4\" (UniqueName: \"kubernetes.io/projected/80cbfa96-879b-48ea-8732-d50ebebea182-kube-api-access-z67b4\") on node \"ip-10-0-141-128.ec2.internal\" DevicePath \"\""
Sep 12 00:22:11.607195 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:22:11.607017    2571 reconciler_common.go:299] "Volume detached for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/80cbfa96-879b-48ea-8732-d50ebebea182-image-registry-private-configuration\") on node \"ip-10-0-141-128.ec2.internal\" DevicePath \"\""
Sep 12 00:22:11.607195 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:22:11.607031    2571 reconciler_common.go:299] "Volume detached for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/80cbfa96-879b-48ea-8732-d50ebebea182-ca-trust-extracted\") on node \"ip-10-0-141-128.ec2.internal\" DevicePath \"\""
Sep 12 00:22:11.607195 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:22:11.607043    2571 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/80cbfa96-879b-48ea-8732-d50ebebea182-registry-tls\") on node \"ip-10-0-141-128.ec2.internal\" DevicePath \"\""
Sep 12 00:22:11.607195 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:22:11.607054    2571 reconciler_common.go:299] "Volume detached for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/80cbfa96-879b-48ea-8732-d50ebebea182-bound-sa-token\") on node \"ip-10-0-141-128.ec2.internal\" DevicePath \"\""
Sep 12 00:22:11.607195 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:22:11.607063    2571 reconciler_common.go:299] "Volume detached for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/80cbfa96-879b-48ea-8732-d50ebebea182-registry-certificates\") on node \"ip-10-0-141-128.ec2.internal\" DevicePath \"\""
Sep 12 00:22:11.607195 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:22:11.607074    2571 reconciler_common.go:299] "Volume detached for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/80cbfa96-879b-48ea-8732-d50ebebea182-installation-pull-secrets\") on node \"ip-10-0-141-128.ec2.internal\" DevicePath \"\""
Sep 12 00:22:12.363427 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:22:12.363393    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-744fcccbf-5vhvb" event={"ID":"80cbfa96-879b-48ea-8732-d50ebebea182","Type":"ContainerDied","Data":"6e3c502d3fa8cd2fbf23b30d68728207705ea6401bcff9d7aa8a87df3d0db7de"}
Sep 12 00:22:12.363873 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:22:12.363475    2571 scope.go:117] "RemoveContainer" containerID="cd9b431a5120d48ed4ad8117e03c153087687bcab8e9304912b036c5a6e1649c"
Sep 12 00:22:12.363873 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:22:12.363401    2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-744fcccbf-5vhvb"
Sep 12 00:22:12.391149 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:22:12.391122    2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-744fcccbf-5vhvb"]
Sep 12 00:22:12.394541 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:22:12.394516    2571 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-image-registry/image-registry-744fcccbf-5vhvb"]
Sep 12 00:22:13.155656 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:22:13.155620    2571 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="80cbfa96-879b-48ea-8732-d50ebebea182" path="/var/lib/kubelet/pods/80cbfa96-879b-48ea-8732-d50ebebea182/volumes"
Sep 12 00:22:26.240106 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:22:26.240034    2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Sep 12 00:22:26.240619 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:22:26.240344    2571 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="c65dfaaf-3dd5-451f-98c1-7ae44786fb2a" containerName="prometheus" containerID="cri-o://34cc2f3b6cddb3895459b0c9a20c2cd6e5aa267a5e1cb8e2e5bc79da8e09a1cc" gracePeriod=600
Sep 12 00:22:26.240619 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:22:26.240379    2571 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="c65dfaaf-3dd5-451f-98c1-7ae44786fb2a" containerName="config-reloader" containerID="cri-o://d56861000d33e8db1c5e5200802d1330b5c6ad7cf8aee53e003279cdd4cf0cfc" gracePeriod=600
Sep 12 00:22:26.240619 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:22:26.240386    2571 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="c65dfaaf-3dd5-451f-98c1-7ae44786fb2a" containerName="kube-rbac-proxy-web" containerID="cri-o://d2be2428757c823ea80d8e3553e6d444104f6fa3e9c1aa371d9725c5a8bd5a8a" gracePeriod=600
Sep 12 00:22:26.240619 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:22:26.240378    2571 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="c65dfaaf-3dd5-451f-98c1-7ae44786fb2a" containerName="kube-rbac-proxy" containerID="cri-o://005838029f7431e414e24e713fb4656d8153e62831d3e5f83f2f51e00bb9753a" gracePeriod=600
Sep 12 00:22:26.240619 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:22:26.240436    2571 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="c65dfaaf-3dd5-451f-98c1-7ae44786fb2a" containerName="kube-rbac-proxy-thanos" containerID="cri-o://04268be4110e6b64ebe94e5e690d28565b1895e70929b873987f0e0ae5e6d448" gracePeriod=600
Sep 12 00:22:26.240619 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:22:26.240385    2571 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="c65dfaaf-3dd5-451f-98c1-7ae44786fb2a" containerName="thanos-sidecar" containerID="cri-o://2f2255ca86826463228a6ec8aba88c8c9798277a74652b98297046fe6560f416" gracePeriod=600
Sep 12 00:22:26.418447 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:22:26.418421    2571 generic.go:356] "Generic (PLEG): container finished" podID="c65dfaaf-3dd5-451f-98c1-7ae44786fb2a" containerID="04268be4110e6b64ebe94e5e690d28565b1895e70929b873987f0e0ae5e6d448" exitCode=0
Sep 12 00:22:26.418447 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:22:26.418445    2571 generic.go:356] "Generic (PLEG): container finished" podID="c65dfaaf-3dd5-451f-98c1-7ae44786fb2a" containerID="005838029f7431e414e24e713fb4656d8153e62831d3e5f83f2f51e00bb9753a" exitCode=0
Sep 12 00:22:26.418605 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:22:26.418456    2571 generic.go:356] "Generic (PLEG): container finished" podID="c65dfaaf-3dd5-451f-98c1-7ae44786fb2a" containerID="2f2255ca86826463228a6ec8aba88c8c9798277a74652b98297046fe6560f416" exitCode=0
Sep 12 00:22:26.418605 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:22:26.418464    2571 generic.go:356] "Generic (PLEG): container finished" podID="c65dfaaf-3dd5-451f-98c1-7ae44786fb2a" containerID="d56861000d33e8db1c5e5200802d1330b5c6ad7cf8aee53e003279cdd4cf0cfc" exitCode=0
Sep 12 00:22:26.418605 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:22:26.418474    2571 generic.go:356] "Generic (PLEG): container finished" podID="c65dfaaf-3dd5-451f-98c1-7ae44786fb2a" containerID="34cc2f3b6cddb3895459b0c9a20c2cd6e5aa267a5e1cb8e2e5bc79da8e09a1cc" exitCode=0
Sep 12 00:22:26.418605 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:22:26.418492    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"c65dfaaf-3dd5-451f-98c1-7ae44786fb2a","Type":"ContainerDied","Data":"04268be4110e6b64ebe94e5e690d28565b1895e70929b873987f0e0ae5e6d448"}
Sep 12 00:22:26.418605 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:22:26.418553    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"c65dfaaf-3dd5-451f-98c1-7ae44786fb2a","Type":"ContainerDied","Data":"005838029f7431e414e24e713fb4656d8153e62831d3e5f83f2f51e00bb9753a"}
Sep 12 00:22:26.418605 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:22:26.418582    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"c65dfaaf-3dd5-451f-98c1-7ae44786fb2a","Type":"ContainerDied","Data":"2f2255ca86826463228a6ec8aba88c8c9798277a74652b98297046fe6560f416"}
Sep 12 00:22:26.418794 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:22:26.418621    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"c65dfaaf-3dd5-451f-98c1-7ae44786fb2a","Type":"ContainerDied","Data":"d56861000d33e8db1c5e5200802d1330b5c6ad7cf8aee53e003279cdd4cf0cfc"}
Sep 12 00:22:26.418794 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:22:26.418647    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"c65dfaaf-3dd5-451f-98c1-7ae44786fb2a","Type":"ContainerDied","Data":"34cc2f3b6cddb3895459b0c9a20c2cd6e5aa267a5e1cb8e2e5bc79da8e09a1cc"}
Sep 12 00:22:27.426721 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:22:27.426685    2571 generic.go:356] "Generic (PLEG): container finished" podID="c65dfaaf-3dd5-451f-98c1-7ae44786fb2a" containerID="d2be2428757c823ea80d8e3553e6d444104f6fa3e9c1aa371d9725c5a8bd5a8a" exitCode=0
Sep 12 00:22:27.427046 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:22:27.426763    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"c65dfaaf-3dd5-451f-98c1-7ae44786fb2a","Type":"ContainerDied","Data":"d2be2428757c823ea80d8e3553e6d444104f6fa3e9c1aa371d9725c5a8bd5a8a"}
Sep 12 00:22:27.477849 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:22:27.477829    2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Sep 12 00:22:27.524896 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:22:27.524871    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c65dfaaf-3dd5-451f-98c1-7ae44786fb2a-prometheus-trusted-ca-bundle\") pod \"c65dfaaf-3dd5-451f-98c1-7ae44786fb2a\" (UID: \"c65dfaaf-3dd5-451f-98c1-7ae44786fb2a\") "
Sep 12 00:22:27.525021 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:22:27.524918    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/c65dfaaf-3dd5-451f-98c1-7ae44786fb2a-thanos-prometheus-http-client-file\") pod \"c65dfaaf-3dd5-451f-98c1-7ae44786fb2a\" (UID: \"c65dfaaf-3dd5-451f-98c1-7ae44786fb2a\") "
Sep 12 00:22:27.525021 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:22:27.524941    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/c65dfaaf-3dd5-451f-98c1-7ae44786fb2a-prometheus-k8s-db\") pod \"c65dfaaf-3dd5-451f-98c1-7ae44786fb2a\" (UID: \"c65dfaaf-3dd5-451f-98c1-7ae44786fb2a\") "
Sep 12 00:22:27.525021 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:22:27.524962    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/c65dfaaf-3dd5-451f-98c1-7ae44786fb2a-prometheus-k8s-rulefiles-0\") pod \"c65dfaaf-3dd5-451f-98c1-7ae44786fb2a\" (UID: \"c65dfaaf-3dd5-451f-98c1-7ae44786fb2a\") "
Sep 12 00:22:27.525021 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:22:27.524977    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/c65dfaaf-3dd5-451f-98c1-7ae44786fb2a-secret-prometheus-k8s-tls\") pod \"c65dfaaf-3dd5-451f-98c1-7ae44786fb2a\" (UID: \"c65dfaaf-3dd5-451f-98c1-7ae44786fb2a\") "
Sep 12 00:22:27.525021 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:22:27.524991    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/c65dfaaf-3dd5-451f-98c1-7ae44786fb2a-tls-assets\") pod \"c65dfaaf-3dd5-451f-98c1-7ae44786fb2a\" (UID: \"c65dfaaf-3dd5-451f-98c1-7ae44786fb2a\") "
Sep 12 00:22:27.525021 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:22:27.525014    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/c65dfaaf-3dd5-451f-98c1-7ae44786fb2a-web-config\") pod \"c65dfaaf-3dd5-451f-98c1-7ae44786fb2a\" (UID: \"c65dfaaf-3dd5-451f-98c1-7ae44786fb2a\") "
Sep 12 00:22:27.525303 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:22:27.525044    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/c65dfaaf-3dd5-451f-98c1-7ae44786fb2a-config\") pod \"c65dfaaf-3dd5-451f-98c1-7ae44786fb2a\" (UID: \"c65dfaaf-3dd5-451f-98c1-7ae44786fb2a\") "
Sep 12 00:22:27.525303 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:22:27.525069    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/c65dfaaf-3dd5-451f-98c1-7ae44786fb2a-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"c65dfaaf-3dd5-451f-98c1-7ae44786fb2a\" (UID: \"c65dfaaf-3dd5-451f-98c1-7ae44786fb2a\") "
Sep 12 00:22:27.525410 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:22:27.525335    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c65dfaaf-3dd5-451f-98c1-7ae44786fb2a-prometheus-trusted-ca-bundle" (OuterVolumeSpecName: "prometheus-trusted-ca-bundle") pod "c65dfaaf-3dd5-451f-98c1-7ae44786fb2a" (UID: "c65dfaaf-3dd5-451f-98c1-7ae44786fb2a"). InnerVolumeSpecName "prometheus-trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 12 00:22:27.526508 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:22:27.525545    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/c65dfaaf-3dd5-451f-98c1-7ae44786fb2a-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"c65dfaaf-3dd5-451f-98c1-7ae44786fb2a\" (UID: \"c65dfaaf-3dd5-451f-98c1-7ae44786fb2a\") "
Sep 12 00:22:27.526508 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:22:27.525639    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c65dfaaf-3dd5-451f-98c1-7ae44786fb2a-configmap-serving-certs-ca-bundle\") pod \"c65dfaaf-3dd5-451f-98c1-7ae44786fb2a\" (UID: \"c65dfaaf-3dd5-451f-98c1-7ae44786fb2a\") "
Sep 12 00:22:27.526508 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:22:27.525695    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/c65dfaaf-3dd5-451f-98c1-7ae44786fb2a-config-out\") pod \"c65dfaaf-3dd5-451f-98c1-7ae44786fb2a\" (UID: \"c65dfaaf-3dd5-451f-98c1-7ae44786fb2a\") "
Sep 12 00:22:27.526508 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:22:27.525725    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/c65dfaaf-3dd5-451f-98c1-7ae44786fb2a-prometheus-k8s-rulefiles-1\") pod \"c65dfaaf-3dd5-451f-98c1-7ae44786fb2a\" (UID: \"c65dfaaf-3dd5-451f-98c1-7ae44786fb2a\") "
Sep 12 00:22:27.526508 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:22:27.525753    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/c65dfaaf-3dd5-451f-98c1-7ae44786fb2a-secret-metrics-client-certs\") pod \"c65dfaaf-3dd5-451f-98c1-7ae44786fb2a\" (UID: \"c65dfaaf-3dd5-451f-98c1-7ae44786fb2a\") "
Sep 12 00:22:27.526508 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:22:27.525782    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c65dfaaf-3dd5-451f-98c1-7ae44786fb2a-configmap-kubelet-serving-ca-bundle\") pod \"c65dfaaf-3dd5-451f-98c1-7ae44786fb2a\" (UID: \"c65dfaaf-3dd5-451f-98c1-7ae44786fb2a\") "
Sep 12 00:22:27.526508 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:22:27.525813    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/c65dfaaf-3dd5-451f-98c1-7ae44786fb2a-prometheus-k8s-rulefiles-2\") pod \"c65dfaaf-3dd5-451f-98c1-7ae44786fb2a\" (UID: \"c65dfaaf-3dd5-451f-98c1-7ae44786fb2a\") "
Sep 12 00:22:27.526508 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:22:27.525852    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/c65dfaaf-3dd5-451f-98c1-7ae44786fb2a-secret-grpc-tls\") pod \"c65dfaaf-3dd5-451f-98c1-7ae44786fb2a\" (UID: \"c65dfaaf-3dd5-451f-98c1-7ae44786fb2a\") "
Sep 12 00:22:27.526508 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:22:27.525889    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/c65dfaaf-3dd5-451f-98c1-7ae44786fb2a-configmap-metrics-client-ca\") pod \"c65dfaaf-3dd5-451f-98c1-7ae44786fb2a\" (UID: \"c65dfaaf-3dd5-451f-98c1-7ae44786fb2a\") "
Sep 12 00:22:27.526508 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:22:27.525913    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-9lhpr\" (UniqueName: \"kubernetes.io/projected/c65dfaaf-3dd5-451f-98c1-7ae44786fb2a-kube-api-access-9lhpr\") pod \"c65dfaaf-3dd5-451f-98c1-7ae44786fb2a\" (UID: \"c65dfaaf-3dd5-451f-98c1-7ae44786fb2a\") "
Sep 12 00:22:27.526508 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:22:27.525941    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/c65dfaaf-3dd5-451f-98c1-7ae44786fb2a-secret-kube-rbac-proxy\") pod \"c65dfaaf-3dd5-451f-98c1-7ae44786fb2a\" (UID: \"c65dfaaf-3dd5-451f-98c1-7ae44786fb2a\") "
Sep 12 00:22:27.526508 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:22:27.526183    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c65dfaaf-3dd5-451f-98c1-7ae44786fb2a-prometheus-k8s-db" (OuterVolumeSpecName: "prometheus-k8s-db") pod "c65dfaaf-3dd5-451f-98c1-7ae44786fb2a" (UID: "c65dfaaf-3dd5-451f-98c1-7ae44786fb2a"). InnerVolumeSpecName "prometheus-k8s-db". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 12 00:22:27.526508 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:22:27.526232    2571 reconciler_common.go:299] "Volume detached for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c65dfaaf-3dd5-451f-98c1-7ae44786fb2a-prometheus-trusted-ca-bundle\") on node \"ip-10-0-141-128.ec2.internal\" DevicePath \"\""
Sep 12 00:22:27.527281 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:22:27.526892    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c65dfaaf-3dd5-451f-98c1-7ae44786fb2a-prometheus-k8s-rulefiles-0" (OuterVolumeSpecName: "prometheus-k8s-rulefiles-0") pod "c65dfaaf-3dd5-451f-98c1-7ae44786fb2a" (UID: "c65dfaaf-3dd5-451f-98c1-7ae44786fb2a"). InnerVolumeSpecName "prometheus-k8s-rulefiles-0". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 12 00:22:27.527662 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:22:27.527473    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c65dfaaf-3dd5-451f-98c1-7ae44786fb2a-configmap-kubelet-serving-ca-bundle" (OuterVolumeSpecName: "configmap-kubelet-serving-ca-bundle") pod "c65dfaaf-3dd5-451f-98c1-7ae44786fb2a" (UID: "c65dfaaf-3dd5-451f-98c1-7ae44786fb2a"). InnerVolumeSpecName "configmap-kubelet-serving-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 12 00:22:27.527766 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:22:27.527743    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c65dfaaf-3dd5-451f-98c1-7ae44786fb2a-configmap-metrics-client-ca" (OuterVolumeSpecName: "configmap-metrics-client-ca") pod "c65dfaaf-3dd5-451f-98c1-7ae44786fb2a" (UID: "c65dfaaf-3dd5-451f-98c1-7ae44786fb2a"). InnerVolumeSpecName "configmap-metrics-client-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 12 00:22:27.528246 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:22:27.528048    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c65dfaaf-3dd5-451f-98c1-7ae44786fb2a-configmap-serving-certs-ca-bundle" (OuterVolumeSpecName: "configmap-serving-certs-ca-bundle") pod "c65dfaaf-3dd5-451f-98c1-7ae44786fb2a" (UID: "c65dfaaf-3dd5-451f-98c1-7ae44786fb2a"). InnerVolumeSpecName "configmap-serving-certs-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 12 00:22:27.528246 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:22:27.528065    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c65dfaaf-3dd5-451f-98c1-7ae44786fb2a-prometheus-k8s-rulefiles-1" (OuterVolumeSpecName: "prometheus-k8s-rulefiles-1") pod "c65dfaaf-3dd5-451f-98c1-7ae44786fb2a" (UID: "c65dfaaf-3dd5-451f-98c1-7ae44786fb2a"). InnerVolumeSpecName "prometheus-k8s-rulefiles-1". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 12 00:22:27.528580 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:22:27.528524    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c65dfaaf-3dd5-451f-98c1-7ae44786fb2a-prometheus-k8s-rulefiles-2" (OuterVolumeSpecName: "prometheus-k8s-rulefiles-2") pod "c65dfaaf-3dd5-451f-98c1-7ae44786fb2a" (UID: "c65dfaaf-3dd5-451f-98c1-7ae44786fb2a"). InnerVolumeSpecName "prometheus-k8s-rulefiles-2". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 12 00:22:27.528781 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:22:27.528758    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c65dfaaf-3dd5-451f-98c1-7ae44786fb2a-thanos-prometheus-http-client-file" (OuterVolumeSpecName: "thanos-prometheus-http-client-file") pod "c65dfaaf-3dd5-451f-98c1-7ae44786fb2a" (UID: "c65dfaaf-3dd5-451f-98c1-7ae44786fb2a"). InnerVolumeSpecName "thanos-prometheus-http-client-file". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 12 00:22:27.528984 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:22:27.528958    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c65dfaaf-3dd5-451f-98c1-7ae44786fb2a-secret-prometheus-k8s-kube-rbac-proxy-web" (OuterVolumeSpecName: "secret-prometheus-k8s-kube-rbac-proxy-web") pod "c65dfaaf-3dd5-451f-98c1-7ae44786fb2a" (UID: "c65dfaaf-3dd5-451f-98c1-7ae44786fb2a"). InnerVolumeSpecName "secret-prometheus-k8s-kube-rbac-proxy-web". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 12 00:22:27.529201 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:22:27.529174    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c65dfaaf-3dd5-451f-98c1-7ae44786fb2a-config" (OuterVolumeSpecName: "config") pod "c65dfaaf-3dd5-451f-98c1-7ae44786fb2a" (UID: "c65dfaaf-3dd5-451f-98c1-7ae44786fb2a"). InnerVolumeSpecName "config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 12 00:22:27.530772 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:22:27.530748    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c65dfaaf-3dd5-451f-98c1-7ae44786fb2a-config-out" (OuterVolumeSpecName: "config-out") pod "c65dfaaf-3dd5-451f-98c1-7ae44786fb2a" (UID: "c65dfaaf-3dd5-451f-98c1-7ae44786fb2a"). InnerVolumeSpecName "config-out". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 12 00:22:27.530859 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:22:27.530774    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c65dfaaf-3dd5-451f-98c1-7ae44786fb2a-secret-prometheus-k8s-tls" (OuterVolumeSpecName: "secret-prometheus-k8s-tls") pod "c65dfaaf-3dd5-451f-98c1-7ae44786fb2a" (UID: "c65dfaaf-3dd5-451f-98c1-7ae44786fb2a"). InnerVolumeSpecName "secret-prometheus-k8s-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 12 00:22:27.530962 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:22:27.530931    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c65dfaaf-3dd5-451f-98c1-7ae44786fb2a-secret-prometheus-k8s-thanos-sidecar-tls" (OuterVolumeSpecName: "secret-prometheus-k8s-thanos-sidecar-tls") pod "c65dfaaf-3dd5-451f-98c1-7ae44786fb2a" (UID: "c65dfaaf-3dd5-451f-98c1-7ae44786fb2a"). InnerVolumeSpecName "secret-prometheus-k8s-thanos-sidecar-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 12 00:22:27.531096 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:22:27.531078    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c65dfaaf-3dd5-451f-98c1-7ae44786fb2a-secret-grpc-tls" (OuterVolumeSpecName: "secret-grpc-tls") pod "c65dfaaf-3dd5-451f-98c1-7ae44786fb2a" (UID: "c65dfaaf-3dd5-451f-98c1-7ae44786fb2a"). InnerVolumeSpecName "secret-grpc-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 12 00:22:27.531220 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:22:27.531196    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c65dfaaf-3dd5-451f-98c1-7ae44786fb2a-tls-assets" (OuterVolumeSpecName: "tls-assets") pod "c65dfaaf-3dd5-451f-98c1-7ae44786fb2a" (UID: "c65dfaaf-3dd5-451f-98c1-7ae44786fb2a"). InnerVolumeSpecName "tls-assets". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 12 00:22:27.531859 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:22:27.531833    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c65dfaaf-3dd5-451f-98c1-7ae44786fb2a-kube-api-access-9lhpr" (OuterVolumeSpecName: "kube-api-access-9lhpr") pod "c65dfaaf-3dd5-451f-98c1-7ae44786fb2a" (UID: "c65dfaaf-3dd5-451f-98c1-7ae44786fb2a"). InnerVolumeSpecName "kube-api-access-9lhpr". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 12 00:22:27.532128 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:22:27.532103    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c65dfaaf-3dd5-451f-98c1-7ae44786fb2a-secret-metrics-client-certs" (OuterVolumeSpecName: "secret-metrics-client-certs") pod "c65dfaaf-3dd5-451f-98c1-7ae44786fb2a" (UID: "c65dfaaf-3dd5-451f-98c1-7ae44786fb2a"). InnerVolumeSpecName "secret-metrics-client-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 12 00:22:27.532477 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:22:27.532454    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c65dfaaf-3dd5-451f-98c1-7ae44786fb2a-secret-kube-rbac-proxy" (OuterVolumeSpecName: "secret-kube-rbac-proxy") pod "c65dfaaf-3dd5-451f-98c1-7ae44786fb2a" (UID: "c65dfaaf-3dd5-451f-98c1-7ae44786fb2a"). InnerVolumeSpecName "secret-kube-rbac-proxy". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 12 00:22:27.539480 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:22:27.539445    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c65dfaaf-3dd5-451f-98c1-7ae44786fb2a-web-config" (OuterVolumeSpecName: "web-config") pod "c65dfaaf-3dd5-451f-98c1-7ae44786fb2a" (UID: "c65dfaaf-3dd5-451f-98c1-7ae44786fb2a"). InnerVolumeSpecName "web-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 12 00:22:27.627058 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:22:27.626988    2571 reconciler_common.go:299] "Volume detached for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/c65dfaaf-3dd5-451f-98c1-7ae44786fb2a-config-out\") on node \"ip-10-0-141-128.ec2.internal\" DevicePath \"\""
Sep 12 00:22:27.627058 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:22:27.627021    2571 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/c65dfaaf-3dd5-451f-98c1-7ae44786fb2a-prometheus-k8s-rulefiles-1\") on node \"ip-10-0-141-128.ec2.internal\" DevicePath \"\""
Sep 12 00:22:27.627058 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:22:27.627032    2571 reconciler_common.go:299] "Volume detached for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/c65dfaaf-3dd5-451f-98c1-7ae44786fb2a-secret-metrics-client-certs\") on node \"ip-10-0-141-128.ec2.internal\" DevicePath \"\""
Sep 12 00:22:27.627058 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:22:27.627042    2571 reconciler_common.go:299] "Volume detached for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c65dfaaf-3dd5-451f-98c1-7ae44786fb2a-configmap-kubelet-serving-ca-bundle\") on node \"ip-10-0-141-128.ec2.internal\" DevicePath \"\""
Sep 12 00:22:27.627058 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:22:27.627051    2571 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/c65dfaaf-3dd5-451f-98c1-7ae44786fb2a-prometheus-k8s-rulefiles-2\") on node \"ip-10-0-141-128.ec2.internal\" DevicePath \"\""
Sep 12 00:22:27.627058 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:22:27.627060    2571 reconciler_common.go:299] "Volume detached for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/c65dfaaf-3dd5-451f-98c1-7ae44786fb2a-secret-grpc-tls\") on node \"ip-10-0-141-128.ec2.internal\" DevicePath \"\""
Sep 12 00:22:27.627058 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:22:27.627069    2571 reconciler_common.go:299] "Volume detached for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/c65dfaaf-3dd5-451f-98c1-7ae44786fb2a-configmap-metrics-client-ca\") on node \"ip-10-0-141-128.ec2.internal\" DevicePath \"\""
Sep 12 00:22:27.627335 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:22:27.627077    2571 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9lhpr\" (UniqueName: \"kubernetes.io/projected/c65dfaaf-3dd5-451f-98c1-7ae44786fb2a-kube-api-access-9lhpr\") on node \"ip-10-0-141-128.ec2.internal\" DevicePath \"\""
Sep 12 00:22:27.627335 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:22:27.627085    2571 reconciler_common.go:299] "Volume detached for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/c65dfaaf-3dd5-451f-98c1-7ae44786fb2a-secret-kube-rbac-proxy\") on node \"ip-10-0-141-128.ec2.internal\" DevicePath \"\""
Sep 12 00:22:27.627335 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:22:27.627093    2571 reconciler_common.go:299] "Volume detached for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/c65dfaaf-3dd5-451f-98c1-7ae44786fb2a-thanos-prometheus-http-client-file\") on node \"ip-10-0-141-128.ec2.internal\" DevicePath \"\""
Sep 12 00:22:27.627335 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:22:27.627101    2571 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/c65dfaaf-3dd5-451f-98c1-7ae44786fb2a-prometheus-k8s-db\") on node \"ip-10-0-141-128.ec2.internal\" DevicePath \"\""
Sep 12 00:22:27.627335 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:22:27.627108    2571 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/c65dfaaf-3dd5-451f-98c1-7ae44786fb2a-prometheus-k8s-rulefiles-0\") on node \"ip-10-0-141-128.ec2.internal\" DevicePath \"\""
Sep 12 00:22:27.627335 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:22:27.627116    2571 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/c65dfaaf-3dd5-451f-98c1-7ae44786fb2a-secret-prometheus-k8s-tls\") on node \"ip-10-0-141-128.ec2.internal\" DevicePath \"\""
Sep 12 00:22:27.627335 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:22:27.627124    2571 reconciler_common.go:299] "Volume detached for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/c65dfaaf-3dd5-451f-98c1-7ae44786fb2a-tls-assets\") on node \"ip-10-0-141-128.ec2.internal\" DevicePath \"\""
Sep 12 00:22:27.627335 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:22:27.627132    2571 reconciler_common.go:299] "Volume detached for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/c65dfaaf-3dd5-451f-98c1-7ae44786fb2a-web-config\") on node \"ip-10-0-141-128.ec2.internal\" DevicePath \"\""
Sep 12 00:22:27.627335 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:22:27.627139    2571 reconciler_common.go:299] "Volume detached for volume \"config\" (UniqueName: \"kubernetes.io/secret/c65dfaaf-3dd5-451f-98c1-7ae44786fb2a-config\") on node \"ip-10-0-141-128.ec2.internal\" DevicePath \"\""
Sep 12 00:22:27.627335 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:22:27.627147    2571 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/c65dfaaf-3dd5-451f-98c1-7ae44786fb2a-secret-prometheus-k8s-kube-rbac-proxy-web\") on node \"ip-10-0-141-128.ec2.internal\" DevicePath \"\""
Sep 12 00:22:27.627335 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:22:27.627155    2571 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/c65dfaaf-3dd5-451f-98c1-7ae44786fb2a-secret-prometheus-k8s-thanos-sidecar-tls\") on node \"ip-10-0-141-128.ec2.internal\" DevicePath \"\""
Sep 12 00:22:27.627335 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:22:27.627164    2571 reconciler_common.go:299] "Volume detached for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c65dfaaf-3dd5-451f-98c1-7ae44786fb2a-configmap-serving-certs-ca-bundle\") on node \"ip-10-0-141-128.ec2.internal\" DevicePath \"\""
Sep 12 00:22:28.433593 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:22:28.433564    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"c65dfaaf-3dd5-451f-98c1-7ae44786fb2a","Type":"ContainerDied","Data":"519e7ec966928fb94c3fb1e1d0b7b4a2f8036fb262dd1bc0d6e7211c3a991795"}
Sep 12 00:22:28.433923 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:22:28.433611    2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Sep 12 00:22:28.433923 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:22:28.433656    2571 scope.go:117] "RemoveContainer" containerID="04268be4110e6b64ebe94e5e690d28565b1895e70929b873987f0e0ae5e6d448"
Sep 12 00:22:28.441773 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:22:28.441755    2571 scope.go:117] "RemoveContainer" containerID="005838029f7431e414e24e713fb4656d8153e62831d3e5f83f2f51e00bb9753a"
Sep 12 00:22:28.448249 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:22:28.448232    2571 scope.go:117] "RemoveContainer" containerID="d2be2428757c823ea80d8e3553e6d444104f6fa3e9c1aa371d9725c5a8bd5a8a"
Sep 12 00:22:28.454804 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:22:28.454783    2571 scope.go:117] "RemoveContainer" containerID="2f2255ca86826463228a6ec8aba88c8c9798277a74652b98297046fe6560f416"
Sep 12 00:22:28.455822 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:22:28.455673    2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Sep 12 00:22:28.462313 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:22:28.462297    2571 scope.go:117] "RemoveContainer" containerID="d56861000d33e8db1c5e5200802d1330b5c6ad7cf8aee53e003279cdd4cf0cfc"
Sep 12 00:22:28.463225 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:22:28.463208    2571 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Sep 12 00:22:28.468704 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:22:28.468688    2571 scope.go:117] "RemoveContainer" containerID="34cc2f3b6cddb3895459b0c9a20c2cd6e5aa267a5e1cb8e2e5bc79da8e09a1cc"
Sep 12 00:22:28.475673 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:22:28.475650    2571 scope.go:117] "RemoveContainer" containerID="5aaa27351c7ca308a5a2217bf3e199ec076d7da50d9e83e73d2e6ea94d7d2c9d"
Sep 12 00:22:28.491809 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:22:28.491788    2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Sep 12 00:22:28.492042 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:22:28.492031    2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c65dfaaf-3dd5-451f-98c1-7ae44786fb2a" containerName="prometheus"
Sep 12 00:22:28.492087 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:22:28.492043    2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="c65dfaaf-3dd5-451f-98c1-7ae44786fb2a" containerName="prometheus"
Sep 12 00:22:28.492087 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:22:28.492051    2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c65dfaaf-3dd5-451f-98c1-7ae44786fb2a" containerName="init-config-reloader"
Sep 12 00:22:28.492087 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:22:28.492057    2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="c65dfaaf-3dd5-451f-98c1-7ae44786fb2a" containerName="init-config-reloader"
Sep 12 00:22:28.492087 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:22:28.492065    2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c65dfaaf-3dd5-451f-98c1-7ae44786fb2a" containerName="kube-rbac-proxy"
Sep 12 00:22:28.492087 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:22:28.492072    2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="c65dfaaf-3dd5-451f-98c1-7ae44786fb2a" containerName="kube-rbac-proxy"
Sep 12 00:22:28.492087 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:22:28.492079    2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="80cbfa96-879b-48ea-8732-d50ebebea182" containerName="registry"
Sep 12 00:22:28.492087 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:22:28.492084    2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="80cbfa96-879b-48ea-8732-d50ebebea182" containerName="registry"
Sep 12 00:22:28.492300 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:22:28.492093    2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c65dfaaf-3dd5-451f-98c1-7ae44786fb2a" containerName="thanos-sidecar"
Sep 12 00:22:28.492300 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:22:28.492098    2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="c65dfaaf-3dd5-451f-98c1-7ae44786fb2a" containerName="thanos-sidecar"
Sep 12 00:22:28.492300 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:22:28.492105    2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c65dfaaf-3dd5-451f-98c1-7ae44786fb2a" containerName="kube-rbac-proxy-web"
Sep 12 00:22:28.492300 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:22:28.492110    2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="c65dfaaf-3dd5-451f-98c1-7ae44786fb2a" containerName="kube-rbac-proxy-web"
Sep 12 00:22:28.492300 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:22:28.492116    2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c65dfaaf-3dd5-451f-98c1-7ae44786fb2a" containerName="config-reloader"
Sep 12 00:22:28.492300 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:22:28.492121    2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="c65dfaaf-3dd5-451f-98c1-7ae44786fb2a" containerName="config-reloader"
Sep 12 00:22:28.492300 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:22:28.492129    2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c65dfaaf-3dd5-451f-98c1-7ae44786fb2a" containerName="kube-rbac-proxy-thanos"
Sep 12 00:22:28.492300 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:22:28.492133    2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="c65dfaaf-3dd5-451f-98c1-7ae44786fb2a" containerName="kube-rbac-proxy-thanos"
Sep 12 00:22:28.492300 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:22:28.492175    2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="c65dfaaf-3dd5-451f-98c1-7ae44786fb2a" containerName="kube-rbac-proxy"
Sep 12 00:22:28.492300 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:22:28.492182    2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="c65dfaaf-3dd5-451f-98c1-7ae44786fb2a" containerName="thanos-sidecar"
Sep 12 00:22:28.492300 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:22:28.492188    2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="c65dfaaf-3dd5-451f-98c1-7ae44786fb2a" containerName="kube-rbac-proxy-web"
Sep 12 00:22:28.492300 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:22:28.492195    2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="c65dfaaf-3dd5-451f-98c1-7ae44786fb2a" containerName="kube-rbac-proxy-thanos"
Sep 12 00:22:28.492300 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:22:28.492202    2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="c65dfaaf-3dd5-451f-98c1-7ae44786fb2a" containerName="prometheus"
Sep 12 00:22:28.492300 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:22:28.492207    2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="c65dfaaf-3dd5-451f-98c1-7ae44786fb2a" containerName="config-reloader"
Sep 12 00:22:28.492300 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:22:28.492213    2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="80cbfa96-879b-48ea-8732-d50ebebea182" containerName="registry"
Sep 12 00:22:28.497181 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:22:28.497166    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Sep 12 00:22:28.499539 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:22:28.499522    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-web-config\""
Sep 12 00:22:28.499631 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:22:28.499562    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-client-certs\""
Sep 12 00:22:28.499687 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:22:28.499660    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-grpc-tls-2dt13jv4hi1jn\""
Sep 12 00:22:28.499764 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:22:28.499746    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"serving-certs-ca-bundle\""
Sep 12 00:22:28.499815 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:22:28.499746    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-prometheus-http-client-file\""
Sep 12 00:22:28.500128 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:22:28.500111    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls-assets-0\""
Sep 12 00:22:28.500222 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:22:28.500199    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-dockercfg-r8wkk\""
Sep 12 00:22:28.500331 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:22:28.500199    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-2\""
Sep 12 00:22:28.500331 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:22:28.500237    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-1\""
Sep 12 00:22:28.500331 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:22:28.500206    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-kube-rbac-proxy-web\""
Sep 12 00:22:28.500331 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:22:28.500322    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls\""
Sep 12 00:22:28.500565 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:22:28.500546    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-sidecar-tls\""
Sep 12 00:22:28.500731 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:22:28.500715    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-rbac-proxy\""
Sep 12 00:22:28.501296 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:22:28.501274    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s\""
Sep 12 00:22:28.502674 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:22:28.502657    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-0\""
Sep 12 00:22:28.505884 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:22:28.505864    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-trusted-ca-bundle\""
Sep 12 00:22:28.511765 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:22:28.511748    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Sep 12 00:22:28.532983 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:22:28.532959    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/2ab53334-b18b-4282-a586-2a1eac8e2575-config\") pod \"prometheus-k8s-0\" (UID: \"2ab53334-b18b-4282-a586-2a1eac8e2575\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 12 00:22:28.533099 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:22:28.532992    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/2ab53334-b18b-4282-a586-2a1eac8e2575-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"2ab53334-b18b-4282-a586-2a1eac8e2575\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 12 00:22:28.533099 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:22:28.533009    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/2ab53334-b18b-4282-a586-2a1eac8e2575-web-config\") pod \"prometheus-k8s-0\" (UID: \"2ab53334-b18b-4282-a586-2a1eac8e2575\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 12 00:22:28.533099 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:22:28.533037    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2ab53334-b18b-4282-a586-2a1eac8e2575-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"2ab53334-b18b-4282-a586-2a1eac8e2575\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 12 00:22:28.533099 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:22:28.533078    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/2ab53334-b18b-4282-a586-2a1eac8e2575-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"2ab53334-b18b-4282-a586-2a1eac8e2575\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 12 00:22:28.533283 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:22:28.533194    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2ab53334-b18b-4282-a586-2a1eac8e2575-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"2ab53334-b18b-4282-a586-2a1eac8e2575\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 12 00:22:28.533283 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:22:28.533226    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2ab53334-b18b-4282-a586-2a1eac8e2575-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"2ab53334-b18b-4282-a586-2a1eac8e2575\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 12 00:22:28.533283 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:22:28.533254    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/2ab53334-b18b-4282-a586-2a1eac8e2575-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"2ab53334-b18b-4282-a586-2a1eac8e2575\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 12 00:22:28.533385 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:22:28.533285    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-47rxg\" (UniqueName: \"kubernetes.io/projected/2ab53334-b18b-4282-a586-2a1eac8e2575-kube-api-access-47rxg\") pod \"prometheus-k8s-0\" (UID: \"2ab53334-b18b-4282-a586-2a1eac8e2575\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 12 00:22:28.533385 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:22:28.533322    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/2ab53334-b18b-4282-a586-2a1eac8e2575-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"2ab53334-b18b-4282-a586-2a1eac8e2575\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 12 00:22:28.533385 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:22:28.533350    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/2ab53334-b18b-4282-a586-2a1eac8e2575-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"2ab53334-b18b-4282-a586-2a1eac8e2575\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 12 00:22:28.533385 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:22:28.533381    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/2ab53334-b18b-4282-a586-2a1eac8e2575-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"2ab53334-b18b-4282-a586-2a1eac8e2575\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 12 00:22:28.533502 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:22:28.533416    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/2ab53334-b18b-4282-a586-2a1eac8e2575-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"2ab53334-b18b-4282-a586-2a1eac8e2575\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 12 00:22:28.533502 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:22:28.533446    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/2ab53334-b18b-4282-a586-2a1eac8e2575-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"2ab53334-b18b-4282-a586-2a1eac8e2575\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 12 00:22:28.533502 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:22:28.533472    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/2ab53334-b18b-4282-a586-2a1eac8e2575-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"2ab53334-b18b-4282-a586-2a1eac8e2575\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 12 00:22:28.533502 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:22:28.533491    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/2ab53334-b18b-4282-a586-2a1eac8e2575-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"2ab53334-b18b-4282-a586-2a1eac8e2575\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 12 00:22:28.533639 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:22:28.533515    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/2ab53334-b18b-4282-a586-2a1eac8e2575-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"2ab53334-b18b-4282-a586-2a1eac8e2575\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 12 00:22:28.533639 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:22:28.533531    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/2ab53334-b18b-4282-a586-2a1eac8e2575-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"2ab53334-b18b-4282-a586-2a1eac8e2575\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 12 00:22:28.533639 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:22:28.533547    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/2ab53334-b18b-4282-a586-2a1eac8e2575-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"2ab53334-b18b-4282-a586-2a1eac8e2575\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 12 00:22:28.533639 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:22:28.533562    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/2ab53334-b18b-4282-a586-2a1eac8e2575-config-out\") pod \"prometheus-k8s-0\" (UID: \"2ab53334-b18b-4282-a586-2a1eac8e2575\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 12 00:22:28.634058 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:22:28.634004    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/2ab53334-b18b-4282-a586-2a1eac8e2575-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"2ab53334-b18b-4282-a586-2a1eac8e2575\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 12 00:22:28.634058 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:22:28.634046    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2ab53334-b18b-4282-a586-2a1eac8e2575-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"2ab53334-b18b-4282-a586-2a1eac8e2575\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 12 00:22:28.634194 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:22:28.634062    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2ab53334-b18b-4282-a586-2a1eac8e2575-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"2ab53334-b18b-4282-a586-2a1eac8e2575\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 12 00:22:28.634194 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:22:28.634077    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/2ab53334-b18b-4282-a586-2a1eac8e2575-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"2ab53334-b18b-4282-a586-2a1eac8e2575\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 12 00:22:28.634194 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:22:28.634113    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-47rxg\" (UniqueName: \"kubernetes.io/projected/2ab53334-b18b-4282-a586-2a1eac8e2575-kube-api-access-47rxg\") pod \"prometheus-k8s-0\" (UID: \"2ab53334-b18b-4282-a586-2a1eac8e2575\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 12 00:22:28.634194 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:22:28.634153    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/2ab53334-b18b-4282-a586-2a1eac8e2575-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"2ab53334-b18b-4282-a586-2a1eac8e2575\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 12 00:22:28.634194 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:22:28.634182    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/2ab53334-b18b-4282-a586-2a1eac8e2575-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"2ab53334-b18b-4282-a586-2a1eac8e2575\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 12 00:22:28.634420 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:22:28.634217    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/2ab53334-b18b-4282-a586-2a1eac8e2575-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"2ab53334-b18b-4282-a586-2a1eac8e2575\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 12 00:22:28.634420 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:22:28.634240    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/2ab53334-b18b-4282-a586-2a1eac8e2575-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"2ab53334-b18b-4282-a586-2a1eac8e2575\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 12 00:22:28.634420 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:22:28.634270    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/2ab53334-b18b-4282-a586-2a1eac8e2575-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"2ab53334-b18b-4282-a586-2a1eac8e2575\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 12 00:22:28.634420 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:22:28.634298    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/2ab53334-b18b-4282-a586-2a1eac8e2575-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"2ab53334-b18b-4282-a586-2a1eac8e2575\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 12 00:22:28.634420 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:22:28.634324    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/2ab53334-b18b-4282-a586-2a1eac8e2575-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"2ab53334-b18b-4282-a586-2a1eac8e2575\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 12 00:22:28.634420 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:22:28.634362    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/2ab53334-b18b-4282-a586-2a1eac8e2575-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"2ab53334-b18b-4282-a586-2a1eac8e2575\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 12 00:22:28.634420 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:22:28.634385    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/2ab53334-b18b-4282-a586-2a1eac8e2575-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"2ab53334-b18b-4282-a586-2a1eac8e2575\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 12 00:22:28.634420 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:22:28.634390    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/2ab53334-b18b-4282-a586-2a1eac8e2575-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"2ab53334-b18b-4282-a586-2a1eac8e2575\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 12 00:22:28.634420 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:22:28.634411    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/2ab53334-b18b-4282-a586-2a1eac8e2575-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"2ab53334-b18b-4282-a586-2a1eac8e2575\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 12 00:22:28.634875 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:22:28.634433    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/2ab53334-b18b-4282-a586-2a1eac8e2575-config-out\") pod \"prometheus-k8s-0\" (UID: \"2ab53334-b18b-4282-a586-2a1eac8e2575\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 12 00:22:28.634875 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:22:28.634463    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/2ab53334-b18b-4282-a586-2a1eac8e2575-config\") pod \"prometheus-k8s-0\" (UID: \"2ab53334-b18b-4282-a586-2a1eac8e2575\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 12 00:22:28.634875 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:22:28.634497    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/2ab53334-b18b-4282-a586-2a1eac8e2575-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"2ab53334-b18b-4282-a586-2a1eac8e2575\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 12 00:22:28.634875 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:22:28.634527    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/2ab53334-b18b-4282-a586-2a1eac8e2575-web-config\") pod \"prometheus-k8s-0\" (UID: \"2ab53334-b18b-4282-a586-2a1eac8e2575\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 12 00:22:28.634875 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:22:28.634579    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2ab53334-b18b-4282-a586-2a1eac8e2575-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"2ab53334-b18b-4282-a586-2a1eac8e2575\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 12 00:22:28.634875 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:22:28.634814    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/2ab53334-b18b-4282-a586-2a1eac8e2575-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"2ab53334-b18b-4282-a586-2a1eac8e2575\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 12 00:22:28.634875 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:22:28.634835    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2ab53334-b18b-4282-a586-2a1eac8e2575-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"2ab53334-b18b-4282-a586-2a1eac8e2575\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 12 00:22:28.635290 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:22:28.635267    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2ab53334-b18b-4282-a586-2a1eac8e2575-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"2ab53334-b18b-4282-a586-2a1eac8e2575\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 12 00:22:28.635807 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:22:28.635778    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2ab53334-b18b-4282-a586-2a1eac8e2575-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"2ab53334-b18b-4282-a586-2a1eac8e2575\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 12 00:22:28.636450 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:22:28.636425    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/2ab53334-b18b-4282-a586-2a1eac8e2575-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"2ab53334-b18b-4282-a586-2a1eac8e2575\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 12 00:22:28.637308 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:22:28.637279    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/2ab53334-b18b-4282-a586-2a1eac8e2575-config-out\") pod \"prometheus-k8s-0\" (UID: \"2ab53334-b18b-4282-a586-2a1eac8e2575\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 12 00:22:28.637308 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:22:28.637284    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/2ab53334-b18b-4282-a586-2a1eac8e2575-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"2ab53334-b18b-4282-a586-2a1eac8e2575\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 12 00:22:28.637460 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:22:28.637378    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/2ab53334-b18b-4282-a586-2a1eac8e2575-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"2ab53334-b18b-4282-a586-2a1eac8e2575\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 12 00:22:28.637908 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:22:28.637882    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/2ab53334-b18b-4282-a586-2a1eac8e2575-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"2ab53334-b18b-4282-a586-2a1eac8e2575\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 12 00:22:28.638309 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:22:28.638282    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/2ab53334-b18b-4282-a586-2a1eac8e2575-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"2ab53334-b18b-4282-a586-2a1eac8e2575\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 12 00:22:28.639079 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:22:28.638423    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/2ab53334-b18b-4282-a586-2a1eac8e2575-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"2ab53334-b18b-4282-a586-2a1eac8e2575\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 12 00:22:28.639079 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:22:28.638554    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/secret/2ab53334-b18b-4282-a586-2a1eac8e2575-config\") pod \"prometheus-k8s-0\" (UID: \"2ab53334-b18b-4282-a586-2a1eac8e2575\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 12 00:22:28.639079 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:22:28.638967    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/2ab53334-b18b-4282-a586-2a1eac8e2575-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"2ab53334-b18b-4282-a586-2a1eac8e2575\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 12 00:22:28.639079 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:22:28.639012    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/2ab53334-b18b-4282-a586-2a1eac8e2575-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"2ab53334-b18b-4282-a586-2a1eac8e2575\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 12 00:22:28.639764 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:22:28.639741    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/2ab53334-b18b-4282-a586-2a1eac8e2575-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"2ab53334-b18b-4282-a586-2a1eac8e2575\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 12 00:22:28.639865 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:22:28.639800    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/2ab53334-b18b-4282-a586-2a1eac8e2575-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"2ab53334-b18b-4282-a586-2a1eac8e2575\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 12 00:22:28.640471 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:22:28.640450    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/2ab53334-b18b-4282-a586-2a1eac8e2575-web-config\") pod \"prometheus-k8s-0\" (UID: \"2ab53334-b18b-4282-a586-2a1eac8e2575\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 12 00:22:28.640535 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:22:28.640511    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/2ab53334-b18b-4282-a586-2a1eac8e2575-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"2ab53334-b18b-4282-a586-2a1eac8e2575\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 12 00:22:28.641729 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:22:28.641713    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-47rxg\" (UniqueName: \"kubernetes.io/projected/2ab53334-b18b-4282-a586-2a1eac8e2575-kube-api-access-47rxg\") pod \"prometheus-k8s-0\" (UID: \"2ab53334-b18b-4282-a586-2a1eac8e2575\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 12 00:22:28.807884 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:22:28.807848    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Sep 12 00:22:29.091424 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:22:29.091396    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Sep 12 00:22:29.094040 ip-10-0-141-128 kubenswrapper[2571]: W0912 00:22:29.094013    2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2ab53334_b18b_4282_a586_2a1eac8e2575.slice/crio-ec6d70617425b82ddf5016d14208c84d15b3f4f07a7397df2272f14dbfd75640 WatchSource:0}: Error finding container ec6d70617425b82ddf5016d14208c84d15b3f4f07a7397df2272f14dbfd75640: Status 404 returned error can't find the container with id ec6d70617425b82ddf5016d14208c84d15b3f4f07a7397df2272f14dbfd75640
Sep 12 00:22:29.139689 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:22:29.139661    2571 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c65dfaaf-3dd5-451f-98c1-7ae44786fb2a" path="/var/lib/kubelet/pods/c65dfaaf-3dd5-451f-98c1-7ae44786fb2a/volumes"
Sep 12 00:22:29.438601 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:22:29.438562    2571 generic.go:356] "Generic (PLEG): container finished" podID="2ab53334-b18b-4282-a586-2a1eac8e2575" containerID="0344af460a11404c75de5fb217a65df31afd4de744ebd517af34a19968e3421e" exitCode=0
Sep 12 00:22:29.438923 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:22:29.438651    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"2ab53334-b18b-4282-a586-2a1eac8e2575","Type":"ContainerDied","Data":"0344af460a11404c75de5fb217a65df31afd4de744ebd517af34a19968e3421e"}
Sep 12 00:22:29.438923 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:22:29.438722    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"2ab53334-b18b-4282-a586-2a1eac8e2575","Type":"ContainerStarted","Data":"ec6d70617425b82ddf5016d14208c84d15b3f4f07a7397df2272f14dbfd75640"}
Sep 12 00:22:30.446203 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:22:30.446170    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"2ab53334-b18b-4282-a586-2a1eac8e2575","Type":"ContainerStarted","Data":"3e673e8fb75b0aa9e6f8e22e3ced9662db4fc461946e51720f74aada9c02cb62"}
Sep 12 00:22:30.446623 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:22:30.446252    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"2ab53334-b18b-4282-a586-2a1eac8e2575","Type":"ContainerStarted","Data":"a686dd97d0b7961c276b049238d3fc22198ba2f01d9a5a55ade18d3adddf1aec"}
Sep 12 00:22:30.446623 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:22:30.446278    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"2ab53334-b18b-4282-a586-2a1eac8e2575","Type":"ContainerStarted","Data":"54fc20ab4b7a96259d8d245561acf04a0ba0777d8c8d4a504a07ce189b4b999b"}
Sep 12 00:22:30.446623 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:22:30.446308    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"2ab53334-b18b-4282-a586-2a1eac8e2575","Type":"ContainerStarted","Data":"d612d0d815e76d1dadfe2c2d1878aaba5c2b5e94155e027b9ee702cd437a5cb8"}
Sep 12 00:22:30.446623 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:22:30.446332    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"2ab53334-b18b-4282-a586-2a1eac8e2575","Type":"ContainerStarted","Data":"5ea61043bbb51071846be55788c41cba73a3545cd3bbbdbf49252d0e33619a5f"}
Sep 12 00:22:30.446623 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:22:30.446354    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"2ab53334-b18b-4282-a586-2a1eac8e2575","Type":"ContainerStarted","Data":"43099f0107f7e96e7d697cb891329858477443bd75ab9a0584fc4bbae7d3f9ca"}
Sep 12 00:22:30.476604 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:22:30.476144    2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-k8s-0" podStartSLOduration=2.461967063 podStartE2EDuration="2.476123219s" totalImagesPullingTime="0s" totalInitContainerRuntime="14.156156ms" isStatefulPod=false podCreationTimestamp="2026-09-12 00:22:28 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-12 00:22:30.473456327 +0000 UTC m=+361.745096830" watchObservedRunningTime="2026-09-12 00:22:30.476123219 +0000 UTC m=+361.747763732"
Sep 12 00:22:33.808842 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:22:33.808798    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-k8s-0"
Sep 12 00:22:42.142778 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:22:42.142745    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/6d479776-cc30-4bea-bb3a-8b4c259d59c7-metrics-certs\") pod \"network-metrics-daemon-bdp25\" (UID: \"6d479776-cc30-4bea-bb3a-8b4c259d59c7\") " pod="openshift-multus/network-metrics-daemon-bdp25"
Sep 12 00:22:42.145044 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:22:42.145011    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/6d479776-cc30-4bea-bb3a-8b4c259d59c7-metrics-certs\") pod \"network-metrics-daemon-bdp25\" (UID: \"6d479776-cc30-4bea-bb3a-8b4c259d59c7\") " pod="openshift-multus/network-metrics-daemon-bdp25"
Sep 12 00:22:42.337990 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:22:42.337959    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-vv7j9\""
Sep 12 00:22:42.345948 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:22:42.345926    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-bdp25"
Sep 12 00:22:42.613106 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:22:42.613076    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-bdp25"]
Sep 12 00:22:42.616174 ip-10-0-141-128 kubenswrapper[2571]: W0912 00:22:42.616139    2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6d479776_cc30_4bea_bb3a_8b4c259d59c7.slice/crio-9aae2f171d199ff460c9c770adf0c40a8cb1c25f9fb5aa204948b90f0374fc2a WatchSource:0}: Error finding container 9aae2f171d199ff460c9c770adf0c40a8cb1c25f9fb5aa204948b90f0374fc2a: Status 404 returned error can't find the container with id 9aae2f171d199ff460c9c770adf0c40a8cb1c25f9fb5aa204948b90f0374fc2a
Sep 12 00:22:42.618067 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:22:42.618048    2571 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 12 00:22:43.496251 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:22:43.496211    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-bdp25" event={"ID":"6d479776-cc30-4bea-bb3a-8b4c259d59c7","Type":"ContainerStarted","Data":"9aae2f171d199ff460c9c770adf0c40a8cb1c25f9fb5aa204948b90f0374fc2a"}
Sep 12 00:22:44.501239 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:22:44.501201    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-bdp25" event={"ID":"6d479776-cc30-4bea-bb3a-8b4c259d59c7","Type":"ContainerStarted","Data":"13e99e2a768edd1e608d699438aea5bcdcd4a0f072725b8e7b3db9c77751d64a"}
Sep 12 00:22:44.501569 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:22:44.501266    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-bdp25" event={"ID":"6d479776-cc30-4bea-bb3a-8b4c259d59c7","Type":"ContainerStarted","Data":"0ee7e2a3ecec6abcd003bfebd8a6c725e24144b5c01e0fee1ee95060b3e70c74"}
Sep 12 00:22:44.518206 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:22:44.518159    2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-bdp25" podStartSLOduration=373.385868609 podStartE2EDuration="6m14.518143652s" totalImagesPullingTime="1.132275043s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-12 00:16:30 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-12 00:22:44.515861036 +0000 UTC m=+375.787501545" watchObservedRunningTime="2026-09-12 00:22:44.518143652 +0000 UTC m=+375.789784161"
Sep 12 00:23:28.808875 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:23:28.808842    2571 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/prometheus-k8s-0"
Sep 12 00:23:28.823840 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:23:28.823818    2571 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/prometheus-k8s-0"
Sep 12 00:23:29.682984 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:23:29.682956    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-k8s-0"
Sep 12 00:25:55.529064 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:25:55.529030    2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-8446f459d6-dngdm"]
Sep 12 00:25:55.532238 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:25:55.532221    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-8446f459d6-dngdm"
Sep 12 00:25:55.534705 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:25:55.534669    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\""
Sep 12 00:25:55.534705 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:25:55.534687    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-mb7xc\""
Sep 12 00:25:55.534881 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:25:55.534677    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\""
Sep 12 00:25:55.534881 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:25:55.534780    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\""
Sep 12 00:25:55.535026 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:25:55.535013    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\""
Sep 12 00:25:55.550419 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:25:55.550395    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-8446f459d6-dngdm"]
Sep 12 00:25:55.586533 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:25:55.586511    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/618d0f14-c54a-4aec-bba9-3d1c67ab23a7-tmp\") pod \"opendatahub-operator-controller-manager-8446f459d6-dngdm\" (UID: \"618d0f14-c54a-4aec-bba9-3d1c67ab23a7\") " pod="opendatahub/opendatahub-operator-controller-manager-8446f459d6-dngdm"
Sep 12 00:25:55.586651 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:25:55.586543    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/618d0f14-c54a-4aec-bba9-3d1c67ab23a7-webhook-cert\") pod \"opendatahub-operator-controller-manager-8446f459d6-dngdm\" (UID: \"618d0f14-c54a-4aec-bba9-3d1c67ab23a7\") " pod="opendatahub/opendatahub-operator-controller-manager-8446f459d6-dngdm"
Sep 12 00:25:55.586651 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:25:55.586578    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hfml4\" (UniqueName: \"kubernetes.io/projected/618d0f14-c54a-4aec-bba9-3d1c67ab23a7-kube-api-access-hfml4\") pod \"opendatahub-operator-controller-manager-8446f459d6-dngdm\" (UID: \"618d0f14-c54a-4aec-bba9-3d1c67ab23a7\") " pod="opendatahub/opendatahub-operator-controller-manager-8446f459d6-dngdm"
Sep 12 00:25:55.586651 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:25:55.586636    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/618d0f14-c54a-4aec-bba9-3d1c67ab23a7-apiservice-cert\") pod \"opendatahub-operator-controller-manager-8446f459d6-dngdm\" (UID: \"618d0f14-c54a-4aec-bba9-3d1c67ab23a7\") " pod="opendatahub/opendatahub-operator-controller-manager-8446f459d6-dngdm"
Sep 12 00:25:55.586778 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:25:55.586655    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/618d0f14-c54a-4aec-bba9-3d1c67ab23a7-manifests\") pod \"opendatahub-operator-controller-manager-8446f459d6-dngdm\" (UID: \"618d0f14-c54a-4aec-bba9-3d1c67ab23a7\") " pod="opendatahub/opendatahub-operator-controller-manager-8446f459d6-dngdm"
Sep 12 00:25:55.688325 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:25:55.687791    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/618d0f14-c54a-4aec-bba9-3d1c67ab23a7-manifests\") pod \"opendatahub-operator-controller-manager-8446f459d6-dngdm\" (UID: \"618d0f14-c54a-4aec-bba9-3d1c67ab23a7\") " pod="opendatahub/opendatahub-operator-controller-manager-8446f459d6-dngdm"
Sep 12 00:25:55.688325 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:25:55.687853    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/618d0f14-c54a-4aec-bba9-3d1c67ab23a7-tmp\") pod \"opendatahub-operator-controller-manager-8446f459d6-dngdm\" (UID: \"618d0f14-c54a-4aec-bba9-3d1c67ab23a7\") " pod="opendatahub/opendatahub-operator-controller-manager-8446f459d6-dngdm"
Sep 12 00:25:55.688325 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:25:55.687894    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/618d0f14-c54a-4aec-bba9-3d1c67ab23a7-webhook-cert\") pod \"opendatahub-operator-controller-manager-8446f459d6-dngdm\" (UID: \"618d0f14-c54a-4aec-bba9-3d1c67ab23a7\") " pod="opendatahub/opendatahub-operator-controller-manager-8446f459d6-dngdm"
Sep 12 00:25:55.688325 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:25:55.687963    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hfml4\" (UniqueName: \"kubernetes.io/projected/618d0f14-c54a-4aec-bba9-3d1c67ab23a7-kube-api-access-hfml4\") pod \"opendatahub-operator-controller-manager-8446f459d6-dngdm\" (UID: \"618d0f14-c54a-4aec-bba9-3d1c67ab23a7\") " pod="opendatahub/opendatahub-operator-controller-manager-8446f459d6-dngdm"
Sep 12 00:25:55.688325 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:25:55.687998    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/618d0f14-c54a-4aec-bba9-3d1c67ab23a7-apiservice-cert\") pod \"opendatahub-operator-controller-manager-8446f459d6-dngdm\" (UID: \"618d0f14-c54a-4aec-bba9-3d1c67ab23a7\") " pod="opendatahub/opendatahub-operator-controller-manager-8446f459d6-dngdm"
Sep 12 00:25:55.688872 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:25:55.688845    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/618d0f14-c54a-4aec-bba9-3d1c67ab23a7-manifests\") pod \"opendatahub-operator-controller-manager-8446f459d6-dngdm\" (UID: \"618d0f14-c54a-4aec-bba9-3d1c67ab23a7\") " pod="opendatahub/opendatahub-operator-controller-manager-8446f459d6-dngdm"
Sep 12 00:25:55.691993 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:25:55.691971    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/618d0f14-c54a-4aec-bba9-3d1c67ab23a7-webhook-cert\") pod \"opendatahub-operator-controller-manager-8446f459d6-dngdm\" (UID: \"618d0f14-c54a-4aec-bba9-3d1c67ab23a7\") " pod="opendatahub/opendatahub-operator-controller-manager-8446f459d6-dngdm"
Sep 12 00:25:55.693031 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:25:55.692985    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/618d0f14-c54a-4aec-bba9-3d1c67ab23a7-tmp\") pod \"opendatahub-operator-controller-manager-8446f459d6-dngdm\" (UID: \"618d0f14-c54a-4aec-bba9-3d1c67ab23a7\") " pod="opendatahub/opendatahub-operator-controller-manager-8446f459d6-dngdm"
Sep 12 00:25:55.693031 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:25:55.693022    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/618d0f14-c54a-4aec-bba9-3d1c67ab23a7-apiservice-cert\") pod \"opendatahub-operator-controller-manager-8446f459d6-dngdm\" (UID: \"618d0f14-c54a-4aec-bba9-3d1c67ab23a7\") " pod="opendatahub/opendatahub-operator-controller-manager-8446f459d6-dngdm"
Sep 12 00:25:55.697735 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:25:55.697709    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hfml4\" (UniqueName: \"kubernetes.io/projected/618d0f14-c54a-4aec-bba9-3d1c67ab23a7-kube-api-access-hfml4\") pod \"opendatahub-operator-controller-manager-8446f459d6-dngdm\" (UID: \"618d0f14-c54a-4aec-bba9-3d1c67ab23a7\") " pod="opendatahub/opendatahub-operator-controller-manager-8446f459d6-dngdm"
Sep 12 00:25:55.842971 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:25:55.842894    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-8446f459d6-dngdm"
Sep 12 00:25:56.128267 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:25:56.128245    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-8446f459d6-dngdm"]
Sep 12 00:25:56.130764 ip-10-0-141-128 kubenswrapper[2571]: W0912 00:25:56.130736    2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod618d0f14_c54a_4aec_bba9_3d1c67ab23a7.slice/crio-ed3613b34f2d56350c43ecaa94000950a3cc72529edb42f9ee6bf3affc99a09b WatchSource:0}: Error finding container ed3613b34f2d56350c43ecaa94000950a3cc72529edb42f9ee6bf3affc99a09b: Status 404 returned error can't find the container with id ed3613b34f2d56350c43ecaa94000950a3cc72529edb42f9ee6bf3affc99a09b
Sep 12 00:25:56.212826 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:25:56.212800    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-8446f459d6-dngdm" event={"ID":"618d0f14-c54a-4aec-bba9-3d1c67ab23a7","Type":"ContainerStarted","Data":"ed3613b34f2d56350c43ecaa94000950a3cc72529edb42f9ee6bf3affc99a09b"}
Sep 12 00:26:00.232418 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:26:00.232380    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-8446f459d6-dngdm" event={"ID":"618d0f14-c54a-4aec-bba9-3d1c67ab23a7","Type":"ContainerStarted","Data":"5240bff7baad20a1bae6b203e2a14b43908c0ef2cb39f4d9312eb988f45026ab"}
Sep 12 00:26:01.238578 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:26:01.238544    2571 generic.go:356] "Generic (PLEG): container finished" podID="618d0f14-c54a-4aec-bba9-3d1c67ab23a7" containerID="5240bff7baad20a1bae6b203e2a14b43908c0ef2cb39f4d9312eb988f45026ab" exitCode=0
Sep 12 00:26:01.238946 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:26:01.238623    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-8446f459d6-dngdm" event={"ID":"618d0f14-c54a-4aec-bba9-3d1c67ab23a7","Type":"ContainerDied","Data":"5240bff7baad20a1bae6b203e2a14b43908c0ef2cb39f4d9312eb988f45026ab"}
Sep 12 00:26:02.244457 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:26:02.244421    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-8446f459d6-dngdm" event={"ID":"618d0f14-c54a-4aec-bba9-3d1c67ab23a7","Type":"ContainerStarted","Data":"9c3cb52b3dd1492eb60bd8cf9bb76cc2c882916868f5087e9cd24f014203861a"}
Sep 12 00:26:02.244852 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:26:02.244554    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-8446f459d6-dngdm"
Sep 12 00:26:02.267127 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:26:02.267074    2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-8446f459d6-dngdm" podStartSLOduration=2.723461172 podStartE2EDuration="7.267058704s" totalImagesPullingTime="3.477073369s" totalInitContainerRuntime="1.066524163s" isStatefulPod=false podCreationTimestamp="2026-09-12 00:25:55 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-12 00:26:02.26466702 +0000 UTC m=+573.536307531" watchObservedRunningTime="2026-09-12 00:26:02.267058704 +0000 UTC m=+573.538699213"
Sep 12 00:26:13.250681 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:26:13.250652    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-8446f459d6-dngdm"
Sep 12 00:26:15.342249 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:26:15.342210    2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/kube-auth-proxy-76b8774589-59jxv"]
Sep 12 00:26:15.355985 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:26:15.355958    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-76b8774589-59jxv"]
Sep 12 00:26:15.356110 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:26:15.356046    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-76b8774589-59jxv"
Sep 12 00:26:15.358733 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:26:15.358705    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-tls\""
Sep 12 00:26:15.358864 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:26:15.358815    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-creds\""
Sep 12 00:26:15.445693 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:26:15.445665    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-22xp7\" (UniqueName: \"kubernetes.io/projected/e4d384f3-026e-4343-bada-ddc631f69d53-kube-api-access-22xp7\") pod \"kube-auth-proxy-76b8774589-59jxv\" (UID: \"e4d384f3-026e-4343-bada-ddc631f69d53\") " pod="openshift-ingress/kube-auth-proxy-76b8774589-59jxv"
Sep 12 00:26:15.445693 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:26:15.445692    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/e4d384f3-026e-4343-bada-ddc631f69d53-tmp\") pod \"kube-auth-proxy-76b8774589-59jxv\" (UID: \"e4d384f3-026e-4343-bada-ddc631f69d53\") " pod="openshift-ingress/kube-auth-proxy-76b8774589-59jxv"
Sep 12 00:26:15.445892 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:26:15.445788    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e4d384f3-026e-4343-bada-ddc631f69d53-tls-certs\") pod \"kube-auth-proxy-76b8774589-59jxv\" (UID: \"e4d384f3-026e-4343-bada-ddc631f69d53\") " pod="openshift-ingress/kube-auth-proxy-76b8774589-59jxv"
Sep 12 00:26:15.546768 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:26:15.546739    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e4d384f3-026e-4343-bada-ddc631f69d53-tls-certs\") pod \"kube-auth-proxy-76b8774589-59jxv\" (UID: \"e4d384f3-026e-4343-bada-ddc631f69d53\") " pod="openshift-ingress/kube-auth-proxy-76b8774589-59jxv"
Sep 12 00:26:15.546891 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:26:15.546779    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-22xp7\" (UniqueName: \"kubernetes.io/projected/e4d384f3-026e-4343-bada-ddc631f69d53-kube-api-access-22xp7\") pod \"kube-auth-proxy-76b8774589-59jxv\" (UID: \"e4d384f3-026e-4343-bada-ddc631f69d53\") " pod="openshift-ingress/kube-auth-proxy-76b8774589-59jxv"
Sep 12 00:26:15.546891 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:26:15.546795    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/e4d384f3-026e-4343-bada-ddc631f69d53-tmp\") pod \"kube-auth-proxy-76b8774589-59jxv\" (UID: \"e4d384f3-026e-4343-bada-ddc631f69d53\") " pod="openshift-ingress/kube-auth-proxy-76b8774589-59jxv"
Sep 12 00:26:15.546891 ip-10-0-141-128 kubenswrapper[2571]: E0912 00:26:15.546871    2571 secret.go:189] Couldn't get secret openshift-ingress/kube-auth-proxy-tls: secret "kube-auth-proxy-tls" not found
Sep 12 00:26:15.547041 ip-10-0-141-128 kubenswrapper[2571]: E0912 00:26:15.546934    2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e4d384f3-026e-4343-bada-ddc631f69d53-tls-certs podName:e4d384f3-026e-4343-bada-ddc631f69d53 nodeName:}" failed. No retries permitted until 2026-09-12 00:26:16.04691909 +0000 UTC m=+587.318559597 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "tls-certs" (UniqueName: "kubernetes.io/secret/e4d384f3-026e-4343-bada-ddc631f69d53-tls-certs") pod "kube-auth-proxy-76b8774589-59jxv" (UID: "e4d384f3-026e-4343-bada-ddc631f69d53") : secret "kube-auth-proxy-tls" not found
Sep 12 00:26:15.548858 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:26:15.548842    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/e4d384f3-026e-4343-bada-ddc631f69d53-tmp\") pod \"kube-auth-proxy-76b8774589-59jxv\" (UID: \"e4d384f3-026e-4343-bada-ddc631f69d53\") " pod="openshift-ingress/kube-auth-proxy-76b8774589-59jxv"
Sep 12 00:26:15.561067 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:26:15.561048    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-22xp7\" (UniqueName: \"kubernetes.io/projected/e4d384f3-026e-4343-bada-ddc631f69d53-kube-api-access-22xp7\") pod \"kube-auth-proxy-76b8774589-59jxv\" (UID: \"e4d384f3-026e-4343-bada-ddc631f69d53\") " pod="openshift-ingress/kube-auth-proxy-76b8774589-59jxv"
Sep 12 00:26:16.051441 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:26:16.051407    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e4d384f3-026e-4343-bada-ddc631f69d53-tls-certs\") pod \"kube-auth-proxy-76b8774589-59jxv\" (UID: \"e4d384f3-026e-4343-bada-ddc631f69d53\") " pod="openshift-ingress/kube-auth-proxy-76b8774589-59jxv"
Sep 12 00:26:16.053803 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:26:16.053781    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e4d384f3-026e-4343-bada-ddc631f69d53-tls-certs\") pod \"kube-auth-proxy-76b8774589-59jxv\" (UID: \"e4d384f3-026e-4343-bada-ddc631f69d53\") " pod="openshift-ingress/kube-auth-proxy-76b8774589-59jxv"
Sep 12 00:26:16.266398 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:26:16.266365    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-76b8774589-59jxv"
Sep 12 00:26:16.539526 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:26:16.539501    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-76b8774589-59jxv"]
Sep 12 00:26:16.542228 ip-10-0-141-128 kubenswrapper[2571]: W0912 00:26:16.542197    2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode4d384f3_026e_4343_bada_ddc631f69d53.slice/crio-1c75f6895649d099c4747ac113e71b26b94f4a452936df0df6fe71386daef587 WatchSource:0}: Error finding container 1c75f6895649d099c4747ac113e71b26b94f4a452936df0df6fe71386daef587: Status 404 returned error can't find the container with id 1c75f6895649d099c4747ac113e71b26b94f4a452936df0df6fe71386daef587
Sep 12 00:26:17.306041 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:26:17.306004    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-76b8774589-59jxv" event={"ID":"e4d384f3-026e-4343-bada-ddc631f69d53","Type":"ContainerStarted","Data":"1c75f6895649d099c4747ac113e71b26b94f4a452936df0df6fe71386daef587"}
Sep 12 00:26:20.320766 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:26:20.320730    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-76b8774589-59jxv" event={"ID":"e4d384f3-026e-4343-bada-ddc631f69d53","Type":"ContainerStarted","Data":"6c5574bb0664935855c0f0a214c98df1d5bac9b3e0b7ef3535037c4c45e7b583"}
Sep 12 00:26:20.343150 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:26:20.343103    2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/kube-auth-proxy-76b8774589-59jxv" podStartSLOduration=2.573169714 podStartE2EDuration="5.343089663s" totalImagesPullingTime="2.769919949s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-12 00:26:15 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-12 00:26:20.341948831 +0000 UTC m=+591.613589358" watchObservedRunningTime="2026-09-12 00:26:20.343089663 +0000 UTC m=+591.614730169"
Sep 12 00:26:26.558992 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:26:26.558955    2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-s4l9d"]
Sep 12 00:26:26.562291 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:26:26.562275    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/model-serving-api-54d89bd5fc-s4l9d"
Sep 12 00:26:26.564438 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:26:26.564411    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"model-serving-api-tls\""
Sep 12 00:26:26.564438 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:26:26.564426    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"model-serving-api-dockercfg-pdgrs\""
Sep 12 00:26:26.572718 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:26:26.572695    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-s4l9d"]
Sep 12 00:26:26.635518 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:26:26.635497    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-q95kx\" (UniqueName: \"kubernetes.io/projected/f4408e93-cbb1-4571-8b85-db15938ea0c0-kube-api-access-q95kx\") pod \"model-serving-api-54d89bd5fc-s4l9d\" (UID: \"f4408e93-cbb1-4571-8b85-db15938ea0c0\") " pod="opendatahub/model-serving-api-54d89bd5fc-s4l9d"
Sep 12 00:26:26.635647 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:26:26.635528    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f4408e93-cbb1-4571-8b85-db15938ea0c0-tls-certs\") pod \"model-serving-api-54d89bd5fc-s4l9d\" (UID: \"f4408e93-cbb1-4571-8b85-db15938ea0c0\") " pod="opendatahub/model-serving-api-54d89bd5fc-s4l9d"
Sep 12 00:26:26.736799 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:26:26.736776    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-q95kx\" (UniqueName: \"kubernetes.io/projected/f4408e93-cbb1-4571-8b85-db15938ea0c0-kube-api-access-q95kx\") pod \"model-serving-api-54d89bd5fc-s4l9d\" (UID: \"f4408e93-cbb1-4571-8b85-db15938ea0c0\") " pod="opendatahub/model-serving-api-54d89bd5fc-s4l9d"
Sep 12 00:26:26.736928 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:26:26.736815    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f4408e93-cbb1-4571-8b85-db15938ea0c0-tls-certs\") pod \"model-serving-api-54d89bd5fc-s4l9d\" (UID: \"f4408e93-cbb1-4571-8b85-db15938ea0c0\") " pod="opendatahub/model-serving-api-54d89bd5fc-s4l9d"
Sep 12 00:26:26.739095 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:26:26.739077    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f4408e93-cbb1-4571-8b85-db15938ea0c0-tls-certs\") pod \"model-serving-api-54d89bd5fc-s4l9d\" (UID: \"f4408e93-cbb1-4571-8b85-db15938ea0c0\") " pod="opendatahub/model-serving-api-54d89bd5fc-s4l9d"
Sep 12 00:26:26.744754 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:26:26.744733    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-q95kx\" (UniqueName: \"kubernetes.io/projected/f4408e93-cbb1-4571-8b85-db15938ea0c0-kube-api-access-q95kx\") pod \"model-serving-api-54d89bd5fc-s4l9d\" (UID: \"f4408e93-cbb1-4571-8b85-db15938ea0c0\") " pod="opendatahub/model-serving-api-54d89bd5fc-s4l9d"
Sep 12 00:26:26.872503 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:26:26.872450    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/model-serving-api-54d89bd5fc-s4l9d"
Sep 12 00:26:27.141386 ip-10-0-141-128 kubenswrapper[2571]: W0912 00:26:27.141360    2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf4408e93_cbb1_4571_8b85_db15938ea0c0.slice/crio-e1bde1275cb1f2a823d96f7e3a8f52639347c5d1345bc5078885f05c14c27966 WatchSource:0}: Error finding container e1bde1275cb1f2a823d96f7e3a8f52639347c5d1345bc5078885f05c14c27966: Status 404 returned error can't find the container with id e1bde1275cb1f2a823d96f7e3a8f52639347c5d1345bc5078885f05c14c27966
Sep 12 00:26:27.141691 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:26:27.141669    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-s4l9d"]
Sep 12 00:26:27.353944 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:26:27.353908    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/model-serving-api-54d89bd5fc-s4l9d" event={"ID":"f4408e93-cbb1-4571-8b85-db15938ea0c0","Type":"ContainerStarted","Data":"e1bde1275cb1f2a823d96f7e3a8f52639347c5d1345bc5078885f05c14c27966"}
Sep 12 00:26:29.363439 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:26:29.363368    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/model-serving-api-54d89bd5fc-s4l9d" event={"ID":"f4408e93-cbb1-4571-8b85-db15938ea0c0","Type":"ContainerStarted","Data":"965a98f03a99ec60e29cf54da4f66f3a58c3772f34363ec117888dd4f3aac1f8"}
Sep 12 00:26:29.363768 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:26:29.363481    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/model-serving-api-54d89bd5fc-s4l9d"
Sep 12 00:26:29.378849 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:26:29.378734    2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/model-serving-api-54d89bd5fc-s4l9d" podStartSLOduration=1.545175016 podStartE2EDuration="3.378718623s" totalImagesPullingTime="1.833543607s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-12 00:26:26 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-12 00:26:29.378039205 +0000 UTC m=+600.649679712" watchObservedRunningTime="2026-09-12 00:26:29.378718623 +0000 UTC m=+600.650359130"
Sep 12 00:26:32.414170 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:26:32.414138    2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/istiod-openshift-gateway-5677ccb49b-m9kwv"]
Sep 12 00:26:32.418994 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:26:32.418972    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-m9kwv"
Sep 12 00:26:32.421606 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:26:32.421562    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-tls\""
Sep 12 00:26:32.421606 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:26:32.421578    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-openshift-gateway-dockercfg-qt589\""
Sep 12 00:26:32.421777 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:26:32.421562    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"cacerts\""
Sep 12 00:26:32.421777 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:26:32.421727    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\""
Sep 12 00:26:32.421912 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:26:32.421737    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istio-kubeconfig\""
Sep 12 00:26:32.430640 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:26:32.430620    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-5677ccb49b-m9kwv"]
Sep 12 00:26:32.584730 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:26:32.584697    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/af206a61-acc5-4738-a83f-55c131da2017-local-certs\") pod \"istiod-openshift-gateway-5677ccb49b-m9kwv\" (UID: \"af206a61-acc5-4738-a83f-55c131da2017\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-m9kwv"
Sep 12 00:26:32.584884 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:26:32.584755    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/af206a61-acc5-4738-a83f-55c131da2017-istio-token\") pod \"istiod-openshift-gateway-5677ccb49b-m9kwv\" (UID: \"af206a61-acc5-4738-a83f-55c131da2017\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-m9kwv"
Sep 12 00:26:32.584884 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:26:32.584812    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/af206a61-acc5-4738-a83f-55c131da2017-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-5677ccb49b-m9kwv\" (UID: \"af206a61-acc5-4738-a83f-55c131da2017\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-m9kwv"
Sep 12 00:26:32.584884 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:26:32.584851    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/af206a61-acc5-4738-a83f-55c131da2017-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-5677ccb49b-m9kwv\" (UID: \"af206a61-acc5-4738-a83f-55c131da2017\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-m9kwv"
Sep 12 00:26:32.584884 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:26:32.584872    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/af206a61-acc5-4738-a83f-55c131da2017-cacerts\") pod \"istiod-openshift-gateway-5677ccb49b-m9kwv\" (UID: \"af206a61-acc5-4738-a83f-55c131da2017\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-m9kwv"
Sep 12 00:26:32.585057 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:26:32.584907    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/af206a61-acc5-4738-a83f-55c131da2017-istio-kubeconfig\") pod \"istiod-openshift-gateway-5677ccb49b-m9kwv\" (UID: \"af206a61-acc5-4738-a83f-55c131da2017\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-m9kwv"
Sep 12 00:26:32.585057 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:26:32.584948    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dwqzx\" (UniqueName: \"kubernetes.io/projected/af206a61-acc5-4738-a83f-55c131da2017-kube-api-access-dwqzx\") pod \"istiod-openshift-gateway-5677ccb49b-m9kwv\" (UID: \"af206a61-acc5-4738-a83f-55c131da2017\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-m9kwv"
Sep 12 00:26:32.686053 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:26:32.686026    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/af206a61-acc5-4738-a83f-55c131da2017-local-certs\") pod \"istiod-openshift-gateway-5677ccb49b-m9kwv\" (UID: \"af206a61-acc5-4738-a83f-55c131da2017\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-m9kwv"
Sep 12 00:26:32.686188 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:26:32.686081    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/af206a61-acc5-4738-a83f-55c131da2017-istio-token\") pod \"istiod-openshift-gateway-5677ccb49b-m9kwv\" (UID: \"af206a61-acc5-4738-a83f-55c131da2017\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-m9kwv"
Sep 12 00:26:32.686188 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:26:32.686104    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/af206a61-acc5-4738-a83f-55c131da2017-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-5677ccb49b-m9kwv\" (UID: \"af206a61-acc5-4738-a83f-55c131da2017\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-m9kwv"
Sep 12 00:26:32.686188 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:26:32.686121    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/af206a61-acc5-4738-a83f-55c131da2017-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-5677ccb49b-m9kwv\" (UID: \"af206a61-acc5-4738-a83f-55c131da2017\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-m9kwv"
Sep 12 00:26:32.686188 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:26:32.686154    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/af206a61-acc5-4738-a83f-55c131da2017-cacerts\") pod \"istiod-openshift-gateway-5677ccb49b-m9kwv\" (UID: \"af206a61-acc5-4738-a83f-55c131da2017\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-m9kwv"
Sep 12 00:26:32.686188 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:26:32.686180    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/af206a61-acc5-4738-a83f-55c131da2017-istio-kubeconfig\") pod \"istiod-openshift-gateway-5677ccb49b-m9kwv\" (UID: \"af206a61-acc5-4738-a83f-55c131da2017\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-m9kwv"
Sep 12 00:26:32.686381 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:26:32.686208    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dwqzx\" (UniqueName: \"kubernetes.io/projected/af206a61-acc5-4738-a83f-55c131da2017-kube-api-access-dwqzx\") pod \"istiod-openshift-gateway-5677ccb49b-m9kwv\" (UID: \"af206a61-acc5-4738-a83f-55c131da2017\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-m9kwv"
Sep 12 00:26:32.687069 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:26:32.687037    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/af206a61-acc5-4738-a83f-55c131da2017-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-5677ccb49b-m9kwv\" (UID: \"af206a61-acc5-4738-a83f-55c131da2017\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-m9kwv"
Sep 12 00:26:32.688425 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:26:32.688396    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/af206a61-acc5-4738-a83f-55c131da2017-local-certs\") pod \"istiod-openshift-gateway-5677ccb49b-m9kwv\" (UID: \"af206a61-acc5-4738-a83f-55c131da2017\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-m9kwv"
Sep 12 00:26:32.688551 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:26:32.688526    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/af206a61-acc5-4738-a83f-55c131da2017-istio-kubeconfig\") pod \"istiod-openshift-gateway-5677ccb49b-m9kwv\" (UID: \"af206a61-acc5-4738-a83f-55c131da2017\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-m9kwv"
Sep 12 00:26:32.688659 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:26:32.688637    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/af206a61-acc5-4738-a83f-55c131da2017-cacerts\") pod \"istiod-openshift-gateway-5677ccb49b-m9kwv\" (UID: \"af206a61-acc5-4738-a83f-55c131da2017\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-m9kwv"
Sep 12 00:26:32.688721 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:26:32.688669    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/af206a61-acc5-4738-a83f-55c131da2017-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-5677ccb49b-m9kwv\" (UID: \"af206a61-acc5-4738-a83f-55c131da2017\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-m9kwv"
Sep 12 00:26:32.693973 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:26:32.693947    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dwqzx\" (UniqueName: \"kubernetes.io/projected/af206a61-acc5-4738-a83f-55c131da2017-kube-api-access-dwqzx\") pod \"istiod-openshift-gateway-5677ccb49b-m9kwv\" (UID: \"af206a61-acc5-4738-a83f-55c131da2017\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-m9kwv"
Sep 12 00:26:32.694323 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:26:32.694306    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/af206a61-acc5-4738-a83f-55c131da2017-istio-token\") pod \"istiod-openshift-gateway-5677ccb49b-m9kwv\" (UID: \"af206a61-acc5-4738-a83f-55c131da2017\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-m9kwv"
Sep 12 00:26:32.729442 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:26:32.729416    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-m9kwv"
Sep 12 00:26:33.036048 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:26:33.035970    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-5677ccb49b-m9kwv"]
Sep 12 00:26:33.039258 ip-10-0-141-128 kubenswrapper[2571]: W0912 00:26:33.039226    2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podaf206a61_acc5_4738_a83f_55c131da2017.slice/crio-5a83caa2c1121736a6e5c995e1fa6f18c99c48b75313ba39cd688af60c2107f1 WatchSource:0}: Error finding container 5a83caa2c1121736a6e5c995e1fa6f18c99c48b75313ba39cd688af60c2107f1: Status 404 returned error can't find the container with id 5a83caa2c1121736a6e5c995e1fa6f18c99c48b75313ba39cd688af60c2107f1
Sep 12 00:26:33.386487 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:26:33.386402    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-m9kwv" event={"ID":"af206a61-acc5-4738-a83f-55c131da2017","Type":"ContainerStarted","Data":"5a83caa2c1121736a6e5c995e1fa6f18c99c48b75313ba39cd688af60c2107f1"}
Sep 12 00:26:35.019124 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:26:35.019090    2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/kserve-controller-manager-d6f4b65b8-tzqmv"]
Sep 12 00:26:35.027416 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:26:35.027370    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/kserve-controller-manager-d6f4b65b8-tzqmv"
Sep 12 00:26:35.030154 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:26:35.030128    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"kserve-webhook-server-cert\""
Sep 12 00:26:35.030291 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:26:35.030161    2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-4bzq6"]
Sep 12 00:26:35.030405 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:26:35.030390    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"kserve-controller-manager-dockercfg-n64zq\""
Sep 12 00:26:35.036802 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:26:35.036481    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/kserve-controller-manager-d6f4b65b8-tzqmv"]
Sep 12 00:26:35.036802 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:26:35.036614    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-4bzq6"
Sep 12 00:26:35.039454 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:26:35.039244    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"llmisvc-controller-manager-dockercfg-rfs9q\""
Sep 12 00:26:35.040411 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:26:35.040226    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"llmisvc-webhook-server-cert\""
Sep 12 00:26:35.051217 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:26:35.051137    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-4bzq6"]
Sep 12 00:26:35.209502 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:26:35.209461    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/9d018424-bcd8-40d8-980a-33214e4ff532-cert\") pod \"kserve-controller-manager-d6f4b65b8-tzqmv\" (UID: \"9d018424-bcd8-40d8-980a-33214e4ff532\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-tzqmv"
Sep 12 00:26:35.209702 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:26:35.209512    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gdlxq\" (UniqueName: \"kubernetes.io/projected/9d018424-bcd8-40d8-980a-33214e4ff532-kube-api-access-gdlxq\") pod \"kserve-controller-manager-d6f4b65b8-tzqmv\" (UID: \"9d018424-bcd8-40d8-980a-33214e4ff532\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-tzqmv"
Sep 12 00:26:35.209702 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:26:35.209574    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/b6f1b332-882c-4dbf-907e-8b201235ab1c-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-4bzq6\" (UID: \"b6f1b332-882c-4dbf-907e-8b201235ab1c\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-4bzq6"
Sep 12 00:26:35.209702 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:26:35.209633    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7rk8s\" (UniqueName: \"kubernetes.io/projected/b6f1b332-882c-4dbf-907e-8b201235ab1c-kube-api-access-7rk8s\") pod \"llmisvc-controller-manager-777c6bd7d5-4bzq6\" (UID: \"b6f1b332-882c-4dbf-907e-8b201235ab1c\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-4bzq6"
Sep 12 00:26:35.311043 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:26:35.310955    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/9d018424-bcd8-40d8-980a-33214e4ff532-cert\") pod \"kserve-controller-manager-d6f4b65b8-tzqmv\" (UID: \"9d018424-bcd8-40d8-980a-33214e4ff532\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-tzqmv"
Sep 12 00:26:35.311043 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:26:35.310998    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gdlxq\" (UniqueName: \"kubernetes.io/projected/9d018424-bcd8-40d8-980a-33214e4ff532-kube-api-access-gdlxq\") pod \"kserve-controller-manager-d6f4b65b8-tzqmv\" (UID: \"9d018424-bcd8-40d8-980a-33214e4ff532\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-tzqmv"
Sep 12 00:26:35.311043 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:26:35.311042    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/b6f1b332-882c-4dbf-907e-8b201235ab1c-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-4bzq6\" (UID: \"b6f1b332-882c-4dbf-907e-8b201235ab1c\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-4bzq6"
Sep 12 00:26:35.311326 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:26:35.311095    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7rk8s\" (UniqueName: \"kubernetes.io/projected/b6f1b332-882c-4dbf-907e-8b201235ab1c-kube-api-access-7rk8s\") pod \"llmisvc-controller-manager-777c6bd7d5-4bzq6\" (UID: \"b6f1b332-882c-4dbf-907e-8b201235ab1c\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-4bzq6"
Sep 12 00:26:35.311326 ip-10-0-141-128 kubenswrapper[2571]: E0912 00:26:35.311135    2571 secret.go:189] Couldn't get secret opendatahub/kserve-webhook-server-cert: secret "kserve-webhook-server-cert" not found
Sep 12 00:26:35.311326 ip-10-0-141-128 kubenswrapper[2571]: E0912 00:26:35.311213    2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9d018424-bcd8-40d8-980a-33214e4ff532-cert podName:9d018424-bcd8-40d8-980a-33214e4ff532 nodeName:}" failed. No retries permitted until 2026-09-12 00:26:35.811191199 +0000 UTC m=+607.082831699 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/9d018424-bcd8-40d8-980a-33214e4ff532-cert") pod "kserve-controller-manager-d6f4b65b8-tzqmv" (UID: "9d018424-bcd8-40d8-980a-33214e4ff532") : secret "kserve-webhook-server-cert" not found
Sep 12 00:26:35.311472 ip-10-0-141-128 kubenswrapper[2571]: E0912 00:26:35.311373    2571 secret.go:189] Couldn't get secret opendatahub/llmisvc-webhook-server-cert: secret "llmisvc-webhook-server-cert" not found
Sep 12 00:26:35.311472 ip-10-0-141-128 kubenswrapper[2571]: E0912 00:26:35.311420    2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b6f1b332-882c-4dbf-907e-8b201235ab1c-cert podName:b6f1b332-882c-4dbf-907e-8b201235ab1c nodeName:}" failed. No retries permitted until 2026-09-12 00:26:35.811403958 +0000 UTC m=+607.083044453 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/b6f1b332-882c-4dbf-907e-8b201235ab1c-cert") pod "llmisvc-controller-manager-777c6bd7d5-4bzq6" (UID: "b6f1b332-882c-4dbf-907e-8b201235ab1c") : secret "llmisvc-webhook-server-cert" not found
Sep 12 00:26:35.331880 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:26:35.331853    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7rk8s\" (UniqueName: \"kubernetes.io/projected/b6f1b332-882c-4dbf-907e-8b201235ab1c-kube-api-access-7rk8s\") pod \"llmisvc-controller-manager-777c6bd7d5-4bzq6\" (UID: \"b6f1b332-882c-4dbf-907e-8b201235ab1c\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-4bzq6"
Sep 12 00:26:35.333098 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:26:35.333074    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gdlxq\" (UniqueName: \"kubernetes.io/projected/9d018424-bcd8-40d8-980a-33214e4ff532-kube-api-access-gdlxq\") pod \"kserve-controller-manager-d6f4b65b8-tzqmv\" (UID: \"9d018424-bcd8-40d8-980a-33214e4ff532\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-tzqmv"
Sep 12 00:26:35.533560 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:26:35.533526    2571 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236196Ki","pods":"250"}
Sep 12 00:26:35.533653 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:26:35.533608    2571 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236196Ki","pods":"250"}
Sep 12 00:26:35.815751 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:26:35.815715    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/9d018424-bcd8-40d8-980a-33214e4ff532-cert\") pod \"kserve-controller-manager-d6f4b65b8-tzqmv\" (UID: \"9d018424-bcd8-40d8-980a-33214e4ff532\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-tzqmv"
Sep 12 00:26:35.815919 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:26:35.815759    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/b6f1b332-882c-4dbf-907e-8b201235ab1c-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-4bzq6\" (UID: \"b6f1b332-882c-4dbf-907e-8b201235ab1c\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-4bzq6"
Sep 12 00:26:35.818282 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:26:35.818260    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/b6f1b332-882c-4dbf-907e-8b201235ab1c-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-4bzq6\" (UID: \"b6f1b332-882c-4dbf-907e-8b201235ab1c\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-4bzq6"
Sep 12 00:26:35.818382 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:26:35.818301    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/9d018424-bcd8-40d8-980a-33214e4ff532-cert\") pod \"kserve-controller-manager-d6f4b65b8-tzqmv\" (UID: \"9d018424-bcd8-40d8-980a-33214e4ff532\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-tzqmv"
Sep 12 00:26:35.943247 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:26:35.943215    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/kserve-controller-manager-d6f4b65b8-tzqmv"
Sep 12 00:26:35.958442 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:26:35.958415    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-4bzq6"
Sep 12 00:26:36.223170 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:26:36.223069    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/kserve-controller-manager-d6f4b65b8-tzqmv"]
Sep 12 00:26:36.230210 ip-10-0-141-128 kubenswrapper[2571]: W0912 00:26:36.230162    2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9d018424_bcd8_40d8_980a_33214e4ff532.slice/crio-4ee6aeb2a166ecd076e9ff1c275a0cd0fb958556f04270fe9edc3b3255f0d3f7 WatchSource:0}: Error finding container 4ee6aeb2a166ecd076e9ff1c275a0cd0fb958556f04270fe9edc3b3255f0d3f7: Status 404 returned error can't find the container with id 4ee6aeb2a166ecd076e9ff1c275a0cd0fb958556f04270fe9edc3b3255f0d3f7
Sep 12 00:26:36.249653 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:26:36.249626    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-4bzq6"]
Sep 12 00:26:36.253272 ip-10-0-141-128 kubenswrapper[2571]: W0912 00:26:36.253243    2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-podb6f1b332_882c_4dbf_907e_8b201235ab1c.slice/crio-c238dc8d423bd246ec250ad1e6e37241dc59eb1f274416158fe2aa1ab5f03121 WatchSource:0}: Error finding container c238dc8d423bd246ec250ad1e6e37241dc59eb1f274416158fe2aa1ab5f03121: Status 404 returned error can't find the container with id c238dc8d423bd246ec250ad1e6e37241dc59eb1f274416158fe2aa1ab5f03121
Sep 12 00:26:36.402548 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:26:36.402513    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-4bzq6" event={"ID":"b6f1b332-882c-4dbf-907e-8b201235ab1c","Type":"ContainerStarted","Data":"c238dc8d423bd246ec250ad1e6e37241dc59eb1f274416158fe2aa1ab5f03121"}
Sep 12 00:26:36.403723 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:26:36.403694    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/kserve-controller-manager-d6f4b65b8-tzqmv" event={"ID":"9d018424-bcd8-40d8-980a-33214e4ff532","Type":"ContainerStarted","Data":"4ee6aeb2a166ecd076e9ff1c275a0cd0fb958556f04270fe9edc3b3255f0d3f7"}
Sep 12 00:26:36.405251 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:26:36.405226    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-m9kwv" event={"ID":"af206a61-acc5-4738-a83f-55c131da2017","Type":"ContainerStarted","Data":"b8ef1c296e9f3cafdb3edf4933052315320a01a084114ae965c11f295aea889d"}
Sep 12 00:26:36.405481 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:26:36.405465    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-m9kwv"
Sep 12 00:26:36.407019 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:26:36.406997    2571 patch_prober.go:28] interesting pod/istiod-openshift-gateway-5677ccb49b-m9kwv container/discovery namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Sep 12 00:26:36.407116 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:26:36.407042    2571 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-m9kwv" podUID="af206a61-acc5-4738-a83f-55c131da2017" containerName="discovery" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Sep 12 00:26:36.442005 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:26:36.441943    2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-m9kwv" podStartSLOduration=1.950240225 podStartE2EDuration="4.441926102s" totalImagesPullingTime="2.491685877s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-12 00:26:32 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-12 00:26:36.440615367 +0000 UTC m=+607.712255887" watchObservedRunningTime="2026-09-12 00:26:36.441926102 +0000 UTC m=+607.713566612"
Sep 12 00:26:37.411881 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:26:37.411850    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-m9kwv"
Sep 12 00:26:39.422902 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:26:39.422801    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/kserve-controller-manager-d6f4b65b8-tzqmv" event={"ID":"9d018424-bcd8-40d8-980a-33214e4ff532","Type":"ContainerStarted","Data":"92a2d8b376e747d07540872cd6393e324ef41ee02317e670116f254ebebfbcf1"}
Sep 12 00:26:39.423317 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:26:39.422933    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/kserve-controller-manager-d6f4b65b8-tzqmv"
Sep 12 00:26:39.424430 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:26:39.424404    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-4bzq6" event={"ID":"b6f1b332-882c-4dbf-907e-8b201235ab1c","Type":"ContainerStarted","Data":"d38e179f2e2d0c8004866389a0eeaf87b2fbf72c3d6861967ba59ba0f1002482"}
Sep 12 00:26:39.424569 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:26:39.424467    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-4bzq6"
Sep 12 00:26:39.438869 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:26:39.438816    2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/kserve-controller-manager-d6f4b65b8-tzqmv" podStartSLOduration=2.153251168 podStartE2EDuration="4.438799833s" totalImagesPullingTime="2.285548665s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-12 00:26:35 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-12 00:26:39.437689569 +0000 UTC m=+610.709330079" watchObservedRunningTime="2026-09-12 00:26:39.438799833 +0000 UTC m=+610.710440349"
Sep 12 00:26:39.453845 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:26:39.453790    2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-4bzq6" podStartSLOduration=1.599437499 podStartE2EDuration="4.453777033s" totalImagesPullingTime="2.854339534s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-12 00:26:35 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-12 00:26:39.452406995 +0000 UTC m=+610.724047505" watchObservedRunningTime="2026-09-12 00:26:39.453777033 +0000 UTC m=+610.725417541"
Sep 12 00:26:40.371617 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:26:40.371577    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/model-serving-api-54d89bd5fc-s4l9d"
Sep 12 00:27:10.431519 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:27:10.431439    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-4bzq6"
Sep 12 00:27:10.434662 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:27:10.434638    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/kserve-controller-manager-d6f4b65b8-tzqmv"
Sep 12 00:27:49.276849 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:27:49.276810    2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-operator-76b478bd78-vdsxb"]
Sep 12 00:27:49.280301 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:27:49.280283    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-operator-76b478bd78-vdsxb"
Sep 12 00:27:49.283107 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:27:49.283082    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\""
Sep 12 00:27:49.283228 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:27:49.283209    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-operator-dockercfg-7tdsj\""
Sep 12 00:27:49.283316 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:27:49.283209    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\""
Sep 12 00:27:49.291523 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:27:49.291488    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-operator-76b478bd78-vdsxb"]
Sep 12 00:27:49.329302 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:27:49.329272    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-t4dkx\" (UniqueName: \"kubernetes.io/projected/3f21ce58-966b-4f83-a1e2-55cbbe91871e-kube-api-access-t4dkx\") pod \"authorino-operator-76b478bd78-vdsxb\" (UID: \"3f21ce58-966b-4f83-a1e2-55cbbe91871e\") " pod="kuadrant-system/authorino-operator-76b478bd78-vdsxb"
Sep 12 00:27:49.430344 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:27:49.430319    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-t4dkx\" (UniqueName: \"kubernetes.io/projected/3f21ce58-966b-4f83-a1e2-55cbbe91871e-kube-api-access-t4dkx\") pod \"authorino-operator-76b478bd78-vdsxb\" (UID: \"3f21ce58-966b-4f83-a1e2-55cbbe91871e\") " pod="kuadrant-system/authorino-operator-76b478bd78-vdsxb"
Sep 12 00:27:49.448736 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:27:49.448713    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-t4dkx\" (UniqueName: \"kubernetes.io/projected/3f21ce58-966b-4f83-a1e2-55cbbe91871e-kube-api-access-t4dkx\") pod \"authorino-operator-76b478bd78-vdsxb\" (UID: \"3f21ce58-966b-4f83-a1e2-55cbbe91871e\") " pod="kuadrant-system/authorino-operator-76b478bd78-vdsxb"
Sep 12 00:27:49.591724 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:27:49.591657    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-operator-76b478bd78-vdsxb"
Sep 12 00:27:49.863475 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:27:49.863452    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-operator-76b478bd78-vdsxb"]
Sep 12 00:27:49.866296 ip-10-0-141-128 kubenswrapper[2571]: W0912 00:27:49.866259    2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3f21ce58_966b_4f83_a1e2_55cbbe91871e.slice/crio-07a0348957396971304f421d1525281a335dd06980bc04fcac3f62f1f0b3a4ac WatchSource:0}: Error finding container 07a0348957396971304f421d1525281a335dd06980bc04fcac3f62f1f0b3a4ac: Status 404 returned error can't find the container with id 07a0348957396971304f421d1525281a335dd06980bc04fcac3f62f1f0b3a4ac
Sep 12 00:27:49.868518 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:27:49.868500    2571 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 12 00:27:50.741209 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:27:50.741134    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-operator-76b478bd78-vdsxb" event={"ID":"3f21ce58-966b-4f83-a1e2-55cbbe91871e","Type":"ContainerStarted","Data":"07a0348957396971304f421d1525281a335dd06980bc04fcac3f62f1f0b3a4ac"}
Sep 12 00:27:52.754413 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:27:52.754374    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-operator-76b478bd78-vdsxb" event={"ID":"3f21ce58-966b-4f83-a1e2-55cbbe91871e","Type":"ContainerStarted","Data":"bf0ddc4fc65455a591172b115e5083b510d37e0dbd41bec564f4c46c4998a327"}
Sep 12 00:27:52.755000 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:27:52.754509    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/authorino-operator-76b478bd78-vdsxb"
Sep 12 00:27:52.776196 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:27:52.776131    2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-operator-76b478bd78-vdsxb" podStartSLOduration=1.428397775 podStartE2EDuration="3.776110678s" totalImagesPullingTime="2.347712903s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-12 00:27:49 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-12 00:27:52.773849347 +0000 UTC m=+684.045489856" watchObservedRunningTime="2026-09-12 00:27:52.776110678 +0000 UTC m=+684.047751189"
Sep 12 00:28:03.761846 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:28:03.761815    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/authorino-operator-76b478bd78-vdsxb"
Sep 12 00:28:04.927595 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:28:04.927563    2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5f685c496f-2rkbj"]
Sep 12 00:28:04.931288 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:28:04.931268    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-5f685c496f-2rkbj"
Sep 12 00:28:04.933930 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:28:04.933912    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-d8v2s\""
Sep 12 00:28:04.947397 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:28:04.947369    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5f685c496f-2rkbj"]
Sep 12 00:28:05.055951 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:28:05.055922    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/481c7eb7-febd-4dde-b983-8e4dcf84a3ae-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-5f685c496f-2rkbj\" (UID: \"481c7eb7-febd-4dde-b983-8e4dcf84a3ae\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5f685c496f-2rkbj"
Sep 12 00:28:05.056076 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:28:05.055978    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2gjl7\" (UniqueName: \"kubernetes.io/projected/481c7eb7-febd-4dde-b983-8e4dcf84a3ae-kube-api-access-2gjl7\") pod \"kuadrant-operator-controller-manager-5f685c496f-2rkbj\" (UID: \"481c7eb7-febd-4dde-b983-8e4dcf84a3ae\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5f685c496f-2rkbj"
Sep 12 00:28:05.156641 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:28:05.156605    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2gjl7\" (UniqueName: \"kubernetes.io/projected/481c7eb7-febd-4dde-b983-8e4dcf84a3ae-kube-api-access-2gjl7\") pod \"kuadrant-operator-controller-manager-5f685c496f-2rkbj\" (UID: \"481c7eb7-febd-4dde-b983-8e4dcf84a3ae\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5f685c496f-2rkbj"
Sep 12 00:28:05.156782 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:28:05.156690    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/481c7eb7-febd-4dde-b983-8e4dcf84a3ae-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-5f685c496f-2rkbj\" (UID: \"481c7eb7-febd-4dde-b983-8e4dcf84a3ae\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5f685c496f-2rkbj"
Sep 12 00:28:05.157069 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:28:05.157046    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/481c7eb7-febd-4dde-b983-8e4dcf84a3ae-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-5f685c496f-2rkbj\" (UID: \"481c7eb7-febd-4dde-b983-8e4dcf84a3ae\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5f685c496f-2rkbj"
Sep 12 00:28:05.164517 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:28:05.164493    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2gjl7\" (UniqueName: \"kubernetes.io/projected/481c7eb7-febd-4dde-b983-8e4dcf84a3ae-kube-api-access-2gjl7\") pod \"kuadrant-operator-controller-manager-5f685c496f-2rkbj\" (UID: \"481c7eb7-febd-4dde-b983-8e4dcf84a3ae\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5f685c496f-2rkbj"
Sep 12 00:28:05.241465 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:28:05.241426    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-5f685c496f-2rkbj"
Sep 12 00:28:05.517368 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:28:05.517280    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5f685c496f-2rkbj"]
Sep 12 00:28:05.520113 ip-10-0-141-128 kubenswrapper[2571]: W0912 00:28:05.520084    2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod481c7eb7_febd_4dde_b983_8e4dcf84a3ae.slice/crio-74d33e6ea64843f705484a9e7df38f88c4b8a9d7df537c5b8a5c9b178fe61dde WatchSource:0}: Error finding container 74d33e6ea64843f705484a9e7df38f88c4b8a9d7df537c5b8a5c9b178fe61dde: Status 404 returned error can't find the container with id 74d33e6ea64843f705484a9e7df38f88c4b8a9d7df537c5b8a5c9b178fe61dde
Sep 12 00:28:05.695068 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:28:05.695038    2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5f685c496f-2rkbj"]
Sep 12 00:28:05.706548 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:28:05.706514    2571 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5f685c496f-2rkbj"]
Sep 12 00:28:05.724733 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:28:05.724700    2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5f685c496f-85jmk"]
Sep 12 00:28:05.729517 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:28:05.729496    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-5f685c496f-85jmk"
Sep 12 00:28:05.758185 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:28:05.758158    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5f685c496f-85jmk"]
Sep 12 00:28:05.864000 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:28:05.863918    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/c7c52007-596b-4733-ac34-d5cd40897fcd-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-5f685c496f-85jmk\" (UID: \"c7c52007-596b-4733-ac34-d5cd40897fcd\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5f685c496f-85jmk"
Sep 12 00:28:05.864000 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:28:05.863954    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kd6kb\" (UniqueName: \"kubernetes.io/projected/c7c52007-596b-4733-ac34-d5cd40897fcd-kube-api-access-kd6kb\") pod \"kuadrant-operator-controller-manager-5f685c496f-85jmk\" (UID: \"c7c52007-596b-4733-ac34-d5cd40897fcd\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5f685c496f-85jmk"
Sep 12 00:28:05.964517 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:28:05.964490    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/c7c52007-596b-4733-ac34-d5cd40897fcd-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-5f685c496f-85jmk\" (UID: \"c7c52007-596b-4733-ac34-d5cd40897fcd\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5f685c496f-85jmk"
Sep 12 00:28:05.964895 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:28:05.964523    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kd6kb\" (UniqueName: \"kubernetes.io/projected/c7c52007-596b-4733-ac34-d5cd40897fcd-kube-api-access-kd6kb\") pod \"kuadrant-operator-controller-manager-5f685c496f-85jmk\" (UID: \"c7c52007-596b-4733-ac34-d5cd40897fcd\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5f685c496f-85jmk"
Sep 12 00:28:05.964944 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:28:05.964911    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/c7c52007-596b-4733-ac34-d5cd40897fcd-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-5f685c496f-85jmk\" (UID: \"c7c52007-596b-4733-ac34-d5cd40897fcd\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5f685c496f-85jmk"
Sep 12 00:28:05.987267 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:28:05.987244    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kd6kb\" (UniqueName: \"kubernetes.io/projected/c7c52007-596b-4733-ac34-d5cd40897fcd-kube-api-access-kd6kb\") pod \"kuadrant-operator-controller-manager-5f685c496f-85jmk\" (UID: \"c7c52007-596b-4733-ac34-d5cd40897fcd\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5f685c496f-85jmk"
Sep 12 00:28:06.042013 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:28:06.041981    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-5f685c496f-85jmk"
Sep 12 00:28:06.317820 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:28:06.317794    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5f685c496f-85jmk"]
Sep 12 00:28:06.318897 ip-10-0-141-128 kubenswrapper[2571]: W0912 00:28:06.318869    2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc7c52007_596b_4733_ac34_d5cd40897fcd.slice/crio-91d9ca49b10e5ac4516655f1328c8a15978efe0882e4204e433bb075335165c5 WatchSource:0}: Error finding container 91d9ca49b10e5ac4516655f1328c8a15978efe0882e4204e433bb075335165c5: Status 404 returned error can't find the container with id 91d9ca49b10e5ac4516655f1328c8a15978efe0882e4204e433bb075335165c5
Sep 12 00:28:06.823651 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:28:06.823613    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-5f685c496f-85jmk" event={"ID":"c7c52007-596b-4733-ac34-d5cd40897fcd","Type":"ContainerStarted","Data":"91d9ca49b10e5ac4516655f1328c8a15978efe0882e4204e433bb075335165c5"}
Sep 12 00:28:10.846390 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:28:10.846349    2571 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-5f685c496f-2rkbj" podUID="481c7eb7-febd-4dde-b983-8e4dcf84a3ae" containerName="manager" containerID="cri-o://882b4072085a398bf68921298d19b88599b6ed9ce537a3855b2be181f819d0d4" gracePeriod=2
Sep 12 00:28:10.848097 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:28:10.848064    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-5f685c496f-85jmk" event={"ID":"c7c52007-596b-4733-ac34-d5cd40897fcd","Type":"ContainerStarted","Data":"43460b0b23153f79555103b09db87ac1bb1abc57b8a991b54ff10926f4a276c5"}
Sep 12 00:28:10.848241 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:28:10.848162    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-5f685c496f-85jmk"
Sep 12 00:28:10.848438 ip-10-0-141-128 kubenswrapper[2571]: E0912 00:28:10.848412    2571 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-5f685c496f-2rkbj\" is forbidden: User \"system:node:ip-10-0-141-128.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-141-128.ec2.internal' and this object" podUID="481c7eb7-febd-4dde-b983-8e4dcf84a3ae" pod="kuadrant-system/kuadrant-operator-controller-manager-5f685c496f-2rkbj"
Sep 12 00:28:10.850100 ip-10-0-141-128 kubenswrapper[2571]: E0912 00:28:10.850075    2571 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-5f685c496f-2rkbj\" is forbidden: User \"system:node:ip-10-0-141-128.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-141-128.ec2.internal' and this object" podUID="481c7eb7-febd-4dde-b983-8e4dcf84a3ae" pod="kuadrant-system/kuadrant-operator-controller-manager-5f685c496f-2rkbj"
Sep 12 00:28:10.872971 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:28:10.872925    2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-5f685c496f-85jmk" podStartSLOduration=2.309362332 podStartE2EDuration="5.872909413s" totalImagesPullingTime="3.563547081s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-12 00:28:05 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-12 00:28:10.870395613 +0000 UTC m=+702.142036124" watchObservedRunningTime="2026-09-12 00:28:10.872909413 +0000 UTC m=+702.144549925"
Sep 12 00:28:11.086246 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:28:11.086224    2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-5f685c496f-2rkbj"
Sep 12 00:28:11.088381 ip-10-0-141-128 kubenswrapper[2571]: E0912 00:28:11.088348    2571 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-5f685c496f-2rkbj\" is forbidden: User \"system:node:ip-10-0-141-128.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-141-128.ec2.internal' and this object" podUID="481c7eb7-febd-4dde-b983-8e4dcf84a3ae" pod="kuadrant-system/kuadrant-operator-controller-manager-5f685c496f-2rkbj"
Sep 12 00:28:11.216300 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:28:11.216274    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-2gjl7\" (UniqueName: \"kubernetes.io/projected/481c7eb7-febd-4dde-b983-8e4dcf84a3ae-kube-api-access-2gjl7\") pod \"481c7eb7-febd-4dde-b983-8e4dcf84a3ae\" (UID: \"481c7eb7-febd-4dde-b983-8e4dcf84a3ae\") "
Sep 12 00:28:11.216408 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:28:11.216370    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/481c7eb7-febd-4dde-b983-8e4dcf84a3ae-extensions-socket-volume\") pod \"481c7eb7-febd-4dde-b983-8e4dcf84a3ae\" (UID: \"481c7eb7-febd-4dde-b983-8e4dcf84a3ae\") "
Sep 12 00:28:11.216654 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:28:11.216634    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/481c7eb7-febd-4dde-b983-8e4dcf84a3ae-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "481c7eb7-febd-4dde-b983-8e4dcf84a3ae" (UID: "481c7eb7-febd-4dde-b983-8e4dcf84a3ae"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 12 00:28:11.218364 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:28:11.218336    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/481c7eb7-febd-4dde-b983-8e4dcf84a3ae-kube-api-access-2gjl7" (OuterVolumeSpecName: "kube-api-access-2gjl7") pod "481c7eb7-febd-4dde-b983-8e4dcf84a3ae" (UID: "481c7eb7-febd-4dde-b983-8e4dcf84a3ae"). InnerVolumeSpecName "kube-api-access-2gjl7". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 12 00:28:11.317369 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:28:11.317338    2571 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/481c7eb7-febd-4dde-b983-8e4dcf84a3ae-extensions-socket-volume\") on node \"ip-10-0-141-128.ec2.internal\" DevicePath \"\""
Sep 12 00:28:11.317369 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:28:11.317363    2571 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2gjl7\" (UniqueName: \"kubernetes.io/projected/481c7eb7-febd-4dde-b983-8e4dcf84a3ae-kube-api-access-2gjl7\") on node \"ip-10-0-141-128.ec2.internal\" DevicePath \"\""
Sep 12 00:28:11.854028 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:28:11.853994    2571 generic.go:356] "Generic (PLEG): container finished" podID="481c7eb7-febd-4dde-b983-8e4dcf84a3ae" containerID="882b4072085a398bf68921298d19b88599b6ed9ce537a3855b2be181f819d0d4" exitCode=2
Sep 12 00:28:11.854496 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:28:11.854034    2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-5f685c496f-2rkbj"
Sep 12 00:28:11.854496 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:28:11.854105    2571 scope.go:117] "RemoveContainer" containerID="882b4072085a398bf68921298d19b88599b6ed9ce537a3855b2be181f819d0d4"
Sep 12 00:28:11.856365 ip-10-0-141-128 kubenswrapper[2571]: E0912 00:28:11.856335    2571 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-5f685c496f-2rkbj\" is forbidden: User \"system:node:ip-10-0-141-128.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-141-128.ec2.internal' and this object" podUID="481c7eb7-febd-4dde-b983-8e4dcf84a3ae" pod="kuadrant-system/kuadrant-operator-controller-manager-5f685c496f-2rkbj"
Sep 12 00:28:11.862468 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:28:11.862450    2571 scope.go:117] "RemoveContainer" containerID="882b4072085a398bf68921298d19b88599b6ed9ce537a3855b2be181f819d0d4"
Sep 12 00:28:11.862761 ip-10-0-141-128 kubenswrapper[2571]: E0912 00:28:11.862741    2571 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"882b4072085a398bf68921298d19b88599b6ed9ce537a3855b2be181f819d0d4\": container with ID starting with 882b4072085a398bf68921298d19b88599b6ed9ce537a3855b2be181f819d0d4 not found: ID does not exist" containerID="882b4072085a398bf68921298d19b88599b6ed9ce537a3855b2be181f819d0d4"
Sep 12 00:28:11.862829 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:28:11.862769    2571 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"882b4072085a398bf68921298d19b88599b6ed9ce537a3855b2be181f819d0d4"} err="failed to get container status \"882b4072085a398bf68921298d19b88599b6ed9ce537a3855b2be181f819d0d4\": rpc error: code = NotFound desc = could not find container \"882b4072085a398bf68921298d19b88599b6ed9ce537a3855b2be181f819d0d4\": container with ID starting with 882b4072085a398bf68921298d19b88599b6ed9ce537a3855b2be181f819d0d4 not found: ID does not exist"
Sep 12 00:28:11.863802 ip-10-0-141-128 kubenswrapper[2571]: E0912 00:28:11.863783    2571 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-5f685c496f-2rkbj\" is forbidden: User \"system:node:ip-10-0-141-128.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-141-128.ec2.internal' and this object" podUID="481c7eb7-febd-4dde-b983-8e4dcf84a3ae" pod="kuadrant-system/kuadrant-operator-controller-manager-5f685c496f-2rkbj"
Sep 12 00:28:13.140523 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:28:13.140490    2571 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="481c7eb7-febd-4dde-b983-8e4dcf84a3ae" path="/var/lib/kubelet/pods/481c7eb7-febd-4dde-b983-8e4dcf84a3ae/volumes"
Sep 12 00:28:21.857099 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:28:21.857070    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-5f685c496f-85jmk"
Sep 12 00:28:34.467358 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:28:34.467317    2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5f685c496f-85jmk"]
Sep 12 00:28:34.467740 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:28:34.467605    2571 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-5f685c496f-85jmk" podUID="c7c52007-596b-4733-ac34-d5cd40897fcd" containerName="manager" containerID="cri-o://43460b0b23153f79555103b09db87ac1bb1abc57b8a991b54ff10926f4a276c5" gracePeriod=10
Sep 12 00:28:34.708413 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:28:34.708391    2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-5f685c496f-85jmk"
Sep 12 00:28:34.806722 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:28:34.806651    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/c7c52007-596b-4733-ac34-d5cd40897fcd-extensions-socket-volume\") pod \"c7c52007-596b-4733-ac34-d5cd40897fcd\" (UID: \"c7c52007-596b-4733-ac34-d5cd40897fcd\") "
Sep 12 00:28:34.806722 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:28:34.806694    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-kd6kb\" (UniqueName: \"kubernetes.io/projected/c7c52007-596b-4733-ac34-d5cd40897fcd-kube-api-access-kd6kb\") pod \"c7c52007-596b-4733-ac34-d5cd40897fcd\" (UID: \"c7c52007-596b-4733-ac34-d5cd40897fcd\") "
Sep 12 00:28:34.806983 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:28:34.806960    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c7c52007-596b-4733-ac34-d5cd40897fcd-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "c7c52007-596b-4733-ac34-d5cd40897fcd" (UID: "c7c52007-596b-4733-ac34-d5cd40897fcd"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 12 00:28:34.808668 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:28:34.808647    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c7c52007-596b-4733-ac34-d5cd40897fcd-kube-api-access-kd6kb" (OuterVolumeSpecName: "kube-api-access-kd6kb") pod "c7c52007-596b-4733-ac34-d5cd40897fcd" (UID: "c7c52007-596b-4733-ac34-d5cd40897fcd"). InnerVolumeSpecName "kube-api-access-kd6kb". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 12 00:28:34.907495 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:28:34.907468    2571 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/c7c52007-596b-4733-ac34-d5cd40897fcd-extensions-socket-volume\") on node \"ip-10-0-141-128.ec2.internal\" DevicePath \"\""
Sep 12 00:28:34.907495 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:28:34.907492    2571 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-kd6kb\" (UniqueName: \"kubernetes.io/projected/c7c52007-596b-4733-ac34-d5cd40897fcd-kube-api-access-kd6kb\") on node \"ip-10-0-141-128.ec2.internal\" DevicePath \"\""
Sep 12 00:28:34.953370 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:28:34.953345    2571 generic.go:356] "Generic (PLEG): container finished" podID="c7c52007-596b-4733-ac34-d5cd40897fcd" containerID="43460b0b23153f79555103b09db87ac1bb1abc57b8a991b54ff10926f4a276c5" exitCode=0
Sep 12 00:28:34.953466 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:28:34.953378    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-5f685c496f-85jmk" event={"ID":"c7c52007-596b-4733-ac34-d5cd40897fcd","Type":"ContainerDied","Data":"43460b0b23153f79555103b09db87ac1bb1abc57b8a991b54ff10926f4a276c5"}
Sep 12 00:28:34.953466 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:28:34.953388    2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-5f685c496f-85jmk"
Sep 12 00:28:34.953466 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:28:34.953430    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-5f685c496f-85jmk" event={"ID":"c7c52007-596b-4733-ac34-d5cd40897fcd","Type":"ContainerDied","Data":"91d9ca49b10e5ac4516655f1328c8a15978efe0882e4204e433bb075335165c5"}
Sep 12 00:28:34.953466 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:28:34.953459    2571 scope.go:117] "RemoveContainer" containerID="43460b0b23153f79555103b09db87ac1bb1abc57b8a991b54ff10926f4a276c5"
Sep 12 00:28:34.961816 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:28:34.961798    2571 scope.go:117] "RemoveContainer" containerID="43460b0b23153f79555103b09db87ac1bb1abc57b8a991b54ff10926f4a276c5"
Sep 12 00:28:34.962071 ip-10-0-141-128 kubenswrapper[2571]: E0912 00:28:34.962050    2571 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"43460b0b23153f79555103b09db87ac1bb1abc57b8a991b54ff10926f4a276c5\": container with ID starting with 43460b0b23153f79555103b09db87ac1bb1abc57b8a991b54ff10926f4a276c5 not found: ID does not exist" containerID="43460b0b23153f79555103b09db87ac1bb1abc57b8a991b54ff10926f4a276c5"
Sep 12 00:28:34.962119 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:28:34.962078    2571 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"43460b0b23153f79555103b09db87ac1bb1abc57b8a991b54ff10926f4a276c5"} err="failed to get container status \"43460b0b23153f79555103b09db87ac1bb1abc57b8a991b54ff10926f4a276c5\": rpc error: code = NotFound desc = could not find container \"43460b0b23153f79555103b09db87ac1bb1abc57b8a991b54ff10926f4a276c5\": container with ID starting with 43460b0b23153f79555103b09db87ac1bb1abc57b8a991b54ff10926f4a276c5 not found: ID does not exist"
Sep 12 00:28:34.973967 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:28:34.973945    2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5f685c496f-85jmk"]
Sep 12 00:28:34.978375 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:28:34.978356    2571 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5f685c496f-85jmk"]
Sep 12 00:28:35.140730 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:28:35.140667    2571 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c7c52007-596b-4733-ac34-d5cd40897fcd" path="/var/lib/kubelet/pods/c7c52007-596b-4733-ac34-d5cd40897fcd/volumes"
Sep 12 00:28:38.270842 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:28:38.270807    2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-svr8q"]
Sep 12 00:28:38.271220 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:28:38.271151    2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="481c7eb7-febd-4dde-b983-8e4dcf84a3ae" containerName="manager"
Sep 12 00:28:38.271220 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:28:38.271162    2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="481c7eb7-febd-4dde-b983-8e4dcf84a3ae" containerName="manager"
Sep 12 00:28:38.271220 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:28:38.271169    2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c7c52007-596b-4733-ac34-d5cd40897fcd" containerName="manager"
Sep 12 00:28:38.271220 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:28:38.271174    2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="c7c52007-596b-4733-ac34-d5cd40897fcd" containerName="manager"
Sep 12 00:28:38.271356 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:28:38.271226    2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="481c7eb7-febd-4dde-b983-8e4dcf84a3ae" containerName="manager"
Sep 12 00:28:38.271356 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:28:38.271236    2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="c7c52007-596b-4733-ac34-d5cd40897fcd" containerName="manager"
Sep 12 00:28:38.275108 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:28:38.275089    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-svr8q"
Sep 12 00:28:38.277987 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:28:38.277954    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"maas-default-gateway-openshift-default-dockercfg-z6jml\""
Sep 12 00:28:38.285747 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:28:38.285721    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-svr8q"]
Sep 12 00:28:38.332488 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:28:38.332454    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/06461169-3e71-40a2-a0dc-e52b89db9f42-istio-token\") pod \"maas-default-gateway-openshift-default-78bb78ff78-svr8q\" (UID: \"06461169-3e71-40a2-a0dc-e52b89db9f42\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-svr8q"
Sep 12 00:28:38.332488 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:28:38.332491    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/06461169-3e71-40a2-a0dc-e52b89db9f42-credential-socket\") pod \"maas-default-gateway-openshift-default-78bb78ff78-svr8q\" (UID: \"06461169-3e71-40a2-a0dc-e52b89db9f42\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-svr8q"
Sep 12 00:28:38.332718 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:28:38.332521    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/06461169-3e71-40a2-a0dc-e52b89db9f42-workload-socket\") pod \"maas-default-gateway-openshift-default-78bb78ff78-svr8q\" (UID: \"06461169-3e71-40a2-a0dc-e52b89db9f42\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-svr8q"
Sep 12 00:28:38.332718 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:28:38.332614    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/06461169-3e71-40a2-a0dc-e52b89db9f42-istio-data\") pod \"maas-default-gateway-openshift-default-78bb78ff78-svr8q\" (UID: \"06461169-3e71-40a2-a0dc-e52b89db9f42\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-svr8q"
Sep 12 00:28:38.332718 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:28:38.332655    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/06461169-3e71-40a2-a0dc-e52b89db9f42-workload-certs\") pod \"maas-default-gateway-openshift-default-78bb78ff78-svr8q\" (UID: \"06461169-3e71-40a2-a0dc-e52b89db9f42\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-svr8q"
Sep 12 00:28:38.332718 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:28:38.332700    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/06461169-3e71-40a2-a0dc-e52b89db9f42-istio-envoy\") pod \"maas-default-gateway-openshift-default-78bb78ff78-svr8q\" (UID: \"06461169-3e71-40a2-a0dc-e52b89db9f42\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-svr8q"
Sep 12 00:28:38.332718 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:28:38.332715    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/06461169-3e71-40a2-a0dc-e52b89db9f42-istio-podinfo\") pod \"maas-default-gateway-openshift-default-78bb78ff78-svr8q\" (UID: \"06461169-3e71-40a2-a0dc-e52b89db9f42\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-svr8q"
Sep 12 00:28:38.333046 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:28:38.332744    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xhhg6\" (UniqueName: \"kubernetes.io/projected/06461169-3e71-40a2-a0dc-e52b89db9f42-kube-api-access-xhhg6\") pod \"maas-default-gateway-openshift-default-78bb78ff78-svr8q\" (UID: \"06461169-3e71-40a2-a0dc-e52b89db9f42\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-svr8q"
Sep 12 00:28:38.333046 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:28:38.332761    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/06461169-3e71-40a2-a0dc-e52b89db9f42-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-78bb78ff78-svr8q\" (UID: \"06461169-3e71-40a2-a0dc-e52b89db9f42\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-svr8q"
Sep 12 00:28:38.433337 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:28:38.433304    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/06461169-3e71-40a2-a0dc-e52b89db9f42-istio-data\") pod \"maas-default-gateway-openshift-default-78bb78ff78-svr8q\" (UID: \"06461169-3e71-40a2-a0dc-e52b89db9f42\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-svr8q"
Sep 12 00:28:38.433487 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:28:38.433343    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/06461169-3e71-40a2-a0dc-e52b89db9f42-workload-certs\") pod \"maas-default-gateway-openshift-default-78bb78ff78-svr8q\" (UID: \"06461169-3e71-40a2-a0dc-e52b89db9f42\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-svr8q"
Sep 12 00:28:38.433487 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:28:38.433364    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/06461169-3e71-40a2-a0dc-e52b89db9f42-istio-envoy\") pod \"maas-default-gateway-openshift-default-78bb78ff78-svr8q\" (UID: \"06461169-3e71-40a2-a0dc-e52b89db9f42\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-svr8q"
Sep 12 00:28:38.433487 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:28:38.433380    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/06461169-3e71-40a2-a0dc-e52b89db9f42-istio-podinfo\") pod \"maas-default-gateway-openshift-default-78bb78ff78-svr8q\" (UID: \"06461169-3e71-40a2-a0dc-e52b89db9f42\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-svr8q"
Sep 12 00:28:38.433487 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:28:38.433406    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xhhg6\" (UniqueName: \"kubernetes.io/projected/06461169-3e71-40a2-a0dc-e52b89db9f42-kube-api-access-xhhg6\") pod \"maas-default-gateway-openshift-default-78bb78ff78-svr8q\" (UID: \"06461169-3e71-40a2-a0dc-e52b89db9f42\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-svr8q"
Sep 12 00:28:38.433487 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:28:38.433422    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/06461169-3e71-40a2-a0dc-e52b89db9f42-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-78bb78ff78-svr8q\" (UID: \"06461169-3e71-40a2-a0dc-e52b89db9f42\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-svr8q"
Sep 12 00:28:38.433487 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:28:38.433456    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/06461169-3e71-40a2-a0dc-e52b89db9f42-istio-token\") pod \"maas-default-gateway-openshift-default-78bb78ff78-svr8q\" (UID: \"06461169-3e71-40a2-a0dc-e52b89db9f42\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-svr8q"
Sep 12 00:28:38.433487 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:28:38.433472    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/06461169-3e71-40a2-a0dc-e52b89db9f42-credential-socket\") pod \"maas-default-gateway-openshift-default-78bb78ff78-svr8q\" (UID: \"06461169-3e71-40a2-a0dc-e52b89db9f42\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-svr8q"
Sep 12 00:28:38.433901 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:28:38.433496    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/06461169-3e71-40a2-a0dc-e52b89db9f42-workload-socket\") pod \"maas-default-gateway-openshift-default-78bb78ff78-svr8q\" (UID: \"06461169-3e71-40a2-a0dc-e52b89db9f42\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-svr8q"
Sep 12 00:28:38.433901 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:28:38.433767    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/06461169-3e71-40a2-a0dc-e52b89db9f42-istio-data\") pod \"maas-default-gateway-openshift-default-78bb78ff78-svr8q\" (UID: \"06461169-3e71-40a2-a0dc-e52b89db9f42\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-svr8q"
Sep 12 00:28:38.434120 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:28:38.434095    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/06461169-3e71-40a2-a0dc-e52b89db9f42-workload-socket\") pod \"maas-default-gateway-openshift-default-78bb78ff78-svr8q\" (UID: \"06461169-3e71-40a2-a0dc-e52b89db9f42\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-svr8q"
Sep 12 00:28:38.434259 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:28:38.434237    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/06461169-3e71-40a2-a0dc-e52b89db9f42-credential-socket\") pod \"maas-default-gateway-openshift-default-78bb78ff78-svr8q\" (UID: \"06461169-3e71-40a2-a0dc-e52b89db9f42\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-svr8q"
Sep 12 00:28:38.434404 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:28:38.434378    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/06461169-3e71-40a2-a0dc-e52b89db9f42-workload-certs\") pod \"maas-default-gateway-openshift-default-78bb78ff78-svr8q\" (UID: \"06461169-3e71-40a2-a0dc-e52b89db9f42\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-svr8q"
Sep 12 00:28:38.434527 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:28:38.434453    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/06461169-3e71-40a2-a0dc-e52b89db9f42-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-78bb78ff78-svr8q\" (UID: \"06461169-3e71-40a2-a0dc-e52b89db9f42\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-svr8q"
Sep 12 00:28:38.435891 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:28:38.435871    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/06461169-3e71-40a2-a0dc-e52b89db9f42-istio-envoy\") pod \"maas-default-gateway-openshift-default-78bb78ff78-svr8q\" (UID: \"06461169-3e71-40a2-a0dc-e52b89db9f42\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-svr8q"
Sep 12 00:28:38.436119 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:28:38.436101    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/06461169-3e71-40a2-a0dc-e52b89db9f42-istio-podinfo\") pod \"maas-default-gateway-openshift-default-78bb78ff78-svr8q\" (UID: \"06461169-3e71-40a2-a0dc-e52b89db9f42\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-svr8q"
Sep 12 00:28:38.441002 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:28:38.440974    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/06461169-3e71-40a2-a0dc-e52b89db9f42-istio-token\") pod \"maas-default-gateway-openshift-default-78bb78ff78-svr8q\" (UID: \"06461169-3e71-40a2-a0dc-e52b89db9f42\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-svr8q"
Sep 12 00:28:38.441269 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:28:38.441250    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xhhg6\" (UniqueName: \"kubernetes.io/projected/06461169-3e71-40a2-a0dc-e52b89db9f42-kube-api-access-xhhg6\") pod \"maas-default-gateway-openshift-default-78bb78ff78-svr8q\" (UID: \"06461169-3e71-40a2-a0dc-e52b89db9f42\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-svr8q"
Sep 12 00:28:38.587560 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:28:38.587487    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-svr8q"
Sep 12 00:28:38.859986 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:28:38.859961    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-svr8q"]
Sep 12 00:28:38.862002 ip-10-0-141-128 kubenswrapper[2571]: W0912 00:28:38.861973    2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod06461169_3e71_40a2_a0dc_e52b89db9f42.slice/crio-49b9bac4d431111f081f8e78d04d49ec686e91dafb0857ff6767d2d70c5c9cd6 WatchSource:0}: Error finding container 49b9bac4d431111f081f8e78d04d49ec686e91dafb0857ff6767d2d70c5c9cd6: Status 404 returned error can't find the container with id 49b9bac4d431111f081f8e78d04d49ec686e91dafb0857ff6767d2d70c5c9cd6
Sep 12 00:28:38.972710 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:28:38.972683    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-svr8q" event={"ID":"06461169-3e71-40a2-a0dc-e52b89db9f42","Type":"ContainerStarted","Data":"49b9bac4d431111f081f8e78d04d49ec686e91dafb0857ff6767d2d70c5c9cd6"}
Sep 12 00:28:41.725295 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:28:41.725261    2571 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236196Ki","pods":"250"}
Sep 12 00:28:41.725564 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:28:41.725340    2571 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236196Ki","pods":"250"}
Sep 12 00:28:41.725564 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:28:41.725366    2571 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236196Ki","pods":"250"}
Sep 12 00:28:41.990133 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:28:41.990046    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-svr8q" event={"ID":"06461169-3e71-40a2-a0dc-e52b89db9f42","Type":"ContainerStarted","Data":"731ce6496d2e4a0d3fd0856b02f10a59d93bde66cc255b792ed83ce1aa0218ec"}
Sep 12 00:28:42.009722 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:28:42.009676    2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-svr8q" podStartSLOduration=1.148633823 podStartE2EDuration="4.009662209s" totalImagesPullingTime="2.861028386s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-12 00:28:38 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-12 00:28:42.008269656 +0000 UTC m=+733.279910166" watchObservedRunningTime="2026-09-12 00:28:42.009662209 +0000 UTC m=+733.281302718"
Sep 12 00:28:42.587875 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:28:42.587838    2571 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-svr8q"
Sep 12 00:28:42.592448 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:28:42.592424    2571 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-svr8q"
Sep 12 00:28:42.995905 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:28:42.995879    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-svr8q"
Sep 12 00:28:42.996800 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:28:42.996783    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-svr8q"
Sep 12 00:28:56.350634 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:28:56.350600    2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-limitador-8588749977-d6qll"]
Sep 12 00:28:56.353852 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:28:56.353836    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-8588749977-d6qll"
Sep 12 00:28:56.356128 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:28:56.356103    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"limitador-limits-config-limitador\""
Sep 12 00:28:56.356254 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:28:56.356182    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-kshlz\""
Sep 12 00:28:56.360803 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:28:56.360782    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-8588749977-d6qll"]
Sep 12 00:28:56.448473 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:28:56.448440    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-8588749977-d6qll"]
Sep 12 00:28:56.490868 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:28:56.490838    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/cc27ab80-b6a6-42e4-b46b-444815f35e60-config-file\") pod \"limitador-limitador-8588749977-d6qll\" (UID: \"cc27ab80-b6a6-42e4-b46b-444815f35e60\") " pod="kuadrant-system/limitador-limitador-8588749977-d6qll"
Sep 12 00:28:56.490984 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:28:56.490944    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-stklr\" (UniqueName: \"kubernetes.io/projected/cc27ab80-b6a6-42e4-b46b-444815f35e60-kube-api-access-stklr\") pod \"limitador-limitador-8588749977-d6qll\" (UID: \"cc27ab80-b6a6-42e4-b46b-444815f35e60\") " pod="kuadrant-system/limitador-limitador-8588749977-d6qll"
Sep 12 00:28:56.591385 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:28:56.591341    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-stklr\" (UniqueName: \"kubernetes.io/projected/cc27ab80-b6a6-42e4-b46b-444815f35e60-kube-api-access-stklr\") pod \"limitador-limitador-8588749977-d6qll\" (UID: \"cc27ab80-b6a6-42e4-b46b-444815f35e60\") " pod="kuadrant-system/limitador-limitador-8588749977-d6qll"
Sep 12 00:28:56.591547 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:28:56.591405    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/cc27ab80-b6a6-42e4-b46b-444815f35e60-config-file\") pod \"limitador-limitador-8588749977-d6qll\" (UID: \"cc27ab80-b6a6-42e4-b46b-444815f35e60\") " pod="kuadrant-system/limitador-limitador-8588749977-d6qll"
Sep 12 00:28:56.592004 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:28:56.591982    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/cc27ab80-b6a6-42e4-b46b-444815f35e60-config-file\") pod \"limitador-limitador-8588749977-d6qll\" (UID: \"cc27ab80-b6a6-42e4-b46b-444815f35e60\") " pod="kuadrant-system/limitador-limitador-8588749977-d6qll"
Sep 12 00:28:56.599095 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:28:56.599068    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-stklr\" (UniqueName: \"kubernetes.io/projected/cc27ab80-b6a6-42e4-b46b-444815f35e60-kube-api-access-stklr\") pod \"limitador-limitador-8588749977-d6qll\" (UID: \"cc27ab80-b6a6-42e4-b46b-444815f35e60\") " pod="kuadrant-system/limitador-limitador-8588749977-d6qll"
Sep 12 00:28:56.664141 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:28:56.664081    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-8588749977-d6qll"
Sep 12 00:28:56.935745 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:28:56.935722    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-8588749977-d6qll"]
Sep 12 00:28:56.937614 ip-10-0-141-128 kubenswrapper[2571]: W0912 00:28:56.937575    2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcc27ab80_b6a6_42e4_b46b_444815f35e60.slice/crio-4118b046eb07fc87fef75e1355298019661c7a05b024aa53f472905854df1b38 WatchSource:0}: Error finding container 4118b046eb07fc87fef75e1355298019661c7a05b024aa53f472905854df1b38: Status 404 returned error can't find the container with id 4118b046eb07fc87fef75e1355298019661c7a05b024aa53f472905854df1b38
Sep 12 00:28:57.056788 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:28:57.056755    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-8588749977-d6qll" event={"ID":"cc27ab80-b6a6-42e4-b46b-444815f35e60","Type":"ContainerStarted","Data":"4118b046eb07fc87fef75e1355298019661c7a05b024aa53f472905854df1b38"}
Sep 12 00:28:59.068395 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:28:59.068360    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-8588749977-d6qll" event={"ID":"cc27ab80-b6a6-42e4-b46b-444815f35e60","Type":"ContainerStarted","Data":"a06752759c68c0dda5d3daa25869a192fe9cb43292037bf5553f60a7ca660a3a"}
Sep 12 00:28:59.068872 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:28:59.068478    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-limitador-8588749977-d6qll"
Sep 12 00:28:59.085026 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:28:59.084982    2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-limitador-8588749977-d6qll" podStartSLOduration=1.8729265069999999 podStartE2EDuration="3.084968162s" totalImagesPullingTime="1.212041655s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-12 00:28:56 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-12 00:28:59.083037576 +0000 UTC m=+750.354678082" watchObservedRunningTime="2026-09-12 00:28:59.084968162 +0000 UTC m=+750.356608671"
Sep 12 00:29:09.635702 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:29:09.635660    2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-limitador-8588749977-d6qll"]
Sep 12 00:29:09.636181 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:29:09.635963    2571 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/limitador-limitador-8588749977-d6qll" podUID="cc27ab80-b6a6-42e4-b46b-444815f35e60" containerName="limitador" containerID="cri-o://a06752759c68c0dda5d3daa25869a192fe9cb43292037bf5553f60a7ca660a3a" gracePeriod=30
Sep 12 00:29:09.636838 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:29:09.636689    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-limitador-8588749977-d6qll"
Sep 12 00:29:10.133640 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:29:10.133602    2571 generic.go:356] "Generic (PLEG): container finished" podID="cc27ab80-b6a6-42e4-b46b-444815f35e60" containerID="a06752759c68c0dda5d3daa25869a192fe9cb43292037bf5553f60a7ca660a3a" exitCode=0
Sep 12 00:29:10.133906 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:29:10.133622    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-8588749977-d6qll" event={"ID":"cc27ab80-b6a6-42e4-b46b-444815f35e60","Type":"ContainerDied","Data":"a06752759c68c0dda5d3daa25869a192fe9cb43292037bf5553f60a7ca660a3a"}
Sep 12 00:29:10.184987 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:29:10.184960    2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-8588749977-d6qll"
Sep 12 00:29:10.310370 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:29:10.310278    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-stklr\" (UniqueName: \"kubernetes.io/projected/cc27ab80-b6a6-42e4-b46b-444815f35e60-kube-api-access-stklr\") pod \"cc27ab80-b6a6-42e4-b46b-444815f35e60\" (UID: \"cc27ab80-b6a6-42e4-b46b-444815f35e60\") "
Sep 12 00:29:10.310370 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:29:10.310356    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/cc27ab80-b6a6-42e4-b46b-444815f35e60-config-file\") pod \"cc27ab80-b6a6-42e4-b46b-444815f35e60\" (UID: \"cc27ab80-b6a6-42e4-b46b-444815f35e60\") "
Sep 12 00:29:10.310736 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:29:10.310711    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/cc27ab80-b6a6-42e4-b46b-444815f35e60-config-file" (OuterVolumeSpecName: "config-file") pod "cc27ab80-b6a6-42e4-b46b-444815f35e60" (UID: "cc27ab80-b6a6-42e4-b46b-444815f35e60"). InnerVolumeSpecName "config-file". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 12 00:29:10.313135 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:29:10.313102    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/cc27ab80-b6a6-42e4-b46b-444815f35e60-kube-api-access-stklr" (OuterVolumeSpecName: "kube-api-access-stklr") pod "cc27ab80-b6a6-42e4-b46b-444815f35e60" (UID: "cc27ab80-b6a6-42e4-b46b-444815f35e60"). InnerVolumeSpecName "kube-api-access-stklr". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 12 00:29:10.411530 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:29:10.411490    2571 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-stklr\" (UniqueName: \"kubernetes.io/projected/cc27ab80-b6a6-42e4-b46b-444815f35e60-kube-api-access-stklr\") on node \"ip-10-0-141-128.ec2.internal\" DevicePath \"\""
Sep 12 00:29:10.411530 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:29:10.411522    2571 reconciler_common.go:299] "Volume detached for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/cc27ab80-b6a6-42e4-b46b-444815f35e60-config-file\") on node \"ip-10-0-141-128.ec2.internal\" DevicePath \"\""
Sep 12 00:29:11.142909 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:29:11.142870    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-8588749977-d6qll" event={"ID":"cc27ab80-b6a6-42e4-b46b-444815f35e60","Type":"ContainerDied","Data":"4118b046eb07fc87fef75e1355298019661c7a05b024aa53f472905854df1b38"}
Sep 12 00:29:11.142909 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:29:11.142901    2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-8588749977-d6qll"
Sep 12 00:29:11.143417 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:29:11.142935    2571 scope.go:117] "RemoveContainer" containerID="a06752759c68c0dda5d3daa25869a192fe9cb43292037bf5553f60a7ca660a3a"
Sep 12 00:29:11.164508 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:29:11.164476    2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-limitador-8588749977-d6qll"]
Sep 12 00:29:11.168001 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:29:11.167981    2571 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/limitador-limitador-8588749977-d6qll"]
Sep 12 00:29:13.140890 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:29:13.140854    2571 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="cc27ab80-b6a6-42e4-b46b-444815f35e60" path="/var/lib/kubelet/pods/cc27ab80-b6a6-42e4-b46b-444815f35e60/volumes"
Sep 12 00:29:15.057461 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:29:15.057428    2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/postgres-69fb4594cf-2bhcg"]
Sep 12 00:29:15.058032 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:29:15.058010    2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="cc27ab80-b6a6-42e4-b46b-444815f35e60" containerName="limitador"
Sep 12 00:29:15.058084 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:29:15.058037    2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="cc27ab80-b6a6-42e4-b46b-444815f35e60" containerName="limitador"
Sep 12 00:29:15.058149 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:29:15.058137    2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="cc27ab80-b6a6-42e4-b46b-444815f35e60" containerName="limitador"
Sep 12 00:29:15.062883 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:29:15.062859    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/postgres-69fb4594cf-2bhcg"
Sep 12 00:29:15.065444 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:29:15.065422    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"postgres-creds\""
Sep 12 00:29:15.065934 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:29:15.065909    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 12 00:29:15.066150 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:29:15.066130    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\""
Sep 12 00:29:15.066260 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:29:15.066173    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"default-dockercfg-fgs9v\""
Sep 12 00:29:15.067506 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:29:15.067483    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/postgres-69fb4594cf-2bhcg"]
Sep 12 00:29:15.154887 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:29:15.154860    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-s5b29\" (UniqueName: \"kubernetes.io/projected/af70f745-1b33-4737-82f1-04159afc98ca-kube-api-access-s5b29\") pod \"postgres-69fb4594cf-2bhcg\" (UID: \"af70f745-1b33-4737-82f1-04159afc98ca\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-2bhcg"
Sep 12 00:29:15.155037 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:29:15.154937    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/af70f745-1b33-4737-82f1-04159afc98ca-data\") pod \"postgres-69fb4594cf-2bhcg\" (UID: \"af70f745-1b33-4737-82f1-04159afc98ca\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-2bhcg"
Sep 12 00:29:15.255570 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:29:15.255536    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/af70f745-1b33-4737-82f1-04159afc98ca-data\") pod \"postgres-69fb4594cf-2bhcg\" (UID: \"af70f745-1b33-4737-82f1-04159afc98ca\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-2bhcg"
Sep 12 00:29:15.255570 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:29:15.255580    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-s5b29\" (UniqueName: \"kubernetes.io/projected/af70f745-1b33-4737-82f1-04159afc98ca-kube-api-access-s5b29\") pod \"postgres-69fb4594cf-2bhcg\" (UID: \"af70f745-1b33-4737-82f1-04159afc98ca\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-2bhcg"
Sep 12 00:29:15.256000 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:29:15.255975    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/af70f745-1b33-4737-82f1-04159afc98ca-data\") pod \"postgres-69fb4594cf-2bhcg\" (UID: \"af70f745-1b33-4737-82f1-04159afc98ca\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-2bhcg"
Sep 12 00:29:15.263024 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:29:15.262998    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-s5b29\" (UniqueName: \"kubernetes.io/projected/af70f745-1b33-4737-82f1-04159afc98ca-kube-api-access-s5b29\") pod \"postgres-69fb4594cf-2bhcg\" (UID: \"af70f745-1b33-4737-82f1-04159afc98ca\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-2bhcg"
Sep 12 00:29:15.381213 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:29:15.381142    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/postgres-69fb4594cf-2bhcg"
Sep 12 00:29:15.653617 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:29:15.653575    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/postgres-69fb4594cf-2bhcg"]
Sep 12 00:29:15.655459 ip-10-0-141-128 kubenswrapper[2571]: W0912 00:29:15.655431    2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podaf70f745_1b33_4737_82f1_04159afc98ca.slice/crio-09d7b3c3754f8d1e5276758d3dca574daf7543f805a29d6c1c301ffb4c5d0900 WatchSource:0}: Error finding container 09d7b3c3754f8d1e5276758d3dca574daf7543f805a29d6c1c301ffb4c5d0900: Status 404 returned error can't find the container with id 09d7b3c3754f8d1e5276758d3dca574daf7543f805a29d6c1c301ffb4c5d0900
Sep 12 00:29:16.167256 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:29:16.167218    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/postgres-69fb4594cf-2bhcg" event={"ID":"af70f745-1b33-4737-82f1-04159afc98ca","Type":"ContainerStarted","Data":"09d7b3c3754f8d1e5276758d3dca574daf7543f805a29d6c1c301ffb4c5d0900"}
Sep 12 00:29:21.195675 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:29:21.195632    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/postgres-69fb4594cf-2bhcg" event={"ID":"af70f745-1b33-4737-82f1-04159afc98ca","Type":"ContainerStarted","Data":"8228b8467c3f2f71772b8fecb10b5b3e48b6c5622383dc5eb8ebd5774bf38e22"}
Sep 12 00:29:21.196225 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:29:21.195751    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/postgres-69fb4594cf-2bhcg"
Sep 12 00:29:21.211839 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:29:21.211794    2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/postgres-69fb4594cf-2bhcg" podStartSLOduration=1.467038962 podStartE2EDuration="6.211779483s" totalImagesPullingTime="4.744740521s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-12 00:29:15 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-12 00:29:21.210375819 +0000 UTC m=+772.482016329" watchObservedRunningTime="2026-09-12 00:29:21.211779483 +0000 UTC m=+772.483420039"
Sep 12 00:29:27.229453 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:29:27.229425    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/postgres-69fb4594cf-2bhcg"
Sep 12 00:29:34.364731 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:29:34.364692    2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-5d489bdc4c-bb6cg"]
Sep 12 00:29:34.370745 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:29:34.370722    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-5d489bdc4c-bb6cg"
Sep 12 00:29:34.373308 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:29:34.373281    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-metrics-tls\""
Sep 12 00:29:34.373442 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:29:34.373287    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-km42l\""
Sep 12 00:29:34.373510 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:29:34.373452    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\""
Sep 12 00:29:34.374295 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:29:34.374280    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\""
Sep 12 00:29:34.382119 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:29:34.382084    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-5d489bdc4c-bb6cg"]
Sep 12 00:29:34.426670 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:29:34.426640    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/fb9aac00-40d0-4cb5-9634-380b7eb5ec74-metrics-certs\") pod \"maas-controller-5d489bdc4c-bb6cg\" (UID: \"fb9aac00-40d0-4cb5-9634-380b7eb5ec74\") " pod="opendatahub/maas-controller-5d489bdc4c-bb6cg"
Sep 12 00:29:34.426803 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:29:34.426683    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4szzf\" (UniqueName: \"kubernetes.io/projected/fb9aac00-40d0-4cb5-9634-380b7eb5ec74-kube-api-access-4szzf\") pod \"maas-controller-5d489bdc4c-bb6cg\" (UID: \"fb9aac00-40d0-4cb5-9634-380b7eb5ec74\") " pod="opendatahub/maas-controller-5d489bdc4c-bb6cg"
Sep 12 00:29:34.426803 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:29:34.426754    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/fb9aac00-40d0-4cb5-9634-380b7eb5ec74-webhook-cert\") pod \"maas-controller-5d489bdc4c-bb6cg\" (UID: \"fb9aac00-40d0-4cb5-9634-380b7eb5ec74\") " pod="opendatahub/maas-controller-5d489bdc4c-bb6cg"
Sep 12 00:29:34.527201 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:29:34.527172    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/fb9aac00-40d0-4cb5-9634-380b7eb5ec74-metrics-certs\") pod \"maas-controller-5d489bdc4c-bb6cg\" (UID: \"fb9aac00-40d0-4cb5-9634-380b7eb5ec74\") " pod="opendatahub/maas-controller-5d489bdc4c-bb6cg"
Sep 12 00:29:34.527343 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:29:34.527204    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4szzf\" (UniqueName: \"kubernetes.io/projected/fb9aac00-40d0-4cb5-9634-380b7eb5ec74-kube-api-access-4szzf\") pod \"maas-controller-5d489bdc4c-bb6cg\" (UID: \"fb9aac00-40d0-4cb5-9634-380b7eb5ec74\") " pod="opendatahub/maas-controller-5d489bdc4c-bb6cg"
Sep 12 00:29:34.527343 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:29:34.527235    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/fb9aac00-40d0-4cb5-9634-380b7eb5ec74-webhook-cert\") pod \"maas-controller-5d489bdc4c-bb6cg\" (UID: \"fb9aac00-40d0-4cb5-9634-380b7eb5ec74\") " pod="opendatahub/maas-controller-5d489bdc4c-bb6cg"
Sep 12 00:29:34.527343 ip-10-0-141-128 kubenswrapper[2571]: E0912 00:29:34.527325    2571 secret.go:189] Couldn't get secret opendatahub/maas-controller-webhook-cert: secret "maas-controller-webhook-cert" not found
Sep 12 00:29:34.527466 ip-10-0-141-128 kubenswrapper[2571]: E0912 00:29:34.527383    2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/fb9aac00-40d0-4cb5-9634-380b7eb5ec74-webhook-cert podName:fb9aac00-40d0-4cb5-9634-380b7eb5ec74 nodeName:}" failed. No retries permitted until 2026-09-12 00:29:35.02736472 +0000 UTC m=+786.299005227 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "webhook-cert" (UniqueName: "kubernetes.io/secret/fb9aac00-40d0-4cb5-9634-380b7eb5ec74-webhook-cert") pod "maas-controller-5d489bdc4c-bb6cg" (UID: "fb9aac00-40d0-4cb5-9634-380b7eb5ec74") : secret "maas-controller-webhook-cert" not found
Sep 12 00:29:34.529837 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:29:34.529815    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/fb9aac00-40d0-4cb5-9634-380b7eb5ec74-metrics-certs\") pod \"maas-controller-5d489bdc4c-bb6cg\" (UID: \"fb9aac00-40d0-4cb5-9634-380b7eb5ec74\") " pod="opendatahub/maas-controller-5d489bdc4c-bb6cg"
Sep 12 00:29:34.531289 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:29:34.531269    2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-57d99b475d-b9nc7"]
Sep 12 00:29:34.535467 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:29:34.535450    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-57d99b475d-b9nc7"
Sep 12 00:29:34.538859 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:29:34.538839    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4szzf\" (UniqueName: \"kubernetes.io/projected/fb9aac00-40d0-4cb5-9634-380b7eb5ec74-kube-api-access-4szzf\") pod \"maas-controller-5d489bdc4c-bb6cg\" (UID: \"fb9aac00-40d0-4cb5-9634-380b7eb5ec74\") " pod="opendatahub/maas-controller-5d489bdc4c-bb6cg"
Sep 12 00:29:34.545117 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:29:34.545088    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-57d99b475d-b9nc7"]
Sep 12 00:29:34.628118 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:29:34.628058    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/0759af38-2732-4bb6-b33b-4fa561ea1486-webhook-cert\") pod \"maas-controller-57d99b475d-b9nc7\" (UID: \"0759af38-2732-4bb6-b33b-4fa561ea1486\") " pod="opendatahub/maas-controller-57d99b475d-b9nc7"
Sep 12 00:29:34.628118 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:29:34.628095    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qr88c\" (UniqueName: \"kubernetes.io/projected/0759af38-2732-4bb6-b33b-4fa561ea1486-kube-api-access-qr88c\") pod \"maas-controller-57d99b475d-b9nc7\" (UID: \"0759af38-2732-4bb6-b33b-4fa561ea1486\") " pod="opendatahub/maas-controller-57d99b475d-b9nc7"
Sep 12 00:29:34.628300 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:29:34.628198    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/0759af38-2732-4bb6-b33b-4fa561ea1486-metrics-certs\") pod \"maas-controller-57d99b475d-b9nc7\" (UID: \"0759af38-2732-4bb6-b33b-4fa561ea1486\") " pod="opendatahub/maas-controller-57d99b475d-b9nc7"
Sep 12 00:29:34.729475 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:29:34.729448    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qr88c\" (UniqueName: \"kubernetes.io/projected/0759af38-2732-4bb6-b33b-4fa561ea1486-kube-api-access-qr88c\") pod \"maas-controller-57d99b475d-b9nc7\" (UID: \"0759af38-2732-4bb6-b33b-4fa561ea1486\") " pod="opendatahub/maas-controller-57d99b475d-b9nc7"
Sep 12 00:29:34.729619 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:29:34.729523    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/0759af38-2732-4bb6-b33b-4fa561ea1486-metrics-certs\") pod \"maas-controller-57d99b475d-b9nc7\" (UID: \"0759af38-2732-4bb6-b33b-4fa561ea1486\") " pod="opendatahub/maas-controller-57d99b475d-b9nc7"
Sep 12 00:29:34.729695 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:29:34.729631    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/0759af38-2732-4bb6-b33b-4fa561ea1486-webhook-cert\") pod \"maas-controller-57d99b475d-b9nc7\" (UID: \"0759af38-2732-4bb6-b33b-4fa561ea1486\") " pod="opendatahub/maas-controller-57d99b475d-b9nc7"
Sep 12 00:29:34.731888 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:29:34.731864    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/0759af38-2732-4bb6-b33b-4fa561ea1486-metrics-certs\") pod \"maas-controller-57d99b475d-b9nc7\" (UID: \"0759af38-2732-4bb6-b33b-4fa561ea1486\") " pod="opendatahub/maas-controller-57d99b475d-b9nc7"
Sep 12 00:29:34.731989 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:29:34.731870    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/0759af38-2732-4bb6-b33b-4fa561ea1486-webhook-cert\") pod \"maas-controller-57d99b475d-b9nc7\" (UID: \"0759af38-2732-4bb6-b33b-4fa561ea1486\") " pod="opendatahub/maas-controller-57d99b475d-b9nc7"
Sep 12 00:29:34.736960 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:29:34.736937    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qr88c\" (UniqueName: \"kubernetes.io/projected/0759af38-2732-4bb6-b33b-4fa561ea1486-kube-api-access-qr88c\") pod \"maas-controller-57d99b475d-b9nc7\" (UID: \"0759af38-2732-4bb6-b33b-4fa561ea1486\") " pod="opendatahub/maas-controller-57d99b475d-b9nc7"
Sep 12 00:29:34.858753 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:29:34.858719    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-57d99b475d-b9nc7"
Sep 12 00:29:35.033471 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:29:35.033436    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/fb9aac00-40d0-4cb5-9634-380b7eb5ec74-webhook-cert\") pod \"maas-controller-5d489bdc4c-bb6cg\" (UID: \"fb9aac00-40d0-4cb5-9634-380b7eb5ec74\") " pod="opendatahub/maas-controller-5d489bdc4c-bb6cg"
Sep 12 00:29:35.035722 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:29:35.035696    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/fb9aac00-40d0-4cb5-9634-380b7eb5ec74-webhook-cert\") pod \"maas-controller-5d489bdc4c-bb6cg\" (UID: \"fb9aac00-40d0-4cb5-9634-380b7eb5ec74\") " pod="opendatahub/maas-controller-5d489bdc4c-bb6cg"
Sep 12 00:29:35.126734 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:29:35.126712    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-57d99b475d-b9nc7"]
Sep 12 00:29:35.128686 ip-10-0-141-128 kubenswrapper[2571]: W0912 00:29:35.128663    2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0759af38_2732_4bb6_b33b_4fa561ea1486.slice/crio-14b3bb08e877fef60f076a425adbcb3a59466b9ffd334a59045a6d592efb4591 WatchSource:0}: Error finding container 14b3bb08e877fef60f076a425adbcb3a59466b9ffd334a59045a6d592efb4591: Status 404 returned error can't find the container with id 14b3bb08e877fef60f076a425adbcb3a59466b9ffd334a59045a6d592efb4591
Sep 12 00:29:35.258695 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:29:35.258663    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-57d99b475d-b9nc7" event={"ID":"0759af38-2732-4bb6-b33b-4fa561ea1486","Type":"ContainerStarted","Data":"14b3bb08e877fef60f076a425adbcb3a59466b9ffd334a59045a6d592efb4591"}
Sep 12 00:29:35.284804 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:29:35.284778    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-5d489bdc4c-bb6cg"
Sep 12 00:29:35.555958 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:29:35.555927    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-5d489bdc4c-bb6cg"]
Sep 12 00:29:35.558956 ip-10-0-141-128 kubenswrapper[2571]: W0912 00:29:35.558927    2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfb9aac00_40d0_4cb5_9634_380b7eb5ec74.slice/crio-76292be9eff60cf77a8e5f4ca832c15921aeefa0df3734d1a766896ab8467f29 WatchSource:0}: Error finding container 76292be9eff60cf77a8e5f4ca832c15921aeefa0df3734d1a766896ab8467f29: Status 404 returned error can't find the container with id 76292be9eff60cf77a8e5f4ca832c15921aeefa0df3734d1a766896ab8467f29
Sep 12 00:29:36.269483 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:29:36.269389    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-5d489bdc4c-bb6cg" event={"ID":"fb9aac00-40d0-4cb5-9634-380b7eb5ec74","Type":"ContainerStarted","Data":"76292be9eff60cf77a8e5f4ca832c15921aeefa0df3734d1a766896ab8467f29"}
Sep 12 00:29:38.280898 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:29:38.280857    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-5d489bdc4c-bb6cg" event={"ID":"fb9aac00-40d0-4cb5-9634-380b7eb5ec74","Type":"ContainerStarted","Data":"ad9d3e40110831bab94f28000644fe436af0aa3bd5afd895db59a7d3ca0e61b5"}
Sep 12 00:29:38.281354 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:29:38.280997    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-5d489bdc4c-bb6cg"
Sep 12 00:29:38.282425 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:29:38.282402    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-57d99b475d-b9nc7" event={"ID":"0759af38-2732-4bb6-b33b-4fa561ea1486","Type":"ContainerStarted","Data":"df1d323fdc706a416b1d2807f06fb501c0083fee9a7be91704d37e5544bc8878"}
Sep 12 00:29:38.282534 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:29:38.282517    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-57d99b475d-b9nc7"
Sep 12 00:29:38.297664 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:29:38.297620    2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-5d489bdc4c-bb6cg" podStartSLOduration=2.116415235 podStartE2EDuration="4.297580034s" totalImagesPullingTime="2.181164799s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-12 00:29:34 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-12 00:29:38.296990807 +0000 UTC m=+789.568631317" watchObservedRunningTime="2026-09-12 00:29:38.297580034 +0000 UTC m=+789.569220570"
Sep 12 00:29:38.315473 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:29:38.315430    2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-57d99b475d-b9nc7" podStartSLOduration=1.710486138 podStartE2EDuration="4.315416827s" totalImagesPullingTime="2.604930689s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-12 00:29:34 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-12 00:29:38.313803856 +0000 UTC m=+789.585444365" watchObservedRunningTime="2026-09-12 00:29:38.315416827 +0000 UTC m=+789.587057332"
Sep 12 00:29:49.290470 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:29:49.290441    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-5d489bdc4c-bb6cg"
Sep 12 00:29:49.290876 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:29:49.290510    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-57d99b475d-b9nc7"
Sep 12 00:29:49.350471 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:29:49.350427    2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-5d489bdc4c-bb6cg"]
Sep 12 00:29:49.351119 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:29:49.350760    2571 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/maas-controller-5d489bdc4c-bb6cg" podUID="fb9aac00-40d0-4cb5-9634-380b7eb5ec74" containerName="manager" containerID="cri-o://ad9d3e40110831bab94f28000644fe436af0aa3bd5afd895db59a7d3ca0e61b5" gracePeriod=30
Sep 12 00:29:49.603064 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:29:49.603037    2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-5d489bdc4c-bb6cg"
Sep 12 00:29:49.672090 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:29:49.672061    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-4szzf\" (UniqueName: \"kubernetes.io/projected/fb9aac00-40d0-4cb5-9634-380b7eb5ec74-kube-api-access-4szzf\") pod \"fb9aac00-40d0-4cb5-9634-380b7eb5ec74\" (UID: \"fb9aac00-40d0-4cb5-9634-380b7eb5ec74\") "
Sep 12 00:29:49.672286 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:29:49.672104    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/fb9aac00-40d0-4cb5-9634-380b7eb5ec74-webhook-cert\") pod \"fb9aac00-40d0-4cb5-9634-380b7eb5ec74\" (UID: \"fb9aac00-40d0-4cb5-9634-380b7eb5ec74\") "
Sep 12 00:29:49.672286 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:29:49.672229    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/fb9aac00-40d0-4cb5-9634-380b7eb5ec74-metrics-certs\") pod \"fb9aac00-40d0-4cb5-9634-380b7eb5ec74\" (UID: \"fb9aac00-40d0-4cb5-9634-380b7eb5ec74\") "
Sep 12 00:29:49.674309 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:29:49.674276    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/fb9aac00-40d0-4cb5-9634-380b7eb5ec74-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "fb9aac00-40d0-4cb5-9634-380b7eb5ec74" (UID: "fb9aac00-40d0-4cb5-9634-380b7eb5ec74"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 12 00:29:49.674409 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:29:49.674316    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fb9aac00-40d0-4cb5-9634-380b7eb5ec74-kube-api-access-4szzf" (OuterVolumeSpecName: "kube-api-access-4szzf") pod "fb9aac00-40d0-4cb5-9634-380b7eb5ec74" (UID: "fb9aac00-40d0-4cb5-9634-380b7eb5ec74"). InnerVolumeSpecName "kube-api-access-4szzf". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 12 00:29:49.674409 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:29:49.674332    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/fb9aac00-40d0-4cb5-9634-380b7eb5ec74-webhook-cert" (OuterVolumeSpecName: "webhook-cert") pod "fb9aac00-40d0-4cb5-9634-380b7eb5ec74" (UID: "fb9aac00-40d0-4cb5-9634-380b7eb5ec74"). InnerVolumeSpecName "webhook-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 12 00:29:49.773942 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:29:49.773903    2571 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/fb9aac00-40d0-4cb5-9634-380b7eb5ec74-metrics-certs\") on node \"ip-10-0-141-128.ec2.internal\" DevicePath \"\""
Sep 12 00:29:49.773942 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:29:49.773936    2571 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4szzf\" (UniqueName: \"kubernetes.io/projected/fb9aac00-40d0-4cb5-9634-380b7eb5ec74-kube-api-access-4szzf\") on node \"ip-10-0-141-128.ec2.internal\" DevicePath \"\""
Sep 12 00:29:49.773942 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:29:49.773946    2571 reconciler_common.go:299] "Volume detached for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/fb9aac00-40d0-4cb5-9634-380b7eb5ec74-webhook-cert\") on node \"ip-10-0-141-128.ec2.internal\" DevicePath \"\""
Sep 12 00:29:50.337314 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:29:50.337278    2571 generic.go:356] "Generic (PLEG): container finished" podID="fb9aac00-40d0-4cb5-9634-380b7eb5ec74" containerID="ad9d3e40110831bab94f28000644fe436af0aa3bd5afd895db59a7d3ca0e61b5" exitCode=0
Sep 12 00:29:50.337735 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:29:50.337326    2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-5d489bdc4c-bb6cg"
Sep 12 00:29:50.337735 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:29:50.337366    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-5d489bdc4c-bb6cg" event={"ID":"fb9aac00-40d0-4cb5-9634-380b7eb5ec74","Type":"ContainerDied","Data":"ad9d3e40110831bab94f28000644fe436af0aa3bd5afd895db59a7d3ca0e61b5"}
Sep 12 00:29:50.337735 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:29:50.337422    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-5d489bdc4c-bb6cg" event={"ID":"fb9aac00-40d0-4cb5-9634-380b7eb5ec74","Type":"ContainerDied","Data":"76292be9eff60cf77a8e5f4ca832c15921aeefa0df3734d1a766896ab8467f29"}
Sep 12 00:29:50.337735 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:29:50.337450    2571 scope.go:117] "RemoveContainer" containerID="ad9d3e40110831bab94f28000644fe436af0aa3bd5afd895db59a7d3ca0e61b5"
Sep 12 00:29:50.346807 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:29:50.346788    2571 scope.go:117] "RemoveContainer" containerID="ad9d3e40110831bab94f28000644fe436af0aa3bd5afd895db59a7d3ca0e61b5"
Sep 12 00:29:50.347099 ip-10-0-141-128 kubenswrapper[2571]: E0912 00:29:50.347074    2571 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ad9d3e40110831bab94f28000644fe436af0aa3bd5afd895db59a7d3ca0e61b5\": container with ID starting with ad9d3e40110831bab94f28000644fe436af0aa3bd5afd895db59a7d3ca0e61b5 not found: ID does not exist" containerID="ad9d3e40110831bab94f28000644fe436af0aa3bd5afd895db59a7d3ca0e61b5"
Sep 12 00:29:50.347193 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:29:50.347105    2571 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ad9d3e40110831bab94f28000644fe436af0aa3bd5afd895db59a7d3ca0e61b5"} err="failed to get container status \"ad9d3e40110831bab94f28000644fe436af0aa3bd5afd895db59a7d3ca0e61b5\": rpc error: code = NotFound desc = could not find container \"ad9d3e40110831bab94f28000644fe436af0aa3bd5afd895db59a7d3ca0e61b5\": container with ID starting with ad9d3e40110831bab94f28000644fe436af0aa3bd5afd895db59a7d3ca0e61b5 not found: ID does not exist"
Sep 12 00:29:50.358723 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:29:50.358696    2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-5d489bdc4c-bb6cg"]
Sep 12 00:29:50.362336 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:29:50.362313    2571 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-5d489bdc4c-bb6cg"]
Sep 12 00:29:51.140971 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:29:51.140937    2571 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="fb9aac00-40d0-4cb5-9634-380b7eb5ec74" path="/var/lib/kubelet/pods/fb9aac00-40d0-4cb5-9634-380b7eb5ec74/volumes"
Sep 12 00:30:11.008227 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:30:11.008144    2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-6b768f88d8-nv8zh"]
Sep 12 00:30:11.008722 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:30:11.008555    2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="fb9aac00-40d0-4cb5-9634-380b7eb5ec74" containerName="manager"
Sep 12 00:30:11.008722 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:30:11.008572    2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="fb9aac00-40d0-4cb5-9634-380b7eb5ec74" containerName="manager"
Sep 12 00:30:11.008722 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:30:11.008676    2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="fb9aac00-40d0-4cb5-9634-380b7eb5ec74" containerName="manager"
Sep 12 00:30:11.014535 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:30:11.014518    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-6b768f88d8-nv8zh"
Sep 12 00:30:11.017062 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:30:11.016999    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-serving-cert\""
Sep 12 00:30:11.017062 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:30:11.016999    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-metrics-tls\""
Sep 12 00:30:11.017228 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:30:11.017101    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-f4scw\""
Sep 12 00:30:11.024431 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:30:11.023919    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-6b768f88d8-nv8zh"]
Sep 12 00:30:11.153091 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:30:11.153058    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-f4z74\" (UniqueName: \"kubernetes.io/projected/f77dec72-81c1-4510-a251-07cda55f274f-kube-api-access-f4z74\") pod \"maas-api-6b768f88d8-nv8zh\" (UID: \"f77dec72-81c1-4510-a251-07cda55f274f\") " pod="odh-ai-gateway-infra/maas-api-6b768f88d8-nv8zh"
Sep 12 00:30:11.153264 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:30:11.153118    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f77dec72-81c1-4510-a251-07cda55f274f-openshift-service-ca\") pod \"maas-api-6b768f88d8-nv8zh\" (UID: \"f77dec72-81c1-4510-a251-07cda55f274f\") " pod="odh-ai-gateway-infra/maas-api-6b768f88d8-nv8zh"
Sep 12 00:30:11.153264 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:30:11.153193    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f77dec72-81c1-4510-a251-07cda55f274f-metrics-certs\") pod \"maas-api-6b768f88d8-nv8zh\" (UID: \"f77dec72-81c1-4510-a251-07cda55f274f\") " pod="odh-ai-gateway-infra/maas-api-6b768f88d8-nv8zh"
Sep 12 00:30:11.153367 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:30:11.153265    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f77dec72-81c1-4510-a251-07cda55f274f-maas-api-tls\") pod \"maas-api-6b768f88d8-nv8zh\" (UID: \"f77dec72-81c1-4510-a251-07cda55f274f\") " pod="odh-ai-gateway-infra/maas-api-6b768f88d8-nv8zh"
Sep 12 00:30:11.254633 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:30:11.254579    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f77dec72-81c1-4510-a251-07cda55f274f-openshift-service-ca\") pod \"maas-api-6b768f88d8-nv8zh\" (UID: \"f77dec72-81c1-4510-a251-07cda55f274f\") " pod="odh-ai-gateway-infra/maas-api-6b768f88d8-nv8zh"
Sep 12 00:30:11.254766 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:30:11.254653    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f77dec72-81c1-4510-a251-07cda55f274f-metrics-certs\") pod \"maas-api-6b768f88d8-nv8zh\" (UID: \"f77dec72-81c1-4510-a251-07cda55f274f\") " pod="odh-ai-gateway-infra/maas-api-6b768f88d8-nv8zh"
Sep 12 00:30:11.254766 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:30:11.254710    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f77dec72-81c1-4510-a251-07cda55f274f-maas-api-tls\") pod \"maas-api-6b768f88d8-nv8zh\" (UID: \"f77dec72-81c1-4510-a251-07cda55f274f\") " pod="odh-ai-gateway-infra/maas-api-6b768f88d8-nv8zh"
Sep 12 00:30:11.254847 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:30:11.254765    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-f4z74\" (UniqueName: \"kubernetes.io/projected/f77dec72-81c1-4510-a251-07cda55f274f-kube-api-access-f4z74\") pod \"maas-api-6b768f88d8-nv8zh\" (UID: \"f77dec72-81c1-4510-a251-07cda55f274f\") " pod="odh-ai-gateway-infra/maas-api-6b768f88d8-nv8zh"
Sep 12 00:30:11.254847 ip-10-0-141-128 kubenswrapper[2571]: E0912 00:30:11.254792    2571 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-metrics-tls: secret "maas-api-metrics-tls" not found
Sep 12 00:30:11.254917 ip-10-0-141-128 kubenswrapper[2571]: E0912 00:30:11.254843    2571 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-serving-cert: secret "maas-api-serving-cert" not found
Sep 12 00:30:11.254917 ip-10-0-141-128 kubenswrapper[2571]: E0912 00:30:11.254871    2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f77dec72-81c1-4510-a251-07cda55f274f-metrics-certs podName:f77dec72-81c1-4510-a251-07cda55f274f nodeName:}" failed. No retries permitted until 2026-09-12 00:30:11.754850707 +0000 UTC m=+823.026491201 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/f77dec72-81c1-4510-a251-07cda55f274f-metrics-certs") pod "maas-api-6b768f88d8-nv8zh" (UID: "f77dec72-81c1-4510-a251-07cda55f274f") : secret "maas-api-metrics-tls" not found
Sep 12 00:30:11.254917 ip-10-0-141-128 kubenswrapper[2571]: E0912 00:30:11.254888    2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f77dec72-81c1-4510-a251-07cda55f274f-maas-api-tls podName:f77dec72-81c1-4510-a251-07cda55f274f nodeName:}" failed. No retries permitted until 2026-09-12 00:30:11.75487906 +0000 UTC m=+823.026519547 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/f77dec72-81c1-4510-a251-07cda55f274f-maas-api-tls") pod "maas-api-6b768f88d8-nv8zh" (UID: "f77dec72-81c1-4510-a251-07cda55f274f") : secret "maas-api-serving-cert" not found
Sep 12 00:30:11.255282 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:30:11.255262    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f77dec72-81c1-4510-a251-07cda55f274f-openshift-service-ca\") pod \"maas-api-6b768f88d8-nv8zh\" (UID: \"f77dec72-81c1-4510-a251-07cda55f274f\") " pod="odh-ai-gateway-infra/maas-api-6b768f88d8-nv8zh"
Sep 12 00:30:11.263007 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:30:11.262954    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-f4z74\" (UniqueName: \"kubernetes.io/projected/f77dec72-81c1-4510-a251-07cda55f274f-kube-api-access-f4z74\") pod \"maas-api-6b768f88d8-nv8zh\" (UID: \"f77dec72-81c1-4510-a251-07cda55f274f\") " pod="odh-ai-gateway-infra/maas-api-6b768f88d8-nv8zh"
Sep 12 00:30:11.491740 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:30:11.491707    2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-59fbc47bf8-p2vwz"]
Sep 12 00:30:11.495225 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:30:11.495205    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-59fbc47bf8-p2vwz"
Sep 12 00:30:11.497914 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:30:11.497891    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins\""
Sep 12 00:30:11.506624 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:30:11.506605    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-59fbc47bf8-p2vwz"]
Sep 12 00:30:11.518436 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:30:11.518379    2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-5b54b7fbb7-btmpn"]
Sep 12 00:30:11.522433 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:30:11.522413    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-5b54b7fbb7-btmpn"
Sep 12 00:30:11.524695 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:30:11.524680    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-dockercfg-4mvwq\""
Sep 12 00:30:11.532523 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:30:11.532497    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-5b54b7fbb7-btmpn"]
Sep 12 00:30:11.659200 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:30:11.659167    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7476l\" (UniqueName: \"kubernetes.io/projected/817b7aba-94b1-4306-b26a-027e1e0b7297-kube-api-access-7476l\") pod \"payload-processing-5b54b7fbb7-btmpn\" (UID: \"817b7aba-94b1-4306-b26a-027e1e0b7297\") " pod="openshift-ingress/payload-processing-5b54b7fbb7-btmpn"
Sep 12 00:30:11.659344 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:30:11.659304    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/817b7aba-94b1-4306-b26a-027e1e0b7297-plugins-config-volume\") pod \"payload-processing-5b54b7fbb7-btmpn\" (UID: \"817b7aba-94b1-4306-b26a-027e1e0b7297\") " pod="openshift-ingress/payload-processing-5b54b7fbb7-btmpn"
Sep 12 00:30:11.659387 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:30:11.659349    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cpn8l\" (UniqueName: \"kubernetes.io/projected/e335c05b-bd1d-468c-b882-f8a372524c56-kube-api-access-cpn8l\") pod \"payload-pre-processing-59fbc47bf8-p2vwz\" (UID: \"e335c05b-bd1d-468c-b882-f8a372524c56\") " pod="openshift-ingress/payload-pre-processing-59fbc47bf8-p2vwz"
Sep 12 00:30:11.659387 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:30:11.659381    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e335c05b-bd1d-468c-b882-f8a372524c56-plugins-config-volume\") pod \"payload-pre-processing-59fbc47bf8-p2vwz\" (UID: \"e335c05b-bd1d-468c-b882-f8a372524c56\") " pod="openshift-ingress/payload-pre-processing-59fbc47bf8-p2vwz"
Sep 12 00:30:11.760540 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:30:11.760503    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f77dec72-81c1-4510-a251-07cda55f274f-maas-api-tls\") pod \"maas-api-6b768f88d8-nv8zh\" (UID: \"f77dec72-81c1-4510-a251-07cda55f274f\") " pod="odh-ai-gateway-infra/maas-api-6b768f88d8-nv8zh"
Sep 12 00:30:11.760741 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:30:11.760555    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cpn8l\" (UniqueName: \"kubernetes.io/projected/e335c05b-bd1d-468c-b882-f8a372524c56-kube-api-access-cpn8l\") pod \"payload-pre-processing-59fbc47bf8-p2vwz\" (UID: \"e335c05b-bd1d-468c-b882-f8a372524c56\") " pod="openshift-ingress/payload-pre-processing-59fbc47bf8-p2vwz"
Sep 12 00:30:11.760741 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:30:11.760621    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e335c05b-bd1d-468c-b882-f8a372524c56-plugins-config-volume\") pod \"payload-pre-processing-59fbc47bf8-p2vwz\" (UID: \"e335c05b-bd1d-468c-b882-f8a372524c56\") " pod="openshift-ingress/payload-pre-processing-59fbc47bf8-p2vwz"
Sep 12 00:30:11.760741 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:30:11.760718    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7476l\" (UniqueName: \"kubernetes.io/projected/817b7aba-94b1-4306-b26a-027e1e0b7297-kube-api-access-7476l\") pod \"payload-processing-5b54b7fbb7-btmpn\" (UID: \"817b7aba-94b1-4306-b26a-027e1e0b7297\") " pod="openshift-ingress/payload-processing-5b54b7fbb7-btmpn"
Sep 12 00:30:11.760910 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:30:11.760757    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f77dec72-81c1-4510-a251-07cda55f274f-metrics-certs\") pod \"maas-api-6b768f88d8-nv8zh\" (UID: \"f77dec72-81c1-4510-a251-07cda55f274f\") " pod="odh-ai-gateway-infra/maas-api-6b768f88d8-nv8zh"
Sep 12 00:30:11.760910 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:30:11.760825    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/817b7aba-94b1-4306-b26a-027e1e0b7297-plugins-config-volume\") pod \"payload-processing-5b54b7fbb7-btmpn\" (UID: \"817b7aba-94b1-4306-b26a-027e1e0b7297\") " pod="openshift-ingress/payload-processing-5b54b7fbb7-btmpn"
Sep 12 00:30:11.761433 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:30:11.761409    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e335c05b-bd1d-468c-b882-f8a372524c56-plugins-config-volume\") pod \"payload-pre-processing-59fbc47bf8-p2vwz\" (UID: \"e335c05b-bd1d-468c-b882-f8a372524c56\") " pod="openshift-ingress/payload-pre-processing-59fbc47bf8-p2vwz"
Sep 12 00:30:11.761559 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:30:11.761537    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/817b7aba-94b1-4306-b26a-027e1e0b7297-plugins-config-volume\") pod \"payload-processing-5b54b7fbb7-btmpn\" (UID: \"817b7aba-94b1-4306-b26a-027e1e0b7297\") " pod="openshift-ingress/payload-processing-5b54b7fbb7-btmpn"
Sep 12 00:30:11.763117 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:30:11.763089    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f77dec72-81c1-4510-a251-07cda55f274f-metrics-certs\") pod \"maas-api-6b768f88d8-nv8zh\" (UID: \"f77dec72-81c1-4510-a251-07cda55f274f\") " pod="odh-ai-gateway-infra/maas-api-6b768f88d8-nv8zh"
Sep 12 00:30:11.763206 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:30:11.763131    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f77dec72-81c1-4510-a251-07cda55f274f-maas-api-tls\") pod \"maas-api-6b768f88d8-nv8zh\" (UID: \"f77dec72-81c1-4510-a251-07cda55f274f\") " pod="odh-ai-gateway-infra/maas-api-6b768f88d8-nv8zh"
Sep 12 00:30:11.768482 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:30:11.768432    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7476l\" (UniqueName: \"kubernetes.io/projected/817b7aba-94b1-4306-b26a-027e1e0b7297-kube-api-access-7476l\") pod \"payload-processing-5b54b7fbb7-btmpn\" (UID: \"817b7aba-94b1-4306-b26a-027e1e0b7297\") " pod="openshift-ingress/payload-processing-5b54b7fbb7-btmpn"
Sep 12 00:30:11.768575 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:30:11.768529    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cpn8l\" (UniqueName: \"kubernetes.io/projected/e335c05b-bd1d-468c-b882-f8a372524c56-kube-api-access-cpn8l\") pod \"payload-pre-processing-59fbc47bf8-p2vwz\" (UID: \"e335c05b-bd1d-468c-b882-f8a372524c56\") " pod="openshift-ingress/payload-pre-processing-59fbc47bf8-p2vwz"
Sep 12 00:30:11.806340 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:30:11.806317    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-59fbc47bf8-p2vwz"
Sep 12 00:30:11.834191 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:30:11.834169    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-5b54b7fbb7-btmpn"
Sep 12 00:30:11.927647 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:30:11.927621    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-6b768f88d8-nv8zh"
Sep 12 00:30:12.094134 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:30:12.094115    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-59fbc47bf8-p2vwz"]
Sep 12 00:30:12.096661 ip-10-0-141-128 kubenswrapper[2571]: W0912 00:30:12.096632    2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode335c05b_bd1d_468c_b882_f8a372524c56.slice/crio-54335649fbd94de9e9e55a16992258f88c1814d157c28e961d1a86e369577e83 WatchSource:0}: Error finding container 54335649fbd94de9e9e55a16992258f88c1814d157c28e961d1a86e369577e83: Status 404 returned error can't find the container with id 54335649fbd94de9e9e55a16992258f88c1814d157c28e961d1a86e369577e83
Sep 12 00:30:12.116886 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:30:12.116850    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-5b54b7fbb7-btmpn"]
Sep 12 00:30:12.118615 ip-10-0-141-128 kubenswrapper[2571]: W0912 00:30:12.118571    2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod817b7aba_94b1_4306_b26a_027e1e0b7297.slice/crio-487c1370ca1f72be7a51e93553ad2a15a8da9d0f6e01435ac9e2c2425970d16b WatchSource:0}: Error finding container 487c1370ca1f72be7a51e93553ad2a15a8da9d0f6e01435ac9e2c2425970d16b: Status 404 returned error can't find the container with id 487c1370ca1f72be7a51e93553ad2a15a8da9d0f6e01435ac9e2c2425970d16b
Sep 12 00:30:12.206730 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:30:12.206702    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-6b768f88d8-nv8zh"]
Sep 12 00:30:12.209617 ip-10-0-141-128 kubenswrapper[2571]: W0912 00:30:12.209574    2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf77dec72_81c1_4510_a251_07cda55f274f.slice/crio-d97fc3a5e3a4f75db6b194dd2de061d2ce248bd854fc0ff9aa8c87ae589d3424 WatchSource:0}: Error finding container d97fc3a5e3a4f75db6b194dd2de061d2ce248bd854fc0ff9aa8c87ae589d3424: Status 404 returned error can't find the container with id d97fc3a5e3a4f75db6b194dd2de061d2ce248bd854fc0ff9aa8c87ae589d3424
Sep 12 00:30:12.444235 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:30:12.444193    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-59fbc47bf8-p2vwz" event={"ID":"e335c05b-bd1d-468c-b882-f8a372524c56","Type":"ContainerStarted","Data":"54335649fbd94de9e9e55a16992258f88c1814d157c28e961d1a86e369577e83"}
Sep 12 00:30:12.445249 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:30:12.445226    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-6b768f88d8-nv8zh" event={"ID":"f77dec72-81c1-4510-a251-07cda55f274f","Type":"ContainerStarted","Data":"d97fc3a5e3a4f75db6b194dd2de061d2ce248bd854fc0ff9aa8c87ae589d3424"}
Sep 12 00:30:12.446266 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:30:12.446247    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-5b54b7fbb7-btmpn" event={"ID":"817b7aba-94b1-4306-b26a-027e1e0b7297","Type":"ContainerStarted","Data":"487c1370ca1f72be7a51e93553ad2a15a8da9d0f6e01435ac9e2c2425970d16b"}
Sep 12 00:30:16.496092 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:30:16.496062    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-59fbc47bf8-p2vwz" event={"ID":"e335c05b-bd1d-468c-b882-f8a372524c56","Type":"ContainerStarted","Data":"1a34713cc08910d95ca6cc26290a60561cfa05226a6f94affbed594d2252cf88"}
Sep 12 00:30:16.496717 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:30:16.496133    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-59fbc47bf8-p2vwz"
Sep 12 00:30:16.497887 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:30:16.497860    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-6b768f88d8-nv8zh" event={"ID":"f77dec72-81c1-4510-a251-07cda55f274f","Type":"ContainerStarted","Data":"3601896e14e6510fb55df38e6ce6674dbd8f69b82290b99e2c03a293cc240494"}
Sep 12 00:30:16.498011 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:30:16.497947    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-6b768f88d8-nv8zh"
Sep 12 00:30:16.499688 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:30:16.499667    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-5b54b7fbb7-btmpn" event={"ID":"817b7aba-94b1-4306-b26a-027e1e0b7297","Type":"ContainerStarted","Data":"75fe6e0d09546a5387f7c435f1baa29add38d066a07b77b128bcb27ecd13fef4"}
Sep 12 00:30:16.499804 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:30:16.499790    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-5b54b7fbb7-btmpn"
Sep 12 00:30:16.519067 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:30:16.514761    2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-59fbc47bf8-p2vwz" podStartSLOduration=1.7108867399999999 podStartE2EDuration="5.514744093s" totalImagesPullingTime="3.803857353s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-12 00:30:11 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-12 00:30:16.511276275 +0000 UTC m=+827.782916784" watchObservedRunningTime="2026-09-12 00:30:16.514744093 +0000 UTC m=+827.786384603"
Sep 12 00:30:16.528549 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:30:16.528508    2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-6b768f88d8-nv8zh" podStartSLOduration=2.909940531 podStartE2EDuration="6.528496267s" totalImagesPullingTime="3.618555736s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-12 00:30:10 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-12 00:30:16.526936101 +0000 UTC m=+827.798576611" watchObservedRunningTime="2026-09-12 00:30:16.528496267 +0000 UTC m=+827.800136775"
Sep 12 00:30:16.542545 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:30:16.542503    2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-5b54b7fbb7-btmpn" podStartSLOduration=1.7539902459999999 podStartE2EDuration="5.542491384s" totalImagesPullingTime="3.788501138s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-12 00:30:11 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-12 00:30:16.540666622 +0000 UTC m=+827.812307132" watchObservedRunningTime="2026-09-12 00:30:16.542491384 +0000 UTC m=+827.814131893"
Sep 12 00:30:21.525072 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:30:21.525034    2571 generic.go:356] "Generic (PLEG): container finished" podID="06461169-3e71-40a2-a0dc-e52b89db9f42" containerID="731ce6496d2e4a0d3fd0856b02f10a59d93bde66cc255b792ed83ce1aa0218ec" exitCode=0
Sep 12 00:30:21.525461 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:30:21.525109    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-svr8q" event={"ID":"06461169-3e71-40a2-a0dc-e52b89db9f42","Type":"ContainerDied","Data":"731ce6496d2e4a0d3fd0856b02f10a59d93bde66cc255b792ed83ce1aa0218ec"}
Sep 12 00:30:21.525522 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:30:21.525504    2571 scope.go:117] "RemoveContainer" containerID="731ce6496d2e4a0d3fd0856b02f10a59d93bde66cc255b792ed83ce1aa0218ec"
Sep 12 00:30:21.526463 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:30:21.526439    2571 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236196Ki","pods":"250"}
Sep 12 00:30:21.526536 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:30:21.526501    2571 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236196Ki","pods":"250"}
Sep 12 00:30:21.526536 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:30:21.526528    2571 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236196Ki","pods":"250"}
Sep 12 00:30:22.511146 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:30:22.511115    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-6b768f88d8-nv8zh"
Sep 12 00:30:22.531962 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:30:22.531933    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-svr8q" event={"ID":"06461169-3e71-40a2-a0dc-e52b89db9f42","Type":"ContainerStarted","Data":"6fc4b5a8a577afca8e66d1a7b5565fa67a1e8ac9c3b7ee37dc47c8a5d2e3fcd9"}
Sep 12 00:30:22.588073 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:30:22.588035    2571 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-78bb78ff78-svr8q container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.31:15021/healthz/ready\": dial tcp 10.132.0.31:15021: connect: connection refused" start-of-body=
Sep 12 00:30:22.588342 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:30:22.588313    2571 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-svr8q" podUID="06461169-3e71-40a2-a0dc-e52b89db9f42" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.31:15021/healthz/ready\": dial tcp 10.132.0.31:15021: connect: connection refused"
Sep 12 00:30:22.588492 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:30:22.588479    2571 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-svr8q"
Sep 12 00:30:23.588740 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:30:23.588706    2571 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-78bb78ff78-svr8q container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.31:15021/healthz/ready\": dial tcp 10.132.0.31:15021: connect: connection refused" start-of-body=
Sep 12 00:30:23.589101 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:30:23.588759    2571 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-svr8q" podUID="06461169-3e71-40a2-a0dc-e52b89db9f42" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.31:15021/healthz/ready\": dial tcp 10.132.0.31:15021: connect: connection refused"
Sep 12 00:30:24.588409 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:30:24.588373    2571 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-78bb78ff78-svr8q container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.31:15021/healthz/ready\": dial tcp 10.132.0.31:15021: connect: connection refused" start-of-body=
Sep 12 00:30:24.588703 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:30:24.588671    2571 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-svr8q" podUID="06461169-3e71-40a2-a0dc-e52b89db9f42" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.31:15021/healthz/ready\": dial tcp 10.132.0.31:15021: connect: connection refused"
Sep 12 00:30:25.588152 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:30:25.588108    2571 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-78bb78ff78-svr8q container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.31:15021/healthz/ready\": dial tcp 10.132.0.31:15021: connect: connection refused" start-of-body=
Sep 12 00:30:25.588686 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:30:25.588175    2571 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-svr8q" podUID="06461169-3e71-40a2-a0dc-e52b89db9f42" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.31:15021/healthz/ready\": dial tcp 10.132.0.31:15021: connect: connection refused"
Sep 12 00:30:26.588607 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:30:26.588556    2571 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-78bb78ff78-svr8q container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.31:15021/healthz/ready\": dial tcp 10.132.0.31:15021: connect: connection refused" start-of-body=
Sep 12 00:30:26.589174 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:30:26.589141    2571 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-svr8q" podUID="06461169-3e71-40a2-a0dc-e52b89db9f42" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.31:15021/healthz/ready\": dial tcp 10.132.0.31:15021: connect: connection refused"
Sep 12 00:30:27.507516 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:30:27.507483    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-59fbc47bf8-p2vwz"
Sep 12 00:30:27.508150 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:30:27.508127    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-5b54b7fbb7-btmpn"
Sep 12 00:30:27.588026 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:30:27.587989    2571 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-78bb78ff78-svr8q container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.31:15021/healthz/ready\": dial tcp 10.132.0.31:15021: connect: connection refused" start-of-body=
Sep 12 00:30:27.588299 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:30:27.588269    2571 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-svr8q" podUID="06461169-3e71-40a2-a0dc-e52b89db9f42" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.31:15021/healthz/ready\": dial tcp 10.132.0.31:15021: connect: connection refused"
Sep 12 00:30:27.996782 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:30:27.996744    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-svr8q"
Sep 12 00:30:28.591540 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:30:28.591509    2571 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-svr8q"
Sep 12 00:30:28.592347 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:30:28.592325    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-svr8q"
Sep 12 00:30:39.428732 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:30:39.428700    2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-distinct-simulated-kserve-59975d69f9-qggd6"]
Sep 12 00:30:39.431510 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:30:39.431495    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-qggd6"
Sep 12 00:30:39.434984 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:30:39.434961    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\""
Sep 12 00:30:39.435117 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:30:39.435017    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\""
Sep 12 00:30:39.435117 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:30:39.435029    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-rjscr\""
Sep 12 00:30:39.435117 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:30:39.435045    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-distinct-simulated-kserve-self-signed-certs\""
Sep 12 00:30:39.439945 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:30:39.439922    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-simulated-kserve-59975d69f9-qggd6"]
Sep 12 00:30:39.533904 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:30:39.533872    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/1dc7f7b3-d9b5-40af-9e34-c8c8af685c05-model-cache\") pod \"e2e-distinct-simulated-kserve-59975d69f9-qggd6\" (UID: \"1dc7f7b3-d9b5-40af-9e34-c8c8af685c05\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-qggd6"
Sep 12 00:30:39.534059 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:30:39.533919    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1dc7f7b3-d9b5-40af-9e34-c8c8af685c05-tmp-dir\") pod \"e2e-distinct-simulated-kserve-59975d69f9-qggd6\" (UID: \"1dc7f7b3-d9b5-40af-9e34-c8c8af685c05\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-qggd6"
Sep 12 00:30:39.534059 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:30:39.534010    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/1dc7f7b3-d9b5-40af-9e34-c8c8af685c05-dshm\") pod \"e2e-distinct-simulated-kserve-59975d69f9-qggd6\" (UID: \"1dc7f7b3-d9b5-40af-9e34-c8c8af685c05\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-qggd6"
Sep 12 00:30:39.534059 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:30:39.534036    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6fr8j\" (UniqueName: \"kubernetes.io/projected/1dc7f7b3-d9b5-40af-9e34-c8c8af685c05-kube-api-access-6fr8j\") pod \"e2e-distinct-simulated-kserve-59975d69f9-qggd6\" (UID: \"1dc7f7b3-d9b5-40af-9e34-c8c8af685c05\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-qggd6"
Sep 12 00:30:39.534179 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:30:39.534060    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/1dc7f7b3-d9b5-40af-9e34-c8c8af685c05-home\") pod \"e2e-distinct-simulated-kserve-59975d69f9-qggd6\" (UID: \"1dc7f7b3-d9b5-40af-9e34-c8c8af685c05\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-qggd6"
Sep 12 00:30:39.534179 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:30:39.534088    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/1dc7f7b3-d9b5-40af-9e34-c8c8af685c05-tls-certs\") pod \"e2e-distinct-simulated-kserve-59975d69f9-qggd6\" (UID: \"1dc7f7b3-d9b5-40af-9e34-c8c8af685c05\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-qggd6"
Sep 12 00:30:39.641929 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:30:39.636134    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/1dc7f7b3-d9b5-40af-9e34-c8c8af685c05-model-cache\") pod \"e2e-distinct-simulated-kserve-59975d69f9-qggd6\" (UID: \"1dc7f7b3-d9b5-40af-9e34-c8c8af685c05\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-qggd6"
Sep 12 00:30:39.641929 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:30:39.636211    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1dc7f7b3-d9b5-40af-9e34-c8c8af685c05-tmp-dir\") pod \"e2e-distinct-simulated-kserve-59975d69f9-qggd6\" (UID: \"1dc7f7b3-d9b5-40af-9e34-c8c8af685c05\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-qggd6"
Sep 12 00:30:39.641929 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:30:39.636299    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/1dc7f7b3-d9b5-40af-9e34-c8c8af685c05-dshm\") pod \"e2e-distinct-simulated-kserve-59975d69f9-qggd6\" (UID: \"1dc7f7b3-d9b5-40af-9e34-c8c8af685c05\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-qggd6"
Sep 12 00:30:39.641929 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:30:39.636329    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6fr8j\" (UniqueName: \"kubernetes.io/projected/1dc7f7b3-d9b5-40af-9e34-c8c8af685c05-kube-api-access-6fr8j\") pod \"e2e-distinct-simulated-kserve-59975d69f9-qggd6\" (UID: \"1dc7f7b3-d9b5-40af-9e34-c8c8af685c05\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-qggd6"
Sep 12 00:30:39.641929 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:30:39.636364    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/1dc7f7b3-d9b5-40af-9e34-c8c8af685c05-home\") pod \"e2e-distinct-simulated-kserve-59975d69f9-qggd6\" (UID: \"1dc7f7b3-d9b5-40af-9e34-c8c8af685c05\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-qggd6"
Sep 12 00:30:39.641929 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:30:39.636394    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/1dc7f7b3-d9b5-40af-9e34-c8c8af685c05-tls-certs\") pod \"e2e-distinct-simulated-kserve-59975d69f9-qggd6\" (UID: \"1dc7f7b3-d9b5-40af-9e34-c8c8af685c05\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-qggd6"
Sep 12 00:30:39.641929 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:30:39.639631    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/1dc7f7b3-d9b5-40af-9e34-c8c8af685c05-tls-certs\") pod \"e2e-distinct-simulated-kserve-59975d69f9-qggd6\" (UID: \"1dc7f7b3-d9b5-40af-9e34-c8c8af685c05\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-qggd6"
Sep 12 00:30:39.641929 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:30:39.639937    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/1dc7f7b3-d9b5-40af-9e34-c8c8af685c05-model-cache\") pod \"e2e-distinct-simulated-kserve-59975d69f9-qggd6\" (UID: \"1dc7f7b3-d9b5-40af-9e34-c8c8af685c05\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-qggd6"
Sep 12 00:30:39.641929 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:30:39.640167    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1dc7f7b3-d9b5-40af-9e34-c8c8af685c05-tmp-dir\") pod \"e2e-distinct-simulated-kserve-59975d69f9-qggd6\" (UID: \"1dc7f7b3-d9b5-40af-9e34-c8c8af685c05\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-qggd6"
Sep 12 00:30:39.642503 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:30:39.641981    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/1dc7f7b3-d9b5-40af-9e34-c8c8af685c05-home\") pod \"e2e-distinct-simulated-kserve-59975d69f9-qggd6\" (UID: \"1dc7f7b3-d9b5-40af-9e34-c8c8af685c05\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-qggd6"
Sep 12 00:30:39.643000 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:30:39.642979    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/1dc7f7b3-d9b5-40af-9e34-c8c8af685c05-dshm\") pod \"e2e-distinct-simulated-kserve-59975d69f9-qggd6\" (UID: \"1dc7f7b3-d9b5-40af-9e34-c8c8af685c05\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-qggd6"
Sep 12 00:30:39.649074 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:30:39.649051    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6fr8j\" (UniqueName: \"kubernetes.io/projected/1dc7f7b3-d9b5-40af-9e34-c8c8af685c05-kube-api-access-6fr8j\") pod \"e2e-distinct-simulated-kserve-59975d69f9-qggd6\" (UID: \"1dc7f7b3-d9b5-40af-9e34-c8c8af685c05\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-qggd6"
Sep 12 00:30:39.743013 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:30:39.742989    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-qggd6"
Sep 12 00:30:40.012238 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:30:40.012202    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-simulated-kserve-59975d69f9-qggd6"]
Sep 12 00:30:40.014315 ip-10-0-141-128 kubenswrapper[2571]: W0912 00:30:40.014289    2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1dc7f7b3_d9b5_40af_9e34_c8c8af685c05.slice/crio-9019bd93ce6973fcdc51ec706cd8614f4fc2d9bf59a211c4c02a39e066a35adb WatchSource:0}: Error finding container 9019bd93ce6973fcdc51ec706cd8614f4fc2d9bf59a211c4c02a39e066a35adb: Status 404 returned error can't find the container with id 9019bd93ce6973fcdc51ec706cd8614f4fc2d9bf59a211c4c02a39e066a35adb
Sep 12 00:30:40.628136 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:30:40.628083    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-qggd6" event={"ID":"1dc7f7b3-d9b5-40af-9e34-c8c8af685c05","Type":"ContainerStarted","Data":"9019bd93ce6973fcdc51ec706cd8614f4fc2d9bf59a211c4c02a39e066a35adb"}
Sep 12 00:30:41.634037 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:30:41.634003    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-qggd6" event={"ID":"1dc7f7b3-d9b5-40af-9e34-c8c8af685c05","Type":"ContainerStarted","Data":"bac1ed0c66d7a6a2ebcfea4526d8b70381ede56ee447a773374449feaff744f3"}
Sep 12 00:30:41.649920 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:30:41.649864    2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-qggd6" podStartSLOduration=1.6745577630000001 podStartE2EDuration="2.64984955s" totalImagesPullingTime="975.291787ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-12 00:30:39 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-12 00:30:41.648739226 +0000 UTC m=+852.920379733" watchObservedRunningTime="2026-09-12 00:30:41.64984955 +0000 UTC m=+852.921490058"
Sep 12 00:30:49.743754 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:30:49.743722    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-qggd6"
Sep 12 00:30:49.744298 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:30:49.743797    2571 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-qggd6"
Sep 12 00:30:49.756167 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:30:49.756142    2571 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-qggd6"
Sep 12 00:30:50.686778 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:30:50.686752    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-qggd6"
Sep 12 00:33:15.396737 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:33:15.396705    2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w4-139288-openshift-default-8557bc4874-7zvml"]
Sep 12 00:33:15.400451 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:33:15.400430    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-139288-openshift-default-8557bc4874-7zvml"
Sep 12 00:33:15.403073 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:33:15.403052    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-worker-w4-139288-openshift-default-dockercfg-ddf9d\""
Sep 12 00:33:15.409832 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:33:15.409804    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w4-139288-openshift-default-8557bc4874-7zvml"]
Sep 12 00:33:15.469635 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:33:15.466041    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d7e353f5-8ba3-41a4-bd94-5e2c0cf9fa46-istio-envoy\") pod \"e2e-worker-w4-139288-openshift-default-8557bc4874-7zvml\" (UID: \"d7e353f5-8ba3-41a4-bd94-5e2c0cf9fa46\") " pod="openshift-ingress/e2e-worker-w4-139288-openshift-default-8557bc4874-7zvml"
Sep 12 00:33:15.469635 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:33:15.466111    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d7e353f5-8ba3-41a4-bd94-5e2c0cf9fa46-istio-podinfo\") pod \"e2e-worker-w4-139288-openshift-default-8557bc4874-7zvml\" (UID: \"d7e353f5-8ba3-41a4-bd94-5e2c0cf9fa46\") " pod="openshift-ingress/e2e-worker-w4-139288-openshift-default-8557bc4874-7zvml"
Sep 12 00:33:15.469635 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:33:15.466143    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gxw5z\" (UniqueName: \"kubernetes.io/projected/d7e353f5-8ba3-41a4-bd94-5e2c0cf9fa46-kube-api-access-gxw5z\") pod \"e2e-worker-w4-139288-openshift-default-8557bc4874-7zvml\" (UID: \"d7e353f5-8ba3-41a4-bd94-5e2c0cf9fa46\") " pod="openshift-ingress/e2e-worker-w4-139288-openshift-default-8557bc4874-7zvml"
Sep 12 00:33:15.469635 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:33:15.466178    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d7e353f5-8ba3-41a4-bd94-5e2c0cf9fa46-istio-data\") pod \"e2e-worker-w4-139288-openshift-default-8557bc4874-7zvml\" (UID: \"d7e353f5-8ba3-41a4-bd94-5e2c0cf9fa46\") " pod="openshift-ingress/e2e-worker-w4-139288-openshift-default-8557bc4874-7zvml"
Sep 12 00:33:15.469635 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:33:15.466205    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d7e353f5-8ba3-41a4-bd94-5e2c0cf9fa46-workload-certs\") pod \"e2e-worker-w4-139288-openshift-default-8557bc4874-7zvml\" (UID: \"d7e353f5-8ba3-41a4-bd94-5e2c0cf9fa46\") " pod="openshift-ingress/e2e-worker-w4-139288-openshift-default-8557bc4874-7zvml"
Sep 12 00:33:15.469635 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:33:15.466288    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d7e353f5-8ba3-41a4-bd94-5e2c0cf9fa46-istio-token\") pod \"e2e-worker-w4-139288-openshift-default-8557bc4874-7zvml\" (UID: \"d7e353f5-8ba3-41a4-bd94-5e2c0cf9fa46\") " pod="openshift-ingress/e2e-worker-w4-139288-openshift-default-8557bc4874-7zvml"
Sep 12 00:33:15.469635 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:33:15.466327    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d7e353f5-8ba3-41a4-bd94-5e2c0cf9fa46-workload-socket\") pod \"e2e-worker-w4-139288-openshift-default-8557bc4874-7zvml\" (UID: \"d7e353f5-8ba3-41a4-bd94-5e2c0cf9fa46\") " pod="openshift-ingress/e2e-worker-w4-139288-openshift-default-8557bc4874-7zvml"
Sep 12 00:33:15.469635 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:33:15.466353    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d7e353f5-8ba3-41a4-bd94-5e2c0cf9fa46-istiod-ca-cert\") pod \"e2e-worker-w4-139288-openshift-default-8557bc4874-7zvml\" (UID: \"d7e353f5-8ba3-41a4-bd94-5e2c0cf9fa46\") " pod="openshift-ingress/e2e-worker-w4-139288-openshift-default-8557bc4874-7zvml"
Sep 12 00:33:15.469635 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:33:15.466382    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d7e353f5-8ba3-41a4-bd94-5e2c0cf9fa46-credential-socket\") pod \"e2e-worker-w4-139288-openshift-default-8557bc4874-7zvml\" (UID: \"d7e353f5-8ba3-41a4-bd94-5e2c0cf9fa46\") " pod="openshift-ingress/e2e-worker-w4-139288-openshift-default-8557bc4874-7zvml"
Sep 12 00:33:15.470653 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:33:15.470625    2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-inf-a-a4df66-openshift-default-7cc987766d-5t9vl"]
Sep 12 00:33:15.474669 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:33:15.474621    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-a4df66-openshift-default-7cc987766d-5t9vl"
Sep 12 00:33:15.477316 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:33:15.477294    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-inf-a-a4df66-openshift-default-dockercfg-qqszh\""
Sep 12 00:33:15.496965 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:33:15.496936    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-a-a4df66-openshift-default-7cc987766d-5t9vl"]
Sep 12 00:33:15.568403 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:33:15.568346    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d7e353f5-8ba3-41a4-bd94-5e2c0cf9fa46-credential-socket\") pod \"e2e-worker-w4-139288-openshift-default-8557bc4874-7zvml\" (UID: \"d7e353f5-8ba3-41a4-bd94-5e2c0cf9fa46\") " pod="openshift-ingress/e2e-worker-w4-139288-openshift-default-8557bc4874-7zvml"
Sep 12 00:33:15.568962 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:33:15.568935    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d7e353f5-8ba3-41a4-bd94-5e2c0cf9fa46-credential-socket\") pod \"e2e-worker-w4-139288-openshift-default-8557bc4874-7zvml\" (UID: \"d7e353f5-8ba3-41a4-bd94-5e2c0cf9fa46\") " pod="openshift-ingress/e2e-worker-w4-139288-openshift-default-8557bc4874-7zvml"
Sep 12 00:33:15.569076 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:33:15.569006    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7d867c39-4af2-46d5-9388-95af564c321e-istio-data\") pod \"e2e-inf-a-a4df66-openshift-default-7cc987766d-5t9vl\" (UID: \"7d867c39-4af2-46d5-9388-95af564c321e\") " pod="openshift-ingress/e2e-inf-a-a4df66-openshift-default-7cc987766d-5t9vl"
Sep 12 00:33:15.569133 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:33:15.569086    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vgtd9\" (UniqueName: \"kubernetes.io/projected/7d867c39-4af2-46d5-9388-95af564c321e-kube-api-access-vgtd9\") pod \"e2e-inf-a-a4df66-openshift-default-7cc987766d-5t9vl\" (UID: \"7d867c39-4af2-46d5-9388-95af564c321e\") " pod="openshift-ingress/e2e-inf-a-a4df66-openshift-default-7cc987766d-5t9vl"
Sep 12 00:33:15.569186 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:33:15.569140    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7d867c39-4af2-46d5-9388-95af564c321e-istio-token\") pod \"e2e-inf-a-a4df66-openshift-default-7cc987766d-5t9vl\" (UID: \"7d867c39-4af2-46d5-9388-95af564c321e\") " pod="openshift-ingress/e2e-inf-a-a4df66-openshift-default-7cc987766d-5t9vl"
Sep 12 00:33:15.569186 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:33:15.569172    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d7e353f5-8ba3-41a4-bd94-5e2c0cf9fa46-workload-certs\") pod \"e2e-worker-w4-139288-openshift-default-8557bc4874-7zvml\" (UID: \"d7e353f5-8ba3-41a4-bd94-5e2c0cf9fa46\") " pod="openshift-ingress/e2e-worker-w4-139288-openshift-default-8557bc4874-7zvml"
Sep 12 00:33:15.569288 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:33:15.569215    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d7e353f5-8ba3-41a4-bd94-5e2c0cf9fa46-istio-data\") pod \"e2e-worker-w4-139288-openshift-default-8557bc4874-7zvml\" (UID: \"d7e353f5-8ba3-41a4-bd94-5e2c0cf9fa46\") " pod="openshift-ingress/e2e-worker-w4-139288-openshift-default-8557bc4874-7zvml"
Sep 12 00:33:15.569342 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:33:15.569296    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7d867c39-4af2-46d5-9388-95af564c321e-credential-socket\") pod \"e2e-inf-a-a4df66-openshift-default-7cc987766d-5t9vl\" (UID: \"7d867c39-4af2-46d5-9388-95af564c321e\") " pod="openshift-ingress/e2e-inf-a-a4df66-openshift-default-7cc987766d-5t9vl"
Sep 12 00:33:15.569342 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:33:15.569331    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d7e353f5-8ba3-41a4-bd94-5e2c0cf9fa46-istio-envoy\") pod \"e2e-worker-w4-139288-openshift-default-8557bc4874-7zvml\" (UID: \"d7e353f5-8ba3-41a4-bd94-5e2c0cf9fa46\") " pod="openshift-ingress/e2e-worker-w4-139288-openshift-default-8557bc4874-7zvml"
Sep 12 00:33:15.569573 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:33:15.569552    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7d867c39-4af2-46d5-9388-95af564c321e-istio-podinfo\") pod \"e2e-inf-a-a4df66-openshift-default-7cc987766d-5t9vl\" (UID: \"7d867c39-4af2-46d5-9388-95af564c321e\") " pod="openshift-ingress/e2e-inf-a-a4df66-openshift-default-7cc987766d-5t9vl"
Sep 12 00:33:15.569951 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:33:15.569925    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d7e353f5-8ba3-41a4-bd94-5e2c0cf9fa46-workload-certs\") pod \"e2e-worker-w4-139288-openshift-default-8557bc4874-7zvml\" (UID: \"d7e353f5-8ba3-41a4-bd94-5e2c0cf9fa46\") " pod="openshift-ingress/e2e-worker-w4-139288-openshift-default-8557bc4874-7zvml"
Sep 12 00:33:15.570030 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:33:15.570010    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d7e353f5-8ba3-41a4-bd94-5e2c0cf9fa46-istio-podinfo\") pod \"e2e-worker-w4-139288-openshift-default-8557bc4874-7zvml\" (UID: \"d7e353f5-8ba3-41a4-bd94-5e2c0cf9fa46\") " pod="openshift-ingress/e2e-worker-w4-139288-openshift-default-8557bc4874-7zvml"
Sep 12 00:33:15.570304 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:33:15.570282    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d7e353f5-8ba3-41a4-bd94-5e2c0cf9fa46-istio-data\") pod \"e2e-worker-w4-139288-openshift-default-8557bc4874-7zvml\" (UID: \"d7e353f5-8ba3-41a4-bd94-5e2c0cf9fa46\") " pod="openshift-ingress/e2e-worker-w4-139288-openshift-default-8557bc4874-7zvml"
Sep 12 00:33:15.570550 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:33:15.570530    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gxw5z\" (UniqueName: \"kubernetes.io/projected/d7e353f5-8ba3-41a4-bd94-5e2c0cf9fa46-kube-api-access-gxw5z\") pod \"e2e-worker-w4-139288-openshift-default-8557bc4874-7zvml\" (UID: \"d7e353f5-8ba3-41a4-bd94-5e2c0cf9fa46\") " pod="openshift-ingress/e2e-worker-w4-139288-openshift-default-8557bc4874-7zvml"
Sep 12 00:33:15.570628 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:33:15.570605    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7d867c39-4af2-46d5-9388-95af564c321e-istio-envoy\") pod \"e2e-inf-a-a4df66-openshift-default-7cc987766d-5t9vl\" (UID: \"7d867c39-4af2-46d5-9388-95af564c321e\") " pod="openshift-ingress/e2e-inf-a-a4df66-openshift-default-7cc987766d-5t9vl"
Sep 12 00:33:15.570696 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:33:15.570648    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7d867c39-4af2-46d5-9388-95af564c321e-istiod-ca-cert\") pod \"e2e-inf-a-a4df66-openshift-default-7cc987766d-5t9vl\" (UID: \"7d867c39-4af2-46d5-9388-95af564c321e\") " pod="openshift-ingress/e2e-inf-a-a4df66-openshift-default-7cc987766d-5t9vl"
Sep 12 00:33:15.570696 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:33:15.570692    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d7e353f5-8ba3-41a4-bd94-5e2c0cf9fa46-workload-socket\") pod \"e2e-worker-w4-139288-openshift-default-8557bc4874-7zvml\" (UID: \"d7e353f5-8ba3-41a4-bd94-5e2c0cf9fa46\") " pod="openshift-ingress/e2e-worker-w4-139288-openshift-default-8557bc4874-7zvml"
Sep 12 00:33:15.570809 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:33:15.570716    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d7e353f5-8ba3-41a4-bd94-5e2c0cf9fa46-istiod-ca-cert\") pod \"e2e-worker-w4-139288-openshift-default-8557bc4874-7zvml\" (UID: \"d7e353f5-8ba3-41a4-bd94-5e2c0cf9fa46\") " pod="openshift-ingress/e2e-worker-w4-139288-openshift-default-8557bc4874-7zvml"
Sep 12 00:33:15.570809 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:33:15.570766    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7d867c39-4af2-46d5-9388-95af564c321e-workload-socket\") pod \"e2e-inf-a-a4df66-openshift-default-7cc987766d-5t9vl\" (UID: \"7d867c39-4af2-46d5-9388-95af564c321e\") " pod="openshift-ingress/e2e-inf-a-a4df66-openshift-default-7cc987766d-5t9vl"
Sep 12 00:33:15.570911 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:33:15.570820    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7d867c39-4af2-46d5-9388-95af564c321e-workload-certs\") pod \"e2e-inf-a-a4df66-openshift-default-7cc987766d-5t9vl\" (UID: \"7d867c39-4af2-46d5-9388-95af564c321e\") " pod="openshift-ingress/e2e-inf-a-a4df66-openshift-default-7cc987766d-5t9vl"
Sep 12 00:33:15.570911 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:33:15.570850    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d7e353f5-8ba3-41a4-bd94-5e2c0cf9fa46-istio-token\") pod \"e2e-worker-w4-139288-openshift-default-8557bc4874-7zvml\" (UID: \"d7e353f5-8ba3-41a4-bd94-5e2c0cf9fa46\") " pod="openshift-ingress/e2e-worker-w4-139288-openshift-default-8557bc4874-7zvml"
Sep 12 00:33:15.571552 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:33:15.571527    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d7e353f5-8ba3-41a4-bd94-5e2c0cf9fa46-workload-socket\") pod \"e2e-worker-w4-139288-openshift-default-8557bc4874-7zvml\" (UID: \"d7e353f5-8ba3-41a4-bd94-5e2c0cf9fa46\") " pod="openshift-ingress/e2e-worker-w4-139288-openshift-default-8557bc4874-7zvml"
Sep 12 00:33:15.571944 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:33:15.571906    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d7e353f5-8ba3-41a4-bd94-5e2c0cf9fa46-istiod-ca-cert\") pod \"e2e-worker-w4-139288-openshift-default-8557bc4874-7zvml\" (UID: \"d7e353f5-8ba3-41a4-bd94-5e2c0cf9fa46\") " pod="openshift-ingress/e2e-worker-w4-139288-openshift-default-8557bc4874-7zvml"
Sep 12 00:33:15.572423 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:33:15.572400    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d7e353f5-8ba3-41a4-bd94-5e2c0cf9fa46-istio-envoy\") pod \"e2e-worker-w4-139288-openshift-default-8557bc4874-7zvml\" (UID: \"d7e353f5-8ba3-41a4-bd94-5e2c0cf9fa46\") " pod="openshift-ingress/e2e-worker-w4-139288-openshift-default-8557bc4874-7zvml"
Sep 12 00:33:15.573718 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:33:15.573697    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d7e353f5-8ba3-41a4-bd94-5e2c0cf9fa46-istio-podinfo\") pod \"e2e-worker-w4-139288-openshift-default-8557bc4874-7zvml\" (UID: \"d7e353f5-8ba3-41a4-bd94-5e2c0cf9fa46\") " pod="openshift-ingress/e2e-worker-w4-139288-openshift-default-8557bc4874-7zvml"
Sep 12 00:33:15.579280 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:33:15.579258    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gxw5z\" (UniqueName: \"kubernetes.io/projected/d7e353f5-8ba3-41a4-bd94-5e2c0cf9fa46-kube-api-access-gxw5z\") pod \"e2e-worker-w4-139288-openshift-default-8557bc4874-7zvml\" (UID: \"d7e353f5-8ba3-41a4-bd94-5e2c0cf9fa46\") " pod="openshift-ingress/e2e-worker-w4-139288-openshift-default-8557bc4874-7zvml"
Sep 12 00:33:15.584610 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:33:15.584559    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d7e353f5-8ba3-41a4-bd94-5e2c0cf9fa46-istio-token\") pod \"e2e-worker-w4-139288-openshift-default-8557bc4874-7zvml\" (UID: \"d7e353f5-8ba3-41a4-bd94-5e2c0cf9fa46\") " pod="openshift-ingress/e2e-worker-w4-139288-openshift-default-8557bc4874-7zvml"
Sep 12 00:33:15.672016 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:33:15.671934    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7d867c39-4af2-46d5-9388-95af564c321e-istio-data\") pod \"e2e-inf-a-a4df66-openshift-default-7cc987766d-5t9vl\" (UID: \"7d867c39-4af2-46d5-9388-95af564c321e\") " pod="openshift-ingress/e2e-inf-a-a4df66-openshift-default-7cc987766d-5t9vl"
Sep 12 00:33:15.672016 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:33:15.671992    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vgtd9\" (UniqueName: \"kubernetes.io/projected/7d867c39-4af2-46d5-9388-95af564c321e-kube-api-access-vgtd9\") pod \"e2e-inf-a-a4df66-openshift-default-7cc987766d-5t9vl\" (UID: \"7d867c39-4af2-46d5-9388-95af564c321e\") " pod="openshift-ingress/e2e-inf-a-a4df66-openshift-default-7cc987766d-5t9vl"
Sep 12 00:33:15.672235 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:33:15.672029    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7d867c39-4af2-46d5-9388-95af564c321e-istio-token\") pod \"e2e-inf-a-a4df66-openshift-default-7cc987766d-5t9vl\" (UID: \"7d867c39-4af2-46d5-9388-95af564c321e\") " pod="openshift-ingress/e2e-inf-a-a4df66-openshift-default-7cc987766d-5t9vl"
Sep 12 00:33:15.672235 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:33:15.672092    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7d867c39-4af2-46d5-9388-95af564c321e-credential-socket\") pod \"e2e-inf-a-a4df66-openshift-default-7cc987766d-5t9vl\" (UID: \"7d867c39-4af2-46d5-9388-95af564c321e\") " pod="openshift-ingress/e2e-inf-a-a4df66-openshift-default-7cc987766d-5t9vl"
Sep 12 00:33:15.672235 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:33:15.672139    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7d867c39-4af2-46d5-9388-95af564c321e-istio-podinfo\") pod \"e2e-inf-a-a4df66-openshift-default-7cc987766d-5t9vl\" (UID: \"7d867c39-4af2-46d5-9388-95af564c321e\") " pod="openshift-ingress/e2e-inf-a-a4df66-openshift-default-7cc987766d-5t9vl"
Sep 12 00:33:15.672235 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:33:15.672168    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7d867c39-4af2-46d5-9388-95af564c321e-istio-envoy\") pod \"e2e-inf-a-a4df66-openshift-default-7cc987766d-5t9vl\" (UID: \"7d867c39-4af2-46d5-9388-95af564c321e\") " pod="openshift-ingress/e2e-inf-a-a4df66-openshift-default-7cc987766d-5t9vl"
Sep 12 00:33:15.672235 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:33:15.672195    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7d867c39-4af2-46d5-9388-95af564c321e-istiod-ca-cert\") pod \"e2e-inf-a-a4df66-openshift-default-7cc987766d-5t9vl\" (UID: \"7d867c39-4af2-46d5-9388-95af564c321e\") " pod="openshift-ingress/e2e-inf-a-a4df66-openshift-default-7cc987766d-5t9vl"
Sep 12 00:33:15.672235 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:33:15.672233    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7d867c39-4af2-46d5-9388-95af564c321e-workload-socket\") pod \"e2e-inf-a-a4df66-openshift-default-7cc987766d-5t9vl\" (UID: \"7d867c39-4af2-46d5-9388-95af564c321e\") " pod="openshift-ingress/e2e-inf-a-a4df66-openshift-default-7cc987766d-5t9vl"
Sep 12 00:33:15.672549 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:33:15.672264    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7d867c39-4af2-46d5-9388-95af564c321e-workload-certs\") pod \"e2e-inf-a-a4df66-openshift-default-7cc987766d-5t9vl\" (UID: \"7d867c39-4af2-46d5-9388-95af564c321e\") " pod="openshift-ingress/e2e-inf-a-a4df66-openshift-default-7cc987766d-5t9vl"
Sep 12 00:33:15.672549 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:33:15.672364    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7d867c39-4af2-46d5-9388-95af564c321e-istio-data\") pod \"e2e-inf-a-a4df66-openshift-default-7cc987766d-5t9vl\" (UID: \"7d867c39-4af2-46d5-9388-95af564c321e\") " pod="openshift-ingress/e2e-inf-a-a4df66-openshift-default-7cc987766d-5t9vl"
Sep 12 00:33:15.672682 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:33:15.672637    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7d867c39-4af2-46d5-9388-95af564c321e-workload-certs\") pod \"e2e-inf-a-a4df66-openshift-default-7cc987766d-5t9vl\" (UID: \"7d867c39-4af2-46d5-9388-95af564c321e\") " pod="openshift-ingress/e2e-inf-a-a4df66-openshift-default-7cc987766d-5t9vl"
Sep 12 00:33:15.672740 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:33:15.672693    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7d867c39-4af2-46d5-9388-95af564c321e-credential-socket\") pod \"e2e-inf-a-a4df66-openshift-default-7cc987766d-5t9vl\" (UID: \"7d867c39-4af2-46d5-9388-95af564c321e\") " pod="openshift-ingress/e2e-inf-a-a4df66-openshift-default-7cc987766d-5t9vl"
Sep 12 00:33:15.672894 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:33:15.672873    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7d867c39-4af2-46d5-9388-95af564c321e-workload-socket\") pod \"e2e-inf-a-a4df66-openshift-default-7cc987766d-5t9vl\" (UID: \"7d867c39-4af2-46d5-9388-95af564c321e\") " pod="openshift-ingress/e2e-inf-a-a4df66-openshift-default-7cc987766d-5t9vl"
Sep 12 00:33:15.673062 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:33:15.673042    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7d867c39-4af2-46d5-9388-95af564c321e-istiod-ca-cert\") pod \"e2e-inf-a-a4df66-openshift-default-7cc987766d-5t9vl\" (UID: \"7d867c39-4af2-46d5-9388-95af564c321e\") " pod="openshift-ingress/e2e-inf-a-a4df66-openshift-default-7cc987766d-5t9vl"
Sep 12 00:33:15.674655 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:33:15.674625    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7d867c39-4af2-46d5-9388-95af564c321e-istio-envoy\") pod \"e2e-inf-a-a4df66-openshift-default-7cc987766d-5t9vl\" (UID: \"7d867c39-4af2-46d5-9388-95af564c321e\") " pod="openshift-ingress/e2e-inf-a-a4df66-openshift-default-7cc987766d-5t9vl"
Sep 12 00:33:15.675190 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:33:15.675172    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7d867c39-4af2-46d5-9388-95af564c321e-istio-podinfo\") pod \"e2e-inf-a-a4df66-openshift-default-7cc987766d-5t9vl\" (UID: \"7d867c39-4af2-46d5-9388-95af564c321e\") " pod="openshift-ingress/e2e-inf-a-a4df66-openshift-default-7cc987766d-5t9vl"
Sep 12 00:33:15.680928 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:33:15.680906    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vgtd9\" (UniqueName: \"kubernetes.io/projected/7d867c39-4af2-46d5-9388-95af564c321e-kube-api-access-vgtd9\") pod \"e2e-inf-a-a4df66-openshift-default-7cc987766d-5t9vl\" (UID: \"7d867c39-4af2-46d5-9388-95af564c321e\") " pod="openshift-ingress/e2e-inf-a-a4df66-openshift-default-7cc987766d-5t9vl"
Sep 12 00:33:15.681105 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:33:15.681085    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7d867c39-4af2-46d5-9388-95af564c321e-istio-token\") pod \"e2e-inf-a-a4df66-openshift-default-7cc987766d-5t9vl\" (UID: \"7d867c39-4af2-46d5-9388-95af564c321e\") " pod="openshift-ingress/e2e-inf-a-a4df66-openshift-default-7cc987766d-5t9vl"
Sep 12 00:33:15.715480 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:33:15.715320    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-139288-openshift-default-8557bc4874-7zvml"
Sep 12 00:33:15.797709 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:33:15.797647    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-a4df66-openshift-default-7cc987766d-5t9vl"
Sep 12 00:33:16.001901 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:33:16.001805    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w4-139288-openshift-default-8557bc4874-7zvml"]
Sep 12 00:33:16.004581 ip-10-0-141-128 kubenswrapper[2571]: W0912 00:33:16.004556    2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd7e353f5_8ba3_41a4_bd94_5e2c0cf9fa46.slice/crio-f4a2a9b6703dea3eda8f4219e98a57264c4f13d1cae654f2a6c65d8f8005ae2a WatchSource:0}: Error finding container f4a2a9b6703dea3eda8f4219e98a57264c4f13d1cae654f2a6c65d8f8005ae2a: Status 404 returned error can't find the container with id f4a2a9b6703dea3eda8f4219e98a57264c4f13d1cae654f2a6c65d8f8005ae2a
Sep 12 00:33:16.006693 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:33:16.006663    2571 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236196Ki","pods":"250"}
Sep 12 00:33:16.006754 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:33:16.006726    2571 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236196Ki","pods":"250"}
Sep 12 00:33:16.006792 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:33:16.006753    2571 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236196Ki","pods":"250"}
Sep 12 00:33:16.075354 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:33:16.075333    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-a-a4df66-openshift-default-7cc987766d-5t9vl"]
Sep 12 00:33:16.077191 ip-10-0-141-128 kubenswrapper[2571]: W0912 00:33:16.077165    2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7d867c39_4af2_46d5_9388_95af564c321e.slice/crio-234239077e35f21b559fa266c6c98c838e35b6550fd1cbd28b551479203694d8 WatchSource:0}: Error finding container 234239077e35f21b559fa266c6c98c838e35b6550fd1cbd28b551479203694d8: Status 404 returned error can't find the container with id 234239077e35f21b559fa266c6c98c838e35b6550fd1cbd28b551479203694d8
Sep 12 00:33:16.079822 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:33:16.079784    2571 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236196Ki","pods":"250"}
Sep 12 00:33:16.079920 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:33:16.079867    2571 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236196Ki","pods":"250"}
Sep 12 00:33:16.079920 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:33:16.079910    2571 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236196Ki","pods":"250"}
Sep 12 00:33:16.364338 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:33:16.364222    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-a4df66-openshift-default-7cc987766d-5t9vl" event={"ID":"7d867c39-4af2-46d5-9388-95af564c321e","Type":"ContainerStarted","Data":"32d161283eb48801c85c052c2573fdbcf60768b47ff0dda2b6f6622f93a51e7e"}
Sep 12 00:33:16.364338 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:33:16.364293    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-a4df66-openshift-default-7cc987766d-5t9vl" event={"ID":"7d867c39-4af2-46d5-9388-95af564c321e","Type":"ContainerStarted","Data":"234239077e35f21b559fa266c6c98c838e35b6550fd1cbd28b551479203694d8"}
Sep 12 00:33:16.366440 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:33:16.366410    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-139288-openshift-default-8557bc4874-7zvml" event={"ID":"d7e353f5-8ba3-41a4-bd94-5e2c0cf9fa46","Type":"ContainerStarted","Data":"0016a348a7c14fe82d8c59241a135afe29e55cc506d42caec078fd91aacce125"}
Sep 12 00:33:16.366855 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:33:16.366467    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-139288-openshift-default-8557bc4874-7zvml" event={"ID":"d7e353f5-8ba3-41a4-bd94-5e2c0cf9fa46","Type":"ContainerStarted","Data":"f4a2a9b6703dea3eda8f4219e98a57264c4f13d1cae654f2a6c65d8f8005ae2a"}
Sep 12 00:33:16.382433 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:33:16.382377    2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-inf-a-a4df66-openshift-default-7cc987766d-5t9vl" podStartSLOduration=1.382362457 podStartE2EDuration="1.382362457s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-12 00:33:15 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-12 00:33:16.381490921 +0000 UTC m=+1007.653131432" watchObservedRunningTime="2026-09-12 00:33:16.382362457 +0000 UTC m=+1007.654002965"
Sep 12 00:33:16.399215 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:33:16.399172    2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w4-139288-openshift-default-8557bc4874-7zvml" podStartSLOduration=1.399157019 podStartE2EDuration="1.399157019s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-12 00:33:15 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-12 00:33:16.397904085 +0000 UTC m=+1007.669544606" watchObservedRunningTime="2026-09-12 00:33:16.399157019 +0000 UTC m=+1007.670797528"
Sep 12 00:33:16.716391 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:33:16.716357    2571 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w4-139288-openshift-default-8557bc4874-7zvml"
Sep 12 00:33:16.798393 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:33:16.798364    2571 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-a-a4df66-openshift-default-7cc987766d-5t9vl"
Sep 12 00:33:17.721539 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:33:17.721508    2571 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w4-139288-openshift-default-8557bc4874-7zvml"
Sep 12 00:33:17.802683 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:33:17.802654    2571 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-a-a4df66-openshift-default-7cc987766d-5t9vl"
Sep 12 00:33:18.378184 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:33:18.378152    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w4-139288-openshift-default-8557bc4874-7zvml"
Sep 12 00:33:18.378362 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:33:18.378208    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-a-a4df66-openshift-default-7cc987766d-5t9vl"
Sep 12 00:33:18.379042 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:33:18.379020    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-a-a4df66-openshift-default-7cc987766d-5t9vl"
Sep 12 00:33:18.379132 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:33:18.379071    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w4-139288-openshift-default-8557bc4874-7zvml"
Sep 12 00:33:19.738048 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:33:19.738016    2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-a4df66-7764776995-d48lt"]
Sep 12 00:33:19.742704 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:33:19.742671    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-a4df66-7764776995-d48lt"
Sep 12 00:33:19.744979 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:33:19.744958    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-inf-a-a4df66-serving-cert\""
Sep 12 00:33:19.757196 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:33:19.757034    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-a4df66-7764776995-d48lt"]
Sep 12 00:33:19.813152 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:33:19.813116    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7f5d4ed3-6d89-4f38-a7f1-e7654c6dc733-metrics-certs\") pod \"maas-api-e2e-inf-a-a4df66-7764776995-d48lt\" (UID: \"7f5d4ed3-6d89-4f38-a7f1-e7654c6dc733\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-a4df66-7764776995-d48lt"
Sep 12 00:33:19.813263 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:33:19.813209    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kvkds\" (UniqueName: \"kubernetes.io/projected/7f5d4ed3-6d89-4f38-a7f1-e7654c6dc733-kube-api-access-kvkds\") pod \"maas-api-e2e-inf-a-a4df66-7764776995-d48lt\" (UID: \"7f5d4ed3-6d89-4f38-a7f1-e7654c6dc733\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-a4df66-7764776995-d48lt"
Sep 12 00:33:19.813263 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:33:19.813244    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/7f5d4ed3-6d89-4f38-a7f1-e7654c6dc733-openshift-service-ca\") pod \"maas-api-e2e-inf-a-a4df66-7764776995-d48lt\" (UID: \"7f5d4ed3-6d89-4f38-a7f1-e7654c6dc733\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-a4df66-7764776995-d48lt"
Sep 12 00:33:19.813353 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:33:19.813276    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/7f5d4ed3-6d89-4f38-a7f1-e7654c6dc733-maas-api-tls\") pod \"maas-api-e2e-inf-a-a4df66-7764776995-d48lt\" (UID: \"7f5d4ed3-6d89-4f38-a7f1-e7654c6dc733\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-a4df66-7764776995-d48lt"
Sep 12 00:33:19.914745 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:33:19.914710    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kvkds\" (UniqueName: \"kubernetes.io/projected/7f5d4ed3-6d89-4f38-a7f1-e7654c6dc733-kube-api-access-kvkds\") pod \"maas-api-e2e-inf-a-a4df66-7764776995-d48lt\" (UID: \"7f5d4ed3-6d89-4f38-a7f1-e7654c6dc733\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-a4df66-7764776995-d48lt"
Sep 12 00:33:19.914936 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:33:19.914755    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/7f5d4ed3-6d89-4f38-a7f1-e7654c6dc733-openshift-service-ca\") pod \"maas-api-e2e-inf-a-a4df66-7764776995-d48lt\" (UID: \"7f5d4ed3-6d89-4f38-a7f1-e7654c6dc733\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-a4df66-7764776995-d48lt"
Sep 12 00:33:19.914936 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:33:19.914785    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/7f5d4ed3-6d89-4f38-a7f1-e7654c6dc733-maas-api-tls\") pod \"maas-api-e2e-inf-a-a4df66-7764776995-d48lt\" (UID: \"7f5d4ed3-6d89-4f38-a7f1-e7654c6dc733\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-a4df66-7764776995-d48lt"
Sep 12 00:33:19.914936 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:33:19.914824    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7f5d4ed3-6d89-4f38-a7f1-e7654c6dc733-metrics-certs\") pod \"maas-api-e2e-inf-a-a4df66-7764776995-d48lt\" (UID: \"7f5d4ed3-6d89-4f38-a7f1-e7654c6dc733\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-a4df66-7764776995-d48lt"
Sep 12 00:33:19.915111 ip-10-0-141-128 kubenswrapper[2571]: E0912 00:33:19.914948    2571 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-inf-a-a4df66-serving-cert: secret "maas-api-e2e-inf-a-a4df66-serving-cert" not found
Sep 12 00:33:19.915111 ip-10-0-141-128 kubenswrapper[2571]: E0912 00:33:19.915024    2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7f5d4ed3-6d89-4f38-a7f1-e7654c6dc733-maas-api-tls podName:7f5d4ed3-6d89-4f38-a7f1-e7654c6dc733 nodeName:}" failed. No retries permitted until 2026-09-12 00:33:20.415004501 +0000 UTC m=+1011.686644994 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/7f5d4ed3-6d89-4f38-a7f1-e7654c6dc733-maas-api-tls") pod "maas-api-e2e-inf-a-a4df66-7764776995-d48lt" (UID: "7f5d4ed3-6d89-4f38-a7f1-e7654c6dc733") : secret "maas-api-e2e-inf-a-a4df66-serving-cert" not found
Sep 12 00:33:19.915640 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:33:19.915614    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/7f5d4ed3-6d89-4f38-a7f1-e7654c6dc733-openshift-service-ca\") pod \"maas-api-e2e-inf-a-a4df66-7764776995-d48lt\" (UID: \"7f5d4ed3-6d89-4f38-a7f1-e7654c6dc733\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-a4df66-7764776995-d48lt"
Sep 12 00:33:19.917651 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:33:19.917622    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7f5d4ed3-6d89-4f38-a7f1-e7654c6dc733-metrics-certs\") pod \"maas-api-e2e-inf-a-a4df66-7764776995-d48lt\" (UID: \"7f5d4ed3-6d89-4f38-a7f1-e7654c6dc733\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-a4df66-7764776995-d48lt"
Sep 12 00:33:19.923281 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:33:19.923258    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kvkds\" (UniqueName: \"kubernetes.io/projected/7f5d4ed3-6d89-4f38-a7f1-e7654c6dc733-kube-api-access-kvkds\") pod \"maas-api-e2e-inf-a-a4df66-7764776995-d48lt\" (UID: \"7f5d4ed3-6d89-4f38-a7f1-e7654c6dc733\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-a4df66-7764776995-d48lt"
Sep 12 00:33:20.420230 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:33:20.420194    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/7f5d4ed3-6d89-4f38-a7f1-e7654c6dc733-maas-api-tls\") pod \"maas-api-e2e-inf-a-a4df66-7764776995-d48lt\" (UID: \"7f5d4ed3-6d89-4f38-a7f1-e7654c6dc733\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-a4df66-7764776995-d48lt"
Sep 12 00:33:20.425449 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:33:20.425422    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/7f5d4ed3-6d89-4f38-a7f1-e7654c6dc733-maas-api-tls\") pod \"maas-api-e2e-inf-a-a4df66-7764776995-d48lt\" (UID: \"7f5d4ed3-6d89-4f38-a7f1-e7654c6dc733\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-a4df66-7764776995-d48lt"
Sep 12 00:33:20.568752 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:33:20.568719    2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-a-a4df66-7dd5b76c4-vqffm"]
Sep 12 00:33:20.572519 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:33:20.572501    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-a4df66-7dd5b76c4-vqffm"
Sep 12 00:33:20.576103 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:33:20.576083    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-inf-a-a4df66-dockercfg-kvg8g\""
Sep 12 00:33:20.576213 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:33:20.576126    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-inf-a-a4df66\""
Sep 12 00:33:20.585785 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:33:20.585765    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-a-a4df66-7dd5b76c4-vqffm"]
Sep 12 00:33:20.628150 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:33:20.628114    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9smhz\" (UniqueName: \"kubernetes.io/projected/41f32f06-ebed-400a-9624-1ffe8602eab3-kube-api-access-9smhz\") pod \"payload-pre-processing-e2e-inf-a-a4df66-7dd5b76c4-vqffm\" (UID: \"41f32f06-ebed-400a-9624-1ffe8602eab3\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-a-a4df66-7dd5b76c4-vqffm"
Sep 12 00:33:20.628346 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:33:20.628213    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/41f32f06-ebed-400a-9624-1ffe8602eab3-plugins-config-volume\") pod \"payload-pre-processing-e2e-inf-a-a4df66-7dd5b76c4-vqffm\" (UID: \"41f32f06-ebed-400a-9624-1ffe8602eab3\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-a-a4df66-7dd5b76c4-vqffm"
Sep 12 00:33:20.659889 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:33:20.659859    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-a4df66-7764776995-d48lt"
Sep 12 00:33:20.730416 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:33:20.729313    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9smhz\" (UniqueName: \"kubernetes.io/projected/41f32f06-ebed-400a-9624-1ffe8602eab3-kube-api-access-9smhz\") pod \"payload-pre-processing-e2e-inf-a-a4df66-7dd5b76c4-vqffm\" (UID: \"41f32f06-ebed-400a-9624-1ffe8602eab3\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-a-a4df66-7dd5b76c4-vqffm"
Sep 12 00:33:20.730416 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:33:20.729423    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/41f32f06-ebed-400a-9624-1ffe8602eab3-plugins-config-volume\") pod \"payload-pre-processing-e2e-inf-a-a4df66-7dd5b76c4-vqffm\" (UID: \"41f32f06-ebed-400a-9624-1ffe8602eab3\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-a-a4df66-7dd5b76c4-vqffm"
Sep 12 00:33:20.730416 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:33:20.730386    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/41f32f06-ebed-400a-9624-1ffe8602eab3-plugins-config-volume\") pod \"payload-pre-processing-e2e-inf-a-a4df66-7dd5b76c4-vqffm\" (UID: \"41f32f06-ebed-400a-9624-1ffe8602eab3\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-a-a4df66-7dd5b76c4-vqffm"
Sep 12 00:33:20.738034 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:33:20.737978    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9smhz\" (UniqueName: \"kubernetes.io/projected/41f32f06-ebed-400a-9624-1ffe8602eab3-kube-api-access-9smhz\") pod \"payload-pre-processing-e2e-inf-a-a4df66-7dd5b76c4-vqffm\" (UID: \"41f32f06-ebed-400a-9624-1ffe8602eab3\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-a-a4df66-7dd5b76c4-vqffm"
Sep 12 00:33:20.885706 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:33:20.885676    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-a4df66-7dd5b76c4-vqffm"
Sep 12 00:33:20.943519 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:33:20.943478    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-a4df66-7764776995-d48lt"]
Sep 12 00:33:20.947251 ip-10-0-141-128 kubenswrapper[2571]: W0912 00:33:20.947145    2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7f5d4ed3_6d89_4f38_a7f1_e7654c6dc733.slice/crio-95eb3a29b4b9d6af240fdc41c27c4146cba4161730ffa0cc9c99e32260d854b7 WatchSource:0}: Error finding container 95eb3a29b4b9d6af240fdc41c27c4146cba4161730ffa0cc9c99e32260d854b7: Status 404 returned error can't find the container with id 95eb3a29b4b9d6af240fdc41c27c4146cba4161730ffa0cc9c99e32260d854b7
Sep 12 00:33:20.948931 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:33:20.948821    2571 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 12 00:33:21.168039 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:33:21.168016    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-a-a4df66-7dd5b76c4-vqffm"]
Sep 12 00:33:21.170794 ip-10-0-141-128 kubenswrapper[2571]: W0912 00:33:21.170688    2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod41f32f06_ebed_400a_9624_1ffe8602eab3.slice/crio-0385bb7dec03400dd37af4918bb25d12a951811b4eeb90b27afdea0fbc4de4dc WatchSource:0}: Error finding container 0385bb7dec03400dd37af4918bb25d12a951811b4eeb90b27afdea0fbc4de4dc: Status 404 returned error can't find the container with id 0385bb7dec03400dd37af4918bb25d12a951811b4eeb90b27afdea0fbc4de4dc
Sep 12 00:33:21.395947 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:33:21.395905    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-a4df66-7dd5b76c4-vqffm" event={"ID":"41f32f06-ebed-400a-9624-1ffe8602eab3","Type":"ContainerStarted","Data":"1a63cd7f992bff25bbfa80331be521349b0ad9ca272a8ba90154ddcb57aaf0f6"}
Sep 12 00:33:21.396130 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:33:21.396045    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-a4df66-7dd5b76c4-vqffm"
Sep 12 00:33:21.396130 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:33:21.396086    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-a4df66-7dd5b76c4-vqffm" event={"ID":"41f32f06-ebed-400a-9624-1ffe8602eab3","Type":"ContainerStarted","Data":"0385bb7dec03400dd37af4918bb25d12a951811b4eeb90b27afdea0fbc4de4dc"}
Sep 12 00:33:21.397364 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:33:21.397340    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-a4df66-7764776995-d48lt" event={"ID":"7f5d4ed3-6d89-4f38-a7f1-e7654c6dc733","Type":"ContainerStarted","Data":"95eb3a29b4b9d6af240fdc41c27c4146cba4161730ffa0cc9c99e32260d854b7"}
Sep 12 00:33:21.414494 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:33:21.414423    2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-a4df66-7dd5b76c4-vqffm" podStartSLOduration=1.414409603 podStartE2EDuration="1.414409603s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-12 00:33:20 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-12 00:33:21.413516508 +0000 UTC m=+1012.685157031" watchObservedRunningTime="2026-09-12 00:33:21.414409603 +0000 UTC m=+1012.686050128"
Sep 12 00:33:21.624965 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:33:21.624932    2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w0-3c466c-5485d64cc6-p6d9z"]
Sep 12 00:33:21.628731 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:33:21.628707    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-3c466c-5485d64cc6-p6d9z"
Sep 12 00:33:21.631019 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:33:21.630986    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-worker-w0-3c466c-serving-cert\""
Sep 12 00:33:21.644005 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:33:21.643980    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w0-3c466c-5485d64cc6-p6d9z"]
Sep 12 00:33:21.742404 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:33:21.742321    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zc75b\" (UniqueName: \"kubernetes.io/projected/95707da3-a50f-48dc-8f29-7e3de07bab92-kube-api-access-zc75b\") pod \"maas-api-e2e-worker-w0-3c466c-5485d64cc6-p6d9z\" (UID: \"95707da3-a50f-48dc-8f29-7e3de07bab92\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-3c466c-5485d64cc6-p6d9z"
Sep 12 00:33:21.742677 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:33:21.742654    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/95707da3-a50f-48dc-8f29-7e3de07bab92-openshift-service-ca\") pod \"maas-api-e2e-worker-w0-3c466c-5485d64cc6-p6d9z\" (UID: \"95707da3-a50f-48dc-8f29-7e3de07bab92\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-3c466c-5485d64cc6-p6d9z"
Sep 12 00:33:21.743412 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:33:21.743385    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/95707da3-a50f-48dc-8f29-7e3de07bab92-metrics-certs\") pod \"maas-api-e2e-worker-w0-3c466c-5485d64cc6-p6d9z\" (UID: \"95707da3-a50f-48dc-8f29-7e3de07bab92\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-3c466c-5485d64cc6-p6d9z"
Sep 12 00:33:21.743562 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:33:21.743454    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/95707da3-a50f-48dc-8f29-7e3de07bab92-maas-api-tls\") pod \"maas-api-e2e-worker-w0-3c466c-5485d64cc6-p6d9z\" (UID: \"95707da3-a50f-48dc-8f29-7e3de07bab92\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-3c466c-5485d64cc6-p6d9z"
Sep 12 00:33:21.844781 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:33:21.844747    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/95707da3-a50f-48dc-8f29-7e3de07bab92-metrics-certs\") pod \"maas-api-e2e-worker-w0-3c466c-5485d64cc6-p6d9z\" (UID: \"95707da3-a50f-48dc-8f29-7e3de07bab92\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-3c466c-5485d64cc6-p6d9z"
Sep 12 00:33:21.844955 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:33:21.844805    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/95707da3-a50f-48dc-8f29-7e3de07bab92-maas-api-tls\") pod \"maas-api-e2e-worker-w0-3c466c-5485d64cc6-p6d9z\" (UID: \"95707da3-a50f-48dc-8f29-7e3de07bab92\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-3c466c-5485d64cc6-p6d9z"
Sep 12 00:33:21.844955 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:33:21.844885    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zc75b\" (UniqueName: \"kubernetes.io/projected/95707da3-a50f-48dc-8f29-7e3de07bab92-kube-api-access-zc75b\") pod \"maas-api-e2e-worker-w0-3c466c-5485d64cc6-p6d9z\" (UID: \"95707da3-a50f-48dc-8f29-7e3de07bab92\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-3c466c-5485d64cc6-p6d9z"
Sep 12 00:33:21.844955 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:33:21.844933    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/95707da3-a50f-48dc-8f29-7e3de07bab92-openshift-service-ca\") pod \"maas-api-e2e-worker-w0-3c466c-5485d64cc6-p6d9z\" (UID: \"95707da3-a50f-48dc-8f29-7e3de07bab92\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-3c466c-5485d64cc6-p6d9z"
Sep 12 00:33:21.845126 ip-10-0-141-128 kubenswrapper[2571]: E0912 00:33:21.844948    2571 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-worker-w0-3c466c-serving-cert: secret "maas-api-e2e-worker-w0-3c466c-serving-cert" not found
Sep 12 00:33:21.845126 ip-10-0-141-128 kubenswrapper[2571]: E0912 00:33:21.845021    2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/95707da3-a50f-48dc-8f29-7e3de07bab92-maas-api-tls podName:95707da3-a50f-48dc-8f29-7e3de07bab92 nodeName:}" failed. No retries permitted until 2026-09-12 00:33:22.344999014 +0000 UTC m=+1013.616639512 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/95707da3-a50f-48dc-8f29-7e3de07bab92-maas-api-tls") pod "maas-api-e2e-worker-w0-3c466c-5485d64cc6-p6d9z" (UID: "95707da3-a50f-48dc-8f29-7e3de07bab92") : secret "maas-api-e2e-worker-w0-3c466c-serving-cert" not found
Sep 12 00:33:21.845576 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:33:21.845553    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/95707da3-a50f-48dc-8f29-7e3de07bab92-openshift-service-ca\") pod \"maas-api-e2e-worker-w0-3c466c-5485d64cc6-p6d9z\" (UID: \"95707da3-a50f-48dc-8f29-7e3de07bab92\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-3c466c-5485d64cc6-p6d9z"
Sep 12 00:33:21.847151 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:33:21.847135    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/95707da3-a50f-48dc-8f29-7e3de07bab92-metrics-certs\") pod \"maas-api-e2e-worker-w0-3c466c-5485d64cc6-p6d9z\" (UID: \"95707da3-a50f-48dc-8f29-7e3de07bab92\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-3c466c-5485d64cc6-p6d9z"
Sep 12 00:33:21.858927 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:33:21.858907    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zc75b\" (UniqueName: \"kubernetes.io/projected/95707da3-a50f-48dc-8f29-7e3de07bab92-kube-api-access-zc75b\") pod \"maas-api-e2e-worker-w0-3c466c-5485d64cc6-p6d9z\" (UID: \"95707da3-a50f-48dc-8f29-7e3de07bab92\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-3c466c-5485d64cc6-p6d9z"
Sep 12 00:33:22.349229 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:33:22.349184    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/95707da3-a50f-48dc-8f29-7e3de07bab92-maas-api-tls\") pod \"maas-api-e2e-worker-w0-3c466c-5485d64cc6-p6d9z\" (UID: \"95707da3-a50f-48dc-8f29-7e3de07bab92\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-3c466c-5485d64cc6-p6d9z"
Sep 12 00:33:22.351030 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:33:22.351006    2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w0-3c466c-7445648d54-7tl8k"]
Sep 12 00:33:22.351795 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:33:22.351773    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/95707da3-a50f-48dc-8f29-7e3de07bab92-maas-api-tls\") pod \"maas-api-e2e-worker-w0-3c466c-5485d64cc6-p6d9z\" (UID: \"95707da3-a50f-48dc-8f29-7e3de07bab92\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-3c466c-5485d64cc6-p6d9z"
Sep 12 00:33:22.354789 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:33:22.354775    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-3c466c-7445648d54-7tl8k"
Sep 12 00:33:22.366606 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:33:22.357463    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-worker-w0-3c466c\""
Sep 12 00:33:22.366606 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:33:22.364121    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w0-3c466c-7445648d54-7tl8k"]
Sep 12 00:33:22.404705 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:33:22.404673    2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w0-3c466c-59bdd48b77-sv6rh"]
Sep 12 00:33:22.408996 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:33:22.408764    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-a4df66-7764776995-d48lt" event={"ID":"7f5d4ed3-6d89-4f38-a7f1-e7654c6dc733","Type":"ContainerStarted","Data":"eafdfa3d4bb3bc641b4ba4347f7abd6a05229a04cf1c679daaee5db5699bbba5"}
Sep 12 00:33:22.408996 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:33:22.408808    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-a4df66-7764776995-d48lt"
Sep 12 00:33:22.408996 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:33:22.408912    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-worker-w0-3c466c-59bdd48b77-sv6rh"
Sep 12 00:33:22.411115 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:33:22.411094    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-worker-w0-3c466c-dockercfg-lsrwq\""
Sep 12 00:33:22.415810 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:33:22.415785    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w0-3c466c-59bdd48b77-sv6rh"]
Sep 12 00:33:22.421456 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:33:22.421387    2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-a4df66-7764776995-d48lt" podStartSLOduration=2.898968617 podStartE2EDuration="3.421370518s" totalImagesPullingTime="522.401901ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-12 00:33:19 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-12 00:33:22.420619272 +0000 UTC m=+1013.692259783" watchObservedRunningTime="2026-09-12 00:33:22.421370518 +0000 UTC m=+1013.693011028"
Sep 12 00:33:22.450225 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:33:22.450200    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/66a162de-bb65-4c00-b677-628ef558832f-plugins-config-volume\") pod \"payload-pre-processing-e2e-worker-w0-3c466c-7445648d54-7tl8k\" (UID: \"66a162de-bb65-4c00-b677-628ef558832f\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-3c466c-7445648d54-7tl8k"
Sep 12 00:33:22.450319 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:33:22.450304    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-w2dbt\" (UniqueName: \"kubernetes.io/projected/66a162de-bb65-4c00-b677-628ef558832f-kube-api-access-w2dbt\") pod \"payload-pre-processing-e2e-worker-w0-3c466c-7445648d54-7tl8k\" (UID: \"66a162de-bb65-4c00-b677-628ef558832f\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-3c466c-7445648d54-7tl8k"
Sep 12 00:33:22.541857 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:33:22.541819    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-3c466c-5485d64cc6-p6d9z"
Sep 12 00:33:22.554115 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:33:22.553888    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/66a162de-bb65-4c00-b677-628ef558832f-plugins-config-volume\") pod \"payload-pre-processing-e2e-worker-w0-3c466c-7445648d54-7tl8k\" (UID: \"66a162de-bb65-4c00-b677-628ef558832f\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-3c466c-7445648d54-7tl8k"
Sep 12 00:33:22.554115 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:33:22.554041    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b0fe6c78-af34-4703-a109-628c4a86ea57-plugins-config-volume\") pod \"payload-processing-e2e-worker-w0-3c466c-59bdd48b77-sv6rh\" (UID: \"b0fe6c78-af34-4703-a109-628c4a86ea57\") " pod="openshift-ingress/payload-processing-e2e-worker-w0-3c466c-59bdd48b77-sv6rh"
Sep 12 00:33:22.554115 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:33:22.554097    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bt5zv\" (UniqueName: \"kubernetes.io/projected/b0fe6c78-af34-4703-a109-628c4a86ea57-kube-api-access-bt5zv\") pod \"payload-processing-e2e-worker-w0-3c466c-59bdd48b77-sv6rh\" (UID: \"b0fe6c78-af34-4703-a109-628c4a86ea57\") " pod="openshift-ingress/payload-processing-e2e-worker-w0-3c466c-59bdd48b77-sv6rh"
Sep 12 00:33:22.554284 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:33:22.554219    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-w2dbt\" (UniqueName: \"kubernetes.io/projected/66a162de-bb65-4c00-b677-628ef558832f-kube-api-access-w2dbt\") pod \"payload-pre-processing-e2e-worker-w0-3c466c-7445648d54-7tl8k\" (UID: \"66a162de-bb65-4c00-b677-628ef558832f\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-3c466c-7445648d54-7tl8k"
Sep 12 00:33:22.556065 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:33:22.556043    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/66a162de-bb65-4c00-b677-628ef558832f-plugins-config-volume\") pod \"payload-pre-processing-e2e-worker-w0-3c466c-7445648d54-7tl8k\" (UID: \"66a162de-bb65-4c00-b677-628ef558832f\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-3c466c-7445648d54-7tl8k"
Sep 12 00:33:22.564474 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:33:22.564365    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-w2dbt\" (UniqueName: \"kubernetes.io/projected/66a162de-bb65-4c00-b677-628ef558832f-kube-api-access-w2dbt\") pod \"payload-pre-processing-e2e-worker-w0-3c466c-7445648d54-7tl8k\" (UID: \"66a162de-bb65-4c00-b677-628ef558832f\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-3c466c-7445648d54-7tl8k"
Sep 12 00:33:22.654770 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:33:22.654740    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b0fe6c78-af34-4703-a109-628c4a86ea57-plugins-config-volume\") pod \"payload-processing-e2e-worker-w0-3c466c-59bdd48b77-sv6rh\" (UID: \"b0fe6c78-af34-4703-a109-628c4a86ea57\") " pod="openshift-ingress/payload-processing-e2e-worker-w0-3c466c-59bdd48b77-sv6rh"
Sep 12 00:33:22.654911 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:33:22.654790    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bt5zv\" (UniqueName: \"kubernetes.io/projected/b0fe6c78-af34-4703-a109-628c4a86ea57-kube-api-access-bt5zv\") pod \"payload-processing-e2e-worker-w0-3c466c-59bdd48b77-sv6rh\" (UID: \"b0fe6c78-af34-4703-a109-628c4a86ea57\") " pod="openshift-ingress/payload-processing-e2e-worker-w0-3c466c-59bdd48b77-sv6rh"
Sep 12 00:33:22.655809 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:33:22.655781    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b0fe6c78-af34-4703-a109-628c4a86ea57-plugins-config-volume\") pod \"payload-processing-e2e-worker-w0-3c466c-59bdd48b77-sv6rh\" (UID: \"b0fe6c78-af34-4703-a109-628c4a86ea57\") " pod="openshift-ingress/payload-processing-e2e-worker-w0-3c466c-59bdd48b77-sv6rh"
Sep 12 00:33:22.663174 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:33:22.663102    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bt5zv\" (UniqueName: \"kubernetes.io/projected/b0fe6c78-af34-4703-a109-628c4a86ea57-kube-api-access-bt5zv\") pod \"payload-processing-e2e-worker-w0-3c466c-59bdd48b77-sv6rh\" (UID: \"b0fe6c78-af34-4703-a109-628c4a86ea57\") " pod="openshift-ingress/payload-processing-e2e-worker-w0-3c466c-59bdd48b77-sv6rh"
Sep 12 00:33:22.678993 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:33:22.678968    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-3c466c-7445648d54-7tl8k"
Sep 12 00:33:22.723036 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:33:22.723001    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-worker-w0-3c466c-59bdd48b77-sv6rh"
Sep 12 00:33:22.831970 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:33:22.831944    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w0-3c466c-5485d64cc6-p6d9z"]
Sep 12 00:33:22.834037 ip-10-0-141-128 kubenswrapper[2571]: W0912 00:33:22.833999    2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod95707da3_a50f_48dc_8f29_7e3de07bab92.slice/crio-7f4f3272708cc219cb2e73a26f7158e1cca40fdc0122ddf31ca587747a4f0aae WatchSource:0}: Error finding container 7f4f3272708cc219cb2e73a26f7158e1cca40fdc0122ddf31ca587747a4f0aae: Status 404 returned error can't find the container with id 7f4f3272708cc219cb2e73a26f7158e1cca40fdc0122ddf31ca587747a4f0aae
Sep 12 00:33:22.956280 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:33:22.956239    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w0-3c466c-7445648d54-7tl8k"]
Sep 12 00:33:22.958217 ip-10-0-141-128 kubenswrapper[2571]: W0912 00:33:22.958189    2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod66a162de_bb65_4c00_b677_628ef558832f.slice/crio-82a78536d52acdab53fc2e0f9c52d78812742ff5b143a0e71d70a059ed893e5e WatchSource:0}: Error finding container 82a78536d52acdab53fc2e0f9c52d78812742ff5b143a0e71d70a059ed893e5e: Status 404 returned error can't find the container with id 82a78536d52acdab53fc2e0f9c52d78812742ff5b143a0e71d70a059ed893e5e
Sep 12 00:33:23.031203 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:33:23.031157    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w0-3c466c-59bdd48b77-sv6rh"]
Sep 12 00:33:23.034439 ip-10-0-141-128 kubenswrapper[2571]: W0912 00:33:23.034400    2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb0fe6c78_af34_4703_a109_628c4a86ea57.slice/crio-eaf5411ecedc7ddbab3fc98ee3ea50a4d0ef0e294df9022b435012f344806b57 WatchSource:0}: Error finding container eaf5411ecedc7ddbab3fc98ee3ea50a4d0ef0e294df9022b435012f344806b57: Status 404 returned error can't find the container with id eaf5411ecedc7ddbab3fc98ee3ea50a4d0ef0e294df9022b435012f344806b57
Sep 12 00:33:23.420741 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:33:23.420654    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-3c466c-5485d64cc6-p6d9z" event={"ID":"95707da3-a50f-48dc-8f29-7e3de07bab92","Type":"ContainerStarted","Data":"5c196010a06c8807018fc5b1c11e6a222398a46bc4453779d7118ce437f56805"}
Sep 12 00:33:23.420741 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:33:23.420715    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-3c466c-5485d64cc6-p6d9z" event={"ID":"95707da3-a50f-48dc-8f29-7e3de07bab92","Type":"ContainerStarted","Data":"7f4f3272708cc219cb2e73a26f7158e1cca40fdc0122ddf31ca587747a4f0aae"}
Sep 12 00:33:23.421277 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:33:23.420848    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-3c466c-5485d64cc6-p6d9z"
Sep 12 00:33:23.422527 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:33:23.422448    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-worker-w0-3c466c-59bdd48b77-sv6rh" event={"ID":"b0fe6c78-af34-4703-a109-628c4a86ea57","Type":"ContainerStarted","Data":"55787cbbfb6f302299e73b321fdc7aff038a6036f680c8164b7ff5641c86625a"}
Sep 12 00:33:23.422663 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:33:23.422530    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-worker-w0-3c466c-59bdd48b77-sv6rh"
Sep 12 00:33:23.422663 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:33:23.422557    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-worker-w0-3c466c-59bdd48b77-sv6rh" event={"ID":"b0fe6c78-af34-4703-a109-628c4a86ea57","Type":"ContainerStarted","Data":"eaf5411ecedc7ddbab3fc98ee3ea50a4d0ef0e294df9022b435012f344806b57"}
Sep 12 00:33:23.424097 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:33:23.423925    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-3c466c-7445648d54-7tl8k" event={"ID":"66a162de-bb65-4c00-b677-628ef558832f","Type":"ContainerStarted","Data":"4450c08db7824f193cec9ac8d2fd818d3e5209e147f7350f1f8757d6c79fe1a8"}
Sep 12 00:33:23.424097 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:33:23.423960    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-3c466c-7445648d54-7tl8k" event={"ID":"66a162de-bb65-4c00-b677-628ef558832f","Type":"ContainerStarted","Data":"82a78536d52acdab53fc2e0f9c52d78812742ff5b143a0e71d70a059ed893e5e"}
Sep 12 00:33:23.424097 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:33:23.424023    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-3c466c-7445648d54-7tl8k"
Sep 12 00:33:23.440904 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:33:23.440844    2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-3c466c-5485d64cc6-p6d9z" podStartSLOduration=2.072939445 podStartE2EDuration="2.440823605s" totalImagesPullingTime="367.88416ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-12 00:33:21 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-12 00:33:23.43806092 +0000 UTC m=+1014.709701428" watchObservedRunningTime="2026-09-12 00:33:23.440823605 +0000 UTC m=+1014.712464114"
Sep 12 00:33:23.456294 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:33:23.456193    2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-3c466c-7445648d54-7tl8k" podStartSLOduration=1.456176492 podStartE2EDuration="1.456176492s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-12 00:33:22 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-12 00:33:23.454113946 +0000 UTC m=+1014.725754455" watchObservedRunningTime="2026-09-12 00:33:23.456176492 +0000 UTC m=+1014.727817000"
Sep 12 00:33:23.474177 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:33:23.474112    2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-worker-w0-3c466c-59bdd48b77-sv6rh" podStartSLOduration=1.47409014 podStartE2EDuration="1.47409014s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-12 00:33:22 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-12 00:33:23.47245406 +0000 UTC m=+1014.744094570" watchObservedRunningTime="2026-09-12 00:33:23.47409014 +0000 UTC m=+1014.745730651"
Sep 12 00:33:28.436028 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:33:28.435995    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-a4df66-7764776995-d48lt"
Sep 12 00:33:29.436085 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:33:29.436056    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-3c466c-5485d64cc6-p6d9z"
Sep 12 00:33:31.066613 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:33:31.064631    2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-autores-4aaf1a-869d96c798-2mg2b"]
Sep 12 00:33:31.089604 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:33:31.086853    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-autores-4aaf1a-869d96c798-2mg2b"
Sep 12 00:33:31.127259 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:33:31.125720    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-autores-4aaf1a-serving-cert\""
Sep 12 00:33:31.179354 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:33:31.179318    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-autores-4aaf1a-869d96c798-2mg2b"]
Sep 12 00:33:31.256990 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:33:31.256939    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/4ba72a97-317b-438a-a952-938cf6e75a07-maas-api-tls\") pod \"maas-api-e2e-autores-4aaf1a-869d96c798-2mg2b\" (UID: \"4ba72a97-317b-438a-a952-938cf6e75a07\") " pod="odh-ai-gateway-infra/maas-api-e2e-autores-4aaf1a-869d96c798-2mg2b"
Sep 12 00:33:31.257145 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:33:31.257108    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/4ba72a97-317b-438a-a952-938cf6e75a07-metrics-certs\") pod \"maas-api-e2e-autores-4aaf1a-869d96c798-2mg2b\" (UID: \"4ba72a97-317b-438a-a952-938cf6e75a07\") " pod="odh-ai-gateway-infra/maas-api-e2e-autores-4aaf1a-869d96c798-2mg2b"
Sep 12 00:33:31.257202 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:33:31.257147    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-n8dhz\" (UniqueName: \"kubernetes.io/projected/4ba72a97-317b-438a-a952-938cf6e75a07-kube-api-access-n8dhz\") pod \"maas-api-e2e-autores-4aaf1a-869d96c798-2mg2b\" (UID: \"4ba72a97-317b-438a-a952-938cf6e75a07\") " pod="odh-ai-gateway-infra/maas-api-e2e-autores-4aaf1a-869d96c798-2mg2b"
Sep 12 00:33:31.257236 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:33:31.257206    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/4ba72a97-317b-438a-a952-938cf6e75a07-openshift-service-ca\") pod \"maas-api-e2e-autores-4aaf1a-869d96c798-2mg2b\" (UID: \"4ba72a97-317b-438a-a952-938cf6e75a07\") " pod="odh-ai-gateway-infra/maas-api-e2e-autores-4aaf1a-869d96c798-2mg2b"
Sep 12 00:33:31.358300 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:33:31.358219    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/4ba72a97-317b-438a-a952-938cf6e75a07-metrics-certs\") pod \"maas-api-e2e-autores-4aaf1a-869d96c798-2mg2b\" (UID: \"4ba72a97-317b-438a-a952-938cf6e75a07\") " pod="odh-ai-gateway-infra/maas-api-e2e-autores-4aaf1a-869d96c798-2mg2b"
Sep 12 00:33:31.358300 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:33:31.358264    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-n8dhz\" (UniqueName: \"kubernetes.io/projected/4ba72a97-317b-438a-a952-938cf6e75a07-kube-api-access-n8dhz\") pod \"maas-api-e2e-autores-4aaf1a-869d96c798-2mg2b\" (UID: \"4ba72a97-317b-438a-a952-938cf6e75a07\") " pod="odh-ai-gateway-infra/maas-api-e2e-autores-4aaf1a-869d96c798-2mg2b"
Sep 12 00:33:31.358300 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:33:31.358300    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/4ba72a97-317b-438a-a952-938cf6e75a07-openshift-service-ca\") pod \"maas-api-e2e-autores-4aaf1a-869d96c798-2mg2b\" (UID: \"4ba72a97-317b-438a-a952-938cf6e75a07\") " pod="odh-ai-gateway-infra/maas-api-e2e-autores-4aaf1a-869d96c798-2mg2b"
Sep 12 00:33:31.358573 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:33:31.358340    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/4ba72a97-317b-438a-a952-938cf6e75a07-maas-api-tls\") pod \"maas-api-e2e-autores-4aaf1a-869d96c798-2mg2b\" (UID: \"4ba72a97-317b-438a-a952-938cf6e75a07\") " pod="odh-ai-gateway-infra/maas-api-e2e-autores-4aaf1a-869d96c798-2mg2b"
Sep 12 00:33:31.358874 ip-10-0-141-128 kubenswrapper[2571]: E0912 00:33:31.358468    2571 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-autores-4aaf1a-serving-cert: secret "maas-api-e2e-autores-4aaf1a-serving-cert" not found
Sep 12 00:33:31.358964 ip-10-0-141-128 kubenswrapper[2571]: E0912 00:33:31.358949    2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4ba72a97-317b-438a-a952-938cf6e75a07-maas-api-tls podName:4ba72a97-317b-438a-a952-938cf6e75a07 nodeName:}" failed. No retries permitted until 2026-09-12 00:33:31.858925155 +0000 UTC m=+1023.130565649 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/4ba72a97-317b-438a-a952-938cf6e75a07-maas-api-tls") pod "maas-api-e2e-autores-4aaf1a-869d96c798-2mg2b" (UID: "4ba72a97-317b-438a-a952-938cf6e75a07") : secret "maas-api-e2e-autores-4aaf1a-serving-cert" not found
Sep 12 00:33:31.359288 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:33:31.359249    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/4ba72a97-317b-438a-a952-938cf6e75a07-openshift-service-ca\") pod \"maas-api-e2e-autores-4aaf1a-869d96c798-2mg2b\" (UID: \"4ba72a97-317b-438a-a952-938cf6e75a07\") " pod="odh-ai-gateway-infra/maas-api-e2e-autores-4aaf1a-869d96c798-2mg2b"
Sep 12 00:33:31.361036 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:33:31.361013    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/4ba72a97-317b-438a-a952-938cf6e75a07-metrics-certs\") pod \"maas-api-e2e-autores-4aaf1a-869d96c798-2mg2b\" (UID: \"4ba72a97-317b-438a-a952-938cf6e75a07\") " pod="odh-ai-gateway-infra/maas-api-e2e-autores-4aaf1a-869d96c798-2mg2b"
Sep 12 00:33:31.390609 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:33:31.390088    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-n8dhz\" (UniqueName: \"kubernetes.io/projected/4ba72a97-317b-438a-a952-938cf6e75a07-kube-api-access-n8dhz\") pod \"maas-api-e2e-autores-4aaf1a-869d96c798-2mg2b\" (UID: \"4ba72a97-317b-438a-a952-938cf6e75a07\") " pod="odh-ai-gateway-infra/maas-api-e2e-autores-4aaf1a-869d96c798-2mg2b"
Sep 12 00:33:31.871764 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:33:31.871168    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/4ba72a97-317b-438a-a952-938cf6e75a07-maas-api-tls\") pod \"maas-api-e2e-autores-4aaf1a-869d96c798-2mg2b\" (UID: \"4ba72a97-317b-438a-a952-938cf6e75a07\") " pod="odh-ai-gateway-infra/maas-api-e2e-autores-4aaf1a-869d96c798-2mg2b"
Sep 12 00:33:31.879150 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:33:31.879095    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/4ba72a97-317b-438a-a952-938cf6e75a07-maas-api-tls\") pod \"maas-api-e2e-autores-4aaf1a-869d96c798-2mg2b\" (UID: \"4ba72a97-317b-438a-a952-938cf6e75a07\") " pod="odh-ai-gateway-infra/maas-api-e2e-autores-4aaf1a-869d96c798-2mg2b"
Sep 12 00:33:32.060288 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:33:32.060083    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-autores-4aaf1a-869d96c798-2mg2b"
Sep 12 00:33:32.359873 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:33:32.359836    2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-1b1c80d8-c55659db4-txtpz"]
Sep 12 00:33:32.375906 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:33:32.375878    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-1b1c80d8-c55659db4-txtpz"
Sep 12 00:33:32.391569 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:33:32.389035    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ait-1b1c80d8-serving-cert\""
Sep 12 00:33:32.419604 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:33:32.412568    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-a4df66-7dd5b76c4-vqffm"
Sep 12 00:33:32.433483 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:33:32.433455    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-1b1c80d8-c55659db4-txtpz"]
Sep 12 00:33:32.460358 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:33:32.460334    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-autores-4aaf1a-869d96c798-2mg2b"]
Sep 12 00:33:32.497613 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:33:32.497022    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-autores-4aaf1a-869d96c798-2mg2b" event={"ID":"4ba72a97-317b-438a-a952-938cf6e75a07","Type":"ContainerStarted","Data":"e78ab1649601ecc277706157b418ce22621830039ff7990fa413d6af7064a523"}
Sep 12 00:33:32.497742 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:33:32.497609    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ef90d781-117b-46cb-bed1-cd575cdd3a11-openshift-service-ca\") pod \"maas-api-e2e-ait-1b1c80d8-c55659db4-txtpz\" (UID: \"ef90d781-117b-46cb-bed1-cd575cdd3a11\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-1b1c80d8-c55659db4-txtpz"
Sep 12 00:33:32.497810 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:33:32.497745    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ef90d781-117b-46cb-bed1-cd575cdd3a11-maas-api-tls\") pod \"maas-api-e2e-ait-1b1c80d8-c55659db4-txtpz\" (UID: \"ef90d781-117b-46cb-bed1-cd575cdd3a11\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-1b1c80d8-c55659db4-txtpz"
Sep 12 00:33:32.497810 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:33:32.497778    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gnfrk\" (UniqueName: \"kubernetes.io/projected/ef90d781-117b-46cb-bed1-cd575cdd3a11-kube-api-access-gnfrk\") pod \"maas-api-e2e-ait-1b1c80d8-c55659db4-txtpz\" (UID: \"ef90d781-117b-46cb-bed1-cd575cdd3a11\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-1b1c80d8-c55659db4-txtpz"
Sep 12 00:33:32.497922 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:33:32.497813    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ef90d781-117b-46cb-bed1-cd575cdd3a11-metrics-certs\") pod \"maas-api-e2e-ait-1b1c80d8-c55659db4-txtpz\" (UID: \"ef90d781-117b-46cb-bed1-cd575cdd3a11\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-1b1c80d8-c55659db4-txtpz"
Sep 12 00:33:32.608239 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:33:32.608198    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ef90d781-117b-46cb-bed1-cd575cdd3a11-maas-api-tls\") pod \"maas-api-e2e-ait-1b1c80d8-c55659db4-txtpz\" (UID: \"ef90d781-117b-46cb-bed1-cd575cdd3a11\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-1b1c80d8-c55659db4-txtpz"
Sep 12 00:33:32.608516 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:33:32.608493    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gnfrk\" (UniqueName: \"kubernetes.io/projected/ef90d781-117b-46cb-bed1-cd575cdd3a11-kube-api-access-gnfrk\") pod \"maas-api-e2e-ait-1b1c80d8-c55659db4-txtpz\" (UID: \"ef90d781-117b-46cb-bed1-cd575cdd3a11\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-1b1c80d8-c55659db4-txtpz"
Sep 12 00:33:32.608936 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:33:32.608900    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ef90d781-117b-46cb-bed1-cd575cdd3a11-metrics-certs\") pod \"maas-api-e2e-ait-1b1c80d8-c55659db4-txtpz\" (UID: \"ef90d781-117b-46cb-bed1-cd575cdd3a11\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-1b1c80d8-c55659db4-txtpz"
Sep 12 00:33:32.609166 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:33:32.609147    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ef90d781-117b-46cb-bed1-cd575cdd3a11-openshift-service-ca\") pod \"maas-api-e2e-ait-1b1c80d8-c55659db4-txtpz\" (UID: \"ef90d781-117b-46cb-bed1-cd575cdd3a11\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-1b1c80d8-c55659db4-txtpz"
Sep 12 00:33:32.610116 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:33:32.610093    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ef90d781-117b-46cb-bed1-cd575cdd3a11-openshift-service-ca\") pod \"maas-api-e2e-ait-1b1c80d8-c55659db4-txtpz\" (UID: \"ef90d781-117b-46cb-bed1-cd575cdd3a11\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-1b1c80d8-c55659db4-txtpz"
Sep 12 00:33:32.610545 ip-10-0-141-128 kubenswrapper[2571]: E0912 00:33:32.610527    2571 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-ait-1b1c80d8-serving-cert: secret "maas-api-e2e-ait-1b1c80d8-serving-cert" not found
Sep 12 00:33:32.610742 ip-10-0-141-128 kubenswrapper[2571]: E0912 00:33:32.610728    2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ef90d781-117b-46cb-bed1-cd575cdd3a11-maas-api-tls podName:ef90d781-117b-46cb-bed1-cd575cdd3a11 nodeName:}" failed. No retries permitted until 2026-09-12 00:33:33.110705057 +0000 UTC m=+1024.382345545 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/ef90d781-117b-46cb-bed1-cd575cdd3a11-maas-api-tls") pod "maas-api-e2e-ait-1b1c80d8-c55659db4-txtpz" (UID: "ef90d781-117b-46cb-bed1-cd575cdd3a11") : secret "maas-api-e2e-ait-1b1c80d8-serving-cert" not found
Sep 12 00:33:32.624371 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:33:32.624235    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ef90d781-117b-46cb-bed1-cd575cdd3a11-metrics-certs\") pod \"maas-api-e2e-ait-1b1c80d8-c55659db4-txtpz\" (UID: \"ef90d781-117b-46cb-bed1-cd575cdd3a11\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-1b1c80d8-c55659db4-txtpz"
Sep 12 00:33:32.626390 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:33:32.626358    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gnfrk\" (UniqueName: \"kubernetes.io/projected/ef90d781-117b-46cb-bed1-cd575cdd3a11-kube-api-access-gnfrk\") pod \"maas-api-e2e-ait-1b1c80d8-c55659db4-txtpz\" (UID: \"ef90d781-117b-46cb-bed1-cd575cdd3a11\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-1b1c80d8-c55659db4-txtpz"
Sep 12 00:33:33.122246 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:33:33.122204    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ef90d781-117b-46cb-bed1-cd575cdd3a11-maas-api-tls\") pod \"maas-api-e2e-ait-1b1c80d8-c55659db4-txtpz\" (UID: \"ef90d781-117b-46cb-bed1-cd575cdd3a11\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-1b1c80d8-c55659db4-txtpz"
Sep 12 00:33:33.127406 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:33:33.127358    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ef90d781-117b-46cb-bed1-cd575cdd3a11-maas-api-tls\") pod \"maas-api-e2e-ait-1b1c80d8-c55659db4-txtpz\" (UID: \"ef90d781-117b-46cb-bed1-cd575cdd3a11\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-1b1c80d8-c55659db4-txtpz"
Sep 12 00:33:33.244849 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:33:33.244812    2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-1b1c80d8-57bf44c4f4-lrk22"]
Sep 12 00:33:33.248710 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:33:33.248689    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-1b1c80d8-57bf44c4f4-lrk22"
Sep 12 00:33:33.251777 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:33:33.251490    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-ait-1b1c80d8\""
Sep 12 00:33:33.251777 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:33:33.251675    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-ait-1b1c80d8-dockercfg-jb9z6\""
Sep 12 00:33:33.257851 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:33:33.257820    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-1b1c80d8-57bf44c4f4-lrk22"]
Sep 12 00:33:33.324410 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:33:33.324366    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/af68fd51-f790-4e42-842d-30b8118a4b5e-plugins-config-volume\") pod \"payload-pre-processing-e2e-ait-1b1c80d8-57bf44c4f4-lrk22\" (UID: \"af68fd51-f790-4e42-842d-30b8118a4b5e\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-1b1c80d8-57bf44c4f4-lrk22"
Sep 12 00:33:33.324704 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:33:33.324459    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kb8wv\" (UniqueName: \"kubernetes.io/projected/af68fd51-f790-4e42-842d-30b8118a4b5e-kube-api-access-kb8wv\") pod \"payload-pre-processing-e2e-ait-1b1c80d8-57bf44c4f4-lrk22\" (UID: \"af68fd51-f790-4e42-842d-30b8118a4b5e\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-1b1c80d8-57bf44c4f4-lrk22"
Sep 12 00:33:33.338992 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:33:33.338953    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-1b1c80d8-c55659db4-txtpz"
Sep 12 00:33:33.426728 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:33:33.426538    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kb8wv\" (UniqueName: \"kubernetes.io/projected/af68fd51-f790-4e42-842d-30b8118a4b5e-kube-api-access-kb8wv\") pod \"payload-pre-processing-e2e-ait-1b1c80d8-57bf44c4f4-lrk22\" (UID: \"af68fd51-f790-4e42-842d-30b8118a4b5e\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-1b1c80d8-57bf44c4f4-lrk22"
Sep 12 00:33:33.426728 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:33:33.426649    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/af68fd51-f790-4e42-842d-30b8118a4b5e-plugins-config-volume\") pod \"payload-pre-processing-e2e-ait-1b1c80d8-57bf44c4f4-lrk22\" (UID: \"af68fd51-f790-4e42-842d-30b8118a4b5e\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-1b1c80d8-57bf44c4f4-lrk22"
Sep 12 00:33:33.427922 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:33:33.427780    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/af68fd51-f790-4e42-842d-30b8118a4b5e-plugins-config-volume\") pod \"payload-pre-processing-e2e-ait-1b1c80d8-57bf44c4f4-lrk22\" (UID: \"af68fd51-f790-4e42-842d-30b8118a4b5e\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-1b1c80d8-57bf44c4f4-lrk22"
Sep 12 00:33:33.436209 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:33:33.436163    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kb8wv\" (UniqueName: \"kubernetes.io/projected/af68fd51-f790-4e42-842d-30b8118a4b5e-kube-api-access-kb8wv\") pod \"payload-pre-processing-e2e-ait-1b1c80d8-57bf44c4f4-lrk22\" (UID: \"af68fd51-f790-4e42-842d-30b8118a4b5e\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-1b1c80d8-57bf44c4f4-lrk22"
Sep 12 00:33:33.508347 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:33:33.508169    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-autores-4aaf1a-869d96c798-2mg2b" event={"ID":"4ba72a97-317b-438a-a952-938cf6e75a07","Type":"ContainerStarted","Data":"50b44b9f613543147d5767d7212f5ca456c065375829074938128188c5c53c62"}
Sep 12 00:33:33.508347 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:33:33.508256    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-autores-4aaf1a-869d96c798-2mg2b"
Sep 12 00:33:33.528325 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:33:33.528268    2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-autores-4aaf1a-869d96c798-2mg2b" podStartSLOduration=2.1332213270000002 podStartE2EDuration="2.528251533s" totalImagesPullingTime="395.030206ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-12 00:33:31 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-12 00:33:33.526494155 +0000 UTC m=+1024.798134664" watchObservedRunningTime="2026-09-12 00:33:33.528251533 +0000 UTC m=+1024.799892041"
Sep 12 00:33:33.560987 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:33:33.560960    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-1b1c80d8-57bf44c4f4-lrk22"
Sep 12 00:33:33.676194 ip-10-0-141-128 kubenswrapper[2571]: W0912 00:33:33.674642    2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podef90d781_117b_46cb_bed1_cd575cdd3a11.slice/crio-86e12c54330190a5e70e3400d06c2a773552c09a0e16ace58b9e161e50fef3fe WatchSource:0}: Error finding container 86e12c54330190a5e70e3400d06c2a773552c09a0e16ace58b9e161e50fef3fe: Status 404 returned error can't find the container with id 86e12c54330190a5e70e3400d06c2a773552c09a0e16ace58b9e161e50fef3fe
Sep 12 00:33:33.683785 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:33:33.679687    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-1b1c80d8-c55659db4-txtpz"]
Sep 12 00:33:33.901742 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:33:33.901653    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-1b1c80d8-57bf44c4f4-lrk22"]
Sep 12 00:33:34.423797 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:33:34.423703    2571 patch_prober.go:28] interesting pod/e2e-inf-a-a4df66-openshift-default-7cc987766d-5t9vl container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.41:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Sep 12 00:33:34.423797 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:33:34.423772    2571 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-inf-a-a4df66-openshift-default-7cc987766d-5t9vl" podUID="7d867c39-4af2-46d5-9388-95af564c321e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.41:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Sep 12 00:33:34.433333 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:33:34.433305    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-3c466c-7445648d54-7tl8k"
Sep 12 00:33:34.433716 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:33:34.433386    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-worker-w0-3c466c-59bdd48b77-sv6rh"
Sep 12 00:33:34.520543 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:33:34.520384    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-1b1c80d8-57bf44c4f4-lrk22" event={"ID":"af68fd51-f790-4e42-842d-30b8118a4b5e","Type":"ContainerStarted","Data":"a1dcfd8cf7c461a326224d4103bd0cb0e61cf8c82e7a9d48478bc8e8c087f51a"}
Sep 12 00:33:34.520543 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:33:34.520451    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-1b1c80d8-57bf44c4f4-lrk22" event={"ID":"af68fd51-f790-4e42-842d-30b8118a4b5e","Type":"ContainerStarted","Data":"b87f37dd181761469d3854b35dbd0b55b90445d97ea7f61e0e5a00d9155d1987"}
Sep 12 00:33:34.520543 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:33:34.520508    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-ait-1b1c80d8-57bf44c4f4-lrk22"
Sep 12 00:33:34.523050 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:33:34.523015    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-1b1c80d8-c55659db4-txtpz" event={"ID":"ef90d781-117b-46cb-bed1-cd575cdd3a11","Type":"ContainerStarted","Data":"8c8d31706f3b198a52edb0ceec7af9d0a897073eb5a304628e240a97ba067077"}
Sep 12 00:33:34.523180 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:33:34.523067    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-1b1c80d8-c55659db4-txtpz" event={"ID":"ef90d781-117b-46cb-bed1-cd575cdd3a11","Type":"ContainerStarted","Data":"86e12c54330190a5e70e3400d06c2a773552c09a0e16ace58b9e161e50fef3fe"}
Sep 12 00:33:34.523180 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:33:34.523124    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-1b1c80d8-c55659db4-txtpz"
Sep 12 00:33:34.540481 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:33:34.540144    2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-ait-1b1c80d8-57bf44c4f4-lrk22" podStartSLOduration=1.5401237399999999 podStartE2EDuration="1.54012374s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-12 00:33:33 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-12 00:33:34.5397836 +0000 UTC m=+1025.811424110" watchObservedRunningTime="2026-09-12 00:33:34.54012374 +0000 UTC m=+1025.811764269"
Sep 12 00:33:34.558676 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:33:34.558574    2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ait-1b1c80d8-c55659db4-txtpz" podStartSLOduration=2.167933645 podStartE2EDuration="2.558555093s" totalImagesPullingTime="390.621448ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-12 00:33:32 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-12 00:33:34.55705454 +0000 UTC m=+1025.828695050" watchObservedRunningTime="2026-09-12 00:33:34.558555093 +0000 UTC m=+1025.830195604"
Sep 12 00:33:37.543820 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:33:37.543789    2571 generic.go:356] "Generic (PLEG): container finished" podID="7d867c39-4af2-46d5-9388-95af564c321e" containerID="32d161283eb48801c85c052c2573fdbcf60768b47ff0dda2b6f6622f93a51e7e" exitCode=0
Sep 12 00:33:37.544208 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:33:37.543861    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-a4df66-openshift-default-7cc987766d-5t9vl" event={"ID":"7d867c39-4af2-46d5-9388-95af564c321e","Type":"ContainerDied","Data":"32d161283eb48801c85c052c2573fdbcf60768b47ff0dda2b6f6622f93a51e7e"}
Sep 12 00:33:37.544364 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:33:37.544346    2571 scope.go:117] "RemoveContainer" containerID="32d161283eb48801c85c052c2573fdbcf60768b47ff0dda2b6f6622f93a51e7e"
Sep 12 00:33:37.545645 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:33:37.545618    2571 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236196Ki","pods":"250"}
Sep 12 00:33:37.545731 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:33:37.545691    2571 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236196Ki","pods":"250"}
Sep 12 00:33:37.545731 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:33:37.545696    2571 generic.go:356] "Generic (PLEG): container finished" podID="d7e353f5-8ba3-41a4-bd94-5e2c0cf9fa46" containerID="0016a348a7c14fe82d8c59241a135afe29e55cc506d42caec078fd91aacce125" exitCode=0
Sep 12 00:33:37.545731 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:33:37.545722    2571 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236196Ki","pods":"250"}
Sep 12 00:33:37.545829 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:33:37.545753    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-139288-openshift-default-8557bc4874-7zvml" event={"ID":"d7e353f5-8ba3-41a4-bd94-5e2c0cf9fa46","Type":"ContainerDied","Data":"0016a348a7c14fe82d8c59241a135afe29e55cc506d42caec078fd91aacce125"}
Sep 12 00:33:37.546140 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:33:37.546108    2571 scope.go:117] "RemoveContainer" containerID="0016a348a7c14fe82d8c59241a135afe29e55cc506d42caec078fd91aacce125"
Sep 12 00:33:37.547290 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:33:37.547264    2571 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236196Ki","pods":"250"}
Sep 12 00:33:37.547369 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:33:37.547340    2571 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236196Ki","pods":"250"}
Sep 12 00:33:37.547406 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:33:37.547389    2571 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236196Ki","pods":"250"}
Sep 12 00:33:38.553432 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:33:38.553394    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-a4df66-openshift-default-7cc987766d-5t9vl" event={"ID":"7d867c39-4af2-46d5-9388-95af564c321e","Type":"ContainerStarted","Data":"0d94e10b694aa046763161a199735a67e603742c8595ff67a64eb8ce0bd990c7"}
Sep 12 00:33:38.555831 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:33:38.555807    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-139288-openshift-default-8557bc4874-7zvml" event={"ID":"d7e353f5-8ba3-41a4-bd94-5e2c0cf9fa46","Type":"ContainerStarted","Data":"bc83423552782511e280547bfcd44127e59b7e2bee34bac0a69fecbbc0c621ea"}
Sep 12 00:33:38.715626 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:33:38.715549    2571 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w4-139288-openshift-default-8557bc4874-7zvml"
Sep 12 00:33:38.715813 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:33:38.715792    2571 patch_prober.go:28] interesting pod/e2e-worker-w4-139288-openshift-default-8557bc4874-7zvml container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.40:15021/healthz/ready\": dial tcp 10.132.0.40:15021: connect: connection refused" start-of-body=
Sep 12 00:33:38.715892 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:33:38.715856    2571 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-139288-openshift-default-8557bc4874-7zvml" podUID="d7e353f5-8ba3-41a4-bd94-5e2c0cf9fa46" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.40:15021/healthz/ready\": dial tcp 10.132.0.40:15021: connect: connection refused"
Sep 12 00:33:38.798655 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:33:38.798626    2571 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-a-a4df66-openshift-default-7cc987766d-5t9vl"
Sep 12 00:33:38.798874 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:33:38.798845    2571 patch_prober.go:28] interesting pod/e2e-inf-a-a4df66-openshift-default-7cc987766d-5t9vl container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.41:15021/healthz/ready\": dial tcp 10.132.0.41:15021: connect: connection refused" start-of-body=
Sep 12 00:33:38.798924 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:33:38.798902    2571 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-a4df66-openshift-default-7cc987766d-5t9vl" podUID="7d867c39-4af2-46d5-9388-95af564c321e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.41:15021/healthz/ready\": dial tcp 10.132.0.41:15021: connect: connection refused"
Sep 12 00:33:39.532457 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:33:39.532427    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-autores-4aaf1a-869d96c798-2mg2b"
Sep 12 00:33:39.716176 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:33:39.716141    2571 patch_prober.go:28] interesting pod/e2e-worker-w4-139288-openshift-default-8557bc4874-7zvml container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.40:15021/healthz/ready\": dial tcp 10.132.0.40:15021: connect: connection refused" start-of-body=
Sep 12 00:33:39.716718 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:33:39.716690    2571 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-139288-openshift-default-8557bc4874-7zvml" podUID="d7e353f5-8ba3-41a4-bd94-5e2c0cf9fa46" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.40:15021/healthz/ready\": dial tcp 10.132.0.40:15021: connect: connection refused"
Sep 12 00:33:39.799684 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:33:39.799060    2571 patch_prober.go:28] interesting pod/e2e-inf-a-a4df66-openshift-default-7cc987766d-5t9vl container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.41:15021/healthz/ready\": dial tcp 10.132.0.41:15021: connect: connection refused" start-of-body=
Sep 12 00:33:39.799684 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:33:39.799126    2571 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-a4df66-openshift-default-7cc987766d-5t9vl" podUID="7d867c39-4af2-46d5-9388-95af564c321e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.41:15021/healthz/ready\": dial tcp 10.132.0.41:15021: connect: connection refused"
Sep 12 00:33:40.537552 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:33:40.537520    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-1b1c80d8-c55659db4-txtpz"
Sep 12 00:33:40.720864 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:33:40.720705    2571 patch_prober.go:28] interesting pod/e2e-worker-w4-139288-openshift-default-8557bc4874-7zvml container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.40:15021/healthz/ready\": dial tcp 10.132.0.40:15021: connect: connection refused" start-of-body=
Sep 12 00:33:40.720864 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:33:40.720779    2571 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-139288-openshift-default-8557bc4874-7zvml" podUID="d7e353f5-8ba3-41a4-bd94-5e2c0cf9fa46" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.40:15021/healthz/ready\": dial tcp 10.132.0.40:15021: connect: connection refused"
Sep 12 00:33:40.798656 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:33:40.798027    2571 patch_prober.go:28] interesting pod/e2e-inf-a-a4df66-openshift-default-7cc987766d-5t9vl container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.41:15021/healthz/ready\": dial tcp 10.132.0.41:15021: connect: connection refused" start-of-body=
Sep 12 00:33:40.798656 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:33:40.798087    2571 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-a4df66-openshift-default-7cc987766d-5t9vl" podUID="7d867c39-4af2-46d5-9388-95af564c321e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.41:15021/healthz/ready\": dial tcp 10.132.0.41:15021: connect: connection refused"
Sep 12 00:33:41.716640 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:33:41.716577    2571 patch_prober.go:28] interesting pod/e2e-worker-w4-139288-openshift-default-8557bc4874-7zvml container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.40:15021/healthz/ready\": dial tcp 10.132.0.40:15021: connect: connection refused" start-of-body=
Sep 12 00:33:41.716823 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:33:41.716655    2571 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-139288-openshift-default-8557bc4874-7zvml" podUID="d7e353f5-8ba3-41a4-bd94-5e2c0cf9fa46" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.40:15021/healthz/ready\": dial tcp 10.132.0.40:15021: connect: connection refused"
Sep 12 00:33:41.818610 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:33:41.815161    2571 patch_prober.go:28] interesting pod/e2e-inf-a-a4df66-openshift-default-7cc987766d-5t9vl container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.41:15021/healthz/ready\": dial tcp 10.132.0.41:15021: connect: connection refused" start-of-body=
Sep 12 00:33:41.818610 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:33:41.815225    2571 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-a4df66-openshift-default-7cc987766d-5t9vl" podUID="7d867c39-4af2-46d5-9388-95af564c321e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.41:15021/healthz/ready\": dial tcp 10.132.0.41:15021: connect: connection refused"
Sep 12 00:33:42.715838 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:33:42.715801    2571 patch_prober.go:28] interesting pod/e2e-worker-w4-139288-openshift-default-8557bc4874-7zvml container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.40:15021/healthz/ready\": dial tcp 10.132.0.40:15021: connect: connection refused" start-of-body=
Sep 12 00:33:42.716139 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:33:42.716107    2571 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-139288-openshift-default-8557bc4874-7zvml" podUID="d7e353f5-8ba3-41a4-bd94-5e2c0cf9fa46" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.40:15021/healthz/ready\": dial tcp 10.132.0.40:15021: connect: connection refused"
Sep 12 00:33:42.802019 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:33:42.801973    2571 patch_prober.go:28] interesting pod/e2e-inf-a-a4df66-openshift-default-7cc987766d-5t9vl container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.41:15021/healthz/ready\": dial tcp 10.132.0.41:15021: connect: connection refused" start-of-body=
Sep 12 00:33:42.802224 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:33:42.802075    2571 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-a4df66-openshift-default-7cc987766d-5t9vl" podUID="7d867c39-4af2-46d5-9388-95af564c321e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.41:15021/healthz/ready\": dial tcp 10.132.0.41:15021: connect: connection refused"
Sep 12 00:33:43.716230 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:33:43.716182    2571 patch_prober.go:28] interesting pod/e2e-worker-w4-139288-openshift-default-8557bc4874-7zvml container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.40:15021/healthz/ready\": dial tcp 10.132.0.40:15021: connect: connection refused" start-of-body=
Sep 12 00:33:43.716721 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:33:43.716256    2571 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-139288-openshift-default-8557bc4874-7zvml" podUID="d7e353f5-8ba3-41a4-bd94-5e2c0cf9fa46" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.40:15021/healthz/ready\": dial tcp 10.132.0.40:15021: connect: connection refused"
Sep 12 00:33:43.799009 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:33:43.798971    2571 patch_prober.go:28] interesting pod/e2e-inf-a-a4df66-openshift-default-7cc987766d-5t9vl container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.41:15021/healthz/ready\": dial tcp 10.132.0.41:15021: connect: connection refused" start-of-body=
Sep 12 00:33:43.799261 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:33:43.799233    2571 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-a4df66-openshift-default-7cc987766d-5t9vl" podUID="7d867c39-4af2-46d5-9388-95af564c321e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.41:15021/healthz/ready\": dial tcp 10.132.0.41:15021: connect: connection refused"
Sep 12 00:33:44.799037 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:33:44.798995    2571 patch_prober.go:28] interesting pod/e2e-inf-a-a4df66-openshift-default-7cc987766d-5t9vl container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.41:15021/healthz/ready\": dial tcp 10.132.0.41:15021: connect: connection refused" start-of-body=
Sep 12 00:33:44.799527 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:33:44.799063    2571 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-a4df66-openshift-default-7cc987766d-5t9vl" podUID="7d867c39-4af2-46d5-9388-95af564c321e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.41:15021/healthz/ready\": dial tcp 10.132.0.41:15021: connect: connection refused"
Sep 12 00:33:44.833164 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:33:44.832913    2571 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w4-139288-openshift-default-8557bc4874-7zvml"
Sep 12 00:33:44.833901 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:33:44.833473    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w4-139288-openshift-default-8557bc4874-7zvml"
Sep 12 00:33:44.834856 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:33:44.834835    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w4-139288-openshift-default-8557bc4874-7zvml"
Sep 12 00:33:45.532999 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:33:45.532972    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-ait-1b1c80d8-57bf44c4f4-lrk22"
Sep 12 00:33:45.802633 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:33:45.802551    2571 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-a-a4df66-openshift-default-7cc987766d-5t9vl"
Sep 12 00:33:45.802996 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:33:45.802935    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-a-a4df66-openshift-default-7cc987766d-5t9vl"
Sep 12 00:33:45.803513 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:33:45.803495    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-a-a4df66-openshift-default-7cc987766d-5t9vl"
Sep 12 00:33:46.454763 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:33:46.454725    2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-b-39d42b-55fc64bff5-cw8nr"]
Sep 12 00:33:46.458083 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:33:46.458058    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-b-39d42b-55fc64bff5-cw8nr"
Sep 12 00:33:46.460470 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:33:46.460450    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-inf-b-39d42b\""
Sep 12 00:33:46.460582 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:33:46.460472    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-inf-b-39d42b-dockercfg-kbfgg\""
Sep 12 00:33:46.477172 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:33:46.477148    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-b-39d42b-55fc64bff5-cw8nr"]
Sep 12 00:33:46.591419 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:33:46.591382    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/68dad0be-3290-4975-86d4-657968eb899a-plugins-config-volume\") pod \"payload-processing-e2e-inf-b-39d42b-55fc64bff5-cw8nr\" (UID: \"68dad0be-3290-4975-86d4-657968eb899a\") " pod="openshift-ingress/payload-processing-e2e-inf-b-39d42b-55fc64bff5-cw8nr"
Sep 12 00:33:46.591600 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:33:46.591531    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tlffh\" (UniqueName: \"kubernetes.io/projected/68dad0be-3290-4975-86d4-657968eb899a-kube-api-access-tlffh\") pod \"payload-processing-e2e-inf-b-39d42b-55fc64bff5-cw8nr\" (UID: \"68dad0be-3290-4975-86d4-657968eb899a\") " pod="openshift-ingress/payload-processing-e2e-inf-b-39d42b-55fc64bff5-cw8nr"
Sep 12 00:33:46.692254 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:33:46.692225    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tlffh\" (UniqueName: \"kubernetes.io/projected/68dad0be-3290-4975-86d4-657968eb899a-kube-api-access-tlffh\") pod \"payload-processing-e2e-inf-b-39d42b-55fc64bff5-cw8nr\" (UID: \"68dad0be-3290-4975-86d4-657968eb899a\") " pod="openshift-ingress/payload-processing-e2e-inf-b-39d42b-55fc64bff5-cw8nr"
Sep 12 00:33:46.692437 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:33:46.692377    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/68dad0be-3290-4975-86d4-657968eb899a-plugins-config-volume\") pod \"payload-processing-e2e-inf-b-39d42b-55fc64bff5-cw8nr\" (UID: \"68dad0be-3290-4975-86d4-657968eb899a\") " pod="openshift-ingress/payload-processing-e2e-inf-b-39d42b-55fc64bff5-cw8nr"
Sep 12 00:33:46.693270 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:33:46.693246    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/68dad0be-3290-4975-86d4-657968eb899a-plugins-config-volume\") pod \"payload-processing-e2e-inf-b-39d42b-55fc64bff5-cw8nr\" (UID: \"68dad0be-3290-4975-86d4-657968eb899a\") " pod="openshift-ingress/payload-processing-e2e-inf-b-39d42b-55fc64bff5-cw8nr"
Sep 12 00:33:46.699423 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:33:46.699400    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tlffh\" (UniqueName: \"kubernetes.io/projected/68dad0be-3290-4975-86d4-657968eb899a-kube-api-access-tlffh\") pod \"payload-processing-e2e-inf-b-39d42b-55fc64bff5-cw8nr\" (UID: \"68dad0be-3290-4975-86d4-657968eb899a\") " pod="openshift-ingress/payload-processing-e2e-inf-b-39d42b-55fc64bff5-cw8nr"
Sep 12 00:33:46.778290 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:33:46.778226    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-b-39d42b-55fc64bff5-cw8nr"
Sep 12 00:33:47.060113 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:33:47.060035    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-b-39d42b-55fc64bff5-cw8nr"]
Sep 12 00:33:47.064928 ip-10-0-141-128 kubenswrapper[2571]: W0912 00:33:47.064896    2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod68dad0be_3290_4975_86d4_657968eb899a.slice/crio-4e32ef566006d5210480339d5433c4fb7f3c325623149900a32170d13080b5d6 WatchSource:0}: Error finding container 4e32ef566006d5210480339d5433c4fb7f3c325623149900a32170d13080b5d6: Status 404 returned error can't find the container with id 4e32ef566006d5210480339d5433c4fb7f3c325623149900a32170d13080b5d6
Sep 12 00:33:47.624240 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:33:47.624201    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-b-39d42b-55fc64bff5-cw8nr" event={"ID":"68dad0be-3290-4975-86d4-657968eb899a","Type":"ContainerStarted","Data":"51f0baabaef9a8d1d506e226ef8c491805baae29a5114fb76d03762868cfc03e"}
Sep 12 00:33:47.624443 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:33:47.624302    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-b-39d42b-55fc64bff5-cw8nr" event={"ID":"68dad0be-3290-4975-86d4-657968eb899a","Type":"ContainerStarted","Data":"4e32ef566006d5210480339d5433c4fb7f3c325623149900a32170d13080b5d6"}
Sep 12 00:33:47.665604 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:33:47.665529    2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-inf-b-39d42b-55fc64bff5-cw8nr" podStartSLOduration=1.665512719 podStartE2EDuration="1.665512719s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-12 00:33:46 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-12 00:33:47.662202644 +0000 UTC m=+1038.933843154" watchObservedRunningTime="2026-09-12 00:33:47.665512719 +0000 UTC m=+1038.937153356"
Sep 12 00:33:48.629667 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:33:48.629639    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-inf-b-39d42b-55fc64bff5-cw8nr"
Sep 12 00:33:59.637648 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:33:59.637617    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-inf-b-39d42b-55fc64bff5-cw8nr"
Sep 12 00:34:18.978286 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:34:18.977978    2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-1b1c80d8-c55659db4-txtpz"]
Sep 12 00:34:18.978286 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:34:18.978264    2571 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-ait-1b1c80d8-c55659db4-txtpz" podUID="ef90d781-117b-46cb-bed1-cd575cdd3a11" containerName="maas-api" containerID="cri-o://8c8d31706f3b198a52edb0ceec7af9d0a897073eb5a304628e240a97ba067077" gracePeriod=30
Sep 12 00:34:19.043386 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:34:19.043339    2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-1b1c80d8-57bf44c4f4-lrk22"]
Sep 12 00:34:19.043946 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:34:19.043878    2571 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-ait-1b1c80d8-57bf44c4f4-lrk22" podUID="af68fd51-f790-4e42-842d-30b8118a4b5e" containerName="payload-pre-processing" containerID="cri-o://a1dcfd8cf7c461a326224d4103bd0cb0e61cf8c82e7a9d48478bc8e8c087f51a" gracePeriod=30
Sep 12 00:34:19.354805 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:34:19.354745    2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-1b1c80d8-c55659db4-txtpz"
Sep 12 00:34:19.377500 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:34:19.377470    2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-1b1c80d8-57bf44c4f4-lrk22"
Sep 12 00:34:19.424847 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:34:19.424811    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/af68fd51-f790-4e42-842d-30b8118a4b5e-plugins-config-volume\") pod \"af68fd51-f790-4e42-842d-30b8118a4b5e\" (UID: \"af68fd51-f790-4e42-842d-30b8118a4b5e\") "
Sep 12 00:34:19.424993 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:34:19.424915    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ef90d781-117b-46cb-bed1-cd575cdd3a11-openshift-service-ca\") pod \"ef90d781-117b-46cb-bed1-cd575cdd3a11\" (UID: \"ef90d781-117b-46cb-bed1-cd575cdd3a11\") "
Sep 12 00:34:19.424993 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:34:19.424952    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ef90d781-117b-46cb-bed1-cd575cdd3a11-maas-api-tls\") pod \"ef90d781-117b-46cb-bed1-cd575cdd3a11\" (UID: \"ef90d781-117b-46cb-bed1-cd575cdd3a11\") "
Sep 12 00:34:19.424993 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:34:19.424980    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-kb8wv\" (UniqueName: \"kubernetes.io/projected/af68fd51-f790-4e42-842d-30b8118a4b5e-kube-api-access-kb8wv\") pod \"af68fd51-f790-4e42-842d-30b8118a4b5e\" (UID: \"af68fd51-f790-4e42-842d-30b8118a4b5e\") "
Sep 12 00:34:19.425157 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:34:19.425035    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-gnfrk\" (UniqueName: \"kubernetes.io/projected/ef90d781-117b-46cb-bed1-cd575cdd3a11-kube-api-access-gnfrk\") pod \"ef90d781-117b-46cb-bed1-cd575cdd3a11\" (UID: \"ef90d781-117b-46cb-bed1-cd575cdd3a11\") "
Sep 12 00:34:19.425157 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:34:19.425070    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ef90d781-117b-46cb-bed1-cd575cdd3a11-metrics-certs\") pod \"ef90d781-117b-46cb-bed1-cd575cdd3a11\" (UID: \"ef90d781-117b-46cb-bed1-cd575cdd3a11\") "
Sep 12 00:34:19.425276 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:34:19.425210    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/af68fd51-f790-4e42-842d-30b8118a4b5e-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "af68fd51-f790-4e42-842d-30b8118a4b5e" (UID: "af68fd51-f790-4e42-842d-30b8118a4b5e"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 12 00:34:19.425332 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:34:19.425268    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ef90d781-117b-46cb-bed1-cd575cdd3a11-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "ef90d781-117b-46cb-bed1-cd575cdd3a11" (UID: "ef90d781-117b-46cb-bed1-cd575cdd3a11"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 12 00:34:19.425568 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:34:19.425544    2571 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ef90d781-117b-46cb-bed1-cd575cdd3a11-openshift-service-ca\") on node \"ip-10-0-141-128.ec2.internal\" DevicePath \"\""
Sep 12 00:34:19.425682 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:34:19.425573    2571 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/af68fd51-f790-4e42-842d-30b8118a4b5e-plugins-config-volume\") on node \"ip-10-0-141-128.ec2.internal\" DevicePath \"\""
Sep 12 00:34:19.427308 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:34:19.427281    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/af68fd51-f790-4e42-842d-30b8118a4b5e-kube-api-access-kb8wv" (OuterVolumeSpecName: "kube-api-access-kb8wv") pod "af68fd51-f790-4e42-842d-30b8118a4b5e" (UID: "af68fd51-f790-4e42-842d-30b8118a4b5e"). InnerVolumeSpecName "kube-api-access-kb8wv". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 12 00:34:19.427411 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:34:19.427317    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ef90d781-117b-46cb-bed1-cd575cdd3a11-kube-api-access-gnfrk" (OuterVolumeSpecName: "kube-api-access-gnfrk") pod "ef90d781-117b-46cb-bed1-cd575cdd3a11" (UID: "ef90d781-117b-46cb-bed1-cd575cdd3a11"). InnerVolumeSpecName "kube-api-access-gnfrk". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 12 00:34:19.427734 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:34:19.427712    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ef90d781-117b-46cb-bed1-cd575cdd3a11-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "ef90d781-117b-46cb-bed1-cd575cdd3a11" (UID: "ef90d781-117b-46cb-bed1-cd575cdd3a11"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 12 00:34:19.427792 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:34:19.427739    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ef90d781-117b-46cb-bed1-cd575cdd3a11-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "ef90d781-117b-46cb-bed1-cd575cdd3a11" (UID: "ef90d781-117b-46cb-bed1-cd575cdd3a11"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 12 00:34:19.526295 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:34:19.526204    2571 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-gnfrk\" (UniqueName: \"kubernetes.io/projected/ef90d781-117b-46cb-bed1-cd575cdd3a11-kube-api-access-gnfrk\") on node \"ip-10-0-141-128.ec2.internal\" DevicePath \"\""
Sep 12 00:34:19.526295 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:34:19.526237    2571 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ef90d781-117b-46cb-bed1-cd575cdd3a11-metrics-certs\") on node \"ip-10-0-141-128.ec2.internal\" DevicePath \"\""
Sep 12 00:34:19.526295 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:34:19.526247    2571 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ef90d781-117b-46cb-bed1-cd575cdd3a11-maas-api-tls\") on node \"ip-10-0-141-128.ec2.internal\" DevicePath \"\""
Sep 12 00:34:19.526295 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:34:19.526256    2571 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-kb8wv\" (UniqueName: \"kubernetes.io/projected/af68fd51-f790-4e42-842d-30b8118a4b5e-kube-api-access-kb8wv\") on node \"ip-10-0-141-128.ec2.internal\" DevicePath \"\""
Sep 12 00:34:19.805856 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:34:19.805765    2571 generic.go:356] "Generic (PLEG): container finished" podID="ef90d781-117b-46cb-bed1-cd575cdd3a11" containerID="8c8d31706f3b198a52edb0ceec7af9d0a897073eb5a304628e240a97ba067077" exitCode=0
Sep 12 00:34:19.805856 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:34:19.805819    2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-1b1c80d8-c55659db4-txtpz"
Sep 12 00:34:19.805856 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:34:19.805837    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-1b1c80d8-c55659db4-txtpz" event={"ID":"ef90d781-117b-46cb-bed1-cd575cdd3a11","Type":"ContainerDied","Data":"8c8d31706f3b198a52edb0ceec7af9d0a897073eb5a304628e240a97ba067077"}
Sep 12 00:34:19.806123 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:34:19.805897    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-1b1c80d8-c55659db4-txtpz" event={"ID":"ef90d781-117b-46cb-bed1-cd575cdd3a11","Type":"ContainerDied","Data":"86e12c54330190a5e70e3400d06c2a773552c09a0e16ace58b9e161e50fef3fe"}
Sep 12 00:34:19.806123 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:34:19.805926    2571 scope.go:117] "RemoveContainer" containerID="8c8d31706f3b198a52edb0ceec7af9d0a897073eb5a304628e240a97ba067077"
Sep 12 00:34:19.807419 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:34:19.807308    2571 generic.go:356] "Generic (PLEG): container finished" podID="af68fd51-f790-4e42-842d-30b8118a4b5e" containerID="a1dcfd8cf7c461a326224d4103bd0cb0e61cf8c82e7a9d48478bc8e8c087f51a" exitCode=0
Sep 12 00:34:19.807419 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:34:19.807350    2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-1b1c80d8-57bf44c4f4-lrk22"
Sep 12 00:34:19.807419 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:34:19.807362    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-1b1c80d8-57bf44c4f4-lrk22" event={"ID":"af68fd51-f790-4e42-842d-30b8118a4b5e","Type":"ContainerDied","Data":"a1dcfd8cf7c461a326224d4103bd0cb0e61cf8c82e7a9d48478bc8e8c087f51a"}
Sep 12 00:34:19.807419 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:34:19.807406    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-1b1c80d8-57bf44c4f4-lrk22" event={"ID":"af68fd51-f790-4e42-842d-30b8118a4b5e","Type":"ContainerDied","Data":"b87f37dd181761469d3854b35dbd0b55b90445d97ea7f61e0e5a00d9155d1987"}
Sep 12 00:34:19.816994 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:34:19.816977    2571 scope.go:117] "RemoveContainer" containerID="8c8d31706f3b198a52edb0ceec7af9d0a897073eb5a304628e240a97ba067077"
Sep 12 00:34:19.817274 ip-10-0-141-128 kubenswrapper[2571]: E0912 00:34:19.817247    2571 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8c8d31706f3b198a52edb0ceec7af9d0a897073eb5a304628e240a97ba067077\": container with ID starting with 8c8d31706f3b198a52edb0ceec7af9d0a897073eb5a304628e240a97ba067077 not found: ID does not exist" containerID="8c8d31706f3b198a52edb0ceec7af9d0a897073eb5a304628e240a97ba067077"
Sep 12 00:34:19.817391 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:34:19.817284    2571 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8c8d31706f3b198a52edb0ceec7af9d0a897073eb5a304628e240a97ba067077"} err="failed to get container status \"8c8d31706f3b198a52edb0ceec7af9d0a897073eb5a304628e240a97ba067077\": rpc error: code = NotFound desc = could not find container \"8c8d31706f3b198a52edb0ceec7af9d0a897073eb5a304628e240a97ba067077\": container with ID starting with 8c8d31706f3b198a52edb0ceec7af9d0a897073eb5a304628e240a97ba067077 not found: ID does not exist"
Sep 12 00:34:19.817391 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:34:19.817304    2571 scope.go:117] "RemoveContainer" containerID="a1dcfd8cf7c461a326224d4103bd0cb0e61cf8c82e7a9d48478bc8e8c087f51a"
Sep 12 00:34:19.826463 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:34:19.826446    2571 scope.go:117] "RemoveContainer" containerID="a1dcfd8cf7c461a326224d4103bd0cb0e61cf8c82e7a9d48478bc8e8c087f51a"
Sep 12 00:34:19.826764 ip-10-0-141-128 kubenswrapper[2571]: E0912 00:34:19.826746    2571 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a1dcfd8cf7c461a326224d4103bd0cb0e61cf8c82e7a9d48478bc8e8c087f51a\": container with ID starting with a1dcfd8cf7c461a326224d4103bd0cb0e61cf8c82e7a9d48478bc8e8c087f51a not found: ID does not exist" containerID="a1dcfd8cf7c461a326224d4103bd0cb0e61cf8c82e7a9d48478bc8e8c087f51a"
Sep 12 00:34:19.826814 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:34:19.826770    2571 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a1dcfd8cf7c461a326224d4103bd0cb0e61cf8c82e7a9d48478bc8e8c087f51a"} err="failed to get container status \"a1dcfd8cf7c461a326224d4103bd0cb0e61cf8c82e7a9d48478bc8e8c087f51a\": rpc error: code = NotFound desc = could not find container \"a1dcfd8cf7c461a326224d4103bd0cb0e61cf8c82e7a9d48478bc8e8c087f51a\": container with ID starting with a1dcfd8cf7c461a326224d4103bd0cb0e61cf8c82e7a9d48478bc8e8c087f51a not found: ID does not exist"
Sep 12 00:34:19.831179 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:34:19.831159    2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-1b1c80d8-c55659db4-txtpz"]
Sep 12 00:34:19.835912 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:34:19.835889    2571 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-1b1c80d8-c55659db4-txtpz"]
Sep 12 00:34:19.852294 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:34:19.852264    2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-1b1c80d8-57bf44c4f4-lrk22"]
Sep 12 00:34:19.863051 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:34:19.863027    2571 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-1b1c80d8-57bf44c4f4-lrk22"]
Sep 12 00:34:21.141407 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:34:21.141374    2571 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="af68fd51-f790-4e42-842d-30b8118a4b5e" path="/var/lib/kubelet/pods/af68fd51-f790-4e42-842d-30b8118a4b5e/volumes"
Sep 12 00:34:21.141823 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:34:21.141808    2571 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ef90d781-117b-46cb-bed1-cd575cdd3a11" path="/var/lib/kubelet/pods/ef90d781-117b-46cb-bed1-cd575cdd3a11/volumes"
Sep 12 00:34:25.087838 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:34:25.087753    2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-migrate-e6d0c6f2-57c7f74fc4-mnk9s"]
Sep 12 00:34:25.088418 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:34:25.088398    2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="af68fd51-f790-4e42-842d-30b8118a4b5e" containerName="payload-pre-processing"
Sep 12 00:34:25.088505 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:34:25.088420    2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="af68fd51-f790-4e42-842d-30b8118a4b5e" containerName="payload-pre-processing"
Sep 12 00:34:25.088505 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:34:25.088432    2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ef90d781-117b-46cb-bed1-cd575cdd3a11" containerName="maas-api"
Sep 12 00:34:25.088505 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:34:25.088439    2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="ef90d781-117b-46cb-bed1-cd575cdd3a11" containerName="maas-api"
Sep 12 00:34:25.088685 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:34:25.088549    2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="af68fd51-f790-4e42-842d-30b8118a4b5e" containerName="payload-pre-processing"
Sep 12 00:34:25.088685 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:34:25.088564    2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="ef90d781-117b-46cb-bed1-cd575cdd3a11" containerName="maas-api"
Sep 12 00:34:25.090839 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:34:25.090813    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-migrate-e6d0c6f2-57c7f74fc4-mnk9s"
Sep 12 00:34:25.093296 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:34:25.093272    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-migrate-e6d0c6f2\""
Sep 12 00:34:25.093375 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:34:25.093272    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-migrate-e6d0c6f2-dockercfg-7rswm\""
Sep 12 00:34:25.100410 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:34:25.100387    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-migrate-e6d0c6f2-57c7f74fc4-mnk9s"]
Sep 12 00:34:25.184827 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:34:25.184795    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-44fh2\" (UniqueName: \"kubernetes.io/projected/6f9922f6-d585-4315-8d14-fa75ba1fb143-kube-api-access-44fh2\") pod \"payload-pre-processing-e2e-migrate-e6d0c6f2-57c7f74fc4-mnk9s\" (UID: \"6f9922f6-d585-4315-8d14-fa75ba1fb143\") " pod="openshift-ingress/payload-pre-processing-e2e-migrate-e6d0c6f2-57c7f74fc4-mnk9s"
Sep 12 00:34:25.185016 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:34:25.184914    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/6f9922f6-d585-4315-8d14-fa75ba1fb143-plugins-config-volume\") pod \"payload-pre-processing-e2e-migrate-e6d0c6f2-57c7f74fc4-mnk9s\" (UID: \"6f9922f6-d585-4315-8d14-fa75ba1fb143\") " pod="openshift-ingress/payload-pre-processing-e2e-migrate-e6d0c6f2-57c7f74fc4-mnk9s"
Sep 12 00:34:25.285859 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:34:25.285823    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/6f9922f6-d585-4315-8d14-fa75ba1fb143-plugins-config-volume\") pod \"payload-pre-processing-e2e-migrate-e6d0c6f2-57c7f74fc4-mnk9s\" (UID: \"6f9922f6-d585-4315-8d14-fa75ba1fb143\") " pod="openshift-ingress/payload-pre-processing-e2e-migrate-e6d0c6f2-57c7f74fc4-mnk9s"
Sep 12 00:34:25.286038 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:34:25.285999    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-44fh2\" (UniqueName: \"kubernetes.io/projected/6f9922f6-d585-4315-8d14-fa75ba1fb143-kube-api-access-44fh2\") pod \"payload-pre-processing-e2e-migrate-e6d0c6f2-57c7f74fc4-mnk9s\" (UID: \"6f9922f6-d585-4315-8d14-fa75ba1fb143\") " pod="openshift-ingress/payload-pre-processing-e2e-migrate-e6d0c6f2-57c7f74fc4-mnk9s"
Sep 12 00:34:25.286552 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:34:25.286528    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/6f9922f6-d585-4315-8d14-fa75ba1fb143-plugins-config-volume\") pod \"payload-pre-processing-e2e-migrate-e6d0c6f2-57c7f74fc4-mnk9s\" (UID: \"6f9922f6-d585-4315-8d14-fa75ba1fb143\") " pod="openshift-ingress/payload-pre-processing-e2e-migrate-e6d0c6f2-57c7f74fc4-mnk9s"
Sep 12 00:34:25.293491 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:34:25.293470    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-44fh2\" (UniqueName: \"kubernetes.io/projected/6f9922f6-d585-4315-8d14-fa75ba1fb143-kube-api-access-44fh2\") pod \"payload-pre-processing-e2e-migrate-e6d0c6f2-57c7f74fc4-mnk9s\" (UID: \"6f9922f6-d585-4315-8d14-fa75ba1fb143\") " pod="openshift-ingress/payload-pre-processing-e2e-migrate-e6d0c6f2-57c7f74fc4-mnk9s"
Sep 12 00:34:25.402750 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:34:25.402667    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-migrate-e6d0c6f2-57c7f74fc4-mnk9s"
Sep 12 00:34:25.681170 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:34:25.681146    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-migrate-e6d0c6f2-57c7f74fc4-mnk9s"]
Sep 12 00:34:25.682869 ip-10-0-141-128 kubenswrapper[2571]: W0912 00:34:25.682846    2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6f9922f6_d585_4315_8d14_fa75ba1fb143.slice/crio-ee1116a6a50236612d19e7aa96ff2504694257079d1ab25486ffd767bc5ab52c WatchSource:0}: Error finding container ee1116a6a50236612d19e7aa96ff2504694257079d1ab25486ffd767bc5ab52c: Status 404 returned error can't find the container with id ee1116a6a50236612d19e7aa96ff2504694257079d1ab25486ffd767bc5ab52c
Sep 12 00:34:25.843937 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:34:25.843894    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-migrate-e6d0c6f2-57c7f74fc4-mnk9s" event={"ID":"6f9922f6-d585-4315-8d14-fa75ba1fb143","Type":"ContainerStarted","Data":"1de110a2b31b7f139783dba08c769aea9e36fea502132450c86b55e9de425937"}
Sep 12 00:34:25.844117 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:34:25.843959    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-migrate-e6d0c6f2-57c7f74fc4-mnk9s" event={"ID":"6f9922f6-d585-4315-8d14-fa75ba1fb143","Type":"ContainerStarted","Data":"ee1116a6a50236612d19e7aa96ff2504694257079d1ab25486ffd767bc5ab52c"}
Sep 12 00:34:25.844117 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:34:25.843989    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-migrate-e6d0c6f2-57c7f74fc4-mnk9s"
Sep 12 00:34:25.860077 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:34:25.860029    2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-migrate-e6d0c6f2-57c7f74fc4-mnk9s" podStartSLOduration=0.860014619 podStartE2EDuration="860.014619ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-12 00:34:25 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-12 00:34:25.857869416 +0000 UTC m=+1077.129510002" watchObservedRunningTime="2026-09-12 00:34:25.860014619 +0000 UTC m=+1077.131655168"
Sep 12 00:34:36.851287 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:34:36.851254    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-migrate-e6d0c6f2-57c7f74fc4-mnk9s"
Sep 12 00:34:55.240037 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:34:55.240004    2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-migrate-e6d0c6f2-57c7f74fc4-mnk9s"]
Sep 12 00:34:55.240670 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:34:55.240242    2571 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-migrate-e6d0c6f2-57c7f74fc4-mnk9s" podUID="6f9922f6-d585-4315-8d14-fa75ba1fb143" containerName="payload-pre-processing" containerID="cri-o://1de110a2b31b7f139783dba08c769aea9e36fea502132450c86b55e9de425937" gracePeriod=30
Sep 12 00:34:55.524054 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:34:55.524022    2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-migrate-e6d0c6f2-57c7f74fc4-mnk9s"
Sep 12 00:34:55.590688 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:34:55.590654    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/6f9922f6-d585-4315-8d14-fa75ba1fb143-plugins-config-volume\") pod \"6f9922f6-d585-4315-8d14-fa75ba1fb143\" (UID: \"6f9922f6-d585-4315-8d14-fa75ba1fb143\") "
Sep 12 00:34:55.590873 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:34:55.590746    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-44fh2\" (UniqueName: \"kubernetes.io/projected/6f9922f6-d585-4315-8d14-fa75ba1fb143-kube-api-access-44fh2\") pod \"6f9922f6-d585-4315-8d14-fa75ba1fb143\" (UID: \"6f9922f6-d585-4315-8d14-fa75ba1fb143\") "
Sep 12 00:34:55.591062 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:34:55.591035    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6f9922f6-d585-4315-8d14-fa75ba1fb143-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "6f9922f6-d585-4315-8d14-fa75ba1fb143" (UID: "6f9922f6-d585-4315-8d14-fa75ba1fb143"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 12 00:34:55.592828 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:34:55.592795    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6f9922f6-d585-4315-8d14-fa75ba1fb143-kube-api-access-44fh2" (OuterVolumeSpecName: "kube-api-access-44fh2") pod "6f9922f6-d585-4315-8d14-fa75ba1fb143" (UID: "6f9922f6-d585-4315-8d14-fa75ba1fb143"). InnerVolumeSpecName "kube-api-access-44fh2". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 12 00:34:55.692039 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:34:55.692004    2571 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/6f9922f6-d585-4315-8d14-fa75ba1fb143-plugins-config-volume\") on node \"ip-10-0-141-128.ec2.internal\" DevicePath \"\""
Sep 12 00:34:55.692039 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:34:55.692032    2571 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-44fh2\" (UniqueName: \"kubernetes.io/projected/6f9922f6-d585-4315-8d14-fa75ba1fb143-kube-api-access-44fh2\") on node \"ip-10-0-141-128.ec2.internal\" DevicePath \"\""
Sep 12 00:34:56.007511 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:34:56.007478    2571 generic.go:356] "Generic (PLEG): container finished" podID="6f9922f6-d585-4315-8d14-fa75ba1fb143" containerID="1de110a2b31b7f139783dba08c769aea9e36fea502132450c86b55e9de425937" exitCode=0
Sep 12 00:34:56.007715 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:34:56.007523    2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-migrate-e6d0c6f2-57c7f74fc4-mnk9s"
Sep 12 00:34:56.007715 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:34:56.007567    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-migrate-e6d0c6f2-57c7f74fc4-mnk9s" event={"ID":"6f9922f6-d585-4315-8d14-fa75ba1fb143","Type":"ContainerDied","Data":"1de110a2b31b7f139783dba08c769aea9e36fea502132450c86b55e9de425937"}
Sep 12 00:34:56.007715 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:34:56.007652    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-migrate-e6d0c6f2-57c7f74fc4-mnk9s" event={"ID":"6f9922f6-d585-4315-8d14-fa75ba1fb143","Type":"ContainerDied","Data":"ee1116a6a50236612d19e7aa96ff2504694257079d1ab25486ffd767bc5ab52c"}
Sep 12 00:34:56.007715 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:34:56.007682    2571 scope.go:117] "RemoveContainer" containerID="1de110a2b31b7f139783dba08c769aea9e36fea502132450c86b55e9de425937"
Sep 12 00:34:56.017213 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:34:56.017197    2571 scope.go:117] "RemoveContainer" containerID="1de110a2b31b7f139783dba08c769aea9e36fea502132450c86b55e9de425937"
Sep 12 00:34:56.017448 ip-10-0-141-128 kubenswrapper[2571]: E0912 00:34:56.017426    2571 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1de110a2b31b7f139783dba08c769aea9e36fea502132450c86b55e9de425937\": container with ID starting with 1de110a2b31b7f139783dba08c769aea9e36fea502132450c86b55e9de425937 not found: ID does not exist" containerID="1de110a2b31b7f139783dba08c769aea9e36fea502132450c86b55e9de425937"
Sep 12 00:34:56.017515 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:34:56.017459    2571 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1de110a2b31b7f139783dba08c769aea9e36fea502132450c86b55e9de425937"} err="failed to get container status \"1de110a2b31b7f139783dba08c769aea9e36fea502132450c86b55e9de425937\": rpc error: code = NotFound desc = could not find container \"1de110a2b31b7f139783dba08c769aea9e36fea502132450c86b55e9de425937\": container with ID starting with 1de110a2b31b7f139783dba08c769aea9e36fea502132450c86b55e9de425937 not found: ID does not exist"
Sep 12 00:34:56.028531 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:34:56.028503    2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-migrate-e6d0c6f2-57c7f74fc4-mnk9s"]
Sep 12 00:34:56.032025 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:34:56.032005    2571 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-migrate-e6d0c6f2-57c7f74fc4-mnk9s"]
Sep 12 00:34:57.141831 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:34:57.141803    2571 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6f9922f6-d585-4315-8d14-fa75ba1fb143" path="/var/lib/kubelet/pods/6f9922f6-d585-4315-8d14-fa75ba1fb143/volumes"
Sep 12 00:35:33.994721 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:35:33.994682    2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-derive-9c414efe-openshift-default-69dd7c69b8-ll5vt"]
Sep 12 00:35:33.995477 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:35:33.995453    2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6f9922f6-d585-4315-8d14-fa75ba1fb143" containerName="payload-pre-processing"
Sep 12 00:35:33.995523 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:35:33.995482    2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="6f9922f6-d585-4315-8d14-fa75ba1fb143" containerName="payload-pre-processing"
Sep 12 00:35:33.995669 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:35:33.995653    2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="6f9922f6-d585-4315-8d14-fa75ba1fb143" containerName="payload-pre-processing"
Sep 12 00:35:34.000519 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:35:34.000496    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-9c414efe-openshift-default-69dd7c69b8-ll5vt"
Sep 12 00:35:34.003472 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:35:34.003453    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-derive-9c414efe-openshift-default-dockercfg-87465\""
Sep 12 00:35:34.012405 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:35:34.012378    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-derive-9c414efe-openshift-default-69dd7c69b8-ll5vt"]
Sep 12 00:35:34.069778 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:35:34.069749    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6f71f6b9-0aa8-4baa-bb5d-4ab9046e4ced-workload-certs\") pod \"e2e-derive-9c414efe-openshift-default-69dd7c69b8-ll5vt\" (UID: \"6f71f6b9-0aa8-4baa-bb5d-4ab9046e4ced\") " pod="openshift-ingress/e2e-derive-9c414efe-openshift-default-69dd7c69b8-ll5vt"
Sep 12 00:35:34.069930 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:35:34.069791    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6f71f6b9-0aa8-4baa-bb5d-4ab9046e4ced-credential-socket\") pod \"e2e-derive-9c414efe-openshift-default-69dd7c69b8-ll5vt\" (UID: \"6f71f6b9-0aa8-4baa-bb5d-4ab9046e4ced\") " pod="openshift-ingress/e2e-derive-9c414efe-openshift-default-69dd7c69b8-ll5vt"
Sep 12 00:35:34.069930 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:35:34.069895    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6f71f6b9-0aa8-4baa-bb5d-4ab9046e4ced-istio-data\") pod \"e2e-derive-9c414efe-openshift-default-69dd7c69b8-ll5vt\" (UID: \"6f71f6b9-0aa8-4baa-bb5d-4ab9046e4ced\") " pod="openshift-ingress/e2e-derive-9c414efe-openshift-default-69dd7c69b8-ll5vt"
Sep 12 00:35:34.070045 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:35:34.069948    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6f71f6b9-0aa8-4baa-bb5d-4ab9046e4ced-istio-podinfo\") pod \"e2e-derive-9c414efe-openshift-default-69dd7c69b8-ll5vt\" (UID: \"6f71f6b9-0aa8-4baa-bb5d-4ab9046e4ced\") " pod="openshift-ingress/e2e-derive-9c414efe-openshift-default-69dd7c69b8-ll5vt"
Sep 12 00:35:34.070045 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:35:34.070016    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-scdhz\" (UniqueName: \"kubernetes.io/projected/6f71f6b9-0aa8-4baa-bb5d-4ab9046e4ced-kube-api-access-scdhz\") pod \"e2e-derive-9c414efe-openshift-default-69dd7c69b8-ll5vt\" (UID: \"6f71f6b9-0aa8-4baa-bb5d-4ab9046e4ced\") " pod="openshift-ingress/e2e-derive-9c414efe-openshift-default-69dd7c69b8-ll5vt"
Sep 12 00:35:34.070224 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:35:34.070090    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6f71f6b9-0aa8-4baa-bb5d-4ab9046e4ced-workload-socket\") pod \"e2e-derive-9c414efe-openshift-default-69dd7c69b8-ll5vt\" (UID: \"6f71f6b9-0aa8-4baa-bb5d-4ab9046e4ced\") " pod="openshift-ingress/e2e-derive-9c414efe-openshift-default-69dd7c69b8-ll5vt"
Sep 12 00:35:34.070224 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:35:34.070126    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6f71f6b9-0aa8-4baa-bb5d-4ab9046e4ced-istio-envoy\") pod \"e2e-derive-9c414efe-openshift-default-69dd7c69b8-ll5vt\" (UID: \"6f71f6b9-0aa8-4baa-bb5d-4ab9046e4ced\") " pod="openshift-ingress/e2e-derive-9c414efe-openshift-default-69dd7c69b8-ll5vt"
Sep 12 00:35:34.070224 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:35:34.070148    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6f71f6b9-0aa8-4baa-bb5d-4ab9046e4ced-istiod-ca-cert\") pod \"e2e-derive-9c414efe-openshift-default-69dd7c69b8-ll5vt\" (UID: \"6f71f6b9-0aa8-4baa-bb5d-4ab9046e4ced\") " pod="openshift-ingress/e2e-derive-9c414efe-openshift-default-69dd7c69b8-ll5vt"
Sep 12 00:35:34.070224 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:35:34.070179    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6f71f6b9-0aa8-4baa-bb5d-4ab9046e4ced-istio-token\") pod \"e2e-derive-9c414efe-openshift-default-69dd7c69b8-ll5vt\" (UID: \"6f71f6b9-0aa8-4baa-bb5d-4ab9046e4ced\") " pod="openshift-ingress/e2e-derive-9c414efe-openshift-default-69dd7c69b8-ll5vt"
Sep 12 00:35:34.171537 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:35:34.171503    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6f71f6b9-0aa8-4baa-bb5d-4ab9046e4ced-credential-socket\") pod \"e2e-derive-9c414efe-openshift-default-69dd7c69b8-ll5vt\" (UID: \"6f71f6b9-0aa8-4baa-bb5d-4ab9046e4ced\") " pod="openshift-ingress/e2e-derive-9c414efe-openshift-default-69dd7c69b8-ll5vt"
Sep 12 00:35:34.171736 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:35:34.171574    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6f71f6b9-0aa8-4baa-bb5d-4ab9046e4ced-istio-data\") pod \"e2e-derive-9c414efe-openshift-default-69dd7c69b8-ll5vt\" (UID: \"6f71f6b9-0aa8-4baa-bb5d-4ab9046e4ced\") " pod="openshift-ingress/e2e-derive-9c414efe-openshift-default-69dd7c69b8-ll5vt"
Sep 12 00:35:34.171736 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:35:34.171625    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6f71f6b9-0aa8-4baa-bb5d-4ab9046e4ced-istio-podinfo\") pod \"e2e-derive-9c414efe-openshift-default-69dd7c69b8-ll5vt\" (UID: \"6f71f6b9-0aa8-4baa-bb5d-4ab9046e4ced\") " pod="openshift-ingress/e2e-derive-9c414efe-openshift-default-69dd7c69b8-ll5vt"
Sep 12 00:35:34.171736 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:35:34.171651    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-scdhz\" (UniqueName: \"kubernetes.io/projected/6f71f6b9-0aa8-4baa-bb5d-4ab9046e4ced-kube-api-access-scdhz\") pod \"e2e-derive-9c414efe-openshift-default-69dd7c69b8-ll5vt\" (UID: \"6f71f6b9-0aa8-4baa-bb5d-4ab9046e4ced\") " pod="openshift-ingress/e2e-derive-9c414efe-openshift-default-69dd7c69b8-ll5vt"
Sep 12 00:35:34.171736 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:35:34.171718    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6f71f6b9-0aa8-4baa-bb5d-4ab9046e4ced-workload-socket\") pod \"e2e-derive-9c414efe-openshift-default-69dd7c69b8-ll5vt\" (UID: \"6f71f6b9-0aa8-4baa-bb5d-4ab9046e4ced\") " pod="openshift-ingress/e2e-derive-9c414efe-openshift-default-69dd7c69b8-ll5vt"
Sep 12 00:35:34.171963 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:35:34.171755    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6f71f6b9-0aa8-4baa-bb5d-4ab9046e4ced-istio-envoy\") pod \"e2e-derive-9c414efe-openshift-default-69dd7c69b8-ll5vt\" (UID: \"6f71f6b9-0aa8-4baa-bb5d-4ab9046e4ced\") " pod="openshift-ingress/e2e-derive-9c414efe-openshift-default-69dd7c69b8-ll5vt"
Sep 12 00:35:34.171963 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:35:34.171776    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6f71f6b9-0aa8-4baa-bb5d-4ab9046e4ced-istiod-ca-cert\") pod \"e2e-derive-9c414efe-openshift-default-69dd7c69b8-ll5vt\" (UID: \"6f71f6b9-0aa8-4baa-bb5d-4ab9046e4ced\") " pod="openshift-ingress/e2e-derive-9c414efe-openshift-default-69dd7c69b8-ll5vt"
Sep 12 00:35:34.171963 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:35:34.171802    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6f71f6b9-0aa8-4baa-bb5d-4ab9046e4ced-istio-token\") pod \"e2e-derive-9c414efe-openshift-default-69dd7c69b8-ll5vt\" (UID: \"6f71f6b9-0aa8-4baa-bb5d-4ab9046e4ced\") " pod="openshift-ingress/e2e-derive-9c414efe-openshift-default-69dd7c69b8-ll5vt"
Sep 12 00:35:34.171963 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:35:34.171841    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6f71f6b9-0aa8-4baa-bb5d-4ab9046e4ced-workload-certs\") pod \"e2e-derive-9c414efe-openshift-default-69dd7c69b8-ll5vt\" (UID: \"6f71f6b9-0aa8-4baa-bb5d-4ab9046e4ced\") " pod="openshift-ingress/e2e-derive-9c414efe-openshift-default-69dd7c69b8-ll5vt"
Sep 12 00:35:34.172170 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:35:34.171990    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6f71f6b9-0aa8-4baa-bb5d-4ab9046e4ced-credential-socket\") pod \"e2e-derive-9c414efe-openshift-default-69dd7c69b8-ll5vt\" (UID: \"6f71f6b9-0aa8-4baa-bb5d-4ab9046e4ced\") " pod="openshift-ingress/e2e-derive-9c414efe-openshift-default-69dd7c69b8-ll5vt"
Sep 12 00:35:34.172170 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:35:34.172044    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6f71f6b9-0aa8-4baa-bb5d-4ab9046e4ced-istio-data\") pod \"e2e-derive-9c414efe-openshift-default-69dd7c69b8-ll5vt\" (UID: \"6f71f6b9-0aa8-4baa-bb5d-4ab9046e4ced\") " pod="openshift-ingress/e2e-derive-9c414efe-openshift-default-69dd7c69b8-ll5vt"
Sep 12 00:35:34.172170 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:35:34.172123    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6f71f6b9-0aa8-4baa-bb5d-4ab9046e4ced-workload-certs\") pod \"e2e-derive-9c414efe-openshift-default-69dd7c69b8-ll5vt\" (UID: \"6f71f6b9-0aa8-4baa-bb5d-4ab9046e4ced\") " pod="openshift-ingress/e2e-derive-9c414efe-openshift-default-69dd7c69b8-ll5vt"
Sep 12 00:35:34.172738 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:35:34.172713    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6f71f6b9-0aa8-4baa-bb5d-4ab9046e4ced-istiod-ca-cert\") pod \"e2e-derive-9c414efe-openshift-default-69dd7c69b8-ll5vt\" (UID: \"6f71f6b9-0aa8-4baa-bb5d-4ab9046e4ced\") " pod="openshift-ingress/e2e-derive-9c414efe-openshift-default-69dd7c69b8-ll5vt"
Sep 12 00:35:34.172922 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:35:34.172726    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6f71f6b9-0aa8-4baa-bb5d-4ab9046e4ced-workload-socket\") pod \"e2e-derive-9c414efe-openshift-default-69dd7c69b8-ll5vt\" (UID: \"6f71f6b9-0aa8-4baa-bb5d-4ab9046e4ced\") " pod="openshift-ingress/e2e-derive-9c414efe-openshift-default-69dd7c69b8-ll5vt"
Sep 12 00:35:34.174938 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:35:34.174915    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6f71f6b9-0aa8-4baa-bb5d-4ab9046e4ced-istio-envoy\") pod \"e2e-derive-9c414efe-openshift-default-69dd7c69b8-ll5vt\" (UID: \"6f71f6b9-0aa8-4baa-bb5d-4ab9046e4ced\") " pod="openshift-ingress/e2e-derive-9c414efe-openshift-default-69dd7c69b8-ll5vt"
Sep 12 00:35:34.175066 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:35:34.175043    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6f71f6b9-0aa8-4baa-bb5d-4ab9046e4ced-istio-podinfo\") pod \"e2e-derive-9c414efe-openshift-default-69dd7c69b8-ll5vt\" (UID: \"6f71f6b9-0aa8-4baa-bb5d-4ab9046e4ced\") " pod="openshift-ingress/e2e-derive-9c414efe-openshift-default-69dd7c69b8-ll5vt"
Sep 12 00:35:34.181069 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:35:34.181045    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6f71f6b9-0aa8-4baa-bb5d-4ab9046e4ced-istio-token\") pod \"e2e-derive-9c414efe-openshift-default-69dd7c69b8-ll5vt\" (UID: \"6f71f6b9-0aa8-4baa-bb5d-4ab9046e4ced\") " pod="openshift-ingress/e2e-derive-9c414efe-openshift-default-69dd7c69b8-ll5vt"
Sep 12 00:35:34.181342 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:35:34.181318    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-scdhz\" (UniqueName: \"kubernetes.io/projected/6f71f6b9-0aa8-4baa-bb5d-4ab9046e4ced-kube-api-access-scdhz\") pod \"e2e-derive-9c414efe-openshift-default-69dd7c69b8-ll5vt\" (UID: \"6f71f6b9-0aa8-4baa-bb5d-4ab9046e4ced\") " pod="openshift-ingress/e2e-derive-9c414efe-openshift-default-69dd7c69b8-ll5vt"
Sep 12 00:35:34.313816 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:35:34.313738    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-9c414efe-openshift-default-69dd7c69b8-ll5vt"
Sep 12 00:35:34.591801 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:35:34.591775    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-derive-9c414efe-openshift-default-69dd7c69b8-ll5vt"]
Sep 12 00:35:34.593961 ip-10-0-141-128 kubenswrapper[2571]: W0912 00:35:34.593932    2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6f71f6b9_0aa8_4baa_bb5d_4ab9046e4ced.slice/crio-7dd72bc7ee342813e694b01ef4eeeb77c54e3a634f88350d5b283cbb9a6375c8 WatchSource:0}: Error finding container 7dd72bc7ee342813e694b01ef4eeeb77c54e3a634f88350d5b283cbb9a6375c8: Status 404 returned error can't find the container with id 7dd72bc7ee342813e694b01ef4eeeb77c54e3a634f88350d5b283cbb9a6375c8
Sep 12 00:35:34.596661 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:35:34.596631    2571 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236196Ki","pods":"250"}
Sep 12 00:35:34.596742 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:35:34.596700    2571 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236196Ki","pods":"250"}
Sep 12 00:35:34.596742 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:35:34.596730    2571 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236196Ki","pods":"250"}
Sep 12 00:35:35.220561 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:35:35.220526    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-9c414efe-openshift-default-69dd7c69b8-ll5vt" event={"ID":"6f71f6b9-0aa8-4baa-bb5d-4ab9046e4ced","Type":"ContainerStarted","Data":"e5db2b62ed2bf6fc58be6592353f3cd4abab5f6d0fc9bb2b48734e98e30e5e40"}
Sep 12 00:35:35.220934 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:35:35.220579    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-9c414efe-openshift-default-69dd7c69b8-ll5vt" event={"ID":"6f71f6b9-0aa8-4baa-bb5d-4ab9046e4ced","Type":"ContainerStarted","Data":"7dd72bc7ee342813e694b01ef4eeeb77c54e3a634f88350d5b283cbb9a6375c8"}
Sep 12 00:35:35.241107 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:35:35.241063    2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-derive-9c414efe-openshift-default-69dd7c69b8-ll5vt" podStartSLOduration=2.241047812 podStartE2EDuration="2.241047812s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-12 00:35:33 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-12 00:35:35.238278416 +0000 UTC m=+1146.509918926" watchObservedRunningTime="2026-09-12 00:35:35.241047812 +0000 UTC m=+1146.512688321"
Sep 12 00:35:35.314390 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:35:35.314360    2571 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-derive-9c414efe-openshift-default-69dd7c69b8-ll5vt"
Sep 12 00:35:35.319497 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:35:35.319474    2571 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-derive-9c414efe-openshift-default-69dd7c69b8-ll5vt"
Sep 12 00:35:36.225548 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:35:36.225513    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-derive-9c414efe-openshift-default-69dd7c69b8-ll5vt"
Sep 12 00:35:36.226529 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:35:36.226508    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-derive-9c414efe-openshift-default-69dd7c69b8-ll5vt"
Sep 12 00:35:37.022919 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:35:37.022885    2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-autores-4aaf1a-869d96c798-2mg2b"]
Sep 12 00:35:37.023306 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:35:37.023124    2571 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-autores-4aaf1a-869d96c798-2mg2b" podUID="4ba72a97-317b-438a-a952-938cf6e75a07" containerName="maas-api" containerID="cri-o://50b44b9f613543147d5767d7212f5ca456c065375829074938128188c5c53c62" gracePeriod=30
Sep 12 00:35:37.234453 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:35:37.233960    2571 generic.go:356] "Generic (PLEG): container finished" podID="4ba72a97-317b-438a-a952-938cf6e75a07" containerID="50b44b9f613543147d5767d7212f5ca456c065375829074938128188c5c53c62" exitCode=0
Sep 12 00:35:37.234453 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:35:37.234105    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-autores-4aaf1a-869d96c798-2mg2b" event={"ID":"4ba72a97-317b-438a-a952-938cf6e75a07","Type":"ContainerDied","Data":"50b44b9f613543147d5767d7212f5ca456c065375829074938128188c5c53c62"}
Sep 12 00:35:37.329521 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:35:37.329499    2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-autores-4aaf1a-869d96c798-2mg2b"
Sep 12 00:35:37.405255 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:35:37.405215    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/4ba72a97-317b-438a-a952-938cf6e75a07-openshift-service-ca\") pod \"4ba72a97-317b-438a-a952-938cf6e75a07\" (UID: \"4ba72a97-317b-438a-a952-938cf6e75a07\") "
Sep 12 00:35:37.405435 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:35:37.405316    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/4ba72a97-317b-438a-a952-938cf6e75a07-maas-api-tls\") pod \"4ba72a97-317b-438a-a952-938cf6e75a07\" (UID: \"4ba72a97-317b-438a-a952-938cf6e75a07\") "
Sep 12 00:35:37.405512 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:35:37.405460    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/4ba72a97-317b-438a-a952-938cf6e75a07-metrics-certs\") pod \"4ba72a97-317b-438a-a952-938cf6e75a07\" (UID: \"4ba72a97-317b-438a-a952-938cf6e75a07\") "
Sep 12 00:35:37.405575 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:35:37.405516    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-n8dhz\" (UniqueName: \"kubernetes.io/projected/4ba72a97-317b-438a-a952-938cf6e75a07-kube-api-access-n8dhz\") pod \"4ba72a97-317b-438a-a952-938cf6e75a07\" (UID: \"4ba72a97-317b-438a-a952-938cf6e75a07\") "
Sep 12 00:35:37.405658 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:35:37.405624    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4ba72a97-317b-438a-a952-938cf6e75a07-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "4ba72a97-317b-438a-a952-938cf6e75a07" (UID: "4ba72a97-317b-438a-a952-938cf6e75a07"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 12 00:35:37.405982 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:35:37.405959    2571 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/4ba72a97-317b-438a-a952-938cf6e75a07-openshift-service-ca\") on node \"ip-10-0-141-128.ec2.internal\" DevicePath \"\""
Sep 12 00:35:37.407971 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:35:37.407942    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4ba72a97-317b-438a-a952-938cf6e75a07-kube-api-access-n8dhz" (OuterVolumeSpecName: "kube-api-access-n8dhz") pod "4ba72a97-317b-438a-a952-938cf6e75a07" (UID: "4ba72a97-317b-438a-a952-938cf6e75a07"). InnerVolumeSpecName "kube-api-access-n8dhz". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 12 00:35:37.408110 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:35:37.408070    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4ba72a97-317b-438a-a952-938cf6e75a07-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "4ba72a97-317b-438a-a952-938cf6e75a07" (UID: "4ba72a97-317b-438a-a952-938cf6e75a07"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 12 00:35:37.408110 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:35:37.408086    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4ba72a97-317b-438a-a952-938cf6e75a07-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "4ba72a97-317b-438a-a952-938cf6e75a07" (UID: "4ba72a97-317b-438a-a952-938cf6e75a07"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 12 00:35:37.506783 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:35:37.506752    2571 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/4ba72a97-317b-438a-a952-938cf6e75a07-metrics-certs\") on node \"ip-10-0-141-128.ec2.internal\" DevicePath \"\""
Sep 12 00:35:37.506783 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:35:37.506780    2571 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-n8dhz\" (UniqueName: \"kubernetes.io/projected/4ba72a97-317b-438a-a952-938cf6e75a07-kube-api-access-n8dhz\") on node \"ip-10-0-141-128.ec2.internal\" DevicePath \"\""
Sep 12 00:35:37.506783 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:35:37.506789    2571 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/4ba72a97-317b-438a-a952-938cf6e75a07-maas-api-tls\") on node \"ip-10-0-141-128.ec2.internal\" DevicePath \"\""
Sep 12 00:35:38.240783 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:35:38.240743    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-autores-4aaf1a-869d96c798-2mg2b" event={"ID":"4ba72a97-317b-438a-a952-938cf6e75a07","Type":"ContainerDied","Data":"e78ab1649601ecc277706157b418ce22621830039ff7990fa413d6af7064a523"}
Sep 12 00:35:38.241162 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:35:38.240820    2571 scope.go:117] "RemoveContainer" containerID="50b44b9f613543147d5767d7212f5ca456c065375829074938128188c5c53c62"
Sep 12 00:35:38.241162 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:35:38.240763    2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-autores-4aaf1a-869d96c798-2mg2b"
Sep 12 00:35:38.263101 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:35:38.263067    2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-autores-4aaf1a-869d96c798-2mg2b"]
Sep 12 00:35:38.269101 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:35:38.269070    2571 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-autores-4aaf1a-869d96c798-2mg2b"]
Sep 12 00:35:39.143375 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:35:39.143345    2571 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4ba72a97-317b-438a-a952-938cf6e75a07" path="/var/lib/kubelet/pods/4ba72a97-317b-438a-a952-938cf6e75a07/volumes"
Sep 12 00:36:05.404106 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:36:05.404076    2571 generic.go:356] "Generic (PLEG): container finished" podID="6f71f6b9-0aa8-4baa-bb5d-4ab9046e4ced" containerID="e5db2b62ed2bf6fc58be6592353f3cd4abab5f6d0fc9bb2b48734e98e30e5e40" exitCode=0
Sep 12 00:36:05.404642 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:36:05.404150    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-9c414efe-openshift-default-69dd7c69b8-ll5vt" event={"ID":"6f71f6b9-0aa8-4baa-bb5d-4ab9046e4ced","Type":"ContainerDied","Data":"e5db2b62ed2bf6fc58be6592353f3cd4abab5f6d0fc9bb2b48734e98e30e5e40"}
Sep 12 00:36:05.404724 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:36:05.404677    2571 scope.go:117] "RemoveContainer" containerID="e5db2b62ed2bf6fc58be6592353f3cd4abab5f6d0fc9bb2b48734e98e30e5e40"
Sep 12 00:36:05.405920 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:36:05.405895    2571 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236196Ki","pods":"250"}
Sep 12 00:36:05.405996 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:36:05.405958    2571 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236196Ki","pods":"250"}
Sep 12 00:36:05.405996 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:36:05.405990    2571 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236196Ki","pods":"250"}
Sep 12 00:36:06.225951 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:36:06.225922    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-derive-9c414efe-openshift-default-69dd7c69b8-ll5vt"
Sep 12 00:36:06.411250 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:36:06.411220    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-9c414efe-openshift-default-69dd7c69b8-ll5vt" event={"ID":"6f71f6b9-0aa8-4baa-bb5d-4ab9046e4ced","Type":"ContainerStarted","Data":"8e0de551e5661751bb625f26a85c36b1bc4039e6e615cd79915d7f8b3ca41d0d"}
Sep 12 00:36:07.314943 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:36:07.314909    2571 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-derive-9c414efe-openshift-default-69dd7c69b8-ll5vt"
Sep 12 00:36:07.316108 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:36:07.316085    2571 patch_prober.go:28] interesting pod/e2e-derive-9c414efe-openshift-default-69dd7c69b8-ll5vt container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.52:15021/healthz/ready\": dial tcp 10.132.0.52:15021: connect: connection refused" start-of-body=
Sep 12 00:36:07.316168 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:36:07.316134    2571 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-derive-9c414efe-openshift-default-69dd7c69b8-ll5vt" podUID="6f71f6b9-0aa8-4baa-bb5d-4ab9046e4ced" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.52:15021/healthz/ready\": dial tcp 10.132.0.52:15021: connect: connection refused"
Sep 12 00:36:08.314400 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:36:08.314366    2571 patch_prober.go:28] interesting pod/e2e-derive-9c414efe-openshift-default-69dd7c69b8-ll5vt container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.52:15021/healthz/ready\": dial tcp 10.132.0.52:15021: connect: connection refused" start-of-body=
Sep 12 00:36:08.314817 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:36:08.314422    2571 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-derive-9c414efe-openshift-default-69dd7c69b8-ll5vt" podUID="6f71f6b9-0aa8-4baa-bb5d-4ab9046e4ced" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.52:15021/healthz/ready\": dial tcp 10.132.0.52:15021: connect: connection refused"
Sep 12 00:36:08.795697 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:36:08.795660    2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-explicit-03e1dc-5dff4957c8-mz86p"]
Sep 12 00:36:08.796649 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:36:08.796625    2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4ba72a97-317b-438a-a952-938cf6e75a07" containerName="maas-api"
Sep 12 00:36:08.796803 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:36:08.796791    2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="4ba72a97-317b-438a-a952-938cf6e75a07" containerName="maas-api"
Sep 12 00:36:08.797038 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:36:08.797025    2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="4ba72a97-317b-438a-a952-938cf6e75a07" containerName="maas-api"
Sep 12 00:36:08.800324 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:36:08.800292    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-explicit-03e1dc-5dff4957c8-mz86p"]
Sep 12 00:36:08.800442 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:36:08.800403    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-explicit-03e1dc-5dff4957c8-mz86p"
Sep 12 00:36:08.802882 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:36:08.802861    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-explicit-03e1dc-dockercfg-hgttx\""
Sep 12 00:36:08.803164 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:36:08.802970    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-explicit-03e1dc\""
Sep 12 00:36:08.909610 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:36:08.907140    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vt79n\" (UniqueName: \"kubernetes.io/projected/8c4c6a4c-963a-49d5-a21e-3a3dc53651e9-kube-api-access-vt79n\") pod \"payload-pre-processing-e2e-explicit-03e1dc-5dff4957c8-mz86p\" (UID: \"8c4c6a4c-963a-49d5-a21e-3a3dc53651e9\") " pod="openshift-ingress/payload-pre-processing-e2e-explicit-03e1dc-5dff4957c8-mz86p"
Sep 12 00:36:08.909610 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:36:08.907229    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/8c4c6a4c-963a-49d5-a21e-3a3dc53651e9-plugins-config-volume\") pod \"payload-pre-processing-e2e-explicit-03e1dc-5dff4957c8-mz86p\" (UID: \"8c4c6a4c-963a-49d5-a21e-3a3dc53651e9\") " pod="openshift-ingress/payload-pre-processing-e2e-explicit-03e1dc-5dff4957c8-mz86p"
Sep 12 00:36:09.007895 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:36:09.007863    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vt79n\" (UniqueName: \"kubernetes.io/projected/8c4c6a4c-963a-49d5-a21e-3a3dc53651e9-kube-api-access-vt79n\") pod \"payload-pre-processing-e2e-explicit-03e1dc-5dff4957c8-mz86p\" (UID: \"8c4c6a4c-963a-49d5-a21e-3a3dc53651e9\") " pod="openshift-ingress/payload-pre-processing-e2e-explicit-03e1dc-5dff4957c8-mz86p"
Sep 12 00:36:09.008141 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:36:09.008127    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/8c4c6a4c-963a-49d5-a21e-3a3dc53651e9-plugins-config-volume\") pod \"payload-pre-processing-e2e-explicit-03e1dc-5dff4957c8-mz86p\" (UID: \"8c4c6a4c-963a-49d5-a21e-3a3dc53651e9\") " pod="openshift-ingress/payload-pre-processing-e2e-explicit-03e1dc-5dff4957c8-mz86p"
Sep 12 00:36:09.008792 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:36:09.008773    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/8c4c6a4c-963a-49d5-a21e-3a3dc53651e9-plugins-config-volume\") pod \"payload-pre-processing-e2e-explicit-03e1dc-5dff4957c8-mz86p\" (UID: \"8c4c6a4c-963a-49d5-a21e-3a3dc53651e9\") " pod="openshift-ingress/payload-pre-processing-e2e-explicit-03e1dc-5dff4957c8-mz86p"
Sep 12 00:36:09.016189 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:36:09.016160    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vt79n\" (UniqueName: \"kubernetes.io/projected/8c4c6a4c-963a-49d5-a21e-3a3dc53651e9-kube-api-access-vt79n\") pod \"payload-pre-processing-e2e-explicit-03e1dc-5dff4957c8-mz86p\" (UID: \"8c4c6a4c-963a-49d5-a21e-3a3dc53651e9\") " pod="openshift-ingress/payload-pre-processing-e2e-explicit-03e1dc-5dff4957c8-mz86p"
Sep 12 00:36:09.113533 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:36:09.113447    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-explicit-03e1dc-5dff4957c8-mz86p"
Sep 12 00:36:09.314581 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:36:09.314546    2571 patch_prober.go:28] interesting pod/e2e-derive-9c414efe-openshift-default-69dd7c69b8-ll5vt container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.52:15021/healthz/ready\": dial tcp 10.132.0.52:15021: connect: connection refused" start-of-body=
Sep 12 00:36:09.314959 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:36:09.314609    2571 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-derive-9c414efe-openshift-default-69dd7c69b8-ll5vt" podUID="6f71f6b9-0aa8-4baa-bb5d-4ab9046e4ced" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.52:15021/healthz/ready\": dial tcp 10.132.0.52:15021: connect: connection refused"
Sep 12 00:36:09.416623 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:36:09.416581    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-explicit-03e1dc-5dff4957c8-mz86p"]
Sep 12 00:36:09.418423 ip-10-0-141-128 kubenswrapper[2571]: W0912 00:36:09.418395    2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8c4c6a4c_963a_49d5_a21e_3a3dc53651e9.slice/crio-f6c494a84b6bb87fb00db1b66578161b2697dbf4e623df2d0847682d1d33c1ba WatchSource:0}: Error finding container f6c494a84b6bb87fb00db1b66578161b2697dbf4e623df2d0847682d1d33c1ba: Status 404 returned error can't find the container with id f6c494a84b6bb87fb00db1b66578161b2697dbf4e623df2d0847682d1d33c1ba
Sep 12 00:36:09.431225 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:36:09.431203    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-explicit-03e1dc-5dff4957c8-mz86p" event={"ID":"8c4c6a4c-963a-49d5-a21e-3a3dc53651e9","Type":"ContainerStarted","Data":"f6c494a84b6bb87fb00db1b66578161b2697dbf4e623df2d0847682d1d33c1ba"}
Sep 12 00:36:10.314224 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:36:10.314186    2571 patch_prober.go:28] interesting pod/e2e-derive-9c414efe-openshift-default-69dd7c69b8-ll5vt container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.52:15021/healthz/ready\": dial tcp 10.132.0.52:15021: connect: connection refused" start-of-body=
Sep 12 00:36:10.314404 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:36:10.314243    2571 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-derive-9c414efe-openshift-default-69dd7c69b8-ll5vt" podUID="6f71f6b9-0aa8-4baa-bb5d-4ab9046e4ced" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.52:15021/healthz/ready\": dial tcp 10.132.0.52:15021: connect: connection refused"
Sep 12 00:36:10.439086 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:36:10.439048    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-explicit-03e1dc-5dff4957c8-mz86p" event={"ID":"8c4c6a4c-963a-49d5-a21e-3a3dc53651e9","Type":"ContainerStarted","Data":"138577faaaed33c6b6436bf86fbb43ffc0f4e9791841e3f31a7c1ff744d553d1"}
Sep 12 00:36:10.439501 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:36:10.439166    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-explicit-03e1dc-5dff4957c8-mz86p"
Sep 12 00:36:10.458687 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:36:10.458640    2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-explicit-03e1dc-5dff4957c8-mz86p" podStartSLOduration=2.458626435 podStartE2EDuration="2.458626435s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-12 00:36:08 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-12 00:36:10.457138851 +0000 UTC m=+1181.728779361" watchObservedRunningTime="2026-09-12 00:36:10.458626435 +0000 UTC m=+1181.730266965"
Sep 12 00:36:11.314196 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:36:11.314160    2571 patch_prober.go:28] interesting pod/e2e-derive-9c414efe-openshift-default-69dd7c69b8-ll5vt container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.52:15021/healthz/ready\": dial tcp 10.132.0.52:15021: connect: connection refused" start-of-body=
Sep 12 00:36:11.314369 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:36:11.314213    2571 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-derive-9c414efe-openshift-default-69dd7c69b8-ll5vt" podUID="6f71f6b9-0aa8-4baa-bb5d-4ab9046e4ced" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.52:15021/healthz/ready\": dial tcp 10.132.0.52:15021: connect: connection refused"
Sep 12 00:36:12.314356 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:36:12.314311    2571 patch_prober.go:28] interesting pod/e2e-derive-9c414efe-openshift-default-69dd7c69b8-ll5vt container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.52:15021/healthz/ready\": dial tcp 10.132.0.52:15021: connect: connection refused" start-of-body=
Sep 12 00:36:12.314872 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:36:12.314370    2571 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-derive-9c414efe-openshift-default-69dd7c69b8-ll5vt" podUID="6f71f6b9-0aa8-4baa-bb5d-4ab9046e4ced" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.52:15021/healthz/ready\": dial tcp 10.132.0.52:15021: connect: connection refused"
Sep 12 00:36:13.318734 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:36:13.318700    2571 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-derive-9c414efe-openshift-default-69dd7c69b8-ll5vt"
Sep 12 00:36:13.319085 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:36:13.319050    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-derive-9c414efe-openshift-default-69dd7c69b8-ll5vt"
Sep 12 00:36:13.319788 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:36:13.319766    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-derive-9c414efe-openshift-default-69dd7c69b8-ll5vt"
Sep 12 00:36:21.446311 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:36:21.446282    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-explicit-03e1dc-5dff4957c8-mz86p"
Sep 12 00:36:29.048375 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:36:29.048333    2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-derive-9c414efe-openshift-default-69dd7c69b8-ll5vt"]
Sep 12 00:36:29.048883 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:36:29.048664    2571 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-derive-9c414efe-openshift-default-69dd7c69b8-ll5vt" podUID="6f71f6b9-0aa8-4baa-bb5d-4ab9046e4ced" containerName="istio-proxy" containerID="cri-o://8e0de551e5661751bb625f26a85c36b1bc4039e6e615cd79915d7f8b3ca41d0d" gracePeriod=30
Sep 12 00:36:34.406434 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:36:34.406406    2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-9c414efe-openshift-default-69dd7c69b8-ll5vt"
Sep 12 00:36:34.557554 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:36:34.557475    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6f71f6b9-0aa8-4baa-bb5d-4ab9046e4ced-istio-podinfo\") pod \"6f71f6b9-0aa8-4baa-bb5d-4ab9046e4ced\" (UID: \"6f71f6b9-0aa8-4baa-bb5d-4ab9046e4ced\") "
Sep 12 00:36:34.557554 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:36:34.557522    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6f71f6b9-0aa8-4baa-bb5d-4ab9046e4ced-istio-token\") pod \"6f71f6b9-0aa8-4baa-bb5d-4ab9046e4ced\" (UID: \"6f71f6b9-0aa8-4baa-bb5d-4ab9046e4ced\") "
Sep 12 00:36:34.557803 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:36:34.557570    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-scdhz\" (UniqueName: \"kubernetes.io/projected/6f71f6b9-0aa8-4baa-bb5d-4ab9046e4ced-kube-api-access-scdhz\") pod \"6f71f6b9-0aa8-4baa-bb5d-4ab9046e4ced\" (UID: \"6f71f6b9-0aa8-4baa-bb5d-4ab9046e4ced\") "
Sep 12 00:36:34.557803 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:36:34.557624    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6f71f6b9-0aa8-4baa-bb5d-4ab9046e4ced-credential-socket\") pod \"6f71f6b9-0aa8-4baa-bb5d-4ab9046e4ced\" (UID: \"6f71f6b9-0aa8-4baa-bb5d-4ab9046e4ced\") "
Sep 12 00:36:34.557803 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:36:34.557695    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6f71f6b9-0aa8-4baa-bb5d-4ab9046e4ced-istiod-ca-cert\") pod \"6f71f6b9-0aa8-4baa-bb5d-4ab9046e4ced\" (UID: \"6f71f6b9-0aa8-4baa-bb5d-4ab9046e4ced\") "
Sep 12 00:36:34.557803 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:36:34.557745    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6f71f6b9-0aa8-4baa-bb5d-4ab9046e4ced-workload-socket\") pod \"6f71f6b9-0aa8-4baa-bb5d-4ab9046e4ced\" (UID: \"6f71f6b9-0aa8-4baa-bb5d-4ab9046e4ced\") "
Sep 12 00:36:34.557803 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:36:34.557785    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6f71f6b9-0aa8-4baa-bb5d-4ab9046e4ced-istio-data\") pod \"6f71f6b9-0aa8-4baa-bb5d-4ab9046e4ced\" (UID: \"6f71f6b9-0aa8-4baa-bb5d-4ab9046e4ced\") "
Sep 12 00:36:34.558051 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:36:34.557871    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6f71f6b9-0aa8-4baa-bb5d-4ab9046e4ced-workload-certs\") pod \"6f71f6b9-0aa8-4baa-bb5d-4ab9046e4ced\" (UID: \"6f71f6b9-0aa8-4baa-bb5d-4ab9046e4ced\") "
Sep 12 00:36:34.558051 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:36:34.557882    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6f71f6b9-0aa8-4baa-bb5d-4ab9046e4ced-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "6f71f6b9-0aa8-4baa-bb5d-4ab9046e4ced" (UID: "6f71f6b9-0aa8-4baa-bb5d-4ab9046e4ced"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 12 00:36:34.558051 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:36:34.557896    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6f71f6b9-0aa8-4baa-bb5d-4ab9046e4ced-istio-envoy\") pod \"6f71f6b9-0aa8-4baa-bb5d-4ab9046e4ced\" (UID: \"6f71f6b9-0aa8-4baa-bb5d-4ab9046e4ced\") "
Sep 12 00:36:34.558051 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:36:34.558012    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6f71f6b9-0aa8-4baa-bb5d-4ab9046e4ced-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "6f71f6b9-0aa8-4baa-bb5d-4ab9046e4ced" (UID: "6f71f6b9-0aa8-4baa-bb5d-4ab9046e4ced"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 12 00:36:34.558257 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:36:34.558066    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6f71f6b9-0aa8-4baa-bb5d-4ab9046e4ced-istio-data" (OuterVolumeSpecName: "istio-data") pod "6f71f6b9-0aa8-4baa-bb5d-4ab9046e4ced" (UID: "6f71f6b9-0aa8-4baa-bb5d-4ab9046e4ced"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 12 00:36:34.558257 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:36:34.558101    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6f71f6b9-0aa8-4baa-bb5d-4ab9046e4ced-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "6f71f6b9-0aa8-4baa-bb5d-4ab9046e4ced" (UID: "6f71f6b9-0aa8-4baa-bb5d-4ab9046e4ced"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 12 00:36:34.558257 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:36:34.558239    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6f71f6b9-0aa8-4baa-bb5d-4ab9046e4ced-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "6f71f6b9-0aa8-4baa-bb5d-4ab9046e4ced" (UID: "6f71f6b9-0aa8-4baa-bb5d-4ab9046e4ced"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 12 00:36:34.558457 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:36:34.558320    2571 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6f71f6b9-0aa8-4baa-bb5d-4ab9046e4ced-workload-certs\") on node \"ip-10-0-141-128.ec2.internal\" DevicePath \"\""
Sep 12 00:36:34.558457 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:36:34.558333    2571 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6f71f6b9-0aa8-4baa-bb5d-4ab9046e4ced-credential-socket\") on node \"ip-10-0-141-128.ec2.internal\" DevicePath \"\""
Sep 12 00:36:34.558457 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:36:34.558343    2571 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6f71f6b9-0aa8-4baa-bb5d-4ab9046e4ced-istiod-ca-cert\") on node \"ip-10-0-141-128.ec2.internal\" DevicePath \"\""
Sep 12 00:36:34.558457 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:36:34.558353    2571 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6f71f6b9-0aa8-4baa-bb5d-4ab9046e4ced-workload-socket\") on node \"ip-10-0-141-128.ec2.internal\" DevicePath \"\""
Sep 12 00:36:34.558457 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:36:34.558361    2571 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6f71f6b9-0aa8-4baa-bb5d-4ab9046e4ced-istio-data\") on node \"ip-10-0-141-128.ec2.internal\" DevicePath \"\""
Sep 12 00:36:34.559822 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:36:34.559796    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6f71f6b9-0aa8-4baa-bb5d-4ab9046e4ced-kube-api-access-scdhz" (OuterVolumeSpecName: "kube-api-access-scdhz") pod "6f71f6b9-0aa8-4baa-bb5d-4ab9046e4ced" (UID: "6f71f6b9-0aa8-4baa-bb5d-4ab9046e4ced"). InnerVolumeSpecName "kube-api-access-scdhz". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 12 00:36:34.559929 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:36:34.559826    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6f71f6b9-0aa8-4baa-bb5d-4ab9046e4ced-istio-token" (OuterVolumeSpecName: "istio-token") pod "6f71f6b9-0aa8-4baa-bb5d-4ab9046e4ced" (UID: "6f71f6b9-0aa8-4baa-bb5d-4ab9046e4ced"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 12 00:36:34.559929 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:36:34.559838    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/6f71f6b9-0aa8-4baa-bb5d-4ab9046e4ced-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "6f71f6b9-0aa8-4baa-bb5d-4ab9046e4ced" (UID: "6f71f6b9-0aa8-4baa-bb5d-4ab9046e4ced"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 12 00:36:34.559929 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:36:34.559901    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6f71f6b9-0aa8-4baa-bb5d-4ab9046e4ced-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "6f71f6b9-0aa8-4baa-bb5d-4ab9046e4ced" (UID: "6f71f6b9-0aa8-4baa-bb5d-4ab9046e4ced"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 12 00:36:34.571423 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:36:34.571399    2571 generic.go:356] "Generic (PLEG): container finished" podID="6f71f6b9-0aa8-4baa-bb5d-4ab9046e4ced" containerID="8e0de551e5661751bb625f26a85c36b1bc4039e6e615cd79915d7f8b3ca41d0d" exitCode=0
Sep 12 00:36:34.571529 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:36:34.571454    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-9c414efe-openshift-default-69dd7c69b8-ll5vt" event={"ID":"6f71f6b9-0aa8-4baa-bb5d-4ab9046e4ced","Type":"ContainerDied","Data":"8e0de551e5661751bb625f26a85c36b1bc4039e6e615cd79915d7f8b3ca41d0d"}
Sep 12 00:36:34.571529 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:36:34.571471    2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-9c414efe-openshift-default-69dd7c69b8-ll5vt"
Sep 12 00:36:34.571529 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:36:34.571510    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-9c414efe-openshift-default-69dd7c69b8-ll5vt" event={"ID":"6f71f6b9-0aa8-4baa-bb5d-4ab9046e4ced","Type":"ContainerDied","Data":"7dd72bc7ee342813e694b01ef4eeeb77c54e3a634f88350d5b283cbb9a6375c8"}
Sep 12 00:36:34.571717 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:36:34.571542    2571 scope.go:117] "RemoveContainer" containerID="8e0de551e5661751bb625f26a85c36b1bc4039e6e615cd79915d7f8b3ca41d0d"
Sep 12 00:36:34.582117 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:36:34.582102    2571 scope.go:117] "RemoveContainer" containerID="e5db2b62ed2bf6fc58be6592353f3cd4abab5f6d0fc9bb2b48734e98e30e5e40"
Sep 12 00:36:34.589770 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:36:34.589756    2571 scope.go:117] "RemoveContainer" containerID="8e0de551e5661751bb625f26a85c36b1bc4039e6e615cd79915d7f8b3ca41d0d"
Sep 12 00:36:34.590054 ip-10-0-141-128 kubenswrapper[2571]: E0912 00:36:34.590034    2571 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8e0de551e5661751bb625f26a85c36b1bc4039e6e615cd79915d7f8b3ca41d0d\": container with ID starting with 8e0de551e5661751bb625f26a85c36b1bc4039e6e615cd79915d7f8b3ca41d0d not found: ID does not exist" containerID="8e0de551e5661751bb625f26a85c36b1bc4039e6e615cd79915d7f8b3ca41d0d"
Sep 12 00:36:34.590127 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:36:34.590062    2571 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8e0de551e5661751bb625f26a85c36b1bc4039e6e615cd79915d7f8b3ca41d0d"} err="failed to get container status \"8e0de551e5661751bb625f26a85c36b1bc4039e6e615cd79915d7f8b3ca41d0d\": rpc error: code = NotFound desc = could not find container \"8e0de551e5661751bb625f26a85c36b1bc4039e6e615cd79915d7f8b3ca41d0d\": container with ID starting with 8e0de551e5661751bb625f26a85c36b1bc4039e6e615cd79915d7f8b3ca41d0d not found: ID does not exist"
Sep 12 00:36:34.590127 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:36:34.590081    2571 scope.go:117] "RemoveContainer" containerID="e5db2b62ed2bf6fc58be6592353f3cd4abab5f6d0fc9bb2b48734e98e30e5e40"
Sep 12 00:36:34.590339 ip-10-0-141-128 kubenswrapper[2571]: E0912 00:36:34.590320    2571 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e5db2b62ed2bf6fc58be6592353f3cd4abab5f6d0fc9bb2b48734e98e30e5e40\": container with ID starting with e5db2b62ed2bf6fc58be6592353f3cd4abab5f6d0fc9bb2b48734e98e30e5e40 not found: ID does not exist" containerID="e5db2b62ed2bf6fc58be6592353f3cd4abab5f6d0fc9bb2b48734e98e30e5e40"
Sep 12 00:36:34.590383 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:36:34.590344    2571 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e5db2b62ed2bf6fc58be6592353f3cd4abab5f6d0fc9bb2b48734e98e30e5e40"} err="failed to get container status \"e5db2b62ed2bf6fc58be6592353f3cd4abab5f6d0fc9bb2b48734e98e30e5e40\": rpc error: code = NotFound desc = could not find container \"e5db2b62ed2bf6fc58be6592353f3cd4abab5f6d0fc9bb2b48734e98e30e5e40\": container with ID starting with e5db2b62ed2bf6fc58be6592353f3cd4abab5f6d0fc9bb2b48734e98e30e5e40 not found: ID does not exist"
Sep 12 00:36:34.597457 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:36:34.597436    2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-derive-9c414efe-openshift-default-69dd7c69b8-ll5vt"]
Sep 12 00:36:34.600461 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:36:34.600441    2571 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-derive-9c414efe-openshift-default-69dd7c69b8-ll5vt"]
Sep 12 00:36:34.659876 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:36:34.659855    2571 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-scdhz\" (UniqueName: \"kubernetes.io/projected/6f71f6b9-0aa8-4baa-bb5d-4ab9046e4ced-kube-api-access-scdhz\") on node \"ip-10-0-141-128.ec2.internal\" DevicePath \"\""
Sep 12 00:36:34.659876 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:36:34.659875    2571 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6f71f6b9-0aa8-4baa-bb5d-4ab9046e4ced-istio-envoy\") on node \"ip-10-0-141-128.ec2.internal\" DevicePath \"\""
Sep 12 00:36:34.659998 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:36:34.659885    2571 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6f71f6b9-0aa8-4baa-bb5d-4ab9046e4ced-istio-podinfo\") on node \"ip-10-0-141-128.ec2.internal\" DevicePath \"\""
Sep 12 00:36:34.659998 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:36:34.659892    2571 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6f71f6b9-0aa8-4baa-bb5d-4ab9046e4ced-istio-token\") on node \"ip-10-0-141-128.ec2.internal\" DevicePath \"\""
Sep 12 00:36:35.144144 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:36:35.144113    2571 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6f71f6b9-0aa8-4baa-bb5d-4ab9046e4ced" path="/var/lib/kubelet/pods/6f71f6b9-0aa8-4baa-bb5d-4ab9046e4ced/volumes"
Sep 12 00:37:39.835400 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:37:39.835364    2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-explicit-03e1dc-5dff4957c8-mz86p"]
Sep 12 00:37:39.836264 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:37:39.836208    2571 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-explicit-03e1dc-5dff4957c8-mz86p" podUID="8c4c6a4c-963a-49d5-a21e-3a3dc53651e9" containerName="payload-pre-processing" containerID="cri-o://138577faaaed33c6b6436bf86fbb43ffc0f4e9791841e3f31a7c1ff744d553d1" gracePeriod=30
Sep 12 00:37:40.111809 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:37:40.111775    2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-explicit-03e1dc-5dff4957c8-mz86p"
Sep 12 00:37:40.175669 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:37:40.175630    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-vt79n\" (UniqueName: \"kubernetes.io/projected/8c4c6a4c-963a-49d5-a21e-3a3dc53651e9-kube-api-access-vt79n\") pod \"8c4c6a4c-963a-49d5-a21e-3a3dc53651e9\" (UID: \"8c4c6a4c-963a-49d5-a21e-3a3dc53651e9\") "
Sep 12 00:37:40.175838 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:37:40.175696    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/8c4c6a4c-963a-49d5-a21e-3a3dc53651e9-plugins-config-volume\") pod \"8c4c6a4c-963a-49d5-a21e-3a3dc53651e9\" (UID: \"8c4c6a4c-963a-49d5-a21e-3a3dc53651e9\") "
Sep 12 00:37:40.176371 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:37:40.176342    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8c4c6a4c-963a-49d5-a21e-3a3dc53651e9-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "8c4c6a4c-963a-49d5-a21e-3a3dc53651e9" (UID: "8c4c6a4c-963a-49d5-a21e-3a3dc53651e9"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 12 00:37:40.189921 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:37:40.189870    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8c4c6a4c-963a-49d5-a21e-3a3dc53651e9-kube-api-access-vt79n" (OuterVolumeSpecName: "kube-api-access-vt79n") pod "8c4c6a4c-963a-49d5-a21e-3a3dc53651e9" (UID: "8c4c6a4c-963a-49d5-a21e-3a3dc53651e9"). InnerVolumeSpecName "kube-api-access-vt79n". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 12 00:37:40.277113 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:37:40.277075    2571 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/8c4c6a4c-963a-49d5-a21e-3a3dc53651e9-plugins-config-volume\") on node \"ip-10-0-141-128.ec2.internal\" DevicePath \"\""
Sep 12 00:37:40.277113 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:37:40.277115    2571 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vt79n\" (UniqueName: \"kubernetes.io/projected/8c4c6a4c-963a-49d5-a21e-3a3dc53651e9-kube-api-access-vt79n\") on node \"ip-10-0-141-128.ec2.internal\" DevicePath \"\""
Sep 12 00:37:40.928374 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:37:40.928339    2571 generic.go:356] "Generic (PLEG): container finished" podID="8c4c6a4c-963a-49d5-a21e-3a3dc53651e9" containerID="138577faaaed33c6b6436bf86fbb43ffc0f4e9791841e3f31a7c1ff744d553d1" exitCode=0
Sep 12 00:37:40.928899 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:37:40.928386    2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-explicit-03e1dc-5dff4957c8-mz86p"
Sep 12 00:37:40.928899 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:37:40.928436    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-explicit-03e1dc-5dff4957c8-mz86p" event={"ID":"8c4c6a4c-963a-49d5-a21e-3a3dc53651e9","Type":"ContainerDied","Data":"138577faaaed33c6b6436bf86fbb43ffc0f4e9791841e3f31a7c1ff744d553d1"}
Sep 12 00:37:40.928899 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:37:40.928498    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-explicit-03e1dc-5dff4957c8-mz86p" event={"ID":"8c4c6a4c-963a-49d5-a21e-3a3dc53651e9","Type":"ContainerDied","Data":"f6c494a84b6bb87fb00db1b66578161b2697dbf4e623df2d0847682d1d33c1ba"}
Sep 12 00:37:40.928899 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:37:40.928520    2571 scope.go:117] "RemoveContainer" containerID="138577faaaed33c6b6436bf86fbb43ffc0f4e9791841e3f31a7c1ff744d553d1"
Sep 12 00:37:40.937962 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:37:40.937945    2571 scope.go:117] "RemoveContainer" containerID="138577faaaed33c6b6436bf86fbb43ffc0f4e9791841e3f31a7c1ff744d553d1"
Sep 12 00:37:40.938234 ip-10-0-141-128 kubenswrapper[2571]: E0912 00:37:40.938216    2571 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"138577faaaed33c6b6436bf86fbb43ffc0f4e9791841e3f31a7c1ff744d553d1\": container with ID starting with 138577faaaed33c6b6436bf86fbb43ffc0f4e9791841e3f31a7c1ff744d553d1 not found: ID does not exist" containerID="138577faaaed33c6b6436bf86fbb43ffc0f4e9791841e3f31a7c1ff744d553d1"
Sep 12 00:37:40.938278 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:37:40.938244    2571 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"138577faaaed33c6b6436bf86fbb43ffc0f4e9791841e3f31a7c1ff744d553d1"} err="failed to get container status \"138577faaaed33c6b6436bf86fbb43ffc0f4e9791841e3f31a7c1ff744d553d1\": rpc error: code = NotFound desc = could not find container \"138577faaaed33c6b6436bf86fbb43ffc0f4e9791841e3f31a7c1ff744d553d1\": container with ID starting with 138577faaaed33c6b6436bf86fbb43ffc0f4e9791841e3f31a7c1ff744d553d1 not found: ID does not exist"
Sep 12 00:37:40.954565 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:37:40.954537    2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-explicit-03e1dc-5dff4957c8-mz86p"]
Sep 12 00:37:40.956493 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:37:40.956474    2571 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-explicit-03e1dc-5dff4957c8-mz86p"]
Sep 12 00:37:41.140810 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:37:41.140782    2571 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8c4c6a4c-963a-49d5-a21e-3a3dc53651e9" path="/var/lib/kubelet/pods/8c4c6a4c-963a-49d5-a21e-3a3dc53651e9/volumes"
Sep 12 00:37:47.228032 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:37:47.227992    2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-noten-0a0781-openshift-default-86b89df9c8-rc2kf"]
Sep 12 00:37:47.228628 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:37:47.228474    2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8c4c6a4c-963a-49d5-a21e-3a3dc53651e9" containerName="payload-pre-processing"
Sep 12 00:37:47.228628 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:37:47.228488    2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="8c4c6a4c-963a-49d5-a21e-3a3dc53651e9" containerName="payload-pre-processing"
Sep 12 00:37:47.228628 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:37:47.228498    2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6f71f6b9-0aa8-4baa-bb5d-4ab9046e4ced" containerName="istio-proxy"
Sep 12 00:37:47.228628 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:37:47.228504    2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="6f71f6b9-0aa8-4baa-bb5d-4ab9046e4ced" containerName="istio-proxy"
Sep 12 00:37:47.228628 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:37:47.228526    2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6f71f6b9-0aa8-4baa-bb5d-4ab9046e4ced" containerName="istio-proxy"
Sep 12 00:37:47.228628 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:37:47.228531    2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="6f71f6b9-0aa8-4baa-bb5d-4ab9046e4ced" containerName="istio-proxy"
Sep 12 00:37:47.228628 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:37:47.228609    2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="8c4c6a4c-963a-49d5-a21e-3a3dc53651e9" containerName="payload-pre-processing"
Sep 12 00:37:47.228628 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:37:47.228620    2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="6f71f6b9-0aa8-4baa-bb5d-4ab9046e4ced" containerName="istio-proxy"
Sep 12 00:37:47.228628 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:37:47.228628    2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="6f71f6b9-0aa8-4baa-bb5d-4ab9046e4ced" containerName="istio-proxy"
Sep 12 00:37:47.233232 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:37:47.233209    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-noten-0a0781-openshift-default-86b89df9c8-rc2kf"
Sep 12 00:37:47.235892 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:37:47.235870    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-noten-0a0781-openshift-default-dockercfg-ft28l\""
Sep 12 00:37:47.243009 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:37:47.242985    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-noten-0a0781-openshift-default-86b89df9c8-rc2kf"]
Sep 12 00:37:47.347139 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:37:47.347095    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3d1a958a-0902-4d78-ba0e-e5b829ce347c-istio-envoy\") pod \"e2e-noten-0a0781-openshift-default-86b89df9c8-rc2kf\" (UID: \"3d1a958a-0902-4d78-ba0e-e5b829ce347c\") " pod="openshift-ingress/e2e-noten-0a0781-openshift-default-86b89df9c8-rc2kf"
Sep 12 00:37:47.347346 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:37:47.347148    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3d1a958a-0902-4d78-ba0e-e5b829ce347c-workload-socket\") pod \"e2e-noten-0a0781-openshift-default-86b89df9c8-rc2kf\" (UID: \"3d1a958a-0902-4d78-ba0e-e5b829ce347c\") " pod="openshift-ingress/e2e-noten-0a0781-openshift-default-86b89df9c8-rc2kf"
Sep 12 00:37:47.347346 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:37:47.347178    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3d1a958a-0902-4d78-ba0e-e5b829ce347c-istio-data\") pod \"e2e-noten-0a0781-openshift-default-86b89df9c8-rc2kf\" (UID: \"3d1a958a-0902-4d78-ba0e-e5b829ce347c\") " pod="openshift-ingress/e2e-noten-0a0781-openshift-default-86b89df9c8-rc2kf"
Sep 12 00:37:47.347346 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:37:47.347208    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ldvqc\" (UniqueName: \"kubernetes.io/projected/3d1a958a-0902-4d78-ba0e-e5b829ce347c-kube-api-access-ldvqc\") pod \"e2e-noten-0a0781-openshift-default-86b89df9c8-rc2kf\" (UID: \"3d1a958a-0902-4d78-ba0e-e5b829ce347c\") " pod="openshift-ingress/e2e-noten-0a0781-openshift-default-86b89df9c8-rc2kf"
Sep 12 00:37:47.347346 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:37:47.347241    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3d1a958a-0902-4d78-ba0e-e5b829ce347c-workload-certs\") pod \"e2e-noten-0a0781-openshift-default-86b89df9c8-rc2kf\" (UID: \"3d1a958a-0902-4d78-ba0e-e5b829ce347c\") " pod="openshift-ingress/e2e-noten-0a0781-openshift-default-86b89df9c8-rc2kf"
Sep 12 00:37:47.347346 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:37:47.347269    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3d1a958a-0902-4d78-ba0e-e5b829ce347c-credential-socket\") pod \"e2e-noten-0a0781-openshift-default-86b89df9c8-rc2kf\" (UID: \"3d1a958a-0902-4d78-ba0e-e5b829ce347c\") " pod="openshift-ingress/e2e-noten-0a0781-openshift-default-86b89df9c8-rc2kf"
Sep 12 00:37:47.347346 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:37:47.347290    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3d1a958a-0902-4d78-ba0e-e5b829ce347c-istiod-ca-cert\") pod \"e2e-noten-0a0781-openshift-default-86b89df9c8-rc2kf\" (UID: \"3d1a958a-0902-4d78-ba0e-e5b829ce347c\") " pod="openshift-ingress/e2e-noten-0a0781-openshift-default-86b89df9c8-rc2kf"
Sep 12 00:37:47.347346 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:37:47.347332    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3d1a958a-0902-4d78-ba0e-e5b829ce347c-istio-token\") pod \"e2e-noten-0a0781-openshift-default-86b89df9c8-rc2kf\" (UID: \"3d1a958a-0902-4d78-ba0e-e5b829ce347c\") " pod="openshift-ingress/e2e-noten-0a0781-openshift-default-86b89df9c8-rc2kf"
Sep 12 00:37:47.347778 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:37:47.347483    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3d1a958a-0902-4d78-ba0e-e5b829ce347c-istio-podinfo\") pod \"e2e-noten-0a0781-openshift-default-86b89df9c8-rc2kf\" (UID: \"3d1a958a-0902-4d78-ba0e-e5b829ce347c\") " pod="openshift-ingress/e2e-noten-0a0781-openshift-default-86b89df9c8-rc2kf"
Sep 12 00:37:47.448639 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:37:47.448578    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3d1a958a-0902-4d78-ba0e-e5b829ce347c-workload-certs\") pod \"e2e-noten-0a0781-openshift-default-86b89df9c8-rc2kf\" (UID: \"3d1a958a-0902-4d78-ba0e-e5b829ce347c\") " pod="openshift-ingress/e2e-noten-0a0781-openshift-default-86b89df9c8-rc2kf"
Sep 12 00:37:47.448639 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:37:47.448644    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3d1a958a-0902-4d78-ba0e-e5b829ce347c-credential-socket\") pod \"e2e-noten-0a0781-openshift-default-86b89df9c8-rc2kf\" (UID: \"3d1a958a-0902-4d78-ba0e-e5b829ce347c\") " pod="openshift-ingress/e2e-noten-0a0781-openshift-default-86b89df9c8-rc2kf"
Sep 12 00:37:47.448890 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:37:47.448671    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3d1a958a-0902-4d78-ba0e-e5b829ce347c-istiod-ca-cert\") pod \"e2e-noten-0a0781-openshift-default-86b89df9c8-rc2kf\" (UID: \"3d1a958a-0902-4d78-ba0e-e5b829ce347c\") " pod="openshift-ingress/e2e-noten-0a0781-openshift-default-86b89df9c8-rc2kf"
Sep 12 00:37:47.448890 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:37:47.448711    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3d1a958a-0902-4d78-ba0e-e5b829ce347c-istio-token\") pod \"e2e-noten-0a0781-openshift-default-86b89df9c8-rc2kf\" (UID: \"3d1a958a-0902-4d78-ba0e-e5b829ce347c\") " pod="openshift-ingress/e2e-noten-0a0781-openshift-default-86b89df9c8-rc2kf"
Sep 12 00:37:47.448890 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:37:47.448846    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3d1a958a-0902-4d78-ba0e-e5b829ce347c-istio-podinfo\") pod \"e2e-noten-0a0781-openshift-default-86b89df9c8-rc2kf\" (UID: \"3d1a958a-0902-4d78-ba0e-e5b829ce347c\") " pod="openshift-ingress/e2e-noten-0a0781-openshift-default-86b89df9c8-rc2kf"
Sep 12 00:37:47.448890 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:37:47.448883    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3d1a958a-0902-4d78-ba0e-e5b829ce347c-istio-envoy\") pod \"e2e-noten-0a0781-openshift-default-86b89df9c8-rc2kf\" (UID: \"3d1a958a-0902-4d78-ba0e-e5b829ce347c\") " pod="openshift-ingress/e2e-noten-0a0781-openshift-default-86b89df9c8-rc2kf"
Sep 12 00:37:47.449092 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:37:47.448902    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3d1a958a-0902-4d78-ba0e-e5b829ce347c-workload-socket\") pod \"e2e-noten-0a0781-openshift-default-86b89df9c8-rc2kf\" (UID: \"3d1a958a-0902-4d78-ba0e-e5b829ce347c\") " pod="openshift-ingress/e2e-noten-0a0781-openshift-default-86b89df9c8-rc2kf"
Sep 12 00:37:47.449092 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:37:47.448920    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3d1a958a-0902-4d78-ba0e-e5b829ce347c-istio-data\") pod \"e2e-noten-0a0781-openshift-default-86b89df9c8-rc2kf\" (UID: \"3d1a958a-0902-4d78-ba0e-e5b829ce347c\") " pod="openshift-ingress/e2e-noten-0a0781-openshift-default-86b89df9c8-rc2kf"
Sep 12 00:37:47.449092 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:37:47.448941    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ldvqc\" (UniqueName: \"kubernetes.io/projected/3d1a958a-0902-4d78-ba0e-e5b829ce347c-kube-api-access-ldvqc\") pod \"e2e-noten-0a0781-openshift-default-86b89df9c8-rc2kf\" (UID: \"3d1a958a-0902-4d78-ba0e-e5b829ce347c\") " pod="openshift-ingress/e2e-noten-0a0781-openshift-default-86b89df9c8-rc2kf"
Sep 12 00:37:47.449092 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:37:47.449038    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3d1a958a-0902-4d78-ba0e-e5b829ce347c-workload-certs\") pod \"e2e-noten-0a0781-openshift-default-86b89df9c8-rc2kf\" (UID: \"3d1a958a-0902-4d78-ba0e-e5b829ce347c\") " pod="openshift-ingress/e2e-noten-0a0781-openshift-default-86b89df9c8-rc2kf"
Sep 12 00:37:47.449092 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:37:47.449046    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3d1a958a-0902-4d78-ba0e-e5b829ce347c-credential-socket\") pod \"e2e-noten-0a0781-openshift-default-86b89df9c8-rc2kf\" (UID: \"3d1a958a-0902-4d78-ba0e-e5b829ce347c\") " pod="openshift-ingress/e2e-noten-0a0781-openshift-default-86b89df9c8-rc2kf"
Sep 12 00:37:47.449469 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:37:47.449445    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3d1a958a-0902-4d78-ba0e-e5b829ce347c-workload-socket\") pod \"e2e-noten-0a0781-openshift-default-86b89df9c8-rc2kf\" (UID: \"3d1a958a-0902-4d78-ba0e-e5b829ce347c\") " pod="openshift-ingress/e2e-noten-0a0781-openshift-default-86b89df9c8-rc2kf"
Sep 12 00:37:47.449544 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:37:47.449500    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3d1a958a-0902-4d78-ba0e-e5b829ce347c-istio-data\") pod \"e2e-noten-0a0781-openshift-default-86b89df9c8-rc2kf\" (UID: \"3d1a958a-0902-4d78-ba0e-e5b829ce347c\") " pod="openshift-ingress/e2e-noten-0a0781-openshift-default-86b89df9c8-rc2kf"
Sep 12 00:37:47.449738 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:37:47.449715    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3d1a958a-0902-4d78-ba0e-e5b829ce347c-istiod-ca-cert\") pod \"e2e-noten-0a0781-openshift-default-86b89df9c8-rc2kf\" (UID: \"3d1a958a-0902-4d78-ba0e-e5b829ce347c\") " pod="openshift-ingress/e2e-noten-0a0781-openshift-default-86b89df9c8-rc2kf"
Sep 12 00:37:47.451581 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:37:47.451555    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3d1a958a-0902-4d78-ba0e-e5b829ce347c-istio-envoy\") pod \"e2e-noten-0a0781-openshift-default-86b89df9c8-rc2kf\" (UID: \"3d1a958a-0902-4d78-ba0e-e5b829ce347c\") " pod="openshift-ingress/e2e-noten-0a0781-openshift-default-86b89df9c8-rc2kf"
Sep 12 00:37:47.451773 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:37:47.451750    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3d1a958a-0902-4d78-ba0e-e5b829ce347c-istio-podinfo\") pod \"e2e-noten-0a0781-openshift-default-86b89df9c8-rc2kf\" (UID: \"3d1a958a-0902-4d78-ba0e-e5b829ce347c\") " pod="openshift-ingress/e2e-noten-0a0781-openshift-default-86b89df9c8-rc2kf"
Sep 12 00:37:47.458295 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:37:47.458274    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ldvqc\" (UniqueName: \"kubernetes.io/projected/3d1a958a-0902-4d78-ba0e-e5b829ce347c-kube-api-access-ldvqc\") pod \"e2e-noten-0a0781-openshift-default-86b89df9c8-rc2kf\" (UID: \"3d1a958a-0902-4d78-ba0e-e5b829ce347c\") " pod="openshift-ingress/e2e-noten-0a0781-openshift-default-86b89df9c8-rc2kf"
Sep 12 00:37:47.458547 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:37:47.458527    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3d1a958a-0902-4d78-ba0e-e5b829ce347c-istio-token\") pod \"e2e-noten-0a0781-openshift-default-86b89df9c8-rc2kf\" (UID: \"3d1a958a-0902-4d78-ba0e-e5b829ce347c\") " pod="openshift-ingress/e2e-noten-0a0781-openshift-default-86b89df9c8-rc2kf"
Sep 12 00:37:47.547317 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:37:47.547238    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-noten-0a0781-openshift-default-86b89df9c8-rc2kf"
Sep 12 00:37:47.826270 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:37:47.826242    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-noten-0a0781-openshift-default-86b89df9c8-rc2kf"]
Sep 12 00:37:47.828580 ip-10-0-141-128 kubenswrapper[2571]: W0912 00:37:47.828549    2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3d1a958a_0902_4d78_ba0e_e5b829ce347c.slice/crio-e96d466f03eb586c324b039059790d482c9adeed60ef8ad5eea923e75319cbbf WatchSource:0}: Error finding container e96d466f03eb586c324b039059790d482c9adeed60ef8ad5eea923e75319cbbf: Status 404 returned error can't find the container with id e96d466f03eb586c324b039059790d482c9adeed60ef8ad5eea923e75319cbbf
Sep 12 00:37:47.830980 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:37:47.830939    2571 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236196Ki","pods":"250"}
Sep 12 00:37:47.831075 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:37:47.831035    2571 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236196Ki","pods":"250"}
Sep 12 00:37:47.831134 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:37:47.831102    2571 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236196Ki","pods":"250"}
Sep 12 00:37:47.969473 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:37:47.969432    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-noten-0a0781-openshift-default-86b89df9c8-rc2kf" event={"ID":"3d1a958a-0902-4d78-ba0e-e5b829ce347c","Type":"ContainerStarted","Data":"281538e4ffb0316c6123a8e101401ed9997d604db2bd1b85038666e9f0eac6ff"}
Sep 12 00:37:47.969654 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:37:47.969508    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-noten-0a0781-openshift-default-86b89df9c8-rc2kf" event={"ID":"3d1a958a-0902-4d78-ba0e-e5b829ce347c","Type":"ContainerStarted","Data":"e96d466f03eb586c324b039059790d482c9adeed60ef8ad5eea923e75319cbbf"}
Sep 12 00:37:47.989533 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:37:47.989483    2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-noten-0a0781-openshift-default-86b89df9c8-rc2kf" podStartSLOduration=0.989467038 podStartE2EDuration="989.467038ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-12 00:37:47 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-12 00:37:47.987327239 +0000 UTC m=+1279.258967749" watchObservedRunningTime="2026-09-12 00:37:47.989467038 +0000 UTC m=+1279.261107547"
Sep 12 00:37:48.548270 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:37:48.548230    2571 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-noten-0a0781-openshift-default-86b89df9c8-rc2kf"
Sep 12 00:37:48.553642 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:37:48.553617    2571 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-noten-0a0781-openshift-default-86b89df9c8-rc2kf"
Sep 12 00:37:48.974466 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:37:48.974437    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-noten-0a0781-openshift-default-86b89df9c8-rc2kf"
Sep 12 00:37:48.975261 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:37:48.975244    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-noten-0a0781-openshift-default-86b89df9c8-rc2kf"
Sep 12 00:38:11.273129 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:38:11.273093    2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-d370bb1a-85cf464cdf-mgcsn"]
Sep 12 00:38:11.276948 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:38:11.276931    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-d370bb1a-85cf464cdf-mgcsn"
Sep 12 00:38:11.279527 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:38:11.279504    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-mt-d370bb1a-dockercfg-n4m2r\""
Sep 12 00:38:11.279671 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:38:11.279547    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-mt-d370bb1a\""
Sep 12 00:38:11.290860 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:38:11.290827    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2r4sw\" (UniqueName: \"kubernetes.io/projected/60c04ecd-1559-4438-93df-26749642613c-kube-api-access-2r4sw\") pod \"payload-processing-e2e-mt-d370bb1a-85cf464cdf-mgcsn\" (UID: \"60c04ecd-1559-4438-93df-26749642613c\") " pod="openshift-ingress/payload-processing-e2e-mt-d370bb1a-85cf464cdf-mgcsn"
Sep 12 00:38:11.290974 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:38:11.290883    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/60c04ecd-1559-4438-93df-26749642613c-plugins-config-volume\") pod \"payload-processing-e2e-mt-d370bb1a-85cf464cdf-mgcsn\" (UID: \"60c04ecd-1559-4438-93df-26749642613c\") " pod="openshift-ingress/payload-processing-e2e-mt-d370bb1a-85cf464cdf-mgcsn"
Sep 12 00:38:11.291306 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:38:11.291284    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-d370bb1a-85cf464cdf-mgcsn"]
Sep 12 00:38:11.391444 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:38:11.391416    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2r4sw\" (UniqueName: \"kubernetes.io/projected/60c04ecd-1559-4438-93df-26749642613c-kube-api-access-2r4sw\") pod \"payload-processing-e2e-mt-d370bb1a-85cf464cdf-mgcsn\" (UID: \"60c04ecd-1559-4438-93df-26749642613c\") " pod="openshift-ingress/payload-processing-e2e-mt-d370bb1a-85cf464cdf-mgcsn"
Sep 12 00:38:11.391624 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:38:11.391454    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/60c04ecd-1559-4438-93df-26749642613c-plugins-config-volume\") pod \"payload-processing-e2e-mt-d370bb1a-85cf464cdf-mgcsn\" (UID: \"60c04ecd-1559-4438-93df-26749642613c\") " pod="openshift-ingress/payload-processing-e2e-mt-d370bb1a-85cf464cdf-mgcsn"
Sep 12 00:38:11.392146 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:38:11.392121    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/60c04ecd-1559-4438-93df-26749642613c-plugins-config-volume\") pod \"payload-processing-e2e-mt-d370bb1a-85cf464cdf-mgcsn\" (UID: \"60c04ecd-1559-4438-93df-26749642613c\") " pod="openshift-ingress/payload-processing-e2e-mt-d370bb1a-85cf464cdf-mgcsn"
Sep 12 00:38:11.399146 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:38:11.399123    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2r4sw\" (UniqueName: \"kubernetes.io/projected/60c04ecd-1559-4438-93df-26749642613c-kube-api-access-2r4sw\") pod \"payload-processing-e2e-mt-d370bb1a-85cf464cdf-mgcsn\" (UID: \"60c04ecd-1559-4438-93df-26749642613c\") " pod="openshift-ingress/payload-processing-e2e-mt-d370bb1a-85cf464cdf-mgcsn"
Sep 12 00:38:11.588923 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:38:11.588824    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-d370bb1a-85cf464cdf-mgcsn"
Sep 12 00:38:11.865517 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:38:11.865487    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-d370bb1a-85cf464cdf-mgcsn"]
Sep 12 00:38:11.867659 ip-10-0-141-128 kubenswrapper[2571]: W0912 00:38:11.867624    2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod60c04ecd_1559_4438_93df_26749642613c.slice/crio-e50e840327a8c48d8d1eea4315f16f33c5c61a93e8c1e8587fe00b0d3ac3de4e WatchSource:0}: Error finding container e50e840327a8c48d8d1eea4315f16f33c5c61a93e8c1e8587fe00b0d3ac3de4e: Status 404 returned error can't find the container with id e50e840327a8c48d8d1eea4315f16f33c5c61a93e8c1e8587fe00b0d3ac3de4e
Sep 12 00:38:12.101220 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:38:12.101186    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-d370bb1a-85cf464cdf-mgcsn" event={"ID":"60c04ecd-1559-4438-93df-26749642613c","Type":"ContainerStarted","Data":"1aa365dc2a9d7d32488e067e92cc50445424551b2591c87ba36673e0c2b7e7d2"}
Sep 12 00:38:12.101404 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:38:12.101351    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-mt-d370bb1a-85cf464cdf-mgcsn"
Sep 12 00:38:12.101404 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:38:12.101383    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-d370bb1a-85cf464cdf-mgcsn" event={"ID":"60c04ecd-1559-4438-93df-26749642613c","Type":"ContainerStarted","Data":"e50e840327a8c48d8d1eea4315f16f33c5c61a93e8c1e8587fe00b0d3ac3de4e"}
Sep 12 00:38:12.117834 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:38:12.117740    2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-mt-d370bb1a-85cf464cdf-mgcsn" podStartSLOduration=1.117721348 podStartE2EDuration="1.117721348s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-12 00:38:11 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-12 00:38:12.116693999 +0000 UTC m=+1303.388334522" watchObservedRunningTime="2026-09-12 00:38:12.117721348 +0000 UTC m=+1303.389361859"
Sep 12 00:38:23.108565 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:38:23.108537    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-mt-d370bb1a-85cf464cdf-mgcsn"
Sep 12 00:38:50.477794 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:38:50.477759    2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-d370bb1a-85cf464cdf-mgcsn"]
Sep 12 00:38:50.478390 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:38:50.478017    2571 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-mt-d370bb1a-85cf464cdf-mgcsn" podUID="60c04ecd-1559-4438-93df-26749642613c" containerName="payload-processing" containerID="cri-o://1aa365dc2a9d7d32488e067e92cc50445424551b2591c87ba36673e0c2b7e7d2" gracePeriod=30
Sep 12 00:38:50.759497 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:38:50.759464    2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-d370bb1a-85cf464cdf-mgcsn"
Sep 12 00:38:50.771228 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:38:50.771202    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/60c04ecd-1559-4438-93df-26749642613c-plugins-config-volume\") pod \"60c04ecd-1559-4438-93df-26749642613c\" (UID: \"60c04ecd-1559-4438-93df-26749642613c\") "
Sep 12 00:38:50.771401 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:38:50.771244    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-2r4sw\" (UniqueName: \"kubernetes.io/projected/60c04ecd-1559-4438-93df-26749642613c-kube-api-access-2r4sw\") pod \"60c04ecd-1559-4438-93df-26749642613c\" (UID: \"60c04ecd-1559-4438-93df-26749642613c\") "
Sep 12 00:38:50.771628 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:38:50.771601    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/60c04ecd-1559-4438-93df-26749642613c-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "60c04ecd-1559-4438-93df-26749642613c" (UID: "60c04ecd-1559-4438-93df-26749642613c"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 12 00:38:50.773566 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:38:50.773536    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/60c04ecd-1559-4438-93df-26749642613c-kube-api-access-2r4sw" (OuterVolumeSpecName: "kube-api-access-2r4sw") pod "60c04ecd-1559-4438-93df-26749642613c" (UID: "60c04ecd-1559-4438-93df-26749642613c"). InnerVolumeSpecName "kube-api-access-2r4sw". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 12 00:38:50.873666 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:38:50.873633    2571 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/60c04ecd-1559-4438-93df-26749642613c-plugins-config-volume\") on node \"ip-10-0-141-128.ec2.internal\" DevicePath \"\""
Sep 12 00:38:50.873666 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:38:50.873665    2571 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2r4sw\" (UniqueName: \"kubernetes.io/projected/60c04ecd-1559-4438-93df-26749642613c-kube-api-access-2r4sw\") on node \"ip-10-0-141-128.ec2.internal\" DevicePath \"\""
Sep 12 00:38:51.309107 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:38:51.309073    2571 generic.go:356] "Generic (PLEG): container finished" podID="60c04ecd-1559-4438-93df-26749642613c" containerID="1aa365dc2a9d7d32488e067e92cc50445424551b2591c87ba36673e0c2b7e7d2" exitCode=0
Sep 12 00:38:51.309253 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:38:51.309118    2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-d370bb1a-85cf464cdf-mgcsn"
Sep 12 00:38:51.309253 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:38:51.309128    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-d370bb1a-85cf464cdf-mgcsn" event={"ID":"60c04ecd-1559-4438-93df-26749642613c","Type":"ContainerDied","Data":"1aa365dc2a9d7d32488e067e92cc50445424551b2591c87ba36673e0c2b7e7d2"}
Sep 12 00:38:51.309253 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:38:51.309185    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-d370bb1a-85cf464cdf-mgcsn" event={"ID":"60c04ecd-1559-4438-93df-26749642613c","Type":"ContainerDied","Data":"e50e840327a8c48d8d1eea4315f16f33c5c61a93e8c1e8587fe00b0d3ac3de4e"}
Sep 12 00:38:51.309253 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:38:51.309205    2571 scope.go:117] "RemoveContainer" containerID="1aa365dc2a9d7d32488e067e92cc50445424551b2591c87ba36673e0c2b7e7d2"
Sep 12 00:38:51.317863 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:38:51.317848    2571 scope.go:117] "RemoveContainer" containerID="1aa365dc2a9d7d32488e067e92cc50445424551b2591c87ba36673e0c2b7e7d2"
Sep 12 00:38:51.318101 ip-10-0-141-128 kubenswrapper[2571]: E0912 00:38:51.318085    2571 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1aa365dc2a9d7d32488e067e92cc50445424551b2591c87ba36673e0c2b7e7d2\": container with ID starting with 1aa365dc2a9d7d32488e067e92cc50445424551b2591c87ba36673e0c2b7e7d2 not found: ID does not exist" containerID="1aa365dc2a9d7d32488e067e92cc50445424551b2591c87ba36673e0c2b7e7d2"
Sep 12 00:38:51.318147 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:38:51.318108    2571 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1aa365dc2a9d7d32488e067e92cc50445424551b2591c87ba36673e0c2b7e7d2"} err="failed to get container status \"1aa365dc2a9d7d32488e067e92cc50445424551b2591c87ba36673e0c2b7e7d2\": rpc error: code = NotFound desc = could not find container \"1aa365dc2a9d7d32488e067e92cc50445424551b2591c87ba36673e0c2b7e7d2\": container with ID starting with 1aa365dc2a9d7d32488e067e92cc50445424551b2591c87ba36673e0c2b7e7d2 not found: ID does not exist"
Sep 12 00:38:51.332053 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:38:51.332019    2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-d370bb1a-85cf464cdf-mgcsn"]
Sep 12 00:38:51.333934 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:38:51.333912    2571 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-d370bb1a-85cf464cdf-mgcsn"]
Sep 12 00:38:53.141476 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:38:53.141407    2571 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="60c04ecd-1559-4438-93df-26749642613c" path="/var/lib/kubelet/pods/60c04ecd-1559-4438-93df-26749642613c/volumes"
Sep 12 00:39:10.532465 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:39:10.532433    2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-shared-a-w2-0d3a66-openshift-default-6c7595fdd6-f92vh"]
Sep 12 00:39:10.532954 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:39:10.532940    2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="60c04ecd-1559-4438-93df-26749642613c" containerName="payload-processing"
Sep 12 00:39:10.533004 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:39:10.532957    2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="60c04ecd-1559-4438-93df-26749642613c" containerName="payload-processing"
Sep 12 00:39:10.533057 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:39:10.533047    2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="60c04ecd-1559-4438-93df-26749642613c" containerName="payload-processing"
Sep 12 00:39:10.535838 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:39:10.535820    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-w2-0d3a66-openshift-default-6c7595fdd6-f92vh"
Sep 12 00:39:10.538795 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:39:10.538772    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-shared-a-w2-0d3a66-openshift-default-dockercfg-lmcmk\""
Sep 12 00:39:10.549303 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:39:10.549280    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-a-w2-0d3a66-openshift-default-6c7595fdd6-f92vh"]
Sep 12 00:39:10.671374 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:39:10.671336    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c3891135-55df-4b1f-aaba-ea417c6d1630-istio-podinfo\") pod \"e2e-shared-a-w2-0d3a66-openshift-default-6c7595fdd6-f92vh\" (UID: \"c3891135-55df-4b1f-aaba-ea417c6d1630\") " pod="openshift-ingress/e2e-shared-a-w2-0d3a66-openshift-default-6c7595fdd6-f92vh"
Sep 12 00:39:10.671374 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:39:10.671370    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c3891135-55df-4b1f-aaba-ea417c6d1630-istio-token\") pod \"e2e-shared-a-w2-0d3a66-openshift-default-6c7595fdd6-f92vh\" (UID: \"c3891135-55df-4b1f-aaba-ea417c6d1630\") " pod="openshift-ingress/e2e-shared-a-w2-0d3a66-openshift-default-6c7595fdd6-f92vh"
Sep 12 00:39:10.671573 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:39:10.671447    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wld99\" (UniqueName: \"kubernetes.io/projected/c3891135-55df-4b1f-aaba-ea417c6d1630-kube-api-access-wld99\") pod \"e2e-shared-a-w2-0d3a66-openshift-default-6c7595fdd6-f92vh\" (UID: \"c3891135-55df-4b1f-aaba-ea417c6d1630\") " pod="openshift-ingress/e2e-shared-a-w2-0d3a66-openshift-default-6c7595fdd6-f92vh"
Sep 12 00:39:10.671573 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:39:10.671546    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c3891135-55df-4b1f-aaba-ea417c6d1630-credential-socket\") pod \"e2e-shared-a-w2-0d3a66-openshift-default-6c7595fdd6-f92vh\" (UID: \"c3891135-55df-4b1f-aaba-ea417c6d1630\") " pod="openshift-ingress/e2e-shared-a-w2-0d3a66-openshift-default-6c7595fdd6-f92vh"
Sep 12 00:39:10.671663 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:39:10.671581    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c3891135-55df-4b1f-aaba-ea417c6d1630-workload-certs\") pod \"e2e-shared-a-w2-0d3a66-openshift-default-6c7595fdd6-f92vh\" (UID: \"c3891135-55df-4b1f-aaba-ea417c6d1630\") " pod="openshift-ingress/e2e-shared-a-w2-0d3a66-openshift-default-6c7595fdd6-f92vh"
Sep 12 00:39:10.671663 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:39:10.671628    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c3891135-55df-4b1f-aaba-ea417c6d1630-workload-socket\") pod \"e2e-shared-a-w2-0d3a66-openshift-default-6c7595fdd6-f92vh\" (UID: \"c3891135-55df-4b1f-aaba-ea417c6d1630\") " pod="openshift-ingress/e2e-shared-a-w2-0d3a66-openshift-default-6c7595fdd6-f92vh"
Sep 12 00:39:10.671730 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:39:10.671675    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c3891135-55df-4b1f-aaba-ea417c6d1630-istio-envoy\") pod \"e2e-shared-a-w2-0d3a66-openshift-default-6c7595fdd6-f92vh\" (UID: \"c3891135-55df-4b1f-aaba-ea417c6d1630\") " pod="openshift-ingress/e2e-shared-a-w2-0d3a66-openshift-default-6c7595fdd6-f92vh"
Sep 12 00:39:10.671730 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:39:10.671722    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c3891135-55df-4b1f-aaba-ea417c6d1630-istiod-ca-cert\") pod \"e2e-shared-a-w2-0d3a66-openshift-default-6c7595fdd6-f92vh\" (UID: \"c3891135-55df-4b1f-aaba-ea417c6d1630\") " pod="openshift-ingress/e2e-shared-a-w2-0d3a66-openshift-default-6c7595fdd6-f92vh"
Sep 12 00:39:10.671802 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:39:10.671782    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c3891135-55df-4b1f-aaba-ea417c6d1630-istio-data\") pod \"e2e-shared-a-w2-0d3a66-openshift-default-6c7595fdd6-f92vh\" (UID: \"c3891135-55df-4b1f-aaba-ea417c6d1630\") " pod="openshift-ingress/e2e-shared-a-w2-0d3a66-openshift-default-6c7595fdd6-f92vh"
Sep 12 00:39:10.772538 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:39:10.772498    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c3891135-55df-4b1f-aaba-ea417c6d1630-credential-socket\") pod \"e2e-shared-a-w2-0d3a66-openshift-default-6c7595fdd6-f92vh\" (UID: \"c3891135-55df-4b1f-aaba-ea417c6d1630\") " pod="openshift-ingress/e2e-shared-a-w2-0d3a66-openshift-default-6c7595fdd6-f92vh"
Sep 12 00:39:10.772538 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:39:10.772538    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c3891135-55df-4b1f-aaba-ea417c6d1630-workload-certs\") pod \"e2e-shared-a-w2-0d3a66-openshift-default-6c7595fdd6-f92vh\" (UID: \"c3891135-55df-4b1f-aaba-ea417c6d1630\") " pod="openshift-ingress/e2e-shared-a-w2-0d3a66-openshift-default-6c7595fdd6-f92vh"
Sep 12 00:39:10.772798 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:39:10.772555    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c3891135-55df-4b1f-aaba-ea417c6d1630-workload-socket\") pod \"e2e-shared-a-w2-0d3a66-openshift-default-6c7595fdd6-f92vh\" (UID: \"c3891135-55df-4b1f-aaba-ea417c6d1630\") " pod="openshift-ingress/e2e-shared-a-w2-0d3a66-openshift-default-6c7595fdd6-f92vh"
Sep 12 00:39:10.772798 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:39:10.772575    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c3891135-55df-4b1f-aaba-ea417c6d1630-istio-envoy\") pod \"e2e-shared-a-w2-0d3a66-openshift-default-6c7595fdd6-f92vh\" (UID: \"c3891135-55df-4b1f-aaba-ea417c6d1630\") " pod="openshift-ingress/e2e-shared-a-w2-0d3a66-openshift-default-6c7595fdd6-f92vh"
Sep 12 00:39:10.772798 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:39:10.772630    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c3891135-55df-4b1f-aaba-ea417c6d1630-istiod-ca-cert\") pod \"e2e-shared-a-w2-0d3a66-openshift-default-6c7595fdd6-f92vh\" (UID: \"c3891135-55df-4b1f-aaba-ea417c6d1630\") " pod="openshift-ingress/e2e-shared-a-w2-0d3a66-openshift-default-6c7595fdd6-f92vh"
Sep 12 00:39:10.772798 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:39:10.772660    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c3891135-55df-4b1f-aaba-ea417c6d1630-istio-data\") pod \"e2e-shared-a-w2-0d3a66-openshift-default-6c7595fdd6-f92vh\" (UID: \"c3891135-55df-4b1f-aaba-ea417c6d1630\") " pod="openshift-ingress/e2e-shared-a-w2-0d3a66-openshift-default-6c7595fdd6-f92vh"
Sep 12 00:39:10.772798 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:39:10.772704    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c3891135-55df-4b1f-aaba-ea417c6d1630-istio-podinfo\") pod \"e2e-shared-a-w2-0d3a66-openshift-default-6c7595fdd6-f92vh\" (UID: \"c3891135-55df-4b1f-aaba-ea417c6d1630\") " pod="openshift-ingress/e2e-shared-a-w2-0d3a66-openshift-default-6c7595fdd6-f92vh"
Sep 12 00:39:10.772798 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:39:10.772732    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c3891135-55df-4b1f-aaba-ea417c6d1630-istio-token\") pod \"e2e-shared-a-w2-0d3a66-openshift-default-6c7595fdd6-f92vh\" (UID: \"c3891135-55df-4b1f-aaba-ea417c6d1630\") " pod="openshift-ingress/e2e-shared-a-w2-0d3a66-openshift-default-6c7595fdd6-f92vh"
Sep 12 00:39:10.772798 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:39:10.772776    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wld99\" (UniqueName: \"kubernetes.io/projected/c3891135-55df-4b1f-aaba-ea417c6d1630-kube-api-access-wld99\") pod \"e2e-shared-a-w2-0d3a66-openshift-default-6c7595fdd6-f92vh\" (UID: \"c3891135-55df-4b1f-aaba-ea417c6d1630\") " pod="openshift-ingress/e2e-shared-a-w2-0d3a66-openshift-default-6c7595fdd6-f92vh"
Sep 12 00:39:10.773165 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:39:10.772975    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c3891135-55df-4b1f-aaba-ea417c6d1630-credential-socket\") pod \"e2e-shared-a-w2-0d3a66-openshift-default-6c7595fdd6-f92vh\" (UID: \"c3891135-55df-4b1f-aaba-ea417c6d1630\") " pod="openshift-ingress/e2e-shared-a-w2-0d3a66-openshift-default-6c7595fdd6-f92vh"
Sep 12 00:39:10.773165 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:39:10.773052    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c3891135-55df-4b1f-aaba-ea417c6d1630-workload-socket\") pod \"e2e-shared-a-w2-0d3a66-openshift-default-6c7595fdd6-f92vh\" (UID: \"c3891135-55df-4b1f-aaba-ea417c6d1630\") " pod="openshift-ingress/e2e-shared-a-w2-0d3a66-openshift-default-6c7595fdd6-f92vh"
Sep 12 00:39:10.773531 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:39:10.773306    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c3891135-55df-4b1f-aaba-ea417c6d1630-istio-data\") pod \"e2e-shared-a-w2-0d3a66-openshift-default-6c7595fdd6-f92vh\" (UID: \"c3891135-55df-4b1f-aaba-ea417c6d1630\") " pod="openshift-ingress/e2e-shared-a-w2-0d3a66-openshift-default-6c7595fdd6-f92vh"
Sep 12 00:39:10.774022 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:39:10.773994    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c3891135-55df-4b1f-aaba-ea417c6d1630-istiod-ca-cert\") pod \"e2e-shared-a-w2-0d3a66-openshift-default-6c7595fdd6-f92vh\" (UID: \"c3891135-55df-4b1f-aaba-ea417c6d1630\") " pod="openshift-ingress/e2e-shared-a-w2-0d3a66-openshift-default-6c7595fdd6-f92vh"
Sep 12 00:39:10.781275 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:39:10.775346    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c3891135-55df-4b1f-aaba-ea417c6d1630-istio-envoy\") pod \"e2e-shared-a-w2-0d3a66-openshift-default-6c7595fdd6-f92vh\" (UID: \"c3891135-55df-4b1f-aaba-ea417c6d1630\") " pod="openshift-ingress/e2e-shared-a-w2-0d3a66-openshift-default-6c7595fdd6-f92vh"
Sep 12 00:39:10.782624 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:39:10.781408    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c3891135-55df-4b1f-aaba-ea417c6d1630-workload-certs\") pod \"e2e-shared-a-w2-0d3a66-openshift-default-6c7595fdd6-f92vh\" (UID: \"c3891135-55df-4b1f-aaba-ea417c6d1630\") " pod="openshift-ingress/e2e-shared-a-w2-0d3a66-openshift-default-6c7595fdd6-f92vh"
Sep 12 00:39:10.782624 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:39:10.781857    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c3891135-55df-4b1f-aaba-ea417c6d1630-istio-podinfo\") pod \"e2e-shared-a-w2-0d3a66-openshift-default-6c7595fdd6-f92vh\" (UID: \"c3891135-55df-4b1f-aaba-ea417c6d1630\") " pod="openshift-ingress/e2e-shared-a-w2-0d3a66-openshift-default-6c7595fdd6-f92vh"
Sep 12 00:39:10.783270 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:39:10.783241    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c3891135-55df-4b1f-aaba-ea417c6d1630-istio-token\") pod \"e2e-shared-a-w2-0d3a66-openshift-default-6c7595fdd6-f92vh\" (UID: \"c3891135-55df-4b1f-aaba-ea417c6d1630\") " pod="openshift-ingress/e2e-shared-a-w2-0d3a66-openshift-default-6c7595fdd6-f92vh"
Sep 12 00:39:10.783337 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:39:10.783310    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wld99\" (UniqueName: \"kubernetes.io/projected/c3891135-55df-4b1f-aaba-ea417c6d1630-kube-api-access-wld99\") pod \"e2e-shared-a-w2-0d3a66-openshift-default-6c7595fdd6-f92vh\" (UID: \"c3891135-55df-4b1f-aaba-ea417c6d1630\") " pod="openshift-ingress/e2e-shared-a-w2-0d3a66-openshift-default-6c7595fdd6-f92vh"
Sep 12 00:39:10.848021 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:39:10.847981    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-w2-0d3a66-openshift-default-6c7595fdd6-f92vh"
Sep 12 00:39:11.126469 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:39:11.126440    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-a-w2-0d3a66-openshift-default-6c7595fdd6-f92vh"]
Sep 12 00:39:11.127800 ip-10-0-141-128 kubenswrapper[2571]: W0912 00:39:11.127774    2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc3891135_55df_4b1f_aaba_ea417c6d1630.slice/crio-49bee2b95289bc9c243a94e3687db4abc91433aa46ea1a3f786a799fb89a3b36 WatchSource:0}: Error finding container 49bee2b95289bc9c243a94e3687db4abc91433aa46ea1a3f786a799fb89a3b36: Status 404 returned error can't find the container with id 49bee2b95289bc9c243a94e3687db4abc91433aa46ea1a3f786a799fb89a3b36
Sep 12 00:39:11.130131 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:39:11.130099    2571 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236196Ki","pods":"250"}
Sep 12 00:39:11.130203 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:39:11.130161    2571 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236196Ki","pods":"250"}
Sep 12 00:39:11.130203 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:39:11.130186    2571 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236196Ki","pods":"250"}
Sep 12 00:39:11.422060 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:39:11.421966    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w2-0d3a66-openshift-default-6c7595fdd6-f92vh" event={"ID":"c3891135-55df-4b1f-aaba-ea417c6d1630","Type":"ContainerStarted","Data":"a6d921ba1cd2404679eafa67a015b6103aab40f95765250c2fb99c2fa7e65c67"}
Sep 12 00:39:11.422060 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:39:11.422026    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w2-0d3a66-openshift-default-6c7595fdd6-f92vh" event={"ID":"c3891135-55df-4b1f-aaba-ea417c6d1630","Type":"ContainerStarted","Data":"49bee2b95289bc9c243a94e3687db4abc91433aa46ea1a3f786a799fb89a3b36"}
Sep 12 00:39:11.442241 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:39:11.442191    2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-shared-a-w2-0d3a66-openshift-default-6c7595fdd6-f92vh" podStartSLOduration=1.442171002 podStartE2EDuration="1.442171002s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-12 00:39:10 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-12 00:39:11.441236195 +0000 UTC m=+1362.712876705" watchObservedRunningTime="2026-09-12 00:39:11.442171002 +0000 UTC m=+1362.713811512"
Sep 12 00:39:11.848350 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:39:11.848323    2571 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-a-w2-0d3a66-openshift-default-6c7595fdd6-f92vh"
Sep 12 00:39:12.854618 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:39:12.854561    2571 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-a-w2-0d3a66-openshift-default-6c7595fdd6-f92vh"
Sep 12 00:39:13.433736 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:39:13.433706    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-a-w2-0d3a66-openshift-default-6c7595fdd6-f92vh"
Sep 12 00:39:13.434693 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:39:13.434670    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-a-w2-0d3a66-openshift-default-6c7595fdd6-f92vh"
Sep 12 00:39:32.019324 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:39:32.019249    2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-a4df66-7764776995-d48lt"]
Sep 12 00:39:32.019857 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:39:32.019484    2571 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-a4df66-7764776995-d48lt" podUID="7f5d4ed3-6d89-4f38-a7f1-e7654c6dc733" containerName="maas-api" containerID="cri-o://eafdfa3d4bb3bc641b4ba4347f7abd6a05229a04cf1c679daaee5db5699bbba5" gracePeriod=30
Sep 12 00:39:32.082064 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:39:32.080778    2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-a-a4df66-7dd5b76c4-vqffm"]
Sep 12 00:39:32.082064 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:39:32.081038    2571 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-a4df66-7dd5b76c4-vqffm" podUID="41f32f06-ebed-400a-9624-1ffe8602eab3" containerName="payload-pre-processing" containerID="cri-o://1a63cd7f992bff25bbfa80331be521349b0ad9ca272a8ba90154ddcb57aaf0f6" gracePeriod=30
Sep 12 00:39:32.444752 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:39:32.444717    2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-a4df66-7764776995-d48lt"
Sep 12 00:39:32.505126 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:39:32.505100    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-kvkds\" (UniqueName: \"kubernetes.io/projected/7f5d4ed3-6d89-4f38-a7f1-e7654c6dc733-kube-api-access-kvkds\") pod \"7f5d4ed3-6d89-4f38-a7f1-e7654c6dc733\" (UID: \"7f5d4ed3-6d89-4f38-a7f1-e7654c6dc733\") "
Sep 12 00:39:32.505396 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:39:32.505377    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/7f5d4ed3-6d89-4f38-a7f1-e7654c6dc733-maas-api-tls\") pod \"7f5d4ed3-6d89-4f38-a7f1-e7654c6dc733\" (UID: \"7f5d4ed3-6d89-4f38-a7f1-e7654c6dc733\") "
Sep 12 00:39:32.505580 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:39:32.505565    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7f5d4ed3-6d89-4f38-a7f1-e7654c6dc733-metrics-certs\") pod \"7f5d4ed3-6d89-4f38-a7f1-e7654c6dc733\" (UID: \"7f5d4ed3-6d89-4f38-a7f1-e7654c6dc733\") "
Sep 12 00:39:32.505745 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:39:32.505727    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/7f5d4ed3-6d89-4f38-a7f1-e7654c6dc733-openshift-service-ca\") pod \"7f5d4ed3-6d89-4f38-a7f1-e7654c6dc733\" (UID: \"7f5d4ed3-6d89-4f38-a7f1-e7654c6dc733\") "
Sep 12 00:39:32.506134 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:39:32.506107    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7f5d4ed3-6d89-4f38-a7f1-e7654c6dc733-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "7f5d4ed3-6d89-4f38-a7f1-e7654c6dc733" (UID: "7f5d4ed3-6d89-4f38-a7f1-e7654c6dc733"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 12 00:39:32.508102 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:39:32.508017    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/7f5d4ed3-6d89-4f38-a7f1-e7654c6dc733-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "7f5d4ed3-6d89-4f38-a7f1-e7654c6dc733" (UID: "7f5d4ed3-6d89-4f38-a7f1-e7654c6dc733"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 12 00:39:32.508450 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:39:32.508407    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7f5d4ed3-6d89-4f38-a7f1-e7654c6dc733-kube-api-access-kvkds" (OuterVolumeSpecName: "kube-api-access-kvkds") pod "7f5d4ed3-6d89-4f38-a7f1-e7654c6dc733" (UID: "7f5d4ed3-6d89-4f38-a7f1-e7654c6dc733"). InnerVolumeSpecName "kube-api-access-kvkds". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 12 00:39:32.508826 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:39:32.508797    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/7f5d4ed3-6d89-4f38-a7f1-e7654c6dc733-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "7f5d4ed3-6d89-4f38-a7f1-e7654c6dc733" (UID: "7f5d4ed3-6d89-4f38-a7f1-e7654c6dc733"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 12 00:39:32.515236 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:39:32.515212    2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-a4df66-7dd5b76c4-vqffm"
Sep 12 00:39:32.537646 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:39:32.537615    2571 generic.go:356] "Generic (PLEG): container finished" podID="7f5d4ed3-6d89-4f38-a7f1-e7654c6dc733" containerID="eafdfa3d4bb3bc641b4ba4347f7abd6a05229a04cf1c679daaee5db5699bbba5" exitCode=0
Sep 12 00:39:32.537826 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:39:32.537645    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-a4df66-7764776995-d48lt" event={"ID":"7f5d4ed3-6d89-4f38-a7f1-e7654c6dc733","Type":"ContainerDied","Data":"eafdfa3d4bb3bc641b4ba4347f7abd6a05229a04cf1c679daaee5db5699bbba5"}
Sep 12 00:39:32.537826 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:39:32.537702    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-a4df66-7764776995-d48lt" event={"ID":"7f5d4ed3-6d89-4f38-a7f1-e7654c6dc733","Type":"ContainerDied","Data":"95eb3a29b4b9d6af240fdc41c27c4146cba4161730ffa0cc9c99e32260d854b7"}
Sep 12 00:39:32.537826 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:39:32.537657    2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-a4df66-7764776995-d48lt"
Sep 12 00:39:32.537826 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:39:32.537723    2571 scope.go:117] "RemoveContainer" containerID="eafdfa3d4bb3bc641b4ba4347f7abd6a05229a04cf1c679daaee5db5699bbba5"
Sep 12 00:39:32.539287 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:39:32.539264    2571 generic.go:356] "Generic (PLEG): container finished" podID="41f32f06-ebed-400a-9624-1ffe8602eab3" containerID="1a63cd7f992bff25bbfa80331be521349b0ad9ca272a8ba90154ddcb57aaf0f6" exitCode=0
Sep 12 00:39:32.539405 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:39:32.539313    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-a4df66-7dd5b76c4-vqffm" event={"ID":"41f32f06-ebed-400a-9624-1ffe8602eab3","Type":"ContainerDied","Data":"1a63cd7f992bff25bbfa80331be521349b0ad9ca272a8ba90154ddcb57aaf0f6"}
Sep 12 00:39:32.539405 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:39:32.539320    2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-a4df66-7dd5b76c4-vqffm"
Sep 12 00:39:32.539405 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:39:32.539346    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-a4df66-7dd5b76c4-vqffm" event={"ID":"41f32f06-ebed-400a-9624-1ffe8602eab3","Type":"ContainerDied","Data":"0385bb7dec03400dd37af4918bb25d12a951811b4eeb90b27afdea0fbc4de4dc"}
Sep 12 00:39:32.551380 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:39:32.551362    2571 scope.go:117] "RemoveContainer" containerID="eafdfa3d4bb3bc641b4ba4347f7abd6a05229a04cf1c679daaee5db5699bbba5"
Sep 12 00:39:32.551722 ip-10-0-141-128 kubenswrapper[2571]: E0912 00:39:32.551703    2571 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"eafdfa3d4bb3bc641b4ba4347f7abd6a05229a04cf1c679daaee5db5699bbba5\": container with ID starting with eafdfa3d4bb3bc641b4ba4347f7abd6a05229a04cf1c679daaee5db5699bbba5 not found: ID does not exist" containerID="eafdfa3d4bb3bc641b4ba4347f7abd6a05229a04cf1c679daaee5db5699bbba5"
Sep 12 00:39:32.551812 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:39:32.551735    2571 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"eafdfa3d4bb3bc641b4ba4347f7abd6a05229a04cf1c679daaee5db5699bbba5"} err="failed to get container status \"eafdfa3d4bb3bc641b4ba4347f7abd6a05229a04cf1c679daaee5db5699bbba5\": rpc error: code = NotFound desc = could not find container \"eafdfa3d4bb3bc641b4ba4347f7abd6a05229a04cf1c679daaee5db5699bbba5\": container with ID starting with eafdfa3d4bb3bc641b4ba4347f7abd6a05229a04cf1c679daaee5db5699bbba5 not found: ID does not exist"
Sep 12 00:39:32.551812 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:39:32.551759    2571 scope.go:117] "RemoveContainer" containerID="1a63cd7f992bff25bbfa80331be521349b0ad9ca272a8ba90154ddcb57aaf0f6"
Sep 12 00:39:32.597125 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:39:32.597075    2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-a4df66-7764776995-d48lt"]
Sep 12 00:39:32.598084 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:39:32.597616    2571 scope.go:117] "RemoveContainer" containerID="1a63cd7f992bff25bbfa80331be521349b0ad9ca272a8ba90154ddcb57aaf0f6"
Sep 12 00:39:32.598494 ip-10-0-141-128 kubenswrapper[2571]: E0912 00:39:32.598443    2571 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1a63cd7f992bff25bbfa80331be521349b0ad9ca272a8ba90154ddcb57aaf0f6\": container with ID starting with 1a63cd7f992bff25bbfa80331be521349b0ad9ca272a8ba90154ddcb57aaf0f6 not found: ID does not exist" containerID="1a63cd7f992bff25bbfa80331be521349b0ad9ca272a8ba90154ddcb57aaf0f6"
Sep 12 00:39:32.598906 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:39:32.598504    2571 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1a63cd7f992bff25bbfa80331be521349b0ad9ca272a8ba90154ddcb57aaf0f6"} err="failed to get container status \"1a63cd7f992bff25bbfa80331be521349b0ad9ca272a8ba90154ddcb57aaf0f6\": rpc error: code = NotFound desc = could not find container \"1a63cd7f992bff25bbfa80331be521349b0ad9ca272a8ba90154ddcb57aaf0f6\": container with ID starting with 1a63cd7f992bff25bbfa80331be521349b0ad9ca272a8ba90154ddcb57aaf0f6 not found: ID does not exist"
Sep 12 00:39:32.607288 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:39:32.607258    2571 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-a4df66-7764776995-d48lt"]
Sep 12 00:39:32.607535 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:39:32.607513    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-9smhz\" (UniqueName: \"kubernetes.io/projected/41f32f06-ebed-400a-9624-1ffe8602eab3-kube-api-access-9smhz\") pod \"41f32f06-ebed-400a-9624-1ffe8602eab3\" (UID: \"41f32f06-ebed-400a-9624-1ffe8602eab3\") "
Sep 12 00:39:32.607641 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:39:32.607613    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/41f32f06-ebed-400a-9624-1ffe8602eab3-plugins-config-volume\") pod \"41f32f06-ebed-400a-9624-1ffe8602eab3\" (UID: \"41f32f06-ebed-400a-9624-1ffe8602eab3\") "
Sep 12 00:39:32.607882 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:39:32.607867    2571 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7f5d4ed3-6d89-4f38-a7f1-e7654c6dc733-metrics-certs\") on node \"ip-10-0-141-128.ec2.internal\" DevicePath \"\""
Sep 12 00:39:32.607882 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:39:32.607884    2571 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/7f5d4ed3-6d89-4f38-a7f1-e7654c6dc733-openshift-service-ca\") on node \"ip-10-0-141-128.ec2.internal\" DevicePath \"\""
Sep 12 00:39:32.608019 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:39:32.607894    2571 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-kvkds\" (UniqueName: \"kubernetes.io/projected/7f5d4ed3-6d89-4f38-a7f1-e7654c6dc733-kube-api-access-kvkds\") on node \"ip-10-0-141-128.ec2.internal\" DevicePath \"\""
Sep 12 00:39:32.608019 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:39:32.607903    2571 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/7f5d4ed3-6d89-4f38-a7f1-e7654c6dc733-maas-api-tls\") on node \"ip-10-0-141-128.ec2.internal\" DevicePath \"\""
Sep 12 00:39:32.608457 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:39:32.608433    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/41f32f06-ebed-400a-9624-1ffe8602eab3-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "41f32f06-ebed-400a-9624-1ffe8602eab3" (UID: "41f32f06-ebed-400a-9624-1ffe8602eab3"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 12 00:39:32.609825 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:39:32.609799    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/41f32f06-ebed-400a-9624-1ffe8602eab3-kube-api-access-9smhz" (OuterVolumeSpecName: "kube-api-access-9smhz") pod "41f32f06-ebed-400a-9624-1ffe8602eab3" (UID: "41f32f06-ebed-400a-9624-1ffe8602eab3"). InnerVolumeSpecName "kube-api-access-9smhz". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 12 00:39:32.708792 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:39:32.708759    2571 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9smhz\" (UniqueName: \"kubernetes.io/projected/41f32f06-ebed-400a-9624-1ffe8602eab3-kube-api-access-9smhz\") on node \"ip-10-0-141-128.ec2.internal\" DevicePath \"\""
Sep 12 00:39:32.708792 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:39:32.708787    2571 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/41f32f06-ebed-400a-9624-1ffe8602eab3-plugins-config-volume\") on node \"ip-10-0-141-128.ec2.internal\" DevicePath \"\""
Sep 12 00:39:32.877010 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:39:32.876967    2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-a-a4df66-7dd5b76c4-vqffm"]
Sep 12 00:39:32.884627 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:39:32.884562    2571 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-a-a4df66-7dd5b76c4-vqffm"]
Sep 12 00:39:33.141413 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:39:33.141337    2571 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="41f32f06-ebed-400a-9624-1ffe8602eab3" path="/var/lib/kubelet/pods/41f32f06-ebed-400a-9624-1ffe8602eab3/volumes"
Sep 12 00:39:33.141805 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:39:33.141786    2571 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7f5d4ed3-6d89-4f38-a7f1-e7654c6dc733" path="/var/lib/kubelet/pods/7f5d4ed3-6d89-4f38-a7f1-e7654c6dc733/volumes"
Sep 12 00:39:33.408261 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:39:33.408176    2571 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-a4df66-7dd5b76c4-vqffm" podUID="41f32f06-ebed-400a-9624-1ffe8602eab3" containerName="payload-pre-processing" probeResult="failure" output="dial tcp 10.132.0.43:9004: i/o timeout"
Sep 12 00:39:38.996188 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:39:38.996140    2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-shared-b-w2-f12ab7-openshift-default-678468bf74-42mbj"]
Sep 12 00:39:38.996906 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:39:38.996883    2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="41f32f06-ebed-400a-9624-1ffe8602eab3" containerName="payload-pre-processing"
Sep 12 00:39:38.997007 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:39:38.996908    2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="41f32f06-ebed-400a-9624-1ffe8602eab3" containerName="payload-pre-processing"
Sep 12 00:39:38.997007 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:39:38.996937    2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7f5d4ed3-6d89-4f38-a7f1-e7654c6dc733" containerName="maas-api"
Sep 12 00:39:38.997007 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:39:38.996945    2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="7f5d4ed3-6d89-4f38-a7f1-e7654c6dc733" containerName="maas-api"
Sep 12 00:39:38.997106 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:39:38.997094    2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="41f32f06-ebed-400a-9624-1ffe8602eab3" containerName="payload-pre-processing"
Sep 12 00:39:38.997143 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:39:38.997111    2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="7f5d4ed3-6d89-4f38-a7f1-e7654c6dc733" containerName="maas-api"
Sep 12 00:39:39.002417 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:39:39.002395    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-w2-f12ab7-openshift-default-678468bf74-42mbj"
Sep 12 00:39:39.004928 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:39:39.004907    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-shared-b-w2-f12ab7-openshift-default-dockercfg-xw7dz\""
Sep 12 00:39:39.009627 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:39:39.009597    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-b-w2-f12ab7-openshift-default-678468bf74-42mbj"]
Sep 12 00:39:39.080099 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:39:39.080060    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/31532d6d-06e5-4f45-922c-184e13d910fe-istio-podinfo\") pod \"e2e-shared-b-w2-f12ab7-openshift-default-678468bf74-42mbj\" (UID: \"31532d6d-06e5-4f45-922c-184e13d910fe\") " pod="openshift-ingress/e2e-shared-b-w2-f12ab7-openshift-default-678468bf74-42mbj"
Sep 12 00:39:39.080290 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:39:39.080113    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/31532d6d-06e5-4f45-922c-184e13d910fe-istio-envoy\") pod \"e2e-shared-b-w2-f12ab7-openshift-default-678468bf74-42mbj\" (UID: \"31532d6d-06e5-4f45-922c-184e13d910fe\") " pod="openshift-ingress/e2e-shared-b-w2-f12ab7-openshift-default-678468bf74-42mbj"
Sep 12 00:39:39.080290 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:39:39.080152    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/31532d6d-06e5-4f45-922c-184e13d910fe-istio-token\") pod \"e2e-shared-b-w2-f12ab7-openshift-default-678468bf74-42mbj\" (UID: \"31532d6d-06e5-4f45-922c-184e13d910fe\") " pod="openshift-ingress/e2e-shared-b-w2-f12ab7-openshift-default-678468bf74-42mbj"
Sep 12 00:39:39.080290 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:39:39.080194    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-g9lpr\" (UniqueName: \"kubernetes.io/projected/31532d6d-06e5-4f45-922c-184e13d910fe-kube-api-access-g9lpr\") pod \"e2e-shared-b-w2-f12ab7-openshift-default-678468bf74-42mbj\" (UID: \"31532d6d-06e5-4f45-922c-184e13d910fe\") " pod="openshift-ingress/e2e-shared-b-w2-f12ab7-openshift-default-678468bf74-42mbj"
Sep 12 00:39:39.080459 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:39:39.080325    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/31532d6d-06e5-4f45-922c-184e13d910fe-workload-certs\") pod \"e2e-shared-b-w2-f12ab7-openshift-default-678468bf74-42mbj\" (UID: \"31532d6d-06e5-4f45-922c-184e13d910fe\") " pod="openshift-ingress/e2e-shared-b-w2-f12ab7-openshift-default-678468bf74-42mbj"
Sep 12 00:39:39.080459 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:39:39.080388    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/31532d6d-06e5-4f45-922c-184e13d910fe-credential-socket\") pod \"e2e-shared-b-w2-f12ab7-openshift-default-678468bf74-42mbj\" (UID: \"31532d6d-06e5-4f45-922c-184e13d910fe\") " pod="openshift-ingress/e2e-shared-b-w2-f12ab7-openshift-default-678468bf74-42mbj"
Sep 12 00:39:39.080459 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:39:39.080433    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/31532d6d-06e5-4f45-922c-184e13d910fe-istiod-ca-cert\") pod \"e2e-shared-b-w2-f12ab7-openshift-default-678468bf74-42mbj\" (UID: \"31532d6d-06e5-4f45-922c-184e13d910fe\") " pod="openshift-ingress/e2e-shared-b-w2-f12ab7-openshift-default-678468bf74-42mbj"
Sep 12 00:39:39.080645 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:39:39.080522    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/31532d6d-06e5-4f45-922c-184e13d910fe-workload-socket\") pod \"e2e-shared-b-w2-f12ab7-openshift-default-678468bf74-42mbj\" (UID: \"31532d6d-06e5-4f45-922c-184e13d910fe\") " pod="openshift-ingress/e2e-shared-b-w2-f12ab7-openshift-default-678468bf74-42mbj"
Sep 12 00:39:39.080645 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:39:39.080559    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/31532d6d-06e5-4f45-922c-184e13d910fe-istio-data\") pod \"e2e-shared-b-w2-f12ab7-openshift-default-678468bf74-42mbj\" (UID: \"31532d6d-06e5-4f45-922c-184e13d910fe\") " pod="openshift-ingress/e2e-shared-b-w2-f12ab7-openshift-default-678468bf74-42mbj"
Sep 12 00:39:39.181528 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:39:39.181489    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/31532d6d-06e5-4f45-922c-184e13d910fe-istio-envoy\") pod \"e2e-shared-b-w2-f12ab7-openshift-default-678468bf74-42mbj\" (UID: \"31532d6d-06e5-4f45-922c-184e13d910fe\") " pod="openshift-ingress/e2e-shared-b-w2-f12ab7-openshift-default-678468bf74-42mbj"
Sep 12 00:39:39.181528 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:39:39.181529    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/31532d6d-06e5-4f45-922c-184e13d910fe-istio-token\") pod \"e2e-shared-b-w2-f12ab7-openshift-default-678468bf74-42mbj\" (UID: \"31532d6d-06e5-4f45-922c-184e13d910fe\") " pod="openshift-ingress/e2e-shared-b-w2-f12ab7-openshift-default-678468bf74-42mbj"
Sep 12 00:39:39.181799 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:39:39.181547    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-g9lpr\" (UniqueName: \"kubernetes.io/projected/31532d6d-06e5-4f45-922c-184e13d910fe-kube-api-access-g9lpr\") pod \"e2e-shared-b-w2-f12ab7-openshift-default-678468bf74-42mbj\" (UID: \"31532d6d-06e5-4f45-922c-184e13d910fe\") " pod="openshift-ingress/e2e-shared-b-w2-f12ab7-openshift-default-678468bf74-42mbj"
Sep 12 00:39:39.181799 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:39:39.181612    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/31532d6d-06e5-4f45-922c-184e13d910fe-workload-certs\") pod \"e2e-shared-b-w2-f12ab7-openshift-default-678468bf74-42mbj\" (UID: \"31532d6d-06e5-4f45-922c-184e13d910fe\") " pod="openshift-ingress/e2e-shared-b-w2-f12ab7-openshift-default-678468bf74-42mbj"
Sep 12 00:39:39.181799 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:39:39.181653    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/31532d6d-06e5-4f45-922c-184e13d910fe-credential-socket\") pod \"e2e-shared-b-w2-f12ab7-openshift-default-678468bf74-42mbj\" (UID: \"31532d6d-06e5-4f45-922c-184e13d910fe\") " pod="openshift-ingress/e2e-shared-b-w2-f12ab7-openshift-default-678468bf74-42mbj"
Sep 12 00:39:39.181799 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:39:39.181678    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/31532d6d-06e5-4f45-922c-184e13d910fe-istiod-ca-cert\") pod \"e2e-shared-b-w2-f12ab7-openshift-default-678468bf74-42mbj\" (UID: \"31532d6d-06e5-4f45-922c-184e13d910fe\") " pod="openshift-ingress/e2e-shared-b-w2-f12ab7-openshift-default-678468bf74-42mbj"
Sep 12 00:39:39.181799 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:39:39.181741    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/31532d6d-06e5-4f45-922c-184e13d910fe-workload-socket\") pod \"e2e-shared-b-w2-f12ab7-openshift-default-678468bf74-42mbj\" (UID: \"31532d6d-06e5-4f45-922c-184e13d910fe\") " pod="openshift-ingress/e2e-shared-b-w2-f12ab7-openshift-default-678468bf74-42mbj"
Sep 12 00:39:39.181799 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:39:39.181774    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/31532d6d-06e5-4f45-922c-184e13d910fe-istio-data\") pod \"e2e-shared-b-w2-f12ab7-openshift-default-678468bf74-42mbj\" (UID: \"31532d6d-06e5-4f45-922c-184e13d910fe\") " pod="openshift-ingress/e2e-shared-b-w2-f12ab7-openshift-default-678468bf74-42mbj"
Sep 12 00:39:39.182115 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:39:39.181850    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/31532d6d-06e5-4f45-922c-184e13d910fe-istio-podinfo\") pod \"e2e-shared-b-w2-f12ab7-openshift-default-678468bf74-42mbj\" (UID: \"31532d6d-06e5-4f45-922c-184e13d910fe\") " pod="openshift-ingress/e2e-shared-b-w2-f12ab7-openshift-default-678468bf74-42mbj"
Sep 12 00:39:39.182415 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:39:39.182379    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/31532d6d-06e5-4f45-922c-184e13d910fe-workload-certs\") pod \"e2e-shared-b-w2-f12ab7-openshift-default-678468bf74-42mbj\" (UID: \"31532d6d-06e5-4f45-922c-184e13d910fe\") " pod="openshift-ingress/e2e-shared-b-w2-f12ab7-openshift-default-678468bf74-42mbj"
Sep 12 00:39:39.182543 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:39:39.182522    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/31532d6d-06e5-4f45-922c-184e13d910fe-workload-socket\") pod \"e2e-shared-b-w2-f12ab7-openshift-default-678468bf74-42mbj\" (UID: \"31532d6d-06e5-4f45-922c-184e13d910fe\") " pod="openshift-ingress/e2e-shared-b-w2-f12ab7-openshift-default-678468bf74-42mbj"
Sep 12 00:39:39.182635 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:39:39.182551    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/31532d6d-06e5-4f45-922c-184e13d910fe-istiod-ca-cert\") pod \"e2e-shared-b-w2-f12ab7-openshift-default-678468bf74-42mbj\" (UID: \"31532d6d-06e5-4f45-922c-184e13d910fe\") " pod="openshift-ingress/e2e-shared-b-w2-f12ab7-openshift-default-678468bf74-42mbj"
Sep 12 00:39:39.182727 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:39:39.182701    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/31532d6d-06e5-4f45-922c-184e13d910fe-istio-data\") pod \"e2e-shared-b-w2-f12ab7-openshift-default-678468bf74-42mbj\" (UID: \"31532d6d-06e5-4f45-922c-184e13d910fe\") " pod="openshift-ingress/e2e-shared-b-w2-f12ab7-openshift-default-678468bf74-42mbj"
Sep 12 00:39:39.182858 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:39:39.182824    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/31532d6d-06e5-4f45-922c-184e13d910fe-credential-socket\") pod \"e2e-shared-b-w2-f12ab7-openshift-default-678468bf74-42mbj\" (UID: \"31532d6d-06e5-4f45-922c-184e13d910fe\") " pod="openshift-ingress/e2e-shared-b-w2-f12ab7-openshift-default-678468bf74-42mbj"
Sep 12 00:39:39.184351 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:39:39.184331    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/31532d6d-06e5-4f45-922c-184e13d910fe-istio-podinfo\") pod \"e2e-shared-b-w2-f12ab7-openshift-default-678468bf74-42mbj\" (UID: \"31532d6d-06e5-4f45-922c-184e13d910fe\") " pod="openshift-ingress/e2e-shared-b-w2-f12ab7-openshift-default-678468bf74-42mbj"
Sep 12 00:39:39.184729 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:39:39.184701    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/31532d6d-06e5-4f45-922c-184e13d910fe-istio-envoy\") pod \"e2e-shared-b-w2-f12ab7-openshift-default-678468bf74-42mbj\" (UID: \"31532d6d-06e5-4f45-922c-184e13d910fe\") " pod="openshift-ingress/e2e-shared-b-w2-f12ab7-openshift-default-678468bf74-42mbj"
Sep 12 00:39:39.188910 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:39:39.188892    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-g9lpr\" (UniqueName: \"kubernetes.io/projected/31532d6d-06e5-4f45-922c-184e13d910fe-kube-api-access-g9lpr\") pod \"e2e-shared-b-w2-f12ab7-openshift-default-678468bf74-42mbj\" (UID: \"31532d6d-06e5-4f45-922c-184e13d910fe\") " pod="openshift-ingress/e2e-shared-b-w2-f12ab7-openshift-default-678468bf74-42mbj"
Sep 12 00:39:39.189224 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:39:39.189209    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/31532d6d-06e5-4f45-922c-184e13d910fe-istio-token\") pod \"e2e-shared-b-w2-f12ab7-openshift-default-678468bf74-42mbj\" (UID: \"31532d6d-06e5-4f45-922c-184e13d910fe\") " pod="openshift-ingress/e2e-shared-b-w2-f12ab7-openshift-default-678468bf74-42mbj"
Sep 12 00:39:39.317811 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:39:39.317734    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-w2-f12ab7-openshift-default-678468bf74-42mbj"
Sep 12 00:39:39.607198 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:39:39.607162    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-b-w2-f12ab7-openshift-default-678468bf74-42mbj"]
Sep 12 00:39:39.607514 ip-10-0-141-128 kubenswrapper[2571]: W0912 00:39:39.607467    2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod31532d6d_06e5_4f45_922c_184e13d910fe.slice/crio-14d088cc81ba01a477e742160f247a6572a9889bf2ada614aaa3d9388fd6303a WatchSource:0}: Error finding container 14d088cc81ba01a477e742160f247a6572a9889bf2ada614aaa3d9388fd6303a: Status 404 returned error can't find the container with id 14d088cc81ba01a477e742160f247a6572a9889bf2ada614aaa3d9388fd6303a
Sep 12 00:39:39.610150 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:39:39.610109    2571 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236196Ki","pods":"250"}
Sep 12 00:39:39.610231 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:39:39.610216    2571 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236196Ki","pods":"250"}
Sep 12 00:39:39.610285 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:39:39.610264    2571 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236196Ki","pods":"250"}
Sep 12 00:39:40.594186 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:39:40.594149    2571 generic.go:356] "Generic (PLEG): container finished" podID="c3891135-55df-4b1f-aaba-ea417c6d1630" containerID="a6d921ba1cd2404679eafa67a015b6103aab40f95765250c2fb99c2fa7e65c67" exitCode=0
Sep 12 00:39:40.594756 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:39:40.594249    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w2-0d3a66-openshift-default-6c7595fdd6-f92vh" event={"ID":"c3891135-55df-4b1f-aaba-ea417c6d1630","Type":"ContainerDied","Data":"a6d921ba1cd2404679eafa67a015b6103aab40f95765250c2fb99c2fa7e65c67"}
Sep 12 00:39:40.594881 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:39:40.594862    2571 scope.go:117] "RemoveContainer" containerID="a6d921ba1cd2404679eafa67a015b6103aab40f95765250c2fb99c2fa7e65c67"
Sep 12 00:39:40.596000 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:39:40.595977    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w2-f12ab7-openshift-default-678468bf74-42mbj" event={"ID":"31532d6d-06e5-4f45-922c-184e13d910fe","Type":"ContainerStarted","Data":"42ee050857dfd38d4e3f907134e7e6c185c8d74bb1d1a99c15edb52334fa7e41"}
Sep 12 00:39:40.596081 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:39:40.596039    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w2-f12ab7-openshift-default-678468bf74-42mbj" event={"ID":"31532d6d-06e5-4f45-922c-184e13d910fe","Type":"ContainerStarted","Data":"14d088cc81ba01a477e742160f247a6572a9889bf2ada614aaa3d9388fd6303a"}
Sep 12 00:39:40.596119 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:39:40.596083    2571 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236196Ki","pods":"250"}
Sep 12 00:39:40.596157 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:39:40.596147    2571 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236196Ki","pods":"250"}
Sep 12 00:39:40.596217 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:39:40.596204    2571 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236196Ki","pods":"250"}
Sep 12 00:39:40.631041 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:39:40.630873    2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-shared-b-w2-f12ab7-openshift-default-678468bf74-42mbj" podStartSLOduration=2.630822903 podStartE2EDuration="2.630822903s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-12 00:39:38 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-12 00:39:40.630001198 +0000 UTC m=+1391.901641710" watchObservedRunningTime="2026-09-12 00:39:40.630822903 +0000 UTC m=+1391.902463412"
Sep 12 00:39:41.318746 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:39:41.318712    2571 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-b-w2-f12ab7-openshift-default-678468bf74-42mbj"
Sep 12 00:39:41.323736 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:39:41.323712    2571 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-b-w2-f12ab7-openshift-default-678468bf74-42mbj"
Sep 12 00:39:41.604151 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:39:41.604070    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w2-0d3a66-openshift-default-6c7595fdd6-f92vh" event={"ID":"c3891135-55df-4b1f-aaba-ea417c6d1630","Type":"ContainerStarted","Data":"b50efbdaa00cf8db4e7158811c8629ecc26250aca1c03b8249c85b33ae3d9d4e"}
Sep 12 00:39:41.604567 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:39:41.604353    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-b-w2-f12ab7-openshift-default-678468bf74-42mbj"
Sep 12 00:39:41.605434 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:39:41.605416    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-b-w2-f12ab7-openshift-default-678468bf74-42mbj"
Sep 12 00:39:41.848668 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:39:41.848630    2571 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-a-w2-0d3a66-openshift-default-6c7595fdd6-f92vh"
Sep 12 00:39:41.849935 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:39:41.849913    2571 patch_prober.go:28] interesting pod/e2e-shared-a-w2-0d3a66-openshift-default-6c7595fdd6-f92vh container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.56:15021/healthz/ready\": dial tcp 10.132.0.56:15021: connect: connection refused" start-of-body=
Sep 12 00:39:41.850003 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:39:41.849965    2571 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w2-0d3a66-openshift-default-6c7595fdd6-f92vh" podUID="c3891135-55df-4b1f-aaba-ea417c6d1630" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.56:15021/healthz/ready\": dial tcp 10.132.0.56:15021: connect: connection refused"
Sep 12 00:39:42.506424 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:39:42.506391    2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-noten-0a0781-openshift-default-86b89df9c8-rc2kf"]
Sep 12 00:39:42.506908 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:39:42.506874    2571 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-noten-0a0781-openshift-default-86b89df9c8-rc2kf" podUID="3d1a958a-0902-4d78-ba0e-e5b829ce347c" containerName="istio-proxy" containerID="cri-o://281538e4ffb0316c6123a8e101401ed9997d604db2bd1b85038666e9f0eac6ff" gracePeriod=30
Sep 12 00:39:42.848698 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:39:42.848618    2571 patch_prober.go:28] interesting pod/e2e-shared-a-w2-0d3a66-openshift-default-6c7595fdd6-f92vh container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.56:15021/healthz/ready\": dial tcp 10.132.0.56:15021: connect: connection refused" start-of-body=
Sep 12 00:39:42.848698 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:39:42.848675    2571 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w2-0d3a66-openshift-default-6c7595fdd6-f92vh" podUID="c3891135-55df-4b1f-aaba-ea417c6d1630" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.56:15021/healthz/ready\": dial tcp 10.132.0.56:15021: connect: connection refused"
Sep 12 00:39:43.433830 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:39:43.433795    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-a-w2-0d3a66-openshift-default-6c7595fdd6-f92vh"
Sep 12 00:39:43.848765 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:39:43.848688    2571 patch_prober.go:28] interesting pod/e2e-shared-a-w2-0d3a66-openshift-default-6c7595fdd6-f92vh container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.56:15021/healthz/ready\": dial tcp 10.132.0.56:15021: connect: connection refused" start-of-body=
Sep 12 00:39:43.848765 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:39:43.848753    2571 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w2-0d3a66-openshift-default-6c7595fdd6-f92vh" podUID="c3891135-55df-4b1f-aaba-ea417c6d1630" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.56:15021/healthz/ready\": dial tcp 10.132.0.56:15021: connect: connection refused"
Sep 12 00:39:44.849113 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:39:44.849076    2571 patch_prober.go:28] interesting pod/e2e-shared-a-w2-0d3a66-openshift-default-6c7595fdd6-f92vh container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.56:15021/healthz/ready\": dial tcp 10.132.0.56:15021: connect: connection refused" start-of-body=
Sep 12 00:39:44.849497 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:39:44.849141    2571 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w2-0d3a66-openshift-default-6c7595fdd6-f92vh" podUID="c3891135-55df-4b1f-aaba-ea417c6d1630" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.56:15021/healthz/ready\": dial tcp 10.132.0.56:15021: connect: connection refused"
Sep 12 00:39:45.848330 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:39:45.848293    2571 patch_prober.go:28] interesting pod/e2e-shared-a-w2-0d3a66-openshift-default-6c7595fdd6-f92vh container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.56:15021/healthz/ready\": dial tcp 10.132.0.56:15021: connect: connection refused" start-of-body=
Sep 12 00:39:45.848633 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:39:45.848605    2571 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w2-0d3a66-openshift-default-6c7595fdd6-f92vh" podUID="c3891135-55df-4b1f-aaba-ea417c6d1630" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.56:15021/healthz/ready\": dial tcp 10.132.0.56:15021: connect: connection refused"
Sep 12 00:39:46.791645 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:39:46.786631    2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w2-f12ab7-59b8766c78-m6swk"]
Sep 12 00:39:46.795638 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:39:46.794835    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w2-f12ab7-59b8766c78-m6swk"]
Sep 12 00:39:46.795638 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:39:46.794953    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-f12ab7-59b8766c78-m6swk"
Sep 12 00:39:46.798357 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:39:46.798339    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-shared-b-w2-f12ab7\""
Sep 12 00:39:46.798505 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:39:46.798410    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-shared-b-w2-f12ab7-dockercfg-dshq4\""
Sep 12 00:39:46.817695 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:39:46.817662    2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w2-f12ab7-6bdf9c9f7c-dwjcr"]
Sep 12 00:39:46.819991 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:39:46.819976    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-b-w2-f12ab7-6bdf9c9f7c-dwjcr"
Sep 12 00:39:46.829374 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:39:46.829354    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w2-f12ab7-6bdf9c9f7c-dwjcr"]
Sep 12 00:39:46.848522 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:39:46.848492    2571 patch_prober.go:28] interesting pod/e2e-shared-a-w2-0d3a66-openshift-default-6c7595fdd6-f92vh container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.56:15021/healthz/ready\": dial tcp 10.132.0.56:15021: connect: connection refused" start-of-body=
Sep 12 00:39:46.848658 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:39:46.848537    2571 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w2-0d3a66-openshift-default-6c7595fdd6-f92vh" podUID="c3891135-55df-4b1f-aaba-ea417c6d1630" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.56:15021/healthz/ready\": dial tcp 10.132.0.56:15021: connect: connection refused"
Sep 12 00:39:46.892612 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:39:46.890727    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/66da68b7-6480-47e9-aa60-3798029410e2-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-b-w2-f12ab7-59b8766c78-m6swk\" (UID: \"66da68b7-6480-47e9-aa60-3798029410e2\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-f12ab7-59b8766c78-m6swk"
Sep 12 00:39:46.892612 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:39:46.891008    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-49zht\" (UniqueName: \"kubernetes.io/projected/66da68b7-6480-47e9-aa60-3798029410e2-kube-api-access-49zht\") pod \"payload-pre-processing-e2e-shared-b-w2-f12ab7-59b8766c78-m6swk\" (UID: \"66da68b7-6480-47e9-aa60-3798029410e2\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-f12ab7-59b8766c78-m6swk"
Sep 12 00:39:46.992542 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:39:46.992500    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9t5d6\" (UniqueName: \"kubernetes.io/projected/746506d4-a95a-415d-8c3d-fcb8c0198373-kube-api-access-9t5d6\") pod \"payload-processing-e2e-shared-b-w2-f12ab7-6bdf9c9f7c-dwjcr\" (UID: \"746506d4-a95a-415d-8c3d-fcb8c0198373\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w2-f12ab7-6bdf9c9f7c-dwjcr"
Sep 12 00:39:46.992744 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:39:46.992574    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-49zht\" (UniqueName: \"kubernetes.io/projected/66da68b7-6480-47e9-aa60-3798029410e2-kube-api-access-49zht\") pod \"payload-pre-processing-e2e-shared-b-w2-f12ab7-59b8766c78-m6swk\" (UID: \"66da68b7-6480-47e9-aa60-3798029410e2\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-f12ab7-59b8766c78-m6swk"
Sep 12 00:39:46.992744 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:39:46.992630    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/746506d4-a95a-415d-8c3d-fcb8c0198373-plugins-config-volume\") pod \"payload-processing-e2e-shared-b-w2-f12ab7-6bdf9c9f7c-dwjcr\" (UID: \"746506d4-a95a-415d-8c3d-fcb8c0198373\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w2-f12ab7-6bdf9c9f7c-dwjcr"
Sep 12 00:39:46.992744 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:39:46.992722    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/66da68b7-6480-47e9-aa60-3798029410e2-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-b-w2-f12ab7-59b8766c78-m6swk\" (UID: \"66da68b7-6480-47e9-aa60-3798029410e2\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-f12ab7-59b8766c78-m6swk"
Sep 12 00:39:46.993432 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:39:46.993405    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/66da68b7-6480-47e9-aa60-3798029410e2-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-b-w2-f12ab7-59b8766c78-m6swk\" (UID: \"66da68b7-6480-47e9-aa60-3798029410e2\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-f12ab7-59b8766c78-m6swk"
Sep 12 00:39:47.000792 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:39:47.000770    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-49zht\" (UniqueName: \"kubernetes.io/projected/66da68b7-6480-47e9-aa60-3798029410e2-kube-api-access-49zht\") pod \"payload-pre-processing-e2e-shared-b-w2-f12ab7-59b8766c78-m6swk\" (UID: \"66da68b7-6480-47e9-aa60-3798029410e2\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-f12ab7-59b8766c78-m6swk"
Sep 12 00:39:47.093818 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:39:47.093720    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9t5d6\" (UniqueName: \"kubernetes.io/projected/746506d4-a95a-415d-8c3d-fcb8c0198373-kube-api-access-9t5d6\") pod \"payload-processing-e2e-shared-b-w2-f12ab7-6bdf9c9f7c-dwjcr\" (UID: \"746506d4-a95a-415d-8c3d-fcb8c0198373\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w2-f12ab7-6bdf9c9f7c-dwjcr"
Sep 12 00:39:47.093818 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:39:47.093782    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/746506d4-a95a-415d-8c3d-fcb8c0198373-plugins-config-volume\") pod \"payload-processing-e2e-shared-b-w2-f12ab7-6bdf9c9f7c-dwjcr\" (UID: \"746506d4-a95a-415d-8c3d-fcb8c0198373\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w2-f12ab7-6bdf9c9f7c-dwjcr"
Sep 12 00:39:47.094527 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:39:47.094501    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/746506d4-a95a-415d-8c3d-fcb8c0198373-plugins-config-volume\") pod \"payload-processing-e2e-shared-b-w2-f12ab7-6bdf9c9f7c-dwjcr\" (UID: \"746506d4-a95a-415d-8c3d-fcb8c0198373\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w2-f12ab7-6bdf9c9f7c-dwjcr"
Sep 12 00:39:47.101519 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:39:47.101493    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9t5d6\" (UniqueName: \"kubernetes.io/projected/746506d4-a95a-415d-8c3d-fcb8c0198373-kube-api-access-9t5d6\") pod \"payload-processing-e2e-shared-b-w2-f12ab7-6bdf9c9f7c-dwjcr\" (UID: \"746506d4-a95a-415d-8c3d-fcb8c0198373\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w2-f12ab7-6bdf9c9f7c-dwjcr"
Sep 12 00:39:47.106334 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:39:47.106310    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-f12ab7-59b8766c78-m6swk"
Sep 12 00:39:47.131097 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:39:47.131020    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-b-w2-f12ab7-6bdf9c9f7c-dwjcr"
Sep 12 00:39:47.389702 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:39:47.389670    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w2-f12ab7-59b8766c78-m6swk"]
Sep 12 00:39:47.391389 ip-10-0-141-128 kubenswrapper[2571]: W0912 00:39:47.391361    2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod66da68b7_6480_47e9_aa60_3798029410e2.slice/crio-599bded7f6f5c8b10e758a0e0d3354b9f8e61df712b54d154cee649ed945171d WatchSource:0}: Error finding container 599bded7f6f5c8b10e758a0e0d3354b9f8e61df712b54d154cee649ed945171d: Status 404 returned error can't find the container with id 599bded7f6f5c8b10e758a0e0d3354b9f8e61df712b54d154cee649ed945171d
Sep 12 00:39:47.438753 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:39:47.438724    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w2-f12ab7-6bdf9c9f7c-dwjcr"]
Sep 12 00:39:47.443949 ip-10-0-141-128 kubenswrapper[2571]: W0912 00:39:47.443917    2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod746506d4_a95a_415d_8c3d_fcb8c0198373.slice/crio-f32d0338e9c03374a56ee95fce5e1b5f3af60e8f0f931164013750038151527d WatchSource:0}: Error finding container f32d0338e9c03374a56ee95fce5e1b5f3af60e8f0f931164013750038151527d: Status 404 returned error can't find the container with id f32d0338e9c03374a56ee95fce5e1b5f3af60e8f0f931164013750038151527d
Sep 12 00:39:47.645180 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:39:47.645098    2571 generic.go:356] "Generic (PLEG): container finished" podID="3d1a958a-0902-4d78-ba0e-e5b829ce347c" containerID="281538e4ffb0316c6123a8e101401ed9997d604db2bd1b85038666e9f0eac6ff" exitCode=0
Sep 12 00:39:47.645333 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:39:47.645170    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-noten-0a0781-openshift-default-86b89df9c8-rc2kf" event={"ID":"3d1a958a-0902-4d78-ba0e-e5b829ce347c","Type":"ContainerDied","Data":"281538e4ffb0316c6123a8e101401ed9997d604db2bd1b85038666e9f0eac6ff"}
Sep 12 00:39:47.647112 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:39:47.646998    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-f12ab7-59b8766c78-m6swk" event={"ID":"66da68b7-6480-47e9-aa60-3798029410e2","Type":"ContainerStarted","Data":"cd81b7dc9acf8b5683cb0804c24546233537eb118353b0f6155aa236dd8edeb6"}
Sep 12 00:39:47.647112 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:39:47.647054    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-f12ab7-59b8766c78-m6swk"
Sep 12 00:39:47.647112 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:39:47.647080    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-f12ab7-59b8766c78-m6swk" event={"ID":"66da68b7-6480-47e9-aa60-3798029410e2","Type":"ContainerStarted","Data":"599bded7f6f5c8b10e758a0e0d3354b9f8e61df712b54d154cee649ed945171d"}
Sep 12 00:39:47.648740 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:39:47.648706    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-b-w2-f12ab7-6bdf9c9f7c-dwjcr" event={"ID":"746506d4-a95a-415d-8c3d-fcb8c0198373","Type":"ContainerStarted","Data":"16e50e2d62d6b4f9fa8678cabe2a94c0deba3a909b12429ae2b3a1dee5672e66"}
Sep 12 00:39:47.648848 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:39:47.648762    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-b-w2-f12ab7-6bdf9c9f7c-dwjcr" event={"ID":"746506d4-a95a-415d-8c3d-fcb8c0198373","Type":"ContainerStarted","Data":"f32d0338e9c03374a56ee95fce5e1b5f3af60e8f0f931164013750038151527d"}
Sep 12 00:39:47.648848 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:39:47.648797    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-shared-b-w2-f12ab7-6bdf9c9f7c-dwjcr"
Sep 12 00:39:47.664246 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:39:47.664199    2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-f12ab7-59b8766c78-m6swk" podStartSLOduration=1.664180493 podStartE2EDuration="1.664180493s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-12 00:39:46 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-12 00:39:47.663267545 +0000 UTC m=+1398.934908047" watchObservedRunningTime="2026-09-12 00:39:47.664180493 +0000 UTC m=+1398.935820998"
Sep 12 00:39:47.761016 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:39:47.760991    2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-noten-0a0781-openshift-default-86b89df9c8-rc2kf"
Sep 12 00:39:47.782342 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:39:47.782288    2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-shared-b-w2-f12ab7-6bdf9c9f7c-dwjcr" podStartSLOduration=1.782266803 podStartE2EDuration="1.782266803s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-12 00:39:46 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-12 00:39:47.680073564 +0000 UTC m=+1398.951714073" watchObservedRunningTime="2026-09-12 00:39:47.782266803 +0000 UTC m=+1399.053907316"
Sep 12 00:39:47.853548 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:39:47.853505    2571 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-a-w2-0d3a66-openshift-default-6c7595fdd6-f92vh"
Sep 12 00:39:47.854818 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:39:47.854794    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-a-w2-0d3a66-openshift-default-6c7595fdd6-f92vh"
Sep 12 00:39:47.906238 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:39:47.906106    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3d1a958a-0902-4d78-ba0e-e5b829ce347c-istio-token\") pod \"3d1a958a-0902-4d78-ba0e-e5b829ce347c\" (UID: \"3d1a958a-0902-4d78-ba0e-e5b829ce347c\") "
Sep 12 00:39:47.906238 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:39:47.906176    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3d1a958a-0902-4d78-ba0e-e5b829ce347c-istio-podinfo\") pod \"3d1a958a-0902-4d78-ba0e-e5b829ce347c\" (UID: \"3d1a958a-0902-4d78-ba0e-e5b829ce347c\") "
Sep 12 00:39:47.906238 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:39:47.906216    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3d1a958a-0902-4d78-ba0e-e5b829ce347c-workload-socket\") pod \"3d1a958a-0902-4d78-ba0e-e5b829ce347c\" (UID: \"3d1a958a-0902-4d78-ba0e-e5b829ce347c\") "
Sep 12 00:39:47.907133 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:39:47.906466    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3d1a958a-0902-4d78-ba0e-e5b829ce347c-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "3d1a958a-0902-4d78-ba0e-e5b829ce347c" (UID: "3d1a958a-0902-4d78-ba0e-e5b829ce347c"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 12 00:39:47.907339 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:39:47.907317    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3d1a958a-0902-4d78-ba0e-e5b829ce347c-workload-certs\") pod \"3d1a958a-0902-4d78-ba0e-e5b829ce347c\" (UID: \"3d1a958a-0902-4d78-ba0e-e5b829ce347c\") "
Sep 12 00:39:47.907491 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:39:47.907472    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3d1a958a-0902-4d78-ba0e-e5b829ce347c-credential-socket\") pod \"3d1a958a-0902-4d78-ba0e-e5b829ce347c\" (UID: \"3d1a958a-0902-4d78-ba0e-e5b829ce347c\") "
Sep 12 00:39:47.907703 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:39:47.907686    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3d1a958a-0902-4d78-ba0e-e5b829ce347c-istio-envoy\") pod \"3d1a958a-0902-4d78-ba0e-e5b829ce347c\" (UID: \"3d1a958a-0902-4d78-ba0e-e5b829ce347c\") "
Sep 12 00:39:47.907923 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:39:47.907708    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3d1a958a-0902-4d78-ba0e-e5b829ce347c-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "3d1a958a-0902-4d78-ba0e-e5b829ce347c" (UID: "3d1a958a-0902-4d78-ba0e-e5b829ce347c"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 12 00:39:47.908024 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:39:47.907981    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3d1a958a-0902-4d78-ba0e-e5b829ce347c-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "3d1a958a-0902-4d78-ba0e-e5b829ce347c" (UID: "3d1a958a-0902-4d78-ba0e-e5b829ce347c"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 12 00:39:47.908320 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:39:47.907850    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-ldvqc\" (UniqueName: \"kubernetes.io/projected/3d1a958a-0902-4d78-ba0e-e5b829ce347c-kube-api-access-ldvqc\") pod \"3d1a958a-0902-4d78-ba0e-e5b829ce347c\" (UID: \"3d1a958a-0902-4d78-ba0e-e5b829ce347c\") "
Sep 12 00:39:47.908530 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:39:47.908514    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3d1a958a-0902-4d78-ba0e-e5b829ce347c-istiod-ca-cert\") pod \"3d1a958a-0902-4d78-ba0e-e5b829ce347c\" (UID: \"3d1a958a-0902-4d78-ba0e-e5b829ce347c\") "
Sep 12 00:39:47.909065 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:39:47.909048    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3d1a958a-0902-4d78-ba0e-e5b829ce347c-istio-data\") pod \"3d1a958a-0902-4d78-ba0e-e5b829ce347c\" (UID: \"3d1a958a-0902-4d78-ba0e-e5b829ce347c\") "
Sep 12 00:39:47.909642 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:39:47.909622    2571 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3d1a958a-0902-4d78-ba0e-e5b829ce347c-workload-socket\") on node \"ip-10-0-141-128.ec2.internal\" DevicePath \"\""
Sep 12 00:39:47.909820 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:39:47.909754    2571 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3d1a958a-0902-4d78-ba0e-e5b829ce347c-workload-certs\") on node \"ip-10-0-141-128.ec2.internal\" DevicePath \"\""
Sep 12 00:39:47.909934 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:39:47.909920    2571 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3d1a958a-0902-4d78-ba0e-e5b829ce347c-credential-socket\") on node \"ip-10-0-141-128.ec2.internal\" DevicePath \"\""
Sep 12 00:39:47.910316 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:39:47.909000    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3d1a958a-0902-4d78-ba0e-e5b829ce347c-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "3d1a958a-0902-4d78-ba0e-e5b829ce347c" (UID: "3d1a958a-0902-4d78-ba0e-e5b829ce347c"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 12 00:39:47.910472 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:39:47.910293    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3d1a958a-0902-4d78-ba0e-e5b829ce347c-istio-data" (OuterVolumeSpecName: "istio-data") pod "3d1a958a-0902-4d78-ba0e-e5b829ce347c" (UID: "3d1a958a-0902-4d78-ba0e-e5b829ce347c"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 12 00:39:47.911282 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:39:47.911252    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3d1a958a-0902-4d78-ba0e-e5b829ce347c-istio-token" (OuterVolumeSpecName: "istio-token") pod "3d1a958a-0902-4d78-ba0e-e5b829ce347c" (UID: "3d1a958a-0902-4d78-ba0e-e5b829ce347c"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 12 00:39:47.913521 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:39:47.913487    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3d1a958a-0902-4d78-ba0e-e5b829ce347c-kube-api-access-ldvqc" (OuterVolumeSpecName: "kube-api-access-ldvqc") pod "3d1a958a-0902-4d78-ba0e-e5b829ce347c" (UID: "3d1a958a-0902-4d78-ba0e-e5b829ce347c"). InnerVolumeSpecName "kube-api-access-ldvqc". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 12 00:39:47.913743 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:39:47.913713    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3d1a958a-0902-4d78-ba0e-e5b829ce347c-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "3d1a958a-0902-4d78-ba0e-e5b829ce347c" (UID: "3d1a958a-0902-4d78-ba0e-e5b829ce347c"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 12 00:39:47.916460 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:39:47.916409    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/3d1a958a-0902-4d78-ba0e-e5b829ce347c-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "3d1a958a-0902-4d78-ba0e-e5b829ce347c" (UID: "3d1a958a-0902-4d78-ba0e-e5b829ce347c"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 12 00:39:48.012196 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:39:48.011889    2571 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3d1a958a-0902-4d78-ba0e-e5b829ce347c-istio-token\") on node \"ip-10-0-141-128.ec2.internal\" DevicePath \"\""
Sep 12 00:39:48.012196 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:39:48.011920    2571 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3d1a958a-0902-4d78-ba0e-e5b829ce347c-istio-podinfo\") on node \"ip-10-0-141-128.ec2.internal\" DevicePath \"\""
Sep 12 00:39:48.012196 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:39:48.011933    2571 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3d1a958a-0902-4d78-ba0e-e5b829ce347c-istio-envoy\") on node \"ip-10-0-141-128.ec2.internal\" DevicePath \"\""
Sep 12 00:39:48.012196 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:39:48.011962    2571 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ldvqc\" (UniqueName: \"kubernetes.io/projected/3d1a958a-0902-4d78-ba0e-e5b829ce347c-kube-api-access-ldvqc\") on node \"ip-10-0-141-128.ec2.internal\" DevicePath \"\""
Sep 12 00:39:48.012196 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:39:48.012000    2571 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3d1a958a-0902-4d78-ba0e-e5b829ce347c-istiod-ca-cert\") on node \"ip-10-0-141-128.ec2.internal\" DevicePath \"\""
Sep 12 00:39:48.012196 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:39:48.012019    2571 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3d1a958a-0902-4d78-ba0e-e5b829ce347c-istio-data\") on node \"ip-10-0-141-128.ec2.internal\" DevicePath \"\""
Sep 12 00:39:48.655441 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:39:48.655410    2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-noten-0a0781-openshift-default-86b89df9c8-rc2kf"
Sep 12 00:39:48.655657 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:39:48.655448    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-noten-0a0781-openshift-default-86b89df9c8-rc2kf" event={"ID":"3d1a958a-0902-4d78-ba0e-e5b829ce347c","Type":"ContainerDied","Data":"e96d466f03eb586c324b039059790d482c9adeed60ef8ad5eea923e75319cbbf"}
Sep 12 00:39:48.655657 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:39:48.655511    2571 scope.go:117] "RemoveContainer" containerID="281538e4ffb0316c6123a8e101401ed9997d604db2bd1b85038666e9f0eac6ff"
Sep 12 00:39:48.678653 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:39:48.678629    2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-noten-0a0781-openshift-default-86b89df9c8-rc2kf"]
Sep 12 00:39:48.686744 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:39:48.686718    2571 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-noten-0a0781-openshift-default-86b89df9c8-rc2kf"]
Sep 12 00:39:49.141568 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:39:49.141540    2571 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3d1a958a-0902-4d78-ba0e-e5b829ce347c" path="/var/lib/kubelet/pods/3d1a958a-0902-4d78-ba0e-e5b829ce347c/volumes"
Sep 12 00:39:58.658577 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:39:58.658545    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-f12ab7-59b8766c78-m6swk"
Sep 12 00:39:58.659033 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:39:58.658649    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-shared-b-w2-f12ab7-6bdf9c9f7c-dwjcr"
Sep 12 00:40:03.741756 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:40:03.741672    2571 generic.go:356] "Generic (PLEG): container finished" podID="31532d6d-06e5-4f45-922c-184e13d910fe" containerID="42ee050857dfd38d4e3f907134e7e6c185c8d74bb1d1a99c15edb52334fa7e41" exitCode=0
Sep 12 00:40:03.741756 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:40:03.741725    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w2-f12ab7-openshift-default-678468bf74-42mbj" event={"ID":"31532d6d-06e5-4f45-922c-184e13d910fe","Type":"ContainerDied","Data":"42ee050857dfd38d4e3f907134e7e6c185c8d74bb1d1a99c15edb52334fa7e41"}
Sep 12 00:40:03.742181 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:40:03.742104    2571 scope.go:117] "RemoveContainer" containerID="42ee050857dfd38d4e3f907134e7e6c185c8d74bb1d1a99c15edb52334fa7e41"
Sep 12 00:40:03.743338 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:40:03.743309    2571 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236196Ki","pods":"250"}
Sep 12 00:40:03.743401 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:40:03.743382    2571 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236196Ki","pods":"250"}
Sep 12 00:40:03.743438 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:40:03.743412    2571 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236196Ki","pods":"250"}
Sep 12 00:40:04.749305 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:40:04.749269    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w2-f12ab7-openshift-default-678468bf74-42mbj" event={"ID":"31532d6d-06e5-4f45-922c-184e13d910fe","Type":"ContainerStarted","Data":"d7cd1eff2923a1b9e6d23cd056c1399b3ac9012e15dd445a1a656cedfa921208"}
Sep 12 00:40:05.318601 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:40:05.318561    2571 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-b-w2-f12ab7-openshift-default-678468bf74-42mbj"
Sep 12 00:40:05.318868 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:40:05.318845    2571 patch_prober.go:28] interesting pod/e2e-shared-b-w2-f12ab7-openshift-default-678468bf74-42mbj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.57:15021/healthz/ready\": dial tcp 10.132.0.57:15021: connect: connection refused" start-of-body=
Sep 12 00:40:05.318920 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:40:05.318897    2571 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w2-f12ab7-openshift-default-678468bf74-42mbj" podUID="31532d6d-06e5-4f45-922c-184e13d910fe" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.57:15021/healthz/ready\": dial tcp 10.132.0.57:15021: connect: connection refused"
Sep 12 00:40:06.318366 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:40:06.318330    2571 patch_prober.go:28] interesting pod/e2e-shared-b-w2-f12ab7-openshift-default-678468bf74-42mbj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.57:15021/healthz/ready\": dial tcp 10.132.0.57:15021: connect: connection refused" start-of-body=
Sep 12 00:40:06.318793 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:40:06.318383    2571 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w2-f12ab7-openshift-default-678468bf74-42mbj" podUID="31532d6d-06e5-4f45-922c-184e13d910fe" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.57:15021/healthz/ready\": dial tcp 10.132.0.57:15021: connect: connection refused"
Sep 12 00:40:07.318988 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:40:07.318952    2571 patch_prober.go:28] interesting pod/e2e-shared-b-w2-f12ab7-openshift-default-678468bf74-42mbj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.57:15021/healthz/ready\": dial tcp 10.132.0.57:15021: connect: connection refused" start-of-body=
Sep 12 00:40:07.319389 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:40:07.319009    2571 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w2-f12ab7-openshift-default-678468bf74-42mbj" podUID="31532d6d-06e5-4f45-922c-184e13d910fe" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.57:15021/healthz/ready\": dial tcp 10.132.0.57:15021: connect: connection refused"
Sep 12 00:40:08.318996 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:40:08.318959    2571 patch_prober.go:28] interesting pod/e2e-shared-b-w2-f12ab7-openshift-default-678468bf74-42mbj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.57:15021/healthz/ready\": dial tcp 10.132.0.57:15021: connect: connection refused" start-of-body=
Sep 12 00:40:08.319392 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:40:08.319015    2571 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w2-f12ab7-openshift-default-678468bf74-42mbj" podUID="31532d6d-06e5-4f45-922c-184e13d910fe" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.57:15021/healthz/ready\": dial tcp 10.132.0.57:15021: connect: connection refused"
Sep 12 00:40:09.318485 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:40:09.318453    2571 patch_prober.go:28] interesting pod/e2e-shared-b-w2-f12ab7-openshift-default-678468bf74-42mbj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.57:15021/healthz/ready\": dial tcp 10.132.0.57:15021: connect: connection refused" start-of-body=
Sep 12 00:40:09.318692 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:40:09.318506    2571 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w2-f12ab7-openshift-default-678468bf74-42mbj" podUID="31532d6d-06e5-4f45-922c-184e13d910fe" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.57:15021/healthz/ready\": dial tcp 10.132.0.57:15021: connect: connection refused"
Sep 12 00:40:10.322632 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:40:10.322570    2571 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-b-w2-f12ab7-openshift-default-678468bf74-42mbj"
Sep 12 00:40:10.323071 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:40:10.322886    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-b-w2-f12ab7-openshift-default-678468bf74-42mbj"
Sep 12 00:40:10.323615 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:40:10.323575    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-b-w2-f12ab7-openshift-default-678468bf74-42mbj"
Sep 12 00:40:20.615142 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:40:20.615101    2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-a-a4df66-openshift-default-7cc987766d-5t9vl"]
Sep 12 00:40:20.615638 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:40:20.615383    2571 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-inf-a-a4df66-openshift-default-7cc987766d-5t9vl" podUID="7d867c39-4af2-46d5-9388-95af564c321e" containerName="istio-proxy" containerID="cri-o://0d94e10b694aa046763161a199735a67e603742c8595ff67a64eb8ce0bd990c7" gracePeriod=30
Sep 12 00:40:25.873283 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:40:25.873261    2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-a4df66-openshift-default-7cc987766d-5t9vl"
Sep 12 00:40:25.881277 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:40:25.881222    2571 generic.go:356] "Generic (PLEG): container finished" podID="7d867c39-4af2-46d5-9388-95af564c321e" containerID="0d94e10b694aa046763161a199735a67e603742c8595ff67a64eb8ce0bd990c7" exitCode=0
Sep 12 00:40:25.881372 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:40:25.881273    2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-a4df66-openshift-default-7cc987766d-5t9vl"
Sep 12 00:40:25.881372 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:40:25.881296    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-a4df66-openshift-default-7cc987766d-5t9vl" event={"ID":"7d867c39-4af2-46d5-9388-95af564c321e","Type":"ContainerDied","Data":"0d94e10b694aa046763161a199735a67e603742c8595ff67a64eb8ce0bd990c7"}
Sep 12 00:40:25.881372 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:40:25.881344    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-a4df66-openshift-default-7cc987766d-5t9vl" event={"ID":"7d867c39-4af2-46d5-9388-95af564c321e","Type":"ContainerDied","Data":"234239077e35f21b559fa266c6c98c838e35b6550fd1cbd28b551479203694d8"}
Sep 12 00:40:25.881372 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:40:25.881364    2571 scope.go:117] "RemoveContainer" containerID="0d94e10b694aa046763161a199735a67e603742c8595ff67a64eb8ce0bd990c7"
Sep 12 00:40:25.890025 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:40:25.889999    2571 scope.go:117] "RemoveContainer" containerID="32d161283eb48801c85c052c2573fdbcf60768b47ff0dda2b6f6622f93a51e7e"
Sep 12 00:40:25.897975 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:40:25.897958    2571 scope.go:117] "RemoveContainer" containerID="0d94e10b694aa046763161a199735a67e603742c8595ff67a64eb8ce0bd990c7"
Sep 12 00:40:25.898232 ip-10-0-141-128 kubenswrapper[2571]: E0912 00:40:25.898203    2571 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0d94e10b694aa046763161a199735a67e603742c8595ff67a64eb8ce0bd990c7\": container with ID starting with 0d94e10b694aa046763161a199735a67e603742c8595ff67a64eb8ce0bd990c7 not found: ID does not exist" containerID="0d94e10b694aa046763161a199735a67e603742c8595ff67a64eb8ce0bd990c7"
Sep 12 00:40:25.898336 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:40:25.898241    2571 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0d94e10b694aa046763161a199735a67e603742c8595ff67a64eb8ce0bd990c7"} err="failed to get container status \"0d94e10b694aa046763161a199735a67e603742c8595ff67a64eb8ce0bd990c7\": rpc error: code = NotFound desc = could not find container \"0d94e10b694aa046763161a199735a67e603742c8595ff67a64eb8ce0bd990c7\": container with ID starting with 0d94e10b694aa046763161a199735a67e603742c8595ff67a64eb8ce0bd990c7 not found: ID does not exist"
Sep 12 00:40:25.898336 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:40:25.898267    2571 scope.go:117] "RemoveContainer" containerID="32d161283eb48801c85c052c2573fdbcf60768b47ff0dda2b6f6622f93a51e7e"
Sep 12 00:40:25.898619 ip-10-0-141-128 kubenswrapper[2571]: E0912 00:40:25.898575    2571 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"32d161283eb48801c85c052c2573fdbcf60768b47ff0dda2b6f6622f93a51e7e\": container with ID starting with 32d161283eb48801c85c052c2573fdbcf60768b47ff0dda2b6f6622f93a51e7e not found: ID does not exist" containerID="32d161283eb48801c85c052c2573fdbcf60768b47ff0dda2b6f6622f93a51e7e"
Sep 12 00:40:25.898679 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:40:25.898627    2571 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"32d161283eb48801c85c052c2573fdbcf60768b47ff0dda2b6f6622f93a51e7e"} err="failed to get container status \"32d161283eb48801c85c052c2573fdbcf60768b47ff0dda2b6f6622f93a51e7e\": rpc error: code = NotFound desc = could not find container \"32d161283eb48801c85c052c2573fdbcf60768b47ff0dda2b6f6622f93a51e7e\": container with ID starting with 32d161283eb48801c85c052c2573fdbcf60768b47ff0dda2b6f6622f93a51e7e not found: ID does not exist"
Sep 12 00:40:26.008204 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:40:26.008170    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7d867c39-4af2-46d5-9388-95af564c321e-istio-token\") pod \"7d867c39-4af2-46d5-9388-95af564c321e\" (UID: \"7d867c39-4af2-46d5-9388-95af564c321e\") "
Sep 12 00:40:26.008365 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:40:26.008212    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7d867c39-4af2-46d5-9388-95af564c321e-istio-envoy\") pod \"7d867c39-4af2-46d5-9388-95af564c321e\" (UID: \"7d867c39-4af2-46d5-9388-95af564c321e\") "
Sep 12 00:40:26.008365 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:40:26.008248    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7d867c39-4af2-46d5-9388-95af564c321e-istiod-ca-cert\") pod \"7d867c39-4af2-46d5-9388-95af564c321e\" (UID: \"7d867c39-4af2-46d5-9388-95af564c321e\") "
Sep 12 00:40:26.008365 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:40:26.008279    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7d867c39-4af2-46d5-9388-95af564c321e-istio-data\") pod \"7d867c39-4af2-46d5-9388-95af564c321e\" (UID: \"7d867c39-4af2-46d5-9388-95af564c321e\") "
Sep 12 00:40:26.008365 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:40:26.008311    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7d867c39-4af2-46d5-9388-95af564c321e-istio-podinfo\") pod \"7d867c39-4af2-46d5-9388-95af564c321e\" (UID: \"7d867c39-4af2-46d5-9388-95af564c321e\") "
Sep 12 00:40:26.008365 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:40:26.008333    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-vgtd9\" (UniqueName: \"kubernetes.io/projected/7d867c39-4af2-46d5-9388-95af564c321e-kube-api-access-vgtd9\") pod \"7d867c39-4af2-46d5-9388-95af564c321e\" (UID: \"7d867c39-4af2-46d5-9388-95af564c321e\") "
Sep 12 00:40:26.008365 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:40:26.008358    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7d867c39-4af2-46d5-9388-95af564c321e-workload-socket\") pod \"7d867c39-4af2-46d5-9388-95af564c321e\" (UID: \"7d867c39-4af2-46d5-9388-95af564c321e\") "
Sep 12 00:40:26.008730 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:40:26.008409    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7d867c39-4af2-46d5-9388-95af564c321e-workload-certs\") pod \"7d867c39-4af2-46d5-9388-95af564c321e\" (UID: \"7d867c39-4af2-46d5-9388-95af564c321e\") "
Sep 12 00:40:26.008730 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:40:26.008483    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7d867c39-4af2-46d5-9388-95af564c321e-credential-socket\") pod \"7d867c39-4af2-46d5-9388-95af564c321e\" (UID: \"7d867c39-4af2-46d5-9388-95af564c321e\") "
Sep 12 00:40:26.008730 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:40:26.008614    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7d867c39-4af2-46d5-9388-95af564c321e-istio-data" (OuterVolumeSpecName: "istio-data") pod "7d867c39-4af2-46d5-9388-95af564c321e" (UID: "7d867c39-4af2-46d5-9388-95af564c321e"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 12 00:40:26.008730 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:40:26.008717    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7d867c39-4af2-46d5-9388-95af564c321e-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "7d867c39-4af2-46d5-9388-95af564c321e" (UID: "7d867c39-4af2-46d5-9388-95af564c321e"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 12 00:40:26.008937 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:40:26.008891    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7d867c39-4af2-46d5-9388-95af564c321e-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "7d867c39-4af2-46d5-9388-95af564c321e" (UID: "7d867c39-4af2-46d5-9388-95af564c321e"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 12 00:40:26.008996 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:40:26.008936    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7d867c39-4af2-46d5-9388-95af564c321e-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "7d867c39-4af2-46d5-9388-95af564c321e" (UID: "7d867c39-4af2-46d5-9388-95af564c321e"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 12 00:40:26.009092 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:40:26.009070    2571 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7d867c39-4af2-46d5-9388-95af564c321e-credential-socket\") on node \"ip-10-0-141-128.ec2.internal\" DevicePath \"\""
Sep 12 00:40:26.009157 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:40:26.009096    2571 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7d867c39-4af2-46d5-9388-95af564c321e-istiod-ca-cert\") on node \"ip-10-0-141-128.ec2.internal\" DevicePath \"\""
Sep 12 00:40:26.009157 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:40:26.009110    2571 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7d867c39-4af2-46d5-9388-95af564c321e-istio-data\") on node \"ip-10-0-141-128.ec2.internal\" DevicePath \"\""
Sep 12 00:40:26.009157 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:40:26.009123    2571 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7d867c39-4af2-46d5-9388-95af564c321e-workload-socket\") on node \"ip-10-0-141-128.ec2.internal\" DevicePath \"\""
Sep 12 00:40:26.009316 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:40:26.009181    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7d867c39-4af2-46d5-9388-95af564c321e-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "7d867c39-4af2-46d5-9388-95af564c321e" (UID: "7d867c39-4af2-46d5-9388-95af564c321e"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 12 00:40:26.010506 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:40:26.010478    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7d867c39-4af2-46d5-9388-95af564c321e-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "7d867c39-4af2-46d5-9388-95af564c321e" (UID: "7d867c39-4af2-46d5-9388-95af564c321e"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 12 00:40:26.010795 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:40:26.010776    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7d867c39-4af2-46d5-9388-95af564c321e-kube-api-access-vgtd9" (OuterVolumeSpecName: "kube-api-access-vgtd9") pod "7d867c39-4af2-46d5-9388-95af564c321e" (UID: "7d867c39-4af2-46d5-9388-95af564c321e"). InnerVolumeSpecName "kube-api-access-vgtd9". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 12 00:40:26.010894 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:40:26.010821    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7d867c39-4af2-46d5-9388-95af564c321e-istio-token" (OuterVolumeSpecName: "istio-token") pod "7d867c39-4af2-46d5-9388-95af564c321e" (UID: "7d867c39-4af2-46d5-9388-95af564c321e"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 12 00:40:26.010894 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:40:26.010877    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/7d867c39-4af2-46d5-9388-95af564c321e-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "7d867c39-4af2-46d5-9388-95af564c321e" (UID: "7d867c39-4af2-46d5-9388-95af564c321e"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 12 00:40:26.109989 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:40:26.109964    2571 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7d867c39-4af2-46d5-9388-95af564c321e-istio-token\") on node \"ip-10-0-141-128.ec2.internal\" DevicePath \"\""
Sep 12 00:40:26.109989 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:40:26.109986    2571 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7d867c39-4af2-46d5-9388-95af564c321e-istio-envoy\") on node \"ip-10-0-141-128.ec2.internal\" DevicePath \"\""
Sep 12 00:40:26.109989 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:40:26.109994    2571 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7d867c39-4af2-46d5-9388-95af564c321e-istio-podinfo\") on node \"ip-10-0-141-128.ec2.internal\" DevicePath \"\""
Sep 12 00:40:26.110185 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:40:26.110003    2571 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vgtd9\" (UniqueName: \"kubernetes.io/projected/7d867c39-4af2-46d5-9388-95af564c321e-kube-api-access-vgtd9\") on node \"ip-10-0-141-128.ec2.internal\" DevicePath \"\""
Sep 12 00:40:26.110185 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:40:26.110014    2571 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7d867c39-4af2-46d5-9388-95af564c321e-workload-certs\") on node \"ip-10-0-141-128.ec2.internal\" DevicePath \"\""
Sep 12 00:40:26.203627 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:40:26.203596    2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-a-a4df66-openshift-default-7cc987766d-5t9vl"]
Sep 12 00:40:26.207003 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:40:26.206979    2571 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-inf-a-a4df66-openshift-default-7cc987766d-5t9vl"]
Sep 12 00:40:27.141428 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:40:27.141395    2571 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7d867c39-4af2-46d5-9388-95af564c321e" path="/var/lib/kubelet/pods/7d867c39-4af2-46d5-9388-95af564c321e/volumes"
Sep 12 00:40:29.431225 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:40:29.431189    2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-cfdf5d6c-5469d66b49-857sd"]
Sep 12 00:40:29.433570 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:40:29.433541    2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7d867c39-4af2-46d5-9388-95af564c321e" containerName="istio-proxy"
Sep 12 00:40:29.433570 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:40:29.433569    2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="7d867c39-4af2-46d5-9388-95af564c321e" containerName="istio-proxy"
Sep 12 00:40:29.433762 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:40:29.433614    2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3d1a958a-0902-4d78-ba0e-e5b829ce347c" containerName="istio-proxy"
Sep 12 00:40:29.433762 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:40:29.433623    2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="3d1a958a-0902-4d78-ba0e-e5b829ce347c" containerName="istio-proxy"
Sep 12 00:40:29.433762 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:40:29.433728    2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="7d867c39-4af2-46d5-9388-95af564c321e" containerName="istio-proxy"
Sep 12 00:40:29.433762 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:40:29.433741    2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="3d1a958a-0902-4d78-ba0e-e5b829ce347c" containerName="istio-proxy"
Sep 12 00:40:29.433762 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:40:29.433753    2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="7d867c39-4af2-46d5-9388-95af564c321e" containerName="istio-proxy"
Sep 12 00:40:29.438885 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:40:29.438864    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-cfdf5d6c-5469d66b49-857sd"
Sep 12 00:40:29.440563 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:40:29.440539    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/8148ef6a-0344-4a4d-87b4-1c4b205e3e43-metrics-certs\") pod \"maas-api-e2e-mt-cfdf5d6c-5469d66b49-857sd\" (UID: \"8148ef6a-0344-4a4d-87b4-1c4b205e3e43\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-cfdf5d6c-5469d66b49-857sd"
Sep 12 00:40:29.440670 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:40:29.440605    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zd7vd\" (UniqueName: \"kubernetes.io/projected/8148ef6a-0344-4a4d-87b4-1c4b205e3e43-kube-api-access-zd7vd\") pod \"maas-api-e2e-mt-cfdf5d6c-5469d66b49-857sd\" (UID: \"8148ef6a-0344-4a4d-87b4-1c4b205e3e43\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-cfdf5d6c-5469d66b49-857sd"
Sep 12 00:40:29.440670 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:40:29.440650    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/8148ef6a-0344-4a4d-87b4-1c4b205e3e43-maas-api-tls\") pod \"maas-api-e2e-mt-cfdf5d6c-5469d66b49-857sd\" (UID: \"8148ef6a-0344-4a4d-87b4-1c4b205e3e43\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-cfdf5d6c-5469d66b49-857sd"
Sep 12 00:40:29.440758 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:40:29.440722    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/8148ef6a-0344-4a4d-87b4-1c4b205e3e43-openshift-service-ca\") pod \"maas-api-e2e-mt-cfdf5d6c-5469d66b49-857sd\" (UID: \"8148ef6a-0344-4a4d-87b4-1c4b205e3e43\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-cfdf5d6c-5469d66b49-857sd"
Sep 12 00:40:29.441292 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:40:29.441273    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-mt-cfdf5d6c-serving-cert\""
Sep 12 00:40:29.446100 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:40:29.446080    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-cfdf5d6c-5469d66b49-857sd"]
Sep 12 00:40:29.542653 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:40:29.542553    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/8148ef6a-0344-4a4d-87b4-1c4b205e3e43-metrics-certs\") pod \"maas-api-e2e-mt-cfdf5d6c-5469d66b49-857sd\" (UID: \"8148ef6a-0344-4a4d-87b4-1c4b205e3e43\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-cfdf5d6c-5469d66b49-857sd"
Sep 12 00:40:29.542888 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:40:29.542867    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zd7vd\" (UniqueName: \"kubernetes.io/projected/8148ef6a-0344-4a4d-87b4-1c4b205e3e43-kube-api-access-zd7vd\") pod \"maas-api-e2e-mt-cfdf5d6c-5469d66b49-857sd\" (UID: \"8148ef6a-0344-4a4d-87b4-1c4b205e3e43\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-cfdf5d6c-5469d66b49-857sd"
Sep 12 00:40:29.543028 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:40:29.543015    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/8148ef6a-0344-4a4d-87b4-1c4b205e3e43-maas-api-tls\") pod \"maas-api-e2e-mt-cfdf5d6c-5469d66b49-857sd\" (UID: \"8148ef6a-0344-4a4d-87b4-1c4b205e3e43\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-cfdf5d6c-5469d66b49-857sd"
Sep 12 00:40:29.543191 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:40:29.543177    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/8148ef6a-0344-4a4d-87b4-1c4b205e3e43-openshift-service-ca\") pod \"maas-api-e2e-mt-cfdf5d6c-5469d66b49-857sd\" (UID: \"8148ef6a-0344-4a4d-87b4-1c4b205e3e43\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-cfdf5d6c-5469d66b49-857sd"
Sep 12 00:40:29.544079 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:40:29.544051    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/8148ef6a-0344-4a4d-87b4-1c4b205e3e43-openshift-service-ca\") pod \"maas-api-e2e-mt-cfdf5d6c-5469d66b49-857sd\" (UID: \"8148ef6a-0344-4a4d-87b4-1c4b205e3e43\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-cfdf5d6c-5469d66b49-857sd"
Sep 12 00:40:29.544962 ip-10-0-141-128 kubenswrapper[2571]: E0912 00:40:29.544943    2571 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-mt-cfdf5d6c-serving-cert: secret "maas-api-e2e-mt-cfdf5d6c-serving-cert" not found
Sep 12 00:40:29.545135 ip-10-0-141-128 kubenswrapper[2571]: E0912 00:40:29.545123    2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8148ef6a-0344-4a4d-87b4-1c4b205e3e43-maas-api-tls podName:8148ef6a-0344-4a4d-87b4-1c4b205e3e43 nodeName:}" failed. No retries permitted until 2026-09-12 00:40:30.045101787 +0000 UTC m=+1441.316742288 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/8148ef6a-0344-4a4d-87b4-1c4b205e3e43-maas-api-tls") pod "maas-api-e2e-mt-cfdf5d6c-5469d66b49-857sd" (UID: "8148ef6a-0344-4a4d-87b4-1c4b205e3e43") : secret "maas-api-e2e-mt-cfdf5d6c-serving-cert" not found
Sep 12 00:40:29.548373 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:40:29.548350    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/8148ef6a-0344-4a4d-87b4-1c4b205e3e43-metrics-certs\") pod \"maas-api-e2e-mt-cfdf5d6c-5469d66b49-857sd\" (UID: \"8148ef6a-0344-4a4d-87b4-1c4b205e3e43\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-cfdf5d6c-5469d66b49-857sd"
Sep 12 00:40:29.554281 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:40:29.554252    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zd7vd\" (UniqueName: \"kubernetes.io/projected/8148ef6a-0344-4a4d-87b4-1c4b205e3e43-kube-api-access-zd7vd\") pod \"maas-api-e2e-mt-cfdf5d6c-5469d66b49-857sd\" (UID: \"8148ef6a-0344-4a4d-87b4-1c4b205e3e43\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-cfdf5d6c-5469d66b49-857sd"
Sep 12 00:40:30.047015 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:40:30.046974    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/8148ef6a-0344-4a4d-87b4-1c4b205e3e43-maas-api-tls\") pod \"maas-api-e2e-mt-cfdf5d6c-5469d66b49-857sd\" (UID: \"8148ef6a-0344-4a4d-87b4-1c4b205e3e43\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-cfdf5d6c-5469d66b49-857sd"
Sep 12 00:40:30.049339 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:40:30.049313    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/8148ef6a-0344-4a4d-87b4-1c4b205e3e43-maas-api-tls\") pod \"maas-api-e2e-mt-cfdf5d6c-5469d66b49-857sd\" (UID: \"8148ef6a-0344-4a4d-87b4-1c4b205e3e43\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-cfdf5d6c-5469d66b49-857sd"
Sep 12 00:40:30.053196 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:40:30.053176    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-cfdf5d6c-5469d66b49-857sd"
Sep 12 00:40:30.324198 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:40:30.324166    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-cfdf5d6c-5469d66b49-857sd"]
Sep 12 00:40:30.326170 ip-10-0-141-128 kubenswrapper[2571]: W0912 00:40:30.326143    2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8148ef6a_0344_4a4d_87b4_1c4b205e3e43.slice/crio-a97a0a8525156354fa82b1cdd7228e8b0e1264343f8761d8097e3253bbdda60e WatchSource:0}: Error finding container a97a0a8525156354fa82b1cdd7228e8b0e1264343f8761d8097e3253bbdda60e: Status 404 returned error can't find the container with id a97a0a8525156354fa82b1cdd7228e8b0e1264343f8761d8097e3253bbdda60e
Sep 12 00:40:30.327612 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:40:30.327579    2571 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 12 00:40:30.911727 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:40:30.911692    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-cfdf5d6c-5469d66b49-857sd" event={"ID":"8148ef6a-0344-4a4d-87b4-1c4b205e3e43","Type":"ContainerStarted","Data":"2afac891334d2191974749f7f917d2ecdcc9c901852e249ee49190bcdb55f440"}
Sep 12 00:40:30.912213 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:40:30.911740    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-cfdf5d6c-5469d66b49-857sd" event={"ID":"8148ef6a-0344-4a4d-87b4-1c4b205e3e43","Type":"ContainerStarted","Data":"a97a0a8525156354fa82b1cdd7228e8b0e1264343f8761d8097e3253bbdda60e"}
Sep 12 00:40:30.912213 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:40:30.911771    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-cfdf5d6c-5469d66b49-857sd"
Sep 12 00:40:30.929927 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:40:30.929873    2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-mt-cfdf5d6c-5469d66b49-857sd" podStartSLOduration=1.525435359 podStartE2EDuration="1.929856227s" totalImagesPullingTime="404.420868ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-12 00:40:29 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-12 00:40:30.929076287 +0000 UTC m=+1442.200716798" watchObservedRunningTime="2026-09-12 00:40:30.929856227 +0000 UTC m=+1442.201496755"
Sep 12 00:40:31.179777 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:40:31.179691    2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-b-39d42b-55fc64bff5-cw8nr"]
Sep 12 00:40:31.180366 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:40:31.180334    2571 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-inf-b-39d42b-55fc64bff5-cw8nr" podUID="68dad0be-3290-4975-86d4-657968eb899a" containerName="payload-processing" containerID="cri-o://51f0baabaef9a8d1d506e226ef8c491805baae29a5114fb76d03762868cfc03e" gracePeriod=30
Sep 12 00:40:31.453640 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:40:31.453614    2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-b-39d42b-55fc64bff5-cw8nr"
Sep 12 00:40:31.461749 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:40:31.461723    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/68dad0be-3290-4975-86d4-657968eb899a-plugins-config-volume\") pod \"68dad0be-3290-4975-86d4-657968eb899a\" (UID: \"68dad0be-3290-4975-86d4-657968eb899a\") "
Sep 12 00:40:31.461914 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:40:31.461820    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-tlffh\" (UniqueName: \"kubernetes.io/projected/68dad0be-3290-4975-86d4-657968eb899a-kube-api-access-tlffh\") pod \"68dad0be-3290-4975-86d4-657968eb899a\" (UID: \"68dad0be-3290-4975-86d4-657968eb899a\") "
Sep 12 00:40:31.462172 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:40:31.462146    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/68dad0be-3290-4975-86d4-657968eb899a-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "68dad0be-3290-4975-86d4-657968eb899a" (UID: "68dad0be-3290-4975-86d4-657968eb899a"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 12 00:40:31.463966 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:40:31.463939    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/68dad0be-3290-4975-86d4-657968eb899a-kube-api-access-tlffh" (OuterVolumeSpecName: "kube-api-access-tlffh") pod "68dad0be-3290-4975-86d4-657968eb899a" (UID: "68dad0be-3290-4975-86d4-657968eb899a"). InnerVolumeSpecName "kube-api-access-tlffh". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 12 00:40:31.563570 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:40:31.563533    2571 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-tlffh\" (UniqueName: \"kubernetes.io/projected/68dad0be-3290-4975-86d4-657968eb899a-kube-api-access-tlffh\") on node \"ip-10-0-141-128.ec2.internal\" DevicePath \"\""
Sep 12 00:40:31.563570 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:40:31.563573    2571 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/68dad0be-3290-4975-86d4-657968eb899a-plugins-config-volume\") on node \"ip-10-0-141-128.ec2.internal\" DevicePath \"\""
Sep 12 00:40:31.918003 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:40:31.917916    2571 generic.go:356] "Generic (PLEG): container finished" podID="68dad0be-3290-4975-86d4-657968eb899a" containerID="51f0baabaef9a8d1d506e226ef8c491805baae29a5114fb76d03762868cfc03e" exitCode=0
Sep 12 00:40:31.918003 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:40:31.917964    2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-b-39d42b-55fc64bff5-cw8nr"
Sep 12 00:40:31.918468 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:40:31.918002    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-b-39d42b-55fc64bff5-cw8nr" event={"ID":"68dad0be-3290-4975-86d4-657968eb899a","Type":"ContainerDied","Data":"51f0baabaef9a8d1d506e226ef8c491805baae29a5114fb76d03762868cfc03e"}
Sep 12 00:40:31.918468 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:40:31.918061    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-b-39d42b-55fc64bff5-cw8nr" event={"ID":"68dad0be-3290-4975-86d4-657968eb899a","Type":"ContainerDied","Data":"4e32ef566006d5210480339d5433c4fb7f3c325623149900a32170d13080b5d6"}
Sep 12 00:40:31.918468 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:40:31.918093    2571 scope.go:117] "RemoveContainer" containerID="51f0baabaef9a8d1d506e226ef8c491805baae29a5114fb76d03762868cfc03e"
Sep 12 00:40:31.934793 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:40:31.934774    2571 scope.go:117] "RemoveContainer" containerID="51f0baabaef9a8d1d506e226ef8c491805baae29a5114fb76d03762868cfc03e"
Sep 12 00:40:31.935116 ip-10-0-141-128 kubenswrapper[2571]: E0912 00:40:31.935096    2571 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"51f0baabaef9a8d1d506e226ef8c491805baae29a5114fb76d03762868cfc03e\": container with ID starting with 51f0baabaef9a8d1d506e226ef8c491805baae29a5114fb76d03762868cfc03e not found: ID does not exist" containerID="51f0baabaef9a8d1d506e226ef8c491805baae29a5114fb76d03762868cfc03e"
Sep 12 00:40:31.935210 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:40:31.935129    2571 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"51f0baabaef9a8d1d506e226ef8c491805baae29a5114fb76d03762868cfc03e"} err="failed to get container status \"51f0baabaef9a8d1d506e226ef8c491805baae29a5114fb76d03762868cfc03e\": rpc error: code = NotFound desc = could not find container \"51f0baabaef9a8d1d506e226ef8c491805baae29a5114fb76d03762868cfc03e\": container with ID starting with 51f0baabaef9a8d1d506e226ef8c491805baae29a5114fb76d03762868cfc03e not found: ID does not exist"
Sep 12 00:40:31.943378 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:40:31.943354    2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-b-39d42b-55fc64bff5-cw8nr"]
Sep 12 00:40:31.947217 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:40:31.947195    2571 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-b-39d42b-55fc64bff5-cw8nr"]
Sep 12 00:40:33.141782 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:40:33.141748    2571 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="68dad0be-3290-4975-86d4-657968eb899a" path="/var/lib/kubelet/pods/68dad0be-3290-4975-86d4-657968eb899a/volumes"
Sep 12 00:40:36.924233 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:40:36.924203    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-cfdf5d6c-5469d66b49-857sd"
Sep 12 00:41:01.589695 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:41:01.589660    2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-shared-a-w3-e88680-openshift-default-74bc5c555d-t5984"]
Sep 12 00:41:01.590409 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:41:01.590381    2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="68dad0be-3290-4975-86d4-657968eb899a" containerName="payload-processing"
Sep 12 00:41:01.590409 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:41:01.590411    2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="68dad0be-3290-4975-86d4-657968eb899a" containerName="payload-processing"
Sep 12 00:41:01.590624 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:41:01.590430    2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7d867c39-4af2-46d5-9388-95af564c321e" containerName="istio-proxy"
Sep 12 00:41:01.590624 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:41:01.590438    2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="7d867c39-4af2-46d5-9388-95af564c321e" containerName="istio-proxy"
Sep 12 00:41:01.590624 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:41:01.590556    2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="68dad0be-3290-4975-86d4-657968eb899a" containerName="payload-processing"
Sep 12 00:41:01.594573 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:41:01.594549    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-w3-e88680-openshift-default-74bc5c555d-t5984"
Sep 12 00:41:01.596892 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:41:01.596869    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-shared-a-w3-e88680-openshift-default-dockercfg-n44gc\""
Sep 12 00:41:01.605513 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:41:01.605484    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-a-w3-e88680-openshift-default-74bc5c555d-t5984"]
Sep 12 00:41:01.761054 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:41:01.761017    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e0489e12-24e5-436d-85c9-5a4fd75e2609-istiod-ca-cert\") pod \"e2e-shared-a-w3-e88680-openshift-default-74bc5c555d-t5984\" (UID: \"e0489e12-24e5-436d-85c9-5a4fd75e2609\") " pod="openshift-ingress/e2e-shared-a-w3-e88680-openshift-default-74bc5c555d-t5984"
Sep 12 00:41:01.761248 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:41:01.761072    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e0489e12-24e5-436d-85c9-5a4fd75e2609-credential-socket\") pod \"e2e-shared-a-w3-e88680-openshift-default-74bc5c555d-t5984\" (UID: \"e0489e12-24e5-436d-85c9-5a4fd75e2609\") " pod="openshift-ingress/e2e-shared-a-w3-e88680-openshift-default-74bc5c555d-t5984"
Sep 12 00:41:01.761248 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:41:01.761138    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e0489e12-24e5-436d-85c9-5a4fd75e2609-istio-podinfo\") pod \"e2e-shared-a-w3-e88680-openshift-default-74bc5c555d-t5984\" (UID: \"e0489e12-24e5-436d-85c9-5a4fd75e2609\") " pod="openshift-ingress/e2e-shared-a-w3-e88680-openshift-default-74bc5c555d-t5984"
Sep 12 00:41:01.761248 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:41:01.761162    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e0489e12-24e5-436d-85c9-5a4fd75e2609-istio-token\") pod \"e2e-shared-a-w3-e88680-openshift-default-74bc5c555d-t5984\" (UID: \"e0489e12-24e5-436d-85c9-5a4fd75e2609\") " pod="openshift-ingress/e2e-shared-a-w3-e88680-openshift-default-74bc5c555d-t5984"
Sep 12 00:41:01.761248 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:41:01.761199    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e0489e12-24e5-436d-85c9-5a4fd75e2609-workload-socket\") pod \"e2e-shared-a-w3-e88680-openshift-default-74bc5c555d-t5984\" (UID: \"e0489e12-24e5-436d-85c9-5a4fd75e2609\") " pod="openshift-ingress/e2e-shared-a-w3-e88680-openshift-default-74bc5c555d-t5984"
Sep 12 00:41:01.761248 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:41:01.761225    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e0489e12-24e5-436d-85c9-5a4fd75e2609-istio-data\") pod \"e2e-shared-a-w3-e88680-openshift-default-74bc5c555d-t5984\" (UID: \"e0489e12-24e5-436d-85c9-5a4fd75e2609\") " pod="openshift-ingress/e2e-shared-a-w3-e88680-openshift-default-74bc5c555d-t5984"
Sep 12 00:41:01.761503 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:41:01.761297    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e0489e12-24e5-436d-85c9-5a4fd75e2609-workload-certs\") pod \"e2e-shared-a-w3-e88680-openshift-default-74bc5c555d-t5984\" (UID: \"e0489e12-24e5-436d-85c9-5a4fd75e2609\") " pod="openshift-ingress/e2e-shared-a-w3-e88680-openshift-default-74bc5c555d-t5984"
Sep 12 00:41:01.761503 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:41:01.761340    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tzcwd\" (UniqueName: \"kubernetes.io/projected/e0489e12-24e5-436d-85c9-5a4fd75e2609-kube-api-access-tzcwd\") pod \"e2e-shared-a-w3-e88680-openshift-default-74bc5c555d-t5984\" (UID: \"e0489e12-24e5-436d-85c9-5a4fd75e2609\") " pod="openshift-ingress/e2e-shared-a-w3-e88680-openshift-default-74bc5c555d-t5984"
Sep 12 00:41:01.761503 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:41:01.761426    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e0489e12-24e5-436d-85c9-5a4fd75e2609-istio-envoy\") pod \"e2e-shared-a-w3-e88680-openshift-default-74bc5c555d-t5984\" (UID: \"e0489e12-24e5-436d-85c9-5a4fd75e2609\") " pod="openshift-ingress/e2e-shared-a-w3-e88680-openshift-default-74bc5c555d-t5984"
Sep 12 00:41:01.863030 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:41:01.862945    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e0489e12-24e5-436d-85c9-5a4fd75e2609-istio-envoy\") pod \"e2e-shared-a-w3-e88680-openshift-default-74bc5c555d-t5984\" (UID: \"e0489e12-24e5-436d-85c9-5a4fd75e2609\") " pod="openshift-ingress/e2e-shared-a-w3-e88680-openshift-default-74bc5c555d-t5984"
Sep 12 00:41:01.863030 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:41:01.863014    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e0489e12-24e5-436d-85c9-5a4fd75e2609-istiod-ca-cert\") pod \"e2e-shared-a-w3-e88680-openshift-default-74bc5c555d-t5984\" (UID: \"e0489e12-24e5-436d-85c9-5a4fd75e2609\") " pod="openshift-ingress/e2e-shared-a-w3-e88680-openshift-default-74bc5c555d-t5984"
Sep 12 00:41:01.863254 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:41:01.863048    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e0489e12-24e5-436d-85c9-5a4fd75e2609-credential-socket\") pod \"e2e-shared-a-w3-e88680-openshift-default-74bc5c555d-t5984\" (UID: \"e0489e12-24e5-436d-85c9-5a4fd75e2609\") " pod="openshift-ingress/e2e-shared-a-w3-e88680-openshift-default-74bc5c555d-t5984"
Sep 12 00:41:01.863254 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:41:01.863086    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e0489e12-24e5-436d-85c9-5a4fd75e2609-istio-podinfo\") pod \"e2e-shared-a-w3-e88680-openshift-default-74bc5c555d-t5984\" (UID: \"e0489e12-24e5-436d-85c9-5a4fd75e2609\") " pod="openshift-ingress/e2e-shared-a-w3-e88680-openshift-default-74bc5c555d-t5984"
Sep 12 00:41:01.863254 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:41:01.863115    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e0489e12-24e5-436d-85c9-5a4fd75e2609-istio-token\") pod \"e2e-shared-a-w3-e88680-openshift-default-74bc5c555d-t5984\" (UID: \"e0489e12-24e5-436d-85c9-5a4fd75e2609\") " pod="openshift-ingress/e2e-shared-a-w3-e88680-openshift-default-74bc5c555d-t5984"
Sep 12 00:41:01.863254 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:41:01.863148    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e0489e12-24e5-436d-85c9-5a4fd75e2609-workload-socket\") pod \"e2e-shared-a-w3-e88680-openshift-default-74bc5c555d-t5984\" (UID: \"e0489e12-24e5-436d-85c9-5a4fd75e2609\") " pod="openshift-ingress/e2e-shared-a-w3-e88680-openshift-default-74bc5c555d-t5984"
Sep 12 00:41:01.863254 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:41:01.863176    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e0489e12-24e5-436d-85c9-5a4fd75e2609-istio-data\") pod \"e2e-shared-a-w3-e88680-openshift-default-74bc5c555d-t5984\" (UID: \"e0489e12-24e5-436d-85c9-5a4fd75e2609\") " pod="openshift-ingress/e2e-shared-a-w3-e88680-openshift-default-74bc5c555d-t5984"
Sep 12 00:41:01.863512 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:41:01.863253    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e0489e12-24e5-436d-85c9-5a4fd75e2609-workload-certs\") pod \"e2e-shared-a-w3-e88680-openshift-default-74bc5c555d-t5984\" (UID: \"e0489e12-24e5-436d-85c9-5a4fd75e2609\") " pod="openshift-ingress/e2e-shared-a-w3-e88680-openshift-default-74bc5c555d-t5984"
Sep 12 00:41:01.863512 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:41:01.863295    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tzcwd\" (UniqueName: \"kubernetes.io/projected/e0489e12-24e5-436d-85c9-5a4fd75e2609-kube-api-access-tzcwd\") pod \"e2e-shared-a-w3-e88680-openshift-default-74bc5c555d-t5984\" (UID: \"e0489e12-24e5-436d-85c9-5a4fd75e2609\") " pod="openshift-ingress/e2e-shared-a-w3-e88680-openshift-default-74bc5c555d-t5984"
Sep 12 00:41:01.863512 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:41:01.863492    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e0489e12-24e5-436d-85c9-5a4fd75e2609-credential-socket\") pod \"e2e-shared-a-w3-e88680-openshift-default-74bc5c555d-t5984\" (UID: \"e0489e12-24e5-436d-85c9-5a4fd75e2609\") " pod="openshift-ingress/e2e-shared-a-w3-e88680-openshift-default-74bc5c555d-t5984"
Sep 12 00:41:01.863727 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:41:01.863528    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e0489e12-24e5-436d-85c9-5a4fd75e2609-workload-socket\") pod \"e2e-shared-a-w3-e88680-openshift-default-74bc5c555d-t5984\" (UID: \"e0489e12-24e5-436d-85c9-5a4fd75e2609\") " pod="openshift-ingress/e2e-shared-a-w3-e88680-openshift-default-74bc5c555d-t5984"
Sep 12 00:41:01.863806 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:41:01.863785    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e0489e12-24e5-436d-85c9-5a4fd75e2609-istio-data\") pod \"e2e-shared-a-w3-e88680-openshift-default-74bc5c555d-t5984\" (UID: \"e0489e12-24e5-436d-85c9-5a4fd75e2609\") " pod="openshift-ingress/e2e-shared-a-w3-e88680-openshift-default-74bc5c555d-t5984"
Sep 12 00:41:01.863866 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:41:01.863821    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e0489e12-24e5-436d-85c9-5a4fd75e2609-istiod-ca-cert\") pod \"e2e-shared-a-w3-e88680-openshift-default-74bc5c555d-t5984\" (UID: \"e0489e12-24e5-436d-85c9-5a4fd75e2609\") " pod="openshift-ingress/e2e-shared-a-w3-e88680-openshift-default-74bc5c555d-t5984"
Sep 12 00:41:01.863866 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:41:01.863830    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e0489e12-24e5-436d-85c9-5a4fd75e2609-workload-certs\") pod \"e2e-shared-a-w3-e88680-openshift-default-74bc5c555d-t5984\" (UID: \"e0489e12-24e5-436d-85c9-5a4fd75e2609\") " pod="openshift-ingress/e2e-shared-a-w3-e88680-openshift-default-74bc5c555d-t5984"
Sep 12 00:41:01.866097 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:41:01.866071    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e0489e12-24e5-436d-85c9-5a4fd75e2609-istio-envoy\") pod \"e2e-shared-a-w3-e88680-openshift-default-74bc5c555d-t5984\" (UID: \"e0489e12-24e5-436d-85c9-5a4fd75e2609\") " pod="openshift-ingress/e2e-shared-a-w3-e88680-openshift-default-74bc5c555d-t5984"
Sep 12 00:41:01.866250 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:41:01.866234    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e0489e12-24e5-436d-85c9-5a4fd75e2609-istio-podinfo\") pod \"e2e-shared-a-w3-e88680-openshift-default-74bc5c555d-t5984\" (UID: \"e0489e12-24e5-436d-85c9-5a4fd75e2609\") " pod="openshift-ingress/e2e-shared-a-w3-e88680-openshift-default-74bc5c555d-t5984"
Sep 12 00:41:01.871371 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:41:01.871344    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e0489e12-24e5-436d-85c9-5a4fd75e2609-istio-token\") pod \"e2e-shared-a-w3-e88680-openshift-default-74bc5c555d-t5984\" (UID: \"e0489e12-24e5-436d-85c9-5a4fd75e2609\") " pod="openshift-ingress/e2e-shared-a-w3-e88680-openshift-default-74bc5c555d-t5984"
Sep 12 00:41:01.871455 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:41:01.871378    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tzcwd\" (UniqueName: \"kubernetes.io/projected/e0489e12-24e5-436d-85c9-5a4fd75e2609-kube-api-access-tzcwd\") pod \"e2e-shared-a-w3-e88680-openshift-default-74bc5c555d-t5984\" (UID: \"e0489e12-24e5-436d-85c9-5a4fd75e2609\") " pod="openshift-ingress/e2e-shared-a-w3-e88680-openshift-default-74bc5c555d-t5984"
Sep 12 00:41:01.909480 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:41:01.909453    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-w3-e88680-openshift-default-74bc5c555d-t5984"
Sep 12 00:41:02.190934 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:41:02.190910    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-a-w3-e88680-openshift-default-74bc5c555d-t5984"]
Sep 12 00:41:02.193961 ip-10-0-141-128 kubenswrapper[2571]: W0912 00:41:02.193936    2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode0489e12_24e5_436d_85c9_5a4fd75e2609.slice/crio-98a190b7df758a530cde46ac7fa6107d7888f7be1a2857e8208e2c6bf77997f6 WatchSource:0}: Error finding container 98a190b7df758a530cde46ac7fa6107d7888f7be1a2857e8208e2c6bf77997f6: Status 404 returned error can't find the container with id 98a190b7df758a530cde46ac7fa6107d7888f7be1a2857e8208e2c6bf77997f6
Sep 12 00:41:02.196784 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:41:02.196741    2571 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236196Ki","pods":"250"}
Sep 12 00:41:02.196880 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:41:02.196835    2571 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236196Ki","pods":"250"}
Sep 12 00:41:02.196919 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:41:02.196878    2571 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236196Ki","pods":"250"}
Sep 12 00:41:03.093546 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:41:03.093513    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w3-e88680-openshift-default-74bc5c555d-t5984" event={"ID":"e0489e12-24e5-436d-85c9-5a4fd75e2609","Type":"ContainerStarted","Data":"17a89e1f775a07982df255ca1aa9435ca98aade5e4cdd604ea39a5894274bf25"}
Sep 12 00:41:03.093940 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:41:03.093568    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w3-e88680-openshift-default-74bc5c555d-t5984" event={"ID":"e0489e12-24e5-436d-85c9-5a4fd75e2609","Type":"ContainerStarted","Data":"98a190b7df758a530cde46ac7fa6107d7888f7be1a2857e8208e2c6bf77997f6"}
Sep 12 00:41:03.115193 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:41:03.115142    2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-shared-a-w3-e88680-openshift-default-74bc5c555d-t5984" podStartSLOduration=2.115126651 podStartE2EDuration="2.115126651s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-12 00:41:01 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-12 00:41:03.113497946 +0000 UTC m=+1474.385138482" watchObservedRunningTime="2026-09-12 00:41:03.115126651 +0000 UTC m=+1474.386767160"
Sep 12 00:41:03.910150 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:41:03.910120    2571 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-a-w3-e88680-openshift-default-74bc5c555d-t5984"
Sep 12 00:41:03.915060 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:41:03.915038    2571 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-a-w3-e88680-openshift-default-74bc5c555d-t5984"
Sep 12 00:41:03.971925 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:41:03.971894    2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w0-3c466c-5485d64cc6-p6d9z"]
Sep 12 00:41:03.972387 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:41:03.972350    2571 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-3c466c-5485d64cc6-p6d9z" podUID="95707da3-a50f-48dc-8f29-7e3de07bab92" containerName="maas-api" containerID="cri-o://5c196010a06c8807018fc5b1c11e6a222398a46bc4453779d7118ce437f56805" gracePeriod=30
Sep 12 00:41:04.032388 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:41:04.032350    2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w0-3c466c-59bdd48b77-sv6rh"]
Sep 12 00:41:04.032710 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:41:04.032609    2571 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-worker-w0-3c466c-59bdd48b77-sv6rh" podUID="b0fe6c78-af34-4703-a109-628c4a86ea57" containerName="payload-processing" containerID="cri-o://55787cbbfb6f302299e73b321fdc7aff038a6036f680c8164b7ff5641c86625a" gracePeriod=30
Sep 12 00:41:04.041530 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:41:04.041497    2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w0-3c466c-7445648d54-7tl8k"]
Sep 12 00:41:04.041779 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:41:04.041732    2571 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-3c466c-7445648d54-7tl8k" podUID="66a162de-bb65-4c00-b677-628ef558832f" containerName="payload-pre-processing" containerID="cri-o://4450c08db7824f193cec9ac8d2fd818d3e5209e147f7350f1f8757d6c79fe1a8" gracePeriod=30
Sep 12 00:41:04.103310 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:41:04.103265    2571 generic.go:356] "Generic (PLEG): container finished" podID="95707da3-a50f-48dc-8f29-7e3de07bab92" containerID="5c196010a06c8807018fc5b1c11e6a222398a46bc4453779d7118ce437f56805" exitCode=0
Sep 12 00:41:04.103751 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:41:04.103347    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-3c466c-5485d64cc6-p6d9z" event={"ID":"95707da3-a50f-48dc-8f29-7e3de07bab92","Type":"ContainerDied","Data":"5c196010a06c8807018fc5b1c11e6a222398a46bc4453779d7118ce437f56805"}
Sep 12 00:41:04.103751 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:41:04.103702    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-a-w3-e88680-openshift-default-74bc5c555d-t5984"
Sep 12 00:41:04.104847 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:41:04.104828    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-a-w3-e88680-openshift-default-74bc5c555d-t5984"
Sep 12 00:41:04.366894 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:41:04.366866    2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-worker-w0-3c466c-59bdd48b77-sv6rh"
Sep 12 00:41:04.393359 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:41:04.393335    2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-3c466c-5485d64cc6-p6d9z"
Sep 12 00:41:04.397863 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:41:04.397841    2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-3c466c-7445648d54-7tl8k"
Sep 12 00:41:04.496140 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:41:04.496051    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/95707da3-a50f-48dc-8f29-7e3de07bab92-metrics-certs\") pod \"95707da3-a50f-48dc-8f29-7e3de07bab92\" (UID: \"95707da3-a50f-48dc-8f29-7e3de07bab92\") "
Sep 12 00:41:04.496140 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:41:04.496114    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/95707da3-a50f-48dc-8f29-7e3de07bab92-maas-api-tls\") pod \"95707da3-a50f-48dc-8f29-7e3de07bab92\" (UID: \"95707da3-a50f-48dc-8f29-7e3de07bab92\") "
Sep 12 00:41:04.496341 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:41:04.496167    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/66a162de-bb65-4c00-b677-628ef558832f-plugins-config-volume\") pod \"66a162de-bb65-4c00-b677-628ef558832f\" (UID: \"66a162de-bb65-4c00-b677-628ef558832f\") "
Sep 12 00:41:04.496341 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:41:04.496202    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-bt5zv\" (UniqueName: \"kubernetes.io/projected/b0fe6c78-af34-4703-a109-628c4a86ea57-kube-api-access-bt5zv\") pod \"b0fe6c78-af34-4703-a109-628c4a86ea57\" (UID: \"b0fe6c78-af34-4703-a109-628c4a86ea57\") "
Sep 12 00:41:04.496341 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:41:04.496258    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-w2dbt\" (UniqueName: \"kubernetes.io/projected/66a162de-bb65-4c00-b677-628ef558832f-kube-api-access-w2dbt\") pod \"66a162de-bb65-4c00-b677-628ef558832f\" (UID: \"66a162de-bb65-4c00-b677-628ef558832f\") "
Sep 12 00:41:04.496341 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:41:04.496328    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/95707da3-a50f-48dc-8f29-7e3de07bab92-openshift-service-ca\") pod \"95707da3-a50f-48dc-8f29-7e3de07bab92\" (UID: \"95707da3-a50f-48dc-8f29-7e3de07bab92\") "
Sep 12 00:41:04.496560 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:41:04.496355    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-zc75b\" (UniqueName: \"kubernetes.io/projected/95707da3-a50f-48dc-8f29-7e3de07bab92-kube-api-access-zc75b\") pod \"95707da3-a50f-48dc-8f29-7e3de07bab92\" (UID: \"95707da3-a50f-48dc-8f29-7e3de07bab92\") "
Sep 12 00:41:04.496560 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:41:04.496435    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b0fe6c78-af34-4703-a109-628c4a86ea57-plugins-config-volume\") pod \"b0fe6c78-af34-4703-a109-628c4a86ea57\" (UID: \"b0fe6c78-af34-4703-a109-628c4a86ea57\") "
Sep 12 00:41:04.496560 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:41:04.496517    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/66a162de-bb65-4c00-b677-628ef558832f-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "66a162de-bb65-4c00-b677-628ef558832f" (UID: "66a162de-bb65-4c00-b677-628ef558832f"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 12 00:41:04.496769 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:41:04.496741    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/95707da3-a50f-48dc-8f29-7e3de07bab92-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "95707da3-a50f-48dc-8f29-7e3de07bab92" (UID: "95707da3-a50f-48dc-8f29-7e3de07bab92"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 12 00:41:04.496866 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:41:04.496828    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b0fe6c78-af34-4703-a109-628c4a86ea57-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "b0fe6c78-af34-4703-a109-628c4a86ea57" (UID: "b0fe6c78-af34-4703-a109-628c4a86ea57"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 12 00:41:04.496986 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:41:04.496965    2571 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/66a162de-bb65-4c00-b677-628ef558832f-plugins-config-volume\") on node \"ip-10-0-141-128.ec2.internal\" DevicePath \"\""
Sep 12 00:41:04.497078 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:41:04.496992    2571 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/95707da3-a50f-48dc-8f29-7e3de07bab92-openshift-service-ca\") on node \"ip-10-0-141-128.ec2.internal\" DevicePath \"\""
Sep 12 00:41:04.497078 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:41:04.497006    2571 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b0fe6c78-af34-4703-a109-628c4a86ea57-plugins-config-volume\") on node \"ip-10-0-141-128.ec2.internal\" DevicePath \"\""
Sep 12 00:41:04.498445 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:41:04.498423    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/95707da3-a50f-48dc-8f29-7e3de07bab92-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "95707da3-a50f-48dc-8f29-7e3de07bab92" (UID: "95707da3-a50f-48dc-8f29-7e3de07bab92"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 12 00:41:04.498557 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:41:04.498530    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b0fe6c78-af34-4703-a109-628c4a86ea57-kube-api-access-bt5zv" (OuterVolumeSpecName: "kube-api-access-bt5zv") pod "b0fe6c78-af34-4703-a109-628c4a86ea57" (UID: "b0fe6c78-af34-4703-a109-628c4a86ea57"). InnerVolumeSpecName "kube-api-access-bt5zv". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 12 00:41:04.498557 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:41:04.498535    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/66a162de-bb65-4c00-b677-628ef558832f-kube-api-access-w2dbt" (OuterVolumeSpecName: "kube-api-access-w2dbt") pod "66a162de-bb65-4c00-b677-628ef558832f" (UID: "66a162de-bb65-4c00-b677-628ef558832f"). InnerVolumeSpecName "kube-api-access-w2dbt". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 12 00:41:04.498770 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:41:04.498755    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/95707da3-a50f-48dc-8f29-7e3de07bab92-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "95707da3-a50f-48dc-8f29-7e3de07bab92" (UID: "95707da3-a50f-48dc-8f29-7e3de07bab92"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 12 00:41:04.498855 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:41:04.498840    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/95707da3-a50f-48dc-8f29-7e3de07bab92-kube-api-access-zc75b" (OuterVolumeSpecName: "kube-api-access-zc75b") pod "95707da3-a50f-48dc-8f29-7e3de07bab92" (UID: "95707da3-a50f-48dc-8f29-7e3de07bab92"). InnerVolumeSpecName "kube-api-access-zc75b". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 12 00:41:04.598275 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:41:04.598241    2571 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/95707da3-a50f-48dc-8f29-7e3de07bab92-metrics-certs\") on node \"ip-10-0-141-128.ec2.internal\" DevicePath \"\""
Sep 12 00:41:04.598275 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:41:04.598270    2571 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/95707da3-a50f-48dc-8f29-7e3de07bab92-maas-api-tls\") on node \"ip-10-0-141-128.ec2.internal\" DevicePath \"\""
Sep 12 00:41:04.598275 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:41:04.598281    2571 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-bt5zv\" (UniqueName: \"kubernetes.io/projected/b0fe6c78-af34-4703-a109-628c4a86ea57-kube-api-access-bt5zv\") on node \"ip-10-0-141-128.ec2.internal\" DevicePath \"\""
Sep 12 00:41:04.598512 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:41:04.598291    2571 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-w2dbt\" (UniqueName: \"kubernetes.io/projected/66a162de-bb65-4c00-b677-628ef558832f-kube-api-access-w2dbt\") on node \"ip-10-0-141-128.ec2.internal\" DevicePath \"\""
Sep 12 00:41:04.598512 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:41:04.598300    2571 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-zc75b\" (UniqueName: \"kubernetes.io/projected/95707da3-a50f-48dc-8f29-7e3de07bab92-kube-api-access-zc75b\") on node \"ip-10-0-141-128.ec2.internal\" DevicePath \"\""
Sep 12 00:41:05.110665 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:41:05.110630    2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-3c466c-5485d64cc6-p6d9z"
Sep 12 00:41:05.110665 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:41:05.110644    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-3c466c-5485d64cc6-p6d9z" event={"ID":"95707da3-a50f-48dc-8f29-7e3de07bab92","Type":"ContainerDied","Data":"7f4f3272708cc219cb2e73a26f7158e1cca40fdc0122ddf31ca587747a4f0aae"}
Sep 12 00:41:05.111196 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:41:05.110747    2571 scope.go:117] "RemoveContainer" containerID="5c196010a06c8807018fc5b1c11e6a222398a46bc4453779d7118ce437f56805"
Sep 12 00:41:05.112214 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:41:05.112105    2571 generic.go:356] "Generic (PLEG): container finished" podID="b0fe6c78-af34-4703-a109-628c4a86ea57" containerID="55787cbbfb6f302299e73b321fdc7aff038a6036f680c8164b7ff5641c86625a" exitCode=0
Sep 12 00:41:05.112214 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:41:05.112139    2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-worker-w0-3c466c-59bdd48b77-sv6rh"
Sep 12 00:41:05.112384 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:41:05.112205    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-worker-w0-3c466c-59bdd48b77-sv6rh" event={"ID":"b0fe6c78-af34-4703-a109-628c4a86ea57","Type":"ContainerDied","Data":"55787cbbfb6f302299e73b321fdc7aff038a6036f680c8164b7ff5641c86625a"}
Sep 12 00:41:05.112384 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:41:05.112257    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-worker-w0-3c466c-59bdd48b77-sv6rh" event={"ID":"b0fe6c78-af34-4703-a109-628c4a86ea57","Type":"ContainerDied","Data":"eaf5411ecedc7ddbab3fc98ee3ea50a4d0ef0e294df9022b435012f344806b57"}
Sep 12 00:41:05.113814 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:41:05.113795    2571 generic.go:356] "Generic (PLEG): container finished" podID="66a162de-bb65-4c00-b677-628ef558832f" containerID="4450c08db7824f193cec9ac8d2fd818d3e5209e147f7350f1f8757d6c79fe1a8" exitCode=0
Sep 12 00:41:05.113961 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:41:05.113840    2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-3c466c-7445648d54-7tl8k"
Sep 12 00:41:05.113961 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:41:05.113835    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-3c466c-7445648d54-7tl8k" event={"ID":"66a162de-bb65-4c00-b677-628ef558832f","Type":"ContainerDied","Data":"4450c08db7824f193cec9ac8d2fd818d3e5209e147f7350f1f8757d6c79fe1a8"}
Sep 12 00:41:05.114084 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:41:05.113963    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-3c466c-7445648d54-7tl8k" event={"ID":"66a162de-bb65-4c00-b677-628ef558832f","Type":"ContainerDied","Data":"82a78536d52acdab53fc2e0f9c52d78812742ff5b143a0e71d70a059ed893e5e"}
Sep 12 00:41:05.122355 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:41:05.122329    2571 scope.go:117] "RemoveContainer" containerID="55787cbbfb6f302299e73b321fdc7aff038a6036f680c8164b7ff5641c86625a"
Sep 12 00:41:05.131025 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:41:05.131009    2571 scope.go:117] "RemoveContainer" containerID="55787cbbfb6f302299e73b321fdc7aff038a6036f680c8164b7ff5641c86625a"
Sep 12 00:41:05.131275 ip-10-0-141-128 kubenswrapper[2571]: E0912 00:41:05.131255    2571 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"55787cbbfb6f302299e73b321fdc7aff038a6036f680c8164b7ff5641c86625a\": container with ID starting with 55787cbbfb6f302299e73b321fdc7aff038a6036f680c8164b7ff5641c86625a not found: ID does not exist" containerID="55787cbbfb6f302299e73b321fdc7aff038a6036f680c8164b7ff5641c86625a"
Sep 12 00:41:05.131323 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:41:05.131282    2571 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"55787cbbfb6f302299e73b321fdc7aff038a6036f680c8164b7ff5641c86625a"} err="failed to get container status \"55787cbbfb6f302299e73b321fdc7aff038a6036f680c8164b7ff5641c86625a\": rpc error: code = NotFound desc = could not find container \"55787cbbfb6f302299e73b321fdc7aff038a6036f680c8164b7ff5641c86625a\": container with ID starting with 55787cbbfb6f302299e73b321fdc7aff038a6036f680c8164b7ff5641c86625a not found: ID does not exist"
Sep 12 00:41:05.131323 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:41:05.131298    2571 scope.go:117] "RemoveContainer" containerID="4450c08db7824f193cec9ac8d2fd818d3e5209e147f7350f1f8757d6c79fe1a8"
Sep 12 00:41:05.140540 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:41:05.140506    2571 scope.go:117] "RemoveContainer" containerID="4450c08db7824f193cec9ac8d2fd818d3e5209e147f7350f1f8757d6c79fe1a8"
Sep 12 00:41:05.140841 ip-10-0-141-128 kubenswrapper[2571]: E0912 00:41:05.140818    2571 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4450c08db7824f193cec9ac8d2fd818d3e5209e147f7350f1f8757d6c79fe1a8\": container with ID starting with 4450c08db7824f193cec9ac8d2fd818d3e5209e147f7350f1f8757d6c79fe1a8 not found: ID does not exist" containerID="4450c08db7824f193cec9ac8d2fd818d3e5209e147f7350f1f8757d6c79fe1a8"
Sep 12 00:41:05.140888 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:41:05.140851    2571 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4450c08db7824f193cec9ac8d2fd818d3e5209e147f7350f1f8757d6c79fe1a8"} err="failed to get container status \"4450c08db7824f193cec9ac8d2fd818d3e5209e147f7350f1f8757d6c79fe1a8\": rpc error: code = NotFound desc = could not find container \"4450c08db7824f193cec9ac8d2fd818d3e5209e147f7350f1f8757d6c79fe1a8\": container with ID starting with 4450c08db7824f193cec9ac8d2fd818d3e5209e147f7350f1f8757d6c79fe1a8 not found: ID does not exist"
Sep 12 00:41:05.141667 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:41:05.141650    2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w0-3c466c-5485d64cc6-p6d9z"]
Sep 12 00:41:05.143000 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:41:05.142983    2571 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w0-3c466c-5485d64cc6-p6d9z"]
Sep 12 00:41:05.152804 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:41:05.152784    2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w0-3c466c-59bdd48b77-sv6rh"]
Sep 12 00:41:05.159782 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:41:05.159763    2571 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w0-3c466c-59bdd48b77-sv6rh"]
Sep 12 00:41:05.171353 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:41:05.171329    2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w0-3c466c-7445648d54-7tl8k"]
Sep 12 00:41:05.175337 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:41:05.175316    2571 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w0-3c466c-7445648d54-7tl8k"]
Sep 12 00:41:07.141050 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:41:07.141022    2571 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="66a162de-bb65-4c00-b677-628ef558832f" path="/var/lib/kubelet/pods/66a162de-bb65-4c00-b677-628ef558832f/volumes"
Sep 12 00:41:07.141447 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:41:07.141433    2571 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="95707da3-a50f-48dc-8f29-7e3de07bab92" path="/var/lib/kubelet/pods/95707da3-a50f-48dc-8f29-7e3de07bab92/volumes"
Sep 12 00:41:07.141824 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:41:07.141811    2571 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b0fe6c78-af34-4703-a109-628c4a86ea57" path="/var/lib/kubelet/pods/b0fe6c78-af34-4703-a109-628c4a86ea57/volumes"
Sep 12 00:41:08.328553 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:41:08.328516    2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-e88680-56bc554c8b-j8dc8"]
Sep 12 00:41:08.329282 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:41:08.329259    2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b0fe6c78-af34-4703-a109-628c4a86ea57" containerName="payload-processing"
Sep 12 00:41:08.329355 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:41:08.329287    2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="b0fe6c78-af34-4703-a109-628c4a86ea57" containerName="payload-processing"
Sep 12 00:41:08.329355 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:41:08.329315    2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="95707da3-a50f-48dc-8f29-7e3de07bab92" containerName="maas-api"
Sep 12 00:41:08.329355 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:41:08.329325    2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="95707da3-a50f-48dc-8f29-7e3de07bab92" containerName="maas-api"
Sep 12 00:41:08.329457 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:41:08.329368    2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="66a162de-bb65-4c00-b677-628ef558832f" containerName="payload-pre-processing"
Sep 12 00:41:08.329457 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:41:08.329377    2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="66a162de-bb65-4c00-b677-628ef558832f" containerName="payload-pre-processing"
Sep 12 00:41:08.329521 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:41:08.329471    2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="66a162de-bb65-4c00-b677-628ef558832f" containerName="payload-pre-processing"
Sep 12 00:41:08.329521 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:41:08.329481    2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="95707da3-a50f-48dc-8f29-7e3de07bab92" containerName="maas-api"
Sep 12 00:41:08.329521 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:41:08.329497    2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="b0fe6c78-af34-4703-a109-628c4a86ea57" containerName="payload-processing"
Sep 12 00:41:08.335644 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:41:08.335620    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-e88680-56bc554c8b-j8dc8"
Sep 12 00:41:08.338045 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:41:08.338019    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-shared-a-w3-e88680-serving-cert\""
Sep 12 00:41:08.343108 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:41:08.342915    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-e88680-56bc554c8b-j8dc8"]
Sep 12 00:41:08.435439 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:41:08.435403    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/1c589235-b97a-4cc5-a75f-0670d30a25d0-metrics-certs\") pod \"maas-api-e2e-shared-a-w3-e88680-56bc554c8b-j8dc8\" (UID: \"1c589235-b97a-4cc5-a75f-0670d30a25d0\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-e88680-56bc554c8b-j8dc8"
Sep 12 00:41:08.435633 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:41:08.435445    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/1c589235-b97a-4cc5-a75f-0670d30a25d0-openshift-service-ca\") pod \"maas-api-e2e-shared-a-w3-e88680-56bc554c8b-j8dc8\" (UID: \"1c589235-b97a-4cc5-a75f-0670d30a25d0\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-e88680-56bc554c8b-j8dc8"
Sep 12 00:41:08.435633 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:41:08.435562    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/1c589235-b97a-4cc5-a75f-0670d30a25d0-maas-api-tls\") pod \"maas-api-e2e-shared-a-w3-e88680-56bc554c8b-j8dc8\" (UID: \"1c589235-b97a-4cc5-a75f-0670d30a25d0\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-e88680-56bc554c8b-j8dc8"
Sep 12 00:41:08.438403 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:41:08.438321    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sk7hz\" (UniqueName: \"kubernetes.io/projected/1c589235-b97a-4cc5-a75f-0670d30a25d0-kube-api-access-sk7hz\") pod \"maas-api-e2e-shared-a-w3-e88680-56bc554c8b-j8dc8\" (UID: \"1c589235-b97a-4cc5-a75f-0670d30a25d0\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-e88680-56bc554c8b-j8dc8"
Sep 12 00:41:08.539704 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:41:08.539667    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/1c589235-b97a-4cc5-a75f-0670d30a25d0-metrics-certs\") pod \"maas-api-e2e-shared-a-w3-e88680-56bc554c8b-j8dc8\" (UID: \"1c589235-b97a-4cc5-a75f-0670d30a25d0\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-e88680-56bc554c8b-j8dc8"
Sep 12 00:41:08.539883 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:41:08.539711    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/1c589235-b97a-4cc5-a75f-0670d30a25d0-openshift-service-ca\") pod \"maas-api-e2e-shared-a-w3-e88680-56bc554c8b-j8dc8\" (UID: \"1c589235-b97a-4cc5-a75f-0670d30a25d0\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-e88680-56bc554c8b-j8dc8"
Sep 12 00:41:08.539883 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:41:08.539752    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/1c589235-b97a-4cc5-a75f-0670d30a25d0-maas-api-tls\") pod \"maas-api-e2e-shared-a-w3-e88680-56bc554c8b-j8dc8\" (UID: \"1c589235-b97a-4cc5-a75f-0670d30a25d0\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-e88680-56bc554c8b-j8dc8"
Sep 12 00:41:08.539883 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:41:08.539845    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sk7hz\" (UniqueName: \"kubernetes.io/projected/1c589235-b97a-4cc5-a75f-0670d30a25d0-kube-api-access-sk7hz\") pod \"maas-api-e2e-shared-a-w3-e88680-56bc554c8b-j8dc8\" (UID: \"1c589235-b97a-4cc5-a75f-0670d30a25d0\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-e88680-56bc554c8b-j8dc8"
Sep 12 00:41:08.540043 ip-10-0-141-128 kubenswrapper[2571]: E0912 00:41:08.539924    2571 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-e88680-serving-cert: secret "maas-api-e2e-shared-a-w3-e88680-serving-cert" not found
Sep 12 00:41:08.540043 ip-10-0-141-128 kubenswrapper[2571]: E0912 00:41:08.540002    2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1c589235-b97a-4cc5-a75f-0670d30a25d0-maas-api-tls podName:1c589235-b97a-4cc5-a75f-0670d30a25d0 nodeName:}" failed. No retries permitted until 2026-09-12 00:41:09.039980618 +0000 UTC m=+1480.311621105 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/1c589235-b97a-4cc5-a75f-0670d30a25d0-maas-api-tls") pod "maas-api-e2e-shared-a-w3-e88680-56bc554c8b-j8dc8" (UID: "1c589235-b97a-4cc5-a75f-0670d30a25d0") : secret "maas-api-e2e-shared-a-w3-e88680-serving-cert" not found
Sep 12 00:41:08.540392 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:41:08.540371    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/1c589235-b97a-4cc5-a75f-0670d30a25d0-openshift-service-ca\") pod \"maas-api-e2e-shared-a-w3-e88680-56bc554c8b-j8dc8\" (UID: \"1c589235-b97a-4cc5-a75f-0670d30a25d0\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-e88680-56bc554c8b-j8dc8"
Sep 12 00:41:08.542232 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:41:08.542207    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/1c589235-b97a-4cc5-a75f-0670d30a25d0-metrics-certs\") pod \"maas-api-e2e-shared-a-w3-e88680-56bc554c8b-j8dc8\" (UID: \"1c589235-b97a-4cc5-a75f-0670d30a25d0\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-e88680-56bc554c8b-j8dc8"
Sep 12 00:41:08.547733 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:41:08.547708    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-sk7hz\" (UniqueName: \"kubernetes.io/projected/1c589235-b97a-4cc5-a75f-0670d30a25d0-kube-api-access-sk7hz\") pod \"maas-api-e2e-shared-a-w3-e88680-56bc554c8b-j8dc8\" (UID: \"1c589235-b97a-4cc5-a75f-0670d30a25d0\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-e88680-56bc554c8b-j8dc8"
Sep 12 00:41:09.054220 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:41:09.054168    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/1c589235-b97a-4cc5-a75f-0670d30a25d0-maas-api-tls\") pod \"maas-api-e2e-shared-a-w3-e88680-56bc554c8b-j8dc8\" (UID: \"1c589235-b97a-4cc5-a75f-0670d30a25d0\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-e88680-56bc554c8b-j8dc8"
Sep 12 00:41:09.057674 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:41:09.057467    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/1c589235-b97a-4cc5-a75f-0670d30a25d0-maas-api-tls\") pod \"maas-api-e2e-shared-a-w3-e88680-56bc554c8b-j8dc8\" (UID: \"1c589235-b97a-4cc5-a75f-0670d30a25d0\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-e88680-56bc554c8b-j8dc8"
Sep 12 00:41:09.250858 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:41:09.250817    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-e88680-56bc554c8b-j8dc8"
Sep 12 00:41:09.520832 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:41:09.520806    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-e88680-56bc554c8b-j8dc8"]
Sep 12 00:41:09.522317 ip-10-0-141-128 kubenswrapper[2571]: W0912 00:41:09.522291    2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1c589235_b97a_4cc5_a75f_0670d30a25d0.slice/crio-f48a2d62a9f9579c847554937dfb0bfc606d29137658a84ab2cc20a0a3df326a WatchSource:0}: Error finding container f48a2d62a9f9579c847554937dfb0bfc606d29137658a84ab2cc20a0a3df326a: Status 404 returned error can't find the container with id f48a2d62a9f9579c847554937dfb0bfc606d29137658a84ab2cc20a0a3df326a
Sep 12 00:41:10.154564 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:41:10.154461    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-e88680-56bc554c8b-j8dc8" event={"ID":"1c589235-b97a-4cc5-a75f-0670d30a25d0","Type":"ContainerStarted","Data":"4dfda39c63f69e0c4773acd57d48e00f2a5143d3575d636336931db8ceb998be"}
Sep 12 00:41:10.154564 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:41:10.154512    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-e88680-56bc554c8b-j8dc8" event={"ID":"1c589235-b97a-4cc5-a75f-0670d30a25d0","Type":"ContainerStarted","Data":"f48a2d62a9f9579c847554937dfb0bfc606d29137658a84ab2cc20a0a3df326a"}
Sep 12 00:41:10.154564 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:41:10.154535    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-e88680-56bc554c8b-j8dc8"
Sep 12 00:41:10.174036 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:41:10.173990    2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-e88680-56bc554c8b-j8dc8" podStartSLOduration=1.848478209 podStartE2EDuration="2.173976729s" totalImagesPullingTime="325.49852ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-12 00:41:08 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-12 00:41:10.171402457 +0000 UTC m=+1481.443042967" watchObservedRunningTime="2026-09-12 00:41:10.173976729 +0000 UTC m=+1481.445617263"
Sep 12 00:41:16.165089 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:41:16.165049    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-e88680-56bc554c8b-j8dc8"
Sep 12 00:41:24.235506 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:41:24.235471    2571 generic.go:356] "Generic (PLEG): container finished" podID="e0489e12-24e5-436d-85c9-5a4fd75e2609" containerID="17a89e1f775a07982df255ca1aa9435ca98aade5e4cdd604ea39a5894274bf25" exitCode=0
Sep 12 00:41:24.236038 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:41:24.235548    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w3-e88680-openshift-default-74bc5c555d-t5984" event={"ID":"e0489e12-24e5-436d-85c9-5a4fd75e2609","Type":"ContainerDied","Data":"17a89e1f775a07982df255ca1aa9435ca98aade5e4cdd604ea39a5894274bf25"}
Sep 12 00:41:24.236139 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:41:24.236121    2571 scope.go:117] "RemoveContainer" containerID="17a89e1f775a07982df255ca1aa9435ca98aade5e4cdd604ea39a5894274bf25"
Sep 12 00:41:24.237314 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:41:24.237285    2571 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236196Ki","pods":"250"}
Sep 12 00:41:24.237406 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:41:24.237350    2571 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236196Ki","pods":"250"}
Sep 12 00:41:24.237406 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:41:24.237381    2571 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236196Ki","pods":"250"}
Sep 12 00:41:25.242474 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:41:25.242430    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w3-e88680-openshift-default-74bc5c555d-t5984" event={"ID":"e0489e12-24e5-436d-85c9-5a4fd75e2609","Type":"ContainerStarted","Data":"b8ff125b3eb02fd9991137f6f860052d5ebf423eeaf1457a5deb55d330f17d43"}
Sep 12 00:41:25.909955 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:41:25.909920    2571 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-a-w3-e88680-openshift-default-74bc5c555d-t5984"
Sep 12 00:41:25.910160 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:41:25.910143    2571 patch_prober.go:28] interesting pod/e2e-shared-a-w3-e88680-openshift-default-74bc5c555d-t5984 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.61:15021/healthz/ready\": dial tcp 10.132.0.61:15021: connect: connection refused" start-of-body=
Sep 12 00:41:25.910207 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:41:25.910185    2571 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w3-e88680-openshift-default-74bc5c555d-t5984" podUID="e0489e12-24e5-436d-85c9-5a4fd75e2609" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.61:15021/healthz/ready\": dial tcp 10.132.0.61:15021: connect: connection refused"
Sep 12 00:41:26.910516 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:41:26.910483    2571 patch_prober.go:28] interesting pod/e2e-shared-a-w3-e88680-openshift-default-74bc5c555d-t5984 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.61:15021/healthz/ready\": dial tcp 10.132.0.61:15021: connect: connection refused" start-of-body=
Sep 12 00:41:26.910901 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:41:26.910551    2571 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w3-e88680-openshift-default-74bc5c555d-t5984" podUID="e0489e12-24e5-436d-85c9-5a4fd75e2609" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.61:15021/healthz/ready\": dial tcp 10.132.0.61:15021: connect: connection refused"
Sep 12 00:41:27.910446 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:41:27.910398    2571 patch_prober.go:28] interesting pod/e2e-shared-a-w3-e88680-openshift-default-74bc5c555d-t5984 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.61:15021/healthz/ready\": dial tcp 10.132.0.61:15021: connect: connection refused" start-of-body=
Sep 12 00:41:27.910683 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:41:27.910490    2571 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w3-e88680-openshift-default-74bc5c555d-t5984" podUID="e0489e12-24e5-436d-85c9-5a4fd75e2609" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.61:15021/healthz/ready\": dial tcp 10.132.0.61:15021: connect: connection refused"
Sep 12 00:41:28.909925 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:41:28.909888    2571 patch_prober.go:28] interesting pod/e2e-shared-a-w3-e88680-openshift-default-74bc5c555d-t5984 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.61:15021/healthz/ready\": dial tcp 10.132.0.61:15021: connect: connection refused" start-of-body=
Sep 12 00:41:28.910099 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:41:28.909959    2571 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w3-e88680-openshift-default-74bc5c555d-t5984" podUID="e0489e12-24e5-436d-85c9-5a4fd75e2609" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.61:15021/healthz/ready\": dial tcp 10.132.0.61:15021: connect: connection refused"
Sep 12 00:41:29.910712 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:41:29.910671    2571 patch_prober.go:28] interesting pod/e2e-shared-a-w3-e88680-openshift-default-74bc5c555d-t5984 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.61:15021/healthz/ready\": dial tcp 10.132.0.61:15021: connect: connection refused" start-of-body=
Sep 12 00:41:29.911304 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:41:29.911274    2571 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w3-e88680-openshift-default-74bc5c555d-t5984" podUID="e0489e12-24e5-436d-85c9-5a4fd75e2609" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.61:15021/healthz/ready\": dial tcp 10.132.0.61:15021: connect: connection refused"
Sep 12 00:41:31.113873 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:41:31.113840    2571 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-a-w3-e88680-openshift-default-74bc5c555d-t5984"
Sep 12 00:41:31.114365 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:41:31.114234    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-a-w3-e88680-openshift-default-74bc5c555d-t5984"
Sep 12 00:41:31.115382 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:41:31.115356    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-a-w3-e88680-openshift-default-74bc5c555d-t5984"
Sep 12 00:41:43.308563 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:41:43.308523    2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w3-e37bfb-756cdf47bb-5xkrt"]
Sep 12 00:41:43.313326 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:41:43.313309    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-e37bfb-756cdf47bb-5xkrt"
Sep 12 00:41:43.315978 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:41:43.315952    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-shared-b-w3-e37bfb-dockercfg-mqtrg\""
Sep 12 00:41:43.316096 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:41:43.315952    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-shared-b-w3-e37bfb\""
Sep 12 00:41:43.320866 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:41:43.320843    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w3-e37bfb-756cdf47bb-5xkrt"]
Sep 12 00:41:43.325752 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:41:43.325730    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2hthn\" (UniqueName: \"kubernetes.io/projected/e238a42c-0d7c-4737-88ba-c05d757287b6-kube-api-access-2hthn\") pod \"payload-pre-processing-e2e-shared-b-w3-e37bfb-756cdf47bb-5xkrt\" (UID: \"e238a42c-0d7c-4737-88ba-c05d757287b6\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-e37bfb-756cdf47bb-5xkrt"
Sep 12 00:41:43.325866 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:41:43.325797    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e238a42c-0d7c-4737-88ba-c05d757287b6-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-b-w3-e37bfb-756cdf47bb-5xkrt\" (UID: \"e238a42c-0d7c-4737-88ba-c05d757287b6\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-e37bfb-756cdf47bb-5xkrt"
Sep 12 00:41:43.343400 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:41:43.343373    2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w3-e37bfb-54fd9bcc55-6x788"]
Sep 12 00:41:43.347018 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:41:43.347001    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-b-w3-e37bfb-54fd9bcc55-6x788"
Sep 12 00:41:43.356619 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:41:43.356580    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w3-e37bfb-54fd9bcc55-6x788"]
Sep 12 00:41:43.426893 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:41:43.426855    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e238a42c-0d7c-4737-88ba-c05d757287b6-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-b-w3-e37bfb-756cdf47bb-5xkrt\" (UID: \"e238a42c-0d7c-4737-88ba-c05d757287b6\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-e37bfb-756cdf47bb-5xkrt"
Sep 12 00:41:43.427198 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:41:43.427170    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/96f2b972-193f-41c7-aee5-20da72c3eb14-plugins-config-volume\") pod \"payload-processing-e2e-shared-b-w3-e37bfb-54fd9bcc55-6x788\" (UID: \"96f2b972-193f-41c7-aee5-20da72c3eb14\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w3-e37bfb-54fd9bcc55-6x788"
Sep 12 00:41:43.427447 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:41:43.427426    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2hthn\" (UniqueName: \"kubernetes.io/projected/e238a42c-0d7c-4737-88ba-c05d757287b6-kube-api-access-2hthn\") pod \"payload-pre-processing-e2e-shared-b-w3-e37bfb-756cdf47bb-5xkrt\" (UID: \"e238a42c-0d7c-4737-88ba-c05d757287b6\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-e37bfb-756cdf47bb-5xkrt"
Sep 12 00:41:43.427660 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:41:43.427630    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-td6hf\" (UniqueName: \"kubernetes.io/projected/96f2b972-193f-41c7-aee5-20da72c3eb14-kube-api-access-td6hf\") pod \"payload-processing-e2e-shared-b-w3-e37bfb-54fd9bcc55-6x788\" (UID: \"96f2b972-193f-41c7-aee5-20da72c3eb14\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w3-e37bfb-54fd9bcc55-6x788"
Sep 12 00:41:43.429230 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:41:43.429205    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e238a42c-0d7c-4737-88ba-c05d757287b6-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-b-w3-e37bfb-756cdf47bb-5xkrt\" (UID: \"e238a42c-0d7c-4737-88ba-c05d757287b6\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-e37bfb-756cdf47bb-5xkrt"
Sep 12 00:41:43.439716 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:41:43.439690    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2hthn\" (UniqueName: \"kubernetes.io/projected/e238a42c-0d7c-4737-88ba-c05d757287b6-kube-api-access-2hthn\") pod \"payload-pre-processing-e2e-shared-b-w3-e37bfb-756cdf47bb-5xkrt\" (UID: \"e238a42c-0d7c-4737-88ba-c05d757287b6\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-e37bfb-756cdf47bb-5xkrt"
Sep 12 00:41:43.528915 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:41:43.528875    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-td6hf\" (UniqueName: \"kubernetes.io/projected/96f2b972-193f-41c7-aee5-20da72c3eb14-kube-api-access-td6hf\") pod \"payload-processing-e2e-shared-b-w3-e37bfb-54fd9bcc55-6x788\" (UID: \"96f2b972-193f-41c7-aee5-20da72c3eb14\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w3-e37bfb-54fd9bcc55-6x788"
Sep 12 00:41:43.529105 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:41:43.528965    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/96f2b972-193f-41c7-aee5-20da72c3eb14-plugins-config-volume\") pod \"payload-processing-e2e-shared-b-w3-e37bfb-54fd9bcc55-6x788\" (UID: \"96f2b972-193f-41c7-aee5-20da72c3eb14\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w3-e37bfb-54fd9bcc55-6x788"
Sep 12 00:41:43.529674 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:41:43.529656    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/96f2b972-193f-41c7-aee5-20da72c3eb14-plugins-config-volume\") pod \"payload-processing-e2e-shared-b-w3-e37bfb-54fd9bcc55-6x788\" (UID: \"96f2b972-193f-41c7-aee5-20da72c3eb14\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w3-e37bfb-54fd9bcc55-6x788"
Sep 12 00:41:43.537067 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:41:43.537040    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-td6hf\" (UniqueName: \"kubernetes.io/projected/96f2b972-193f-41c7-aee5-20da72c3eb14-kube-api-access-td6hf\") pod \"payload-processing-e2e-shared-b-w3-e37bfb-54fd9bcc55-6x788\" (UID: \"96f2b972-193f-41c7-aee5-20da72c3eb14\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w3-e37bfb-54fd9bcc55-6x788"
Sep 12 00:41:43.625776 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:41:43.625676    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-e37bfb-756cdf47bb-5xkrt"
Sep 12 00:41:43.659843 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:41:43.659812    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-b-w3-e37bfb-54fd9bcc55-6x788"
Sep 12 00:41:43.903636 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:41:43.901296    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w3-e37bfb-756cdf47bb-5xkrt"]
Sep 12 00:41:43.955300 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:41:43.955272    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w3-e37bfb-54fd9bcc55-6x788"]
Sep 12 00:41:43.957233 ip-10-0-141-128 kubenswrapper[2571]: W0912 00:41:43.957209    2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod96f2b972_193f_41c7_aee5_20da72c3eb14.slice/crio-a6dc79406d25c64a504020e51410faaeff95a900eb385864fa12ce767a0ac65c WatchSource:0}: Error finding container a6dc79406d25c64a504020e51410faaeff95a900eb385864fa12ce767a0ac65c: Status 404 returned error can't find the container with id a6dc79406d25c64a504020e51410faaeff95a900eb385864fa12ce767a0ac65c
Sep 12 00:41:44.369429 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:41:44.369387    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-b-w3-e37bfb-54fd9bcc55-6x788" event={"ID":"96f2b972-193f-41c7-aee5-20da72c3eb14","Type":"ContainerStarted","Data":"288a263cdf7d093fbe5dbb0e8b60afb61f82e2a92dd6b3dc12247789379d0779"}
Sep 12 00:41:44.369914 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:41:44.369448    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-b-w3-e37bfb-54fd9bcc55-6x788" event={"ID":"96f2b972-193f-41c7-aee5-20da72c3eb14","Type":"ContainerStarted","Data":"a6dc79406d25c64a504020e51410faaeff95a900eb385864fa12ce767a0ac65c"}
Sep 12 00:41:44.369914 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:41:44.369578    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-shared-b-w3-e37bfb-54fd9bcc55-6x788"
Sep 12 00:41:44.371298 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:41:44.371271    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-e37bfb-756cdf47bb-5xkrt" event={"ID":"e238a42c-0d7c-4737-88ba-c05d757287b6","Type":"ContainerStarted","Data":"15f13df7ab16b7704c29b987e9300c53fd91376c87a45de2337c8596f6de50d1"}
Sep 12 00:41:44.371425 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:41:44.371322    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-e37bfb-756cdf47bb-5xkrt"
Sep 12 00:41:44.371425 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:41:44.371344    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-e37bfb-756cdf47bb-5xkrt" event={"ID":"e238a42c-0d7c-4737-88ba-c05d757287b6","Type":"ContainerStarted","Data":"3f8dee12860f401361c8d1ea7bddddc7b2743852ba5055c03f2f55e18fcd0801"}
Sep 12 00:41:44.386572 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:41:44.386519    2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-shared-b-w3-e37bfb-54fd9bcc55-6x788" podStartSLOduration=1.386501505 podStartE2EDuration="1.386501505s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-12 00:41:43 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-12 00:41:44.385864361 +0000 UTC m=+1515.657504882" watchObservedRunningTime="2026-09-12 00:41:44.386501505 +0000 UTC m=+1515.658142020"
Sep 12 00:41:44.406685 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:41:44.406559    2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-e37bfb-756cdf47bb-5xkrt" podStartSLOduration=1.406538629 podStartE2EDuration="1.406538629s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-12 00:41:43 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-12 00:41:44.403493247 +0000 UTC m=+1515.675133756" watchObservedRunningTime="2026-09-12 00:41:44.406538629 +0000 UTC m=+1515.678179140"
Sep 12 00:41:55.379997 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:41:55.379909    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-shared-b-w3-e37bfb-54fd9bcc55-6x788"
Sep 12 00:41:55.380375 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:41:55.380141    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-e37bfb-756cdf47bb-5xkrt"
Sep 12 00:41:56.131072 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:41:56.131035    2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-cfdf5d6c-5469d66b49-857sd"]
Sep 12 00:41:56.131324 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:41:56.131298    2571 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-mt-cfdf5d6c-5469d66b49-857sd" podUID="8148ef6a-0344-4a4d-87b4-1c4b205e3e43" containerName="maas-api" containerID="cri-o://2afac891334d2191974749f7f917d2ecdcc9c901852e249ee49190bcdb55f440" gracePeriod=30
Sep 12 00:41:56.389385 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:41:56.389320    2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-cfdf5d6c-5469d66b49-857sd"
Sep 12 00:41:56.445914 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:41:56.445882    2571 generic.go:356] "Generic (PLEG): container finished" podID="8148ef6a-0344-4a4d-87b4-1c4b205e3e43" containerID="2afac891334d2191974749f7f917d2ecdcc9c901852e249ee49190bcdb55f440" exitCode=0
Sep 12 00:41:56.446107 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:41:56.445932    2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-cfdf5d6c-5469d66b49-857sd"
Sep 12 00:41:56.446107 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:41:56.445962    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-cfdf5d6c-5469d66b49-857sd" event={"ID":"8148ef6a-0344-4a4d-87b4-1c4b205e3e43","Type":"ContainerDied","Data":"2afac891334d2191974749f7f917d2ecdcc9c901852e249ee49190bcdb55f440"}
Sep 12 00:41:56.446107 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:41:56.446019    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-cfdf5d6c-5469d66b49-857sd" event={"ID":"8148ef6a-0344-4a4d-87b4-1c4b205e3e43","Type":"ContainerDied","Data":"a97a0a8525156354fa82b1cdd7228e8b0e1264343f8761d8097e3253bbdda60e"}
Sep 12 00:41:56.446107 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:41:56.446033    2571 scope.go:117] "RemoveContainer" containerID="2afac891334d2191974749f7f917d2ecdcc9c901852e249ee49190bcdb55f440"
Sep 12 00:41:56.455706 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:41:56.455689    2571 scope.go:117] "RemoveContainer" containerID="2afac891334d2191974749f7f917d2ecdcc9c901852e249ee49190bcdb55f440"
Sep 12 00:41:56.456024 ip-10-0-141-128 kubenswrapper[2571]: E0912 00:41:56.456005    2571 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2afac891334d2191974749f7f917d2ecdcc9c901852e249ee49190bcdb55f440\": container with ID starting with 2afac891334d2191974749f7f917d2ecdcc9c901852e249ee49190bcdb55f440 not found: ID does not exist" containerID="2afac891334d2191974749f7f917d2ecdcc9c901852e249ee49190bcdb55f440"
Sep 12 00:41:56.456109 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:41:56.456033    2571 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2afac891334d2191974749f7f917d2ecdcc9c901852e249ee49190bcdb55f440"} err="failed to get container status \"2afac891334d2191974749f7f917d2ecdcc9c901852e249ee49190bcdb55f440\": rpc error: code = NotFound desc = could not find container \"2afac891334d2191974749f7f917d2ecdcc9c901852e249ee49190bcdb55f440\": container with ID starting with 2afac891334d2191974749f7f917d2ecdcc9c901852e249ee49190bcdb55f440 not found: ID does not exist"
Sep 12 00:41:56.464631 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:41:56.464454    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/8148ef6a-0344-4a4d-87b4-1c4b205e3e43-maas-api-tls\") pod \"8148ef6a-0344-4a4d-87b4-1c4b205e3e43\" (UID: \"8148ef6a-0344-4a4d-87b4-1c4b205e3e43\") "
Sep 12 00:41:56.464631 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:41:56.464510    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-zd7vd\" (UniqueName: \"kubernetes.io/projected/8148ef6a-0344-4a4d-87b4-1c4b205e3e43-kube-api-access-zd7vd\") pod \"8148ef6a-0344-4a4d-87b4-1c4b205e3e43\" (UID: \"8148ef6a-0344-4a4d-87b4-1c4b205e3e43\") "
Sep 12 00:41:56.464631 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:41:56.464548    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/8148ef6a-0344-4a4d-87b4-1c4b205e3e43-metrics-certs\") pod \"8148ef6a-0344-4a4d-87b4-1c4b205e3e43\" (UID: \"8148ef6a-0344-4a4d-87b4-1c4b205e3e43\") "
Sep 12 00:41:56.465006 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:41:56.464986    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/8148ef6a-0344-4a4d-87b4-1c4b205e3e43-openshift-service-ca\") pod \"8148ef6a-0344-4a4d-87b4-1c4b205e3e43\" (UID: \"8148ef6a-0344-4a4d-87b4-1c4b205e3e43\") "
Sep 12 00:41:56.465508 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:41:56.465469    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8148ef6a-0344-4a4d-87b4-1c4b205e3e43-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "8148ef6a-0344-4a4d-87b4-1c4b205e3e43" (UID: "8148ef6a-0344-4a4d-87b4-1c4b205e3e43"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 12 00:41:56.467332 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:41:56.467193    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8148ef6a-0344-4a4d-87b4-1c4b205e3e43-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "8148ef6a-0344-4a4d-87b4-1c4b205e3e43" (UID: "8148ef6a-0344-4a4d-87b4-1c4b205e3e43"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 12 00:41:56.467332 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:41:56.467295    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8148ef6a-0344-4a4d-87b4-1c4b205e3e43-kube-api-access-zd7vd" (OuterVolumeSpecName: "kube-api-access-zd7vd") pod "8148ef6a-0344-4a4d-87b4-1c4b205e3e43" (UID: "8148ef6a-0344-4a4d-87b4-1c4b205e3e43"). InnerVolumeSpecName "kube-api-access-zd7vd". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 12 00:41:56.467489 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:41:56.467368    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8148ef6a-0344-4a4d-87b4-1c4b205e3e43-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "8148ef6a-0344-4a4d-87b4-1c4b205e3e43" (UID: "8148ef6a-0344-4a4d-87b4-1c4b205e3e43"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 12 00:41:56.566331 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:41:56.566286    2571 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/8148ef6a-0344-4a4d-87b4-1c4b205e3e43-maas-api-tls\") on node \"ip-10-0-141-128.ec2.internal\" DevicePath \"\""
Sep 12 00:41:56.566331 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:41:56.566327    2571 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-zd7vd\" (UniqueName: \"kubernetes.io/projected/8148ef6a-0344-4a4d-87b4-1c4b205e3e43-kube-api-access-zd7vd\") on node \"ip-10-0-141-128.ec2.internal\" DevicePath \"\""
Sep 12 00:41:56.566579 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:41:56.566342    2571 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/8148ef6a-0344-4a4d-87b4-1c4b205e3e43-metrics-certs\") on node \"ip-10-0-141-128.ec2.internal\" DevicePath \"\""
Sep 12 00:41:56.566579 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:41:56.566355    2571 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/8148ef6a-0344-4a4d-87b4-1c4b205e3e43-openshift-service-ca\") on node \"ip-10-0-141-128.ec2.internal\" DevicePath \"\""
Sep 12 00:41:56.767893 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:41:56.767856    2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-cfdf5d6c-5469d66b49-857sd"]
Sep 12 00:41:56.770818 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:41:56.770786    2571 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-cfdf5d6c-5469d66b49-857sd"]
Sep 12 00:41:57.141202 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:41:57.141126    2571 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8148ef6a-0344-4a4d-87b4-1c4b205e3e43" path="/var/lib/kubelet/pods/8148ef6a-0344-4a4d-87b4-1c4b205e3e43/volumes"
Sep 12 00:44:05.030655 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:44:05.030622    2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-api-a-829959-openshift-default-697f5547d4-mdscp"]
Sep 12 00:44:05.031333 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:44:05.031314    2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8148ef6a-0344-4a4d-87b4-1c4b205e3e43" containerName="maas-api"
Sep 12 00:44:05.031386 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:44:05.031337    2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="8148ef6a-0344-4a4d-87b4-1c4b205e3e43" containerName="maas-api"
Sep 12 00:44:05.031462 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:44:05.031451    2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="8148ef6a-0344-4a4d-87b4-1c4b205e3e43" containerName="maas-api"
Sep 12 00:44:05.034961 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:44:05.034934    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-829959-openshift-default-697f5547d4-mdscp"
Sep 12 00:44:05.037580 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:44:05.037557    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-api-a-829959-openshift-default-dockercfg-wnnpx\""
Sep 12 00:44:05.046064 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:44:05.046040    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-a-829959-openshift-default-697f5547d4-mdscp"]
Sep 12 00:44:05.054789 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:44:05.054760    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/537eb92d-71ea-4651-a204-fee3a3728ace-istio-data\") pod \"e2e-api-a-829959-openshift-default-697f5547d4-mdscp\" (UID: \"537eb92d-71ea-4651-a204-fee3a3728ace\") " pod="openshift-ingress/e2e-api-a-829959-openshift-default-697f5547d4-mdscp"
Sep 12 00:44:05.054918 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:44:05.054808    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/537eb92d-71ea-4651-a204-fee3a3728ace-workload-socket\") pod \"e2e-api-a-829959-openshift-default-697f5547d4-mdscp\" (UID: \"537eb92d-71ea-4651-a204-fee3a3728ace\") " pod="openshift-ingress/e2e-api-a-829959-openshift-default-697f5547d4-mdscp"
Sep 12 00:44:05.054918 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:44:05.054835    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/537eb92d-71ea-4651-a204-fee3a3728ace-istiod-ca-cert\") pod \"e2e-api-a-829959-openshift-default-697f5547d4-mdscp\" (UID: \"537eb92d-71ea-4651-a204-fee3a3728ace\") " pod="openshift-ingress/e2e-api-a-829959-openshift-default-697f5547d4-mdscp"
Sep 12 00:44:05.055045 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:44:05.054983    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/537eb92d-71ea-4651-a204-fee3a3728ace-istio-token\") pod \"e2e-api-a-829959-openshift-default-697f5547d4-mdscp\" (UID: \"537eb92d-71ea-4651-a204-fee3a3728ace\") " pod="openshift-ingress/e2e-api-a-829959-openshift-default-697f5547d4-mdscp"
Sep 12 00:44:05.055045 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:44:05.055023    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-swspr\" (UniqueName: \"kubernetes.io/projected/537eb92d-71ea-4651-a204-fee3a3728ace-kube-api-access-swspr\") pod \"e2e-api-a-829959-openshift-default-697f5547d4-mdscp\" (UID: \"537eb92d-71ea-4651-a204-fee3a3728ace\") " pod="openshift-ingress/e2e-api-a-829959-openshift-default-697f5547d4-mdscp"
Sep 12 00:44:05.056152 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:44:05.055100    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/537eb92d-71ea-4651-a204-fee3a3728ace-credential-socket\") pod \"e2e-api-a-829959-openshift-default-697f5547d4-mdscp\" (UID: \"537eb92d-71ea-4651-a204-fee3a3728ace\") " pod="openshift-ingress/e2e-api-a-829959-openshift-default-697f5547d4-mdscp"
Sep 12 00:44:05.056152 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:44:05.055227    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/537eb92d-71ea-4651-a204-fee3a3728ace-istio-podinfo\") pod \"e2e-api-a-829959-openshift-default-697f5547d4-mdscp\" (UID: \"537eb92d-71ea-4651-a204-fee3a3728ace\") " pod="openshift-ingress/e2e-api-a-829959-openshift-default-697f5547d4-mdscp"
Sep 12 00:44:05.056152 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:44:05.055305    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/537eb92d-71ea-4651-a204-fee3a3728ace-workload-certs\") pod \"e2e-api-a-829959-openshift-default-697f5547d4-mdscp\" (UID: \"537eb92d-71ea-4651-a204-fee3a3728ace\") " pod="openshift-ingress/e2e-api-a-829959-openshift-default-697f5547d4-mdscp"
Sep 12 00:44:05.056152 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:44:05.055363    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/537eb92d-71ea-4651-a204-fee3a3728ace-istio-envoy\") pod \"e2e-api-a-829959-openshift-default-697f5547d4-mdscp\" (UID: \"537eb92d-71ea-4651-a204-fee3a3728ace\") " pod="openshift-ingress/e2e-api-a-829959-openshift-default-697f5547d4-mdscp"
Sep 12 00:44:05.158607 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:44:05.157927    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/537eb92d-71ea-4651-a204-fee3a3728ace-istio-token\") pod \"e2e-api-a-829959-openshift-default-697f5547d4-mdscp\" (UID: \"537eb92d-71ea-4651-a204-fee3a3728ace\") " pod="openshift-ingress/e2e-api-a-829959-openshift-default-697f5547d4-mdscp"
Sep 12 00:44:05.158607 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:44:05.158009    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-swspr\" (UniqueName: \"kubernetes.io/projected/537eb92d-71ea-4651-a204-fee3a3728ace-kube-api-access-swspr\") pod \"e2e-api-a-829959-openshift-default-697f5547d4-mdscp\" (UID: \"537eb92d-71ea-4651-a204-fee3a3728ace\") " pod="openshift-ingress/e2e-api-a-829959-openshift-default-697f5547d4-mdscp"
Sep 12 00:44:05.158607 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:44:05.158081    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/537eb92d-71ea-4651-a204-fee3a3728ace-credential-socket\") pod \"e2e-api-a-829959-openshift-default-697f5547d4-mdscp\" (UID: \"537eb92d-71ea-4651-a204-fee3a3728ace\") " pod="openshift-ingress/e2e-api-a-829959-openshift-default-697f5547d4-mdscp"
Sep 12 00:44:05.158607 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:44:05.158120    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/537eb92d-71ea-4651-a204-fee3a3728ace-istio-podinfo\") pod \"e2e-api-a-829959-openshift-default-697f5547d4-mdscp\" (UID: \"537eb92d-71ea-4651-a204-fee3a3728ace\") " pod="openshift-ingress/e2e-api-a-829959-openshift-default-697f5547d4-mdscp"
Sep 12 00:44:05.158607 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:44:05.158153    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/537eb92d-71ea-4651-a204-fee3a3728ace-workload-certs\") pod \"e2e-api-a-829959-openshift-default-697f5547d4-mdscp\" (UID: \"537eb92d-71ea-4651-a204-fee3a3728ace\") " pod="openshift-ingress/e2e-api-a-829959-openshift-default-697f5547d4-mdscp"
Sep 12 00:44:05.158607 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:44:05.158203    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/537eb92d-71ea-4651-a204-fee3a3728ace-istio-envoy\") pod \"e2e-api-a-829959-openshift-default-697f5547d4-mdscp\" (UID: \"537eb92d-71ea-4651-a204-fee3a3728ace\") " pod="openshift-ingress/e2e-api-a-829959-openshift-default-697f5547d4-mdscp"
Sep 12 00:44:05.158607 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:44:05.158272    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/537eb92d-71ea-4651-a204-fee3a3728ace-istio-data\") pod \"e2e-api-a-829959-openshift-default-697f5547d4-mdscp\" (UID: \"537eb92d-71ea-4651-a204-fee3a3728ace\") " pod="openshift-ingress/e2e-api-a-829959-openshift-default-697f5547d4-mdscp"
Sep 12 00:44:05.158607 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:44:05.158318    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/537eb92d-71ea-4651-a204-fee3a3728ace-workload-socket\") pod \"e2e-api-a-829959-openshift-default-697f5547d4-mdscp\" (UID: \"537eb92d-71ea-4651-a204-fee3a3728ace\") " pod="openshift-ingress/e2e-api-a-829959-openshift-default-697f5547d4-mdscp"
Sep 12 00:44:05.158607 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:44:05.158343    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/537eb92d-71ea-4651-a204-fee3a3728ace-istiod-ca-cert\") pod \"e2e-api-a-829959-openshift-default-697f5547d4-mdscp\" (UID: \"537eb92d-71ea-4651-a204-fee3a3728ace\") " pod="openshift-ingress/e2e-api-a-829959-openshift-default-697f5547d4-mdscp"
Sep 12 00:44:05.159232 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:44:05.159186    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/537eb92d-71ea-4651-a204-fee3a3728ace-istiod-ca-cert\") pod \"e2e-api-a-829959-openshift-default-697f5547d4-mdscp\" (UID: \"537eb92d-71ea-4651-a204-fee3a3728ace\") " pod="openshift-ingress/e2e-api-a-829959-openshift-default-697f5547d4-mdscp"
Sep 12 00:44:05.160267 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:44:05.159437    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/537eb92d-71ea-4651-a204-fee3a3728ace-istio-data\") pod \"e2e-api-a-829959-openshift-default-697f5547d4-mdscp\" (UID: \"537eb92d-71ea-4651-a204-fee3a3728ace\") " pod="openshift-ingress/e2e-api-a-829959-openshift-default-697f5547d4-mdscp"
Sep 12 00:44:05.160267 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:44:05.159659    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/537eb92d-71ea-4651-a204-fee3a3728ace-workload-certs\") pod \"e2e-api-a-829959-openshift-default-697f5547d4-mdscp\" (UID: \"537eb92d-71ea-4651-a204-fee3a3728ace\") " pod="openshift-ingress/e2e-api-a-829959-openshift-default-697f5547d4-mdscp"
Sep 12 00:44:05.160267 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:44:05.160022    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/537eb92d-71ea-4651-a204-fee3a3728ace-credential-socket\") pod \"e2e-api-a-829959-openshift-default-697f5547d4-mdscp\" (UID: \"537eb92d-71ea-4651-a204-fee3a3728ace\") " pod="openshift-ingress/e2e-api-a-829959-openshift-default-697f5547d4-mdscp"
Sep 12 00:44:05.160267 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:44:05.160193    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/537eb92d-71ea-4651-a204-fee3a3728ace-workload-socket\") pod \"e2e-api-a-829959-openshift-default-697f5547d4-mdscp\" (UID: \"537eb92d-71ea-4651-a204-fee3a3728ace\") " pod="openshift-ingress/e2e-api-a-829959-openshift-default-697f5547d4-mdscp"
Sep 12 00:44:05.165281 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:44:05.162761    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/537eb92d-71ea-4651-a204-fee3a3728ace-istio-podinfo\") pod \"e2e-api-a-829959-openshift-default-697f5547d4-mdscp\" (UID: \"537eb92d-71ea-4651-a204-fee3a3728ace\") " pod="openshift-ingress/e2e-api-a-829959-openshift-default-697f5547d4-mdscp"
Sep 12 00:44:05.165281 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:44:05.163929    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/537eb92d-71ea-4651-a204-fee3a3728ace-istio-envoy\") pod \"e2e-api-a-829959-openshift-default-697f5547d4-mdscp\" (UID: \"537eb92d-71ea-4651-a204-fee3a3728ace\") " pod="openshift-ingress/e2e-api-a-829959-openshift-default-697f5547d4-mdscp"
Sep 12 00:44:05.170320 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:44:05.168609    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/537eb92d-71ea-4651-a204-fee3a3728ace-istio-token\") pod \"e2e-api-a-829959-openshift-default-697f5547d4-mdscp\" (UID: \"537eb92d-71ea-4651-a204-fee3a3728ace\") " pod="openshift-ingress/e2e-api-a-829959-openshift-default-697f5547d4-mdscp"
Sep 12 00:44:05.170320 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:44:05.170275    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-swspr\" (UniqueName: \"kubernetes.io/projected/537eb92d-71ea-4651-a204-fee3a3728ace-kube-api-access-swspr\") pod \"e2e-api-a-829959-openshift-default-697f5547d4-mdscp\" (UID: \"537eb92d-71ea-4651-a204-fee3a3728ace\") " pod="openshift-ingress/e2e-api-a-829959-openshift-default-697f5547d4-mdscp"
Sep 12 00:44:05.348415 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:44:05.348333    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-829959-openshift-default-697f5547d4-mdscp"
Sep 12 00:44:05.627023 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:44:05.626978    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-a-829959-openshift-default-697f5547d4-mdscp"]
Sep 12 00:44:05.628017 ip-10-0-141-128 kubenswrapper[2571]: W0912 00:44:05.627983    2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod537eb92d_71ea_4651_a204_fee3a3728ace.slice/crio-bb4f14ce467b78ab31a76ccd0f3888a199240d45aca9ae4f6e613978dc115e0d WatchSource:0}: Error finding container bb4f14ce467b78ab31a76ccd0f3888a199240d45aca9ae4f6e613978dc115e0d: Status 404 returned error can't find the container with id bb4f14ce467b78ab31a76ccd0f3888a199240d45aca9ae4f6e613978dc115e0d
Sep 12 00:44:05.637571 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:44:05.630768    2571 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236196Ki","pods":"250"}
Sep 12 00:44:05.637571 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:44:05.630935    2571 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236196Ki","pods":"250"}
Sep 12 00:44:05.637571 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:44:05.631017    2571 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236196Ki","pods":"250"}
Sep 12 00:44:06.137902 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:44:06.137870    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-829959-openshift-default-697f5547d4-mdscp" event={"ID":"537eb92d-71ea-4651-a204-fee3a3728ace","Type":"ContainerStarted","Data":"008bb091ad9633a923f97e1dfb3ba4fd19090eb62151240048687efa42d7a203"}
Sep 12 00:44:06.138291 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:44:06.137916    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-829959-openshift-default-697f5547d4-mdscp" event={"ID":"537eb92d-71ea-4651-a204-fee3a3728ace","Type":"ContainerStarted","Data":"bb4f14ce467b78ab31a76ccd0f3888a199240d45aca9ae4f6e613978dc115e0d"}
Sep 12 00:44:06.160345 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:44:06.160300    2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-api-a-829959-openshift-default-697f5547d4-mdscp" podStartSLOduration=1.160285849 podStartE2EDuration="1.160285849s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-12 00:44:05 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-12 00:44:06.158117445 +0000 UTC m=+1657.429757955" watchObservedRunningTime="2026-09-12 00:44:06.160285849 +0000 UTC m=+1657.431926358"
Sep 12 00:44:06.348940 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:44:06.348910    2571 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-a-829959-openshift-default-697f5547d4-mdscp"
Sep 12 00:44:06.354565 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:44:06.354543    2571 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-a-829959-openshift-default-697f5547d4-mdscp"
Sep 12 00:44:07.144491 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:44:07.144466    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-a-829959-openshift-default-697f5547d4-mdscp"
Sep 12 00:44:07.145403 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:44:07.145384    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-a-829959-openshift-default-697f5547d4-mdscp"
Sep 12 00:44:30.284934 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:44:30.284901    2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w4-139288-dsk8h"]
Sep 12 00:44:30.292231 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:44:30.292187    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w4-139288-dsk8h"
Sep 12 00:44:30.294811 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:44:30.294782    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-rlnvx\""
Sep 12 00:44:30.295568 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:44:30.295533    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w4-139288-dsk8h"]
Sep 12 00:44:30.414491 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:44:30.414455    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/5dd9c16e-644a-421e-813c-4a519128d91c-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-worker-w4-139288-dsk8h\" (UID: \"5dd9c16e-644a-421e-813c-4a519128d91c\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w4-139288-dsk8h"
Sep 12 00:44:30.515657 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:44:30.515621    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/5dd9c16e-644a-421e-813c-4a519128d91c-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-worker-w4-139288-dsk8h\" (UID: \"5dd9c16e-644a-421e-813c-4a519128d91c\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w4-139288-dsk8h"
Sep 12 00:44:30.516256 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:44:30.516234    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/5dd9c16e-644a-421e-813c-4a519128d91c-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-worker-w4-139288-dsk8h\" (UID: \"5dd9c16e-644a-421e-813c-4a519128d91c\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w4-139288-dsk8h"
Sep 12 00:44:30.606941 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:44:30.606854    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w4-139288-dsk8h"
Sep 12 00:44:30.922300 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:44:30.922267    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w4-139288-dsk8h"]
Sep 12 00:44:30.923196 ip-10-0-141-128 kubenswrapper[2571]: W0912 00:44:30.923170    2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5dd9c16e_644a_421e_813c_4a519128d91c.slice/crio-585442e0fae553dc02e07bc4f5287b30436c511ead207d72e5e7b6cd7d28a307 WatchSource:0}: Error finding container 585442e0fae553dc02e07bc4f5287b30436c511ead207d72e5e7b6cd7d28a307: Status 404 returned error can't find the container with id 585442e0fae553dc02e07bc4f5287b30436c511ead207d72e5e7b6cd7d28a307
Sep 12 00:44:31.288025 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:44:31.287988    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w4-139288-dsk8h" event={"ID":"5dd9c16e-644a-421e-813c-4a519128d91c","Type":"ContainerStarted","Data":"585442e0fae553dc02e07bc4f5287b30436c511ead207d72e5e7b6cd7d28a307"}
Sep 12 00:44:33.843702 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:44:33.840528    2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-api-b-593845-openshift-default-54dcc87546-pvswg"]
Sep 12 00:44:33.857875 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:44:33.857838    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-593845-openshift-default-54dcc87546-pvswg"
Sep 12 00:44:33.874310 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:44:33.874272    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-api-b-593845-openshift-default-dockercfg-t4zxd\""
Sep 12 00:44:33.880655 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:44:33.880624    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-b-593845-openshift-default-54dcc87546-pvswg"]
Sep 12 00:44:33.959664 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:44:33.957776    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7eef202a-38a8-4944-a4ef-7ffe4bc0ae1e-istio-podinfo\") pod \"e2e-api-b-593845-openshift-default-54dcc87546-pvswg\" (UID: \"7eef202a-38a8-4944-a4ef-7ffe4bc0ae1e\") " pod="openshift-ingress/e2e-api-b-593845-openshift-default-54dcc87546-pvswg"
Sep 12 00:44:33.959664 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:44:33.957826    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7eef202a-38a8-4944-a4ef-7ffe4bc0ae1e-credential-socket\") pod \"e2e-api-b-593845-openshift-default-54dcc87546-pvswg\" (UID: \"7eef202a-38a8-4944-a4ef-7ffe4bc0ae1e\") " pod="openshift-ingress/e2e-api-b-593845-openshift-default-54dcc87546-pvswg"
Sep 12 00:44:33.959664 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:44:33.957860    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7eef202a-38a8-4944-a4ef-7ffe4bc0ae1e-istio-envoy\") pod \"e2e-api-b-593845-openshift-default-54dcc87546-pvswg\" (UID: \"7eef202a-38a8-4944-a4ef-7ffe4bc0ae1e\") " pod="openshift-ingress/e2e-api-b-593845-openshift-default-54dcc87546-pvswg"
Sep 12 00:44:33.959664 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:44:33.957915    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7eef202a-38a8-4944-a4ef-7ffe4bc0ae1e-istio-data\") pod \"e2e-api-b-593845-openshift-default-54dcc87546-pvswg\" (UID: \"7eef202a-38a8-4944-a4ef-7ffe4bc0ae1e\") " pod="openshift-ingress/e2e-api-b-593845-openshift-default-54dcc87546-pvswg"
Sep 12 00:44:33.959664 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:44:33.957943    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7eef202a-38a8-4944-a4ef-7ffe4bc0ae1e-workload-certs\") pod \"e2e-api-b-593845-openshift-default-54dcc87546-pvswg\" (UID: \"7eef202a-38a8-4944-a4ef-7ffe4bc0ae1e\") " pod="openshift-ingress/e2e-api-b-593845-openshift-default-54dcc87546-pvswg"
Sep 12 00:44:33.959664 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:44:33.957971    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7eef202a-38a8-4944-a4ef-7ffe4bc0ae1e-istiod-ca-cert\") pod \"e2e-api-b-593845-openshift-default-54dcc87546-pvswg\" (UID: \"7eef202a-38a8-4944-a4ef-7ffe4bc0ae1e\") " pod="openshift-ingress/e2e-api-b-593845-openshift-default-54dcc87546-pvswg"
Sep 12 00:44:33.959664 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:44:33.957999    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7eef202a-38a8-4944-a4ef-7ffe4bc0ae1e-workload-socket\") pod \"e2e-api-b-593845-openshift-default-54dcc87546-pvswg\" (UID: \"7eef202a-38a8-4944-a4ef-7ffe4bc0ae1e\") " pod="openshift-ingress/e2e-api-b-593845-openshift-default-54dcc87546-pvswg"
Sep 12 00:44:33.959664 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:44:33.958025    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4j26h\" (UniqueName: \"kubernetes.io/projected/7eef202a-38a8-4944-a4ef-7ffe4bc0ae1e-kube-api-access-4j26h\") pod \"e2e-api-b-593845-openshift-default-54dcc87546-pvswg\" (UID: \"7eef202a-38a8-4944-a4ef-7ffe4bc0ae1e\") " pod="openshift-ingress/e2e-api-b-593845-openshift-default-54dcc87546-pvswg"
Sep 12 00:44:33.959664 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:44:33.958066    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7eef202a-38a8-4944-a4ef-7ffe4bc0ae1e-istio-token\") pod \"e2e-api-b-593845-openshift-default-54dcc87546-pvswg\" (UID: \"7eef202a-38a8-4944-a4ef-7ffe4bc0ae1e\") " pod="openshift-ingress/e2e-api-b-593845-openshift-default-54dcc87546-pvswg"
Sep 12 00:44:34.059192 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:44:34.059141    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7eef202a-38a8-4944-a4ef-7ffe4bc0ae1e-istio-podinfo\") pod \"e2e-api-b-593845-openshift-default-54dcc87546-pvswg\" (UID: \"7eef202a-38a8-4944-a4ef-7ffe4bc0ae1e\") " pod="openshift-ingress/e2e-api-b-593845-openshift-default-54dcc87546-pvswg"
Sep 12 00:44:34.059400 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:44:34.059202    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7eef202a-38a8-4944-a4ef-7ffe4bc0ae1e-credential-socket\") pod \"e2e-api-b-593845-openshift-default-54dcc87546-pvswg\" (UID: \"7eef202a-38a8-4944-a4ef-7ffe4bc0ae1e\") " pod="openshift-ingress/e2e-api-b-593845-openshift-default-54dcc87546-pvswg"
Sep 12 00:44:34.059400 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:44:34.059235    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7eef202a-38a8-4944-a4ef-7ffe4bc0ae1e-istio-envoy\") pod \"e2e-api-b-593845-openshift-default-54dcc87546-pvswg\" (UID: \"7eef202a-38a8-4944-a4ef-7ffe4bc0ae1e\") " pod="openshift-ingress/e2e-api-b-593845-openshift-default-54dcc87546-pvswg"
Sep 12 00:44:34.059400 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:44:34.059283    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7eef202a-38a8-4944-a4ef-7ffe4bc0ae1e-istio-data\") pod \"e2e-api-b-593845-openshift-default-54dcc87546-pvswg\" (UID: \"7eef202a-38a8-4944-a4ef-7ffe4bc0ae1e\") " pod="openshift-ingress/e2e-api-b-593845-openshift-default-54dcc87546-pvswg"
Sep 12 00:44:34.059400 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:44:34.059310    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7eef202a-38a8-4944-a4ef-7ffe4bc0ae1e-workload-certs\") pod \"e2e-api-b-593845-openshift-default-54dcc87546-pvswg\" (UID: \"7eef202a-38a8-4944-a4ef-7ffe4bc0ae1e\") " pod="openshift-ingress/e2e-api-b-593845-openshift-default-54dcc87546-pvswg"
Sep 12 00:44:34.059400 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:44:34.059336    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7eef202a-38a8-4944-a4ef-7ffe4bc0ae1e-istiod-ca-cert\") pod \"e2e-api-b-593845-openshift-default-54dcc87546-pvswg\" (UID: \"7eef202a-38a8-4944-a4ef-7ffe4bc0ae1e\") " pod="openshift-ingress/e2e-api-b-593845-openshift-default-54dcc87546-pvswg"
Sep 12 00:44:34.059400 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:44:34.059364    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7eef202a-38a8-4944-a4ef-7ffe4bc0ae1e-workload-socket\") pod \"e2e-api-b-593845-openshift-default-54dcc87546-pvswg\" (UID: \"7eef202a-38a8-4944-a4ef-7ffe4bc0ae1e\") " pod="openshift-ingress/e2e-api-b-593845-openshift-default-54dcc87546-pvswg"
Sep 12 00:44:34.059400 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:44:34.059389    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4j26h\" (UniqueName: \"kubernetes.io/projected/7eef202a-38a8-4944-a4ef-7ffe4bc0ae1e-kube-api-access-4j26h\") pod \"e2e-api-b-593845-openshift-default-54dcc87546-pvswg\" (UID: \"7eef202a-38a8-4944-a4ef-7ffe4bc0ae1e\") " pod="openshift-ingress/e2e-api-b-593845-openshift-default-54dcc87546-pvswg"
Sep 12 00:44:34.059793 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:44:34.059427    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7eef202a-38a8-4944-a4ef-7ffe4bc0ae1e-istio-token\") pod \"e2e-api-b-593845-openshift-default-54dcc87546-pvswg\" (UID: \"7eef202a-38a8-4944-a4ef-7ffe4bc0ae1e\") " pod="openshift-ingress/e2e-api-b-593845-openshift-default-54dcc87546-pvswg"
Sep 12 00:44:34.059793 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:44:34.059716    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7eef202a-38a8-4944-a4ef-7ffe4bc0ae1e-istio-data\") pod \"e2e-api-b-593845-openshift-default-54dcc87546-pvswg\" (UID: \"7eef202a-38a8-4944-a4ef-7ffe4bc0ae1e\") " pod="openshift-ingress/e2e-api-b-593845-openshift-default-54dcc87546-pvswg"
Sep 12 00:44:34.060289 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:44:34.060260    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7eef202a-38a8-4944-a4ef-7ffe4bc0ae1e-istiod-ca-cert\") pod \"e2e-api-b-593845-openshift-default-54dcc87546-pvswg\" (UID: \"7eef202a-38a8-4944-a4ef-7ffe4bc0ae1e\") " pod="openshift-ingress/e2e-api-b-593845-openshift-default-54dcc87546-pvswg"
Sep 12 00:44:34.060625 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:44:34.060562    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7eef202a-38a8-4944-a4ef-7ffe4bc0ae1e-workload-certs\") pod \"e2e-api-b-593845-openshift-default-54dcc87546-pvswg\" (UID: \"7eef202a-38a8-4944-a4ef-7ffe4bc0ae1e\") " pod="openshift-ingress/e2e-api-b-593845-openshift-default-54dcc87546-pvswg"
Sep 12 00:44:34.060713 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:44:34.060649    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7eef202a-38a8-4944-a4ef-7ffe4bc0ae1e-workload-socket\") pod \"e2e-api-b-593845-openshift-default-54dcc87546-pvswg\" (UID: \"7eef202a-38a8-4944-a4ef-7ffe4bc0ae1e\") " pod="openshift-ingress/e2e-api-b-593845-openshift-default-54dcc87546-pvswg"
Sep 12 00:44:34.061008 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:44:34.060980    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7eef202a-38a8-4944-a4ef-7ffe4bc0ae1e-credential-socket\") pod \"e2e-api-b-593845-openshift-default-54dcc87546-pvswg\" (UID: \"7eef202a-38a8-4944-a4ef-7ffe4bc0ae1e\") " pod="openshift-ingress/e2e-api-b-593845-openshift-default-54dcc87546-pvswg"
Sep 12 00:44:34.062341 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:44:34.062316    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7eef202a-38a8-4944-a4ef-7ffe4bc0ae1e-istio-envoy\") pod \"e2e-api-b-593845-openshift-default-54dcc87546-pvswg\" (UID: \"7eef202a-38a8-4944-a4ef-7ffe4bc0ae1e\") " pod="openshift-ingress/e2e-api-b-593845-openshift-default-54dcc87546-pvswg"
Sep 12 00:44:34.062706 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:44:34.062684    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7eef202a-38a8-4944-a4ef-7ffe4bc0ae1e-istio-podinfo\") pod \"e2e-api-b-593845-openshift-default-54dcc87546-pvswg\" (UID: \"7eef202a-38a8-4944-a4ef-7ffe4bc0ae1e\") " pod="openshift-ingress/e2e-api-b-593845-openshift-default-54dcc87546-pvswg"
Sep 12 00:44:34.068108 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:44:34.068086    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4j26h\" (UniqueName: \"kubernetes.io/projected/7eef202a-38a8-4944-a4ef-7ffe4bc0ae1e-kube-api-access-4j26h\") pod \"e2e-api-b-593845-openshift-default-54dcc87546-pvswg\" (UID: \"7eef202a-38a8-4944-a4ef-7ffe4bc0ae1e\") " pod="openshift-ingress/e2e-api-b-593845-openshift-default-54dcc87546-pvswg"
Sep 12 00:44:34.068504 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:44:34.068465    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7eef202a-38a8-4944-a4ef-7ffe4bc0ae1e-istio-token\") pod \"e2e-api-b-593845-openshift-default-54dcc87546-pvswg\" (UID: \"7eef202a-38a8-4944-a4ef-7ffe4bc0ae1e\") " pod="openshift-ingress/e2e-api-b-593845-openshift-default-54dcc87546-pvswg"
Sep 12 00:44:34.185323 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:44:34.185277    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-593845-openshift-default-54dcc87546-pvswg"
Sep 12 00:44:34.313155 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:44:34.313113    2571 generic.go:356] "Generic (PLEG): container finished" podID="5dd9c16e-644a-421e-813c-4a519128d91c" containerID="6efa982753ed4f291c0e76c75e4a32c893cd41a05326a82608653c09409c206c" exitCode=0
Sep 12 00:44:34.313331 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:44:34.313200    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w4-139288-dsk8h" event={"ID":"5dd9c16e-644a-421e-813c-4a519128d91c","Type":"ContainerDied","Data":"6efa982753ed4f291c0e76c75e4a32c893cd41a05326a82608653c09409c206c"}
Sep 12 00:44:34.472531 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:44:34.472503    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-b-593845-openshift-default-54dcc87546-pvswg"]
Sep 12 00:44:34.478669 ip-10-0-141-128 kubenswrapper[2571]: W0912 00:44:34.478642    2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7eef202a_38a8_4944_a4ef_7ffe4bc0ae1e.slice/crio-79a6ab8485f99c7695af5d7853e29b40c75e5fc3d1bd488364b57c14eecf4263 WatchSource:0}: Error finding container 79a6ab8485f99c7695af5d7853e29b40c75e5fc3d1bd488364b57c14eecf4263: Status 404 returned error can't find the container with id 79a6ab8485f99c7695af5d7853e29b40c75e5fc3d1bd488364b57c14eecf4263
Sep 12 00:44:34.481332 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:44:34.481300    2571 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236196Ki","pods":"250"}
Sep 12 00:44:34.481439 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:44:34.481391    2571 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236196Ki","pods":"250"}
Sep 12 00:44:34.481504 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:44:34.481442    2571 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236196Ki","pods":"250"}
Sep 12 00:44:35.320480 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:44:35.320449    2571 generic.go:356] "Generic (PLEG): container finished" podID="537eb92d-71ea-4651-a204-fee3a3728ace" containerID="008bb091ad9633a923f97e1dfb3ba4fd19090eb62151240048687efa42d7a203" exitCode=0
Sep 12 00:44:35.320480 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:44:35.320476    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-829959-openshift-default-697f5547d4-mdscp" event={"ID":"537eb92d-71ea-4651-a204-fee3a3728ace","Type":"ContainerDied","Data":"008bb091ad9633a923f97e1dfb3ba4fd19090eb62151240048687efa42d7a203"}
Sep 12 00:44:35.321029 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:44:35.321012    2571 scope.go:117] "RemoveContainer" containerID="008bb091ad9633a923f97e1dfb3ba4fd19090eb62151240048687efa42d7a203"
Sep 12 00:44:35.322386 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:44:35.322351    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-593845-openshift-default-54dcc87546-pvswg" event={"ID":"7eef202a-38a8-4944-a4ef-7ffe4bc0ae1e","Type":"ContainerStarted","Data":"be27b8eb97202bb56703d33afaf4a4fe22a873f638c567e18b6964ef87d25601"}
Sep 12 00:44:35.322386 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:44:35.322362    2571 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236196Ki","pods":"250"}
Sep 12 00:44:35.322506 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:44:35.322406    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-593845-openshift-default-54dcc87546-pvswg" event={"ID":"7eef202a-38a8-4944-a4ef-7ffe4bc0ae1e","Type":"ContainerStarted","Data":"79a6ab8485f99c7695af5d7853e29b40c75e5fc3d1bd488364b57c14eecf4263"}
Sep 12 00:44:35.322506 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:44:35.322446    2571 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236196Ki","pods":"250"}
Sep 12 00:44:35.322506 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:44:35.322494    2571 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236196Ki","pods":"250"}
Sep 12 00:44:35.358072 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:44:35.357843    2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-api-b-593845-openshift-default-54dcc87546-pvswg" podStartSLOduration=2.357820636 podStartE2EDuration="2.357820636s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-12 00:44:33 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-12 00:44:35.357517748 +0000 UTC m=+1686.629158258" watchObservedRunningTime="2026-09-12 00:44:35.357820636 +0000 UTC m=+1686.629461148"
Sep 12 00:44:35.528556 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:44:35.528532    2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w4-139288-dsk8h"
Sep 12 00:44:35.690537 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:44:35.690501    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/5dd9c16e-644a-421e-813c-4a519128d91c-maas-api-service-ca\") pod \"5dd9c16e-644a-421e-813c-4a519128d91c\" (UID: \"5dd9c16e-644a-421e-813c-4a519128d91c\") "
Sep 12 00:44:35.690928 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:44:35.690881    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5dd9c16e-644a-421e-813c-4a519128d91c-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "5dd9c16e-644a-421e-813c-4a519128d91c" (UID: "5dd9c16e-644a-421e-813c-4a519128d91c"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 12 00:44:35.791620 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:44:35.791550    2571 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/5dd9c16e-644a-421e-813c-4a519128d91c-maas-api-service-ca\") on node \"ip-10-0-141-128.ec2.internal\" DevicePath \"\""
Sep 12 00:44:36.187128 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:44:36.187076    2571 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-b-593845-openshift-default-54dcc87546-pvswg"
Sep 12 00:44:36.195032 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:44:36.195005    2571 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-b-593845-openshift-default-54dcc87546-pvswg"
Sep 12 00:44:36.328959 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:44:36.328935    2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w4-139288-dsk8h"
Sep 12 00:44:36.328959 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:44:36.328952    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w4-139288-dsk8h" event={"ID":"5dd9c16e-644a-421e-813c-4a519128d91c","Type":"ContainerDied","Data":"585442e0fae553dc02e07bc4f5287b30436c511ead207d72e5e7b6cd7d28a307"}
Sep 12 00:44:36.329402 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:44:36.328998    2571 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="585442e0fae553dc02e07bc4f5287b30436c511ead207d72e5e7b6cd7d28a307"
Sep 12 00:44:36.331080 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:44:36.331052    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-829959-openshift-default-697f5547d4-mdscp" event={"ID":"537eb92d-71ea-4651-a204-fee3a3728ace","Type":"ContainerStarted","Data":"71f953a33cba56abc71d09a499491e906cefaf82a4a72b4f58056e2e06684cb4"}
Sep 12 00:44:36.331317 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:44:36.331303    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-b-593845-openshift-default-54dcc87546-pvswg"
Sep 12 00:44:36.332278 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:44:36.332262    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-b-593845-openshift-default-54dcc87546-pvswg"
Sep 12 00:44:36.348803 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:44:36.348560    2571 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-a-829959-openshift-default-697f5547d4-mdscp"
Sep 12 00:44:36.349856 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:44:36.349830    2571 patch_prober.go:28] interesting pod/e2e-api-a-829959-openshift-default-697f5547d4-mdscp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.65:15021/healthz/ready\": dial tcp 10.132.0.65:15021: connect: connection refused" start-of-body=
Sep 12 00:44:36.349959 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:44:36.349886    2571 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-829959-openshift-default-697f5547d4-mdscp" podUID="537eb92d-71ea-4651-a204-fee3a3728ace" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.65:15021/healthz/ready\": dial tcp 10.132.0.65:15021: connect: connection refused"
Sep 12 00:44:37.145220 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:44:37.145187    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-a-829959-openshift-default-697f5547d4-mdscp"
Sep 12 00:44:37.349333 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:44:37.349299    2571 patch_prober.go:28] interesting pod/e2e-api-a-829959-openshift-default-697f5547d4-mdscp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.65:15021/healthz/ready\": dial tcp 10.132.0.65:15021: connect: connection refused" start-of-body=
Sep 12 00:44:37.349740 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:44:37.349349    2571 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-829959-openshift-default-697f5547d4-mdscp" podUID="537eb92d-71ea-4651-a204-fee3a3728ace" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.65:15021/healthz/ready\": dial tcp 10.132.0.65:15021: connect: connection refused"
Sep 12 00:44:38.349448 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:44:38.349420    2571 patch_prober.go:28] interesting pod/e2e-api-a-829959-openshift-default-697f5547d4-mdscp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.65:15021/healthz/ready\": dial tcp 10.132.0.65:15021: connect: connection refused" start-of-body=
Sep 12 00:44:38.349826 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:44:38.349465    2571 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-829959-openshift-default-697f5547d4-mdscp" podUID="537eb92d-71ea-4651-a204-fee3a3728ace" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.65:15021/healthz/ready\": dial tcp 10.132.0.65:15021: connect: connection refused"
Sep 12 00:44:39.331362 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:44:39.330530    2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-593845-6c44884f86-5wfbd"]
Sep 12 00:44:39.331362 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:44:39.331056    2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5dd9c16e-644a-421e-813c-4a519128d91c" containerName="revoke-keys"
Sep 12 00:44:39.331362 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:44:39.331072    2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="5dd9c16e-644a-421e-813c-4a519128d91c" containerName="revoke-keys"
Sep 12 00:44:39.331362 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:44:39.331157    2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="5dd9c16e-644a-421e-813c-4a519128d91c" containerName="revoke-keys"
Sep 12 00:44:39.337041 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:44:39.337019    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-593845-6c44884f86-5wfbd"
Sep 12 00:44:39.339485 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:44:39.339458    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-api-b-593845-serving-cert\""
Sep 12 00:44:39.342784 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:44:39.342759    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-593845-6c44884f86-5wfbd"]
Sep 12 00:44:39.348872 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:44:39.348834    2571 patch_prober.go:28] interesting pod/e2e-api-a-829959-openshift-default-697f5547d4-mdscp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.65:15021/healthz/ready\": dial tcp 10.132.0.65:15021: connect: connection refused" start-of-body=
Sep 12 00:44:39.349080 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:44:39.349045    2571 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-829959-openshift-default-697f5547d4-mdscp" podUID="537eb92d-71ea-4651-a204-fee3a3728ace" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.65:15021/healthz/ready\": dial tcp 10.132.0.65:15021: connect: connection refused"
Sep 12 00:44:39.437580 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:44:39.437538    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e8528d32-8c40-4464-82d9-d58854afb3ac-openshift-service-ca\") pod \"maas-api-e2e-api-b-593845-6c44884f86-5wfbd\" (UID: \"e8528d32-8c40-4464-82d9-d58854afb3ac\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-593845-6c44884f86-5wfbd"
Sep 12 00:44:39.437580 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:44:39.437597    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dqrbg\" (UniqueName: \"kubernetes.io/projected/e8528d32-8c40-4464-82d9-d58854afb3ac-kube-api-access-dqrbg\") pod \"maas-api-e2e-api-b-593845-6c44884f86-5wfbd\" (UID: \"e8528d32-8c40-4464-82d9-d58854afb3ac\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-593845-6c44884f86-5wfbd"
Sep 12 00:44:39.438025 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:44:39.437783    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e8528d32-8c40-4464-82d9-d58854afb3ac-maas-api-tls\") pod \"maas-api-e2e-api-b-593845-6c44884f86-5wfbd\" (UID: \"e8528d32-8c40-4464-82d9-d58854afb3ac\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-593845-6c44884f86-5wfbd"
Sep 12 00:44:39.438025 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:44:39.437844    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e8528d32-8c40-4464-82d9-d58854afb3ac-metrics-certs\") pod \"maas-api-e2e-api-b-593845-6c44884f86-5wfbd\" (UID: \"e8528d32-8c40-4464-82d9-d58854afb3ac\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-593845-6c44884f86-5wfbd"
Sep 12 00:44:39.539265 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:44:39.539222    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e8528d32-8c40-4464-82d9-d58854afb3ac-maas-api-tls\") pod \"maas-api-e2e-api-b-593845-6c44884f86-5wfbd\" (UID: \"e8528d32-8c40-4464-82d9-d58854afb3ac\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-593845-6c44884f86-5wfbd"
Sep 12 00:44:39.539458 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:44:39.539367    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e8528d32-8c40-4464-82d9-d58854afb3ac-metrics-certs\") pod \"maas-api-e2e-api-b-593845-6c44884f86-5wfbd\" (UID: \"e8528d32-8c40-4464-82d9-d58854afb3ac\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-593845-6c44884f86-5wfbd"
Sep 12 00:44:39.539458 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:44:39.539437    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e8528d32-8c40-4464-82d9-d58854afb3ac-openshift-service-ca\") pod \"maas-api-e2e-api-b-593845-6c44884f86-5wfbd\" (UID: \"e8528d32-8c40-4464-82d9-d58854afb3ac\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-593845-6c44884f86-5wfbd"
Sep 12 00:44:39.539604 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:44:39.539465    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dqrbg\" (UniqueName: \"kubernetes.io/projected/e8528d32-8c40-4464-82d9-d58854afb3ac-kube-api-access-dqrbg\") pod \"maas-api-e2e-api-b-593845-6c44884f86-5wfbd\" (UID: \"e8528d32-8c40-4464-82d9-d58854afb3ac\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-593845-6c44884f86-5wfbd"
Sep 12 00:44:39.540623 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:44:39.540567    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e8528d32-8c40-4464-82d9-d58854afb3ac-openshift-service-ca\") pod \"maas-api-e2e-api-b-593845-6c44884f86-5wfbd\" (UID: \"e8528d32-8c40-4464-82d9-d58854afb3ac\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-593845-6c44884f86-5wfbd"
Sep 12 00:44:39.544301 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:44:39.544280    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e8528d32-8c40-4464-82d9-d58854afb3ac-maas-api-tls\") pod \"maas-api-e2e-api-b-593845-6c44884f86-5wfbd\" (UID: \"e8528d32-8c40-4464-82d9-d58854afb3ac\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-593845-6c44884f86-5wfbd"
Sep 12 00:44:39.545532 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:44:39.545506    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e8528d32-8c40-4464-82d9-d58854afb3ac-metrics-certs\") pod \"maas-api-e2e-api-b-593845-6c44884f86-5wfbd\" (UID: \"e8528d32-8c40-4464-82d9-d58854afb3ac\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-593845-6c44884f86-5wfbd"
Sep 12 00:44:39.547862 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:44:39.547837    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dqrbg\" (UniqueName: \"kubernetes.io/projected/e8528d32-8c40-4464-82d9-d58854afb3ac-kube-api-access-dqrbg\") pod \"maas-api-e2e-api-b-593845-6c44884f86-5wfbd\" (UID: \"e8528d32-8c40-4464-82d9-d58854afb3ac\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-593845-6c44884f86-5wfbd"
Sep 12 00:44:39.654357 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:44:39.654265    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-593845-6c44884f86-5wfbd"
Sep 12 00:44:39.953152 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:44:39.953126    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-593845-6c44884f86-5wfbd"]
Sep 12 00:44:39.955195 ip-10-0-141-128 kubenswrapper[2571]: W0912 00:44:39.955150    2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode8528d32_8c40_4464_82d9_d58854afb3ac.slice/crio-c740a006a6280b58dd5a85d954e104bdb47b9e6be1d6da7f5338b947c18aae31 WatchSource:0}: Error finding container c740a006a6280b58dd5a85d954e104bdb47b9e6be1d6da7f5338b947c18aae31: Status 404 returned error can't find the container with id c740a006a6280b58dd5a85d954e104bdb47b9e6be1d6da7f5338b947c18aae31
Sep 12 00:44:40.349437 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:44:40.349355    2571 patch_prober.go:28] interesting pod/e2e-api-a-829959-openshift-default-697f5547d4-mdscp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.65:15021/healthz/ready\": dial tcp 10.132.0.65:15021: connect: connection refused" start-of-body=
Sep 12 00:44:40.349437 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:44:40.349420    2571 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-829959-openshift-default-697f5547d4-mdscp" podUID="537eb92d-71ea-4651-a204-fee3a3728ace" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.65:15021/healthz/ready\": dial tcp 10.132.0.65:15021: connect: connection refused"
Sep 12 00:44:40.364668 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:44:40.364630    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-593845-6c44884f86-5wfbd" event={"ID":"e8528d32-8c40-4464-82d9-d58854afb3ac","Type":"ContainerStarted","Data":"c740a006a6280b58dd5a85d954e104bdb47b9e6be1d6da7f5338b947c18aae31"}
Sep 12 00:44:41.348793 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:44:41.348755    2571 patch_prober.go:28] interesting pod/e2e-api-a-829959-openshift-default-697f5547d4-mdscp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.65:15021/healthz/ready\": dial tcp 10.132.0.65:15021: connect: connection refused" start-of-body=
Sep 12 00:44:41.349142 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:44:41.348827    2571 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-829959-openshift-default-697f5547d4-mdscp" podUID="537eb92d-71ea-4651-a204-fee3a3728ace" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.65:15021/healthz/ready\": dial tcp 10.132.0.65:15021: connect: connection refused"
Sep 12 00:44:41.374744 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:44:41.374709    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-593845-6c44884f86-5wfbd" event={"ID":"e8528d32-8c40-4464-82d9-d58854afb3ac","Type":"ContainerStarted","Data":"5fa4c3a9f4bbd8e7fe520da8ea8c46a8ae168651170a520ee28e3ac698d6fee1"}
Sep 12 00:44:41.374895 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:44:41.374826    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-593845-6c44884f86-5wfbd"
Sep 12 00:44:41.393612 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:44:41.393518    2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-593845-6c44884f86-5wfbd" podStartSLOduration=1.907407047 podStartE2EDuration="2.393494432s" totalImagesPullingTime="486.087385ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-12 00:44:39 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-12 00:44:41.39010303 +0000 UTC m=+1692.661743537" watchObservedRunningTime="2026-09-12 00:44:41.393494432 +0000 UTC m=+1692.665134941"
Sep 12 00:44:42.353103 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:44:42.353073    2571 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-a-829959-openshift-default-697f5547d4-mdscp"
Sep 12 00:44:42.354285 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:44:42.354267    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-a-829959-openshift-default-697f5547d4-mdscp"
Sep 12 00:44:47.385898 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:44:47.385864    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-593845-6c44884f86-5wfbd"
Sep 12 00:44:49.161196 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:44:49.161160    2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w4-139288-dsk8h"]
Sep 12 00:44:49.186610 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:44:49.183909    2571 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w4-139288-dsk8h"]
Sep 12 00:44:51.141770 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:44:51.141738    2571 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5dd9c16e-644a-421e-813c-4a519128d91c" path="/var/lib/kubelet/pods/5dd9c16e-644a-421e-813c-4a519128d91c/volumes"
Sep 12 00:44:55.942137 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:44:55.942100    2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w4-139288-openshift-default-8557bc4874-7zvml"]
Sep 12 00:44:55.942568 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:44:55.942361    2571 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w4-139288-openshift-default-8557bc4874-7zvml" podUID="d7e353f5-8ba3-41a4-bd94-5e2c0cf9fa46" containerName="istio-proxy" containerID="cri-o://bc83423552782511e280547bfcd44127e59b7e2bee34bac0a69fecbbc0c621ea" gracePeriod=30
Sep 12 00:44:58.478828 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:44:58.478792    2571 generic.go:356] "Generic (PLEG): container finished" podID="7eef202a-38a8-4944-a4ef-7ffe4bc0ae1e" containerID="be27b8eb97202bb56703d33afaf4a4fe22a873f638c567e18b6964ef87d25601" exitCode=0
Sep 12 00:44:58.479252 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:44:58.478856    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-593845-openshift-default-54dcc87546-pvswg" event={"ID":"7eef202a-38a8-4944-a4ef-7ffe4bc0ae1e","Type":"ContainerDied","Data":"be27b8eb97202bb56703d33afaf4a4fe22a873f638c567e18b6964ef87d25601"}
Sep 12 00:44:58.479297 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:44:58.479275    2571 scope.go:117] "RemoveContainer" containerID="be27b8eb97202bb56703d33afaf4a4fe22a873f638c567e18b6964ef87d25601"
Sep 12 00:44:58.480499 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:44:58.480469    2571 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236196Ki","pods":"250"}
Sep 12 00:44:58.480632 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:44:58.480555    2571 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236196Ki","pods":"250"}
Sep 12 00:44:58.480706 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:44:58.480644    2571 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236196Ki","pods":"250"}
Sep 12 00:44:59.490554 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:44:59.490518    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-593845-openshift-default-54dcc87546-pvswg" event={"ID":"7eef202a-38a8-4944-a4ef-7ffe4bc0ae1e","Type":"ContainerStarted","Data":"86e6ac5215ea06b54161259c3e8c1fcf99f1cd0370b23f12941d46da8b8d9a96"}
Sep 12 00:44:59.836250 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:44:59.836150    2571 patch_prober.go:28] interesting pod/e2e-worker-w4-139288-openshift-default-8557bc4874-7zvml container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Sep 12 00:44:59.836250 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:44:59.836222    2571 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w4-139288-openshift-default-8557bc4874-7zvml" podUID="d7e353f5-8ba3-41a4-bd94-5e2c0cf9fa46" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Sep 12 00:45:00.142656 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:45:00.142540    2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w2-f12ab7-29819565-qbvds"]
Sep 12 00:45:00.147938 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:45:00.147909    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w2-f12ab7-29819565-qbvds"
Sep 12 00:45:00.152272 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:45:00.152071    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-rlnvx\""
Sep 12 00:45:00.152431 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:45:00.152343    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w2-f12ab7-29819565-qbvds"]
Sep 12 00:45:00.174611 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:45:00.163168    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w2-f12ab7-29819565-qbvds"
Sep 12 00:45:00.196905 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:45:00.196879    2571 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-b-593845-openshift-default-54dcc87546-pvswg"
Sep 12 00:45:00.197294 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:45:00.197271    2571 patch_prober.go:28] interesting pod/e2e-api-b-593845-openshift-default-54dcc87546-pvswg container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.67:15021/healthz/ready\": dial tcp 10.132.0.67:15021: connect: connection refused" start-of-body=
Sep 12 00:45:00.197480 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:45:00.197456    2571 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-593845-openshift-default-54dcc87546-pvswg" podUID="7eef202a-38a8-4944-a4ef-7ffe4bc0ae1e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.67:15021/healthz/ready\": dial tcp 10.132.0.67:15021: connect: connection refused"
Sep 12 00:45:00.244555 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:45:00.244520    2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-829959-6fs77"]
Sep 12 00:45:00.251926 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:45:00.251902    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-829959-6fs77"
Sep 12 00:45:00.255272 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:45:00.254869    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-829959-6fs77"]
Sep 12 00:45:00.318067 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:45:00.318028    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/2b5cfa38-4c94-42e0-8684-0cdd2a93c3f5-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-a-829959-6fs77\" (UID: \"2b5cfa38-4c94-42e0-8684-0cdd2a93c3f5\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-829959-6fs77"
Sep 12 00:45:00.418903 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:45:00.418808    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/2b5cfa38-4c94-42e0-8684-0cdd2a93c3f5-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-a-829959-6fs77\" (UID: \"2b5cfa38-4c94-42e0-8684-0cdd2a93c3f5\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-829959-6fs77"
Sep 12 00:45:00.419472 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:45:00.419445    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/2b5cfa38-4c94-42e0-8684-0cdd2a93c3f5-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-a-829959-6fs77\" (UID: \"2b5cfa38-4c94-42e0-8684-0cdd2a93c3f5\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-829959-6fs77"
Sep 12 00:45:00.478609 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:45:00.474800    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w2-f12ab7-29819565-qbvds"]
Sep 12 00:45:00.591178 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:45:00.590853    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-829959-6fs77"
Sep 12 00:45:00.872614 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:45:00.866158    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-829959-6fs77"]
Sep 12 00:45:01.186299 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:45:01.186265    2571 patch_prober.go:28] interesting pod/e2e-api-b-593845-openshift-default-54dcc87546-pvswg container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.67:15021/healthz/ready\": dial tcp 10.132.0.67:15021: connect: connection refused" start-of-body=
Sep 12 00:45:01.190693 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:45:01.190656    2571 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-593845-openshift-default-54dcc87546-pvswg" podUID="7eef202a-38a8-4944-a4ef-7ffe4bc0ae1e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.67:15021/healthz/ready\": dial tcp 10.132.0.67:15021: connect: connection refused"
Sep 12 00:45:01.256303 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:45:01.256276    2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-139288-openshift-default-8557bc4874-7zvml"
Sep 12 00:45:01.337687 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:45:01.337658    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d7e353f5-8ba3-41a4-bd94-5e2c0cf9fa46-credential-socket\") pod \"d7e353f5-8ba3-41a4-bd94-5e2c0cf9fa46\" (UID: \"d7e353f5-8ba3-41a4-bd94-5e2c0cf9fa46\") "
Sep 12 00:45:01.337883 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:45:01.337729    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d7e353f5-8ba3-41a4-bd94-5e2c0cf9fa46-istio-envoy\") pod \"d7e353f5-8ba3-41a4-bd94-5e2c0cf9fa46\" (UID: \"d7e353f5-8ba3-41a4-bd94-5e2c0cf9fa46\") "
Sep 12 00:45:01.337883 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:45:01.337749    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d7e353f5-8ba3-41a4-bd94-5e2c0cf9fa46-istio-token\") pod \"d7e353f5-8ba3-41a4-bd94-5e2c0cf9fa46\" (UID: \"d7e353f5-8ba3-41a4-bd94-5e2c0cf9fa46\") "
Sep 12 00:45:01.337883 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:45:01.337769    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d7e353f5-8ba3-41a4-bd94-5e2c0cf9fa46-istio-data\") pod \"d7e353f5-8ba3-41a4-bd94-5e2c0cf9fa46\" (UID: \"d7e353f5-8ba3-41a4-bd94-5e2c0cf9fa46\") "
Sep 12 00:45:01.337883 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:45:01.337786    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d7e353f5-8ba3-41a4-bd94-5e2c0cf9fa46-workload-certs\") pod \"d7e353f5-8ba3-41a4-bd94-5e2c0cf9fa46\" (UID: \"d7e353f5-8ba3-41a4-bd94-5e2c0cf9fa46\") "
Sep 12 00:45:01.337883 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:45:01.337845    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-gxw5z\" (UniqueName: \"kubernetes.io/projected/d7e353f5-8ba3-41a4-bd94-5e2c0cf9fa46-kube-api-access-gxw5z\") pod \"d7e353f5-8ba3-41a4-bd94-5e2c0cf9fa46\" (UID: \"d7e353f5-8ba3-41a4-bd94-5e2c0cf9fa46\") "
Sep 12 00:45:01.337883 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:45:01.337873    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d7e353f5-8ba3-41a4-bd94-5e2c0cf9fa46-workload-socket\") pod \"d7e353f5-8ba3-41a4-bd94-5e2c0cf9fa46\" (UID: \"d7e353f5-8ba3-41a4-bd94-5e2c0cf9fa46\") "
Sep 12 00:45:01.338229 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:45:01.337930    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d7e353f5-8ba3-41a4-bd94-5e2c0cf9fa46-istiod-ca-cert\") pod \"d7e353f5-8ba3-41a4-bd94-5e2c0cf9fa46\" (UID: \"d7e353f5-8ba3-41a4-bd94-5e2c0cf9fa46\") "
Sep 12 00:45:01.338229 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:45:01.337962    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d7e353f5-8ba3-41a4-bd94-5e2c0cf9fa46-istio-podinfo\") pod \"d7e353f5-8ba3-41a4-bd94-5e2c0cf9fa46\" (UID: \"d7e353f5-8ba3-41a4-bd94-5e2c0cf9fa46\") "
Sep 12 00:45:01.338229 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:45:01.337979    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d7e353f5-8ba3-41a4-bd94-5e2c0cf9fa46-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "d7e353f5-8ba3-41a4-bd94-5e2c0cf9fa46" (UID: "d7e353f5-8ba3-41a4-bd94-5e2c0cf9fa46"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 12 00:45:01.338229 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:45:01.338063    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d7e353f5-8ba3-41a4-bd94-5e2c0cf9fa46-istio-data" (OuterVolumeSpecName: "istio-data") pod "d7e353f5-8ba3-41a4-bd94-5e2c0cf9fa46" (UID: "d7e353f5-8ba3-41a4-bd94-5e2c0cf9fa46"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 12 00:45:01.338229 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:45:01.338170    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d7e353f5-8ba3-41a4-bd94-5e2c0cf9fa46-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "d7e353f5-8ba3-41a4-bd94-5e2c0cf9fa46" (UID: "d7e353f5-8ba3-41a4-bd94-5e2c0cf9fa46"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 12 00:45:01.338503 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:45:01.338283    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d7e353f5-8ba3-41a4-bd94-5e2c0cf9fa46-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "d7e353f5-8ba3-41a4-bd94-5e2c0cf9fa46" (UID: "d7e353f5-8ba3-41a4-bd94-5e2c0cf9fa46"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 12 00:45:01.338503 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:45:01.338371    2571 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d7e353f5-8ba3-41a4-bd94-5e2c0cf9fa46-workload-socket\") on node \"ip-10-0-141-128.ec2.internal\" DevicePath \"\""
Sep 12 00:45:01.338503 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:45:01.338388    2571 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d7e353f5-8ba3-41a4-bd94-5e2c0cf9fa46-credential-socket\") on node \"ip-10-0-141-128.ec2.internal\" DevicePath \"\""
Sep 12 00:45:01.338503 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:45:01.338401    2571 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d7e353f5-8ba3-41a4-bd94-5e2c0cf9fa46-istio-data\") on node \"ip-10-0-141-128.ec2.internal\" DevicePath \"\""
Sep 12 00:45:01.338503 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:45:01.338416    2571 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d7e353f5-8ba3-41a4-bd94-5e2c0cf9fa46-workload-certs\") on node \"ip-10-0-141-128.ec2.internal\" DevicePath \"\""
Sep 12 00:45:01.338783 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:45:01.338716    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d7e353f5-8ba3-41a4-bd94-5e2c0cf9fa46-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "d7e353f5-8ba3-41a4-bd94-5e2c0cf9fa46" (UID: "d7e353f5-8ba3-41a4-bd94-5e2c0cf9fa46"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 12 00:45:01.340198 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:45:01.340169    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/d7e353f5-8ba3-41a4-bd94-5e2c0cf9fa46-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "d7e353f5-8ba3-41a4-bd94-5e2c0cf9fa46" (UID: "d7e353f5-8ba3-41a4-bd94-5e2c0cf9fa46"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 12 00:45:01.340325 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:45:01.340304    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d7e353f5-8ba3-41a4-bd94-5e2c0cf9fa46-istio-token" (OuterVolumeSpecName: "istio-token") pod "d7e353f5-8ba3-41a4-bd94-5e2c0cf9fa46" (UID: "d7e353f5-8ba3-41a4-bd94-5e2c0cf9fa46"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 12 00:45:01.340516 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:45:01.340493    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d7e353f5-8ba3-41a4-bd94-5e2c0cf9fa46-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "d7e353f5-8ba3-41a4-bd94-5e2c0cf9fa46" (UID: "d7e353f5-8ba3-41a4-bd94-5e2c0cf9fa46"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 12 00:45:01.340645 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:45:01.340622    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d7e353f5-8ba3-41a4-bd94-5e2c0cf9fa46-kube-api-access-gxw5z" (OuterVolumeSpecName: "kube-api-access-gxw5z") pod "d7e353f5-8ba3-41a4-bd94-5e2c0cf9fa46" (UID: "d7e353f5-8ba3-41a4-bd94-5e2c0cf9fa46"). InnerVolumeSpecName "kube-api-access-gxw5z". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 12 00:45:01.439419 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:45:01.439383    2571 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d7e353f5-8ba3-41a4-bd94-5e2c0cf9fa46-istio-envoy\") on node \"ip-10-0-141-128.ec2.internal\" DevicePath \"\""
Sep 12 00:45:01.439419 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:45:01.439413    2571 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d7e353f5-8ba3-41a4-bd94-5e2c0cf9fa46-istio-token\") on node \"ip-10-0-141-128.ec2.internal\" DevicePath \"\""
Sep 12 00:45:01.439419 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:45:01.439424    2571 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-gxw5z\" (UniqueName: \"kubernetes.io/projected/d7e353f5-8ba3-41a4-bd94-5e2c0cf9fa46-kube-api-access-gxw5z\") on node \"ip-10-0-141-128.ec2.internal\" DevicePath \"\""
Sep 12 00:45:01.439686 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:45:01.439434    2571 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d7e353f5-8ba3-41a4-bd94-5e2c0cf9fa46-istiod-ca-cert\") on node \"ip-10-0-141-128.ec2.internal\" DevicePath \"\""
Sep 12 00:45:01.439686 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:45:01.439444    2571 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d7e353f5-8ba3-41a4-bd94-5e2c0cf9fa46-istio-podinfo\") on node \"ip-10-0-141-128.ec2.internal\" DevicePath \"\""
Sep 12 00:45:01.512894 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:45:01.512788    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w2-f12ab7-29819565-qbvds" event={"ID":"78d2f831-fdc9-4d7f-b363-2c0a3a185b8f","Type":"ContainerStarted","Data":"286db8b787f83a22ec0e798120e1918631badda3aea2cf20ff9f9a2ca3ce5b29"}
Sep 12 00:45:01.512894 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:45:01.512855    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w2-f12ab7-29819565-qbvds" event={"ID":"78d2f831-fdc9-4d7f-b363-2c0a3a185b8f","Type":"ContainerStarted","Data":"8a17fad81c3b29915c8a646d8432108bfb84a71e4ca5e067be335bef26788b6f"}
Sep 12 00:45:01.514451 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:45:01.514422    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-829959-6fs77" event={"ID":"2b5cfa38-4c94-42e0-8684-0cdd2a93c3f5","Type":"ContainerStarted","Data":"82b6f88a306f6036c0f6fb0dd23d1068c191d393849191f3d2531946108d69d9"}
Sep 12 00:45:01.514612 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:45:01.514464    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-829959-6fs77" event={"ID":"2b5cfa38-4c94-42e0-8684-0cdd2a93c3f5","Type":"ContainerStarted","Data":"8cb6663e8d696b55d7fe1bf95f1561d783c617367b04af301d191e96bc57d5bd"}
Sep 12 00:45:01.516146 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:45:01.516123    2571 generic.go:356] "Generic (PLEG): container finished" podID="d7e353f5-8ba3-41a4-bd94-5e2c0cf9fa46" containerID="bc83423552782511e280547bfcd44127e59b7e2bee34bac0a69fecbbc0c621ea" exitCode=0
Sep 12 00:45:01.516263 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:45:01.516166    2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-139288-openshift-default-8557bc4874-7zvml"
Sep 12 00:45:01.516263 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:45:01.516199    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-139288-openshift-default-8557bc4874-7zvml" event={"ID":"d7e353f5-8ba3-41a4-bd94-5e2c0cf9fa46","Type":"ContainerDied","Data":"bc83423552782511e280547bfcd44127e59b7e2bee34bac0a69fecbbc0c621ea"}
Sep 12 00:45:01.516263 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:45:01.516246    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-139288-openshift-default-8557bc4874-7zvml" event={"ID":"d7e353f5-8ba3-41a4-bd94-5e2c0cf9fa46","Type":"ContainerDied","Data":"f4a2a9b6703dea3eda8f4219e98a57264c4f13d1cae654f2a6c65d8f8005ae2a"}
Sep 12 00:45:01.516391 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:45:01.516268    2571 scope.go:117] "RemoveContainer" containerID="bc83423552782511e280547bfcd44127e59b7e2bee34bac0a69fecbbc0c621ea"
Sep 12 00:45:01.527070 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:45:01.527000    2571 scope.go:117] "RemoveContainer" containerID="0016a348a7c14fe82d8c59241a135afe29e55cc506d42caec078fd91aacce125"
Sep 12 00:45:01.527993 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:45:01.527940    2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w2-f12ab7-29819565-qbvds" podStartSLOduration=1.5279271140000001 podStartE2EDuration="1.527927114s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-12 00:45:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-12 00:45:01.526621479 +0000 UTC m=+1712.798261985" watchObservedRunningTime="2026-09-12 00:45:01.527927114 +0000 UTC m=+1712.799567623"
Sep 12 00:45:01.536422 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:45:01.536402    2571 scope.go:117] "RemoveContainer" containerID="bc83423552782511e280547bfcd44127e59b7e2bee34bac0a69fecbbc0c621ea"
Sep 12 00:45:01.536722 ip-10-0-141-128 kubenswrapper[2571]: E0912 00:45:01.536700    2571 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"bc83423552782511e280547bfcd44127e59b7e2bee34bac0a69fecbbc0c621ea\": container with ID starting with bc83423552782511e280547bfcd44127e59b7e2bee34bac0a69fecbbc0c621ea not found: ID does not exist" containerID="bc83423552782511e280547bfcd44127e59b7e2bee34bac0a69fecbbc0c621ea"
Sep 12 00:45:01.536806 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:45:01.536734    2571 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"bc83423552782511e280547bfcd44127e59b7e2bee34bac0a69fecbbc0c621ea"} err="failed to get container status \"bc83423552782511e280547bfcd44127e59b7e2bee34bac0a69fecbbc0c621ea\": rpc error: code = NotFound desc = could not find container \"bc83423552782511e280547bfcd44127e59b7e2bee34bac0a69fecbbc0c621ea\": container with ID starting with bc83423552782511e280547bfcd44127e59b7e2bee34bac0a69fecbbc0c621ea not found: ID does not exist"
Sep 12 00:45:01.536806 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:45:01.536759    2571 scope.go:117] "RemoveContainer" containerID="0016a348a7c14fe82d8c59241a135afe29e55cc506d42caec078fd91aacce125"
Sep 12 00:45:01.537388 ip-10-0-141-128 kubenswrapper[2571]: E0912 00:45:01.537157    2571 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0016a348a7c14fe82d8c59241a135afe29e55cc506d42caec078fd91aacce125\": container with ID starting with 0016a348a7c14fe82d8c59241a135afe29e55cc506d42caec078fd91aacce125 not found: ID does not exist" containerID="0016a348a7c14fe82d8c59241a135afe29e55cc506d42caec078fd91aacce125"
Sep 12 00:45:01.537388 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:45:01.537191    2571 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0016a348a7c14fe82d8c59241a135afe29e55cc506d42caec078fd91aacce125"} err="failed to get container status \"0016a348a7c14fe82d8c59241a135afe29e55cc506d42caec078fd91aacce125\": rpc error: code = NotFound desc = could not find container \"0016a348a7c14fe82d8c59241a135afe29e55cc506d42caec078fd91aacce125\": container with ID starting with 0016a348a7c14fe82d8c59241a135afe29e55cc506d42caec078fd91aacce125 not found: ID does not exist"
Sep 12 00:45:01.540430 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:45:01.540408    2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w4-139288-openshift-default-8557bc4874-7zvml"]
Sep 12 00:45:01.542625 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:45:01.542607    2571 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w4-139288-openshift-default-8557bc4874-7zvml"]
Sep 12 00:45:01.555877 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:45:01.555830    2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-829959-6fs77" podStartSLOduration=1.555818447 podStartE2EDuration="1.555818447s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-12 00:45:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-12 00:45:01.554084145 +0000 UTC m=+1712.825724654" watchObservedRunningTime="2026-09-12 00:45:01.555818447 +0000 UTC m=+1712.827458983"
Sep 12 00:45:02.185925 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:45:02.185884    2571 patch_prober.go:28] interesting pod/e2e-api-b-593845-openshift-default-54dcc87546-pvswg container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.67:15021/healthz/ready\": dial tcp 10.132.0.67:15021: connect: connection refused" start-of-body=
Sep 12 00:45:02.186422 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:45:02.185967    2571 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-593845-openshift-default-54dcc87546-pvswg" podUID="7eef202a-38a8-4944-a4ef-7ffe4bc0ae1e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.67:15021/healthz/ready\": dial tcp 10.132.0.67:15021: connect: connection refused"
Sep 12 00:45:02.536328 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:45:02.536237    2571 generic.go:356] "Generic (PLEG): container finished" podID="78d2f831-fdc9-4d7f-b363-2c0a3a185b8f" containerID="286db8b787f83a22ec0e798120e1918631badda3aea2cf20ff9f9a2ca3ce5b29" exitCode=0
Sep 12 00:45:02.536706 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:45:02.536676    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w2-f12ab7-29819565-qbvds" event={"ID":"78d2f831-fdc9-4d7f-b363-2c0a3a185b8f","Type":"ContainerDied","Data":"286db8b787f83a22ec0e798120e1918631badda3aea2cf20ff9f9a2ca3ce5b29"}
Sep 12 00:45:02.541535 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:45:02.541509    2571 generic.go:356] "Generic (PLEG): container finished" podID="2b5cfa38-4c94-42e0-8684-0cdd2a93c3f5" containerID="82b6f88a306f6036c0f6fb0dd23d1068c191d393849191f3d2531946108d69d9" exitCode=0
Sep 12 00:45:02.541684 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:45:02.541613    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-829959-6fs77" event={"ID":"2b5cfa38-4c94-42e0-8684-0cdd2a93c3f5","Type":"ContainerDied","Data":"82b6f88a306f6036c0f6fb0dd23d1068c191d393849191f3d2531946108d69d9"}
Sep 12 00:45:03.143060 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:45:03.142750    2571 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d7e353f5-8ba3-41a4-bd94-5e2c0cf9fa46" path="/var/lib/kubelet/pods/d7e353f5-8ba3-41a4-bd94-5e2c0cf9fa46/volumes"
Sep 12 00:45:03.186092 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:45:03.186053    2571 patch_prober.go:28] interesting pod/e2e-api-b-593845-openshift-default-54dcc87546-pvswg container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.67:15021/healthz/ready\": dial tcp 10.132.0.67:15021: connect: connection refused" start-of-body=
Sep 12 00:45:03.186688 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:45:03.186656    2571 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-593845-openshift-default-54dcc87546-pvswg" podUID="7eef202a-38a8-4944-a4ef-7ffe4bc0ae1e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.67:15021/healthz/ready\": dial tcp 10.132.0.67:15021: connect: connection refused"
Sep 12 00:45:03.726575 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:45:03.726543    2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w2-f12ab7-29819565-qbvds"
Sep 12 00:45:03.799552 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:45:03.799491    2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-829959-6fs77"
Sep 12 00:45:03.880610 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:45:03.878998    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/2b5cfa38-4c94-42e0-8684-0cdd2a93c3f5-maas-api-service-ca\") pod \"2b5cfa38-4c94-42e0-8684-0cdd2a93c3f5\" (UID: \"2b5cfa38-4c94-42e0-8684-0cdd2a93c3f5\") "
Sep 12 00:45:03.886890 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:45:03.886837    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2b5cfa38-4c94-42e0-8684-0cdd2a93c3f5-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "2b5cfa38-4c94-42e0-8684-0cdd2a93c3f5" (UID: "2b5cfa38-4c94-42e0-8684-0cdd2a93c3f5"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 12 00:45:03.980390 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:45:03.980341    2571 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/2b5cfa38-4c94-42e0-8684-0cdd2a93c3f5-maas-api-service-ca\") on node \"ip-10-0-141-128.ec2.internal\" DevicePath \"\""
Sep 12 00:45:04.186519 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:45:04.186477    2571 patch_prober.go:28] interesting pod/e2e-api-b-593845-openshift-default-54dcc87546-pvswg container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.67:15021/healthz/ready\": dial tcp 10.132.0.67:15021: connect: connection refused" start-of-body=
Sep 12 00:45:04.186981 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:45:04.186542    2571 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-593845-openshift-default-54dcc87546-pvswg" podUID="7eef202a-38a8-4944-a4ef-7ffe4bc0ae1e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.67:15021/healthz/ready\": dial tcp 10.132.0.67:15021: connect: connection refused"
Sep 12 00:45:04.557124 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:45:04.557035    2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w2-f12ab7-29819565-qbvds"
Sep 12 00:45:04.557124 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:45:04.557082    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w2-f12ab7-29819565-qbvds" event={"ID":"78d2f831-fdc9-4d7f-b363-2c0a3a185b8f","Type":"ContainerDied","Data":"8a17fad81c3b29915c8a646d8432108bfb84a71e4ca5e067be335bef26788b6f"}
Sep 12 00:45:04.557370 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:45:04.557137    2571 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="8a17fad81c3b29915c8a646d8432108bfb84a71e4ca5e067be335bef26788b6f"
Sep 12 00:45:04.559381 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:45:04.559350    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-829959-6fs77" event={"ID":"2b5cfa38-4c94-42e0-8684-0cdd2a93c3f5","Type":"ContainerDied","Data":"8cb6663e8d696b55d7fe1bf95f1561d783c617367b04af301d191e96bc57d5bd"}
Sep 12 00:45:04.559525 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:45:04.559397    2571 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="8cb6663e8d696b55d7fe1bf95f1561d783c617367b04af301d191e96bc57d5bd"
Sep 12 00:45:04.559525 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:45:04.559465    2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-829959-6fs77"
Sep 12 00:45:05.190539 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:45:05.190503    2571 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-b-593845-openshift-default-54dcc87546-pvswg"
Sep 12 00:45:05.190953 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:45:05.190916    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-b-593845-openshift-default-54dcc87546-pvswg"
Sep 12 00:45:05.191815 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:45:05.191790    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-b-593845-openshift-default-54dcc87546-pvswg"
Sep 12 00:45:16.412399 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:45:16.412366    2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-829959-6fs77"]
Sep 12 00:45:16.414839 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:45:16.414815    2571 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-829959-6fs77"]
Sep 12 00:45:17.141432 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:45:17.141395    2571 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2b5cfa38-4c94-42e0-8684-0cdd2a93c3f5" path="/var/lib/kubelet/pods/2b5cfa38-4c94-42e0-8684-0cdd2a93c3f5/volumes"
Sep 12 00:45:22.903603 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:45:22.903561    2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-a-829959-openshift-default-697f5547d4-mdscp"]
Sep 12 00:45:22.904013 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:45:22.903803    2571 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-api-a-829959-openshift-default-697f5547d4-mdscp" podUID="537eb92d-71ea-4651-a204-fee3a3728ace" containerName="istio-proxy" containerID="cri-o://71f953a33cba56abc71d09a499491e906cefaf82a4a72b4f58056e2e06684cb4" gracePeriod=30
Sep 12 00:45:27.356076 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:45:27.356037    2571 patch_prober.go:28] interesting pod/e2e-api-a-829959-openshift-default-697f5547d4-mdscp container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Sep 12 00:45:27.356464 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:45:27.356100    2571 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-api-a-829959-openshift-default-697f5547d4-mdscp" podUID="537eb92d-71ea-4651-a204-fee3a3728ace" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Sep 12 00:45:28.162955 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:45:28.162932    2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-829959-openshift-default-697f5547d4-mdscp"
Sep 12 00:45:28.218630 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:45:28.218603    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-swspr\" (UniqueName: \"kubernetes.io/projected/537eb92d-71ea-4651-a204-fee3a3728ace-kube-api-access-swspr\") pod \"537eb92d-71ea-4651-a204-fee3a3728ace\" (UID: \"537eb92d-71ea-4651-a204-fee3a3728ace\") "
Sep 12 00:45:28.218822 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:45:28.218665    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/537eb92d-71ea-4651-a204-fee3a3728ace-istio-envoy\") pod \"537eb92d-71ea-4651-a204-fee3a3728ace\" (UID: \"537eb92d-71ea-4651-a204-fee3a3728ace\") "
Sep 12 00:45:28.218822 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:45:28.218683    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/537eb92d-71ea-4651-a204-fee3a3728ace-istio-token\") pod \"537eb92d-71ea-4651-a204-fee3a3728ace\" (UID: \"537eb92d-71ea-4651-a204-fee3a3728ace\") "
Sep 12 00:45:28.218822 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:45:28.218704    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/537eb92d-71ea-4651-a204-fee3a3728ace-istio-data\") pod \"537eb92d-71ea-4651-a204-fee3a3728ace\" (UID: \"537eb92d-71ea-4651-a204-fee3a3728ace\") "
Sep 12 00:45:28.218822 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:45:28.218748    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/537eb92d-71ea-4651-a204-fee3a3728ace-workload-certs\") pod \"537eb92d-71ea-4651-a204-fee3a3728ace\" (UID: \"537eb92d-71ea-4651-a204-fee3a3728ace\") "
Sep 12 00:45:28.218822 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:45:28.218783    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/537eb92d-71ea-4651-a204-fee3a3728ace-istio-podinfo\") pod \"537eb92d-71ea-4651-a204-fee3a3728ace\" (UID: \"537eb92d-71ea-4651-a204-fee3a3728ace\") "
Sep 12 00:45:28.218822 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:45:28.218806    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/537eb92d-71ea-4651-a204-fee3a3728ace-credential-socket\") pod \"537eb92d-71ea-4651-a204-fee3a3728ace\" (UID: \"537eb92d-71ea-4651-a204-fee3a3728ace\") "
Sep 12 00:45:28.219161 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:45:28.218859    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/537eb92d-71ea-4651-a204-fee3a3728ace-workload-socket\") pod \"537eb92d-71ea-4651-a204-fee3a3728ace\" (UID: \"537eb92d-71ea-4651-a204-fee3a3728ace\") "
Sep 12 00:45:28.219161 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:45:28.218893    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/537eb92d-71ea-4651-a204-fee3a3728ace-istiod-ca-cert\") pod \"537eb92d-71ea-4651-a204-fee3a3728ace\" (UID: \"537eb92d-71ea-4651-a204-fee3a3728ace\") "
Sep 12 00:45:28.219161 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:45:28.219070    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/537eb92d-71ea-4651-a204-fee3a3728ace-istio-data" (OuterVolumeSpecName: "istio-data") pod "537eb92d-71ea-4651-a204-fee3a3728ace" (UID: "537eb92d-71ea-4651-a204-fee3a3728ace"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 12 00:45:28.219333 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:45:28.219222    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/537eb92d-71ea-4651-a204-fee3a3728ace-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "537eb92d-71ea-4651-a204-fee3a3728ace" (UID: "537eb92d-71ea-4651-a204-fee3a3728ace"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 12 00:45:28.219333 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:45:28.219255    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/537eb92d-71ea-4651-a204-fee3a3728ace-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "537eb92d-71ea-4651-a204-fee3a3728ace" (UID: "537eb92d-71ea-4651-a204-fee3a3728ace"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 12 00:45:28.219444 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:45:28.219359    2571 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/537eb92d-71ea-4651-a204-fee3a3728ace-istio-data\") on node \"ip-10-0-141-128.ec2.internal\" DevicePath \"\""
Sep 12 00:45:28.219444 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:45:28.219379    2571 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/537eb92d-71ea-4651-a204-fee3a3728ace-credential-socket\") on node \"ip-10-0-141-128.ec2.internal\" DevicePath \"\""
Sep 12 00:45:28.219444 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:45:28.219394    2571 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/537eb92d-71ea-4651-a204-fee3a3728ace-workload-socket\") on node \"ip-10-0-141-128.ec2.internal\" DevicePath \"\""
Sep 12 00:45:28.219706 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:45:28.219680    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/537eb92d-71ea-4651-a204-fee3a3728ace-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "537eb92d-71ea-4651-a204-fee3a3728ace" (UID: "537eb92d-71ea-4651-a204-fee3a3728ace"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 12 00:45:28.219967 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:45:28.219943    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/537eb92d-71ea-4651-a204-fee3a3728ace-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "537eb92d-71ea-4651-a204-fee3a3728ace" (UID: "537eb92d-71ea-4651-a204-fee3a3728ace"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 12 00:45:28.221246 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:45:28.221218    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/537eb92d-71ea-4651-a204-fee3a3728ace-istio-token" (OuterVolumeSpecName: "istio-token") pod "537eb92d-71ea-4651-a204-fee3a3728ace" (UID: "537eb92d-71ea-4651-a204-fee3a3728ace"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 12 00:45:28.221351 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:45:28.221287    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/537eb92d-71ea-4651-a204-fee3a3728ace-kube-api-access-swspr" (OuterVolumeSpecName: "kube-api-access-swspr") pod "537eb92d-71ea-4651-a204-fee3a3728ace" (UID: "537eb92d-71ea-4651-a204-fee3a3728ace"). InnerVolumeSpecName "kube-api-access-swspr". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 12 00:45:28.221482 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:45:28.221468    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/537eb92d-71ea-4651-a204-fee3a3728ace-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "537eb92d-71ea-4651-a204-fee3a3728ace" (UID: "537eb92d-71ea-4651-a204-fee3a3728ace"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 12 00:45:28.221659 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:45:28.221635    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/537eb92d-71ea-4651-a204-fee3a3728ace-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "537eb92d-71ea-4651-a204-fee3a3728ace" (UID: "537eb92d-71ea-4651-a204-fee3a3728ace"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 12 00:45:28.320110 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:45:28.320081    2571 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/537eb92d-71ea-4651-a204-fee3a3728ace-workload-certs\") on node \"ip-10-0-141-128.ec2.internal\" DevicePath \"\""
Sep 12 00:45:28.320110 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:45:28.320106    2571 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/537eb92d-71ea-4651-a204-fee3a3728ace-istio-podinfo\") on node \"ip-10-0-141-128.ec2.internal\" DevicePath \"\""
Sep 12 00:45:28.320110 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:45:28.320115    2571 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/537eb92d-71ea-4651-a204-fee3a3728ace-istiod-ca-cert\") on node \"ip-10-0-141-128.ec2.internal\" DevicePath \"\""
Sep 12 00:45:28.320303 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:45:28.320124    2571 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-swspr\" (UniqueName: \"kubernetes.io/projected/537eb92d-71ea-4651-a204-fee3a3728ace-kube-api-access-swspr\") on node \"ip-10-0-141-128.ec2.internal\" DevicePath \"\""
Sep 12 00:45:28.320303 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:45:28.320133    2571 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/537eb92d-71ea-4651-a204-fee3a3728ace-istio-envoy\") on node \"ip-10-0-141-128.ec2.internal\" DevicePath \"\""
Sep 12 00:45:28.320303 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:45:28.320141    2571 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/537eb92d-71ea-4651-a204-fee3a3728ace-istio-token\") on node \"ip-10-0-141-128.ec2.internal\" DevicePath \"\""
Sep 12 00:45:28.699805 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:45:28.699772    2571 generic.go:356] "Generic (PLEG): container finished" podID="537eb92d-71ea-4651-a204-fee3a3728ace" containerID="71f953a33cba56abc71d09a499491e906cefaf82a4a72b4f58056e2e06684cb4" exitCode=0
Sep 12 00:45:28.700196 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:45:28.699823    2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-829959-openshift-default-697f5547d4-mdscp"
Sep 12 00:45:28.700196 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:45:28.699852    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-829959-openshift-default-697f5547d4-mdscp" event={"ID":"537eb92d-71ea-4651-a204-fee3a3728ace","Type":"ContainerDied","Data":"71f953a33cba56abc71d09a499491e906cefaf82a4a72b4f58056e2e06684cb4"}
Sep 12 00:45:28.700196 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:45:28.699903    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-829959-openshift-default-697f5547d4-mdscp" event={"ID":"537eb92d-71ea-4651-a204-fee3a3728ace","Type":"ContainerDied","Data":"bb4f14ce467b78ab31a76ccd0f3888a199240d45aca9ae4f6e613978dc115e0d"}
Sep 12 00:45:28.700196 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:45:28.699924    2571 scope.go:117] "RemoveContainer" containerID="71f953a33cba56abc71d09a499491e906cefaf82a4a72b4f58056e2e06684cb4"
Sep 12 00:45:28.710037 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:45:28.710020    2571 scope.go:117] "RemoveContainer" containerID="008bb091ad9633a923f97e1dfb3ba4fd19090eb62151240048687efa42d7a203"
Sep 12 00:45:28.718631 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:45:28.718571    2571 scope.go:117] "RemoveContainer" containerID="71f953a33cba56abc71d09a499491e906cefaf82a4a72b4f58056e2e06684cb4"
Sep 12 00:45:28.718994 ip-10-0-141-128 kubenswrapper[2571]: E0912 00:45:28.718963    2571 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"71f953a33cba56abc71d09a499491e906cefaf82a4a72b4f58056e2e06684cb4\": container with ID starting with 71f953a33cba56abc71d09a499491e906cefaf82a4a72b4f58056e2e06684cb4 not found: ID does not exist" containerID="71f953a33cba56abc71d09a499491e906cefaf82a4a72b4f58056e2e06684cb4"
Sep 12 00:45:28.719092 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:45:28.719005    2571 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"71f953a33cba56abc71d09a499491e906cefaf82a4a72b4f58056e2e06684cb4"} err="failed to get container status \"71f953a33cba56abc71d09a499491e906cefaf82a4a72b4f58056e2e06684cb4\": rpc error: code = NotFound desc = could not find container \"71f953a33cba56abc71d09a499491e906cefaf82a4a72b4f58056e2e06684cb4\": container with ID starting with 71f953a33cba56abc71d09a499491e906cefaf82a4a72b4f58056e2e06684cb4 not found: ID does not exist"
Sep 12 00:45:28.719092 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:45:28.719027    2571 scope.go:117] "RemoveContainer" containerID="008bb091ad9633a923f97e1dfb3ba4fd19090eb62151240048687efa42d7a203"
Sep 12 00:45:28.719385 ip-10-0-141-128 kubenswrapper[2571]: E0912 00:45:28.719363    2571 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"008bb091ad9633a923f97e1dfb3ba4fd19090eb62151240048687efa42d7a203\": container with ID starting with 008bb091ad9633a923f97e1dfb3ba4fd19090eb62151240048687efa42d7a203 not found: ID does not exist" containerID="008bb091ad9633a923f97e1dfb3ba4fd19090eb62151240048687efa42d7a203"
Sep 12 00:45:28.719467 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:45:28.719388    2571 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"008bb091ad9633a923f97e1dfb3ba4fd19090eb62151240048687efa42d7a203"} err="failed to get container status \"008bb091ad9633a923f97e1dfb3ba4fd19090eb62151240048687efa42d7a203\": rpc error: code = NotFound desc = could not find container \"008bb091ad9633a923f97e1dfb3ba4fd19090eb62151240048687efa42d7a203\": container with ID starting with 008bb091ad9633a923f97e1dfb3ba4fd19090eb62151240048687efa42d7a203 not found: ID does not exist"
Sep 12 00:45:28.722045 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:45:28.722021    2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-a-829959-openshift-default-697f5547d4-mdscp"]
Sep 12 00:45:28.724677 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:45:28.724657    2571 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-api-a-829959-openshift-default-697f5547d4-mdscp"]
Sep 12 00:45:29.140954 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:45:29.140873    2571 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="537eb92d-71ea-4651-a204-fee3a3728ace" path="/var/lib/kubelet/pods/537eb92d-71ea-4651-a204-fee3a3728ace/volumes"
Sep 12 00:45:38.866281 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:45:38.866245    2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-593845-6c44884f86-5wfbd"]
Sep 12 00:45:38.866868 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:45:38.866524    2571 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-593845-6c44884f86-5wfbd" podUID="e8528d32-8c40-4464-82d9-d58854afb3ac" containerName="maas-api" containerID="cri-o://5fa4c3a9f4bbd8e7fe520da8ea8c46a8ae168651170a520ee28e3ac698d6fee1" gracePeriod=30
Sep 12 00:45:39.137344 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:45:39.137313    2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-593845-6c44884f86-5wfbd"
Sep 12 00:45:39.241402 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:45:39.241361    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-dqrbg\" (UniqueName: \"kubernetes.io/projected/e8528d32-8c40-4464-82d9-d58854afb3ac-kube-api-access-dqrbg\") pod \"e8528d32-8c40-4464-82d9-d58854afb3ac\" (UID: \"e8528d32-8c40-4464-82d9-d58854afb3ac\") "
Sep 12 00:45:39.241402 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:45:39.241401    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e8528d32-8c40-4464-82d9-d58854afb3ac-openshift-service-ca\") pod \"e8528d32-8c40-4464-82d9-d58854afb3ac\" (UID: \"e8528d32-8c40-4464-82d9-d58854afb3ac\") "
Sep 12 00:45:39.241688 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:45:39.241502    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e8528d32-8c40-4464-82d9-d58854afb3ac-maas-api-tls\") pod \"e8528d32-8c40-4464-82d9-d58854afb3ac\" (UID: \"e8528d32-8c40-4464-82d9-d58854afb3ac\") "
Sep 12 00:45:39.241688 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:45:39.241554    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e8528d32-8c40-4464-82d9-d58854afb3ac-metrics-certs\") pod \"e8528d32-8c40-4464-82d9-d58854afb3ac\" (UID: \"e8528d32-8c40-4464-82d9-d58854afb3ac\") "
Sep 12 00:45:39.241844 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:45:39.241813    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e8528d32-8c40-4464-82d9-d58854afb3ac-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "e8528d32-8c40-4464-82d9-d58854afb3ac" (UID: "e8528d32-8c40-4464-82d9-d58854afb3ac"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 12 00:45:39.241943 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:45:39.241922    2571 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e8528d32-8c40-4464-82d9-d58854afb3ac-openshift-service-ca\") on node \"ip-10-0-141-128.ec2.internal\" DevicePath \"\""
Sep 12 00:45:39.243564 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:45:39.243531    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e8528d32-8c40-4464-82d9-d58854afb3ac-kube-api-access-dqrbg" (OuterVolumeSpecName: "kube-api-access-dqrbg") pod "e8528d32-8c40-4464-82d9-d58854afb3ac" (UID: "e8528d32-8c40-4464-82d9-d58854afb3ac"). InnerVolumeSpecName "kube-api-access-dqrbg". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 12 00:45:39.243700 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:45:39.243634    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e8528d32-8c40-4464-82d9-d58854afb3ac-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "e8528d32-8c40-4464-82d9-d58854afb3ac" (UID: "e8528d32-8c40-4464-82d9-d58854afb3ac"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 12 00:45:39.244151 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:45:39.244128    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e8528d32-8c40-4464-82d9-d58854afb3ac-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "e8528d32-8c40-4464-82d9-d58854afb3ac" (UID: "e8528d32-8c40-4464-82d9-d58854afb3ac"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 12 00:45:39.342969 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:45:39.342935    2571 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e8528d32-8c40-4464-82d9-d58854afb3ac-maas-api-tls\") on node \"ip-10-0-141-128.ec2.internal\" DevicePath \"\""
Sep 12 00:45:39.342969 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:45:39.342962    2571 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e8528d32-8c40-4464-82d9-d58854afb3ac-metrics-certs\") on node \"ip-10-0-141-128.ec2.internal\" DevicePath \"\""
Sep 12 00:45:39.342969 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:45:39.342972    2571 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-dqrbg\" (UniqueName: \"kubernetes.io/projected/e8528d32-8c40-4464-82d9-d58854afb3ac-kube-api-access-dqrbg\") on node \"ip-10-0-141-128.ec2.internal\" DevicePath \"\""
Sep 12 00:45:39.760028 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:45:39.760000    2571 generic.go:356] "Generic (PLEG): container finished" podID="e8528d32-8c40-4464-82d9-d58854afb3ac" containerID="5fa4c3a9f4bbd8e7fe520da8ea8c46a8ae168651170a520ee28e3ac698d6fee1" exitCode=0
Sep 12 00:45:39.760228 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:45:39.760056    2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-593845-6c44884f86-5wfbd"
Sep 12 00:45:39.760228 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:45:39.760058    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-593845-6c44884f86-5wfbd" event={"ID":"e8528d32-8c40-4464-82d9-d58854afb3ac","Type":"ContainerDied","Data":"5fa4c3a9f4bbd8e7fe520da8ea8c46a8ae168651170a520ee28e3ac698d6fee1"}
Sep 12 00:45:39.760228 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:45:39.760123    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-593845-6c44884f86-5wfbd" event={"ID":"e8528d32-8c40-4464-82d9-d58854afb3ac","Type":"ContainerDied","Data":"c740a006a6280b58dd5a85d954e104bdb47b9e6be1d6da7f5338b947c18aae31"}
Sep 12 00:45:39.760228 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:45:39.760154    2571 scope.go:117] "RemoveContainer" containerID="5fa4c3a9f4bbd8e7fe520da8ea8c46a8ae168651170a520ee28e3ac698d6fee1"
Sep 12 00:45:39.769953 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:45:39.769939    2571 scope.go:117] "RemoveContainer" containerID="5fa4c3a9f4bbd8e7fe520da8ea8c46a8ae168651170a520ee28e3ac698d6fee1"
Sep 12 00:45:39.770190 ip-10-0-141-128 kubenswrapper[2571]: E0912 00:45:39.770171    2571 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5fa4c3a9f4bbd8e7fe520da8ea8c46a8ae168651170a520ee28e3ac698d6fee1\": container with ID starting with 5fa4c3a9f4bbd8e7fe520da8ea8c46a8ae168651170a520ee28e3ac698d6fee1 not found: ID does not exist" containerID="5fa4c3a9f4bbd8e7fe520da8ea8c46a8ae168651170a520ee28e3ac698d6fee1"
Sep 12 00:45:39.770237 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:45:39.770196    2571 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5fa4c3a9f4bbd8e7fe520da8ea8c46a8ae168651170a520ee28e3ac698d6fee1"} err="failed to get container status \"5fa4c3a9f4bbd8e7fe520da8ea8c46a8ae168651170a520ee28e3ac698d6fee1\": rpc error: code = NotFound desc = could not find container \"5fa4c3a9f4bbd8e7fe520da8ea8c46a8ae168651170a520ee28e3ac698d6fee1\": container with ID starting with 5fa4c3a9f4bbd8e7fe520da8ea8c46a8ae168651170a520ee28e3ac698d6fee1 not found: ID does not exist"
Sep 12 00:45:39.781312 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:45:39.781291    2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-593845-6c44884f86-5wfbd"]
Sep 12 00:45:39.784631 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:45:39.784610    2571 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-593845-6c44884f86-5wfbd"]
Sep 12 00:45:41.141740 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:45:41.141709    2571 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e8528d32-8c40-4464-82d9-d58854afb3ac" path="/var/lib/kubelet/pods/e8528d32-8c40-4464-82d9-d58854afb3ac/volumes"
Sep 12 00:45:45.434599 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:45:45.434545    2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-b-593845-openshift-default-54dcc87546-pvswg"]
Sep 12 00:45:45.435055 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:45:45.434821    2571 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-api-b-593845-openshift-default-54dcc87546-pvswg" podUID="7eef202a-38a8-4944-a4ef-7ffe4bc0ae1e" containerName="istio-proxy" containerID="cri-o://86e6ac5215ea06b54161259c3e8c1fcf99f1cd0370b23f12941d46da8b8d9a96" gracePeriod=30
Sep 12 00:45:50.194251 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:45:50.194207    2571 patch_prober.go:28] interesting pod/e2e-api-b-593845-openshift-default-54dcc87546-pvswg container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Sep 12 00:45:50.194745 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:45:50.194291    2571 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-api-b-593845-openshift-default-54dcc87546-pvswg" podUID="7eef202a-38a8-4944-a4ef-7ffe4bc0ae1e" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Sep 12 00:45:50.698373 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:45:50.698320    2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-593845-openshift-default-54dcc87546-pvswg"
Sep 12 00:45:50.823094 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:45:50.823061    2571 generic.go:356] "Generic (PLEG): container finished" podID="7eef202a-38a8-4944-a4ef-7ffe4bc0ae1e" containerID="86e6ac5215ea06b54161259c3e8c1fcf99f1cd0370b23f12941d46da8b8d9a96" exitCode=0
Sep 12 00:45:50.823273 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:45:50.823102    2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-593845-openshift-default-54dcc87546-pvswg"
Sep 12 00:45:50.823273 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:45:50.823143    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-593845-openshift-default-54dcc87546-pvswg" event={"ID":"7eef202a-38a8-4944-a4ef-7ffe4bc0ae1e","Type":"ContainerDied","Data":"86e6ac5215ea06b54161259c3e8c1fcf99f1cd0370b23f12941d46da8b8d9a96"}
Sep 12 00:45:50.823273 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:45:50.823196    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-593845-openshift-default-54dcc87546-pvswg" event={"ID":"7eef202a-38a8-4944-a4ef-7ffe4bc0ae1e","Type":"ContainerDied","Data":"79a6ab8485f99c7695af5d7853e29b40c75e5fc3d1bd488364b57c14eecf4263"}
Sep 12 00:45:50.823273 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:45:50.823217    2571 scope.go:117] "RemoveContainer" containerID="86e6ac5215ea06b54161259c3e8c1fcf99f1cd0370b23f12941d46da8b8d9a96"
Sep 12 00:45:50.832302 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:45:50.832279    2571 scope.go:117] "RemoveContainer" containerID="be27b8eb97202bb56703d33afaf4a4fe22a873f638c567e18b6964ef87d25601"
Sep 12 00:45:50.840069 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:45:50.840047    2571 scope.go:117] "RemoveContainer" containerID="86e6ac5215ea06b54161259c3e8c1fcf99f1cd0370b23f12941d46da8b8d9a96"
Sep 12 00:45:50.840342 ip-10-0-141-128 kubenswrapper[2571]: E0912 00:45:50.840323    2571 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"86e6ac5215ea06b54161259c3e8c1fcf99f1cd0370b23f12941d46da8b8d9a96\": container with ID starting with 86e6ac5215ea06b54161259c3e8c1fcf99f1cd0370b23f12941d46da8b8d9a96 not found: ID does not exist" containerID="86e6ac5215ea06b54161259c3e8c1fcf99f1cd0370b23f12941d46da8b8d9a96"
Sep 12 00:45:50.840398 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:45:50.840351    2571 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"86e6ac5215ea06b54161259c3e8c1fcf99f1cd0370b23f12941d46da8b8d9a96"} err="failed to get container status \"86e6ac5215ea06b54161259c3e8c1fcf99f1cd0370b23f12941d46da8b8d9a96\": rpc error: code = NotFound desc = could not find container \"86e6ac5215ea06b54161259c3e8c1fcf99f1cd0370b23f12941d46da8b8d9a96\": container with ID starting with 86e6ac5215ea06b54161259c3e8c1fcf99f1cd0370b23f12941d46da8b8d9a96 not found: ID does not exist"
Sep 12 00:45:50.840398 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:45:50.840367    2571 scope.go:117] "RemoveContainer" containerID="be27b8eb97202bb56703d33afaf4a4fe22a873f638c567e18b6964ef87d25601"
Sep 12 00:45:50.840636 ip-10-0-141-128 kubenswrapper[2571]: E0912 00:45:50.840617    2571 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"be27b8eb97202bb56703d33afaf4a4fe22a873f638c567e18b6964ef87d25601\": container with ID starting with be27b8eb97202bb56703d33afaf4a4fe22a873f638c567e18b6964ef87d25601 not found: ID does not exist" containerID="be27b8eb97202bb56703d33afaf4a4fe22a873f638c567e18b6964ef87d25601"
Sep 12 00:45:50.840706 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:45:50.840645    2571 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"be27b8eb97202bb56703d33afaf4a4fe22a873f638c567e18b6964ef87d25601"} err="failed to get container status \"be27b8eb97202bb56703d33afaf4a4fe22a873f638c567e18b6964ef87d25601\": rpc error: code = NotFound desc = could not find container \"be27b8eb97202bb56703d33afaf4a4fe22a873f638c567e18b6964ef87d25601\": container with ID starting with be27b8eb97202bb56703d33afaf4a4fe22a873f638c567e18b6964ef87d25601 not found: ID does not exist"
Sep 12 00:45:50.855050 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:45:50.855031    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7eef202a-38a8-4944-a4ef-7ffe4bc0ae1e-istio-token\") pod \"7eef202a-38a8-4944-a4ef-7ffe4bc0ae1e\" (UID: \"7eef202a-38a8-4944-a4ef-7ffe4bc0ae1e\") "
Sep 12 00:45:50.855137 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:45:50.855077    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7eef202a-38a8-4944-a4ef-7ffe4bc0ae1e-istio-envoy\") pod \"7eef202a-38a8-4944-a4ef-7ffe4bc0ae1e\" (UID: \"7eef202a-38a8-4944-a4ef-7ffe4bc0ae1e\") "
Sep 12 00:45:50.855137 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:45:50.855102    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7eef202a-38a8-4944-a4ef-7ffe4bc0ae1e-workload-certs\") pod \"7eef202a-38a8-4944-a4ef-7ffe4bc0ae1e\" (UID: \"7eef202a-38a8-4944-a4ef-7ffe4bc0ae1e\") "
Sep 12 00:45:50.855251 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:45:50.855214    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7eef202a-38a8-4944-a4ef-7ffe4bc0ae1e-workload-socket\") pod \"7eef202a-38a8-4944-a4ef-7ffe4bc0ae1e\" (UID: \"7eef202a-38a8-4944-a4ef-7ffe4bc0ae1e\") "
Sep 12 00:45:50.855305 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:45:50.855266    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7eef202a-38a8-4944-a4ef-7ffe4bc0ae1e-istiod-ca-cert\") pod \"7eef202a-38a8-4944-a4ef-7ffe4bc0ae1e\" (UID: \"7eef202a-38a8-4944-a4ef-7ffe4bc0ae1e\") "
Sep 12 00:45:50.855360 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:45:50.855310    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7eef202a-38a8-4944-a4ef-7ffe4bc0ae1e-istio-podinfo\") pod \"7eef202a-38a8-4944-a4ef-7ffe4bc0ae1e\" (UID: \"7eef202a-38a8-4944-a4ef-7ffe4bc0ae1e\") "
Sep 12 00:45:50.855360 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:45:50.855340    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-4j26h\" (UniqueName: \"kubernetes.io/projected/7eef202a-38a8-4944-a4ef-7ffe4bc0ae1e-kube-api-access-4j26h\") pod \"7eef202a-38a8-4944-a4ef-7ffe4bc0ae1e\" (UID: \"7eef202a-38a8-4944-a4ef-7ffe4bc0ae1e\") "
Sep 12 00:45:50.855467 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:45:50.855356    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7eef202a-38a8-4944-a4ef-7ffe4bc0ae1e-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "7eef202a-38a8-4944-a4ef-7ffe4bc0ae1e" (UID: "7eef202a-38a8-4944-a4ef-7ffe4bc0ae1e"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 12 00:45:50.855467 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:45:50.855427    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7eef202a-38a8-4944-a4ef-7ffe4bc0ae1e-credential-socket\") pod \"7eef202a-38a8-4944-a4ef-7ffe4bc0ae1e\" (UID: \"7eef202a-38a8-4944-a4ef-7ffe4bc0ae1e\") "
Sep 12 00:45:50.855565 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:45:50.855463    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7eef202a-38a8-4944-a4ef-7ffe4bc0ae1e-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "7eef202a-38a8-4944-a4ef-7ffe4bc0ae1e" (UID: "7eef202a-38a8-4944-a4ef-7ffe4bc0ae1e"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 12 00:45:50.855653 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:45:50.855565    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7eef202a-38a8-4944-a4ef-7ffe4bc0ae1e-istio-data\") pod \"7eef202a-38a8-4944-a4ef-7ffe4bc0ae1e\" (UID: \"7eef202a-38a8-4944-a4ef-7ffe4bc0ae1e\") "
Sep 12 00:45:50.855798 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:45:50.855765    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7eef202a-38a8-4944-a4ef-7ffe4bc0ae1e-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "7eef202a-38a8-4944-a4ef-7ffe4bc0ae1e" (UID: "7eef202a-38a8-4944-a4ef-7ffe4bc0ae1e"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 12 00:45:50.855798 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:45:50.855700    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7eef202a-38a8-4944-a4ef-7ffe4bc0ae1e-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "7eef202a-38a8-4944-a4ef-7ffe4bc0ae1e" (UID: "7eef202a-38a8-4944-a4ef-7ffe4bc0ae1e"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 12 00:45:50.855996 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:45:50.855977    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7eef202a-38a8-4944-a4ef-7ffe4bc0ae1e-istio-data" (OuterVolumeSpecName: "istio-data") pod "7eef202a-38a8-4944-a4ef-7ffe4bc0ae1e" (UID: "7eef202a-38a8-4944-a4ef-7ffe4bc0ae1e"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 12 00:45:50.857246 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:45:50.857164    2571 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7eef202a-38a8-4944-a4ef-7ffe4bc0ae1e-istiod-ca-cert\") on node \"ip-10-0-141-128.ec2.internal\" DevicePath \"\""
Sep 12 00:45:50.857246 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:45:50.857202    2571 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7eef202a-38a8-4944-a4ef-7ffe4bc0ae1e-credential-socket\") on node \"ip-10-0-141-128.ec2.internal\" DevicePath \"\""
Sep 12 00:45:50.857246 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:45:50.857218    2571 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7eef202a-38a8-4944-a4ef-7ffe4bc0ae1e-istio-data\") on node \"ip-10-0-141-128.ec2.internal\" DevicePath \"\""
Sep 12 00:45:50.857246 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:45:50.857235    2571 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7eef202a-38a8-4944-a4ef-7ffe4bc0ae1e-workload-certs\") on node \"ip-10-0-141-128.ec2.internal\" DevicePath \"\""
Sep 12 00:45:50.857246 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:45:50.857247    2571 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7eef202a-38a8-4944-a4ef-7ffe4bc0ae1e-workload-socket\") on node \"ip-10-0-141-128.ec2.internal\" DevicePath \"\""
Sep 12 00:45:50.862918 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:45:50.862715    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7eef202a-38a8-4944-a4ef-7ffe4bc0ae1e-kube-api-access-4j26h" (OuterVolumeSpecName: "kube-api-access-4j26h") pod "7eef202a-38a8-4944-a4ef-7ffe4bc0ae1e" (UID: "7eef202a-38a8-4944-a4ef-7ffe4bc0ae1e"). InnerVolumeSpecName "kube-api-access-4j26h". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 12 00:45:50.862918 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:45:50.862727    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7eef202a-38a8-4944-a4ef-7ffe4bc0ae1e-istio-token" (OuterVolumeSpecName: "istio-token") pod "7eef202a-38a8-4944-a4ef-7ffe4bc0ae1e" (UID: "7eef202a-38a8-4944-a4ef-7ffe4bc0ae1e"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 12 00:45:50.866020 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:45:50.864708    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7eef202a-38a8-4944-a4ef-7ffe4bc0ae1e-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "7eef202a-38a8-4944-a4ef-7ffe4bc0ae1e" (UID: "7eef202a-38a8-4944-a4ef-7ffe4bc0ae1e"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 12 00:45:50.867084 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:45:50.866938    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/7eef202a-38a8-4944-a4ef-7ffe4bc0ae1e-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "7eef202a-38a8-4944-a4ef-7ffe4bc0ae1e" (UID: "7eef202a-38a8-4944-a4ef-7ffe4bc0ae1e"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 12 00:45:50.958475 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:45:50.958404    2571 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7eef202a-38a8-4944-a4ef-7ffe4bc0ae1e-istio-podinfo\") on node \"ip-10-0-141-128.ec2.internal\" DevicePath \"\""
Sep 12 00:45:50.958475 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:45:50.958430    2571 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4j26h\" (UniqueName: \"kubernetes.io/projected/7eef202a-38a8-4944-a4ef-7ffe4bc0ae1e-kube-api-access-4j26h\") on node \"ip-10-0-141-128.ec2.internal\" DevicePath \"\""
Sep 12 00:45:50.958475 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:45:50.958441    2571 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7eef202a-38a8-4944-a4ef-7ffe4bc0ae1e-istio-token\") on node \"ip-10-0-141-128.ec2.internal\" DevicePath \"\""
Sep 12 00:45:50.958475 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:45:50.958450    2571 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7eef202a-38a8-4944-a4ef-7ffe4bc0ae1e-istio-envoy\") on node \"ip-10-0-141-128.ec2.internal\" DevicePath \"\""
Sep 12 00:45:51.145260 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:45:51.145225    2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-b-593845-openshift-default-54dcc87546-pvswg"]
Sep 12 00:45:51.148314 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:45:51.148289    2571 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-api-b-593845-openshift-default-54dcc87546-pvswg"]
Sep 12 00:45:53.141601 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:45:53.141549    2571 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7eef202a-38a8-4944-a4ef-7ffe4bc0ae1e" path="/var/lib/kubelet/pods/7eef202a-38a8-4944-a4ef-7ffe4bc0ae1e/volumes"
Sep 12 00:46:02.769782 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:46:02.769741    2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-a-w2-0d3a66-openshift-default-6c7595fdd6-f92vh"]
Sep 12 00:46:02.770146 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:46:02.770004    2571 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-shared-a-w2-0d3a66-openshift-default-6c7595fdd6-f92vh" podUID="c3891135-55df-4b1f-aaba-ea417c6d1630" containerName="istio-proxy" containerID="cri-o://b50efbdaa00cf8db4e7158811c8629ecc26250aca1c03b8249c85b33ae3d9d4e" gracePeriod=30
Sep 12 00:46:02.856090 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:46:02.856060    2571 patch_prober.go:28] interesting pod/e2e-shared-a-w2-0d3a66-openshift-default-6c7595fdd6-f92vh container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Sep 12 00:46:02.856245 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:46:02.856125    2571 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-shared-a-w2-0d3a66-openshift-default-6c7595fdd6-f92vh" podUID="c3891135-55df-4b1f-aaba-ea417c6d1630" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Sep 12 00:46:07.928914 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:46:07.928883    2571 generic.go:356] "Generic (PLEG): container finished" podID="c3891135-55df-4b1f-aaba-ea417c6d1630" containerID="b50efbdaa00cf8db4e7158811c8629ecc26250aca1c03b8249c85b33ae3d9d4e" exitCode=0
Sep 12 00:46:07.929286 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:46:07.928930    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w2-0d3a66-openshift-default-6c7595fdd6-f92vh" event={"ID":"c3891135-55df-4b1f-aaba-ea417c6d1630","Type":"ContainerDied","Data":"b50efbdaa00cf8db4e7158811c8629ecc26250aca1c03b8249c85b33ae3d9d4e"}
Sep 12 00:46:07.929286 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:46:07.928979    2571 scope.go:117] "RemoveContainer" containerID="a6d921ba1cd2404679eafa67a015b6103aab40f95765250c2fb99c2fa7e65c67"
Sep 12 00:46:08.025971 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:46:08.025948    2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-w2-0d3a66-openshift-default-6c7595fdd6-f92vh"
Sep 12 00:46:08.119848 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:46:08.119787    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c3891135-55df-4b1f-aaba-ea417c6d1630-workload-socket\") pod \"c3891135-55df-4b1f-aaba-ea417c6d1630\" (UID: \"c3891135-55df-4b1f-aaba-ea417c6d1630\") "
Sep 12 00:46:08.119848 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:46:08.119829    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c3891135-55df-4b1f-aaba-ea417c6d1630-istio-envoy\") pod \"c3891135-55df-4b1f-aaba-ea417c6d1630\" (UID: \"c3891135-55df-4b1f-aaba-ea417c6d1630\") "
Sep 12 00:46:08.120058 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:46:08.119851    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c3891135-55df-4b1f-aaba-ea417c6d1630-credential-socket\") pod \"c3891135-55df-4b1f-aaba-ea417c6d1630\" (UID: \"c3891135-55df-4b1f-aaba-ea417c6d1630\") "
Sep 12 00:46:08.120058 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:46:08.119880    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c3891135-55df-4b1f-aaba-ea417c6d1630-istiod-ca-cert\") pod \"c3891135-55df-4b1f-aaba-ea417c6d1630\" (UID: \"c3891135-55df-4b1f-aaba-ea417c6d1630\") "
Sep 12 00:46:08.120058 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:46:08.119928    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c3891135-55df-4b1f-aaba-ea417c6d1630-istio-data\") pod \"c3891135-55df-4b1f-aaba-ea417c6d1630\" (UID: \"c3891135-55df-4b1f-aaba-ea417c6d1630\") "
Sep 12 00:46:08.120058 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:46:08.119950    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c3891135-55df-4b1f-aaba-ea417c6d1630-istio-podinfo\") pod \"c3891135-55df-4b1f-aaba-ea417c6d1630\" (UID: \"c3891135-55df-4b1f-aaba-ea417c6d1630\") "
Sep 12 00:46:08.120058 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:46:08.119987    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c3891135-55df-4b1f-aaba-ea417c6d1630-istio-token\") pod \"c3891135-55df-4b1f-aaba-ea417c6d1630\" (UID: \"c3891135-55df-4b1f-aaba-ea417c6d1630\") "
Sep 12 00:46:08.120058 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:46:08.120027    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-wld99\" (UniqueName: \"kubernetes.io/projected/c3891135-55df-4b1f-aaba-ea417c6d1630-kube-api-access-wld99\") pod \"c3891135-55df-4b1f-aaba-ea417c6d1630\" (UID: \"c3891135-55df-4b1f-aaba-ea417c6d1630\") "
Sep 12 00:46:08.120403 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:46:08.120061    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c3891135-55df-4b1f-aaba-ea417c6d1630-workload-certs\") pod \"c3891135-55df-4b1f-aaba-ea417c6d1630\" (UID: \"c3891135-55df-4b1f-aaba-ea417c6d1630\") "
Sep 12 00:46:08.120458 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:46:08.120053    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c3891135-55df-4b1f-aaba-ea417c6d1630-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "c3891135-55df-4b1f-aaba-ea417c6d1630" (UID: "c3891135-55df-4b1f-aaba-ea417c6d1630"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 12 00:46:08.120458 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:46:08.120180    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c3891135-55df-4b1f-aaba-ea417c6d1630-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "c3891135-55df-4b1f-aaba-ea417c6d1630" (UID: "c3891135-55df-4b1f-aaba-ea417c6d1630"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 12 00:46:08.120458 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:46:08.120231    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c3891135-55df-4b1f-aaba-ea417c6d1630-istio-data" (OuterVolumeSpecName: "istio-data") pod "c3891135-55df-4b1f-aaba-ea417c6d1630" (UID: "c3891135-55df-4b1f-aaba-ea417c6d1630"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 12 00:46:08.120458 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:46:08.120333    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c3891135-55df-4b1f-aaba-ea417c6d1630-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "c3891135-55df-4b1f-aaba-ea417c6d1630" (UID: "c3891135-55df-4b1f-aaba-ea417c6d1630"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 12 00:46:08.120458 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:46:08.120436    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c3891135-55df-4b1f-aaba-ea417c6d1630-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "c3891135-55df-4b1f-aaba-ea417c6d1630" (UID: "c3891135-55df-4b1f-aaba-ea417c6d1630"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 12 00:46:08.122059 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:46:08.122030    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c3891135-55df-4b1f-aaba-ea417c6d1630-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "c3891135-55df-4b1f-aaba-ea417c6d1630" (UID: "c3891135-55df-4b1f-aaba-ea417c6d1630"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 12 00:46:08.122178 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:46:08.122150    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c3891135-55df-4b1f-aaba-ea417c6d1630-kube-api-access-wld99" (OuterVolumeSpecName: "kube-api-access-wld99") pod "c3891135-55df-4b1f-aaba-ea417c6d1630" (UID: "c3891135-55df-4b1f-aaba-ea417c6d1630"). InnerVolumeSpecName "kube-api-access-wld99". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 12 00:46:08.122310 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:46:08.122290    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c3891135-55df-4b1f-aaba-ea417c6d1630-istio-token" (OuterVolumeSpecName: "istio-token") pod "c3891135-55df-4b1f-aaba-ea417c6d1630" (UID: "c3891135-55df-4b1f-aaba-ea417c6d1630"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 12 00:46:08.122353 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:46:08.122322    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/c3891135-55df-4b1f-aaba-ea417c6d1630-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "c3891135-55df-4b1f-aaba-ea417c6d1630" (UID: "c3891135-55df-4b1f-aaba-ea417c6d1630"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 12 00:46:08.221844 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:46:08.221815    2571 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c3891135-55df-4b1f-aaba-ea417c6d1630-workload-socket\") on node \"ip-10-0-141-128.ec2.internal\" DevicePath \"\""
Sep 12 00:46:08.221844 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:46:08.221840    2571 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c3891135-55df-4b1f-aaba-ea417c6d1630-istio-envoy\") on node \"ip-10-0-141-128.ec2.internal\" DevicePath \"\""
Sep 12 00:46:08.221844 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:46:08.221849    2571 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c3891135-55df-4b1f-aaba-ea417c6d1630-credential-socket\") on node \"ip-10-0-141-128.ec2.internal\" DevicePath \"\""
Sep 12 00:46:08.222057 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:46:08.221858    2571 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c3891135-55df-4b1f-aaba-ea417c6d1630-istiod-ca-cert\") on node \"ip-10-0-141-128.ec2.internal\" DevicePath \"\""
Sep 12 00:46:08.222057 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:46:08.221866    2571 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c3891135-55df-4b1f-aaba-ea417c6d1630-istio-data\") on node \"ip-10-0-141-128.ec2.internal\" DevicePath \"\""
Sep 12 00:46:08.222057 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:46:08.221873    2571 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c3891135-55df-4b1f-aaba-ea417c6d1630-istio-podinfo\") on node \"ip-10-0-141-128.ec2.internal\" DevicePath \"\""
Sep 12 00:46:08.222057 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:46:08.221880    2571 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c3891135-55df-4b1f-aaba-ea417c6d1630-istio-token\") on node \"ip-10-0-141-128.ec2.internal\" DevicePath \"\""
Sep 12 00:46:08.222057 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:46:08.221888    2571 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wld99\" (UniqueName: \"kubernetes.io/projected/c3891135-55df-4b1f-aaba-ea417c6d1630-kube-api-access-wld99\") on node \"ip-10-0-141-128.ec2.internal\" DevicePath \"\""
Sep 12 00:46:08.222057 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:46:08.221895    2571 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c3891135-55df-4b1f-aaba-ea417c6d1630-workload-certs\") on node \"ip-10-0-141-128.ec2.internal\" DevicePath \"\""
Sep 12 00:46:08.935553 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:46:08.935516    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w2-0d3a66-openshift-default-6c7595fdd6-f92vh" event={"ID":"c3891135-55df-4b1f-aaba-ea417c6d1630","Type":"ContainerDied","Data":"49bee2b95289bc9c243a94e3687db4abc91433aa46ea1a3f786a799fb89a3b36"}
Sep 12 00:46:08.935990 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:46:08.935603    2571 scope.go:117] "RemoveContainer" containerID="b50efbdaa00cf8db4e7158811c8629ecc26250aca1c03b8249c85b33ae3d9d4e"
Sep 12 00:46:08.935990 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:46:08.935531    2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-w2-0d3a66-openshift-default-6c7595fdd6-f92vh"
Sep 12 00:46:08.957309 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:46:08.957281    2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-a-w2-0d3a66-openshift-default-6c7595fdd6-f92vh"]
Sep 12 00:46:08.960472 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:46:08.960450    2571 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-shared-a-w2-0d3a66-openshift-default-6c7595fdd6-f92vh"]
Sep 12 00:46:09.146694 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:46:09.146664    2571 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c3891135-55df-4b1f-aaba-ea417c6d1630" path="/var/lib/kubelet/pods/c3891135-55df-4b1f-aaba-ea417c6d1630/volumes"
Sep 12 00:46:13.588954 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:46:13.588903    2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w2-f12ab7-29819565-qbvds"]
Sep 12 00:46:13.590987 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:46:13.590964    2571 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w2-f12ab7-29819565-qbvds"]
Sep 12 00:46:13.611356 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:46:13.611322    2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w2-f12ab7-6bdf9c9f7c-dwjcr"]
Sep 12 00:46:13.612204 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:46:13.611562    2571 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-shared-b-w2-f12ab7-6bdf9c9f7c-dwjcr" podUID="746506d4-a95a-415d-8c3d-fcb8c0198373" containerName="payload-processing" containerID="cri-o://16e50e2d62d6b4f9fa8678cabe2a94c0deba3a909b12429ae2b3a1dee5672e66" gracePeriod=30
Sep 12 00:46:13.619265 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:46:13.618952    2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w2-f12ab7-59b8766c78-m6swk"]
Sep 12 00:46:13.619265 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:46:13.619156    2571 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-f12ab7-59b8766c78-m6swk" podUID="66da68b7-6480-47e9-aa60-3798029410e2" containerName="payload-pre-processing" containerID="cri-o://cd81b7dc9acf8b5683cb0804c24546233537eb118353b0f6155aa236dd8edeb6" gracePeriod=30
Sep 12 00:46:13.911472 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:46:13.911449    2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-b-w2-f12ab7-6bdf9c9f7c-dwjcr"
Sep 12 00:46:13.914340 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:46:13.914315    2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-f12ab7-59b8766c78-m6swk"
Sep 12 00:46:13.970510 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:46:13.970473    2571 generic.go:356] "Generic (PLEG): container finished" podID="746506d4-a95a-415d-8c3d-fcb8c0198373" containerID="16e50e2d62d6b4f9fa8678cabe2a94c0deba3a909b12429ae2b3a1dee5672e66" exitCode=0
Sep 12 00:46:13.970690 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:46:13.970648    2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-b-w2-f12ab7-6bdf9c9f7c-dwjcr"
Sep 12 00:46:13.970998 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:46:13.970974    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-b-w2-f12ab7-6bdf9c9f7c-dwjcr" event={"ID":"746506d4-a95a-415d-8c3d-fcb8c0198373","Type":"ContainerDied","Data":"16e50e2d62d6b4f9fa8678cabe2a94c0deba3a909b12429ae2b3a1dee5672e66"}
Sep 12 00:46:13.971057 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:46:13.971031    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-b-w2-f12ab7-6bdf9c9f7c-dwjcr" event={"ID":"746506d4-a95a-415d-8c3d-fcb8c0198373","Type":"ContainerDied","Data":"f32d0338e9c03374a56ee95fce5e1b5f3af60e8f0f931164013750038151527d"}
Sep 12 00:46:13.971111 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:46:13.971061    2571 scope.go:117] "RemoveContainer" containerID="16e50e2d62d6b4f9fa8678cabe2a94c0deba3a909b12429ae2b3a1dee5672e66"
Sep 12 00:46:13.973552 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:46:13.973527    2571 generic.go:356] "Generic (PLEG): container finished" podID="66da68b7-6480-47e9-aa60-3798029410e2" containerID="cd81b7dc9acf8b5683cb0804c24546233537eb118353b0f6155aa236dd8edeb6" exitCode=0
Sep 12 00:46:13.973773 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:46:13.973646    2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-f12ab7-59b8766c78-m6swk"
Sep 12 00:46:13.973773 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:46:13.973734    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-f12ab7-59b8766c78-m6swk" event={"ID":"66da68b7-6480-47e9-aa60-3798029410e2","Type":"ContainerDied","Data":"cd81b7dc9acf8b5683cb0804c24546233537eb118353b0f6155aa236dd8edeb6"}
Sep 12 00:46:13.973902 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:46:13.973781    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-f12ab7-59b8766c78-m6swk" event={"ID":"66da68b7-6480-47e9-aa60-3798029410e2","Type":"ContainerDied","Data":"599bded7f6f5c8b10e758a0e0d3354b9f8e61df712b54d154cee649ed945171d"}
Sep 12 00:46:13.975850 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:46:13.975824    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-49zht\" (UniqueName: \"kubernetes.io/projected/66da68b7-6480-47e9-aa60-3798029410e2-kube-api-access-49zht\") pod \"66da68b7-6480-47e9-aa60-3798029410e2\" (UID: \"66da68b7-6480-47e9-aa60-3798029410e2\") "
Sep 12 00:46:13.975959 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:46:13.975890    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/746506d4-a95a-415d-8c3d-fcb8c0198373-plugins-config-volume\") pod \"746506d4-a95a-415d-8c3d-fcb8c0198373\" (UID: \"746506d4-a95a-415d-8c3d-fcb8c0198373\") "
Sep 12 00:46:13.976017 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:46:13.975962    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-9t5d6\" (UniqueName: \"kubernetes.io/projected/746506d4-a95a-415d-8c3d-fcb8c0198373-kube-api-access-9t5d6\") pod \"746506d4-a95a-415d-8c3d-fcb8c0198373\" (UID: \"746506d4-a95a-415d-8c3d-fcb8c0198373\") "
Sep 12 00:46:13.976017 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:46:13.976005    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/66da68b7-6480-47e9-aa60-3798029410e2-plugins-config-volume\") pod \"66da68b7-6480-47e9-aa60-3798029410e2\" (UID: \"66da68b7-6480-47e9-aa60-3798029410e2\") "
Sep 12 00:46:13.976691 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:46:13.976643    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/746506d4-a95a-415d-8c3d-fcb8c0198373-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "746506d4-a95a-415d-8c3d-fcb8c0198373" (UID: "746506d4-a95a-415d-8c3d-fcb8c0198373"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 12 00:46:13.976772 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:46:13.976702    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/66da68b7-6480-47e9-aa60-3798029410e2-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "66da68b7-6480-47e9-aa60-3798029410e2" (UID: "66da68b7-6480-47e9-aa60-3798029410e2"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 12 00:46:13.979297 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:46:13.979269    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/66da68b7-6480-47e9-aa60-3798029410e2-kube-api-access-49zht" (OuterVolumeSpecName: "kube-api-access-49zht") pod "66da68b7-6480-47e9-aa60-3798029410e2" (UID: "66da68b7-6480-47e9-aa60-3798029410e2"). InnerVolumeSpecName "kube-api-access-49zht". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 12 00:46:13.980822 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:46:13.980798    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/746506d4-a95a-415d-8c3d-fcb8c0198373-kube-api-access-9t5d6" (OuterVolumeSpecName: "kube-api-access-9t5d6") pod "746506d4-a95a-415d-8c3d-fcb8c0198373" (UID: "746506d4-a95a-415d-8c3d-fcb8c0198373"). InnerVolumeSpecName "kube-api-access-9t5d6". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 12 00:46:13.983830 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:46:13.983796    2571 scope.go:117] "RemoveContainer" containerID="16e50e2d62d6b4f9fa8678cabe2a94c0deba3a909b12429ae2b3a1dee5672e66"
Sep 12 00:46:13.984111 ip-10-0-141-128 kubenswrapper[2571]: E0912 00:46:13.984089    2571 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"16e50e2d62d6b4f9fa8678cabe2a94c0deba3a909b12429ae2b3a1dee5672e66\": container with ID starting with 16e50e2d62d6b4f9fa8678cabe2a94c0deba3a909b12429ae2b3a1dee5672e66 not found: ID does not exist" containerID="16e50e2d62d6b4f9fa8678cabe2a94c0deba3a909b12429ae2b3a1dee5672e66"
Sep 12 00:46:13.984183 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:46:13.984120    2571 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"16e50e2d62d6b4f9fa8678cabe2a94c0deba3a909b12429ae2b3a1dee5672e66"} err="failed to get container status \"16e50e2d62d6b4f9fa8678cabe2a94c0deba3a909b12429ae2b3a1dee5672e66\": rpc error: code = NotFound desc = could not find container \"16e50e2d62d6b4f9fa8678cabe2a94c0deba3a909b12429ae2b3a1dee5672e66\": container with ID starting with 16e50e2d62d6b4f9fa8678cabe2a94c0deba3a909b12429ae2b3a1dee5672e66 not found: ID does not exist"
Sep 12 00:46:13.984183 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:46:13.984139    2571 scope.go:117] "RemoveContainer" containerID="cd81b7dc9acf8b5683cb0804c24546233537eb118353b0f6155aa236dd8edeb6"
Sep 12 00:46:13.993218 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:46:13.993195    2571 scope.go:117] "RemoveContainer" containerID="cd81b7dc9acf8b5683cb0804c24546233537eb118353b0f6155aa236dd8edeb6"
Sep 12 00:46:13.993499 ip-10-0-141-128 kubenswrapper[2571]: E0912 00:46:13.993465    2571 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"cd81b7dc9acf8b5683cb0804c24546233537eb118353b0f6155aa236dd8edeb6\": container with ID starting with cd81b7dc9acf8b5683cb0804c24546233537eb118353b0f6155aa236dd8edeb6 not found: ID does not exist" containerID="cd81b7dc9acf8b5683cb0804c24546233537eb118353b0f6155aa236dd8edeb6"
Sep 12 00:46:13.993574 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:46:13.993506    2571 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"cd81b7dc9acf8b5683cb0804c24546233537eb118353b0f6155aa236dd8edeb6"} err="failed to get container status \"cd81b7dc9acf8b5683cb0804c24546233537eb118353b0f6155aa236dd8edeb6\": rpc error: code = NotFound desc = could not find container \"cd81b7dc9acf8b5683cb0804c24546233537eb118353b0f6155aa236dd8edeb6\": container with ID starting with cd81b7dc9acf8b5683cb0804c24546233537eb118353b0f6155aa236dd8edeb6 not found: ID does not exist"
Sep 12 00:46:14.077373 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:46:14.077348    2571 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-49zht\" (UniqueName: \"kubernetes.io/projected/66da68b7-6480-47e9-aa60-3798029410e2-kube-api-access-49zht\") on node \"ip-10-0-141-128.ec2.internal\" DevicePath \"\""
Sep 12 00:46:14.077373 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:46:14.077371    2571 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/746506d4-a95a-415d-8c3d-fcb8c0198373-plugins-config-volume\") on node \"ip-10-0-141-128.ec2.internal\" DevicePath \"\""
Sep 12 00:46:14.077514 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:46:14.077382    2571 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9t5d6\" (UniqueName: \"kubernetes.io/projected/746506d4-a95a-415d-8c3d-fcb8c0198373-kube-api-access-9t5d6\") on node \"ip-10-0-141-128.ec2.internal\" DevicePath \"\""
Sep 12 00:46:14.077514 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:46:14.077390    2571 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/66da68b7-6480-47e9-aa60-3798029410e2-plugins-config-volume\") on node \"ip-10-0-141-128.ec2.internal\" DevicePath \"\""
Sep 12 00:46:14.295465 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:46:14.295356    2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w2-f12ab7-6bdf9c9f7c-dwjcr"]
Sep 12 00:46:14.298456 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:46:14.298425    2571 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w2-f12ab7-6bdf9c9f7c-dwjcr"]
Sep 12 00:46:14.305496 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:46:14.305452    2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w2-f12ab7-59b8766c78-m6swk"]
Sep 12 00:46:14.307150 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:46:14.307130    2571 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w2-f12ab7-59b8766c78-m6swk"]
Sep 12 00:46:15.140563 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:46:15.140531    2571 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="66da68b7-6480-47e9-aa60-3798029410e2" path="/var/lib/kubelet/pods/66da68b7-6480-47e9-aa60-3798029410e2/volumes"
Sep 12 00:46:15.140949 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:46:15.140906    2571 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="746506d4-a95a-415d-8c3d-fcb8c0198373" path="/var/lib/kubelet/pods/746506d4-a95a-415d-8c3d-fcb8c0198373/volumes"
Sep 12 00:46:15.141151 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:46:15.141138    2571 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="78d2f831-fdc9-4d7f-b363-2c0a3a185b8f" path="/var/lib/kubelet/pods/78d2f831-fdc9-4d7f-b363-2c0a3a185b8f/volumes"
Sep 12 00:46:20.100903 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:46:20.100867    2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-b-w2-f12ab7-openshift-default-678468bf74-42mbj"]
Sep 12 00:46:20.101363 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:46:20.101129    2571 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-shared-b-w2-f12ab7-openshift-default-678468bf74-42mbj" podUID="31532d6d-06e5-4f45-922c-184e13d910fe" containerName="istio-proxy" containerID="cri-o://d7cd1eff2923a1b9e6d23cd056c1399b3ac9012e15dd445a1a656cedfa921208" gracePeriod=30
Sep 12 00:46:25.356950 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:46:25.356924    2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-w2-f12ab7-openshift-default-678468bf74-42mbj"
Sep 12 00:46:25.488824 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:46:25.488792    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/31532d6d-06e5-4f45-922c-184e13d910fe-workload-socket\") pod \"31532d6d-06e5-4f45-922c-184e13d910fe\" (UID: \"31532d6d-06e5-4f45-922c-184e13d910fe\") "
Sep 12 00:46:25.488824 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:46:25.488832    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/31532d6d-06e5-4f45-922c-184e13d910fe-istio-podinfo\") pod \"31532d6d-06e5-4f45-922c-184e13d910fe\" (UID: \"31532d6d-06e5-4f45-922c-184e13d910fe\") "
Sep 12 00:46:25.489087 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:46:25.488871    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/31532d6d-06e5-4f45-922c-184e13d910fe-credential-socket\") pod \"31532d6d-06e5-4f45-922c-184e13d910fe\" (UID: \"31532d6d-06e5-4f45-922c-184e13d910fe\") "
Sep 12 00:46:25.489087 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:46:25.488893    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/31532d6d-06e5-4f45-922c-184e13d910fe-istio-data\") pod \"31532d6d-06e5-4f45-922c-184e13d910fe\" (UID: \"31532d6d-06e5-4f45-922c-184e13d910fe\") "
Sep 12 00:46:25.489087 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:46:25.488912    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/31532d6d-06e5-4f45-922c-184e13d910fe-workload-certs\") pod \"31532d6d-06e5-4f45-922c-184e13d910fe\" (UID: \"31532d6d-06e5-4f45-922c-184e13d910fe\") "
Sep 12 00:46:25.489087 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:46:25.488932    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/31532d6d-06e5-4f45-922c-184e13d910fe-istiod-ca-cert\") pod \"31532d6d-06e5-4f45-922c-184e13d910fe\" (UID: \"31532d6d-06e5-4f45-922c-184e13d910fe\") "
Sep 12 00:46:25.489087 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:46:25.488968    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-g9lpr\" (UniqueName: \"kubernetes.io/projected/31532d6d-06e5-4f45-922c-184e13d910fe-kube-api-access-g9lpr\") pod \"31532d6d-06e5-4f45-922c-184e13d910fe\" (UID: \"31532d6d-06e5-4f45-922c-184e13d910fe\") "
Sep 12 00:46:25.489316 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:46:25.489122    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/31532d6d-06e5-4f45-922c-184e13d910fe-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "31532d6d-06e5-4f45-922c-184e13d910fe" (UID: "31532d6d-06e5-4f45-922c-184e13d910fe"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 12 00:46:25.489316 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:46:25.489220    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/31532d6d-06e5-4f45-922c-184e13d910fe-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "31532d6d-06e5-4f45-922c-184e13d910fe" (UID: "31532d6d-06e5-4f45-922c-184e13d910fe"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 12 00:46:25.489316 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:46:25.489236    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/31532d6d-06e5-4f45-922c-184e13d910fe-istio-token\") pod \"31532d6d-06e5-4f45-922c-184e13d910fe\" (UID: \"31532d6d-06e5-4f45-922c-184e13d910fe\") "
Sep 12 00:46:25.489429 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:46:25.489314    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/31532d6d-06e5-4f45-922c-184e13d910fe-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "31532d6d-06e5-4f45-922c-184e13d910fe" (UID: "31532d6d-06e5-4f45-922c-184e13d910fe"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 12 00:46:25.489429 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:46:25.489300    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/31532d6d-06e5-4f45-922c-184e13d910fe-istio-data" (OuterVolumeSpecName: "istio-data") pod "31532d6d-06e5-4f45-922c-184e13d910fe" (UID: "31532d6d-06e5-4f45-922c-184e13d910fe"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 12 00:46:25.489429 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:46:25.489372    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/31532d6d-06e5-4f45-922c-184e13d910fe-istio-envoy\") pod \"31532d6d-06e5-4f45-922c-184e13d910fe\" (UID: \"31532d6d-06e5-4f45-922c-184e13d910fe\") "
Sep 12 00:46:25.489618 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:46:25.489468    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/31532d6d-06e5-4f45-922c-184e13d910fe-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "31532d6d-06e5-4f45-922c-184e13d910fe" (UID: "31532d6d-06e5-4f45-922c-184e13d910fe"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 12 00:46:25.489785 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:46:25.489764    2571 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/31532d6d-06e5-4f45-922c-184e13d910fe-istio-data\") on node \"ip-10-0-141-128.ec2.internal\" DevicePath \"\""
Sep 12 00:46:25.489869 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:46:25.489790    2571 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/31532d6d-06e5-4f45-922c-184e13d910fe-workload-certs\") on node \"ip-10-0-141-128.ec2.internal\" DevicePath \"\""
Sep 12 00:46:25.489869 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:46:25.489805    2571 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/31532d6d-06e5-4f45-922c-184e13d910fe-istiod-ca-cert\") on node \"ip-10-0-141-128.ec2.internal\" DevicePath \"\""
Sep 12 00:46:25.489869 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:46:25.489817    2571 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/31532d6d-06e5-4f45-922c-184e13d910fe-workload-socket\") on node \"ip-10-0-141-128.ec2.internal\" DevicePath \"\""
Sep 12 00:46:25.489869 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:46:25.489830    2571 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/31532d6d-06e5-4f45-922c-184e13d910fe-credential-socket\") on node \"ip-10-0-141-128.ec2.internal\" DevicePath \"\""
Sep 12 00:46:25.491322 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:46:25.491287    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/31532d6d-06e5-4f45-922c-184e13d910fe-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "31532d6d-06e5-4f45-922c-184e13d910fe" (UID: "31532d6d-06e5-4f45-922c-184e13d910fe"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 12 00:46:25.491322 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:46:25.491306    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/31532d6d-06e5-4f45-922c-184e13d910fe-istio-token" (OuterVolumeSpecName: "istio-token") pod "31532d6d-06e5-4f45-922c-184e13d910fe" (UID: "31532d6d-06e5-4f45-922c-184e13d910fe"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 12 00:46:25.491468 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:46:25.491367    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/31532d6d-06e5-4f45-922c-184e13d910fe-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "31532d6d-06e5-4f45-922c-184e13d910fe" (UID: "31532d6d-06e5-4f45-922c-184e13d910fe"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 12 00:46:25.491468 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:46:25.491363    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/31532d6d-06e5-4f45-922c-184e13d910fe-kube-api-access-g9lpr" (OuterVolumeSpecName: "kube-api-access-g9lpr") pod "31532d6d-06e5-4f45-922c-184e13d910fe" (UID: "31532d6d-06e5-4f45-922c-184e13d910fe"). InnerVolumeSpecName "kube-api-access-g9lpr". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 12 00:46:25.590666 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:46:25.590638    2571 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/31532d6d-06e5-4f45-922c-184e13d910fe-istio-envoy\") on node \"ip-10-0-141-128.ec2.internal\" DevicePath \"\""
Sep 12 00:46:25.590666 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:46:25.590662    2571 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/31532d6d-06e5-4f45-922c-184e13d910fe-istio-podinfo\") on node \"ip-10-0-141-128.ec2.internal\" DevicePath \"\""
Sep 12 00:46:25.590666 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:46:25.590670    2571 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-g9lpr\" (UniqueName: \"kubernetes.io/projected/31532d6d-06e5-4f45-922c-184e13d910fe-kube-api-access-g9lpr\") on node \"ip-10-0-141-128.ec2.internal\" DevicePath \"\""
Sep 12 00:46:25.590910 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:46:25.590679    2571 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/31532d6d-06e5-4f45-922c-184e13d910fe-istio-token\") on node \"ip-10-0-141-128.ec2.internal\" DevicePath \"\""
Sep 12 00:46:26.041849 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:46:26.041819    2571 generic.go:356] "Generic (PLEG): container finished" podID="31532d6d-06e5-4f45-922c-184e13d910fe" containerID="d7cd1eff2923a1b9e6d23cd056c1399b3ac9012e15dd445a1a656cedfa921208" exitCode=0
Sep 12 00:46:26.042010 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:46:26.041869    2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-w2-f12ab7-openshift-default-678468bf74-42mbj"
Sep 12 00:46:26.042010 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:46:26.041909    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w2-f12ab7-openshift-default-678468bf74-42mbj" event={"ID":"31532d6d-06e5-4f45-922c-184e13d910fe","Type":"ContainerDied","Data":"d7cd1eff2923a1b9e6d23cd056c1399b3ac9012e15dd445a1a656cedfa921208"}
Sep 12 00:46:26.042010 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:46:26.041995    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w2-f12ab7-openshift-default-678468bf74-42mbj" event={"ID":"31532d6d-06e5-4f45-922c-184e13d910fe","Type":"ContainerDied","Data":"14d088cc81ba01a477e742160f247a6572a9889bf2ada614aaa3d9388fd6303a"}
Sep 12 00:46:26.042121 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:46:26.042023    2571 scope.go:117] "RemoveContainer" containerID="d7cd1eff2923a1b9e6d23cd056c1399b3ac9012e15dd445a1a656cedfa921208"
Sep 12 00:46:26.051511 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:46:26.051494    2571 scope.go:117] "RemoveContainer" containerID="42ee050857dfd38d4e3f907134e7e6c185c8d74bb1d1a99c15edb52334fa7e41"
Sep 12 00:46:26.059424 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:46:26.059407    2571 scope.go:117] "RemoveContainer" containerID="d7cd1eff2923a1b9e6d23cd056c1399b3ac9012e15dd445a1a656cedfa921208"
Sep 12 00:46:26.059726 ip-10-0-141-128 kubenswrapper[2571]: E0912 00:46:26.059705    2571 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d7cd1eff2923a1b9e6d23cd056c1399b3ac9012e15dd445a1a656cedfa921208\": container with ID starting with d7cd1eff2923a1b9e6d23cd056c1399b3ac9012e15dd445a1a656cedfa921208 not found: ID does not exist" containerID="d7cd1eff2923a1b9e6d23cd056c1399b3ac9012e15dd445a1a656cedfa921208"
Sep 12 00:46:26.059818 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:46:26.059735    2571 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d7cd1eff2923a1b9e6d23cd056c1399b3ac9012e15dd445a1a656cedfa921208"} err="failed to get container status \"d7cd1eff2923a1b9e6d23cd056c1399b3ac9012e15dd445a1a656cedfa921208\": rpc error: code = NotFound desc = could not find container \"d7cd1eff2923a1b9e6d23cd056c1399b3ac9012e15dd445a1a656cedfa921208\": container with ID starting with d7cd1eff2923a1b9e6d23cd056c1399b3ac9012e15dd445a1a656cedfa921208 not found: ID does not exist"
Sep 12 00:46:26.059818 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:46:26.059754    2571 scope.go:117] "RemoveContainer" containerID="42ee050857dfd38d4e3f907134e7e6c185c8d74bb1d1a99c15edb52334fa7e41"
Sep 12 00:46:26.060217 ip-10-0-141-128 kubenswrapper[2571]: E0912 00:46:26.060183    2571 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"42ee050857dfd38d4e3f907134e7e6c185c8d74bb1d1a99c15edb52334fa7e41\": container with ID starting with 42ee050857dfd38d4e3f907134e7e6c185c8d74bb1d1a99c15edb52334fa7e41 not found: ID does not exist" containerID="42ee050857dfd38d4e3f907134e7e6c185c8d74bb1d1a99c15edb52334fa7e41"
Sep 12 00:46:26.060486 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:46:26.060224    2571 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"42ee050857dfd38d4e3f907134e7e6c185c8d74bb1d1a99c15edb52334fa7e41"} err="failed to get container status \"42ee050857dfd38d4e3f907134e7e6c185c8d74bb1d1a99c15edb52334fa7e41\": rpc error: code = NotFound desc = could not find container \"42ee050857dfd38d4e3f907134e7e6c185c8d74bb1d1a99c15edb52334fa7e41\": container with ID starting with 42ee050857dfd38d4e3f907134e7e6c185c8d74bb1d1a99c15edb52334fa7e41 not found: ID does not exist"
Sep 12 00:46:26.062700 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:46:26.062679    2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-b-w2-f12ab7-openshift-default-678468bf74-42mbj"]
Sep 12 00:46:26.068998 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:46:26.068976    2571 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-shared-b-w2-f12ab7-openshift-default-678468bf74-42mbj"]
Sep 12 00:46:26.323626 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:46:26.323524    2571 patch_prober.go:28] interesting pod/e2e-shared-b-w2-f12ab7-openshift-default-678468bf74-42mbj container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.57:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Sep 12 00:46:26.323798 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:46:26.323617    2571 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-shared-b-w2-f12ab7-openshift-default-678468bf74-42mbj" podUID="31532d6d-06e5-4f45-922c-184e13d910fe" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.57:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Sep 12 00:46:27.141633 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:46:27.141579    2571 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="31532d6d-06e5-4f45-922c-184e13d910fe" path="/var/lib/kubelet/pods/31532d6d-06e5-4f45-922c-184e13d910fe/volumes"
Sep 12 00:48:52.380983 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:48:52.380946    2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ipp-a-64e462-openshift-default-67664986bd-989bb"]
Sep 12 00:48:52.381422 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:48:52.381396    2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7eef202a-38a8-4944-a4ef-7ffe4bc0ae1e" containerName="istio-proxy"
Sep 12 00:48:52.381422 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:48:52.381410    2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="7eef202a-38a8-4944-a4ef-7ffe4bc0ae1e" containerName="istio-proxy"
Sep 12 00:48:52.381422 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:48:52.381422    2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="537eb92d-71ea-4651-a204-fee3a3728ace" containerName="istio-proxy"
Sep 12 00:48:52.381544 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:48:52.381428    2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="537eb92d-71ea-4651-a204-fee3a3728ace" containerName="istio-proxy"
Sep 12 00:48:52.381544 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:48:52.381436    2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="66da68b7-6480-47e9-aa60-3798029410e2" containerName="payload-pre-processing"
Sep 12 00:48:52.381544 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:48:52.381445    2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="66da68b7-6480-47e9-aa60-3798029410e2" containerName="payload-pre-processing"
Sep 12 00:48:52.381544 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:48:52.381457    2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c3891135-55df-4b1f-aaba-ea417c6d1630" containerName="istio-proxy"
Sep 12 00:48:52.381544 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:48:52.381462    2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="c3891135-55df-4b1f-aaba-ea417c6d1630" containerName="istio-proxy"
Sep 12 00:48:52.381544 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:48:52.381467    2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c3891135-55df-4b1f-aaba-ea417c6d1630" containerName="istio-proxy"
Sep 12 00:48:52.381544 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:48:52.381472    2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="c3891135-55df-4b1f-aaba-ea417c6d1630" containerName="istio-proxy"
Sep 12 00:48:52.381544 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:48:52.381479    2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="746506d4-a95a-415d-8c3d-fcb8c0198373" containerName="payload-processing"
Sep 12 00:48:52.381544 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:48:52.381485    2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="746506d4-a95a-415d-8c3d-fcb8c0198373" containerName="payload-processing"
Sep 12 00:48:52.381544 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:48:52.381495    2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7eef202a-38a8-4944-a4ef-7ffe4bc0ae1e" containerName="istio-proxy"
Sep 12 00:48:52.381544 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:48:52.381500    2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="7eef202a-38a8-4944-a4ef-7ffe4bc0ae1e" containerName="istio-proxy"
Sep 12 00:48:52.381544 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:48:52.381509    2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d7e353f5-8ba3-41a4-bd94-5e2c0cf9fa46" containerName="istio-proxy"
Sep 12 00:48:52.381544 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:48:52.381515    2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="d7e353f5-8ba3-41a4-bd94-5e2c0cf9fa46" containerName="istio-proxy"
Sep 12 00:48:52.381544 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:48:52.381524    2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="31532d6d-06e5-4f45-922c-184e13d910fe" containerName="istio-proxy"
Sep 12 00:48:52.381544 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:48:52.381530    2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="31532d6d-06e5-4f45-922c-184e13d910fe" containerName="istio-proxy"
Sep 12 00:48:52.381544 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:48:52.381537    2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e8528d32-8c40-4464-82d9-d58854afb3ac" containerName="maas-api"
Sep 12 00:48:52.381544 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:48:52.381543    2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="e8528d32-8c40-4464-82d9-d58854afb3ac" containerName="maas-api"
Sep 12 00:48:52.381544 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:48:52.381550    2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="78d2f831-fdc9-4d7f-b363-2c0a3a185b8f" containerName="cleanup"
Sep 12 00:48:52.382165 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:48:52.381555    2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="78d2f831-fdc9-4d7f-b363-2c0a3a185b8f" containerName="cleanup"
Sep 12 00:48:52.382165 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:48:52.381565    2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="31532d6d-06e5-4f45-922c-184e13d910fe" containerName="istio-proxy"
Sep 12 00:48:52.382165 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:48:52.381571    2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="31532d6d-06e5-4f45-922c-184e13d910fe" containerName="istio-proxy"
Sep 12 00:48:52.382165 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:48:52.381576    2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="537eb92d-71ea-4651-a204-fee3a3728ace" containerName="istio-proxy"
Sep 12 00:48:52.382165 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:48:52.381606    2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="537eb92d-71ea-4651-a204-fee3a3728ace" containerName="istio-proxy"
Sep 12 00:48:52.382165 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:48:52.381614    2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2b5cfa38-4c94-42e0-8684-0cdd2a93c3f5" containerName="revoke-keys"
Sep 12 00:48:52.382165 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:48:52.381619    2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="2b5cfa38-4c94-42e0-8684-0cdd2a93c3f5" containerName="revoke-keys"
Sep 12 00:48:52.382165 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:48:52.381628    2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d7e353f5-8ba3-41a4-bd94-5e2c0cf9fa46" containerName="istio-proxy"
Sep 12 00:48:52.382165 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:48:52.381633    2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="d7e353f5-8ba3-41a4-bd94-5e2c0cf9fa46" containerName="istio-proxy"
Sep 12 00:48:52.382165 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:48:52.381694    2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="537eb92d-71ea-4651-a204-fee3a3728ace" containerName="istio-proxy"
Sep 12 00:48:52.382165 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:48:52.381703    2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="31532d6d-06e5-4f45-922c-184e13d910fe" containerName="istio-proxy"
Sep 12 00:48:52.382165 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:48:52.381709    2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="d7e353f5-8ba3-41a4-bd94-5e2c0cf9fa46" containerName="istio-proxy"
Sep 12 00:48:52.382165 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:48:52.381716    2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="c3891135-55df-4b1f-aaba-ea417c6d1630" containerName="istio-proxy"
Sep 12 00:48:52.382165 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:48:52.381724    2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="31532d6d-06e5-4f45-922c-184e13d910fe" containerName="istio-proxy"
Sep 12 00:48:52.382165 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:48:52.381731    2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="78d2f831-fdc9-4d7f-b363-2c0a3a185b8f" containerName="cleanup"
Sep 12 00:48:52.382165 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:48:52.381739    2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="c3891135-55df-4b1f-aaba-ea417c6d1630" containerName="istio-proxy"
Sep 12 00:48:52.382165 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:48:52.381747    2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="7eef202a-38a8-4944-a4ef-7ffe4bc0ae1e" containerName="istio-proxy"
Sep 12 00:48:52.382165 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:48:52.381756    2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="7eef202a-38a8-4944-a4ef-7ffe4bc0ae1e" containerName="istio-proxy"
Sep 12 00:48:52.382165 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:48:52.381762    2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="e8528d32-8c40-4464-82d9-d58854afb3ac" containerName="maas-api"
Sep 12 00:48:52.382165 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:48:52.381766    2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="2b5cfa38-4c94-42e0-8684-0cdd2a93c3f5" containerName="revoke-keys"
Sep 12 00:48:52.382165 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:48:52.381771    2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="746506d4-a95a-415d-8c3d-fcb8c0198373" containerName="payload-processing"
Sep 12 00:48:52.382165 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:48:52.381778    2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="66da68b7-6480-47e9-aa60-3798029410e2" containerName="payload-pre-processing"
Sep 12 00:48:52.385060 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:48:52.385040    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-a-64e462-openshift-default-67664986bd-989bb"
Sep 12 00:48:52.387710 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:48:52.387683    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ipp-a-64e462-openshift-default-dockercfg-hk9j4\""
Sep 12 00:48:52.400096 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:48:52.400028    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ipp-a-64e462-openshift-default-67664986bd-989bb"]
Sep 12 00:48:52.481976 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:48:52.481937    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d73ec31b-c54e-44df-a448-0a914b938eb7-credential-socket\") pod \"e2e-ipp-a-64e462-openshift-default-67664986bd-989bb\" (UID: \"d73ec31b-c54e-44df-a448-0a914b938eb7\") " pod="openshift-ingress/e2e-ipp-a-64e462-openshift-default-67664986bd-989bb"
Sep 12 00:48:52.482136 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:48:52.481997    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d73ec31b-c54e-44df-a448-0a914b938eb7-workload-socket\") pod \"e2e-ipp-a-64e462-openshift-default-67664986bd-989bb\" (UID: \"d73ec31b-c54e-44df-a448-0a914b938eb7\") " pod="openshift-ingress/e2e-ipp-a-64e462-openshift-default-67664986bd-989bb"
Sep 12 00:48:52.482136 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:48:52.482063    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d73ec31b-c54e-44df-a448-0a914b938eb7-istiod-ca-cert\") pod \"e2e-ipp-a-64e462-openshift-default-67664986bd-989bb\" (UID: \"d73ec31b-c54e-44df-a448-0a914b938eb7\") " pod="openshift-ingress/e2e-ipp-a-64e462-openshift-default-67664986bd-989bb"
Sep 12 00:48:52.482136 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:48:52.482088    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d73ec31b-c54e-44df-a448-0a914b938eb7-istio-podinfo\") pod \"e2e-ipp-a-64e462-openshift-default-67664986bd-989bb\" (UID: \"d73ec31b-c54e-44df-a448-0a914b938eb7\") " pod="openshift-ingress/e2e-ipp-a-64e462-openshift-default-67664986bd-989bb"
Sep 12 00:48:52.482239 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:48:52.482146    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d73ec31b-c54e-44df-a448-0a914b938eb7-workload-certs\") pod \"e2e-ipp-a-64e462-openshift-default-67664986bd-989bb\" (UID: \"d73ec31b-c54e-44df-a448-0a914b938eb7\") " pod="openshift-ingress/e2e-ipp-a-64e462-openshift-default-67664986bd-989bb"
Sep 12 00:48:52.482239 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:48:52.482188    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d73ec31b-c54e-44df-a448-0a914b938eb7-istio-data\") pod \"e2e-ipp-a-64e462-openshift-default-67664986bd-989bb\" (UID: \"d73ec31b-c54e-44df-a448-0a914b938eb7\") " pod="openshift-ingress/e2e-ipp-a-64e462-openshift-default-67664986bd-989bb"
Sep 12 00:48:52.482309 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:48:52.482250    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d73ec31b-c54e-44df-a448-0a914b938eb7-istio-token\") pod \"e2e-ipp-a-64e462-openshift-default-67664986bd-989bb\" (UID: \"d73ec31b-c54e-44df-a448-0a914b938eb7\") " pod="openshift-ingress/e2e-ipp-a-64e462-openshift-default-67664986bd-989bb"
Sep 12 00:48:52.482309 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:48:52.482285    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d73ec31b-c54e-44df-a448-0a914b938eb7-istio-envoy\") pod \"e2e-ipp-a-64e462-openshift-default-67664986bd-989bb\" (UID: \"d73ec31b-c54e-44df-a448-0a914b938eb7\") " pod="openshift-ingress/e2e-ipp-a-64e462-openshift-default-67664986bd-989bb"
Sep 12 00:48:52.482380 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:48:52.482326    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-f9g28\" (UniqueName: \"kubernetes.io/projected/d73ec31b-c54e-44df-a448-0a914b938eb7-kube-api-access-f9g28\") pod \"e2e-ipp-a-64e462-openshift-default-67664986bd-989bb\" (UID: \"d73ec31b-c54e-44df-a448-0a914b938eb7\") " pod="openshift-ingress/e2e-ipp-a-64e462-openshift-default-67664986bd-989bb"
Sep 12 00:48:52.583557 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:48:52.583520    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d73ec31b-c54e-44df-a448-0a914b938eb7-istiod-ca-cert\") pod \"e2e-ipp-a-64e462-openshift-default-67664986bd-989bb\" (UID: \"d73ec31b-c54e-44df-a448-0a914b938eb7\") " pod="openshift-ingress/e2e-ipp-a-64e462-openshift-default-67664986bd-989bb"
Sep 12 00:48:52.583771 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:48:52.583559    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d73ec31b-c54e-44df-a448-0a914b938eb7-istio-podinfo\") pod \"e2e-ipp-a-64e462-openshift-default-67664986bd-989bb\" (UID: \"d73ec31b-c54e-44df-a448-0a914b938eb7\") " pod="openshift-ingress/e2e-ipp-a-64e462-openshift-default-67664986bd-989bb"
Sep 12 00:48:52.583771 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:48:52.583630    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d73ec31b-c54e-44df-a448-0a914b938eb7-workload-certs\") pod \"e2e-ipp-a-64e462-openshift-default-67664986bd-989bb\" (UID: \"d73ec31b-c54e-44df-a448-0a914b938eb7\") " pod="openshift-ingress/e2e-ipp-a-64e462-openshift-default-67664986bd-989bb"
Sep 12 00:48:52.583771 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:48:52.583654    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d73ec31b-c54e-44df-a448-0a914b938eb7-istio-data\") pod \"e2e-ipp-a-64e462-openshift-default-67664986bd-989bb\" (UID: \"d73ec31b-c54e-44df-a448-0a914b938eb7\") " pod="openshift-ingress/e2e-ipp-a-64e462-openshift-default-67664986bd-989bb"
Sep 12 00:48:52.583940 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:48:52.583809    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d73ec31b-c54e-44df-a448-0a914b938eb7-istio-token\") pod \"e2e-ipp-a-64e462-openshift-default-67664986bd-989bb\" (UID: \"d73ec31b-c54e-44df-a448-0a914b938eb7\") " pod="openshift-ingress/e2e-ipp-a-64e462-openshift-default-67664986bd-989bb"
Sep 12 00:48:52.583940 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:48:52.583854    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d73ec31b-c54e-44df-a448-0a914b938eb7-istio-envoy\") pod \"e2e-ipp-a-64e462-openshift-default-67664986bd-989bb\" (UID: \"d73ec31b-c54e-44df-a448-0a914b938eb7\") " pod="openshift-ingress/e2e-ipp-a-64e462-openshift-default-67664986bd-989bb"
Sep 12 00:48:52.583940 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:48:52.583905    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-f9g28\" (UniqueName: \"kubernetes.io/projected/d73ec31b-c54e-44df-a448-0a914b938eb7-kube-api-access-f9g28\") pod \"e2e-ipp-a-64e462-openshift-default-67664986bd-989bb\" (UID: \"d73ec31b-c54e-44df-a448-0a914b938eb7\") " pod="openshift-ingress/e2e-ipp-a-64e462-openshift-default-67664986bd-989bb"
Sep 12 00:48:52.584106 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:48:52.583964    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d73ec31b-c54e-44df-a448-0a914b938eb7-credential-socket\") pod \"e2e-ipp-a-64e462-openshift-default-67664986bd-989bb\" (UID: \"d73ec31b-c54e-44df-a448-0a914b938eb7\") " pod="openshift-ingress/e2e-ipp-a-64e462-openshift-default-67664986bd-989bb"
Sep 12 00:48:52.584106 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:48:52.584011    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d73ec31b-c54e-44df-a448-0a914b938eb7-workload-socket\") pod \"e2e-ipp-a-64e462-openshift-default-67664986bd-989bb\" (UID: \"d73ec31b-c54e-44df-a448-0a914b938eb7\") " pod="openshift-ingress/e2e-ipp-a-64e462-openshift-default-67664986bd-989bb"
Sep 12 00:48:52.584106 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:48:52.584053    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d73ec31b-c54e-44df-a448-0a914b938eb7-workload-certs\") pod \"e2e-ipp-a-64e462-openshift-default-67664986bd-989bb\" (UID: \"d73ec31b-c54e-44df-a448-0a914b938eb7\") " pod="openshift-ingress/e2e-ipp-a-64e462-openshift-default-67664986bd-989bb"
Sep 12 00:48:52.584106 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:48:52.584093    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d73ec31b-c54e-44df-a448-0a914b938eb7-istio-data\") pod \"e2e-ipp-a-64e462-openshift-default-67664986bd-989bb\" (UID: \"d73ec31b-c54e-44df-a448-0a914b938eb7\") " pod="openshift-ingress/e2e-ipp-a-64e462-openshift-default-67664986bd-989bb"
Sep 12 00:48:52.584359 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:48:52.584336    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d73ec31b-c54e-44df-a448-0a914b938eb7-workload-socket\") pod \"e2e-ipp-a-64e462-openshift-default-67664986bd-989bb\" (UID: \"d73ec31b-c54e-44df-a448-0a914b938eb7\") " pod="openshift-ingress/e2e-ipp-a-64e462-openshift-default-67664986bd-989bb"
Sep 12 00:48:52.584436 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:48:52.584415    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d73ec31b-c54e-44df-a448-0a914b938eb7-credential-socket\") pod \"e2e-ipp-a-64e462-openshift-default-67664986bd-989bb\" (UID: \"d73ec31b-c54e-44df-a448-0a914b938eb7\") " pod="openshift-ingress/e2e-ipp-a-64e462-openshift-default-67664986bd-989bb"
Sep 12 00:48:52.584684 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:48:52.584667    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d73ec31b-c54e-44df-a448-0a914b938eb7-istiod-ca-cert\") pod \"e2e-ipp-a-64e462-openshift-default-67664986bd-989bb\" (UID: \"d73ec31b-c54e-44df-a448-0a914b938eb7\") " pod="openshift-ingress/e2e-ipp-a-64e462-openshift-default-67664986bd-989bb"
Sep 12 00:48:52.586024 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:48:52.586002    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d73ec31b-c54e-44df-a448-0a914b938eb7-istio-envoy\") pod \"e2e-ipp-a-64e462-openshift-default-67664986bd-989bb\" (UID: \"d73ec31b-c54e-44df-a448-0a914b938eb7\") " pod="openshift-ingress/e2e-ipp-a-64e462-openshift-default-67664986bd-989bb"
Sep 12 00:48:52.586214 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:48:52.586199    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d73ec31b-c54e-44df-a448-0a914b938eb7-istio-podinfo\") pod \"e2e-ipp-a-64e462-openshift-default-67664986bd-989bb\" (UID: \"d73ec31b-c54e-44df-a448-0a914b938eb7\") " pod="openshift-ingress/e2e-ipp-a-64e462-openshift-default-67664986bd-989bb"
Sep 12 00:48:52.591482 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:48:52.591459    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d73ec31b-c54e-44df-a448-0a914b938eb7-istio-token\") pod \"e2e-ipp-a-64e462-openshift-default-67664986bd-989bb\" (UID: \"d73ec31b-c54e-44df-a448-0a914b938eb7\") " pod="openshift-ingress/e2e-ipp-a-64e462-openshift-default-67664986bd-989bb"
Sep 12 00:48:52.591632 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:48:52.591545    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-f9g28\" (UniqueName: \"kubernetes.io/projected/d73ec31b-c54e-44df-a448-0a914b938eb7-kube-api-access-f9g28\") pod \"e2e-ipp-a-64e462-openshift-default-67664986bd-989bb\" (UID: \"d73ec31b-c54e-44df-a448-0a914b938eb7\") " pod="openshift-ingress/e2e-ipp-a-64e462-openshift-default-67664986bd-989bb"
Sep 12 00:48:52.694938 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:48:52.694913    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-a-64e462-openshift-default-67664986bd-989bb"
Sep 12 00:48:52.971671 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:48:52.971646    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ipp-a-64e462-openshift-default-67664986bd-989bb"]
Sep 12 00:48:52.973497 ip-10-0-141-128 kubenswrapper[2571]: W0912 00:48:52.973462    2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd73ec31b_c54e_44df_a448_0a914b938eb7.slice/crio-f6351ca2f286d529a30f4be64461ca5b9e91f8dade33feac80b82aa5b248f5f6 WatchSource:0}: Error finding container f6351ca2f286d529a30f4be64461ca5b9e91f8dade33feac80b82aa5b248f5f6: Status 404 returned error can't find the container with id f6351ca2f286d529a30f4be64461ca5b9e91f8dade33feac80b82aa5b248f5f6
Sep 12 00:48:52.975933 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:48:52.975903    2571 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236196Ki","pods":"250"}
Sep 12 00:48:52.976013 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:48:52.975967    2571 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236196Ki","pods":"250"}
Sep 12 00:48:52.976013 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:48:52.975997    2571 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236196Ki","pods":"250"}
Sep 12 00:48:53.810528 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:48:53.810491    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-a-64e462-openshift-default-67664986bd-989bb" event={"ID":"d73ec31b-c54e-44df-a448-0a914b938eb7","Type":"ContainerStarted","Data":"878068f959d49e38a7baed0752277da5d96b84be6efbcde359bb40b6f856dbf5"}
Sep 12 00:48:53.811005 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:48:53.810546    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-a-64e462-openshift-default-67664986bd-989bb" event={"ID":"d73ec31b-c54e-44df-a448-0a914b938eb7","Type":"ContainerStarted","Data":"f6351ca2f286d529a30f4be64461ca5b9e91f8dade33feac80b82aa5b248f5f6"}
Sep 12 00:48:53.832117 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:48:53.832069    2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-ipp-a-64e462-openshift-default-67664986bd-989bb" podStartSLOduration=1.83205408 podStartE2EDuration="1.83205408s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-12 00:48:52 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-12 00:48:53.829167289 +0000 UTC m=+1945.100807801" watchObservedRunningTime="2026-09-12 00:48:53.83205408 +0000 UTC m=+1945.103694636"
Sep 12 00:48:54.695809 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:48:54.695780    2571 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ipp-a-64e462-openshift-default-67664986bd-989bb"
Sep 12 00:48:54.700706 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:48:54.700681    2571 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ipp-a-64e462-openshift-default-67664986bd-989bb"
Sep 12 00:48:54.816954 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:48:54.816924    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ipp-a-64e462-openshift-default-67664986bd-989bb"
Sep 12 00:48:54.818015 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:48:54.817997    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ipp-a-64e462-openshift-default-67664986bd-989bb"
Sep 12 00:49:10.857739 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:49:10.857698    2571 patch_prober.go:28] interesting pod/e2e-ipp-a-64e462-openshift-default-67664986bd-989bb container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.71:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Sep 12 00:49:10.858212 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:49:10.857769    2571 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-ipp-a-64e462-openshift-default-67664986bd-989bb" podUID="d73ec31b-c54e-44df-a448-0a914b938eb7" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.71:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Sep 12 00:49:11.912423 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:49:11.912385    2571 generic.go:356] "Generic (PLEG): container finished" podID="d73ec31b-c54e-44df-a448-0a914b938eb7" containerID="878068f959d49e38a7baed0752277da5d96b84be6efbcde359bb40b6f856dbf5" exitCode=0
Sep 12 00:49:11.912895 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:49:11.912460    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-a-64e462-openshift-default-67664986bd-989bb" event={"ID":"d73ec31b-c54e-44df-a448-0a914b938eb7","Type":"ContainerDied","Data":"878068f959d49e38a7baed0752277da5d96b84be6efbcde359bb40b6f856dbf5"}
Sep 12 00:49:11.912895 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:49:11.912872    2571 scope.go:117] "RemoveContainer" containerID="878068f959d49e38a7baed0752277da5d96b84be6efbcde359bb40b6f856dbf5"
Sep 12 00:49:11.913889 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:49:11.913859    2571 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236196Ki","pods":"250"}
Sep 12 00:49:11.914009 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:49:11.913939    2571 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236196Ki","pods":"250"}
Sep 12 00:49:11.914009 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:49:11.913985    2571 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236196Ki","pods":"250"}
Sep 12 00:49:12.922478 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:49:12.922445    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-a-64e462-openshift-default-67664986bd-989bb" event={"ID":"d73ec31b-c54e-44df-a448-0a914b938eb7","Type":"ContainerStarted","Data":"bcc049736be1b73bb6a594d4989b9dac316fe84efb477460339028bdaf2d9e43"}
Sep 12 00:49:13.695494 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:49:13.695454    2571 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ipp-a-64e462-openshift-default-67664986bd-989bb"
Sep 12 00:49:13.695701 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:49:13.695647    2571 patch_prober.go:28] interesting pod/e2e-ipp-a-64e462-openshift-default-67664986bd-989bb container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.71:15021/healthz/ready\": dial tcp 10.132.0.71:15021: connect: connection refused" start-of-body=
Sep 12 00:49:13.695750 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:49:13.695706    2571 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-a-64e462-openshift-default-67664986bd-989bb" podUID="d73ec31b-c54e-44df-a448-0a914b938eb7" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.71:15021/healthz/ready\": dial tcp 10.132.0.71:15021: connect: connection refused"
Sep 12 00:49:14.695977 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:49:14.695943    2571 patch_prober.go:28] interesting pod/e2e-ipp-a-64e462-openshift-default-67664986bd-989bb container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.71:15021/healthz/ready\": dial tcp 10.132.0.71:15021: connect: connection refused" start-of-body=
Sep 12 00:49:14.696428 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:49:14.696016    2571 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-a-64e462-openshift-default-67664986bd-989bb" podUID="d73ec31b-c54e-44df-a448-0a914b938eb7" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.71:15021/healthz/ready\": dial tcp 10.132.0.71:15021: connect: connection refused"
Sep 12 00:49:15.695261 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:49:15.695224    2571 patch_prober.go:28] interesting pod/e2e-ipp-a-64e462-openshift-default-67664986bd-989bb container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.71:15021/healthz/ready\": dial tcp 10.132.0.71:15021: connect: connection refused" start-of-body=
Sep 12 00:49:15.695447 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:49:15.695274    2571 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-a-64e462-openshift-default-67664986bd-989bb" podUID="d73ec31b-c54e-44df-a448-0a914b938eb7" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.71:15021/healthz/ready\": dial tcp 10.132.0.71:15021: connect: connection refused"
Sep 12 00:49:16.695543 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:49:16.695506    2571 patch_prober.go:28] interesting pod/e2e-ipp-a-64e462-openshift-default-67664986bd-989bb container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.71:15021/healthz/ready\": dial tcp 10.132.0.71:15021: connect: connection refused" start-of-body=
Sep 12 00:49:16.695930 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:49:16.695561    2571 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-a-64e462-openshift-default-67664986bd-989bb" podUID="d73ec31b-c54e-44df-a448-0a914b938eb7" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.71:15021/healthz/ready\": dial tcp 10.132.0.71:15021: connect: connection refused"
Sep 12 00:49:17.695775 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:49:17.695738    2571 patch_prober.go:28] interesting pod/e2e-ipp-a-64e462-openshift-default-67664986bd-989bb container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.71:15021/healthz/ready\": dial tcp 10.132.0.71:15021: connect: connection refused" start-of-body=
Sep 12 00:49:17.696160 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:49:17.695796    2571 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-a-64e462-openshift-default-67664986bd-989bb" podUID="d73ec31b-c54e-44df-a448-0a914b938eb7" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.71:15021/healthz/ready\": dial tcp 10.132.0.71:15021: connect: connection refused"
Sep 12 00:49:18.699238 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:49:18.699208    2571 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ipp-a-64e462-openshift-default-67664986bd-989bb"
Sep 12 00:49:18.699631 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:49:18.699568    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ipp-a-64e462-openshift-default-67664986bd-989bb"
Sep 12 00:49:18.700259 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:49:18.700242    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ipp-a-64e462-openshift-default-67664986bd-989bb"
Sep 12 00:49:21.818920 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:49:21.818886    2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ipp-b-1cbe1c-openshift-default-5fc97d867d-9hcqb"]
Sep 12 00:49:21.819405 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:49:21.819391    2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="537eb92d-71ea-4651-a204-fee3a3728ace" containerName="istio-proxy"
Sep 12 00:49:21.819479 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:49:21.819407    2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="d7e353f5-8ba3-41a4-bd94-5e2c0cf9fa46" containerName="istio-proxy"
Sep 12 00:49:21.821474 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:49:21.821453    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-b-1cbe1c-openshift-default-5fc97d867d-9hcqb"
Sep 12 00:49:21.824634 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:49:21.824613    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ipp-b-1cbe1c-openshift-default-dockercfg-985jt\""
Sep 12 00:49:21.834905 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:49:21.834878    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ipp-b-1cbe1c-openshift-default-5fc97d867d-9hcqb"]
Sep 12 00:49:21.871624 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:49:21.871572    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3c451460-9cc7-49b0-ab4f-f361805f15d9-workload-socket\") pod \"e2e-ipp-b-1cbe1c-openshift-default-5fc97d867d-9hcqb\" (UID: \"3c451460-9cc7-49b0-ab4f-f361805f15d9\") " pod="openshift-ingress/e2e-ipp-b-1cbe1c-openshift-default-5fc97d867d-9hcqb"
Sep 12 00:49:21.871738 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:49:21.871646    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3c451460-9cc7-49b0-ab4f-f361805f15d9-istio-envoy\") pod \"e2e-ipp-b-1cbe1c-openshift-default-5fc97d867d-9hcqb\" (UID: \"3c451460-9cc7-49b0-ab4f-f361805f15d9\") " pod="openshift-ingress/e2e-ipp-b-1cbe1c-openshift-default-5fc97d867d-9hcqb"
Sep 12 00:49:21.871738 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:49:21.871684    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3c451460-9cc7-49b0-ab4f-f361805f15d9-istio-data\") pod \"e2e-ipp-b-1cbe1c-openshift-default-5fc97d867d-9hcqb\" (UID: \"3c451460-9cc7-49b0-ab4f-f361805f15d9\") " pod="openshift-ingress/e2e-ipp-b-1cbe1c-openshift-default-5fc97d867d-9hcqb"
Sep 12 00:49:21.871738 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:49:21.871709    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fchxc\" (UniqueName: \"kubernetes.io/projected/3c451460-9cc7-49b0-ab4f-f361805f15d9-kube-api-access-fchxc\") pod \"e2e-ipp-b-1cbe1c-openshift-default-5fc97d867d-9hcqb\" (UID: \"3c451460-9cc7-49b0-ab4f-f361805f15d9\") " pod="openshift-ingress/e2e-ipp-b-1cbe1c-openshift-default-5fc97d867d-9hcqb"
Sep 12 00:49:21.871871 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:49:21.871740    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3c451460-9cc7-49b0-ab4f-f361805f15d9-workload-certs\") pod \"e2e-ipp-b-1cbe1c-openshift-default-5fc97d867d-9hcqb\" (UID: \"3c451460-9cc7-49b0-ab4f-f361805f15d9\") " pod="openshift-ingress/e2e-ipp-b-1cbe1c-openshift-default-5fc97d867d-9hcqb"
Sep 12 00:49:21.871871 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:49:21.871754    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3c451460-9cc7-49b0-ab4f-f361805f15d9-istio-token\") pod \"e2e-ipp-b-1cbe1c-openshift-default-5fc97d867d-9hcqb\" (UID: \"3c451460-9cc7-49b0-ab4f-f361805f15d9\") " pod="openshift-ingress/e2e-ipp-b-1cbe1c-openshift-default-5fc97d867d-9hcqb"
Sep 12 00:49:21.871871 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:49:21.871777    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3c451460-9cc7-49b0-ab4f-f361805f15d9-credential-socket\") pod \"e2e-ipp-b-1cbe1c-openshift-default-5fc97d867d-9hcqb\" (UID: \"3c451460-9cc7-49b0-ab4f-f361805f15d9\") " pod="openshift-ingress/e2e-ipp-b-1cbe1c-openshift-default-5fc97d867d-9hcqb"
Sep 12 00:49:21.871871 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:49:21.871791    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3c451460-9cc7-49b0-ab4f-f361805f15d9-istio-podinfo\") pod \"e2e-ipp-b-1cbe1c-openshift-default-5fc97d867d-9hcqb\" (UID: \"3c451460-9cc7-49b0-ab4f-f361805f15d9\") " pod="openshift-ingress/e2e-ipp-b-1cbe1c-openshift-default-5fc97d867d-9hcqb"
Sep 12 00:49:21.871996 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:49:21.871879    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3c451460-9cc7-49b0-ab4f-f361805f15d9-istiod-ca-cert\") pod \"e2e-ipp-b-1cbe1c-openshift-default-5fc97d867d-9hcqb\" (UID: \"3c451460-9cc7-49b0-ab4f-f361805f15d9\") " pod="openshift-ingress/e2e-ipp-b-1cbe1c-openshift-default-5fc97d867d-9hcqb"
Sep 12 00:49:21.972964 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:49:21.972881    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3c451460-9cc7-49b0-ab4f-f361805f15d9-credential-socket\") pod \"e2e-ipp-b-1cbe1c-openshift-default-5fc97d867d-9hcqb\" (UID: \"3c451460-9cc7-49b0-ab4f-f361805f15d9\") " pod="openshift-ingress/e2e-ipp-b-1cbe1c-openshift-default-5fc97d867d-9hcqb"
Sep 12 00:49:21.972964 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:49:21.972919    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3c451460-9cc7-49b0-ab4f-f361805f15d9-istio-podinfo\") pod \"e2e-ipp-b-1cbe1c-openshift-default-5fc97d867d-9hcqb\" (UID: \"3c451460-9cc7-49b0-ab4f-f361805f15d9\") " pod="openshift-ingress/e2e-ipp-b-1cbe1c-openshift-default-5fc97d867d-9hcqb"
Sep 12 00:49:21.973192 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:49:21.972962    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3c451460-9cc7-49b0-ab4f-f361805f15d9-istiod-ca-cert\") pod \"e2e-ipp-b-1cbe1c-openshift-default-5fc97d867d-9hcqb\" (UID: \"3c451460-9cc7-49b0-ab4f-f361805f15d9\") " pod="openshift-ingress/e2e-ipp-b-1cbe1c-openshift-default-5fc97d867d-9hcqb"
Sep 12 00:49:21.973192 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:49:21.973028    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3c451460-9cc7-49b0-ab4f-f361805f15d9-workload-socket\") pod \"e2e-ipp-b-1cbe1c-openshift-default-5fc97d867d-9hcqb\" (UID: \"3c451460-9cc7-49b0-ab4f-f361805f15d9\") " pod="openshift-ingress/e2e-ipp-b-1cbe1c-openshift-default-5fc97d867d-9hcqb"
Sep 12 00:49:21.973192 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:49:21.973073    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3c451460-9cc7-49b0-ab4f-f361805f15d9-istio-envoy\") pod \"e2e-ipp-b-1cbe1c-openshift-default-5fc97d867d-9hcqb\" (UID: \"3c451460-9cc7-49b0-ab4f-f361805f15d9\") " pod="openshift-ingress/e2e-ipp-b-1cbe1c-openshift-default-5fc97d867d-9hcqb"
Sep 12 00:49:21.973192 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:49:21.973124    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3c451460-9cc7-49b0-ab4f-f361805f15d9-istio-data\") pod \"e2e-ipp-b-1cbe1c-openshift-default-5fc97d867d-9hcqb\" (UID: \"3c451460-9cc7-49b0-ab4f-f361805f15d9\") " pod="openshift-ingress/e2e-ipp-b-1cbe1c-openshift-default-5fc97d867d-9hcqb"
Sep 12 00:49:21.973192 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:49:21.973157    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fchxc\" (UniqueName: \"kubernetes.io/projected/3c451460-9cc7-49b0-ab4f-f361805f15d9-kube-api-access-fchxc\") pod \"e2e-ipp-b-1cbe1c-openshift-default-5fc97d867d-9hcqb\" (UID: \"3c451460-9cc7-49b0-ab4f-f361805f15d9\") " pod="openshift-ingress/e2e-ipp-b-1cbe1c-openshift-default-5fc97d867d-9hcqb"
Sep 12 00:49:21.973461 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:49:21.973206    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3c451460-9cc7-49b0-ab4f-f361805f15d9-workload-certs\") pod \"e2e-ipp-b-1cbe1c-openshift-default-5fc97d867d-9hcqb\" (UID: \"3c451460-9cc7-49b0-ab4f-f361805f15d9\") " pod="openshift-ingress/e2e-ipp-b-1cbe1c-openshift-default-5fc97d867d-9hcqb"
Sep 12 00:49:21.973461 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:49:21.973403    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3c451460-9cc7-49b0-ab4f-f361805f15d9-credential-socket\") pod \"e2e-ipp-b-1cbe1c-openshift-default-5fc97d867d-9hcqb\" (UID: \"3c451460-9cc7-49b0-ab4f-f361805f15d9\") " pod="openshift-ingress/e2e-ipp-b-1cbe1c-openshift-default-5fc97d867d-9hcqb"
Sep 12 00:49:21.973461 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:49:21.973445    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3c451460-9cc7-49b0-ab4f-f361805f15d9-workload-socket\") pod \"e2e-ipp-b-1cbe1c-openshift-default-5fc97d867d-9hcqb\" (UID: \"3c451460-9cc7-49b0-ab4f-f361805f15d9\") " pod="openshift-ingress/e2e-ipp-b-1cbe1c-openshift-default-5fc97d867d-9hcqb"
Sep 12 00:49:21.973461 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:49:21.973457    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3c451460-9cc7-49b0-ab4f-f361805f15d9-istio-token\") pod \"e2e-ipp-b-1cbe1c-openshift-default-5fc97d867d-9hcqb\" (UID: \"3c451460-9cc7-49b0-ab4f-f361805f15d9\") " pod="openshift-ingress/e2e-ipp-b-1cbe1c-openshift-default-5fc97d867d-9hcqb"
Sep 12 00:49:21.973698 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:49:21.973528    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3c451460-9cc7-49b0-ab4f-f361805f15d9-workload-certs\") pod \"e2e-ipp-b-1cbe1c-openshift-default-5fc97d867d-9hcqb\" (UID: \"3c451460-9cc7-49b0-ab4f-f361805f15d9\") " pod="openshift-ingress/e2e-ipp-b-1cbe1c-openshift-default-5fc97d867d-9hcqb"
Sep 12 00:49:21.973698 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:49:21.973527    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3c451460-9cc7-49b0-ab4f-f361805f15d9-istio-data\") pod \"e2e-ipp-b-1cbe1c-openshift-default-5fc97d867d-9hcqb\" (UID: \"3c451460-9cc7-49b0-ab4f-f361805f15d9\") " pod="openshift-ingress/e2e-ipp-b-1cbe1c-openshift-default-5fc97d867d-9hcqb"
Sep 12 00:49:21.973943 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:49:21.973926    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3c451460-9cc7-49b0-ab4f-f361805f15d9-istiod-ca-cert\") pod \"e2e-ipp-b-1cbe1c-openshift-default-5fc97d867d-9hcqb\" (UID: \"3c451460-9cc7-49b0-ab4f-f361805f15d9\") " pod="openshift-ingress/e2e-ipp-b-1cbe1c-openshift-default-5fc97d867d-9hcqb"
Sep 12 00:49:21.975663 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:49:21.975639    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3c451460-9cc7-49b0-ab4f-f361805f15d9-istio-podinfo\") pod \"e2e-ipp-b-1cbe1c-openshift-default-5fc97d867d-9hcqb\" (UID: \"3c451460-9cc7-49b0-ab4f-f361805f15d9\") " pod="openshift-ingress/e2e-ipp-b-1cbe1c-openshift-default-5fc97d867d-9hcqb"
Sep 12 00:49:21.975941 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:49:21.975920    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3c451460-9cc7-49b0-ab4f-f361805f15d9-istio-envoy\") pod \"e2e-ipp-b-1cbe1c-openshift-default-5fc97d867d-9hcqb\" (UID: \"3c451460-9cc7-49b0-ab4f-f361805f15d9\") " pod="openshift-ingress/e2e-ipp-b-1cbe1c-openshift-default-5fc97d867d-9hcqb"
Sep 12 00:49:21.981512 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:49:21.981483    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fchxc\" (UniqueName: \"kubernetes.io/projected/3c451460-9cc7-49b0-ab4f-f361805f15d9-kube-api-access-fchxc\") pod \"e2e-ipp-b-1cbe1c-openshift-default-5fc97d867d-9hcqb\" (UID: \"3c451460-9cc7-49b0-ab4f-f361805f15d9\") " pod="openshift-ingress/e2e-ipp-b-1cbe1c-openshift-default-5fc97d867d-9hcqb"
Sep 12 00:49:21.981638 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:49:21.981548    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3c451460-9cc7-49b0-ab4f-f361805f15d9-istio-token\") pod \"e2e-ipp-b-1cbe1c-openshift-default-5fc97d867d-9hcqb\" (UID: \"3c451460-9cc7-49b0-ab4f-f361805f15d9\") " pod="openshift-ingress/e2e-ipp-b-1cbe1c-openshift-default-5fc97d867d-9hcqb"
Sep 12 00:49:22.133837 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:49:22.133763    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-b-1cbe1c-openshift-default-5fc97d867d-9hcqb"
Sep 12 00:49:22.406757 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:49:22.406732    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ipp-b-1cbe1c-openshift-default-5fc97d867d-9hcqb"]
Sep 12 00:49:22.408171 ip-10-0-141-128 kubenswrapper[2571]: W0912 00:49:22.408146    2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3c451460_9cc7_49b0_ab4f_f361805f15d9.slice/crio-6a0bafe7f1dd957586465cf2c986de45a88b4e26f1d71b2fdcad7f1815def562 WatchSource:0}: Error finding container 6a0bafe7f1dd957586465cf2c986de45a88b4e26f1d71b2fdcad7f1815def562: Status 404 returned error can't find the container with id 6a0bafe7f1dd957586465cf2c986de45a88b4e26f1d71b2fdcad7f1815def562
Sep 12 00:49:22.410454 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:49:22.410420    2571 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236196Ki","pods":"250"}
Sep 12 00:49:22.410536 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:49:22.410480    2571 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236196Ki","pods":"250"}
Sep 12 00:49:22.410536 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:49:22.410510    2571 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236196Ki","pods":"250"}
Sep 12 00:49:22.975370 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:49:22.975326    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-b-1cbe1c-openshift-default-5fc97d867d-9hcqb" event={"ID":"3c451460-9cc7-49b0-ab4f-f361805f15d9","Type":"ContainerStarted","Data":"30e7b187ab20caa4d639899024e8aa5eff94c12dcc7101fefac51bd7eeaf9589"}
Sep 12 00:49:22.975742 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:49:22.975385    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-b-1cbe1c-openshift-default-5fc97d867d-9hcqb" event={"ID":"3c451460-9cc7-49b0-ab4f-f361805f15d9","Type":"ContainerStarted","Data":"6a0bafe7f1dd957586465cf2c986de45a88b4e26f1d71b2fdcad7f1815def562"}
Sep 12 00:49:22.996741 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:49:22.996694    2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-ipp-b-1cbe1c-openshift-default-5fc97d867d-9hcqb" podStartSLOduration=1.9966786380000001 podStartE2EDuration="1.996678638s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-12 00:49:21 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-12 00:49:22.995268262 +0000 UTC m=+1974.266908795" watchObservedRunningTime="2026-09-12 00:49:22.996678638 +0000 UTC m=+1974.268319146"
Sep 12 00:49:23.134574 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:49:23.134543    2571 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ipp-b-1cbe1c-openshift-default-5fc97d867d-9hcqb"
Sep 12 00:49:23.142722 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:49:23.142699    2571 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ipp-b-1cbe1c-openshift-default-5fc97d867d-9hcqb"
Sep 12 00:49:23.981127 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:49:23.981097    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ipp-b-1cbe1c-openshift-default-5fc97d867d-9hcqb"
Sep 12 00:49:23.982255 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:49:23.982234    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ipp-b-1cbe1c-openshift-default-5fc97d867d-9hcqb"
Sep 12 00:49:43.079001 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:49:43.078972    2571 generic.go:356] "Generic (PLEG): container finished" podID="3c451460-9cc7-49b0-ab4f-f361805f15d9" containerID="30e7b187ab20caa4d639899024e8aa5eff94c12dcc7101fefac51bd7eeaf9589" exitCode=0
Sep 12 00:49:43.079428 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:49:43.079058    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-b-1cbe1c-openshift-default-5fc97d867d-9hcqb" event={"ID":"3c451460-9cc7-49b0-ab4f-f361805f15d9","Type":"ContainerDied","Data":"30e7b187ab20caa4d639899024e8aa5eff94c12dcc7101fefac51bd7eeaf9589"}
Sep 12 00:49:43.079473 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:49:43.079447    2571 scope.go:117] "RemoveContainer" containerID="30e7b187ab20caa4d639899024e8aa5eff94c12dcc7101fefac51bd7eeaf9589"
Sep 12 00:49:43.080514 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:49:43.080488    2571 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236196Ki","pods":"250"}
Sep 12 00:49:43.080563 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:49:43.080554    2571 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236196Ki","pods":"250"}
Sep 12 00:49:43.080609 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:49:43.080597    2571 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236196Ki","pods":"250"}
Sep 12 00:49:44.089376 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:49:44.089344    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-b-1cbe1c-openshift-default-5fc97d867d-9hcqb" event={"ID":"3c451460-9cc7-49b0-ab4f-f361805f15d9","Type":"ContainerStarted","Data":"a6e83b7404c55c45e080af62c7987d2934007fca1b332efcf5d42d9729f7d9d1"}
Sep 12 00:49:44.134279 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:49:44.134250    2571 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ipp-b-1cbe1c-openshift-default-5fc97d867d-9hcqb"
Sep 12 00:49:44.134473 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:49:44.134453    2571 patch_prober.go:28] interesting pod/e2e-ipp-b-1cbe1c-openshift-default-5fc97d867d-9hcqb container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.72:15021/healthz/ready\": dial tcp 10.132.0.72:15021: connect: connection refused" start-of-body=
Sep 12 00:49:44.134533 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:49:44.134497    2571 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-b-1cbe1c-openshift-default-5fc97d867d-9hcqb" podUID="3c451460-9cc7-49b0-ab4f-f361805f15d9" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.72:15021/healthz/ready\": dial tcp 10.132.0.72:15021: connect: connection refused"
Sep 12 00:49:45.134733 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:49:45.134698    2571 patch_prober.go:28] interesting pod/e2e-ipp-b-1cbe1c-openshift-default-5fc97d867d-9hcqb container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.72:15021/healthz/ready\": dial tcp 10.132.0.72:15021: connect: connection refused" start-of-body=
Sep 12 00:49:45.135110 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:49:45.134750    2571 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-b-1cbe1c-openshift-default-5fc97d867d-9hcqb" podUID="3c451460-9cc7-49b0-ab4f-f361805f15d9" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.72:15021/healthz/ready\": dial tcp 10.132.0.72:15021: connect: connection refused"
Sep 12 00:49:46.134734 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:49:46.134697    2571 patch_prober.go:28] interesting pod/e2e-ipp-b-1cbe1c-openshift-default-5fc97d867d-9hcqb container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.72:15021/healthz/ready\": dial tcp 10.132.0.72:15021: connect: connection refused" start-of-body=
Sep 12 00:49:46.135206 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:49:46.134768    2571 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-b-1cbe1c-openshift-default-5fc97d867d-9hcqb" podUID="3c451460-9cc7-49b0-ab4f-f361805f15d9" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.72:15021/healthz/ready\": dial tcp 10.132.0.72:15021: connect: connection refused"
Sep 12 00:49:47.134566 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:49:47.134525    2571 patch_prober.go:28] interesting pod/e2e-ipp-b-1cbe1c-openshift-default-5fc97d867d-9hcqb container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.72:15021/healthz/ready\": dial tcp 10.132.0.72:15021: connect: connection refused" start-of-body=
Sep 12 00:49:47.134771 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:49:47.134624    2571 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-b-1cbe1c-openshift-default-5fc97d867d-9hcqb" podUID="3c451460-9cc7-49b0-ab4f-f361805f15d9" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.72:15021/healthz/ready\": dial tcp 10.132.0.72:15021: connect: connection refused"
Sep 12 00:49:48.134576 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:49:48.134543    2571 patch_prober.go:28] interesting pod/e2e-ipp-b-1cbe1c-openshift-default-5fc97d867d-9hcqb container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.72:15021/healthz/ready\": dial tcp 10.132.0.72:15021: connect: connection refused" start-of-body=
Sep 12 00:49:48.134760 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:49:48.134609    2571 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-b-1cbe1c-openshift-default-5fc97d867d-9hcqb" podUID="3c451460-9cc7-49b0-ab4f-f361805f15d9" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.72:15021/healthz/ready\": dial tcp 10.132.0.72:15021: connect: connection refused"
Sep 12 00:49:49.134391 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:49:49.134353    2571 patch_prober.go:28] interesting pod/e2e-ipp-b-1cbe1c-openshift-default-5fc97d867d-9hcqb container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.72:15021/healthz/ready\": dial tcp 10.132.0.72:15021: connect: connection refused" start-of-body=
Sep 12 00:49:49.134779 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:49:49.134419    2571 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-b-1cbe1c-openshift-default-5fc97d867d-9hcqb" podUID="3c451460-9cc7-49b0-ab4f-f361805f15d9" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.72:15021/healthz/ready\": dial tcp 10.132.0.72:15021: connect: connection refused"
Sep 12 00:49:50.138463 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:49:50.138428    2571 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ipp-b-1cbe1c-openshift-default-5fc97d867d-9hcqb"
Sep 12 00:49:50.138857 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:49:50.138762    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ipp-b-1cbe1c-openshift-default-5fc97d867d-9hcqb"
Sep 12 00:49:50.139505 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:49:50.139477    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ipp-b-1cbe1c-openshift-default-5fc97d867d-9hcqb"
Sep 12 00:49:55.101540 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:49:55.101511    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5b54b7fbb7-btmpn_817b7aba-94b1-4306-b26a-027e1e0b7297/payload-processing/0.log"
Sep 12 00:50:46.704612 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:50:46.704559    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5b54b7fbb7-btmpn_817b7aba-94b1-4306-b26a-027e1e0b7297/payload-processing/0.log"
Sep 12 00:50:47.396690 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:50:47.396637    2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ipp-cleanup-b4b570-openshift-default-86b7f9547b-5pgkk"]
Sep 12 00:50:47.401087 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:50:47.401060    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-cleanup-b4b570-openshift-default-86b7f9547b-5pgkk"
Sep 12 00:50:47.403480 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:50:47.403456    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ipp-cleanup-b4b570-openshift-default-dockercfg-mb2gv\""
Sep 12 00:50:47.411197 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:50:47.411171    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ipp-cleanup-b4b570-openshift-default-86b7f9547b-5pgkk"]
Sep 12 00:50:47.477910 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:50:47.477869    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ee555fcf-4c74-4b63-ad98-e9d5c4976697-istio-podinfo\") pod \"e2e-ipp-cleanup-b4b570-openshift-default-86b7f9547b-5pgkk\" (UID: \"ee555fcf-4c74-4b63-ad98-e9d5c4976697\") " pod="openshift-ingress/e2e-ipp-cleanup-b4b570-openshift-default-86b7f9547b-5pgkk"
Sep 12 00:50:47.478082 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:50:47.477915    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ee555fcf-4c74-4b63-ad98-e9d5c4976697-istio-token\") pod \"e2e-ipp-cleanup-b4b570-openshift-default-86b7f9547b-5pgkk\" (UID: \"ee555fcf-4c74-4b63-ad98-e9d5c4976697\") " pod="openshift-ingress/e2e-ipp-cleanup-b4b570-openshift-default-86b7f9547b-5pgkk"
Sep 12 00:50:47.478082 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:50:47.478016    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ee555fcf-4c74-4b63-ad98-e9d5c4976697-istio-envoy\") pod \"e2e-ipp-cleanup-b4b570-openshift-default-86b7f9547b-5pgkk\" (UID: \"ee555fcf-4c74-4b63-ad98-e9d5c4976697\") " pod="openshift-ingress/e2e-ipp-cleanup-b4b570-openshift-default-86b7f9547b-5pgkk"
Sep 12 00:50:47.478082 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:50:47.478059    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ee555fcf-4c74-4b63-ad98-e9d5c4976697-istiod-ca-cert\") pod \"e2e-ipp-cleanup-b4b570-openshift-default-86b7f9547b-5pgkk\" (UID: \"ee555fcf-4c74-4b63-ad98-e9d5c4976697\") " pod="openshift-ingress/e2e-ipp-cleanup-b4b570-openshift-default-86b7f9547b-5pgkk"
Sep 12 00:50:47.478210 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:50:47.478112    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tk4r4\" (UniqueName: \"kubernetes.io/projected/ee555fcf-4c74-4b63-ad98-e9d5c4976697-kube-api-access-tk4r4\") pod \"e2e-ipp-cleanup-b4b570-openshift-default-86b7f9547b-5pgkk\" (UID: \"ee555fcf-4c74-4b63-ad98-e9d5c4976697\") " pod="openshift-ingress/e2e-ipp-cleanup-b4b570-openshift-default-86b7f9547b-5pgkk"
Sep 12 00:50:47.478210 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:50:47.478183    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ee555fcf-4c74-4b63-ad98-e9d5c4976697-istio-data\") pod \"e2e-ipp-cleanup-b4b570-openshift-default-86b7f9547b-5pgkk\" (UID: \"ee555fcf-4c74-4b63-ad98-e9d5c4976697\") " pod="openshift-ingress/e2e-ipp-cleanup-b4b570-openshift-default-86b7f9547b-5pgkk"
Sep 12 00:50:47.478284 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:50:47.478254    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ee555fcf-4c74-4b63-ad98-e9d5c4976697-workload-certs\") pod \"e2e-ipp-cleanup-b4b570-openshift-default-86b7f9547b-5pgkk\" (UID: \"ee555fcf-4c74-4b63-ad98-e9d5c4976697\") " pod="openshift-ingress/e2e-ipp-cleanup-b4b570-openshift-default-86b7f9547b-5pgkk"
Sep 12 00:50:47.478332 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:50:47.478298    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ee555fcf-4c74-4b63-ad98-e9d5c4976697-credential-socket\") pod \"e2e-ipp-cleanup-b4b570-openshift-default-86b7f9547b-5pgkk\" (UID: \"ee555fcf-4c74-4b63-ad98-e9d5c4976697\") " pod="openshift-ingress/e2e-ipp-cleanup-b4b570-openshift-default-86b7f9547b-5pgkk"
Sep 12 00:50:47.478369 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:50:47.478335    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ee555fcf-4c74-4b63-ad98-e9d5c4976697-workload-socket\") pod \"e2e-ipp-cleanup-b4b570-openshift-default-86b7f9547b-5pgkk\" (UID: \"ee555fcf-4c74-4b63-ad98-e9d5c4976697\") " pod="openshift-ingress/e2e-ipp-cleanup-b4b570-openshift-default-86b7f9547b-5pgkk"
Sep 12 00:50:47.579797 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:50:47.579760    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ee555fcf-4c74-4b63-ad98-e9d5c4976697-istio-podinfo\") pod \"e2e-ipp-cleanup-b4b570-openshift-default-86b7f9547b-5pgkk\" (UID: \"ee555fcf-4c74-4b63-ad98-e9d5c4976697\") " pod="openshift-ingress/e2e-ipp-cleanup-b4b570-openshift-default-86b7f9547b-5pgkk"
Sep 12 00:50:47.579797 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:50:47.579802    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ee555fcf-4c74-4b63-ad98-e9d5c4976697-istio-token\") pod \"e2e-ipp-cleanup-b4b570-openshift-default-86b7f9547b-5pgkk\" (UID: \"ee555fcf-4c74-4b63-ad98-e9d5c4976697\") " pod="openshift-ingress/e2e-ipp-cleanup-b4b570-openshift-default-86b7f9547b-5pgkk"
Sep 12 00:50:47.580068 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:50:47.579836    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ee555fcf-4c74-4b63-ad98-e9d5c4976697-istio-envoy\") pod \"e2e-ipp-cleanup-b4b570-openshift-default-86b7f9547b-5pgkk\" (UID: \"ee555fcf-4c74-4b63-ad98-e9d5c4976697\") " pod="openshift-ingress/e2e-ipp-cleanup-b4b570-openshift-default-86b7f9547b-5pgkk"
Sep 12 00:50:47.580068 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:50:47.579857    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ee555fcf-4c74-4b63-ad98-e9d5c4976697-istiod-ca-cert\") pod \"e2e-ipp-cleanup-b4b570-openshift-default-86b7f9547b-5pgkk\" (UID: \"ee555fcf-4c74-4b63-ad98-e9d5c4976697\") " pod="openshift-ingress/e2e-ipp-cleanup-b4b570-openshift-default-86b7f9547b-5pgkk"
Sep 12 00:50:47.580068 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:50:47.579890    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tk4r4\" (UniqueName: \"kubernetes.io/projected/ee555fcf-4c74-4b63-ad98-e9d5c4976697-kube-api-access-tk4r4\") pod \"e2e-ipp-cleanup-b4b570-openshift-default-86b7f9547b-5pgkk\" (UID: \"ee555fcf-4c74-4b63-ad98-e9d5c4976697\") " pod="openshift-ingress/e2e-ipp-cleanup-b4b570-openshift-default-86b7f9547b-5pgkk"
Sep 12 00:50:47.580068 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:50:47.579939    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ee555fcf-4c74-4b63-ad98-e9d5c4976697-istio-data\") pod \"e2e-ipp-cleanup-b4b570-openshift-default-86b7f9547b-5pgkk\" (UID: \"ee555fcf-4c74-4b63-ad98-e9d5c4976697\") " pod="openshift-ingress/e2e-ipp-cleanup-b4b570-openshift-default-86b7f9547b-5pgkk"
Sep 12 00:50:47.580068 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:50:47.580005    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ee555fcf-4c74-4b63-ad98-e9d5c4976697-workload-certs\") pod \"e2e-ipp-cleanup-b4b570-openshift-default-86b7f9547b-5pgkk\" (UID: \"ee555fcf-4c74-4b63-ad98-e9d5c4976697\") " pod="openshift-ingress/e2e-ipp-cleanup-b4b570-openshift-default-86b7f9547b-5pgkk"
Sep 12 00:50:47.580068 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:50:47.580039    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ee555fcf-4c74-4b63-ad98-e9d5c4976697-credential-socket\") pod \"e2e-ipp-cleanup-b4b570-openshift-default-86b7f9547b-5pgkk\" (UID: \"ee555fcf-4c74-4b63-ad98-e9d5c4976697\") " pod="openshift-ingress/e2e-ipp-cleanup-b4b570-openshift-default-86b7f9547b-5pgkk"
Sep 12 00:50:47.580068 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:50:47.580062    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ee555fcf-4c74-4b63-ad98-e9d5c4976697-workload-socket\") pod \"e2e-ipp-cleanup-b4b570-openshift-default-86b7f9547b-5pgkk\" (UID: \"ee555fcf-4c74-4b63-ad98-e9d5c4976697\") " pod="openshift-ingress/e2e-ipp-cleanup-b4b570-openshift-default-86b7f9547b-5pgkk"
Sep 12 00:50:47.580471 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:50:47.580446    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ee555fcf-4c74-4b63-ad98-e9d5c4976697-credential-socket\") pod \"e2e-ipp-cleanup-b4b570-openshift-default-86b7f9547b-5pgkk\" (UID: \"ee555fcf-4c74-4b63-ad98-e9d5c4976697\") " pod="openshift-ingress/e2e-ipp-cleanup-b4b570-openshift-default-86b7f9547b-5pgkk"
Sep 12 00:50:47.580646 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:50:47.580623    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ee555fcf-4c74-4b63-ad98-e9d5c4976697-workload-socket\") pod \"e2e-ipp-cleanup-b4b570-openshift-default-86b7f9547b-5pgkk\" (UID: \"ee555fcf-4c74-4b63-ad98-e9d5c4976697\") " pod="openshift-ingress/e2e-ipp-cleanup-b4b570-openshift-default-86b7f9547b-5pgkk"
Sep 12 00:50:47.580784 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:50:47.580667    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ee555fcf-4c74-4b63-ad98-e9d5c4976697-workload-certs\") pod \"e2e-ipp-cleanup-b4b570-openshift-default-86b7f9547b-5pgkk\" (UID: \"ee555fcf-4c74-4b63-ad98-e9d5c4976697\") " pod="openshift-ingress/e2e-ipp-cleanup-b4b570-openshift-default-86b7f9547b-5pgkk"
Sep 12 00:50:47.581023 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:50:47.580998    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ee555fcf-4c74-4b63-ad98-e9d5c4976697-istiod-ca-cert\") pod \"e2e-ipp-cleanup-b4b570-openshift-default-86b7f9547b-5pgkk\" (UID: \"ee555fcf-4c74-4b63-ad98-e9d5c4976697\") " pod="openshift-ingress/e2e-ipp-cleanup-b4b570-openshift-default-86b7f9547b-5pgkk"
Sep 12 00:50:47.581266 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:50:47.581246    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ee555fcf-4c74-4b63-ad98-e9d5c4976697-istio-data\") pod \"e2e-ipp-cleanup-b4b570-openshift-default-86b7f9547b-5pgkk\" (UID: \"ee555fcf-4c74-4b63-ad98-e9d5c4976697\") " pod="openshift-ingress/e2e-ipp-cleanup-b4b570-openshift-default-86b7f9547b-5pgkk"
Sep 12 00:50:47.582603 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:50:47.582565    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ee555fcf-4c74-4b63-ad98-e9d5c4976697-istio-envoy\") pod \"e2e-ipp-cleanup-b4b570-openshift-default-86b7f9547b-5pgkk\" (UID: \"ee555fcf-4c74-4b63-ad98-e9d5c4976697\") " pod="openshift-ingress/e2e-ipp-cleanup-b4b570-openshift-default-86b7f9547b-5pgkk"
Sep 12 00:50:47.582820 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:50:47.582800    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ee555fcf-4c74-4b63-ad98-e9d5c4976697-istio-podinfo\") pod \"e2e-ipp-cleanup-b4b570-openshift-default-86b7f9547b-5pgkk\" (UID: \"ee555fcf-4c74-4b63-ad98-e9d5c4976697\") " pod="openshift-ingress/e2e-ipp-cleanup-b4b570-openshift-default-86b7f9547b-5pgkk"
Sep 12 00:50:47.589144 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:50:47.589118    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ee555fcf-4c74-4b63-ad98-e9d5c4976697-istio-token\") pod \"e2e-ipp-cleanup-b4b570-openshift-default-86b7f9547b-5pgkk\" (UID: \"ee555fcf-4c74-4b63-ad98-e9d5c4976697\") " pod="openshift-ingress/e2e-ipp-cleanup-b4b570-openshift-default-86b7f9547b-5pgkk"
Sep 12 00:50:47.589735 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:50:47.589715    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tk4r4\" (UniqueName: \"kubernetes.io/projected/ee555fcf-4c74-4b63-ad98-e9d5c4976697-kube-api-access-tk4r4\") pod \"e2e-ipp-cleanup-b4b570-openshift-default-86b7f9547b-5pgkk\" (UID: \"ee555fcf-4c74-4b63-ad98-e9d5c4976697\") " pod="openshift-ingress/e2e-ipp-cleanup-b4b570-openshift-default-86b7f9547b-5pgkk"
Sep 12 00:50:47.716927 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:50:47.716897    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-cleanup-b4b570-openshift-default-86b7f9547b-5pgkk"
Sep 12 00:50:47.998483 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:50:47.998456    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ipp-cleanup-b4b570-openshift-default-86b7f9547b-5pgkk"]
Sep 12 00:50:48.000614 ip-10-0-141-128 kubenswrapper[2571]: W0912 00:50:48.000572    2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podee555fcf_4c74_4b63_ad98_e9d5c4976697.slice/crio-46d82324ce9eb5818c894eea2de4e2ae1a894436cdc60aa7de05447d121a9b08 WatchSource:0}: Error finding container 46d82324ce9eb5818c894eea2de4e2ae1a894436cdc60aa7de05447d121a9b08: Status 404 returned error can't find the container with id 46d82324ce9eb5818c894eea2de4e2ae1a894436cdc60aa7de05447d121a9b08
Sep 12 00:50:48.003400 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:50:48.003365    2571 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236196Ki","pods":"250"}
Sep 12 00:50:48.003506 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:50:48.003433    2571 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236196Ki","pods":"250"}
Sep 12 00:50:48.003506 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:50:48.003466    2571 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236196Ki","pods":"250"}
Sep 12 00:50:48.425712 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:50:48.425611    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-cleanup-b4b570-openshift-default-86b7f9547b-5pgkk" event={"ID":"ee555fcf-4c74-4b63-ad98-e9d5c4976697","Type":"ContainerStarted","Data":"58e659de0af2752162e21f8ae21fe115d795cfd7958893d58b86d6a9dcd7f098"}
Sep 12 00:50:48.425876 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:50:48.425720    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-cleanup-b4b570-openshift-default-86b7f9547b-5pgkk" event={"ID":"ee555fcf-4c74-4b63-ad98-e9d5c4976697","Type":"ContainerStarted","Data":"46d82324ce9eb5818c894eea2de4e2ae1a894436cdc60aa7de05447d121a9b08"}
Sep 12 00:50:48.447173 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:50:48.447111    2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-ipp-cleanup-b4b570-openshift-default-86b7f9547b-5pgkk" podStartSLOduration=1.447092223 podStartE2EDuration="1.447092223s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-12 00:50:47 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-12 00:50:48.445150079 +0000 UTC m=+2059.716790613" watchObservedRunningTime="2026-09-12 00:50:48.447092223 +0000 UTC m=+2059.718732732"
Sep 12 00:50:48.717004 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:50:48.716974    2571 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ipp-cleanup-b4b570-openshift-default-86b7f9547b-5pgkk"
Sep 12 00:50:49.723469 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:50:49.723439    2571 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ipp-cleanup-b4b570-openshift-default-86b7f9547b-5pgkk"
Sep 12 00:50:50.436406 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:50:50.436375    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ipp-cleanup-b4b570-openshift-default-86b7f9547b-5pgkk"
Sep 12 00:50:50.437215 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:50:50.437198    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ipp-cleanup-b4b570-openshift-default-86b7f9547b-5pgkk"
Sep 12 00:50:53.961633 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:50:53.961515    2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-b4b570-94b4f9bc8-6txkd"]
Sep 12 00:50:53.965779 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:50:53.965753    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-b4b570-94b4f9bc8-6txkd"
Sep 12 00:50:53.968555 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:50:53.968534    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-ipp-cleanup-b4b570\""
Sep 12 00:50:53.968697 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:50:53.968549    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-ipp-cleanup-b4b570-dockercfg-lgspr\""
Sep 12 00:50:53.979247 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:50:53.979225    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-b4b570-94b4f9bc8-6txkd"]
Sep 12 00:50:54.048164 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:50:54.048124    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/bb470798-740b-44be-a5dd-48febecd5d30-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-cleanup-b4b570-94b4f9bc8-6txkd\" (UID: \"bb470798-740b-44be-a5dd-48febecd5d30\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-b4b570-94b4f9bc8-6txkd"
Sep 12 00:50:54.048347 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:50:54.048273    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lk5ns\" (UniqueName: \"kubernetes.io/projected/bb470798-740b-44be-a5dd-48febecd5d30-kube-api-access-lk5ns\") pod \"payload-pre-processing-e2e-ipp-cleanup-b4b570-94b4f9bc8-6txkd\" (UID: \"bb470798-740b-44be-a5dd-48febecd5d30\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-b4b570-94b4f9bc8-6txkd"
Sep 12 00:50:54.149256 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:50:54.149223    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lk5ns\" (UniqueName: \"kubernetes.io/projected/bb470798-740b-44be-a5dd-48febecd5d30-kube-api-access-lk5ns\") pod \"payload-pre-processing-e2e-ipp-cleanup-b4b570-94b4f9bc8-6txkd\" (UID: \"bb470798-740b-44be-a5dd-48febecd5d30\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-b4b570-94b4f9bc8-6txkd"
Sep 12 00:50:54.149403 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:50:54.149313    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/bb470798-740b-44be-a5dd-48febecd5d30-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-cleanup-b4b570-94b4f9bc8-6txkd\" (UID: \"bb470798-740b-44be-a5dd-48febecd5d30\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-b4b570-94b4f9bc8-6txkd"
Sep 12 00:50:54.149977 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:50:54.149958    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/bb470798-740b-44be-a5dd-48febecd5d30-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-cleanup-b4b570-94b4f9bc8-6txkd\" (UID: \"bb470798-740b-44be-a5dd-48febecd5d30\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-b4b570-94b4f9bc8-6txkd"
Sep 12 00:50:54.156533 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:50:54.156512    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lk5ns\" (UniqueName: \"kubernetes.io/projected/bb470798-740b-44be-a5dd-48febecd5d30-kube-api-access-lk5ns\") pod \"payload-pre-processing-e2e-ipp-cleanup-b4b570-94b4f9bc8-6txkd\" (UID: \"bb470798-740b-44be-a5dd-48febecd5d30\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-b4b570-94b4f9bc8-6txkd"
Sep 12 00:50:54.278722 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:50:54.278632    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-b4b570-94b4f9bc8-6txkd"
Sep 12 00:50:54.557066 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:50:54.557041    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-b4b570-94b4f9bc8-6txkd"]
Sep 12 00:50:54.559375 ip-10-0-141-128 kubenswrapper[2571]: W0912 00:50:54.559347    2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbb470798_740b_44be_a5dd_48febecd5d30.slice/crio-f30c46a09c13c01a712a1c23cb28db69eaea88a3e386498d18a4e2d418bd85a8 WatchSource:0}: Error finding container f30c46a09c13c01a712a1c23cb28db69eaea88a3e386498d18a4e2d418bd85a8: Status 404 returned error can't find the container with id f30c46a09c13c01a712a1c23cb28db69eaea88a3e386498d18a4e2d418bd85a8
Sep 12 00:50:55.465782 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:50:55.465748    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-b4b570-94b4f9bc8-6txkd" event={"ID":"bb470798-740b-44be-a5dd-48febecd5d30","Type":"ContainerStarted","Data":"de511d7966d4403ee3e1e15d4da448f4fcdd92fc2f5f35c7aa7d4b7388e11874"}
Sep 12 00:50:55.466268 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:50:55.465807    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-b4b570-94b4f9bc8-6txkd" event={"ID":"bb470798-740b-44be-a5dd-48febecd5d30","Type":"ContainerStarted","Data":"f30c46a09c13c01a712a1c23cb28db69eaea88a3e386498d18a4e2d418bd85a8"}
Sep 12 00:50:55.466268 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:50:55.465962    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-b4b570-94b4f9bc8-6txkd"
Sep 12 00:50:55.481519 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:50:55.481462    2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-b4b570-94b4f9bc8-6txkd" podStartSLOduration=2.481444284 podStartE2EDuration="2.481444284s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-12 00:50:53 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-12 00:50:55.480791258 +0000 UTC m=+2066.752431767" watchObservedRunningTime="2026-09-12 00:50:55.481444284 +0000 UTC m=+2066.753084795"
Sep 12 00:51:06.473833 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:51:06.473785    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-b4b570-94b4f9bc8-6txkd"
Sep 12 00:51:10.553763 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:51:10.553674    2571 generic.go:356] "Generic (PLEG): container finished" podID="ee555fcf-4c74-4b63-ad98-e9d5c4976697" containerID="58e659de0af2752162e21f8ae21fe115d795cfd7958893d58b86d6a9dcd7f098" exitCode=0
Sep 12 00:51:10.553763 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:51:10.553750    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-cleanup-b4b570-openshift-default-86b7f9547b-5pgkk" event={"ID":"ee555fcf-4c74-4b63-ad98-e9d5c4976697","Type":"ContainerDied","Data":"58e659de0af2752162e21f8ae21fe115d795cfd7958893d58b86d6a9dcd7f098"}
Sep 12 00:51:10.554266 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:51:10.554159    2571 scope.go:117] "RemoveContainer" containerID="58e659de0af2752162e21f8ae21fe115d795cfd7958893d58b86d6a9dcd7f098"
Sep 12 00:51:10.555252 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:51:10.555208    2571 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236196Ki","pods":"250"}
Sep 12 00:51:10.555311 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:51:10.555294    2571 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236196Ki","pods":"250"}
Sep 12 00:51:10.555353 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:51:10.555326    2571 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236196Ki","pods":"250"}
Sep 12 00:51:11.560835 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:51:11.560800    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-cleanup-b4b570-openshift-default-86b7f9547b-5pgkk" event={"ID":"ee555fcf-4c74-4b63-ad98-e9d5c4976697","Type":"ContainerStarted","Data":"2b518dfb46c31ea2de38f08adf0e36eed23255640d6b7e28c07b3182c8c3845b"}
Sep 12 00:51:11.717529 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:51:11.717489    2571 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ipp-cleanup-b4b570-openshift-default-86b7f9547b-5pgkk"
Sep 12 00:51:11.717808 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:51:11.717783    2571 patch_prober.go:28] interesting pod/e2e-ipp-cleanup-b4b570-openshift-default-86b7f9547b-5pgkk container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.73:15021/healthz/ready\": dial tcp 10.132.0.73:15021: connect: connection refused" start-of-body=
Sep 12 00:51:11.717859 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:51:11.717838    2571 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-cleanup-b4b570-openshift-default-86b7f9547b-5pgkk" podUID="ee555fcf-4c74-4b63-ad98-e9d5c4976697" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.73:15021/healthz/ready\": dial tcp 10.132.0.73:15021: connect: connection refused"
Sep 12 00:51:12.717500 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:51:12.717457    2571 patch_prober.go:28] interesting pod/e2e-ipp-cleanup-b4b570-openshift-default-86b7f9547b-5pgkk container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.73:15021/healthz/ready\": dial tcp 10.132.0.73:15021: connect: connection refused" start-of-body=
Sep 12 00:51:12.718001 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:51:12.717520    2571 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-cleanup-b4b570-openshift-default-86b7f9547b-5pgkk" podUID="ee555fcf-4c74-4b63-ad98-e9d5c4976697" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.73:15021/healthz/ready\": dial tcp 10.132.0.73:15021: connect: connection refused"
Sep 12 00:51:13.717467 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:51:13.717427    2571 patch_prober.go:28] interesting pod/e2e-ipp-cleanup-b4b570-openshift-default-86b7f9547b-5pgkk container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.73:15021/healthz/ready\": dial tcp 10.132.0.73:15021: connect: connection refused" start-of-body=
Sep 12 00:51:13.717670 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:51:13.717485    2571 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-cleanup-b4b570-openshift-default-86b7f9547b-5pgkk" podUID="ee555fcf-4c74-4b63-ad98-e9d5c4976697" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.73:15021/healthz/ready\": dial tcp 10.132.0.73:15021: connect: connection refused"
Sep 12 00:51:14.718075 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:51:14.718035    2571 patch_prober.go:28] interesting pod/e2e-ipp-cleanup-b4b570-openshift-default-86b7f9547b-5pgkk container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.73:15021/healthz/ready\": dial tcp 10.132.0.73:15021: connect: connection refused" start-of-body=
Sep 12 00:51:14.718567 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:51:14.718106    2571 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-cleanup-b4b570-openshift-default-86b7f9547b-5pgkk" podUID="ee555fcf-4c74-4b63-ad98-e9d5c4976697" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.73:15021/healthz/ready\": dial tcp 10.132.0.73:15021: connect: connection refused"
Sep 12 00:51:15.717569 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:51:15.717525    2571 patch_prober.go:28] interesting pod/e2e-ipp-cleanup-b4b570-openshift-default-86b7f9547b-5pgkk container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.73:15021/healthz/ready\": dial tcp 10.132.0.73:15021: connect: connection refused" start-of-body=
Sep 12 00:51:15.717806 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:51:15.717613    2571 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-cleanup-b4b570-openshift-default-86b7f9547b-5pgkk" podUID="ee555fcf-4c74-4b63-ad98-e9d5c4976697" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.73:15021/healthz/ready\": dial tcp 10.132.0.73:15021: connect: connection refused"
Sep 12 00:51:16.876955 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:51:16.876919    2571 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ipp-cleanup-b4b570-openshift-default-86b7f9547b-5pgkk"
Sep 12 00:51:16.877447 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:51:16.877285    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ipp-cleanup-b4b570-openshift-default-86b7f9547b-5pgkk"
Sep 12 00:51:16.878266 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:51:16.878245    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ipp-cleanup-b4b570-openshift-default-86b7f9547b-5pgkk"
Sep 12 00:51:22.530286 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:51:22.530253    2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-b4b570-94b4f9bc8-6txkd"]
Sep 12 00:51:22.530803 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:51:22.530473    2571 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-b4b570-94b4f9bc8-6txkd" podUID="bb470798-740b-44be-a5dd-48febecd5d30" containerName="payload-pre-processing" containerID="cri-o://de511d7966d4403ee3e1e15d4da448f4fcdd92fc2f5f35c7aa7d4b7388e11874" gracePeriod=30
Sep 12 00:51:22.808799 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:51:22.808352    2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-b4b570-94b4f9bc8-6txkd"
Sep 12 00:51:22.845852 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:51:22.845818    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/bb470798-740b-44be-a5dd-48febecd5d30-plugins-config-volume\") pod \"bb470798-740b-44be-a5dd-48febecd5d30\" (UID: \"bb470798-740b-44be-a5dd-48febecd5d30\") "
Sep 12 00:51:22.846033 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:51:22.845877    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-lk5ns\" (UniqueName: \"kubernetes.io/projected/bb470798-740b-44be-a5dd-48febecd5d30-kube-api-access-lk5ns\") pod \"bb470798-740b-44be-a5dd-48febecd5d30\" (UID: \"bb470798-740b-44be-a5dd-48febecd5d30\") "
Sep 12 00:51:22.846171 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:51:22.846146    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/bb470798-740b-44be-a5dd-48febecd5d30-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "bb470798-740b-44be-a5dd-48febecd5d30" (UID: "bb470798-740b-44be-a5dd-48febecd5d30"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 12 00:51:22.846321 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:51:22.846302    2571 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/bb470798-740b-44be-a5dd-48febecd5d30-plugins-config-volume\") on node \"ip-10-0-141-128.ec2.internal\" DevicePath \"\""
Sep 12 00:51:22.848007 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:51:22.847982    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/bb470798-740b-44be-a5dd-48febecd5d30-kube-api-access-lk5ns" (OuterVolumeSpecName: "kube-api-access-lk5ns") pod "bb470798-740b-44be-a5dd-48febecd5d30" (UID: "bb470798-740b-44be-a5dd-48febecd5d30"). InnerVolumeSpecName "kube-api-access-lk5ns". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 12 00:51:22.948047 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:51:22.948006    2571 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-lk5ns\" (UniqueName: \"kubernetes.io/projected/bb470798-740b-44be-a5dd-48febecd5d30-kube-api-access-lk5ns\") on node \"ip-10-0-141-128.ec2.internal\" DevicePath \"\""
Sep 12 00:51:23.640553 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:51:23.640512    2571 generic.go:356] "Generic (PLEG): container finished" podID="bb470798-740b-44be-a5dd-48febecd5d30" containerID="de511d7966d4403ee3e1e15d4da448f4fcdd92fc2f5f35c7aa7d4b7388e11874" exitCode=0
Sep 12 00:51:23.641112 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:51:23.640574    2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-b4b570-94b4f9bc8-6txkd"
Sep 12 00:51:23.641112 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:51:23.640610    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-b4b570-94b4f9bc8-6txkd" event={"ID":"bb470798-740b-44be-a5dd-48febecd5d30","Type":"ContainerDied","Data":"de511d7966d4403ee3e1e15d4da448f4fcdd92fc2f5f35c7aa7d4b7388e11874"}
Sep 12 00:51:23.641112 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:51:23.640669    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-b4b570-94b4f9bc8-6txkd" event={"ID":"bb470798-740b-44be-a5dd-48febecd5d30","Type":"ContainerDied","Data":"f30c46a09c13c01a712a1c23cb28db69eaea88a3e386498d18a4e2d418bd85a8"}
Sep 12 00:51:23.641112 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:51:23.640692    2571 scope.go:117] "RemoveContainer" containerID="de511d7966d4403ee3e1e15d4da448f4fcdd92fc2f5f35c7aa7d4b7388e11874"
Sep 12 00:51:23.649842 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:51:23.649826    2571 scope.go:117] "RemoveContainer" containerID="de511d7966d4403ee3e1e15d4da448f4fcdd92fc2f5f35c7aa7d4b7388e11874"
Sep 12 00:51:23.650094 ip-10-0-141-128 kubenswrapper[2571]: E0912 00:51:23.650079    2571 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"de511d7966d4403ee3e1e15d4da448f4fcdd92fc2f5f35c7aa7d4b7388e11874\": container with ID starting with de511d7966d4403ee3e1e15d4da448f4fcdd92fc2f5f35c7aa7d4b7388e11874 not found: ID does not exist" containerID="de511d7966d4403ee3e1e15d4da448f4fcdd92fc2f5f35c7aa7d4b7388e11874"
Sep 12 00:51:23.650134 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:51:23.650103    2571 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"de511d7966d4403ee3e1e15d4da448f4fcdd92fc2f5f35c7aa7d4b7388e11874"} err="failed to get container status \"de511d7966d4403ee3e1e15d4da448f4fcdd92fc2f5f35c7aa7d4b7388e11874\": rpc error: code = NotFound desc = could not find container \"de511d7966d4403ee3e1e15d4da448f4fcdd92fc2f5f35c7aa7d4b7388e11874\": container with ID starting with de511d7966d4403ee3e1e15d4da448f4fcdd92fc2f5f35c7aa7d4b7388e11874 not found: ID does not exist"
Sep 12 00:51:23.656890 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:51:23.656869    2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-b4b570-94b4f9bc8-6txkd"]
Sep 12 00:51:23.661269 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:51:23.661250    2571 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-b4b570-94b4f9bc8-6txkd"]
Sep 12 00:51:25.140812 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:51:25.140781    2571 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="bb470798-740b-44be-a5dd-48febecd5d30" path="/var/lib/kubelet/pods/bb470798-740b-44be-a5dd-48febecd5d30/volumes"
Sep 12 00:51:29.255148 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:51:29.255115    2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ipp-cleanup-b4b570-openshift-default-86b7f9547b-5pgkk"]
Sep 12 00:51:29.255632 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:51:29.255468    2571 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-ipp-cleanup-b4b570-openshift-default-86b7f9547b-5pgkk" podUID="ee555fcf-4c74-4b63-ad98-e9d5c4976697" containerName="istio-proxy" containerID="cri-o://2b518dfb46c31ea2de38f08adf0e36eed23255640d6b7e28c07b3182c8c3845b" gracePeriod=30
Sep 12 00:51:29.440786 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:51:29.440748    2571 scope.go:117] "RemoveContainer" containerID="82b6f88a306f6036c0f6fb0dd23d1068c191d393849191f3d2531946108d69d9"
Sep 12 00:51:29.449114 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:51:29.449094    2571 scope.go:117] "RemoveContainer" containerID="6efa982753ed4f291c0e76c75e4a32c893cd41a05326a82608653c09409c206c"
Sep 12 00:51:29.456949 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:51:29.456934    2571 scope.go:117] "RemoveContainer" containerID="286db8b787f83a22ec0e798120e1918631badda3aea2cf20ff9f9a2ca3ce5b29"
Sep 12 00:51:31.879573 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:51:31.879537    2571 patch_prober.go:28] interesting pod/e2e-ipp-cleanup-b4b570-openshift-default-86b7f9547b-5pgkk container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Sep 12 00:51:31.880017 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:51:31.879618    2571 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-ipp-cleanup-b4b570-openshift-default-86b7f9547b-5pgkk" podUID="ee555fcf-4c74-4b63-ad98-e9d5c4976697" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Sep 12 00:51:34.524139 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:51:34.524079    2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-cleanup-b4b570-openshift-default-86b7f9547b-5pgkk"
Sep 12 00:51:34.659698 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:51:34.659666    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ee555fcf-4c74-4b63-ad98-e9d5c4976697-istiod-ca-cert\") pod \"ee555fcf-4c74-4b63-ad98-e9d5c4976697\" (UID: \"ee555fcf-4c74-4b63-ad98-e9d5c4976697\") "
Sep 12 00:51:34.659856 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:51:34.659739    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ee555fcf-4c74-4b63-ad98-e9d5c4976697-istio-token\") pod \"ee555fcf-4c74-4b63-ad98-e9d5c4976697\" (UID: \"ee555fcf-4c74-4b63-ad98-e9d5c4976697\") "
Sep 12 00:51:34.659856 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:51:34.659774    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-tk4r4\" (UniqueName: \"kubernetes.io/projected/ee555fcf-4c74-4b63-ad98-e9d5c4976697-kube-api-access-tk4r4\") pod \"ee555fcf-4c74-4b63-ad98-e9d5c4976697\" (UID: \"ee555fcf-4c74-4b63-ad98-e9d5c4976697\") "
Sep 12 00:51:34.659856 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:51:34.659806    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ee555fcf-4c74-4b63-ad98-e9d5c4976697-istio-podinfo\") pod \"ee555fcf-4c74-4b63-ad98-e9d5c4976697\" (UID: \"ee555fcf-4c74-4b63-ad98-e9d5c4976697\") "
Sep 12 00:51:34.659856 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:51:34.659836    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ee555fcf-4c74-4b63-ad98-e9d5c4976697-workload-certs\") pod \"ee555fcf-4c74-4b63-ad98-e9d5c4976697\" (UID: \"ee555fcf-4c74-4b63-ad98-e9d5c4976697\") "
Sep 12 00:51:34.660048 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:51:34.659895    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ee555fcf-4c74-4b63-ad98-e9d5c4976697-istio-data\") pod \"ee555fcf-4c74-4b63-ad98-e9d5c4976697\" (UID: \"ee555fcf-4c74-4b63-ad98-e9d5c4976697\") "
Sep 12 00:51:34.660048 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:51:34.659920    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ee555fcf-4c74-4b63-ad98-e9d5c4976697-credential-socket\") pod \"ee555fcf-4c74-4b63-ad98-e9d5c4976697\" (UID: \"ee555fcf-4c74-4b63-ad98-e9d5c4976697\") "
Sep 12 00:51:34.660048 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:51:34.659943    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ee555fcf-4c74-4b63-ad98-e9d5c4976697-workload-socket\") pod \"ee555fcf-4c74-4b63-ad98-e9d5c4976697\" (UID: \"ee555fcf-4c74-4b63-ad98-e9d5c4976697\") "
Sep 12 00:51:34.660048 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:51:34.659972    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ee555fcf-4c74-4b63-ad98-e9d5c4976697-istio-envoy\") pod \"ee555fcf-4c74-4b63-ad98-e9d5c4976697\" (UID: \"ee555fcf-4c74-4b63-ad98-e9d5c4976697\") "
Sep 12 00:51:34.660261 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:51:34.660045    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ee555fcf-4c74-4b63-ad98-e9d5c4976697-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "ee555fcf-4c74-4b63-ad98-e9d5c4976697" (UID: "ee555fcf-4c74-4b63-ad98-e9d5c4976697"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 12 00:51:34.660261 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:51:34.660222    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ee555fcf-4c74-4b63-ad98-e9d5c4976697-istio-data" (OuterVolumeSpecName: "istio-data") pod "ee555fcf-4c74-4b63-ad98-e9d5c4976697" (UID: "ee555fcf-4c74-4b63-ad98-e9d5c4976697"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 12 00:51:34.660261 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:51:34.660245    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ee555fcf-4c74-4b63-ad98-e9d5c4976697-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "ee555fcf-4c74-4b63-ad98-e9d5c4976697" (UID: "ee555fcf-4c74-4b63-ad98-e9d5c4976697"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 12 00:51:34.660429 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:51:34.660347    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ee555fcf-4c74-4b63-ad98-e9d5c4976697-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "ee555fcf-4c74-4b63-ad98-e9d5c4976697" (UID: "ee555fcf-4c74-4b63-ad98-e9d5c4976697"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 12 00:51:34.660429 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:51:34.660367    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ee555fcf-4c74-4b63-ad98-e9d5c4976697-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "ee555fcf-4c74-4b63-ad98-e9d5c4976697" (UID: "ee555fcf-4c74-4b63-ad98-e9d5c4976697"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 12 00:51:34.660429 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:51:34.660407    2571 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ee555fcf-4c74-4b63-ad98-e9d5c4976697-istiod-ca-cert\") on node \"ip-10-0-141-128.ec2.internal\" DevicePath \"\""
Sep 12 00:51:34.660429 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:51:34.660428    2571 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ee555fcf-4c74-4b63-ad98-e9d5c4976697-workload-certs\") on node \"ip-10-0-141-128.ec2.internal\" DevicePath \"\""
Sep 12 00:51:34.660619 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:51:34.660442    2571 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ee555fcf-4c74-4b63-ad98-e9d5c4976697-istio-data\") on node \"ip-10-0-141-128.ec2.internal\" DevicePath \"\""
Sep 12 00:51:34.662156 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:51:34.662062    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/ee555fcf-4c74-4b63-ad98-e9d5c4976697-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "ee555fcf-4c74-4b63-ad98-e9d5c4976697" (UID: "ee555fcf-4c74-4b63-ad98-e9d5c4976697"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 12 00:51:34.662283 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:51:34.662208    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ee555fcf-4c74-4b63-ad98-e9d5c4976697-kube-api-access-tk4r4" (OuterVolumeSpecName: "kube-api-access-tk4r4") pod "ee555fcf-4c74-4b63-ad98-e9d5c4976697" (UID: "ee555fcf-4c74-4b63-ad98-e9d5c4976697"). InnerVolumeSpecName "kube-api-access-tk4r4". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 12 00:51:34.662364 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:51:34.662346    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ee555fcf-4c74-4b63-ad98-e9d5c4976697-istio-token" (OuterVolumeSpecName: "istio-token") pod "ee555fcf-4c74-4b63-ad98-e9d5c4976697" (UID: "ee555fcf-4c74-4b63-ad98-e9d5c4976697"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 12 00:51:34.662410 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:51:34.662396    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ee555fcf-4c74-4b63-ad98-e9d5c4976697-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "ee555fcf-4c74-4b63-ad98-e9d5c4976697" (UID: "ee555fcf-4c74-4b63-ad98-e9d5c4976697"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 12 00:51:34.705336 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:51:34.705307    2571 generic.go:356] "Generic (PLEG): container finished" podID="ee555fcf-4c74-4b63-ad98-e9d5c4976697" containerID="2b518dfb46c31ea2de38f08adf0e36eed23255640d6b7e28c07b3182c8c3845b" exitCode=0
Sep 12 00:51:34.705472 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:51:34.705356    2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-cleanup-b4b570-openshift-default-86b7f9547b-5pgkk"
Sep 12 00:51:34.705472 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:51:34.705356    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-cleanup-b4b570-openshift-default-86b7f9547b-5pgkk" event={"ID":"ee555fcf-4c74-4b63-ad98-e9d5c4976697","Type":"ContainerDied","Data":"2b518dfb46c31ea2de38f08adf0e36eed23255640d6b7e28c07b3182c8c3845b"}
Sep 12 00:51:34.705611 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:51:34.705487    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-cleanup-b4b570-openshift-default-86b7f9547b-5pgkk" event={"ID":"ee555fcf-4c74-4b63-ad98-e9d5c4976697","Type":"ContainerDied","Data":"46d82324ce9eb5818c894eea2de4e2ae1a894436cdc60aa7de05447d121a9b08"}
Sep 12 00:51:34.705611 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:51:34.705528    2571 scope.go:117] "RemoveContainer" containerID="2b518dfb46c31ea2de38f08adf0e36eed23255640d6b7e28c07b3182c8c3845b"
Sep 12 00:51:34.714463 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:51:34.714448    2571 scope.go:117] "RemoveContainer" containerID="58e659de0af2752162e21f8ae21fe115d795cfd7958893d58b86d6a9dcd7f098"
Sep 12 00:51:34.722304 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:51:34.722287    2571 scope.go:117] "RemoveContainer" containerID="2b518dfb46c31ea2de38f08adf0e36eed23255640d6b7e28c07b3182c8c3845b"
Sep 12 00:51:34.722607 ip-10-0-141-128 kubenswrapper[2571]: E0912 00:51:34.722569    2571 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2b518dfb46c31ea2de38f08adf0e36eed23255640d6b7e28c07b3182c8c3845b\": container with ID starting with 2b518dfb46c31ea2de38f08adf0e36eed23255640d6b7e28c07b3182c8c3845b not found: ID does not exist" containerID="2b518dfb46c31ea2de38f08adf0e36eed23255640d6b7e28c07b3182c8c3845b"
Sep 12 00:51:34.722671 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:51:34.722605    2571 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2b518dfb46c31ea2de38f08adf0e36eed23255640d6b7e28c07b3182c8c3845b"} err="failed to get container status \"2b518dfb46c31ea2de38f08adf0e36eed23255640d6b7e28c07b3182c8c3845b\": rpc error: code = NotFound desc = could not find container \"2b518dfb46c31ea2de38f08adf0e36eed23255640d6b7e28c07b3182c8c3845b\": container with ID starting with 2b518dfb46c31ea2de38f08adf0e36eed23255640d6b7e28c07b3182c8c3845b not found: ID does not exist"
Sep 12 00:51:34.722671 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:51:34.722622    2571 scope.go:117] "RemoveContainer" containerID="58e659de0af2752162e21f8ae21fe115d795cfd7958893d58b86d6a9dcd7f098"
Sep 12 00:51:34.722871 ip-10-0-141-128 kubenswrapper[2571]: E0912 00:51:34.722854    2571 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"58e659de0af2752162e21f8ae21fe115d795cfd7958893d58b86d6a9dcd7f098\": container with ID starting with 58e659de0af2752162e21f8ae21fe115d795cfd7958893d58b86d6a9dcd7f098 not found: ID does not exist" containerID="58e659de0af2752162e21f8ae21fe115d795cfd7958893d58b86d6a9dcd7f098"
Sep 12 00:51:34.722917 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:51:34.722877    2571 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"58e659de0af2752162e21f8ae21fe115d795cfd7958893d58b86d6a9dcd7f098"} err="failed to get container status \"58e659de0af2752162e21f8ae21fe115d795cfd7958893d58b86d6a9dcd7f098\": rpc error: code = NotFound desc = could not find container \"58e659de0af2752162e21f8ae21fe115d795cfd7958893d58b86d6a9dcd7f098\": container with ID starting with 58e659de0af2752162e21f8ae21fe115d795cfd7958893d58b86d6a9dcd7f098 not found: ID does not exist"
Sep 12 00:51:34.726870 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:51:34.726846    2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ipp-cleanup-b4b570-openshift-default-86b7f9547b-5pgkk"]
Sep 12 00:51:34.730150 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:51:34.730130    2571 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ipp-cleanup-b4b570-openshift-default-86b7f9547b-5pgkk"]
Sep 12 00:51:34.761807 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:51:34.761786    2571 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ee555fcf-4c74-4b63-ad98-e9d5c4976697-istio-token\") on node \"ip-10-0-141-128.ec2.internal\" DevicePath \"\""
Sep 12 00:51:34.761901 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:51:34.761810    2571 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-tk4r4\" (UniqueName: \"kubernetes.io/projected/ee555fcf-4c74-4b63-ad98-e9d5c4976697-kube-api-access-tk4r4\") on node \"ip-10-0-141-128.ec2.internal\" DevicePath \"\""
Sep 12 00:51:34.761901 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:51:34.761824    2571 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ee555fcf-4c74-4b63-ad98-e9d5c4976697-istio-podinfo\") on node \"ip-10-0-141-128.ec2.internal\" DevicePath \"\""
Sep 12 00:51:34.761901 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:51:34.761837    2571 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ee555fcf-4c74-4b63-ad98-e9d5c4976697-credential-socket\") on node \"ip-10-0-141-128.ec2.internal\" DevicePath \"\""
Sep 12 00:51:34.761901 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:51:34.761850    2571 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ee555fcf-4c74-4b63-ad98-e9d5c4976697-workload-socket\") on node \"ip-10-0-141-128.ec2.internal\" DevicePath \"\""
Sep 12 00:51:34.761901 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:51:34.761864    2571 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ee555fcf-4c74-4b63-ad98-e9d5c4976697-istio-envoy\") on node \"ip-10-0-141-128.ec2.internal\" DevicePath \"\""
Sep 12 00:51:35.141342 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:51:35.141308    2571 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ee555fcf-4c74-4b63-ad98-e9d5c4976697" path="/var/lib/kubelet/pods/ee555fcf-4c74-4b63-ad98-e9d5c4976697/volumes"
Sep 12 00:52:37.863161 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:52:37.863130    2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ipp-a-64e462-openshift-default-67664986bd-989bb"]
Sep 12 00:52:37.863738 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:52:37.863431    2571 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-ipp-a-64e462-openshift-default-67664986bd-989bb" podUID="d73ec31b-c54e-44df-a448-0a914b938eb7" containerName="istio-proxy" containerID="cri-o://bcc049736be1b73bb6a594d4989b9dac316fe84efb477460339028bdaf2d9e43" gracePeriod=30
Sep 12 00:52:43.055341 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:52:43.055302    2571 generic.go:356] "Generic (PLEG): container finished" podID="d73ec31b-c54e-44df-a448-0a914b938eb7" containerID="bcc049736be1b73bb6a594d4989b9dac316fe84efb477460339028bdaf2d9e43" exitCode=0
Sep 12 00:52:43.055834 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:52:43.055363    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-a-64e462-openshift-default-67664986bd-989bb" event={"ID":"d73ec31b-c54e-44df-a448-0a914b938eb7","Type":"ContainerDied","Data":"bcc049736be1b73bb6a594d4989b9dac316fe84efb477460339028bdaf2d9e43"}
Sep 12 00:52:43.055834 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:52:43.055429    2571 scope.go:117] "RemoveContainer" containerID="878068f959d49e38a7baed0752277da5d96b84be6efbcde359bb40b6f856dbf5"
Sep 12 00:52:43.118069 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:52:43.118048    2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-a-64e462-openshift-default-67664986bd-989bb"
Sep 12 00:52:43.189145 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:52:43.189123    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d73ec31b-c54e-44df-a448-0a914b938eb7-credential-socket\") pod \"d73ec31b-c54e-44df-a448-0a914b938eb7\" (UID: \"d73ec31b-c54e-44df-a448-0a914b938eb7\") "
Sep 12 00:52:43.189291 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:52:43.189152    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d73ec31b-c54e-44df-a448-0a914b938eb7-workload-socket\") pod \"d73ec31b-c54e-44df-a448-0a914b938eb7\" (UID: \"d73ec31b-c54e-44df-a448-0a914b938eb7\") "
Sep 12 00:52:43.189291 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:52:43.189177    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d73ec31b-c54e-44df-a448-0a914b938eb7-workload-certs\") pod \"d73ec31b-c54e-44df-a448-0a914b938eb7\" (UID: \"d73ec31b-c54e-44df-a448-0a914b938eb7\") "
Sep 12 00:52:43.189291 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:52:43.189253    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d73ec31b-c54e-44df-a448-0a914b938eb7-istio-envoy\") pod \"d73ec31b-c54e-44df-a448-0a914b938eb7\" (UID: \"d73ec31b-c54e-44df-a448-0a914b938eb7\") "
Sep 12 00:52:43.189291 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:52:43.189282    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d73ec31b-c54e-44df-a448-0a914b938eb7-istio-podinfo\") pod \"d73ec31b-c54e-44df-a448-0a914b938eb7\" (UID: \"d73ec31b-c54e-44df-a448-0a914b938eb7\") "
Sep 12 00:52:43.189487 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:52:43.189297    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d73ec31b-c54e-44df-a448-0a914b938eb7-istio-data\") pod \"d73ec31b-c54e-44df-a448-0a914b938eb7\" (UID: \"d73ec31b-c54e-44df-a448-0a914b938eb7\") "
Sep 12 00:52:43.189487 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:52:43.189336    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-f9g28\" (UniqueName: \"kubernetes.io/projected/d73ec31b-c54e-44df-a448-0a914b938eb7-kube-api-access-f9g28\") pod \"d73ec31b-c54e-44df-a448-0a914b938eb7\" (UID: \"d73ec31b-c54e-44df-a448-0a914b938eb7\") "
Sep 12 00:52:43.189487 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:52:43.189366    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d73ec31b-c54e-44df-a448-0a914b938eb7-istiod-ca-cert\") pod \"d73ec31b-c54e-44df-a448-0a914b938eb7\" (UID: \"d73ec31b-c54e-44df-a448-0a914b938eb7\") "
Sep 12 00:52:43.189487 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:52:43.189387    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d73ec31b-c54e-44df-a448-0a914b938eb7-istio-token\") pod \"d73ec31b-c54e-44df-a448-0a914b938eb7\" (UID: \"d73ec31b-c54e-44df-a448-0a914b938eb7\") "
Sep 12 00:52:43.189487 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:52:43.189391    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d73ec31b-c54e-44df-a448-0a914b938eb7-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "d73ec31b-c54e-44df-a448-0a914b938eb7" (UID: "d73ec31b-c54e-44df-a448-0a914b938eb7"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 12 00:52:43.189487 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:52:43.189428    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d73ec31b-c54e-44df-a448-0a914b938eb7-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "d73ec31b-c54e-44df-a448-0a914b938eb7" (UID: "d73ec31b-c54e-44df-a448-0a914b938eb7"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 12 00:52:43.189822 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:52:43.189653    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d73ec31b-c54e-44df-a448-0a914b938eb7-istio-data" (OuterVolumeSpecName: "istio-data") pod "d73ec31b-c54e-44df-a448-0a914b938eb7" (UID: "d73ec31b-c54e-44df-a448-0a914b938eb7"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 12 00:52:43.189822 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:52:43.189683    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d73ec31b-c54e-44df-a448-0a914b938eb7-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "d73ec31b-c54e-44df-a448-0a914b938eb7" (UID: "d73ec31b-c54e-44df-a448-0a914b938eb7"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 12 00:52:43.189822 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:52:43.189765    2571 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d73ec31b-c54e-44df-a448-0a914b938eb7-istio-data\") on node \"ip-10-0-141-128.ec2.internal\" DevicePath \"\""
Sep 12 00:52:43.189822 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:52:43.189770    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d73ec31b-c54e-44df-a448-0a914b938eb7-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "d73ec31b-c54e-44df-a448-0a914b938eb7" (UID: "d73ec31b-c54e-44df-a448-0a914b938eb7"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 12 00:52:43.189822 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:52:43.189784    2571 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d73ec31b-c54e-44df-a448-0a914b938eb7-credential-socket\") on node \"ip-10-0-141-128.ec2.internal\" DevicePath \"\""
Sep 12 00:52:43.189822 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:52:43.189798    2571 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d73ec31b-c54e-44df-a448-0a914b938eb7-workload-socket\") on node \"ip-10-0-141-128.ec2.internal\" DevicePath \"\""
Sep 12 00:52:43.191524 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:52:43.191496    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/d73ec31b-c54e-44df-a448-0a914b938eb7-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "d73ec31b-c54e-44df-a448-0a914b938eb7" (UID: "d73ec31b-c54e-44df-a448-0a914b938eb7"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 12 00:52:43.191655 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:52:43.191538    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d73ec31b-c54e-44df-a448-0a914b938eb7-istio-token" (OuterVolumeSpecName: "istio-token") pod "d73ec31b-c54e-44df-a448-0a914b938eb7" (UID: "d73ec31b-c54e-44df-a448-0a914b938eb7"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 12 00:52:43.191655 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:52:43.191548    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d73ec31b-c54e-44df-a448-0a914b938eb7-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "d73ec31b-c54e-44df-a448-0a914b938eb7" (UID: "d73ec31b-c54e-44df-a448-0a914b938eb7"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 12 00:52:43.191655 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:52:43.191621    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d73ec31b-c54e-44df-a448-0a914b938eb7-kube-api-access-f9g28" (OuterVolumeSpecName: "kube-api-access-f9g28") pod "d73ec31b-c54e-44df-a448-0a914b938eb7" (UID: "d73ec31b-c54e-44df-a448-0a914b938eb7"). InnerVolumeSpecName "kube-api-access-f9g28". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 12 00:52:43.290889 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:52:43.290863    2571 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d73ec31b-c54e-44df-a448-0a914b938eb7-istio-podinfo\") on node \"ip-10-0-141-128.ec2.internal\" DevicePath \"\""
Sep 12 00:52:43.290889 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:52:43.290885    2571 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-f9g28\" (UniqueName: \"kubernetes.io/projected/d73ec31b-c54e-44df-a448-0a914b938eb7-kube-api-access-f9g28\") on node \"ip-10-0-141-128.ec2.internal\" DevicePath \"\""
Sep 12 00:52:43.291035 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:52:43.290899    2571 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d73ec31b-c54e-44df-a448-0a914b938eb7-istiod-ca-cert\") on node \"ip-10-0-141-128.ec2.internal\" DevicePath \"\""
Sep 12 00:52:43.291035 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:52:43.290911    2571 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d73ec31b-c54e-44df-a448-0a914b938eb7-istio-token\") on node \"ip-10-0-141-128.ec2.internal\" DevicePath \"\""
Sep 12 00:52:43.291035 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:52:43.290921    2571 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d73ec31b-c54e-44df-a448-0a914b938eb7-workload-certs\") on node \"ip-10-0-141-128.ec2.internal\" DevicePath \"\""
Sep 12 00:52:43.291035 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:52:43.290934    2571 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d73ec31b-c54e-44df-a448-0a914b938eb7-istio-envoy\") on node \"ip-10-0-141-128.ec2.internal\" DevicePath \"\""
Sep 12 00:52:44.061906 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:52:44.061877    2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-a-64e462-openshift-default-67664986bd-989bb"
Sep 12 00:52:44.062290 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:52:44.061884    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-a-64e462-openshift-default-67664986bd-989bb" event={"ID":"d73ec31b-c54e-44df-a448-0a914b938eb7","Type":"ContainerDied","Data":"f6351ca2f286d529a30f4be64461ca5b9e91f8dade33feac80b82aa5b248f5f6"}
Sep 12 00:52:44.062290 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:52:44.062010    2571 scope.go:117] "RemoveContainer" containerID="bcc049736be1b73bb6a594d4989b9dac316fe84efb477460339028bdaf2d9e43"
Sep 12 00:52:44.083229 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:52:44.083205    2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ipp-a-64e462-openshift-default-67664986bd-989bb"]
Sep 12 00:52:44.090550 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:52:44.090523    2571 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ipp-a-64e462-openshift-default-67664986bd-989bb"]
Sep 12 00:52:45.140410 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:52:45.140374    2571 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d73ec31b-c54e-44df-a448-0a914b938eb7" path="/var/lib/kubelet/pods/d73ec31b-c54e-44df-a448-0a914b938eb7/volumes"
Sep 12 00:52:55.464713 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:52:55.464680    2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ipp-b-1cbe1c-openshift-default-5fc97d867d-9hcqb"]
Sep 12 00:52:55.465195 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:52:55.464905    2571 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-ipp-b-1cbe1c-openshift-default-5fc97d867d-9hcqb" podUID="3c451460-9cc7-49b0-ab4f-f361805f15d9" containerName="istio-proxy" containerID="cri-o://a6e83b7404c55c45e080af62c7987d2934007fca1b332efcf5d42d9729f7d9d1" gracePeriod=30
Sep 12 00:53:00.721865 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:53:00.721840    2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-b-1cbe1c-openshift-default-5fc97d867d-9hcqb"
Sep 12 00:53:00.850279 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:53:00.850209    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3c451460-9cc7-49b0-ab4f-f361805f15d9-istio-data\") pod \"3c451460-9cc7-49b0-ab4f-f361805f15d9\" (UID: \"3c451460-9cc7-49b0-ab4f-f361805f15d9\") "
Sep 12 00:53:00.850279 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:53:00.850244    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3c451460-9cc7-49b0-ab4f-f361805f15d9-credential-socket\") pod \"3c451460-9cc7-49b0-ab4f-f361805f15d9\" (UID: \"3c451460-9cc7-49b0-ab4f-f361805f15d9\") "
Sep 12 00:53:00.850279 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:53:00.850279    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3c451460-9cc7-49b0-ab4f-f361805f15d9-workload-socket\") pod \"3c451460-9cc7-49b0-ab4f-f361805f15d9\" (UID: \"3c451460-9cc7-49b0-ab4f-f361805f15d9\") "
Sep 12 00:53:00.850554 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:53:00.850311    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3c451460-9cc7-49b0-ab4f-f361805f15d9-istiod-ca-cert\") pod \"3c451460-9cc7-49b0-ab4f-f361805f15d9\" (UID: \"3c451460-9cc7-49b0-ab4f-f361805f15d9\") "
Sep 12 00:53:00.850554 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:53:00.850363    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3c451460-9cc7-49b0-ab4f-f361805f15d9-workload-certs\") pod \"3c451460-9cc7-49b0-ab4f-f361805f15d9\" (UID: \"3c451460-9cc7-49b0-ab4f-f361805f15d9\") "
Sep 12 00:53:00.850554 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:53:00.850409    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3c451460-9cc7-49b0-ab4f-f361805f15d9-istio-envoy\") pod \"3c451460-9cc7-49b0-ab4f-f361805f15d9\" (UID: \"3c451460-9cc7-49b0-ab4f-f361805f15d9\") "
Sep 12 00:53:00.850554 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:53:00.850436    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-fchxc\" (UniqueName: \"kubernetes.io/projected/3c451460-9cc7-49b0-ab4f-f361805f15d9-kube-api-access-fchxc\") pod \"3c451460-9cc7-49b0-ab4f-f361805f15d9\" (UID: \"3c451460-9cc7-49b0-ab4f-f361805f15d9\") "
Sep 12 00:53:00.850554 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:53:00.850498    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3c451460-9cc7-49b0-ab4f-f361805f15d9-istio-token\") pod \"3c451460-9cc7-49b0-ab4f-f361805f15d9\" (UID: \"3c451460-9cc7-49b0-ab4f-f361805f15d9\") "
Sep 12 00:53:00.850554 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:53:00.850506    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3c451460-9cc7-49b0-ab4f-f361805f15d9-istio-data" (OuterVolumeSpecName: "istio-data") pod "3c451460-9cc7-49b0-ab4f-f361805f15d9" (UID: "3c451460-9cc7-49b0-ab4f-f361805f15d9"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 12 00:53:00.850554 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:53:00.850524    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3c451460-9cc7-49b0-ab4f-f361805f15d9-istio-podinfo\") pod \"3c451460-9cc7-49b0-ab4f-f361805f15d9\" (UID: \"3c451460-9cc7-49b0-ab4f-f361805f15d9\") "
Sep 12 00:53:00.850554 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:53:00.850526    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3c451460-9cc7-49b0-ab4f-f361805f15d9-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "3c451460-9cc7-49b0-ab4f-f361805f15d9" (UID: "3c451460-9cc7-49b0-ab4f-f361805f15d9"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 12 00:53:00.850991 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:53:00.850641    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3c451460-9cc7-49b0-ab4f-f361805f15d9-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "3c451460-9cc7-49b0-ab4f-f361805f15d9" (UID: "3c451460-9cc7-49b0-ab4f-f361805f15d9"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 12 00:53:00.850991 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:53:00.850858    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3c451460-9cc7-49b0-ab4f-f361805f15d9-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "3c451460-9cc7-49b0-ab4f-f361805f15d9" (UID: "3c451460-9cc7-49b0-ab4f-f361805f15d9"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 12 00:53:00.850991 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:53:00.850885    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3c451460-9cc7-49b0-ab4f-f361805f15d9-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "3c451460-9cc7-49b0-ab4f-f361805f15d9" (UID: "3c451460-9cc7-49b0-ab4f-f361805f15d9"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 12 00:53:00.850991 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:53:00.850895    2571 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3c451460-9cc7-49b0-ab4f-f361805f15d9-istio-data\") on node \"ip-10-0-141-128.ec2.internal\" DevicePath \"\""
Sep 12 00:53:00.850991 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:53:00.850926    2571 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3c451460-9cc7-49b0-ab4f-f361805f15d9-credential-socket\") on node \"ip-10-0-141-128.ec2.internal\" DevicePath \"\""
Sep 12 00:53:00.850991 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:53:00.850939    2571 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3c451460-9cc7-49b0-ab4f-f361805f15d9-workload-socket\") on node \"ip-10-0-141-128.ec2.internal\" DevicePath \"\""
Sep 12 00:53:00.852636 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:53:00.852612    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/3c451460-9cc7-49b0-ab4f-f361805f15d9-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "3c451460-9cc7-49b0-ab4f-f361805f15d9" (UID: "3c451460-9cc7-49b0-ab4f-f361805f15d9"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 12 00:53:00.852741 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:53:00.852676    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3c451460-9cc7-49b0-ab4f-f361805f15d9-istio-token" (OuterVolumeSpecName: "istio-token") pod "3c451460-9cc7-49b0-ab4f-f361805f15d9" (UID: "3c451460-9cc7-49b0-ab4f-f361805f15d9"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 12 00:53:00.853004 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:53:00.852979    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3c451460-9cc7-49b0-ab4f-f361805f15d9-kube-api-access-fchxc" (OuterVolumeSpecName: "kube-api-access-fchxc") pod "3c451460-9cc7-49b0-ab4f-f361805f15d9" (UID: "3c451460-9cc7-49b0-ab4f-f361805f15d9"). InnerVolumeSpecName "kube-api-access-fchxc". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 12 00:53:00.853004 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:53:00.852992    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3c451460-9cc7-49b0-ab4f-f361805f15d9-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "3c451460-9cc7-49b0-ab4f-f361805f15d9" (UID: "3c451460-9cc7-49b0-ab4f-f361805f15d9"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 12 00:53:00.951694 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:53:00.951671    2571 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3c451460-9cc7-49b0-ab4f-f361805f15d9-workload-certs\") on node \"ip-10-0-141-128.ec2.internal\" DevicePath \"\""
Sep 12 00:53:00.951694 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:53:00.951692    2571 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3c451460-9cc7-49b0-ab4f-f361805f15d9-istio-envoy\") on node \"ip-10-0-141-128.ec2.internal\" DevicePath \"\""
Sep 12 00:53:00.951822 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:53:00.951701    2571 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-fchxc\" (UniqueName: \"kubernetes.io/projected/3c451460-9cc7-49b0-ab4f-f361805f15d9-kube-api-access-fchxc\") on node \"ip-10-0-141-128.ec2.internal\" DevicePath \"\""
Sep 12 00:53:00.951822 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:53:00.951711    2571 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3c451460-9cc7-49b0-ab4f-f361805f15d9-istio-token\") on node \"ip-10-0-141-128.ec2.internal\" DevicePath \"\""
Sep 12 00:53:00.951822 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:53:00.951720    2571 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3c451460-9cc7-49b0-ab4f-f361805f15d9-istio-podinfo\") on node \"ip-10-0-141-128.ec2.internal\" DevicePath \"\""
Sep 12 00:53:00.951822 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:53:00.951727    2571 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3c451460-9cc7-49b0-ab4f-f361805f15d9-istiod-ca-cert\") on node \"ip-10-0-141-128.ec2.internal\" DevicePath \"\""
Sep 12 00:53:01.153807 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:53:01.153732    2571 generic.go:356] "Generic (PLEG): container finished" podID="3c451460-9cc7-49b0-ab4f-f361805f15d9" containerID="a6e83b7404c55c45e080af62c7987d2934007fca1b332efcf5d42d9729f7d9d1" exitCode=0
Sep 12 00:53:01.153807 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:53:01.153782    2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-b-1cbe1c-openshift-default-5fc97d867d-9hcqb"
Sep 12 00:53:01.153997 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:53:01.153808    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-b-1cbe1c-openshift-default-5fc97d867d-9hcqb" event={"ID":"3c451460-9cc7-49b0-ab4f-f361805f15d9","Type":"ContainerDied","Data":"a6e83b7404c55c45e080af62c7987d2934007fca1b332efcf5d42d9729f7d9d1"}
Sep 12 00:53:01.153997 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:53:01.153851    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-b-1cbe1c-openshift-default-5fc97d867d-9hcqb" event={"ID":"3c451460-9cc7-49b0-ab4f-f361805f15d9","Type":"ContainerDied","Data":"6a0bafe7f1dd957586465cf2c986de45a88b4e26f1d71b2fdcad7f1815def562"}
Sep 12 00:53:01.153997 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:53:01.153880    2571 scope.go:117] "RemoveContainer" containerID="a6e83b7404c55c45e080af62c7987d2934007fca1b332efcf5d42d9729f7d9d1"
Sep 12 00:53:01.163449 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:53:01.163435    2571 scope.go:117] "RemoveContainer" containerID="30e7b187ab20caa4d639899024e8aa5eff94c12dcc7101fefac51bd7eeaf9589"
Sep 12 00:53:01.172445 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:53:01.172417    2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ipp-b-1cbe1c-openshift-default-5fc97d867d-9hcqb"]
Sep 12 00:53:01.173028 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:53:01.173011    2571 scope.go:117] "RemoveContainer" containerID="a6e83b7404c55c45e080af62c7987d2934007fca1b332efcf5d42d9729f7d9d1"
Sep 12 00:53:01.173312 ip-10-0-141-128 kubenswrapper[2571]: E0912 00:53:01.173296    2571 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a6e83b7404c55c45e080af62c7987d2934007fca1b332efcf5d42d9729f7d9d1\": container with ID starting with a6e83b7404c55c45e080af62c7987d2934007fca1b332efcf5d42d9729f7d9d1 not found: ID does not exist" containerID="a6e83b7404c55c45e080af62c7987d2934007fca1b332efcf5d42d9729f7d9d1"
Sep 12 00:53:01.173367 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:53:01.173319    2571 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a6e83b7404c55c45e080af62c7987d2934007fca1b332efcf5d42d9729f7d9d1"} err="failed to get container status \"a6e83b7404c55c45e080af62c7987d2934007fca1b332efcf5d42d9729f7d9d1\": rpc error: code = NotFound desc = could not find container \"a6e83b7404c55c45e080af62c7987d2934007fca1b332efcf5d42d9729f7d9d1\": container with ID starting with a6e83b7404c55c45e080af62c7987d2934007fca1b332efcf5d42d9729f7d9d1 not found: ID does not exist"
Sep 12 00:53:01.173367 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:53:01.173334    2571 scope.go:117] "RemoveContainer" containerID="30e7b187ab20caa4d639899024e8aa5eff94c12dcc7101fefac51bd7eeaf9589"
Sep 12 00:53:01.173645 ip-10-0-141-128 kubenswrapper[2571]: E0912 00:53:01.173627    2571 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"30e7b187ab20caa4d639899024e8aa5eff94c12dcc7101fefac51bd7eeaf9589\": container with ID starting with 30e7b187ab20caa4d639899024e8aa5eff94c12dcc7101fefac51bd7eeaf9589 not found: ID does not exist" containerID="30e7b187ab20caa4d639899024e8aa5eff94c12dcc7101fefac51bd7eeaf9589"
Sep 12 00:53:01.173705 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:53:01.173651    2571 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"30e7b187ab20caa4d639899024e8aa5eff94c12dcc7101fefac51bd7eeaf9589"} err="failed to get container status \"30e7b187ab20caa4d639899024e8aa5eff94c12dcc7101fefac51bd7eeaf9589\": rpc error: code = NotFound desc = could not find container \"30e7b187ab20caa4d639899024e8aa5eff94c12dcc7101fefac51bd7eeaf9589\": container with ID starting with 30e7b187ab20caa4d639899024e8aa5eff94c12dcc7101fefac51bd7eeaf9589 not found: ID does not exist"
Sep 12 00:53:01.174564 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:53:01.174545    2571 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ipp-b-1cbe1c-openshift-default-5fc97d867d-9hcqb"]
Sep 12 00:53:03.141699 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:53:03.141667    2571 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3c451460-9cc7-49b0-ab4f-f361805f15d9" path="/var/lib/kubelet/pods/3c451460-9cc7-49b0-ab4f-f361805f15d9/volumes"
Sep 12 00:53:06.434878 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:53:06.434746    2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-e88680-56bc554c8b-j8dc8"]
Sep 12 00:53:06.435884 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:53:06.435816    2571 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-e88680-56bc554c8b-j8dc8" podUID="1c589235-b97a-4cc5-a75f-0670d30a25d0" containerName="maas-api" containerID="cri-o://4dfda39c63f69e0c4773acd57d48e00f2a5143d3575d636336931db8ceb998be" gracePeriod=30
Sep 12 00:53:06.706661 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:53:06.706639    2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-e88680-56bc554c8b-j8dc8"
Sep 12 00:53:06.797364 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:53:06.797329    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/1c589235-b97a-4cc5-a75f-0670d30a25d0-metrics-certs\") pod \"1c589235-b97a-4cc5-a75f-0670d30a25d0\" (UID: \"1c589235-b97a-4cc5-a75f-0670d30a25d0\") "
Sep 12 00:53:06.797539 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:53:06.797399    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/1c589235-b97a-4cc5-a75f-0670d30a25d0-openshift-service-ca\") pod \"1c589235-b97a-4cc5-a75f-0670d30a25d0\" (UID: \"1c589235-b97a-4cc5-a75f-0670d30a25d0\") "
Sep 12 00:53:06.797539 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:53:06.797437    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/1c589235-b97a-4cc5-a75f-0670d30a25d0-maas-api-tls\") pod \"1c589235-b97a-4cc5-a75f-0670d30a25d0\" (UID: \"1c589235-b97a-4cc5-a75f-0670d30a25d0\") "
Sep 12 00:53:06.797539 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:53:06.797462    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-sk7hz\" (UniqueName: \"kubernetes.io/projected/1c589235-b97a-4cc5-a75f-0670d30a25d0-kube-api-access-sk7hz\") pod \"1c589235-b97a-4cc5-a75f-0670d30a25d0\" (UID: \"1c589235-b97a-4cc5-a75f-0670d30a25d0\") "
Sep 12 00:53:06.797867 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:53:06.797837    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1c589235-b97a-4cc5-a75f-0670d30a25d0-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "1c589235-b97a-4cc5-a75f-0670d30a25d0" (UID: "1c589235-b97a-4cc5-a75f-0670d30a25d0"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 12 00:53:06.799578 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:53:06.799548    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1c589235-b97a-4cc5-a75f-0670d30a25d0-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "1c589235-b97a-4cc5-a75f-0670d30a25d0" (UID: "1c589235-b97a-4cc5-a75f-0670d30a25d0"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 12 00:53:06.799779 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:53:06.799758    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1c589235-b97a-4cc5-a75f-0670d30a25d0-kube-api-access-sk7hz" (OuterVolumeSpecName: "kube-api-access-sk7hz") pod "1c589235-b97a-4cc5-a75f-0670d30a25d0" (UID: "1c589235-b97a-4cc5-a75f-0670d30a25d0"). InnerVolumeSpecName "kube-api-access-sk7hz". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 12 00:53:06.799856 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:53:06.799830    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1c589235-b97a-4cc5-a75f-0670d30a25d0-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "1c589235-b97a-4cc5-a75f-0670d30a25d0" (UID: "1c589235-b97a-4cc5-a75f-0670d30a25d0"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 12 00:53:06.898314 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:53:06.898283    2571 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/1c589235-b97a-4cc5-a75f-0670d30a25d0-openshift-service-ca\") on node \"ip-10-0-141-128.ec2.internal\" DevicePath \"\""
Sep 12 00:53:06.898314 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:53:06.898311    2571 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/1c589235-b97a-4cc5-a75f-0670d30a25d0-maas-api-tls\") on node \"ip-10-0-141-128.ec2.internal\" DevicePath \"\""
Sep 12 00:53:06.898314 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:53:06.898320    2571 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-sk7hz\" (UniqueName: \"kubernetes.io/projected/1c589235-b97a-4cc5-a75f-0670d30a25d0-kube-api-access-sk7hz\") on node \"ip-10-0-141-128.ec2.internal\" DevicePath \"\""
Sep 12 00:53:06.898557 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:53:06.898330    2571 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/1c589235-b97a-4cc5-a75f-0670d30a25d0-metrics-certs\") on node \"ip-10-0-141-128.ec2.internal\" DevicePath \"\""
Sep 12 00:53:07.186759 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:53:07.186732    2571 generic.go:356] "Generic (PLEG): container finished" podID="1c589235-b97a-4cc5-a75f-0670d30a25d0" containerID="4dfda39c63f69e0c4773acd57d48e00f2a5143d3575d636336931db8ceb998be" exitCode=0
Sep 12 00:53:07.186907 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:53:07.186780    2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-e88680-56bc554c8b-j8dc8"
Sep 12 00:53:07.186907 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:53:07.186795    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-e88680-56bc554c8b-j8dc8" event={"ID":"1c589235-b97a-4cc5-a75f-0670d30a25d0","Type":"ContainerDied","Data":"4dfda39c63f69e0c4773acd57d48e00f2a5143d3575d636336931db8ceb998be"}
Sep 12 00:53:07.186907 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:53:07.186840    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-e88680-56bc554c8b-j8dc8" event={"ID":"1c589235-b97a-4cc5-a75f-0670d30a25d0","Type":"ContainerDied","Data":"f48a2d62a9f9579c847554937dfb0bfc606d29137658a84ab2cc20a0a3df326a"}
Sep 12 00:53:07.186907 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:53:07.186861    2571 scope.go:117] "RemoveContainer" containerID="4dfda39c63f69e0c4773acd57d48e00f2a5143d3575d636336931db8ceb998be"
Sep 12 00:53:07.195649 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:53:07.195628    2571 scope.go:117] "RemoveContainer" containerID="4dfda39c63f69e0c4773acd57d48e00f2a5143d3575d636336931db8ceb998be"
Sep 12 00:53:07.195914 ip-10-0-141-128 kubenswrapper[2571]: E0912 00:53:07.195897    2571 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4dfda39c63f69e0c4773acd57d48e00f2a5143d3575d636336931db8ceb998be\": container with ID starting with 4dfda39c63f69e0c4773acd57d48e00f2a5143d3575d636336931db8ceb998be not found: ID does not exist" containerID="4dfda39c63f69e0c4773acd57d48e00f2a5143d3575d636336931db8ceb998be"
Sep 12 00:53:07.195986 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:53:07.195922    2571 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4dfda39c63f69e0c4773acd57d48e00f2a5143d3575d636336931db8ceb998be"} err="failed to get container status \"4dfda39c63f69e0c4773acd57d48e00f2a5143d3575d636336931db8ceb998be\": rpc error: code = NotFound desc = could not find container \"4dfda39c63f69e0c4773acd57d48e00f2a5143d3575d636336931db8ceb998be\": container with ID starting with 4dfda39c63f69e0c4773acd57d48e00f2a5143d3575d636336931db8ceb998be not found: ID does not exist"
Sep 12 00:53:07.203163 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:53:07.203138    2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-e88680-56bc554c8b-j8dc8"]
Sep 12 00:53:07.208817 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:53:07.208798    2571 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-e88680-56bc554c8b-j8dc8"]
Sep 12 00:53:09.141802 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:53:09.141770    2571 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="1c589235-b97a-4cc5-a75f-0670d30a25d0" path="/var/lib/kubelet/pods/1c589235-b97a-4cc5-a75f-0670d30a25d0/volumes"
Sep 12 00:53:13.360056 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:53:13.360022    2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-a-w3-e88680-openshift-default-74bc5c555d-t5984"]
Sep 12 00:53:13.360538 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:53:13.360347    2571 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-shared-a-w3-e88680-openshift-default-74bc5c555d-t5984" podUID="e0489e12-24e5-436d-85c9-5a4fd75e2609" containerName="istio-proxy" containerID="cri-o://b8ff125b3eb02fd9991137f6f860052d5ebf423eeaf1457a5deb55d330f17d43" gracePeriod=30
Sep 12 00:53:16.116728 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:53:16.116692    2571 patch_prober.go:28] interesting pod/e2e-shared-a-w3-e88680-openshift-default-74bc5c555d-t5984 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Sep 12 00:53:16.117088 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:53:16.116763    2571 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-shared-a-w3-e88680-openshift-default-74bc5c555d-t5984" podUID="e0489e12-24e5-436d-85c9-5a4fd75e2609" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Sep 12 00:53:18.618400 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:53:18.618377    2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-w3-e88680-openshift-default-74bc5c555d-t5984"
Sep 12 00:53:18.800216 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:53:18.800184    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-tzcwd\" (UniqueName: \"kubernetes.io/projected/e0489e12-24e5-436d-85c9-5a4fd75e2609-kube-api-access-tzcwd\") pod \"e0489e12-24e5-436d-85c9-5a4fd75e2609\" (UID: \"e0489e12-24e5-436d-85c9-5a4fd75e2609\") "
Sep 12 00:53:18.800396 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:53:18.800279    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e0489e12-24e5-436d-85c9-5a4fd75e2609-istio-token\") pod \"e0489e12-24e5-436d-85c9-5a4fd75e2609\" (UID: \"e0489e12-24e5-436d-85c9-5a4fd75e2609\") "
Sep 12 00:53:18.800396 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:53:18.800307    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e0489e12-24e5-436d-85c9-5a4fd75e2609-istio-envoy\") pod \"e0489e12-24e5-436d-85c9-5a4fd75e2609\" (UID: \"e0489e12-24e5-436d-85c9-5a4fd75e2609\") "
Sep 12 00:53:18.800396 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:53:18.800334    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e0489e12-24e5-436d-85c9-5a4fd75e2609-istio-podinfo\") pod \"e0489e12-24e5-436d-85c9-5a4fd75e2609\" (UID: \"e0489e12-24e5-436d-85c9-5a4fd75e2609\") "
Sep 12 00:53:18.800396 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:53:18.800363    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e0489e12-24e5-436d-85c9-5a4fd75e2609-credential-socket\") pod \"e0489e12-24e5-436d-85c9-5a4fd75e2609\" (UID: \"e0489e12-24e5-436d-85c9-5a4fd75e2609\") "
Sep 12 00:53:18.800396 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:53:18.800394    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e0489e12-24e5-436d-85c9-5a4fd75e2609-workload-socket\") pod \"e0489e12-24e5-436d-85c9-5a4fd75e2609\" (UID: \"e0489e12-24e5-436d-85c9-5a4fd75e2609\") "
Sep 12 00:53:18.800713 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:53:18.800440    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e0489e12-24e5-436d-85c9-5a4fd75e2609-workload-certs\") pod \"e0489e12-24e5-436d-85c9-5a4fd75e2609\" (UID: \"e0489e12-24e5-436d-85c9-5a4fd75e2609\") "
Sep 12 00:53:18.800713 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:53:18.800497    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e0489e12-24e5-436d-85c9-5a4fd75e2609-istiod-ca-cert\") pod \"e0489e12-24e5-436d-85c9-5a4fd75e2609\" (UID: \"e0489e12-24e5-436d-85c9-5a4fd75e2609\") "
Sep 12 00:53:18.800713 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:53:18.800528    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e0489e12-24e5-436d-85c9-5a4fd75e2609-istio-data\") pod \"e0489e12-24e5-436d-85c9-5a4fd75e2609\" (UID: \"e0489e12-24e5-436d-85c9-5a4fd75e2609\") "
Sep 12 00:53:18.800865 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:53:18.800726    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e0489e12-24e5-436d-85c9-5a4fd75e2609-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "e0489e12-24e5-436d-85c9-5a4fd75e2609" (UID: "e0489e12-24e5-436d-85c9-5a4fd75e2609"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 12 00:53:18.800865 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:53:18.800750    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e0489e12-24e5-436d-85c9-5a4fd75e2609-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "e0489e12-24e5-436d-85c9-5a4fd75e2609" (UID: "e0489e12-24e5-436d-85c9-5a4fd75e2609"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 12 00:53:18.800975 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:53:18.800901    2571 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e0489e12-24e5-436d-85c9-5a4fd75e2609-credential-socket\") on node \"ip-10-0-141-128.ec2.internal\" DevicePath \"\""
Sep 12 00:53:18.800975 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:53:18.800923    2571 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e0489e12-24e5-436d-85c9-5a4fd75e2609-workload-socket\") on node \"ip-10-0-141-128.ec2.internal\" DevicePath \"\""
Sep 12 00:53:18.801082 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:53:18.800972    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e0489e12-24e5-436d-85c9-5a4fd75e2609-istio-data" (OuterVolumeSpecName: "istio-data") pod "e0489e12-24e5-436d-85c9-5a4fd75e2609" (UID: "e0489e12-24e5-436d-85c9-5a4fd75e2609"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 12 00:53:18.801134 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:53:18.801036    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e0489e12-24e5-436d-85c9-5a4fd75e2609-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "e0489e12-24e5-436d-85c9-5a4fd75e2609" (UID: "e0489e12-24e5-436d-85c9-5a4fd75e2609"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 12 00:53:18.801175 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:53:18.801114    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e0489e12-24e5-436d-85c9-5a4fd75e2609-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "e0489e12-24e5-436d-85c9-5a4fd75e2609" (UID: "e0489e12-24e5-436d-85c9-5a4fd75e2609"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 12 00:53:18.802452 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:53:18.802431    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e0489e12-24e5-436d-85c9-5a4fd75e2609-istio-token" (OuterVolumeSpecName: "istio-token") pod "e0489e12-24e5-436d-85c9-5a4fd75e2609" (UID: "e0489e12-24e5-436d-85c9-5a4fd75e2609"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 12 00:53:18.802618 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:53:18.802559    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e0489e12-24e5-436d-85c9-5a4fd75e2609-kube-api-access-tzcwd" (OuterVolumeSpecName: "kube-api-access-tzcwd") pod "e0489e12-24e5-436d-85c9-5a4fd75e2609" (UID: "e0489e12-24e5-436d-85c9-5a4fd75e2609"). InnerVolumeSpecName "kube-api-access-tzcwd". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 12 00:53:18.802839 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:53:18.802820    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/e0489e12-24e5-436d-85c9-5a4fd75e2609-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "e0489e12-24e5-436d-85c9-5a4fd75e2609" (UID: "e0489e12-24e5-436d-85c9-5a4fd75e2609"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 12 00:53:18.802898 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:53:18.802836    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e0489e12-24e5-436d-85c9-5a4fd75e2609-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "e0489e12-24e5-436d-85c9-5a4fd75e2609" (UID: "e0489e12-24e5-436d-85c9-5a4fd75e2609"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 12 00:53:18.901985 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:53:18.901961    2571 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e0489e12-24e5-436d-85c9-5a4fd75e2609-istio-token\") on node \"ip-10-0-141-128.ec2.internal\" DevicePath \"\""
Sep 12 00:53:18.901985 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:53:18.901982    2571 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e0489e12-24e5-436d-85c9-5a4fd75e2609-istio-envoy\") on node \"ip-10-0-141-128.ec2.internal\" DevicePath \"\""
Sep 12 00:53:18.901985 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:53:18.901990    2571 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e0489e12-24e5-436d-85c9-5a4fd75e2609-istio-podinfo\") on node \"ip-10-0-141-128.ec2.internal\" DevicePath \"\""
Sep 12 00:53:18.902167 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:53:18.901998    2571 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e0489e12-24e5-436d-85c9-5a4fd75e2609-workload-certs\") on node \"ip-10-0-141-128.ec2.internal\" DevicePath \"\""
Sep 12 00:53:18.902167 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:53:18.902007    2571 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e0489e12-24e5-436d-85c9-5a4fd75e2609-istiod-ca-cert\") on node \"ip-10-0-141-128.ec2.internal\" DevicePath \"\""
Sep 12 00:53:18.902167 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:53:18.902014    2571 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e0489e12-24e5-436d-85c9-5a4fd75e2609-istio-data\") on node \"ip-10-0-141-128.ec2.internal\" DevicePath \"\""
Sep 12 00:53:18.902167 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:53:18.902022    2571 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-tzcwd\" (UniqueName: \"kubernetes.io/projected/e0489e12-24e5-436d-85c9-5a4fd75e2609-kube-api-access-tzcwd\") on node \"ip-10-0-141-128.ec2.internal\" DevicePath \"\""
Sep 12 00:53:19.249366 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:53:19.249330    2571 generic.go:356] "Generic (PLEG): container finished" podID="e0489e12-24e5-436d-85c9-5a4fd75e2609" containerID="b8ff125b3eb02fd9991137f6f860052d5ebf423eeaf1457a5deb55d330f17d43" exitCode=0
Sep 12 00:53:19.249510 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:53:19.249372    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w3-e88680-openshift-default-74bc5c555d-t5984" event={"ID":"e0489e12-24e5-436d-85c9-5a4fd75e2609","Type":"ContainerDied","Data":"b8ff125b3eb02fd9991137f6f860052d5ebf423eeaf1457a5deb55d330f17d43"}
Sep 12 00:53:19.249510 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:53:19.249424    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w3-e88680-openshift-default-74bc5c555d-t5984" event={"ID":"e0489e12-24e5-436d-85c9-5a4fd75e2609","Type":"ContainerDied","Data":"98a190b7df758a530cde46ac7fa6107d7888f7be1a2857e8208e2c6bf77997f6"}
Sep 12 00:53:19.249510 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:53:19.249446    2571 scope.go:117] "RemoveContainer" containerID="b8ff125b3eb02fd9991137f6f860052d5ebf423eeaf1457a5deb55d330f17d43"
Sep 12 00:53:19.249510 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:53:19.249381    2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-w3-e88680-openshift-default-74bc5c555d-t5984"
Sep 12 00:53:19.258133 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:53:19.258118    2571 scope.go:117] "RemoveContainer" containerID="17a89e1f775a07982df255ca1aa9435ca98aade5e4cdd604ea39a5894274bf25"
Sep 12 00:53:19.266518 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:53:19.266456    2571 scope.go:117] "RemoveContainer" containerID="b8ff125b3eb02fd9991137f6f860052d5ebf423eeaf1457a5deb55d330f17d43"
Sep 12 00:53:19.267176 ip-10-0-141-128 kubenswrapper[2571]: E0912 00:53:19.267147    2571 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b8ff125b3eb02fd9991137f6f860052d5ebf423eeaf1457a5deb55d330f17d43\": container with ID starting with b8ff125b3eb02fd9991137f6f860052d5ebf423eeaf1457a5deb55d330f17d43 not found: ID does not exist" containerID="b8ff125b3eb02fd9991137f6f860052d5ebf423eeaf1457a5deb55d330f17d43"
Sep 12 00:53:19.267319 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:53:19.267186    2571 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b8ff125b3eb02fd9991137f6f860052d5ebf423eeaf1457a5deb55d330f17d43"} err="failed to get container status \"b8ff125b3eb02fd9991137f6f860052d5ebf423eeaf1457a5deb55d330f17d43\": rpc error: code = NotFound desc = could not find container \"b8ff125b3eb02fd9991137f6f860052d5ebf423eeaf1457a5deb55d330f17d43\": container with ID starting with b8ff125b3eb02fd9991137f6f860052d5ebf423eeaf1457a5deb55d330f17d43 not found: ID does not exist"
Sep 12 00:53:19.267392 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:53:19.267322    2571 scope.go:117] "RemoveContainer" containerID="17a89e1f775a07982df255ca1aa9435ca98aade5e4cdd604ea39a5894274bf25"
Sep 12 00:53:19.267803 ip-10-0-141-128 kubenswrapper[2571]: E0912 00:53:19.267774    2571 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"17a89e1f775a07982df255ca1aa9435ca98aade5e4cdd604ea39a5894274bf25\": container with ID starting with 17a89e1f775a07982df255ca1aa9435ca98aade5e4cdd604ea39a5894274bf25 not found: ID does not exist" containerID="17a89e1f775a07982df255ca1aa9435ca98aade5e4cdd604ea39a5894274bf25"
Sep 12 00:53:19.267934 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:53:19.267813    2571 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"17a89e1f775a07982df255ca1aa9435ca98aade5e4cdd604ea39a5894274bf25"} err="failed to get container status \"17a89e1f775a07982df255ca1aa9435ca98aade5e4cdd604ea39a5894274bf25\": rpc error: code = NotFound desc = could not find container \"17a89e1f775a07982df255ca1aa9435ca98aade5e4cdd604ea39a5894274bf25\": container with ID starting with 17a89e1f775a07982df255ca1aa9435ca98aade5e4cdd604ea39a5894274bf25 not found: ID does not exist"
Sep 12 00:53:19.269516 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:53:19.269489    2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-a-w3-e88680-openshift-default-74bc5c555d-t5984"]
Sep 12 00:53:19.274930 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:53:19.274910    2571 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-shared-a-w3-e88680-openshift-default-74bc5c555d-t5984"]
Sep 12 00:53:21.140660 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:53:21.140629    2571 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e0489e12-24e5-436d-85c9-5a4fd75e2609" path="/var/lib/kubelet/pods/e0489e12-24e5-436d-85c9-5a4fd75e2609/volumes"
Sep 12 00:53:24.211818 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:53:24.211788    2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w3-e37bfb-54fd9bcc55-6x788"]
Sep 12 00:53:24.212248 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:53:24.211994    2571 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-shared-b-w3-e37bfb-54fd9bcc55-6x788" podUID="96f2b972-193f-41c7-aee5-20da72c3eb14" containerName="payload-processing" containerID="cri-o://288a263cdf7d093fbe5dbb0e8b60afb61f82e2a92dd6b3dc12247789379d0779" gracePeriod=30
Sep 12 00:53:24.224411 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:53:24.223760    2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w3-e37bfb-756cdf47bb-5xkrt"]
Sep 12 00:53:24.224411 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:53:24.223979    2571 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-e37bfb-756cdf47bb-5xkrt" podUID="e238a42c-0d7c-4737-88ba-c05d757287b6" containerName="payload-pre-processing" containerID="cri-o://15f13df7ab16b7704c29b987e9300c53fd91376c87a45de2337c8596f6de50d1" gracePeriod=30
Sep 12 00:53:24.477178 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:53:24.477114    2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-b-w3-e37bfb-54fd9bcc55-6x788"
Sep 12 00:53:24.501211 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:53:24.501184    2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-e37bfb-756cdf47bb-5xkrt"
Sep 12 00:53:24.644949 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:53:24.644916    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-td6hf\" (UniqueName: \"kubernetes.io/projected/96f2b972-193f-41c7-aee5-20da72c3eb14-kube-api-access-td6hf\") pod \"96f2b972-193f-41c7-aee5-20da72c3eb14\" (UID: \"96f2b972-193f-41c7-aee5-20da72c3eb14\") "
Sep 12 00:53:24.645135 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:53:24.644958    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/96f2b972-193f-41c7-aee5-20da72c3eb14-plugins-config-volume\") pod \"96f2b972-193f-41c7-aee5-20da72c3eb14\" (UID: \"96f2b972-193f-41c7-aee5-20da72c3eb14\") "
Sep 12 00:53:24.645135 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:53:24.645001    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-2hthn\" (UniqueName: \"kubernetes.io/projected/e238a42c-0d7c-4737-88ba-c05d757287b6-kube-api-access-2hthn\") pod \"e238a42c-0d7c-4737-88ba-c05d757287b6\" (UID: \"e238a42c-0d7c-4737-88ba-c05d757287b6\") "
Sep 12 00:53:24.645135 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:53:24.645069    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e238a42c-0d7c-4737-88ba-c05d757287b6-plugins-config-volume\") pod \"e238a42c-0d7c-4737-88ba-c05d757287b6\" (UID: \"e238a42c-0d7c-4737-88ba-c05d757287b6\") "
Sep 12 00:53:24.645315 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:53:24.645262    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/96f2b972-193f-41c7-aee5-20da72c3eb14-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "96f2b972-193f-41c7-aee5-20da72c3eb14" (UID: "96f2b972-193f-41c7-aee5-20da72c3eb14"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 12 00:53:24.645456 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:53:24.645434    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e238a42c-0d7c-4737-88ba-c05d757287b6-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "e238a42c-0d7c-4737-88ba-c05d757287b6" (UID: "e238a42c-0d7c-4737-88ba-c05d757287b6"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 12 00:53:24.647098 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:53:24.647077    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e238a42c-0d7c-4737-88ba-c05d757287b6-kube-api-access-2hthn" (OuterVolumeSpecName: "kube-api-access-2hthn") pod "e238a42c-0d7c-4737-88ba-c05d757287b6" (UID: "e238a42c-0d7c-4737-88ba-c05d757287b6"). InnerVolumeSpecName "kube-api-access-2hthn". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 12 00:53:24.647173 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:53:24.647119    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/96f2b972-193f-41c7-aee5-20da72c3eb14-kube-api-access-td6hf" (OuterVolumeSpecName: "kube-api-access-td6hf") pod "96f2b972-193f-41c7-aee5-20da72c3eb14" (UID: "96f2b972-193f-41c7-aee5-20da72c3eb14"). InnerVolumeSpecName "kube-api-access-td6hf". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 12 00:53:24.746042 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:53:24.745971    2571 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e238a42c-0d7c-4737-88ba-c05d757287b6-plugins-config-volume\") on node \"ip-10-0-141-128.ec2.internal\" DevicePath \"\""
Sep 12 00:53:24.746042 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:53:24.745994    2571 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-td6hf\" (UniqueName: \"kubernetes.io/projected/96f2b972-193f-41c7-aee5-20da72c3eb14-kube-api-access-td6hf\") on node \"ip-10-0-141-128.ec2.internal\" DevicePath \"\""
Sep 12 00:53:24.746042 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:53:24.746003    2571 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/96f2b972-193f-41c7-aee5-20da72c3eb14-plugins-config-volume\") on node \"ip-10-0-141-128.ec2.internal\" DevicePath \"\""
Sep 12 00:53:24.746042 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:53:24.746013    2571 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2hthn\" (UniqueName: \"kubernetes.io/projected/e238a42c-0d7c-4737-88ba-c05d757287b6-kube-api-access-2hthn\") on node \"ip-10-0-141-128.ec2.internal\" DevicePath \"\""
Sep 12 00:53:25.282491 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:53:25.282456    2571 generic.go:356] "Generic (PLEG): container finished" podID="e238a42c-0d7c-4737-88ba-c05d757287b6" containerID="15f13df7ab16b7704c29b987e9300c53fd91376c87a45de2337c8596f6de50d1" exitCode=0
Sep 12 00:53:25.282893 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:53:25.282498    2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-e37bfb-756cdf47bb-5xkrt"
Sep 12 00:53:25.282893 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:53:25.282547    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-e37bfb-756cdf47bb-5xkrt" event={"ID":"e238a42c-0d7c-4737-88ba-c05d757287b6","Type":"ContainerDied","Data":"15f13df7ab16b7704c29b987e9300c53fd91376c87a45de2337c8596f6de50d1"}
Sep 12 00:53:25.282893 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:53:25.282632    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-e37bfb-756cdf47bb-5xkrt" event={"ID":"e238a42c-0d7c-4737-88ba-c05d757287b6","Type":"ContainerDied","Data":"3f8dee12860f401361c8d1ea7bddddc7b2743852ba5055c03f2f55e18fcd0801"}
Sep 12 00:53:25.282893 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:53:25.282666    2571 scope.go:117] "RemoveContainer" containerID="15f13df7ab16b7704c29b987e9300c53fd91376c87a45de2337c8596f6de50d1"
Sep 12 00:53:25.283991 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:53:25.283971    2571 generic.go:356] "Generic (PLEG): container finished" podID="96f2b972-193f-41c7-aee5-20da72c3eb14" containerID="288a263cdf7d093fbe5dbb0e8b60afb61f82e2a92dd6b3dc12247789379d0779" exitCode=0
Sep 12 00:53:25.284066 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:53:25.284002    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-b-w3-e37bfb-54fd9bcc55-6x788" event={"ID":"96f2b972-193f-41c7-aee5-20da72c3eb14","Type":"ContainerDied","Data":"288a263cdf7d093fbe5dbb0e8b60afb61f82e2a92dd6b3dc12247789379d0779"}
Sep 12 00:53:25.284066 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:53:25.284023    2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-b-w3-e37bfb-54fd9bcc55-6x788"
Sep 12 00:53:25.284066 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:53:25.284049    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-b-w3-e37bfb-54fd9bcc55-6x788" event={"ID":"96f2b972-193f-41c7-aee5-20da72c3eb14","Type":"ContainerDied","Data":"a6dc79406d25c64a504020e51410faaeff95a900eb385864fa12ce767a0ac65c"}
Sep 12 00:53:25.291149 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:53:25.291134    2571 scope.go:117] "RemoveContainer" containerID="15f13df7ab16b7704c29b987e9300c53fd91376c87a45de2337c8596f6de50d1"
Sep 12 00:53:25.291398 ip-10-0-141-128 kubenswrapper[2571]: E0912 00:53:25.291381    2571 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"15f13df7ab16b7704c29b987e9300c53fd91376c87a45de2337c8596f6de50d1\": container with ID starting with 15f13df7ab16b7704c29b987e9300c53fd91376c87a45de2337c8596f6de50d1 not found: ID does not exist" containerID="15f13df7ab16b7704c29b987e9300c53fd91376c87a45de2337c8596f6de50d1"
Sep 12 00:53:25.291466 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:53:25.291409    2571 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"15f13df7ab16b7704c29b987e9300c53fd91376c87a45de2337c8596f6de50d1"} err="failed to get container status \"15f13df7ab16b7704c29b987e9300c53fd91376c87a45de2337c8596f6de50d1\": rpc error: code = NotFound desc = could not find container \"15f13df7ab16b7704c29b987e9300c53fd91376c87a45de2337c8596f6de50d1\": container with ID starting with 15f13df7ab16b7704c29b987e9300c53fd91376c87a45de2337c8596f6de50d1 not found: ID does not exist"
Sep 12 00:53:25.291466 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:53:25.291431    2571 scope.go:117] "RemoveContainer" containerID="288a263cdf7d093fbe5dbb0e8b60afb61f82e2a92dd6b3dc12247789379d0779"
Sep 12 00:53:25.299263 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:53:25.299246    2571 scope.go:117] "RemoveContainer" containerID="288a263cdf7d093fbe5dbb0e8b60afb61f82e2a92dd6b3dc12247789379d0779"
Sep 12 00:53:25.299525 ip-10-0-141-128 kubenswrapper[2571]: E0912 00:53:25.299507    2571 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"288a263cdf7d093fbe5dbb0e8b60afb61f82e2a92dd6b3dc12247789379d0779\": container with ID starting with 288a263cdf7d093fbe5dbb0e8b60afb61f82e2a92dd6b3dc12247789379d0779 not found: ID does not exist" containerID="288a263cdf7d093fbe5dbb0e8b60afb61f82e2a92dd6b3dc12247789379d0779"
Sep 12 00:53:25.299573 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:53:25.299532    2571 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"288a263cdf7d093fbe5dbb0e8b60afb61f82e2a92dd6b3dc12247789379d0779"} err="failed to get container status \"288a263cdf7d093fbe5dbb0e8b60afb61f82e2a92dd6b3dc12247789379d0779\": rpc error: code = NotFound desc = could not find container \"288a263cdf7d093fbe5dbb0e8b60afb61f82e2a92dd6b3dc12247789379d0779\": container with ID starting with 288a263cdf7d093fbe5dbb0e8b60afb61f82e2a92dd6b3dc12247789379d0779 not found: ID does not exist"
Sep 12 00:53:25.310273 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:53:25.310244    2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w3-e37bfb-54fd9bcc55-6x788"]
Sep 12 00:53:25.317246 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:53:25.317227    2571 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w3-e37bfb-54fd9bcc55-6x788"]
Sep 12 00:53:25.327802 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:53:25.327784    2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w3-e37bfb-756cdf47bb-5xkrt"]
Sep 12 00:53:25.331626 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:53:25.331605    2571 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w3-e37bfb-756cdf47bb-5xkrt"]
Sep 12 00:53:27.141319 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:53:27.141287    2571 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="96f2b972-193f-41c7-aee5-20da72c3eb14" path="/var/lib/kubelet/pods/96f2b972-193f-41c7-aee5-20da72c3eb14/volumes"
Sep 12 00:53:27.141732 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:53:27.141716    2571 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e238a42c-0d7c-4737-88ba-c05d757287b6" path="/var/lib/kubelet/pods/e238a42c-0d7c-4737-88ba-c05d757287b6/volumes"
Sep 12 00:53:33.816176 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:53:33.815878    2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-57d99b475d-b9nc7"]
Sep 12 00:53:33.816176 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:53:33.816147    2571 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/maas-controller-57d99b475d-b9nc7" podUID="0759af38-2732-4bb6-b33b-4fa561ea1486" containerName="manager" containerID="cri-o://df1d323fdc706a416b1d2807f06fb501c0083fee9a7be91704d37e5544bc8878" gracePeriod=30
Sep 12 00:53:34.060555 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:53:34.060533    2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-57d99b475d-b9nc7"
Sep 12 00:53:34.111987 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:53:34.111910    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/0759af38-2732-4bb6-b33b-4fa561ea1486-webhook-cert\") pod \"0759af38-2732-4bb6-b33b-4fa561ea1486\" (UID: \"0759af38-2732-4bb6-b33b-4fa561ea1486\") "
Sep 12 00:53:34.111987 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:53:34.111962    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/0759af38-2732-4bb6-b33b-4fa561ea1486-metrics-certs\") pod \"0759af38-2732-4bb6-b33b-4fa561ea1486\" (UID: \"0759af38-2732-4bb6-b33b-4fa561ea1486\") "
Sep 12 00:53:34.112172 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:53:34.112020    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-qr88c\" (UniqueName: \"kubernetes.io/projected/0759af38-2732-4bb6-b33b-4fa561ea1486-kube-api-access-qr88c\") pod \"0759af38-2732-4bb6-b33b-4fa561ea1486\" (UID: \"0759af38-2732-4bb6-b33b-4fa561ea1486\") "
Sep 12 00:53:34.114030 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:53:34.113998    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0759af38-2732-4bb6-b33b-4fa561ea1486-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "0759af38-2732-4bb6-b33b-4fa561ea1486" (UID: "0759af38-2732-4bb6-b33b-4fa561ea1486"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 12 00:53:34.114136 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:53:34.114025    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0759af38-2732-4bb6-b33b-4fa561ea1486-kube-api-access-qr88c" (OuterVolumeSpecName: "kube-api-access-qr88c") pod "0759af38-2732-4bb6-b33b-4fa561ea1486" (UID: "0759af38-2732-4bb6-b33b-4fa561ea1486"). InnerVolumeSpecName "kube-api-access-qr88c". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 12 00:53:34.114136 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:53:34.114012    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0759af38-2732-4bb6-b33b-4fa561ea1486-webhook-cert" (OuterVolumeSpecName: "webhook-cert") pod "0759af38-2732-4bb6-b33b-4fa561ea1486" (UID: "0759af38-2732-4bb6-b33b-4fa561ea1486"). InnerVolumeSpecName "webhook-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 12 00:53:34.213195 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:53:34.213161    2571 reconciler_common.go:299] "Volume detached for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/0759af38-2732-4bb6-b33b-4fa561ea1486-webhook-cert\") on node \"ip-10-0-141-128.ec2.internal\" DevicePath \"\""
Sep 12 00:53:34.213195 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:53:34.213189    2571 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/0759af38-2732-4bb6-b33b-4fa561ea1486-metrics-certs\") on node \"ip-10-0-141-128.ec2.internal\" DevicePath \"\""
Sep 12 00:53:34.213368 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:53:34.213203    2571 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qr88c\" (UniqueName: \"kubernetes.io/projected/0759af38-2732-4bb6-b33b-4fa561ea1486-kube-api-access-qr88c\") on node \"ip-10-0-141-128.ec2.internal\" DevicePath \"\""
Sep 12 00:53:34.332504 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:53:34.332472    2571 generic.go:356] "Generic (PLEG): container finished" podID="0759af38-2732-4bb6-b33b-4fa561ea1486" containerID="df1d323fdc706a416b1d2807f06fb501c0083fee9a7be91704d37e5544bc8878" exitCode=0
Sep 12 00:53:34.332666 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:53:34.332516    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-57d99b475d-b9nc7" event={"ID":"0759af38-2732-4bb6-b33b-4fa561ea1486","Type":"ContainerDied","Data":"df1d323fdc706a416b1d2807f06fb501c0083fee9a7be91704d37e5544bc8878"}
Sep 12 00:53:34.332666 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:53:34.332566    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-57d99b475d-b9nc7" event={"ID":"0759af38-2732-4bb6-b33b-4fa561ea1486","Type":"ContainerDied","Data":"14b3bb08e877fef60f076a425adbcb3a59466b9ffd334a59045a6d592efb4591"}
Sep 12 00:53:34.332666 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:53:34.332522    2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-57d99b475d-b9nc7"
Sep 12 00:53:34.332666 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:53:34.332610    2571 scope.go:117] "RemoveContainer" containerID="df1d323fdc706a416b1d2807f06fb501c0083fee9a7be91704d37e5544bc8878"
Sep 12 00:53:34.342247 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:53:34.342226    2571 scope.go:117] "RemoveContainer" containerID="df1d323fdc706a416b1d2807f06fb501c0083fee9a7be91704d37e5544bc8878"
Sep 12 00:53:34.342497 ip-10-0-141-128 kubenswrapper[2571]: E0912 00:53:34.342480    2571 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"df1d323fdc706a416b1d2807f06fb501c0083fee9a7be91704d37e5544bc8878\": container with ID starting with df1d323fdc706a416b1d2807f06fb501c0083fee9a7be91704d37e5544bc8878 not found: ID does not exist" containerID="df1d323fdc706a416b1d2807f06fb501c0083fee9a7be91704d37e5544bc8878"
Sep 12 00:53:34.342546 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:53:34.342505    2571 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"df1d323fdc706a416b1d2807f06fb501c0083fee9a7be91704d37e5544bc8878"} err="failed to get container status \"df1d323fdc706a416b1d2807f06fb501c0083fee9a7be91704d37e5544bc8878\": rpc error: code = NotFound desc = could not find container \"df1d323fdc706a416b1d2807f06fb501c0083fee9a7be91704d37e5544bc8878\": container with ID starting with df1d323fdc706a416b1d2807f06fb501c0083fee9a7be91704d37e5544bc8878 not found: ID does not exist"
Sep 12 00:53:34.353835 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:53:34.353813    2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-57d99b475d-b9nc7"]
Sep 12 00:53:34.357535 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:53:34.357514    2571 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-57d99b475d-b9nc7"]
Sep 12 00:53:35.140522 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:53:35.140489    2571 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0759af38-2732-4bb6-b33b-4fa561ea1486" path="/var/lib/kubelet/pods/0759af38-2732-4bb6-b33b-4fa561ea1486/volumes"
Sep 12 00:54:35.189849 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:54:35.189810    2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-57d99b475d-nwgsr"]
Sep 12 00:54:35.190436 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:54:35.190266    2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1c589235-b97a-4cc5-a75f-0670d30a25d0" containerName="maas-api"
Sep 12 00:54:35.190436 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:54:35.190288    2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="1c589235-b97a-4cc5-a75f-0670d30a25d0" containerName="maas-api"
Sep 12 00:54:35.190436 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:54:35.190310    2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0759af38-2732-4bb6-b33b-4fa561ea1486" containerName="manager"
Sep 12 00:54:35.190436 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:54:35.190319    2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="0759af38-2732-4bb6-b33b-4fa561ea1486" containerName="manager"
Sep 12 00:54:35.190436 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:54:35.190330    2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="bb470798-740b-44be-a5dd-48febecd5d30" containerName="payload-pre-processing"
Sep 12 00:54:35.190436 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:54:35.190339    2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="bb470798-740b-44be-a5dd-48febecd5d30" containerName="payload-pre-processing"
Sep 12 00:54:35.190436 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:54:35.190352    2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ee555fcf-4c74-4b63-ad98-e9d5c4976697" containerName="istio-proxy"
Sep 12 00:54:35.190436 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:54:35.190360    2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="ee555fcf-4c74-4b63-ad98-e9d5c4976697" containerName="istio-proxy"
Sep 12 00:54:35.190436 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:54:35.190374    2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d73ec31b-c54e-44df-a448-0a914b938eb7" containerName="istio-proxy"
Sep 12 00:54:35.190436 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:54:35.190383    2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="d73ec31b-c54e-44df-a448-0a914b938eb7" containerName="istio-proxy"
Sep 12 00:54:35.190436 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:54:35.190396    2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e238a42c-0d7c-4737-88ba-c05d757287b6" containerName="payload-pre-processing"
Sep 12 00:54:35.190436 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:54:35.190405    2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="e238a42c-0d7c-4737-88ba-c05d757287b6" containerName="payload-pre-processing"
Sep 12 00:54:35.190436 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:54:35.190421    2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3c451460-9cc7-49b0-ab4f-f361805f15d9" containerName="istio-proxy"
Sep 12 00:54:35.190436 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:54:35.190429    2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="3c451460-9cc7-49b0-ab4f-f361805f15d9" containerName="istio-proxy"
Sep 12 00:54:35.191030 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:54:35.190448    2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ee555fcf-4c74-4b63-ad98-e9d5c4976697" containerName="istio-proxy"
Sep 12 00:54:35.191030 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:54:35.190456    2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="ee555fcf-4c74-4b63-ad98-e9d5c4976697" containerName="istio-proxy"
Sep 12 00:54:35.191030 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:54:35.190465    2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e0489e12-24e5-436d-85c9-5a4fd75e2609" containerName="istio-proxy"
Sep 12 00:54:35.191030 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:54:35.190473    2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="e0489e12-24e5-436d-85c9-5a4fd75e2609" containerName="istio-proxy"
Sep 12 00:54:35.191030 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:54:35.190483    2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="96f2b972-193f-41c7-aee5-20da72c3eb14" containerName="payload-processing"
Sep 12 00:54:35.191030 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:54:35.190491    2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="96f2b972-193f-41c7-aee5-20da72c3eb14" containerName="payload-processing"
Sep 12 00:54:35.191030 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:54:35.190511    2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e0489e12-24e5-436d-85c9-5a4fd75e2609" containerName="istio-proxy"
Sep 12 00:54:35.191030 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:54:35.190519    2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="e0489e12-24e5-436d-85c9-5a4fd75e2609" containerName="istio-proxy"
Sep 12 00:54:35.191030 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:54:35.190527    2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d73ec31b-c54e-44df-a448-0a914b938eb7" containerName="istio-proxy"
Sep 12 00:54:35.191030 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:54:35.190535    2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="d73ec31b-c54e-44df-a448-0a914b938eb7" containerName="istio-proxy"
Sep 12 00:54:35.191030 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:54:35.190657    2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="0759af38-2732-4bb6-b33b-4fa561ea1486" containerName="manager"
Sep 12 00:54:35.191030 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:54:35.190671    2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="1c589235-b97a-4cc5-a75f-0670d30a25d0" containerName="maas-api"
Sep 12 00:54:35.191030 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:54:35.190679    2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="ee555fcf-4c74-4b63-ad98-e9d5c4976697" containerName="istio-proxy"
Sep 12 00:54:35.191030 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:54:35.190690    2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="3c451460-9cc7-49b0-ab4f-f361805f15d9" containerName="istio-proxy"
Sep 12 00:54:35.191030 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:54:35.190699    2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="3c451460-9cc7-49b0-ab4f-f361805f15d9" containerName="istio-proxy"
Sep 12 00:54:35.191030 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:54:35.190706    2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="e0489e12-24e5-436d-85c9-5a4fd75e2609" containerName="istio-proxy"
Sep 12 00:54:35.191030 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:54:35.190711    2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="bb470798-740b-44be-a5dd-48febecd5d30" containerName="payload-pre-processing"
Sep 12 00:54:35.191030 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:54:35.190717    2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="96f2b972-193f-41c7-aee5-20da72c3eb14" containerName="payload-processing"
Sep 12 00:54:35.191030 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:54:35.190723    2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="d73ec31b-c54e-44df-a448-0a914b938eb7" containerName="istio-proxy"
Sep 12 00:54:35.191030 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:54:35.190728    2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="d73ec31b-c54e-44df-a448-0a914b938eb7" containerName="istio-proxy"
Sep 12 00:54:35.191030 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:54:35.190734    2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="e238a42c-0d7c-4737-88ba-c05d757287b6" containerName="payload-pre-processing"
Sep 12 00:54:35.194987 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:54:35.194970    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-57d99b475d-nwgsr"
Sep 12 00:54:35.198902 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:54:35.198883    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-metrics-tls\""
Sep 12 00:54:35.199231 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:54:35.199206    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-km42l\""
Sep 12 00:54:35.199358 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:54:35.199338    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\""
Sep 12 00:54:35.199461 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:54:35.199351    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\""
Sep 12 00:54:35.205934 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:54:35.205911    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-57d99b475d-nwgsr"]
Sep 12 00:54:35.234824 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:54:35.234792    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/9ae90dbd-7aea-4448-9356-d202ce7df1c3-metrics-certs\") pod \"maas-controller-57d99b475d-nwgsr\" (UID: \"9ae90dbd-7aea-4448-9356-d202ce7df1c3\") " pod="opendatahub/maas-controller-57d99b475d-nwgsr"
Sep 12 00:54:35.234979 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:54:35.234830    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-trd7d\" (UniqueName: \"kubernetes.io/projected/9ae90dbd-7aea-4448-9356-d202ce7df1c3-kube-api-access-trd7d\") pod \"maas-controller-57d99b475d-nwgsr\" (UID: \"9ae90dbd-7aea-4448-9356-d202ce7df1c3\") " pod="opendatahub/maas-controller-57d99b475d-nwgsr"
Sep 12 00:54:35.234979 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:54:35.234893    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/9ae90dbd-7aea-4448-9356-d202ce7df1c3-webhook-cert\") pod \"maas-controller-57d99b475d-nwgsr\" (UID: \"9ae90dbd-7aea-4448-9356-d202ce7df1c3\") " pod="opendatahub/maas-controller-57d99b475d-nwgsr"
Sep 12 00:54:35.335698 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:54:35.335657    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/9ae90dbd-7aea-4448-9356-d202ce7df1c3-metrics-certs\") pod \"maas-controller-57d99b475d-nwgsr\" (UID: \"9ae90dbd-7aea-4448-9356-d202ce7df1c3\") " pod="opendatahub/maas-controller-57d99b475d-nwgsr"
Sep 12 00:54:35.335698 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:54:35.335703    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-trd7d\" (UniqueName: \"kubernetes.io/projected/9ae90dbd-7aea-4448-9356-d202ce7df1c3-kube-api-access-trd7d\") pod \"maas-controller-57d99b475d-nwgsr\" (UID: \"9ae90dbd-7aea-4448-9356-d202ce7df1c3\") " pod="opendatahub/maas-controller-57d99b475d-nwgsr"
Sep 12 00:54:35.335916 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:54:35.335765    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/9ae90dbd-7aea-4448-9356-d202ce7df1c3-webhook-cert\") pod \"maas-controller-57d99b475d-nwgsr\" (UID: \"9ae90dbd-7aea-4448-9356-d202ce7df1c3\") " pod="opendatahub/maas-controller-57d99b475d-nwgsr"
Sep 12 00:54:35.338194 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:54:35.338168    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/9ae90dbd-7aea-4448-9356-d202ce7df1c3-webhook-cert\") pod \"maas-controller-57d99b475d-nwgsr\" (UID: \"9ae90dbd-7aea-4448-9356-d202ce7df1c3\") " pod="opendatahub/maas-controller-57d99b475d-nwgsr"
Sep 12 00:54:35.338313 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:54:35.338229    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/9ae90dbd-7aea-4448-9356-d202ce7df1c3-metrics-certs\") pod \"maas-controller-57d99b475d-nwgsr\" (UID: \"9ae90dbd-7aea-4448-9356-d202ce7df1c3\") " pod="opendatahub/maas-controller-57d99b475d-nwgsr"
Sep 12 00:54:35.344347 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:54:35.344328    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-trd7d\" (UniqueName: \"kubernetes.io/projected/9ae90dbd-7aea-4448-9356-d202ce7df1c3-kube-api-access-trd7d\") pod \"maas-controller-57d99b475d-nwgsr\" (UID: \"9ae90dbd-7aea-4448-9356-d202ce7df1c3\") " pod="opendatahub/maas-controller-57d99b475d-nwgsr"
Sep 12 00:54:35.509918 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:54:35.509833    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-57d99b475d-nwgsr"
Sep 12 00:54:35.781941 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:54:35.781918    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-57d99b475d-nwgsr"]
Sep 12 00:54:35.784138 ip-10-0-141-128 kubenswrapper[2571]: W0912 00:54:35.784106    2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9ae90dbd_7aea_4448_9356_d202ce7df1c3.slice/crio-360b0094338ff44622f9f9a07289d349be5d8abd6436883438a7b223d76a97a8 WatchSource:0}: Error finding container 360b0094338ff44622f9f9a07289d349be5d8abd6436883438a7b223d76a97a8: Status 404 returned error can't find the container with id 360b0094338ff44622f9f9a07289d349be5d8abd6436883438a7b223d76a97a8
Sep 12 00:54:35.785702 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:54:35.785682    2571 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 12 00:54:36.632288 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:54:36.632200    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-57d99b475d-nwgsr" event={"ID":"9ae90dbd-7aea-4448-9356-d202ce7df1c3","Type":"ContainerStarted","Data":"b6f52b353a4428aee8ddff51a3e79035066971589a5bd93c78d5128737c9ee8b"}
Sep 12 00:54:36.632288 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:54:36.632266    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-57d99b475d-nwgsr" event={"ID":"9ae90dbd-7aea-4448-9356-d202ce7df1c3","Type":"ContainerStarted","Data":"360b0094338ff44622f9f9a07289d349be5d8abd6436883438a7b223d76a97a8"}
Sep 12 00:54:36.632708 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:54:36.632318    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-57d99b475d-nwgsr"
Sep 12 00:54:36.650115 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:54:36.650067    2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-57d99b475d-nwgsr" podStartSLOduration=1.246605854 podStartE2EDuration="1.650051076s" totalImagesPullingTime="403.445222ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-12 00:54:35 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-12 00:54:36.648771378 +0000 UTC m=+2287.920411889" watchObservedRunningTime="2026-09-12 00:54:36.650051076 +0000 UTC m=+2287.921691609"
Sep 12 00:54:47.640977 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:54:47.640945    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-57d99b475d-nwgsr"
Sep 12 00:54:57.061340 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:54:57.061300    2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-57d99b475d-nwgsr"]
Sep 12 00:54:57.061849 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:54:57.061658    2571 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/maas-controller-57d99b475d-nwgsr" podUID="9ae90dbd-7aea-4448-9356-d202ce7df1c3" containerName="manager" containerID="cri-o://b6f52b353a4428aee8ddff51a3e79035066971589a5bd93c78d5128737c9ee8b" gracePeriod=30
Sep 12 00:54:57.304647 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:54:57.304620    2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-57d99b475d-nwgsr"
Sep 12 00:54:57.430371 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:54:57.430342    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-trd7d\" (UniqueName: \"kubernetes.io/projected/9ae90dbd-7aea-4448-9356-d202ce7df1c3-kube-api-access-trd7d\") pod \"9ae90dbd-7aea-4448-9356-d202ce7df1c3\" (UID: \"9ae90dbd-7aea-4448-9356-d202ce7df1c3\") "
Sep 12 00:54:57.430548 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:54:57.430391    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/9ae90dbd-7aea-4448-9356-d202ce7df1c3-metrics-certs\") pod \"9ae90dbd-7aea-4448-9356-d202ce7df1c3\" (UID: \"9ae90dbd-7aea-4448-9356-d202ce7df1c3\") "
Sep 12 00:54:57.430548 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:54:57.430515    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/9ae90dbd-7aea-4448-9356-d202ce7df1c3-webhook-cert\") pod \"9ae90dbd-7aea-4448-9356-d202ce7df1c3\" (UID: \"9ae90dbd-7aea-4448-9356-d202ce7df1c3\") "
Sep 12 00:54:57.432508 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:54:57.432478    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9ae90dbd-7aea-4448-9356-d202ce7df1c3-kube-api-access-trd7d" (OuterVolumeSpecName: "kube-api-access-trd7d") pod "9ae90dbd-7aea-4448-9356-d202ce7df1c3" (UID: "9ae90dbd-7aea-4448-9356-d202ce7df1c3"). InnerVolumeSpecName "kube-api-access-trd7d". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 12 00:54:57.432508 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:54:57.432501    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9ae90dbd-7aea-4448-9356-d202ce7df1c3-webhook-cert" (OuterVolumeSpecName: "webhook-cert") pod "9ae90dbd-7aea-4448-9356-d202ce7df1c3" (UID: "9ae90dbd-7aea-4448-9356-d202ce7df1c3"). InnerVolumeSpecName "webhook-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 12 00:54:57.432706 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:54:57.432518    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9ae90dbd-7aea-4448-9356-d202ce7df1c3-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "9ae90dbd-7aea-4448-9356-d202ce7df1c3" (UID: "9ae90dbd-7aea-4448-9356-d202ce7df1c3"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 12 00:54:57.531224 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:54:57.531192    2571 reconciler_common.go:299] "Volume detached for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/9ae90dbd-7aea-4448-9356-d202ce7df1c3-webhook-cert\") on node \"ip-10-0-141-128.ec2.internal\" DevicePath \"\""
Sep 12 00:54:57.531224 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:54:57.531218    2571 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-trd7d\" (UniqueName: \"kubernetes.io/projected/9ae90dbd-7aea-4448-9356-d202ce7df1c3-kube-api-access-trd7d\") on node \"ip-10-0-141-128.ec2.internal\" DevicePath \"\""
Sep 12 00:54:57.531224 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:54:57.531228    2571 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/9ae90dbd-7aea-4448-9356-d202ce7df1c3-metrics-certs\") on node \"ip-10-0-141-128.ec2.internal\" DevicePath \"\""
Sep 12 00:54:57.740562 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:54:57.740482    2571 generic.go:356] "Generic (PLEG): container finished" podID="9ae90dbd-7aea-4448-9356-d202ce7df1c3" containerID="b6f52b353a4428aee8ddff51a3e79035066971589a5bd93c78d5128737c9ee8b" exitCode=0
Sep 12 00:54:57.740562 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:54:57.740525    2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-57d99b475d-nwgsr"
Sep 12 00:54:57.740562 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:54:57.740541    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-57d99b475d-nwgsr" event={"ID":"9ae90dbd-7aea-4448-9356-d202ce7df1c3","Type":"ContainerDied","Data":"b6f52b353a4428aee8ddff51a3e79035066971589a5bd93c78d5128737c9ee8b"}
Sep 12 00:54:57.740827 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:54:57.740599    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-57d99b475d-nwgsr" event={"ID":"9ae90dbd-7aea-4448-9356-d202ce7df1c3","Type":"ContainerDied","Data":"360b0094338ff44622f9f9a07289d349be5d8abd6436883438a7b223d76a97a8"}
Sep 12 00:54:57.740827 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:54:57.740622    2571 scope.go:117] "RemoveContainer" containerID="b6f52b353a4428aee8ddff51a3e79035066971589a5bd93c78d5128737c9ee8b"
Sep 12 00:54:57.749224 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:54:57.749209    2571 scope.go:117] "RemoveContainer" containerID="b6f52b353a4428aee8ddff51a3e79035066971589a5bd93c78d5128737c9ee8b"
Sep 12 00:54:57.749497 ip-10-0-141-128 kubenswrapper[2571]: E0912 00:54:57.749479    2571 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b6f52b353a4428aee8ddff51a3e79035066971589a5bd93c78d5128737c9ee8b\": container with ID starting with b6f52b353a4428aee8ddff51a3e79035066971589a5bd93c78d5128737c9ee8b not found: ID does not exist" containerID="b6f52b353a4428aee8ddff51a3e79035066971589a5bd93c78d5128737c9ee8b"
Sep 12 00:54:57.749559 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:54:57.749507    2571 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b6f52b353a4428aee8ddff51a3e79035066971589a5bd93c78d5128737c9ee8b"} err="failed to get container status \"b6f52b353a4428aee8ddff51a3e79035066971589a5bd93c78d5128737c9ee8b\": rpc error: code = NotFound desc = could not find container \"b6f52b353a4428aee8ddff51a3e79035066971589a5bd93c78d5128737c9ee8b\": container with ID starting with b6f52b353a4428aee8ddff51a3e79035066971589a5bd93c78d5128737c9ee8b not found: ID does not exist"
Sep 12 00:54:57.760687 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:54:57.760664    2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-57d99b475d-nwgsr"]
Sep 12 00:54:57.764068 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:54:57.764048    2571 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-57d99b475d-nwgsr"]
Sep 12 00:54:58.572515 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:54:58.572479    2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-57d99b475d-7dt66"]
Sep 12 00:54:58.572903 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:54:58.572895    2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3c451460-9cc7-49b0-ab4f-f361805f15d9" containerName="istio-proxy"
Sep 12 00:54:58.572947 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:54:58.572907    2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="3c451460-9cc7-49b0-ab4f-f361805f15d9" containerName="istio-proxy"
Sep 12 00:54:58.572947 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:54:58.572933    2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9ae90dbd-7aea-4448-9356-d202ce7df1c3" containerName="manager"
Sep 12 00:54:58.572947 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:54:58.572939    2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="9ae90dbd-7aea-4448-9356-d202ce7df1c3" containerName="manager"
Sep 12 00:54:58.573041 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:54:58.572995    2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="e0489e12-24e5-436d-85c9-5a4fd75e2609" containerName="istio-proxy"
Sep 12 00:54:58.573041 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:54:58.573010    2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="9ae90dbd-7aea-4448-9356-d202ce7df1c3" containerName="manager"
Sep 12 00:54:58.573041 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:54:58.573016    2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="ee555fcf-4c74-4b63-ad98-e9d5c4976697" containerName="istio-proxy"
Sep 12 00:54:58.577700 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:54:58.577680    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-57d99b475d-7dt66"
Sep 12 00:54:58.581266 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:54:58.581243    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\""
Sep 12 00:54:58.581397 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:54:58.581268    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-km42l\""
Sep 12 00:54:58.581397 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:54:58.581270    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-metrics-tls\""
Sep 12 00:54:58.581397 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:54:58.581316    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\""
Sep 12 00:54:58.586775 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:54:58.586751    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-57d99b475d-7dt66"]
Sep 12 00:54:58.641494 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:54:58.641463    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/af4c0706-719d-49f9-a742-789cad837da2-metrics-certs\") pod \"maas-controller-57d99b475d-7dt66\" (UID: \"af4c0706-719d-49f9-a742-789cad837da2\") " pod="opendatahub/maas-controller-57d99b475d-7dt66"
Sep 12 00:54:58.641690 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:54:58.641511    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xbwgt\" (UniqueName: \"kubernetes.io/projected/af4c0706-719d-49f9-a742-789cad837da2-kube-api-access-xbwgt\") pod \"maas-controller-57d99b475d-7dt66\" (UID: \"af4c0706-719d-49f9-a742-789cad837da2\") " pod="opendatahub/maas-controller-57d99b475d-7dt66"
Sep 12 00:54:58.641690 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:54:58.641577    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/af4c0706-719d-49f9-a742-789cad837da2-webhook-cert\") pod \"maas-controller-57d99b475d-7dt66\" (UID: \"af4c0706-719d-49f9-a742-789cad837da2\") " pod="opendatahub/maas-controller-57d99b475d-7dt66"
Sep 12 00:54:58.742328 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:54:58.742296    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/af4c0706-719d-49f9-a742-789cad837da2-webhook-cert\") pod \"maas-controller-57d99b475d-7dt66\" (UID: \"af4c0706-719d-49f9-a742-789cad837da2\") " pod="opendatahub/maas-controller-57d99b475d-7dt66"
Sep 12 00:54:58.742514 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:54:58.742356    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/af4c0706-719d-49f9-a742-789cad837da2-metrics-certs\") pod \"maas-controller-57d99b475d-7dt66\" (UID: \"af4c0706-719d-49f9-a742-789cad837da2\") " pod="opendatahub/maas-controller-57d99b475d-7dt66"
Sep 12 00:54:58.742514 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:54:58.742389    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xbwgt\" (UniqueName: \"kubernetes.io/projected/af4c0706-719d-49f9-a742-789cad837da2-kube-api-access-xbwgt\") pod \"maas-controller-57d99b475d-7dt66\" (UID: \"af4c0706-719d-49f9-a742-789cad837da2\") " pod="opendatahub/maas-controller-57d99b475d-7dt66"
Sep 12 00:54:58.744837 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:54:58.744808    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/af4c0706-719d-49f9-a742-789cad837da2-metrics-certs\") pod \"maas-controller-57d99b475d-7dt66\" (UID: \"af4c0706-719d-49f9-a742-789cad837da2\") " pod="opendatahub/maas-controller-57d99b475d-7dt66"
Sep 12 00:54:58.744947 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:54:58.744867    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/af4c0706-719d-49f9-a742-789cad837da2-webhook-cert\") pod \"maas-controller-57d99b475d-7dt66\" (UID: \"af4c0706-719d-49f9-a742-789cad837da2\") " pod="opendatahub/maas-controller-57d99b475d-7dt66"
Sep 12 00:54:58.749797 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:54:58.749773    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xbwgt\" (UniqueName: \"kubernetes.io/projected/af4c0706-719d-49f9-a742-789cad837da2-kube-api-access-xbwgt\") pod \"maas-controller-57d99b475d-7dt66\" (UID: \"af4c0706-719d-49f9-a742-789cad837da2\") " pod="opendatahub/maas-controller-57d99b475d-7dt66"
Sep 12 00:54:58.889020 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:54:58.888943    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-57d99b475d-7dt66"
Sep 12 00:54:59.140971 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:54:59.140902    2571 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9ae90dbd-7aea-4448-9356-d202ce7df1c3" path="/var/lib/kubelet/pods/9ae90dbd-7aea-4448-9356-d202ce7df1c3/volumes"
Sep 12 00:54:59.157767 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:54:59.157745    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-57d99b475d-7dt66"]
Sep 12 00:54:59.160151 ip-10-0-141-128 kubenswrapper[2571]: W0912 00:54:59.160127    2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podaf4c0706_719d_49f9_a742_789cad837da2.slice/crio-217c1999d84c613bcd97f1828c811ab642b855c9cf175454e3576b3f0a2f4df3 WatchSource:0}: Error finding container 217c1999d84c613bcd97f1828c811ab642b855c9cf175454e3576b3f0a2f4df3: Status 404 returned error can't find the container with id 217c1999d84c613bcd97f1828c811ab642b855c9cf175454e3576b3f0a2f4df3
Sep 12 00:54:59.756029 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:54:59.755940    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-57d99b475d-7dt66" event={"ID":"af4c0706-719d-49f9-a742-789cad837da2","Type":"ContainerStarted","Data":"a4ad1331272abccff0b06a73dd10dd83deb4f6577fb38dca210059ae3f5576f5"}
Sep 12 00:54:59.756029 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:54:59.755994    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-57d99b475d-7dt66" event={"ID":"af4c0706-719d-49f9-a742-789cad837da2","Type":"ContainerStarted","Data":"217c1999d84c613bcd97f1828c811ab642b855c9cf175454e3576b3f0a2f4df3"}
Sep 12 00:54:59.756472 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:54:59.756037    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-57d99b475d-7dt66"
Sep 12 00:54:59.773565 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:54:59.773518    2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-57d99b475d-7dt66" podStartSLOduration=1.495325736 podStartE2EDuration="1.773504798s" totalImagesPullingTime="278.179062ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-12 00:54:58 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-12 00:54:59.77154565 +0000 UTC m=+2311.043186186" watchObservedRunningTime="2026-09-12 00:54:59.773504798 +0000 UTC m=+2311.045145304"
Sep 12 00:55:10.763481 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:55:10.763454    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-57d99b475d-7dt66"
Sep 12 00:58:34.463467 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:58:34.463388    2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-6bb595b6d8-lcxkt"]
Sep 12 00:58:34.467810 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:58:34.467783    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-6bb595b6d8-lcxkt"
Sep 12 00:58:34.478305 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:58:34.478268    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-6bb595b6d8-lcxkt"]
Sep 12 00:58:34.596150 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:58:34.596117    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/289aeba1-f639-4532-a1e5-58138f854691-maas-api-tls\") pod \"maas-api-6bb595b6d8-lcxkt\" (UID: \"289aeba1-f639-4532-a1e5-58138f854691\") " pod="odh-ai-gateway-infra/maas-api-6bb595b6d8-lcxkt"
Sep 12 00:58:34.596150 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:58:34.596156    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/289aeba1-f639-4532-a1e5-58138f854691-openshift-service-ca\") pod \"maas-api-6bb595b6d8-lcxkt\" (UID: \"289aeba1-f639-4532-a1e5-58138f854691\") " pod="odh-ai-gateway-infra/maas-api-6bb595b6d8-lcxkt"
Sep 12 00:58:34.596421 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:58:34.596237    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2htmn\" (UniqueName: \"kubernetes.io/projected/289aeba1-f639-4532-a1e5-58138f854691-kube-api-access-2htmn\") pod \"maas-api-6bb595b6d8-lcxkt\" (UID: \"289aeba1-f639-4532-a1e5-58138f854691\") " pod="odh-ai-gateway-infra/maas-api-6bb595b6d8-lcxkt"
Sep 12 00:58:34.596421 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:58:34.596285    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/289aeba1-f639-4532-a1e5-58138f854691-metrics-certs\") pod \"maas-api-6bb595b6d8-lcxkt\" (UID: \"289aeba1-f639-4532-a1e5-58138f854691\") " pod="odh-ai-gateway-infra/maas-api-6bb595b6d8-lcxkt"
Sep 12 00:58:34.697067 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:58:34.697036    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2htmn\" (UniqueName: \"kubernetes.io/projected/289aeba1-f639-4532-a1e5-58138f854691-kube-api-access-2htmn\") pod \"maas-api-6bb595b6d8-lcxkt\" (UID: \"289aeba1-f639-4532-a1e5-58138f854691\") " pod="odh-ai-gateway-infra/maas-api-6bb595b6d8-lcxkt"
Sep 12 00:58:34.697246 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:58:34.697085    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/289aeba1-f639-4532-a1e5-58138f854691-metrics-certs\") pod \"maas-api-6bb595b6d8-lcxkt\" (UID: \"289aeba1-f639-4532-a1e5-58138f854691\") " pod="odh-ai-gateway-infra/maas-api-6bb595b6d8-lcxkt"
Sep 12 00:58:34.697246 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:58:34.697154    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/289aeba1-f639-4532-a1e5-58138f854691-maas-api-tls\") pod \"maas-api-6bb595b6d8-lcxkt\" (UID: \"289aeba1-f639-4532-a1e5-58138f854691\") " pod="odh-ai-gateway-infra/maas-api-6bb595b6d8-lcxkt"
Sep 12 00:58:34.697246 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:58:34.697171    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/289aeba1-f639-4532-a1e5-58138f854691-openshift-service-ca\") pod \"maas-api-6bb595b6d8-lcxkt\" (UID: \"289aeba1-f639-4532-a1e5-58138f854691\") " pod="odh-ai-gateway-infra/maas-api-6bb595b6d8-lcxkt"
Sep 12 00:58:34.697765 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:58:34.697742    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/289aeba1-f639-4532-a1e5-58138f854691-openshift-service-ca\") pod \"maas-api-6bb595b6d8-lcxkt\" (UID: \"289aeba1-f639-4532-a1e5-58138f854691\") " pod="odh-ai-gateway-infra/maas-api-6bb595b6d8-lcxkt"
Sep 12 00:58:34.699483 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:58:34.699459    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/289aeba1-f639-4532-a1e5-58138f854691-maas-api-tls\") pod \"maas-api-6bb595b6d8-lcxkt\" (UID: \"289aeba1-f639-4532-a1e5-58138f854691\") " pod="odh-ai-gateway-infra/maas-api-6bb595b6d8-lcxkt"
Sep 12 00:58:34.699628 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:58:34.699492    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/289aeba1-f639-4532-a1e5-58138f854691-metrics-certs\") pod \"maas-api-6bb595b6d8-lcxkt\" (UID: \"289aeba1-f639-4532-a1e5-58138f854691\") " pod="odh-ai-gateway-infra/maas-api-6bb595b6d8-lcxkt"
Sep 12 00:58:34.704647 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:58:34.704627    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2htmn\" (UniqueName: \"kubernetes.io/projected/289aeba1-f639-4532-a1e5-58138f854691-kube-api-access-2htmn\") pod \"maas-api-6bb595b6d8-lcxkt\" (UID: \"289aeba1-f639-4532-a1e5-58138f854691\") " pod="odh-ai-gateway-infra/maas-api-6bb595b6d8-lcxkt"
Sep 12 00:58:34.780261 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:58:34.780171    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-6bb595b6d8-lcxkt"
Sep 12 00:58:35.068945 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:58:35.068855    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-6bb595b6d8-lcxkt"]
Sep 12 00:58:35.072734 ip-10-0-141-128 kubenswrapper[2571]: W0912 00:58:35.072701    2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod289aeba1_f639_4532_a1e5_58138f854691.slice/crio-898355ab7a99d05f4c93ec20631ef858b46b18a0d13727e85011ad18cc33c4e7 WatchSource:0}: Error finding container 898355ab7a99d05f4c93ec20631ef858b46b18a0d13727e85011ad18cc33c4e7: Status 404 returned error can't find the container with id 898355ab7a99d05f4c93ec20631ef858b46b18a0d13727e85011ad18cc33c4e7
Sep 12 00:58:35.777341 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:58:35.777257    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-6bb595b6d8-lcxkt" event={"ID":"289aeba1-f639-4532-a1e5-58138f854691","Type":"ContainerStarted","Data":"30f381ea9e677b9c9bd052a0b599562a5e167a82416ba5dd9e65ef35e4dad778"}
Sep 12 00:58:35.777341 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:58:35.777317    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-6bb595b6d8-lcxkt" event={"ID":"289aeba1-f639-4532-a1e5-58138f854691","Type":"ContainerStarted","Data":"898355ab7a99d05f4c93ec20631ef858b46b18a0d13727e85011ad18cc33c4e7"}
Sep 12 00:58:35.777851 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:58:35.777372    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-6bb595b6d8-lcxkt"
Sep 12 00:58:35.797737 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:58:35.797695    2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-6bb595b6d8-lcxkt" podStartSLOduration=1.360931972 podStartE2EDuration="1.797680282s" totalImagesPullingTime="436.74831ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-12 00:58:34 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-12 00:58:35.795651434 +0000 UTC m=+2527.067291946" watchObservedRunningTime="2026-09-12 00:58:35.797680282 +0000 UTC m=+2527.069320816"
Sep 12 00:58:41.788473 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:58:41.788442    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-6bb595b6d8-lcxkt"
Sep 12 00:58:49.767416 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:58:49.767381    2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-6b768f88d8-nv8zh"]
Sep 12 00:58:49.768096 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:58:49.767872    2571 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-6b768f88d8-nv8zh" podUID="f77dec72-81c1-4510-a251-07cda55f274f" containerName="maas-api" containerID="cri-o://3601896e14e6510fb55df38e6ce6674dbd8f69b82290b99e2c03a293cc240494" gracePeriod=30
Sep 12 00:58:51.115447 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:58:51.115422    2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-6b768f88d8-nv8zh"
Sep 12 00:58:51.249740 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:58:51.249709    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-f4z74\" (UniqueName: \"kubernetes.io/projected/f77dec72-81c1-4510-a251-07cda55f274f-kube-api-access-f4z74\") pod \"f77dec72-81c1-4510-a251-07cda55f274f\" (UID: \"f77dec72-81c1-4510-a251-07cda55f274f\") "
Sep 12 00:58:51.249912 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:58:51.249778    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f77dec72-81c1-4510-a251-07cda55f274f-metrics-certs\") pod \"f77dec72-81c1-4510-a251-07cda55f274f\" (UID: \"f77dec72-81c1-4510-a251-07cda55f274f\") "
Sep 12 00:58:51.249912 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:58:51.249806    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f77dec72-81c1-4510-a251-07cda55f274f-openshift-service-ca\") pod \"f77dec72-81c1-4510-a251-07cda55f274f\" (UID: \"f77dec72-81c1-4510-a251-07cda55f274f\") "
Sep 12 00:58:51.249912 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:58:51.249870    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f77dec72-81c1-4510-a251-07cda55f274f-maas-api-tls\") pod \"f77dec72-81c1-4510-a251-07cda55f274f\" (UID: \"f77dec72-81c1-4510-a251-07cda55f274f\") "
Sep 12 00:58:51.250264 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:58:51.250225    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f77dec72-81c1-4510-a251-07cda55f274f-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "f77dec72-81c1-4510-a251-07cda55f274f" (UID: "f77dec72-81c1-4510-a251-07cda55f274f"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 12 00:58:51.252017 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:58:51.251993    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f77dec72-81c1-4510-a251-07cda55f274f-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "f77dec72-81c1-4510-a251-07cda55f274f" (UID: "f77dec72-81c1-4510-a251-07cda55f274f"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 12 00:58:51.252128 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:58:51.252022    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f77dec72-81c1-4510-a251-07cda55f274f-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "f77dec72-81c1-4510-a251-07cda55f274f" (UID: "f77dec72-81c1-4510-a251-07cda55f274f"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 12 00:58:51.252128 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:58:51.252032    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f77dec72-81c1-4510-a251-07cda55f274f-kube-api-access-f4z74" (OuterVolumeSpecName: "kube-api-access-f4z74") pod "f77dec72-81c1-4510-a251-07cda55f274f" (UID: "f77dec72-81c1-4510-a251-07cda55f274f"). InnerVolumeSpecName "kube-api-access-f4z74". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 12 00:58:51.350653 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:58:51.350626    2571 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f77dec72-81c1-4510-a251-07cda55f274f-maas-api-tls\") on node \"ip-10-0-141-128.ec2.internal\" DevicePath \"\""
Sep 12 00:58:51.350653 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:58:51.350651    2571 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-f4z74\" (UniqueName: \"kubernetes.io/projected/f77dec72-81c1-4510-a251-07cda55f274f-kube-api-access-f4z74\") on node \"ip-10-0-141-128.ec2.internal\" DevicePath \"\""
Sep 12 00:58:51.350653 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:58:51.350661    2571 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f77dec72-81c1-4510-a251-07cda55f274f-metrics-certs\") on node \"ip-10-0-141-128.ec2.internal\" DevicePath \"\""
Sep 12 00:58:51.350869 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:58:51.350670    2571 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f77dec72-81c1-4510-a251-07cda55f274f-openshift-service-ca\") on node \"ip-10-0-141-128.ec2.internal\" DevicePath \"\""
Sep 12 00:58:51.855973 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:58:51.855940    2571 generic.go:356] "Generic (PLEG): container finished" podID="f77dec72-81c1-4510-a251-07cda55f274f" containerID="3601896e14e6510fb55df38e6ce6674dbd8f69b82290b99e2c03a293cc240494" exitCode=0
Sep 12 00:58:51.856145 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:58:51.855984    2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-6b768f88d8-nv8zh"
Sep 12 00:58:51.856145 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:58:51.856030    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-6b768f88d8-nv8zh" event={"ID":"f77dec72-81c1-4510-a251-07cda55f274f","Type":"ContainerDied","Data":"3601896e14e6510fb55df38e6ce6674dbd8f69b82290b99e2c03a293cc240494"}
Sep 12 00:58:51.856145 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:58:51.856082    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-6b768f88d8-nv8zh" event={"ID":"f77dec72-81c1-4510-a251-07cda55f274f","Type":"ContainerDied","Data":"d97fc3a5e3a4f75db6b194dd2de061d2ce248bd854fc0ff9aa8c87ae589d3424"}
Sep 12 00:58:51.856145 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:58:51.856104    2571 scope.go:117] "RemoveContainer" containerID="3601896e14e6510fb55df38e6ce6674dbd8f69b82290b99e2c03a293cc240494"
Sep 12 00:58:51.865334 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:58:51.865318    2571 scope.go:117] "RemoveContainer" containerID="3601896e14e6510fb55df38e6ce6674dbd8f69b82290b99e2c03a293cc240494"
Sep 12 00:58:51.865581 ip-10-0-141-128 kubenswrapper[2571]: E0912 00:58:51.865562    2571 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3601896e14e6510fb55df38e6ce6674dbd8f69b82290b99e2c03a293cc240494\": container with ID starting with 3601896e14e6510fb55df38e6ce6674dbd8f69b82290b99e2c03a293cc240494 not found: ID does not exist" containerID="3601896e14e6510fb55df38e6ce6674dbd8f69b82290b99e2c03a293cc240494"
Sep 12 00:58:51.865656 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:58:51.865603    2571 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3601896e14e6510fb55df38e6ce6674dbd8f69b82290b99e2c03a293cc240494"} err="failed to get container status \"3601896e14e6510fb55df38e6ce6674dbd8f69b82290b99e2c03a293cc240494\": rpc error: code = NotFound desc = could not find container \"3601896e14e6510fb55df38e6ce6674dbd8f69b82290b99e2c03a293cc240494\": container with ID starting with 3601896e14e6510fb55df38e6ce6674dbd8f69b82290b99e2c03a293cc240494 not found: ID does not exist"
Sep 12 00:58:51.878303 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:58:51.878282    2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-6b768f88d8-nv8zh"]
Sep 12 00:58:51.882625 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:58:51.882604    2571 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-6b768f88d8-nv8zh"]
Sep 12 00:58:53.140605 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:58:53.140554    2571 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f77dec72-81c1-4510-a251-07cda55f274f" path="/var/lib/kubelet/pods/f77dec72-81c1-4510-a251-07cda55f274f/volumes"
Sep 12 00:59:10.135438 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:59:10.135402    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_kserve-controller-manager-d6f4b65b8-tzqmv_9d018424-bcd8-40d8-980a-33214e4ff532/manager/0.log"
Sep 12 00:59:10.257677 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:59:10.257645    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_llmisvc-controller-manager-777c6bd7d5-4bzq6_b6f1b332-882c-4dbf-907e-8b201235ab1c/manager/0.log"
Sep 12 00:59:10.384341 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:59:10.384314    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-57d99b475d-7dt66_af4c0706-719d-49f9-a742-789cad837da2/manager/0.log"
Sep 12 00:59:10.503091 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:59:10.503062    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_model-serving-api-54d89bd5fc-s4l9d_f4408e93-cbb1-4571-8b85-db15938ea0c0/server/0.log"
Sep 12 00:59:10.738078 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:59:10.738052    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-8446f459d6-dngdm_618d0f14-c54a-4aec-bba9-3d1c67ab23a7/copy-manifests/0.log"
Sep 12 00:59:10.745207 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:59:10.745183    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-8446f459d6-dngdm_618d0f14-c54a-4aec-bba9-3d1c67ab23a7/manager/0.log"
Sep 12 00:59:11.471943 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:59:11.471915    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-6bb595b6d8-lcxkt_289aeba1-f639-4532-a1e5-58138f854691/maas-api/0.log"
Sep 12 00:59:11.706651 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:59:11.706620    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_postgres-69fb4594cf-2bhcg_af70f745-1b33-4737-82f1-04159afc98ca/postgres/0.log"
Sep 12 00:59:12.643255 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:59:12.643229    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-operator-76b478bd78-vdsxb_3f21ce58-966b-4f83-a1e2-55cbbe91871e/manager/0.log"
Sep 12 00:59:13.743811 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:59:13.743786    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-5677ccb49b-m9kwv_af206a61-acc5-4738-a83f-55c131da2017/discovery/0.log"
Sep 12 00:59:13.863276 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:59:13.863251    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-76b8774589-59jxv_e4d384f3-026e-4343-bada-ddc631f69d53/kube-auth-proxy/0.log"
Sep 12 00:59:14.119109 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:59:14.119034    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-78bb78ff78-svr8q_06461169-3e71-40a2-a0dc-e52b89db9f42/istio-proxy/1.log"
Sep 12 00:59:14.233000 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:59:14.232970    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-59fbc47bf8-p2vwz_e335c05b-bd1d-468c-b882-f8a372524c56/payload-pre-processing/0.log"
Sep 12 00:59:14.349891 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:59:14.349861    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5b54b7fbb7-btmpn_817b7aba-94b1-4306-b26a-027e1e0b7297/payload-processing/0.log"
Sep 12 00:59:14.469342 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:59:14.469294    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-57b84c8474-8xjql_c2123986-4ae4-40ce-a423-b5e9a4accda2/router/0.log"
Sep 12 00:59:14.957554 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:59:14.957524    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-distinct-simulated-kserve-59975d69f9-qggd6_1dc7f7b3-d9b5-40af-9e34-c8c8af685c05/main/0.log"
Sep 12 00:59:23.747818 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:59:23.747788    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-nxrg5_09b83899-89b7-4f4a-b485-c102295f188e/global-pull-secret-syncer/0.log"
Sep 12 00:59:23.859326 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:59:23.859276    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-r92pg_71d215ed-e9d2-4a8b-bc73-f0ee2e0d83c5/konnectivity-agent/0.log"
Sep 12 00:59:23.923824 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:59:23.923802    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-141-128.ec2.internal_a0171d806087a75aaabfcaff6900d5a8/haproxy/0.log"
Sep 12 00:59:28.380809 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:59:28.380731    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-operator-76b478bd78-vdsxb_3f21ce58-966b-4f83-a1e2-55cbbe91871e/manager/0.log"
Sep 12 00:59:30.422342 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:59:30.422269    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-59fbdd77d-fc26r_951506d4-b1be-4ee7-b72d-1b5dc1d556f2/kube-state-metrics/0.log"
Sep 12 00:59:30.442983 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:59:30.442959    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-59fbdd77d-fc26r_951506d4-b1be-4ee7-b72d-1b5dc1d556f2/kube-rbac-proxy-main/0.log"
Sep 12 00:59:30.470049 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:59:30.470027    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-59fbdd77d-fc26r_951506d4-b1be-4ee7-b72d-1b5dc1d556f2/kube-rbac-proxy-self/0.log"
Sep 12 00:59:30.508758 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:59:30.508737    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_metrics-server-6fc7858648-mvh4h_980eacbb-2f36-403a-a2b8-bce355264f9f/metrics-server/0.log"
Sep 12 00:59:30.537675 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:59:30.537650    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_monitoring-plugin-5d85778b9d-tsthv_3f2f3cbe-620b-4bcb-ba33-109bd0b35b1b/monitoring-plugin/0.log"
Sep 12 00:59:30.570441 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:59:30.570423    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-qgr6s_17f775e6-4bfa-4d7f-aeac-4e66cb551e46/node-exporter/0.log"
Sep 12 00:59:30.596352 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:59:30.596330    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-qgr6s_17f775e6-4bfa-4d7f-aeac-4e66cb551e46/kube-rbac-proxy/0.log"
Sep 12 00:59:30.617050 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:59:30.617029    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-qgr6s_17f775e6-4bfa-4d7f-aeac-4e66cb551e46/init-textfile/0.log"
Sep 12 00:59:30.809802 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:59:30.809776    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-67dcf8c979-n4njq_f0b346be-c2c0-43da-9858-f2e93a091d75/kube-rbac-proxy-main/0.log"
Sep 12 00:59:30.833407 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:59:30.833384    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-67dcf8c979-n4njq_f0b346be-c2c0-43da-9858-f2e93a091d75/kube-rbac-proxy-self/0.log"
Sep 12 00:59:30.859334 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:59:30.859314    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-67dcf8c979-n4njq_f0b346be-c2c0-43da-9858-f2e93a091d75/openshift-state-metrics/0.log"
Sep 12 00:59:30.908691 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:59:30.908656    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_2ab53334-b18b-4282-a586-2a1eac8e2575/prometheus/0.log"
Sep 12 00:59:30.938357 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:59:30.938322    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_2ab53334-b18b-4282-a586-2a1eac8e2575/config-reloader/0.log"
Sep 12 00:59:30.972216 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:59:30.972196    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_2ab53334-b18b-4282-a586-2a1eac8e2575/thanos-sidecar/0.log"
Sep 12 00:59:31.001928 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:59:31.001867    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_2ab53334-b18b-4282-a586-2a1eac8e2575/kube-rbac-proxy-web/0.log"
Sep 12 00:59:31.026169 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:59:31.026145    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_2ab53334-b18b-4282-a586-2a1eac8e2575/kube-rbac-proxy/0.log"
Sep 12 00:59:31.049600 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:59:31.049569    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_2ab53334-b18b-4282-a586-2a1eac8e2575/kube-rbac-proxy-thanos/0.log"
Sep 12 00:59:31.072539 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:59:31.072485    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_2ab53334-b18b-4282-a586-2a1eac8e2575/init-config-reloader/0.log"
Sep 12 00:59:33.794382 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:59:33.794349    2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-wvbw5/perf-node-gather-daemonset-w6mz5"]
Sep 12 00:59:33.794866 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:59:33.794848    2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f77dec72-81c1-4510-a251-07cda55f274f" containerName="maas-api"
Sep 12 00:59:33.794923 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:59:33.794870    2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="f77dec72-81c1-4510-a251-07cda55f274f" containerName="maas-api"
Sep 12 00:59:33.794961 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:59:33.794937    2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="f77dec72-81c1-4510-a251-07cda55f274f" containerName="maas-api"
Sep 12 00:59:33.799265 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:59:33.799246    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-wvbw5/perf-node-gather-daemonset-w6mz5"
Sep 12 00:59:33.801777 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:59:33.801755    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-wvbw5\"/\"openshift-service-ca.crt\""
Sep 12 00:59:33.802306 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:59:33.802289    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-wvbw5\"/\"default-dockercfg-wwsd6\""
Sep 12 00:59:33.802395 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:59:33.802297    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-wvbw5\"/\"kube-root-ca.crt\""
Sep 12 00:59:33.804987 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:59:33.804967    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-wvbw5/perf-node-gather-daemonset-w6mz5"]
Sep 12 00:59:33.931712 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:59:33.931674    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/77bcafb6-4dbe-4161-8277-89233a3476b9-proc\") pod \"perf-node-gather-daemonset-w6mz5\" (UID: \"77bcafb6-4dbe-4161-8277-89233a3476b9\") " pod="openshift-must-gather-wvbw5/perf-node-gather-daemonset-w6mz5"
Sep 12 00:59:33.931897 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:59:33.931730    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/77bcafb6-4dbe-4161-8277-89233a3476b9-podres\") pod \"perf-node-gather-daemonset-w6mz5\" (UID: \"77bcafb6-4dbe-4161-8277-89233a3476b9\") " pod="openshift-must-gather-wvbw5/perf-node-gather-daemonset-w6mz5"
Sep 12 00:59:33.931897 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:59:33.931757    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/77bcafb6-4dbe-4161-8277-89233a3476b9-lib-modules\") pod \"perf-node-gather-daemonset-w6mz5\" (UID: \"77bcafb6-4dbe-4161-8277-89233a3476b9\") " pod="openshift-must-gather-wvbw5/perf-node-gather-daemonset-w6mz5"
Sep 12 00:59:33.931897 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:59:33.931779    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/77bcafb6-4dbe-4161-8277-89233a3476b9-sys\") pod \"perf-node-gather-daemonset-w6mz5\" (UID: \"77bcafb6-4dbe-4161-8277-89233a3476b9\") " pod="openshift-must-gather-wvbw5/perf-node-gather-daemonset-w6mz5"
Sep 12 00:59:33.931897 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:59:33.931813    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rs52c\" (UniqueName: \"kubernetes.io/projected/77bcafb6-4dbe-4161-8277-89233a3476b9-kube-api-access-rs52c\") pod \"perf-node-gather-daemonset-w6mz5\" (UID: \"77bcafb6-4dbe-4161-8277-89233a3476b9\") " pod="openshift-must-gather-wvbw5/perf-node-gather-daemonset-w6mz5"
Sep 12 00:59:34.033326 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:59:34.033290    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/77bcafb6-4dbe-4161-8277-89233a3476b9-podres\") pod \"perf-node-gather-daemonset-w6mz5\" (UID: \"77bcafb6-4dbe-4161-8277-89233a3476b9\") " pod="openshift-must-gather-wvbw5/perf-node-gather-daemonset-w6mz5"
Sep 12 00:59:34.033326 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:59:34.033327    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/77bcafb6-4dbe-4161-8277-89233a3476b9-lib-modules\") pod \"perf-node-gather-daemonset-w6mz5\" (UID: \"77bcafb6-4dbe-4161-8277-89233a3476b9\") " pod="openshift-must-gather-wvbw5/perf-node-gather-daemonset-w6mz5"
Sep 12 00:59:34.033564 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:59:34.033347    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/77bcafb6-4dbe-4161-8277-89233a3476b9-sys\") pod \"perf-node-gather-daemonset-w6mz5\" (UID: \"77bcafb6-4dbe-4161-8277-89233a3476b9\") " pod="openshift-must-gather-wvbw5/perf-node-gather-daemonset-w6mz5"
Sep 12 00:59:34.033564 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:59:34.033375    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rs52c\" (UniqueName: \"kubernetes.io/projected/77bcafb6-4dbe-4161-8277-89233a3476b9-kube-api-access-rs52c\") pod \"perf-node-gather-daemonset-w6mz5\" (UID: \"77bcafb6-4dbe-4161-8277-89233a3476b9\") " pod="openshift-must-gather-wvbw5/perf-node-gather-daemonset-w6mz5"
Sep 12 00:59:34.033564 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:59:34.033446    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/77bcafb6-4dbe-4161-8277-89233a3476b9-sys\") pod \"perf-node-gather-daemonset-w6mz5\" (UID: \"77bcafb6-4dbe-4161-8277-89233a3476b9\") " pod="openshift-must-gather-wvbw5/perf-node-gather-daemonset-w6mz5"
Sep 12 00:59:34.033564 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:59:34.033449    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/77bcafb6-4dbe-4161-8277-89233a3476b9-proc\") pod \"perf-node-gather-daemonset-w6mz5\" (UID: \"77bcafb6-4dbe-4161-8277-89233a3476b9\") " pod="openshift-must-gather-wvbw5/perf-node-gather-daemonset-w6mz5"
Sep 12 00:59:34.033564 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:59:34.033480    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/77bcafb6-4dbe-4161-8277-89233a3476b9-podres\") pod \"perf-node-gather-daemonset-w6mz5\" (UID: \"77bcafb6-4dbe-4161-8277-89233a3476b9\") " pod="openshift-must-gather-wvbw5/perf-node-gather-daemonset-w6mz5"
Sep 12 00:59:34.033564 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:59:34.033482    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/77bcafb6-4dbe-4161-8277-89233a3476b9-lib-modules\") pod \"perf-node-gather-daemonset-w6mz5\" (UID: \"77bcafb6-4dbe-4161-8277-89233a3476b9\") " pod="openshift-must-gather-wvbw5/perf-node-gather-daemonset-w6mz5"
Sep 12 00:59:34.033564 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:59:34.033488    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/77bcafb6-4dbe-4161-8277-89233a3476b9-proc\") pod \"perf-node-gather-daemonset-w6mz5\" (UID: \"77bcafb6-4dbe-4161-8277-89233a3476b9\") " pod="openshift-must-gather-wvbw5/perf-node-gather-daemonset-w6mz5"
Sep 12 00:59:34.042097 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:59:34.042077    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rs52c\" (UniqueName: \"kubernetes.io/projected/77bcafb6-4dbe-4161-8277-89233a3476b9-kube-api-access-rs52c\") pod \"perf-node-gather-daemonset-w6mz5\" (UID: \"77bcafb6-4dbe-4161-8277-89233a3476b9\") " pod="openshift-must-gather-wvbw5/perf-node-gather-daemonset-w6mz5"
Sep 12 00:59:34.111514 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:59:34.111445    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-wvbw5/perf-node-gather-daemonset-w6mz5"
Sep 12 00:59:34.224026 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:59:34.223998    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_downloads-76f99584f4-wqhtj_25e2de93-94e2-4231-97ff-208a1e9694a7/download-server/0.log"
Sep 12 00:59:34.382972 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:59:34.382947    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-wvbw5/perf-node-gather-daemonset-w6mz5"]
Sep 12 00:59:34.385236 ip-10-0-141-128 kubenswrapper[2571]: W0912 00:59:34.385205    2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pod77bcafb6_4dbe_4161_8277_89233a3476b9.slice/crio-424491be8b54ee9c9c9a18640f4e325be6cd37cbbb161054cbe13fbd418d494f WatchSource:0}: Error finding container 424491be8b54ee9c9c9a18640f4e325be6cd37cbbb161054cbe13fbd418d494f: Status 404 returned error can't find the container with id 424491be8b54ee9c9c9a18640f4e325be6cd37cbbb161054cbe13fbd418d494f
Sep 12 00:59:34.892648 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:59:34.892619    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-cluster-storage-operator_volume-data-source-validator-86695b7cc7-b9jtz_eea0700d-afa8-412b-b099-4f6953e54eee/volume-data-source-validator/0.log"
Sep 12 00:59:35.058667 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:59:35.058630    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-wvbw5/perf-node-gather-daemonset-w6mz5" event={"ID":"77bcafb6-4dbe-4161-8277-89233a3476b9","Type":"ContainerStarted","Data":"20a69dc63efbf63c20e6fb8a508e0d79418caa34a31693672a06b54e8393a776"}
Sep 12 00:59:35.058851 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:59:35.058703    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-wvbw5/perf-node-gather-daemonset-w6mz5"
Sep 12 00:59:35.058851 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:59:35.058724    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-wvbw5/perf-node-gather-daemonset-w6mz5" event={"ID":"77bcafb6-4dbe-4161-8277-89233a3476b9","Type":"ContainerStarted","Data":"424491be8b54ee9c9c9a18640f4e325be6cd37cbbb161054cbe13fbd418d494f"}
Sep 12 00:59:35.084142 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:59:35.078439    2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-wvbw5/perf-node-gather-daemonset-w6mz5" podStartSLOduration=2.078419458 podStartE2EDuration="2.078419458s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-12 00:59:33 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-12 00:59:35.076157167 +0000 UTC m=+2586.347797667" watchObservedRunningTime="2026-09-12 00:59:35.078419458 +0000 UTC m=+2586.350059971"
Sep 12 00:59:36.154665 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:59:36.154630    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-fr2kt_3a2114c7-d34a-4249-97a4-65f5375453ce/dns/0.log"
Sep 12 00:59:36.176620 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:59:36.176562    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-fr2kt_3a2114c7-d34a-4249-97a4-65f5375453ce/kube-rbac-proxy/0.log"
Sep 12 00:59:36.268362 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:59:36.268333    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-dnnhn_19b621f9-8fcc-4a57-b13d-862aad3d75ce/dns-node-resolver/0.log"
Sep 12 00:59:36.893854 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:59:36.893825    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-qm6vt_7a6f512a-ad51-4ec7-a9a5-8b6a34cb5bb8/node-ca/0.log"
Sep 12 00:59:38.192932 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:59:38.192898    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-5677ccb49b-m9kwv_af206a61-acc5-4738-a83f-55c131da2017/discovery/0.log"
Sep 12 00:59:38.212421 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:59:38.212395    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-76b8774589-59jxv_e4d384f3-026e-4343-bada-ddc631f69d53/kube-auth-proxy/0.log"
Sep 12 00:59:38.266036 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:59:38.266004    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-78bb78ff78-svr8q_06461169-3e71-40a2-a0dc-e52b89db9f42/istio-proxy/0.log"
Sep 12 00:59:38.268519 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:59:38.268497    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-78bb78ff78-svr8q_06461169-3e71-40a2-a0dc-e52b89db9f42/istio-proxy/1.log"
Sep 12 00:59:38.295440 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:59:38.295410    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-59fbc47bf8-p2vwz_e335c05b-bd1d-468c-b882-f8a372524c56/payload-pre-processing/0.log"
Sep 12 00:59:38.325903 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:59:38.325870    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5b54b7fbb7-btmpn_817b7aba-94b1-4306-b26a-027e1e0b7297/payload-processing/0.log"
Sep 12 00:59:38.349461 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:59:38.349432    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-57b84c8474-8xjql_c2123986-4ae4-40ce-a423-b5e9a4accda2/router/0.log"
Sep 12 00:59:39.270958 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:59:39.270927    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-rlr5m_ba88d840-64fd-419f-8444-ebde7a94c354/serve-healthcheck-canary/0.log"
Sep 12 00:59:39.801081 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:59:39.801045    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-kkm5j_531e7c03-cde5-4559-9d03-403158fb7168/kube-rbac-proxy/0.log"
Sep 12 00:59:39.821318 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:59:39.821281    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-kkm5j_531e7c03-cde5-4559-9d03-403158fb7168/exporter/0.log"
Sep 12 00:59:39.841491 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:59:39.841470    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-kkm5j_531e7c03-cde5-4559-9d03-403158fb7168/extractor/0.log"
Sep 12 00:59:41.074976 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:59:41.074950    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-wvbw5/perf-node-gather-daemonset-w6mz5"
Sep 12 00:59:42.095381 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:59:42.095354    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_kserve-controller-manager-d6f4b65b8-tzqmv_9d018424-bcd8-40d8-980a-33214e4ff532/manager/0.log"
Sep 12 00:59:42.328007 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:59:42.327967    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_llmisvc-controller-manager-777c6bd7d5-4bzq6_b6f1b332-882c-4dbf-907e-8b201235ab1c/manager/0.log"
Sep 12 00:59:42.376189 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:59:42.376113    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-57d99b475d-7dt66_af4c0706-719d-49f9-a742-789cad837da2/manager/0.log"
Sep 12 00:59:42.399615 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:59:42.399582    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_model-serving-api-54d89bd5fc-s4l9d_f4408e93-cbb1-4571-8b85-db15938ea0c0/server/0.log"
Sep 12 00:59:42.508656 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:59:42.508631    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-8446f459d6-dngdm_618d0f14-c54a-4aec-bba9-3d1c67ab23a7/manager/0.log"
Sep 12 00:59:42.530042 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:59:42.530026    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-8446f459d6-dngdm_618d0f14-c54a-4aec-bba9-3d1c67ab23a7/copy-manifests/0.log"
Sep 12 00:59:44.457439 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:59:44.457409    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-5677ccb49b-m9kwv_af206a61-acc5-4738-a83f-55c131da2017/discovery/0.log"
Sep 12 00:59:44.473138 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:59:44.473091    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-76b8774589-59jxv_e4d384f3-026e-4343-bada-ddc631f69d53/kube-auth-proxy/0.log"
Sep 12 00:59:44.508224 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:59:44.508198    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-78bb78ff78-svr8q_06461169-3e71-40a2-a0dc-e52b89db9f42/istio-proxy/0.log"
Sep 12 00:59:44.510711 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:59:44.510695    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-78bb78ff78-svr8q_06461169-3e71-40a2-a0dc-e52b89db9f42/istio-proxy/1.log"
Sep 12 00:59:44.528677 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:59:44.528659    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-59fbc47bf8-p2vwz_e335c05b-bd1d-468c-b882-f8a372524c56/payload-pre-processing/0.log"
Sep 12 00:59:44.551124 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:59:44.551103    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5b54b7fbb7-btmpn_817b7aba-94b1-4306-b26a-027e1e0b7297/payload-processing/0.log"
Sep 12 00:59:44.574415 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:59:44.574399    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-57b84c8474-8xjql_c2123986-4ae4-40ce-a423-b5e9a4accda2/router/0.log"
Sep 12 00:59:46.481085 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:59:46.481055    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-5677ccb49b-m9kwv_af206a61-acc5-4738-a83f-55c131da2017/discovery/0.log"
Sep 12 00:59:46.497723 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:59:46.497698    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-76b8774589-59jxv_e4d384f3-026e-4343-bada-ddc631f69d53/kube-auth-proxy/0.log"
Sep 12 00:59:46.531890 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:59:46.531868    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-78bb78ff78-svr8q_06461169-3e71-40a2-a0dc-e52b89db9f42/istio-proxy/0.log"
Sep 12 00:59:46.534246 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:59:46.534219    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-78bb78ff78-svr8q_06461169-3e71-40a2-a0dc-e52b89db9f42/istio-proxy/1.log"
Sep 12 00:59:46.554065 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:59:46.554043    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-59fbc47bf8-p2vwz_e335c05b-bd1d-468c-b882-f8a372524c56/payload-pre-processing/0.log"
Sep 12 00:59:46.580096 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:59:46.580076    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5b54b7fbb7-btmpn_817b7aba-94b1-4306-b26a-027e1e0b7297/payload-processing/0.log"
Sep 12 00:59:46.596443 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:59:46.596420    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-57b84c8474-8xjql_c2123986-4ae4-40ce-a423-b5e9a4accda2/router/0.log"
Sep 12 00:59:50.623568 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:59:50.623533    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-57l7n_76f830f1-1f3e-4dd4-9deb-023f6ad1e2e4/kube-multus/0.log"
Sep 12 00:59:50.644269 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:59:50.644246    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-62v9r_0a3e3c8c-cafa-4254-a972-fcf494ae4e28/kube-multus-additional-cni-plugins/0.log"
Sep 12 00:59:50.664048 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:59:50.663998    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-62v9r_0a3e3c8c-cafa-4254-a972-fcf494ae4e28/egress-router-binary-copy/0.log"
Sep 12 00:59:50.684094 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:59:50.684078    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-62v9r_0a3e3c8c-cafa-4254-a972-fcf494ae4e28/cni-plugins/0.log"
Sep 12 00:59:50.704302 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:59:50.704285    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-62v9r_0a3e3c8c-cafa-4254-a972-fcf494ae4e28/bond-cni-plugin/0.log"
Sep 12 00:59:50.724057 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:59:50.724035    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-62v9r_0a3e3c8c-cafa-4254-a972-fcf494ae4e28/routeoverride-cni/0.log"
Sep 12 00:59:50.743440 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:59:50.743418    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-62v9r_0a3e3c8c-cafa-4254-a972-fcf494ae4e28/whereabouts-cni-bincopy/0.log"
Sep 12 00:59:50.763503 ip-10-0-141-128 kubenswrapper[2571]: I0912 00:59:50.763485    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-62v9r_0a3e3c8c-cafa-4254-a972-fcf494ae4e28/whereabouts-cni/0.log"