Sep 09 03:13:04.592001 ip-10-0-132-14 systemd[1]: Starting Kubernetes Kubelet...
Sep 09 03:13:05.048241 ip-10-0-132-14 kubenswrapper[2576]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Sep 09 03:13:05.048241 ip-10-0-132-14 kubenswrapper[2576]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version.
Sep 09 03:13:05.048241 ip-10-0-132-14 kubenswrapper[2576]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Sep 09 03:13:05.048241 ip-10-0-132-14 kubenswrapper[2576]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI.
Sep 09 03:13:05.048241 ip-10-0-132-14 kubenswrapper[2576]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Sep 09 03:13:05.049375 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:13:05.049286    2576 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime"
Sep 09 03:13:05.053751 ip-10-0-132-14 kubenswrapper[2576]: W0909 03:13:05.053735    2576 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 09 03:13:05.053751 ip-10-0-132-14 kubenswrapper[2576]: W0909 03:13:05.053751    2576 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 09 03:13:05.053814 ip-10-0-132-14 kubenswrapper[2576]: W0909 03:13:05.053755    2576 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 09 03:13:05.053814 ip-10-0-132-14 kubenswrapper[2576]: W0909 03:13:05.053758    2576 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 09 03:13:05.053814 ip-10-0-132-14 kubenswrapper[2576]: W0909 03:13:05.053762    2576 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 09 03:13:05.053814 ip-10-0-132-14 kubenswrapper[2576]: W0909 03:13:05.053765    2576 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 09 03:13:05.053814 ip-10-0-132-14 kubenswrapper[2576]: W0909 03:13:05.053768    2576 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 09 03:13:05.053814 ip-10-0-132-14 kubenswrapper[2576]: W0909 03:13:05.053771    2576 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 09 03:13:05.053814 ip-10-0-132-14 kubenswrapper[2576]: W0909 03:13:05.053775    2576 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 09 03:13:05.053814 ip-10-0-132-14 kubenswrapper[2576]: W0909 03:13:05.053778    2576 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 09 03:13:05.053814 ip-10-0-132-14 kubenswrapper[2576]: W0909 03:13:05.053781    2576 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 09 03:13:05.053814 ip-10-0-132-14 kubenswrapper[2576]: W0909 03:13:05.053785    2576 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 09 03:13:05.053814 ip-10-0-132-14 kubenswrapper[2576]: W0909 03:13:05.053787    2576 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 09 03:13:05.053814 ip-10-0-132-14 kubenswrapper[2576]: W0909 03:13:05.053790    2576 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 09 03:13:05.053814 ip-10-0-132-14 kubenswrapper[2576]: W0909 03:13:05.053792    2576 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 09 03:13:05.053814 ip-10-0-132-14 kubenswrapper[2576]: W0909 03:13:05.053795    2576 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 09 03:13:05.053814 ip-10-0-132-14 kubenswrapper[2576]: W0909 03:13:05.053798    2576 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 09 03:13:05.053814 ip-10-0-132-14 kubenswrapper[2576]: W0909 03:13:05.053800    2576 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 09 03:13:05.053814 ip-10-0-132-14 kubenswrapper[2576]: W0909 03:13:05.053803    2576 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 09 03:13:05.053814 ip-10-0-132-14 kubenswrapper[2576]: W0909 03:13:05.053805    2576 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 09 03:13:05.053814 ip-10-0-132-14 kubenswrapper[2576]: W0909 03:13:05.053807    2576 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 09 03:13:05.053814 ip-10-0-132-14 kubenswrapper[2576]: W0909 03:13:05.053810    2576 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 09 03:13:05.054288 ip-10-0-132-14 kubenswrapper[2576]: W0909 03:13:05.053813    2576 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 09 03:13:05.054288 ip-10-0-132-14 kubenswrapper[2576]: W0909 03:13:05.053815    2576 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 09 03:13:05.054288 ip-10-0-132-14 kubenswrapper[2576]: W0909 03:13:05.053820    2576 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 09 03:13:05.054288 ip-10-0-132-14 kubenswrapper[2576]: W0909 03:13:05.053824    2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 09 03:13:05.054288 ip-10-0-132-14 kubenswrapper[2576]: W0909 03:13:05.053827    2576 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 09 03:13:05.054288 ip-10-0-132-14 kubenswrapper[2576]: W0909 03:13:05.053831    2576 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 09 03:13:05.054288 ip-10-0-132-14 kubenswrapper[2576]: W0909 03:13:05.053833    2576 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 09 03:13:05.054288 ip-10-0-132-14 kubenswrapper[2576]: W0909 03:13:05.053836    2576 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 09 03:13:05.054288 ip-10-0-132-14 kubenswrapper[2576]: W0909 03:13:05.053839    2576 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 09 03:13:05.054288 ip-10-0-132-14 kubenswrapper[2576]: W0909 03:13:05.053842    2576 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 09 03:13:05.054288 ip-10-0-132-14 kubenswrapper[2576]: W0909 03:13:05.053845    2576 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 09 03:13:05.054288 ip-10-0-132-14 kubenswrapper[2576]: W0909 03:13:05.053848    2576 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 09 03:13:05.054288 ip-10-0-132-14 kubenswrapper[2576]: W0909 03:13:05.053850    2576 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 09 03:13:05.054288 ip-10-0-132-14 kubenswrapper[2576]: W0909 03:13:05.053853    2576 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 09 03:13:05.054288 ip-10-0-132-14 kubenswrapper[2576]: W0909 03:13:05.053856    2576 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 09 03:13:05.054288 ip-10-0-132-14 kubenswrapper[2576]: W0909 03:13:05.053859    2576 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 09 03:13:05.054288 ip-10-0-132-14 kubenswrapper[2576]: W0909 03:13:05.053861    2576 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 09 03:13:05.054288 ip-10-0-132-14 kubenswrapper[2576]: W0909 03:13:05.053864    2576 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 09 03:13:05.054288 ip-10-0-132-14 kubenswrapper[2576]: W0909 03:13:05.053867    2576 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 09 03:13:05.054758 ip-10-0-132-14 kubenswrapper[2576]: W0909 03:13:05.053870    2576 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 09 03:13:05.054758 ip-10-0-132-14 kubenswrapper[2576]: W0909 03:13:05.053873    2576 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 09 03:13:05.054758 ip-10-0-132-14 kubenswrapper[2576]: W0909 03:13:05.053875    2576 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 09 03:13:05.054758 ip-10-0-132-14 kubenswrapper[2576]: W0909 03:13:05.053878    2576 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 09 03:13:05.054758 ip-10-0-132-14 kubenswrapper[2576]: W0909 03:13:05.053880    2576 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 09 03:13:05.054758 ip-10-0-132-14 kubenswrapper[2576]: W0909 03:13:05.053883    2576 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 09 03:13:05.054758 ip-10-0-132-14 kubenswrapper[2576]: W0909 03:13:05.053885    2576 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 09 03:13:05.054758 ip-10-0-132-14 kubenswrapper[2576]: W0909 03:13:05.053887    2576 feature_gate.go:328] unrecognized feature gate: Example
Sep 09 03:13:05.054758 ip-10-0-132-14 kubenswrapper[2576]: W0909 03:13:05.053890    2576 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 09 03:13:05.054758 ip-10-0-132-14 kubenswrapper[2576]: W0909 03:13:05.053892    2576 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 09 03:13:05.054758 ip-10-0-132-14 kubenswrapper[2576]: W0909 03:13:05.053896    2576 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 09 03:13:05.054758 ip-10-0-132-14 kubenswrapper[2576]: W0909 03:13:05.053898    2576 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 09 03:13:05.054758 ip-10-0-132-14 kubenswrapper[2576]: W0909 03:13:05.053901    2576 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 09 03:13:05.054758 ip-10-0-132-14 kubenswrapper[2576]: W0909 03:13:05.053904    2576 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 09 03:13:05.054758 ip-10-0-132-14 kubenswrapper[2576]: W0909 03:13:05.053907    2576 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 09 03:13:05.054758 ip-10-0-132-14 kubenswrapper[2576]: W0909 03:13:05.053909    2576 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 09 03:13:05.054758 ip-10-0-132-14 kubenswrapper[2576]: W0909 03:13:05.053912    2576 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 09 03:13:05.054758 ip-10-0-132-14 kubenswrapper[2576]: W0909 03:13:05.053915    2576 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 09 03:13:05.054758 ip-10-0-132-14 kubenswrapper[2576]: W0909 03:13:05.053917    2576 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 09 03:13:05.054758 ip-10-0-132-14 kubenswrapper[2576]: W0909 03:13:05.053919    2576 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 09 03:13:05.055235 ip-10-0-132-14 kubenswrapper[2576]: W0909 03:13:05.053922    2576 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 09 03:13:05.055235 ip-10-0-132-14 kubenswrapper[2576]: W0909 03:13:05.053924    2576 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 09 03:13:05.055235 ip-10-0-132-14 kubenswrapper[2576]: W0909 03:13:05.053926    2576 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 09 03:13:05.055235 ip-10-0-132-14 kubenswrapper[2576]: W0909 03:13:05.053929    2576 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 09 03:13:05.055235 ip-10-0-132-14 kubenswrapper[2576]: W0909 03:13:05.053931    2576 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 09 03:13:05.055235 ip-10-0-132-14 kubenswrapper[2576]: W0909 03:13:05.053934    2576 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 09 03:13:05.055235 ip-10-0-132-14 kubenswrapper[2576]: W0909 03:13:05.053936    2576 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 09 03:13:05.055235 ip-10-0-132-14 kubenswrapper[2576]: W0909 03:13:05.053939    2576 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 09 03:13:05.055235 ip-10-0-132-14 kubenswrapper[2576]: W0909 03:13:05.053941    2576 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 09 03:13:05.055235 ip-10-0-132-14 kubenswrapper[2576]: W0909 03:13:05.053944    2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 09 03:13:05.055235 ip-10-0-132-14 kubenswrapper[2576]: W0909 03:13:05.053947    2576 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 09 03:13:05.055235 ip-10-0-132-14 kubenswrapper[2576]: W0909 03:13:05.053951    2576 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 09 03:13:05.055235 ip-10-0-132-14 kubenswrapper[2576]: W0909 03:13:05.053953    2576 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 09 03:13:05.055235 ip-10-0-132-14 kubenswrapper[2576]: W0909 03:13:05.053957    2576 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 09 03:13:05.055235 ip-10-0-132-14 kubenswrapper[2576]: W0909 03:13:05.053960    2576 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 09 03:13:05.055235 ip-10-0-132-14 kubenswrapper[2576]: W0909 03:13:05.053962    2576 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 09 03:13:05.055235 ip-10-0-132-14 kubenswrapper[2576]: W0909 03:13:05.053965    2576 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 09 03:13:05.055235 ip-10-0-132-14 kubenswrapper[2576]: W0909 03:13:05.053968    2576 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 09 03:13:05.055235 ip-10-0-132-14 kubenswrapper[2576]: W0909 03:13:05.053971    2576 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 09 03:13:05.055235 ip-10-0-132-14 kubenswrapper[2576]: W0909 03:13:05.053973    2576 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 09 03:13:05.055723 ip-10-0-132-14 kubenswrapper[2576]: W0909 03:13:05.053976    2576 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 09 03:13:05.055723 ip-10-0-132-14 kubenswrapper[2576]: W0909 03:13:05.053979    2576 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 09 03:13:05.055723 ip-10-0-132-14 kubenswrapper[2576]: W0909 03:13:05.053982    2576 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 09 03:13:05.055723 ip-10-0-132-14 kubenswrapper[2576]: W0909 03:13:05.053984    2576 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 09 03:13:05.055723 ip-10-0-132-14 kubenswrapper[2576]: W0909 03:13:05.053987    2576 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 09 03:13:05.055723 ip-10-0-132-14 kubenswrapper[2576]: W0909 03:13:05.053989    2576 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 09 03:13:05.055723 ip-10-0-132-14 kubenswrapper[2576]: W0909 03:13:05.053992    2576 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 09 03:13:05.055723 ip-10-0-132-14 kubenswrapper[2576]: W0909 03:13:05.053996    2576 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 09 03:13:05.055723 ip-10-0-132-14 kubenswrapper[2576]: W0909 03:13:05.054000    2576 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 09 03:13:05.055723 ip-10-0-132-14 kubenswrapper[2576]: W0909 03:13:05.054003    2576 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 09 03:13:05.055723 ip-10-0-132-14 kubenswrapper[2576]: W0909 03:13:05.054006    2576 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 09 03:13:05.055723 ip-10-0-132-14 kubenswrapper[2576]: W0909 03:13:05.054009    2576 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 09 03:13:05.055723 ip-10-0-132-14 kubenswrapper[2576]: W0909 03:13:05.054012    2576 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 09 03:13:05.055723 ip-10-0-132-14 kubenswrapper[2576]: W0909 03:13:05.054015    2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 09 03:13:05.055723 ip-10-0-132-14 kubenswrapper[2576]: W0909 03:13:05.054017    2576 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 09 03:13:05.055723 ip-10-0-132-14 kubenswrapper[2576]: W0909 03:13:05.054021    2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 09 03:13:05.055723 ip-10-0-132-14 kubenswrapper[2576]: W0909 03:13:05.054024    2576 feature_gate.go:328] unrecognized feature gate: Example2
Sep 09 03:13:05.055723 ip-10-0-132-14 kubenswrapper[2576]: W0909 03:13:05.054026    2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 09 03:13:05.056137 ip-10-0-132-14 kubenswrapper[2576]: W0909 03:13:05.055773    2576 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 09 03:13:05.056137 ip-10-0-132-14 kubenswrapper[2576]: W0909 03:13:05.055781    2576 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 09 03:13:05.056137 ip-10-0-132-14 kubenswrapper[2576]: W0909 03:13:05.055784    2576 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 09 03:13:05.056137 ip-10-0-132-14 kubenswrapper[2576]: W0909 03:13:05.055788    2576 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 09 03:13:05.056137 ip-10-0-132-14 kubenswrapper[2576]: W0909 03:13:05.055790    2576 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 09 03:13:05.056137 ip-10-0-132-14 kubenswrapper[2576]: W0909 03:13:05.055793    2576 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 09 03:13:05.056137 ip-10-0-132-14 kubenswrapper[2576]: W0909 03:13:05.055796    2576 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 09 03:13:05.056137 ip-10-0-132-14 kubenswrapper[2576]: W0909 03:13:05.055798    2576 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 09 03:13:05.056137 ip-10-0-132-14 kubenswrapper[2576]: W0909 03:13:05.055802    2576 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 09 03:13:05.056137 ip-10-0-132-14 kubenswrapper[2576]: W0909 03:13:05.055804    2576 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 09 03:13:05.056137 ip-10-0-132-14 kubenswrapper[2576]: W0909 03:13:05.055807    2576 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 09 03:13:05.056137 ip-10-0-132-14 kubenswrapper[2576]: W0909 03:13:05.055810    2576 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 09 03:13:05.056137 ip-10-0-132-14 kubenswrapper[2576]: W0909 03:13:05.055812    2576 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 09 03:13:05.056137 ip-10-0-132-14 kubenswrapper[2576]: W0909 03:13:05.055815    2576 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 09 03:13:05.056137 ip-10-0-132-14 kubenswrapper[2576]: W0909 03:13:05.055818    2576 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 09 03:13:05.056137 ip-10-0-132-14 kubenswrapper[2576]: W0909 03:13:05.055821    2576 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 09 03:13:05.056137 ip-10-0-132-14 kubenswrapper[2576]: W0909 03:13:05.055823    2576 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 09 03:13:05.056137 ip-10-0-132-14 kubenswrapper[2576]: W0909 03:13:05.055825    2576 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 09 03:13:05.056137 ip-10-0-132-14 kubenswrapper[2576]: W0909 03:13:05.055828    2576 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 09 03:13:05.056650 ip-10-0-132-14 kubenswrapper[2576]: W0909 03:13:05.055831    2576 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 09 03:13:05.056650 ip-10-0-132-14 kubenswrapper[2576]: W0909 03:13:05.055833    2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 09 03:13:05.056650 ip-10-0-132-14 kubenswrapper[2576]: W0909 03:13:05.055836    2576 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 09 03:13:05.056650 ip-10-0-132-14 kubenswrapper[2576]: W0909 03:13:05.055838    2576 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 09 03:13:05.056650 ip-10-0-132-14 kubenswrapper[2576]: W0909 03:13:05.055840    2576 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 09 03:13:05.056650 ip-10-0-132-14 kubenswrapper[2576]: W0909 03:13:05.055843    2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 09 03:13:05.056650 ip-10-0-132-14 kubenswrapper[2576]: W0909 03:13:05.055845    2576 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 09 03:13:05.056650 ip-10-0-132-14 kubenswrapper[2576]: W0909 03:13:05.055848    2576 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 09 03:13:05.056650 ip-10-0-132-14 kubenswrapper[2576]: W0909 03:13:05.055858    2576 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 09 03:13:05.056650 ip-10-0-132-14 kubenswrapper[2576]: W0909 03:13:05.055861    2576 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 09 03:13:05.056650 ip-10-0-132-14 kubenswrapper[2576]: W0909 03:13:05.055864    2576 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 09 03:13:05.056650 ip-10-0-132-14 kubenswrapper[2576]: W0909 03:13:05.055866    2576 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 09 03:13:05.056650 ip-10-0-132-14 kubenswrapper[2576]: W0909 03:13:05.055869    2576 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 09 03:13:05.056650 ip-10-0-132-14 kubenswrapper[2576]: W0909 03:13:05.055871    2576 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 09 03:13:05.056650 ip-10-0-132-14 kubenswrapper[2576]: W0909 03:13:05.055875    2576 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 09 03:13:05.056650 ip-10-0-132-14 kubenswrapper[2576]: W0909 03:13:05.055878    2576 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 09 03:13:05.056650 ip-10-0-132-14 kubenswrapper[2576]: W0909 03:13:05.055881    2576 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 09 03:13:05.056650 ip-10-0-132-14 kubenswrapper[2576]: W0909 03:13:05.055883    2576 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 09 03:13:05.056650 ip-10-0-132-14 kubenswrapper[2576]: W0909 03:13:05.055886    2576 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 09 03:13:05.056650 ip-10-0-132-14 kubenswrapper[2576]: W0909 03:13:05.055888    2576 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 09 03:13:05.057161 ip-10-0-132-14 kubenswrapper[2576]: W0909 03:13:05.055891    2576 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 09 03:13:05.057161 ip-10-0-132-14 kubenswrapper[2576]: W0909 03:13:05.055895    2576 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 09 03:13:05.057161 ip-10-0-132-14 kubenswrapper[2576]: W0909 03:13:05.055901    2576 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 09 03:13:05.057161 ip-10-0-132-14 kubenswrapper[2576]: W0909 03:13:05.055905    2576 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 09 03:13:05.057161 ip-10-0-132-14 kubenswrapper[2576]: W0909 03:13:05.055908    2576 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 09 03:13:05.057161 ip-10-0-132-14 kubenswrapper[2576]: W0909 03:13:05.055910    2576 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 09 03:13:05.057161 ip-10-0-132-14 kubenswrapper[2576]: W0909 03:13:05.055913    2576 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 09 03:13:05.057161 ip-10-0-132-14 kubenswrapper[2576]: W0909 03:13:05.055915    2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 09 03:13:05.057161 ip-10-0-132-14 kubenswrapper[2576]: W0909 03:13:05.055918    2576 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 09 03:13:05.057161 ip-10-0-132-14 kubenswrapper[2576]: W0909 03:13:05.055920    2576 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 09 03:13:05.057161 ip-10-0-132-14 kubenswrapper[2576]: W0909 03:13:05.055923    2576 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 09 03:13:05.057161 ip-10-0-132-14 kubenswrapper[2576]: W0909 03:13:05.055925    2576 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 09 03:13:05.057161 ip-10-0-132-14 kubenswrapper[2576]: W0909 03:13:05.055927    2576 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 09 03:13:05.057161 ip-10-0-132-14 kubenswrapper[2576]: W0909 03:13:05.055931    2576 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 09 03:13:05.057161 ip-10-0-132-14 kubenswrapper[2576]: W0909 03:13:05.055934    2576 feature_gate.go:328] unrecognized feature gate: Example2
Sep 09 03:13:05.057161 ip-10-0-132-14 kubenswrapper[2576]: W0909 03:13:05.055937    2576 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 09 03:13:05.057161 ip-10-0-132-14 kubenswrapper[2576]: W0909 03:13:05.055939    2576 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 09 03:13:05.057161 ip-10-0-132-14 kubenswrapper[2576]: W0909 03:13:05.055942    2576 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 09 03:13:05.057161 ip-10-0-132-14 kubenswrapper[2576]: W0909 03:13:05.055944    2576 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 09 03:13:05.057616 ip-10-0-132-14 kubenswrapper[2576]: W0909 03:13:05.055947    2576 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 09 03:13:05.057616 ip-10-0-132-14 kubenswrapper[2576]: W0909 03:13:05.055949    2576 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 09 03:13:05.057616 ip-10-0-132-14 kubenswrapper[2576]: W0909 03:13:05.055952    2576 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 09 03:13:05.057616 ip-10-0-132-14 kubenswrapper[2576]: W0909 03:13:05.055954    2576 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 09 03:13:05.057616 ip-10-0-132-14 kubenswrapper[2576]: W0909 03:13:05.055956    2576 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 09 03:13:05.057616 ip-10-0-132-14 kubenswrapper[2576]: W0909 03:13:05.055959    2576 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 09 03:13:05.057616 ip-10-0-132-14 kubenswrapper[2576]: W0909 03:13:05.055961    2576 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 09 03:13:05.057616 ip-10-0-132-14 kubenswrapper[2576]: W0909 03:13:05.055965    2576 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 09 03:13:05.057616 ip-10-0-132-14 kubenswrapper[2576]: W0909 03:13:05.055967    2576 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 09 03:13:05.057616 ip-10-0-132-14 kubenswrapper[2576]: W0909 03:13:05.055970    2576 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 09 03:13:05.057616 ip-10-0-132-14 kubenswrapper[2576]: W0909 03:13:05.055973    2576 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 09 03:13:05.057616 ip-10-0-132-14 kubenswrapper[2576]: W0909 03:13:05.055975    2576 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 09 03:13:05.057616 ip-10-0-132-14 kubenswrapper[2576]: W0909 03:13:05.055978    2576 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 09 03:13:05.057616 ip-10-0-132-14 kubenswrapper[2576]: W0909 03:13:05.055981    2576 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 09 03:13:05.057616 ip-10-0-132-14 kubenswrapper[2576]: W0909 03:13:05.055983    2576 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 09 03:13:05.057616 ip-10-0-132-14 kubenswrapper[2576]: W0909 03:13:05.055986    2576 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 09 03:13:05.057616 ip-10-0-132-14 kubenswrapper[2576]: W0909 03:13:05.055990    2576 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 09 03:13:05.057616 ip-10-0-132-14 kubenswrapper[2576]: W0909 03:13:05.055992    2576 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 09 03:13:05.057616 ip-10-0-132-14 kubenswrapper[2576]: W0909 03:13:05.055995    2576 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 09 03:13:05.057616 ip-10-0-132-14 kubenswrapper[2576]: W0909 03:13:05.055997    2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 09 03:13:05.058104 ip-10-0-132-14 kubenswrapper[2576]: W0909 03:13:05.055999    2576 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 09 03:13:05.058104 ip-10-0-132-14 kubenswrapper[2576]: W0909 03:13:05.056002    2576 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 09 03:13:05.058104 ip-10-0-132-14 kubenswrapper[2576]: W0909 03:13:05.056004    2576 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 09 03:13:05.058104 ip-10-0-132-14 kubenswrapper[2576]: W0909 03:13:05.056006    2576 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 09 03:13:05.058104 ip-10-0-132-14 kubenswrapper[2576]: W0909 03:13:05.056009    2576 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 09 03:13:05.058104 ip-10-0-132-14 kubenswrapper[2576]: W0909 03:13:05.056011    2576 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 09 03:13:05.058104 ip-10-0-132-14 kubenswrapper[2576]: W0909 03:13:05.056014    2576 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 09 03:13:05.058104 ip-10-0-132-14 kubenswrapper[2576]: W0909 03:13:05.056017    2576 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 09 03:13:05.058104 ip-10-0-132-14 kubenswrapper[2576]: W0909 03:13:05.056020    2576 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 09 03:13:05.058104 ip-10-0-132-14 kubenswrapper[2576]: W0909 03:13:05.056022    2576 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 09 03:13:05.058104 ip-10-0-132-14 kubenswrapper[2576]: W0909 03:13:05.056024    2576 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 09 03:13:05.058104 ip-10-0-132-14 kubenswrapper[2576]: W0909 03:13:05.056027    2576 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 09 03:13:05.058104 ip-10-0-132-14 kubenswrapper[2576]: W0909 03:13:05.056029    2576 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 09 03:13:05.058104 ip-10-0-132-14 kubenswrapper[2576]: W0909 03:13:05.056032    2576 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 09 03:13:05.058104 ip-10-0-132-14 kubenswrapper[2576]: W0909 03:13:05.056035    2576 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 09 03:13:05.058104 ip-10-0-132-14 kubenswrapper[2576]: W0909 03:13:05.056037    2576 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 09 03:13:05.058104 ip-10-0-132-14 kubenswrapper[2576]: W0909 03:13:05.056039    2576 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 09 03:13:05.058104 ip-10-0-132-14 kubenswrapper[2576]: W0909 03:13:05.056042    2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 09 03:13:05.058104 ip-10-0-132-14 kubenswrapper[2576]: W0909 03:13:05.056044    2576 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 09 03:13:05.058104 ip-10-0-132-14 kubenswrapper[2576]: W0909 03:13:05.056048    2576 feature_gate.go:328] unrecognized feature gate: Example
Sep 09 03:13:05.058695 ip-10-0-132-14 kubenswrapper[2576]: W0909 03:13:05.056052    2576 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 09 03:13:05.058695 ip-10-0-132-14 kubenswrapper[2576]: W0909 03:13:05.056433    2576 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 09 03:13:05.058695 ip-10-0-132-14 kubenswrapper[2576]: W0909 03:13:05.056437    2576 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 09 03:13:05.058695 ip-10-0-132-14 kubenswrapper[2576]: W0909 03:13:05.056440    2576 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 09 03:13:05.058695 ip-10-0-132-14 kubenswrapper[2576]: W0909 03:13:05.056443    2576 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 09 03:13:05.058695 ip-10-0-132-14 kubenswrapper[2576]: W0909 03:13:05.056446    2576 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 09 03:13:05.058695 ip-10-0-132-14 kubenswrapper[2576]: W0909 03:13:05.056448    2576 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 09 03:13:05.058695 ip-10-0-132-14 kubenswrapper[2576]: W0909 03:13:05.056451    2576 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 09 03:13:05.058695 ip-10-0-132-14 kubenswrapper[2576]: W0909 03:13:05.056454    2576 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 09 03:13:05.058695 ip-10-0-132-14 kubenswrapper[2576]: W0909 03:13:05.056457    2576 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 09 03:13:05.058695 ip-10-0-132-14 kubenswrapper[2576]: W0909 03:13:05.056459    2576 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 09 03:13:05.058695 ip-10-0-132-14 kubenswrapper[2576]: W0909 03:13:05.056462    2576 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 09 03:13:05.058695 ip-10-0-132-14 kubenswrapper[2576]: W0909 03:13:05.056464    2576 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 09 03:13:05.058695 ip-10-0-132-14 kubenswrapper[2576]: W0909 03:13:05.056467    2576 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 09 03:13:05.058695 ip-10-0-132-14 kubenswrapper[2576]: W0909 03:13:05.056469    2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 09 03:13:05.058695 ip-10-0-132-14 kubenswrapper[2576]: W0909 03:13:05.056472    2576 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 09 03:13:05.058695 ip-10-0-132-14 kubenswrapper[2576]: W0909 03:13:05.056474    2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 09 03:13:05.058695 ip-10-0-132-14 kubenswrapper[2576]: W0909 03:13:05.056477    2576 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 09 03:13:05.058695 ip-10-0-132-14 kubenswrapper[2576]: W0909 03:13:05.056480    2576 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 09 03:13:05.058695 ip-10-0-132-14 kubenswrapper[2576]: W0909 03:13:05.056482    2576 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 09 03:13:05.059172 ip-10-0-132-14 kubenswrapper[2576]: W0909 03:13:05.056484    2576 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 09 03:13:05.059172 ip-10-0-132-14 kubenswrapper[2576]: W0909 03:13:05.056487    2576 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 09 03:13:05.059172 ip-10-0-132-14 kubenswrapper[2576]: W0909 03:13:05.056489    2576 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 09 03:13:05.059172 ip-10-0-132-14 kubenswrapper[2576]: W0909 03:13:05.056492    2576 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 09 03:13:05.059172 ip-10-0-132-14 kubenswrapper[2576]: W0909 03:13:05.056494    2576 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 09 03:13:05.059172 ip-10-0-132-14 kubenswrapper[2576]: W0909 03:13:05.056497    2576 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 09 03:13:05.059172 ip-10-0-132-14 kubenswrapper[2576]: W0909 03:13:05.056500    2576 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 09 03:13:05.059172 ip-10-0-132-14 kubenswrapper[2576]: W0909 03:13:05.056502    2576 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 09 03:13:05.059172 ip-10-0-132-14 kubenswrapper[2576]: W0909 03:13:05.056505    2576 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 09 03:13:05.059172 ip-10-0-132-14 kubenswrapper[2576]: W0909 03:13:05.056507    2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 09 03:13:05.059172 ip-10-0-132-14 kubenswrapper[2576]: W0909 03:13:05.056509    2576 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 09 03:13:05.059172 ip-10-0-132-14 kubenswrapper[2576]: W0909 03:13:05.056514    2576 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 09 03:13:05.059172 ip-10-0-132-14 kubenswrapper[2576]: W0909 03:13:05.056518    2576 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 09 03:13:05.059172 ip-10-0-132-14 kubenswrapper[2576]: W0909 03:13:05.056521    2576 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 09 03:13:05.059172 ip-10-0-132-14 kubenswrapper[2576]: W0909 03:13:05.056523    2576 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 09 03:13:05.059172 ip-10-0-132-14 kubenswrapper[2576]: W0909 03:13:05.056528    2576 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 09 03:13:05.059172 ip-10-0-132-14 kubenswrapper[2576]: W0909 03:13:05.056531    2576 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 09 03:13:05.059172 ip-10-0-132-14 kubenswrapper[2576]: W0909 03:13:05.056534    2576 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 09 03:13:05.059172 ip-10-0-132-14 kubenswrapper[2576]: W0909 03:13:05.056537    2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 09 03:13:05.059637 ip-10-0-132-14 kubenswrapper[2576]: W0909 03:13:05.056540    2576 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 09 03:13:05.059637 ip-10-0-132-14 kubenswrapper[2576]: W0909 03:13:05.056543    2576 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 09 03:13:05.059637 ip-10-0-132-14 kubenswrapper[2576]: W0909 03:13:05.056545    2576 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 09 03:13:05.059637 ip-10-0-132-14 kubenswrapper[2576]: W0909 03:13:05.056548    2576 feature_gate.go:328] unrecognized feature gate: Example2
Sep 09 03:13:05.059637 ip-10-0-132-14 kubenswrapper[2576]: W0909 03:13:05.056550    2576 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 09 03:13:05.059637 ip-10-0-132-14 kubenswrapper[2576]: W0909 03:13:05.056553    2576 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 09 03:13:05.059637 ip-10-0-132-14 kubenswrapper[2576]: W0909 03:13:05.056555    2576 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 09 03:13:05.059637 ip-10-0-132-14 kubenswrapper[2576]: W0909 03:13:05.056557    2576 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 09 03:13:05.059637 ip-10-0-132-14 kubenswrapper[2576]: W0909 03:13:05.056560    2576 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 09 03:13:05.059637 ip-10-0-132-14 kubenswrapper[2576]: W0909 03:13:05.056562    2576 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 09 03:13:05.059637 ip-10-0-132-14 kubenswrapper[2576]: W0909 03:13:05.056564    2576 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 09 03:13:05.059637 ip-10-0-132-14 kubenswrapper[2576]: W0909 03:13:05.056567    2576 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 09 03:13:05.059637 ip-10-0-132-14 kubenswrapper[2576]: W0909 03:13:05.056569    2576 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 09 03:13:05.059637 ip-10-0-132-14 kubenswrapper[2576]: W0909 03:13:05.056571    2576 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 09 03:13:05.059637 ip-10-0-132-14 kubenswrapper[2576]: W0909 03:13:05.056574    2576 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 09 03:13:05.059637 ip-10-0-132-14 kubenswrapper[2576]: W0909 03:13:05.056576    2576 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 09 03:13:05.059637 ip-10-0-132-14 kubenswrapper[2576]: W0909 03:13:05.056578    2576 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 09 03:13:05.059637 ip-10-0-132-14 kubenswrapper[2576]: W0909 03:13:05.056581    2576 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 09 03:13:05.059637 ip-10-0-132-14 kubenswrapper[2576]: W0909 03:13:05.056583    2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 09 03:13:05.059637 ip-10-0-132-14 kubenswrapper[2576]: W0909 03:13:05.056585    2576 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 09 03:13:05.060123 ip-10-0-132-14 kubenswrapper[2576]: W0909 03:13:05.056588    2576 feature_gate.go:328] unrecognized feature gate: Example
Sep 09 03:13:05.060123 ip-10-0-132-14 kubenswrapper[2576]: W0909 03:13:05.056590    2576 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 09 03:13:05.060123 ip-10-0-132-14 kubenswrapper[2576]: W0909 03:13:05.056595    2576 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 09 03:13:05.060123 ip-10-0-132-14 kubenswrapper[2576]: W0909 03:13:05.056598    2576 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 09 03:13:05.060123 ip-10-0-132-14 kubenswrapper[2576]: W0909 03:13:05.056602    2576 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 09 03:13:05.060123 ip-10-0-132-14 kubenswrapper[2576]: W0909 03:13:05.056605    2576 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 09 03:13:05.060123 ip-10-0-132-14 kubenswrapper[2576]: W0909 03:13:05.056608    2576 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 09 03:13:05.060123 ip-10-0-132-14 kubenswrapper[2576]: W0909 03:13:05.056610    2576 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 09 03:13:05.060123 ip-10-0-132-14 kubenswrapper[2576]: W0909 03:13:05.056613    2576 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 09 03:13:05.060123 ip-10-0-132-14 kubenswrapper[2576]: W0909 03:13:05.056616    2576 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 09 03:13:05.060123 ip-10-0-132-14 kubenswrapper[2576]: W0909 03:13:05.056619    2576 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 09 03:13:05.060123 ip-10-0-132-14 kubenswrapper[2576]: W0909 03:13:05.056621    2576 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 09 03:13:05.060123 ip-10-0-132-14 kubenswrapper[2576]: W0909 03:13:05.056639    2576 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 09 03:13:05.060123 ip-10-0-132-14 kubenswrapper[2576]: W0909 03:13:05.056644    2576 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 09 03:13:05.060123 ip-10-0-132-14 kubenswrapper[2576]: W0909 03:13:05.056648    2576 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 09 03:13:05.060123 ip-10-0-132-14 kubenswrapper[2576]: W0909 03:13:05.056652    2576 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 09 03:13:05.060123 ip-10-0-132-14 kubenswrapper[2576]: W0909 03:13:05.056657    2576 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 09 03:13:05.060123 ip-10-0-132-14 kubenswrapper[2576]: W0909 03:13:05.056660    2576 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 09 03:13:05.060123 ip-10-0-132-14 kubenswrapper[2576]: W0909 03:13:05.056672    2576 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 09 03:13:05.060597 ip-10-0-132-14 kubenswrapper[2576]: W0909 03:13:05.056676    2576 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 09 03:13:05.060597 ip-10-0-132-14 kubenswrapper[2576]: W0909 03:13:05.056679    2576 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 09 03:13:05.060597 ip-10-0-132-14 kubenswrapper[2576]: W0909 03:13:05.056681    2576 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 09 03:13:05.060597 ip-10-0-132-14 kubenswrapper[2576]: W0909 03:13:05.056685    2576 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 09 03:13:05.060597 ip-10-0-132-14 kubenswrapper[2576]: W0909 03:13:05.056688    2576 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 09 03:13:05.060597 ip-10-0-132-14 kubenswrapper[2576]: W0909 03:13:05.056690    2576 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 09 03:13:05.060597 ip-10-0-132-14 kubenswrapper[2576]: W0909 03:13:05.056692    2576 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 09 03:13:05.060597 ip-10-0-132-14 kubenswrapper[2576]: W0909 03:13:05.056696    2576 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 09 03:13:05.060597 ip-10-0-132-14 kubenswrapper[2576]: W0909 03:13:05.056698    2576 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 09 03:13:05.060597 ip-10-0-132-14 kubenswrapper[2576]: W0909 03:13:05.056700    2576 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 09 03:13:05.060597 ip-10-0-132-14 kubenswrapper[2576]: W0909 03:13:05.056703    2576 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 09 03:13:05.060597 ip-10-0-132-14 kubenswrapper[2576]: W0909 03:13:05.056705    2576 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 09 03:13:05.060597 ip-10-0-132-14 kubenswrapper[2576]: W0909 03:13:05.056708    2576 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 09 03:13:05.060597 ip-10-0-132-14 kubenswrapper[2576]: W0909 03:13:05.056711    2576 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 09 03:13:05.060597 ip-10-0-132-14 kubenswrapper[2576]: W0909 03:13:05.056714    2576 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 09 03:13:05.060597 ip-10-0-132-14 kubenswrapper[2576]: W0909 03:13:05.056716    2576 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 09 03:13:05.060597 ip-10-0-132-14 kubenswrapper[2576]: W0909 03:13:05.056719    2576 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 09 03:13:05.060597 ip-10-0-132-14 kubenswrapper[2576]: W0909 03:13:05.056722    2576 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 09 03:13:05.060597 ip-10-0-132-14 kubenswrapper[2576]: W0909 03:13:05.056724    2576 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 09 03:13:05.060597 ip-10-0-132-14 kubenswrapper[2576]: W0909 03:13:05.056727    2576 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 09 03:13:05.061105 ip-10-0-132-14 kubenswrapper[2576]: W0909 03:13:05.056729    2576 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 09 03:13:05.061105 ip-10-0-132-14 kubenswrapper[2576]: W0909 03:13:05.056732    2576 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 09 03:13:05.061105 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:13:05.056807    2576 flags.go:64] FLAG: --address="0.0.0.0"
Sep 09 03:13:05.061105 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:13:05.056847    2576 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]"
Sep 09 03:13:05.061105 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:13:05.056853    2576 flags.go:64] FLAG: --anonymous-auth="true"
Sep 09 03:13:05.061105 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:13:05.056858    2576 flags.go:64] FLAG: --application-metrics-count-limit="100"
Sep 09 03:13:05.061105 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:13:05.056862    2576 flags.go:64] FLAG: --authentication-token-webhook="false"
Sep 09 03:13:05.061105 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:13:05.056866    2576 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s"
Sep 09 03:13:05.061105 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:13:05.056870    2576 flags.go:64] FLAG: --authorization-mode="AlwaysAllow"
Sep 09 03:13:05.061105 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:13:05.056875    2576 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s"
Sep 09 03:13:05.061105 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:13:05.056878    2576 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s"
Sep 09 03:13:05.061105 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:13:05.056881    2576 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id"
Sep 09 03:13:05.061105 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:13:05.056885    2576 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig"
Sep 09 03:13:05.061105 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:13:05.056888    2576 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki"
Sep 09 03:13:05.061105 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:13:05.056891    2576 flags.go:64] FLAG: --cgroup-driver="cgroupfs"
Sep 09 03:13:05.061105 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:13:05.056894    2576 flags.go:64] FLAG: --cgroup-root=""
Sep 09 03:13:05.061105 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:13:05.056897    2576 flags.go:64] FLAG: --cgroups-per-qos="true"
Sep 09 03:13:05.061105 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:13:05.056900    2576 flags.go:64] FLAG: --client-ca-file=""
Sep 09 03:13:05.061105 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:13:05.056902    2576 flags.go:64] FLAG: --cloud-provider="external"
Sep 09 03:13:05.061105 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:13:05.056905    2576 flags.go:64] FLAG: --cluster-dns="[]"
Sep 09 03:13:05.061105 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:13:05.056910    2576 flags.go:64] FLAG: --cluster-domain=""
Sep 09 03:13:05.061105 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:13:05.056912    2576 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf"
Sep 09 03:13:05.061105 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:13:05.056915    2576 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d"
Sep 09 03:13:05.061105 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:13:05.056918    2576 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json"
Sep 09 03:13:05.061725 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:13:05.056921    2576 flags.go:64] FLAG: --container-log-max-files="5"
Sep 09 03:13:05.061725 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:13:05.056926    2576 flags.go:64] FLAG: --container-log-max-size="10Mi"
Sep 09 03:13:05.061725 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:13:05.056929    2576 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock"
Sep 09 03:13:05.061725 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:13:05.056932    2576 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock"
Sep 09 03:13:05.061725 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:13:05.056935    2576 flags.go:64] FLAG: --containerd-namespace="k8s.io"
Sep 09 03:13:05.061725 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:13:05.056939    2576 flags.go:64] FLAG: --contention-profiling="false"
Sep 09 03:13:05.061725 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:13:05.056942    2576 flags.go:64] FLAG: --cpu-cfs-quota="true"
Sep 09 03:13:05.061725 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:13:05.056945    2576 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms"
Sep 09 03:13:05.061725 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:13:05.056948    2576 flags.go:64] FLAG: --cpu-manager-policy="none"
Sep 09 03:13:05.061725 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:13:05.056951    2576 flags.go:64] FLAG: --cpu-manager-policy-options=""
Sep 09 03:13:05.061725 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:13:05.056955    2576 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s"
Sep 09 03:13:05.061725 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:13:05.056958    2576 flags.go:64] FLAG: --enable-controller-attach-detach="true"
Sep 09 03:13:05.061725 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:13:05.056961    2576 flags.go:64] FLAG: --enable-debugging-handlers="true"
Sep 09 03:13:05.061725 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:13:05.056964    2576 flags.go:64] FLAG: --enable-load-reader="false"
Sep 09 03:13:05.061725 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:13:05.056967    2576 flags.go:64] FLAG: --enable-server="true"
Sep 09 03:13:05.061725 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:13:05.056970    2576 flags.go:64] FLAG: --enforce-node-allocatable="[pods]"
Sep 09 03:13:05.061725 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:13:05.056974    2576 flags.go:64] FLAG: --event-burst="100"
Sep 09 03:13:05.061725 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:13:05.056977    2576 flags.go:64] FLAG: --event-qps="50"
Sep 09 03:13:05.061725 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:13:05.056980    2576 flags.go:64] FLAG: --event-storage-age-limit="default=0"
Sep 09 03:13:05.061725 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:13:05.056983    2576 flags.go:64] FLAG: --event-storage-event-limit="default=0"
Sep 09 03:13:05.061725 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:13:05.056986    2576 flags.go:64] FLAG: --eviction-hard=""
Sep 09 03:13:05.061725 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:13:05.056990    2576 flags.go:64] FLAG: --eviction-max-pod-grace-period="0"
Sep 09 03:13:05.061725 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:13:05.056993    2576 flags.go:64] FLAG: --eviction-minimum-reclaim=""
Sep 09 03:13:05.061725 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:13:05.056995    2576 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s"
Sep 09 03:13:05.061725 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:13:05.056998    2576 flags.go:64] FLAG: --eviction-soft=""
Sep 09 03:13:05.062328 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:13:05.057001    2576 flags.go:64] FLAG: --eviction-soft-grace-period=""
Sep 09 03:13:05.062328 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:13:05.057004    2576 flags.go:64] FLAG: --exit-on-lock-contention="false"
Sep 09 03:13:05.062328 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:13:05.057007    2576 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false"
Sep 09 03:13:05.062328 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:13:05.057010    2576 flags.go:64] FLAG: --experimental-mounter-path=""
Sep 09 03:13:05.062328 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:13:05.057012    2576 flags.go:64] FLAG: --fail-cgroupv1="false"
Sep 09 03:13:05.062328 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:13:05.057015    2576 flags.go:64] FLAG: --fail-swap-on="true"
Sep 09 03:13:05.062328 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:13:05.057018    2576 flags.go:64] FLAG: --feature-gates=""
Sep 09 03:13:05.062328 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:13:05.057022    2576 flags.go:64] FLAG: --file-check-frequency="20s"
Sep 09 03:13:05.062328 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:13:05.057025    2576 flags.go:64] FLAG: --global-housekeeping-interval="1m0s"
Sep 09 03:13:05.062328 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:13:05.057028    2576 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge"
Sep 09 03:13:05.062328 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:13:05.057031    2576 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1"
Sep 09 03:13:05.062328 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:13:05.057034    2576 flags.go:64] FLAG: --healthz-port="10248"
Sep 09 03:13:05.062328 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:13:05.057037    2576 flags.go:64] FLAG: --help="false"
Sep 09 03:13:05.062328 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:13:05.057040    2576 flags.go:64] FLAG: --hostname-override="ip-10-0-132-14.ec2.internal"
Sep 09 03:13:05.062328 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:13:05.057043    2576 flags.go:64] FLAG: --housekeeping-interval="10s"
Sep 09 03:13:05.062328 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:13:05.057046    2576 flags.go:64] FLAG: --http-check-frequency="20s"
Sep 09 03:13:05.062328 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:13:05.057048    2576 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins"
Sep 09 03:13:05.062328 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:13:05.057051    2576 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml"
Sep 09 03:13:05.062328 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:13:05.057054    2576 flags.go:64] FLAG: --image-gc-high-threshold="85"
Sep 09 03:13:05.062328 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:13:05.057057    2576 flags.go:64] FLAG: --image-gc-low-threshold="80"
Sep 09 03:13:05.062328 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:13:05.057060    2576 flags.go:64] FLAG: --image-service-endpoint=""
Sep 09 03:13:05.062328 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:13:05.057063    2576 flags.go:64] FLAG: --kernel-memcg-notification="false"
Sep 09 03:13:05.062328 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:13:05.057066    2576 flags.go:64] FLAG: --kube-api-burst="100"
Sep 09 03:13:05.062907 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:13:05.057068    2576 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf"
Sep 09 03:13:05.062907 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:13:05.057071    2576 flags.go:64] FLAG: --kube-api-qps="50"
Sep 09 03:13:05.062907 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:13:05.057074    2576 flags.go:64] FLAG: --kube-reserved=""
Sep 09 03:13:05.062907 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:13:05.057077    2576 flags.go:64] FLAG: --kube-reserved-cgroup=""
Sep 09 03:13:05.062907 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:13:05.057080    2576 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig"
Sep 09 03:13:05.062907 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:13:05.057083    2576 flags.go:64] FLAG: --kubelet-cgroups=""
Sep 09 03:13:05.062907 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:13:05.057087    2576 flags.go:64] FLAG: --local-storage-capacity-isolation="true"
Sep 09 03:13:05.062907 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:13:05.057090    2576 flags.go:64] FLAG: --lock-file=""
Sep 09 03:13:05.062907 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:13:05.057093    2576 flags.go:64] FLAG: --log-cadvisor-usage="false"
Sep 09 03:13:05.062907 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:13:05.057095    2576 flags.go:64] FLAG: --log-flush-frequency="5s"
Sep 09 03:13:05.062907 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:13:05.057098    2576 flags.go:64] FLAG: --log-json-info-buffer-size="0"
Sep 09 03:13:05.062907 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:13:05.057103    2576 flags.go:64] FLAG: --log-json-split-stream="false"
Sep 09 03:13:05.062907 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:13:05.057106    2576 flags.go:64] FLAG: --log-text-info-buffer-size="0"
Sep 09 03:13:05.062907 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:13:05.057109    2576 flags.go:64] FLAG: --log-text-split-stream="false"
Sep 09 03:13:05.062907 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:13:05.057111    2576 flags.go:64] FLAG: --logging-format="text"
Sep 09 03:13:05.062907 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:13:05.057116    2576 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id"
Sep 09 03:13:05.062907 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:13:05.057119    2576 flags.go:64] FLAG: --make-iptables-util-chains="true"
Sep 09 03:13:05.062907 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:13:05.057122    2576 flags.go:64] FLAG: --manifest-url=""
Sep 09 03:13:05.062907 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:13:05.057125    2576 flags.go:64] FLAG: --manifest-url-header=""
Sep 09 03:13:05.062907 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:13:05.057130    2576 flags.go:64] FLAG: --max-housekeeping-interval="15s"
Sep 09 03:13:05.062907 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:13:05.057133    2576 flags.go:64] FLAG: --max-open-files="1000000"
Sep 09 03:13:05.062907 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:13:05.057137    2576 flags.go:64] FLAG: --max-pods="110"
Sep 09 03:13:05.062907 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:13:05.057140    2576 flags.go:64] FLAG: --maximum-dead-containers="-1"
Sep 09 03:13:05.062907 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:13:05.057143    2576 flags.go:64] FLAG: --maximum-dead-containers-per-container="1"
Sep 09 03:13:05.062907 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:13:05.057146    2576 flags.go:64] FLAG: --memory-manager-policy="None"
Sep 09 03:13:05.063509 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:13:05.057149    2576 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s"
Sep 09 03:13:05.063509 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:13:05.057152    2576 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s"
Sep 09 03:13:05.063509 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:13:05.057155    2576 flags.go:64] FLAG: --node-ip="0.0.0.0"
Sep 09 03:13:05.063509 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:13:05.057158    2576 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel"
Sep 09 03:13:05.063509 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:13:05.057165    2576 flags.go:64] FLAG: --node-status-max-images="50"
Sep 09 03:13:05.063509 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:13:05.057169    2576 flags.go:64] FLAG: --node-status-update-frequency="10s"
Sep 09 03:13:05.063509 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:13:05.057172    2576 flags.go:64] FLAG: --oom-score-adj="-999"
Sep 09 03:13:05.063509 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:13:05.057175    2576 flags.go:64] FLAG: --pod-cidr=""
Sep 09 03:13:05.063509 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:13:05.057178    2576 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:8ccbacc999dbf9abad8f4009240188312b683f36ced4763eed166c1500c164b1"
Sep 09 03:13:05.063509 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:13:05.057183    2576 flags.go:64] FLAG: --pod-manifest-path=""
Sep 09 03:13:05.063509 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:13:05.057186    2576 flags.go:64] FLAG: --pod-max-pids="-1"
Sep 09 03:13:05.063509 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:13:05.057190    2576 flags.go:64] FLAG: --pods-per-core="0"
Sep 09 03:13:05.063509 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:13:05.057193    2576 flags.go:64] FLAG: --port="10250"
Sep 09 03:13:05.063509 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:13:05.057198    2576 flags.go:64] FLAG: --protect-kernel-defaults="false"
Sep 09 03:13:05.063509 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:13:05.057200    2576 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-07cc70b6853d2f3d3"
Sep 09 03:13:05.063509 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:13:05.057204    2576 flags.go:64] FLAG: --qos-reserved=""
Sep 09 03:13:05.063509 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:13:05.057207    2576 flags.go:64] FLAG: --read-only-port="10255"
Sep 09 03:13:05.063509 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:13:05.057210    2576 flags.go:64] FLAG: --register-node="true"
Sep 09 03:13:05.063509 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:13:05.057213    2576 flags.go:64] FLAG: --register-with-taints=""
Sep 09 03:13:05.063509 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:13:05.057216    2576 flags.go:64] FLAG: --registry-burst="10"
Sep 09 03:13:05.063509 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:13:05.057219    2576 flags.go:64] FLAG: --registry-qps="5"
Sep 09 03:13:05.063509 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:13:05.057222    2576 flags.go:64] FLAG: --reserved-cpus=""
Sep 09 03:13:05.063509 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:13:05.057224    2576 flags.go:64] FLAG: --reserved-memory=""
Sep 09 03:13:05.063509 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:13:05.057231    2576 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf"
Sep 09 03:13:05.063509 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:13:05.057234    2576 flags.go:64] FLAG: --root-dir="/var/lib/kubelet"
Sep 09 03:13:05.064127 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:13:05.057237    2576 flags.go:64] FLAG: --rotate-certificates="false"
Sep 09 03:13:05.064127 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:13:05.057240    2576 flags.go:64] FLAG: --rotate-server-certificates="false"
Sep 09 03:13:05.064127 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:13:05.057243    2576 flags.go:64] FLAG: --runonce="false"
Sep 09 03:13:05.064127 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:13:05.057246    2576 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service"
Sep 09 03:13:05.064127 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:13:05.057249    2576 flags.go:64] FLAG: --runtime-request-timeout="2m0s"
Sep 09 03:13:05.064127 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:13:05.057251    2576 flags.go:64] FLAG: --seccomp-default="false"
Sep 09 03:13:05.064127 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:13:05.057254    2576 flags.go:64] FLAG: --serialize-image-pulls="true"
Sep 09 03:13:05.064127 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:13:05.057257    2576 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s"
Sep 09 03:13:05.064127 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:13:05.057260    2576 flags.go:64] FLAG: --storage-driver-db="cadvisor"
Sep 09 03:13:05.064127 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:13:05.057263    2576 flags.go:64] FLAG: --storage-driver-host="localhost:8086"
Sep 09 03:13:05.064127 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:13:05.057266    2576 flags.go:64] FLAG: --storage-driver-password="root"
Sep 09 03:13:05.064127 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:13:05.057269    2576 flags.go:64] FLAG: --storage-driver-secure="false"
Sep 09 03:13:05.064127 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:13:05.057272    2576 flags.go:64] FLAG: --storage-driver-table="stats"
Sep 09 03:13:05.064127 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:13:05.057274    2576 flags.go:64] FLAG: --storage-driver-user="root"
Sep 09 03:13:05.064127 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:13:05.057277    2576 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s"
Sep 09 03:13:05.064127 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:13:05.057280    2576 flags.go:64] FLAG: --sync-frequency="1m0s"
Sep 09 03:13:05.064127 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:13:05.057283    2576 flags.go:64] FLAG: --system-cgroups=""
Sep 09 03:13:05.064127 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:13:05.057285    2576 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi"
Sep 09 03:13:05.064127 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:13:05.057290    2576 flags.go:64] FLAG: --system-reserved-cgroup=""
Sep 09 03:13:05.064127 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:13:05.057293    2576 flags.go:64] FLAG: --tls-cert-file=""
Sep 09 03:13:05.064127 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:13:05.057297    2576 flags.go:64] FLAG: --tls-cipher-suites="[]"
Sep 09 03:13:05.064127 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:13:05.057301    2576 flags.go:64] FLAG: --tls-min-version=""
Sep 09 03:13:05.064127 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:13:05.057304    2576 flags.go:64] FLAG: --tls-private-key-file=""
Sep 09 03:13:05.064127 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:13:05.057307    2576 flags.go:64] FLAG: --topology-manager-policy="none"
Sep 09 03:13:05.064127 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:13:05.057309    2576 flags.go:64] FLAG: --topology-manager-policy-options=""
Sep 09 03:13:05.064768 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:13:05.057312    2576 flags.go:64] FLAG: --topology-manager-scope="container"
Sep 09 03:13:05.064768 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:13:05.057315    2576 flags.go:64] FLAG: --v="2"
Sep 09 03:13:05.064768 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:13:05.057319    2576 flags.go:64] FLAG: --version="false"
Sep 09 03:13:05.064768 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:13:05.057322    2576 flags.go:64] FLAG: --vmodule=""
Sep 09 03:13:05.064768 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:13:05.057326    2576 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec"
Sep 09 03:13:05.064768 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:13:05.057330    2576 flags.go:64] FLAG: --volume-stats-agg-period="1m0s"
Sep 09 03:13:05.064768 ip-10-0-132-14 kubenswrapper[2576]: W0909 03:13:05.057444    2576 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 09 03:13:05.064768 ip-10-0-132-14 kubenswrapper[2576]: W0909 03:13:05.057447    2576 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 09 03:13:05.064768 ip-10-0-132-14 kubenswrapper[2576]: W0909 03:13:05.057450    2576 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 09 03:13:05.064768 ip-10-0-132-14 kubenswrapper[2576]: W0909 03:13:05.057453    2576 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 09 03:13:05.064768 ip-10-0-132-14 kubenswrapper[2576]: W0909 03:13:05.057456    2576 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 09 03:13:05.064768 ip-10-0-132-14 kubenswrapper[2576]: W0909 03:13:05.057459    2576 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 09 03:13:05.064768 ip-10-0-132-14 kubenswrapper[2576]: W0909 03:13:05.057462    2576 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 09 03:13:05.064768 ip-10-0-132-14 kubenswrapper[2576]: W0909 03:13:05.057465    2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 09 03:13:05.064768 ip-10-0-132-14 kubenswrapper[2576]: W0909 03:13:05.057468    2576 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 09 03:13:05.064768 ip-10-0-132-14 kubenswrapper[2576]: W0909 03:13:05.057471    2576 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 09 03:13:05.064768 ip-10-0-132-14 kubenswrapper[2576]: W0909 03:13:05.057474    2576 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 09 03:13:05.064768 ip-10-0-132-14 kubenswrapper[2576]: W0909 03:13:05.057476    2576 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 09 03:13:05.064768 ip-10-0-132-14 kubenswrapper[2576]: W0909 03:13:05.057479    2576 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 09 03:13:05.064768 ip-10-0-132-14 kubenswrapper[2576]: W0909 03:13:05.057482    2576 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 09 03:13:05.064768 ip-10-0-132-14 kubenswrapper[2576]: W0909 03:13:05.057485    2576 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 09 03:13:05.064768 ip-10-0-132-14 kubenswrapper[2576]: W0909 03:13:05.057487    2576 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 09 03:13:05.065327 ip-10-0-132-14 kubenswrapper[2576]: W0909 03:13:05.057490    2576 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 09 03:13:05.065327 ip-10-0-132-14 kubenswrapper[2576]: W0909 03:13:05.057493    2576 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 09 03:13:05.065327 ip-10-0-132-14 kubenswrapper[2576]: W0909 03:13:05.057495    2576 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 09 03:13:05.065327 ip-10-0-132-14 kubenswrapper[2576]: W0909 03:13:05.057498    2576 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 09 03:13:05.065327 ip-10-0-132-14 kubenswrapper[2576]: W0909 03:13:05.057500    2576 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 09 03:13:05.065327 ip-10-0-132-14 kubenswrapper[2576]: W0909 03:13:05.057506    2576 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 09 03:13:05.065327 ip-10-0-132-14 kubenswrapper[2576]: W0909 03:13:05.057509    2576 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 09 03:13:05.065327 ip-10-0-132-14 kubenswrapper[2576]: W0909 03:13:05.057511    2576 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 09 03:13:05.065327 ip-10-0-132-14 kubenswrapper[2576]: W0909 03:13:05.057514    2576 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 09 03:13:05.065327 ip-10-0-132-14 kubenswrapper[2576]: W0909 03:13:05.057516    2576 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 09 03:13:05.065327 ip-10-0-132-14 kubenswrapper[2576]: W0909 03:13:05.057518    2576 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 09 03:13:05.065327 ip-10-0-132-14 kubenswrapper[2576]: W0909 03:13:05.057521    2576 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 09 03:13:05.065327 ip-10-0-132-14 kubenswrapper[2576]: W0909 03:13:05.057524    2576 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 09 03:13:05.065327 ip-10-0-132-14 kubenswrapper[2576]: W0909 03:13:05.057526    2576 feature_gate.go:328] unrecognized feature gate: Example
Sep 09 03:13:05.065327 ip-10-0-132-14 kubenswrapper[2576]: W0909 03:13:05.057528    2576 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 09 03:13:05.065327 ip-10-0-132-14 kubenswrapper[2576]: W0909 03:13:05.057532    2576 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 09 03:13:05.065327 ip-10-0-132-14 kubenswrapper[2576]: W0909 03:13:05.057535    2576 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 09 03:13:05.065327 ip-10-0-132-14 kubenswrapper[2576]: W0909 03:13:05.057539    2576 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 09 03:13:05.065327 ip-10-0-132-14 kubenswrapper[2576]: W0909 03:13:05.057542    2576 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 09 03:13:05.065327 ip-10-0-132-14 kubenswrapper[2576]: W0909 03:13:05.057545    2576 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 09 03:13:05.065851 ip-10-0-132-14 kubenswrapper[2576]: W0909 03:13:05.057548    2576 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 09 03:13:05.065851 ip-10-0-132-14 kubenswrapper[2576]: W0909 03:13:05.057551    2576 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 09 03:13:05.065851 ip-10-0-132-14 kubenswrapper[2576]: W0909 03:13:05.057554    2576 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 09 03:13:05.065851 ip-10-0-132-14 kubenswrapper[2576]: W0909 03:13:05.057557    2576 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 09 03:13:05.065851 ip-10-0-132-14 kubenswrapper[2576]: W0909 03:13:05.057559    2576 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 09 03:13:05.065851 ip-10-0-132-14 kubenswrapper[2576]: W0909 03:13:05.057562    2576 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 09 03:13:05.065851 ip-10-0-132-14 kubenswrapper[2576]: W0909 03:13:05.057564    2576 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 09 03:13:05.065851 ip-10-0-132-14 kubenswrapper[2576]: W0909 03:13:05.057567    2576 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 09 03:13:05.065851 ip-10-0-132-14 kubenswrapper[2576]: W0909 03:13:05.057569    2576 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 09 03:13:05.065851 ip-10-0-132-14 kubenswrapper[2576]: W0909 03:13:05.057572    2576 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 09 03:13:05.065851 ip-10-0-132-14 kubenswrapper[2576]: W0909 03:13:05.057574    2576 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 09 03:13:05.065851 ip-10-0-132-14 kubenswrapper[2576]: W0909 03:13:05.057577    2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 09 03:13:05.065851 ip-10-0-132-14 kubenswrapper[2576]: W0909 03:13:05.057579    2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 09 03:13:05.065851 ip-10-0-132-14 kubenswrapper[2576]: W0909 03:13:05.057581    2576 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 09 03:13:05.065851 ip-10-0-132-14 kubenswrapper[2576]: W0909 03:13:05.057584    2576 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 09 03:13:05.065851 ip-10-0-132-14 kubenswrapper[2576]: W0909 03:13:05.057586    2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 09 03:13:05.065851 ip-10-0-132-14 kubenswrapper[2576]: W0909 03:13:05.057588    2576 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 09 03:13:05.065851 ip-10-0-132-14 kubenswrapper[2576]: W0909 03:13:05.057592    2576 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 09 03:13:05.065851 ip-10-0-132-14 kubenswrapper[2576]: W0909 03:13:05.057595    2576 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 09 03:13:05.065851 ip-10-0-132-14 kubenswrapper[2576]: W0909 03:13:05.057597    2576 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 09 03:13:05.066621 ip-10-0-132-14 kubenswrapper[2576]: W0909 03:13:05.057600    2576 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 09 03:13:05.066621 ip-10-0-132-14 kubenswrapper[2576]: W0909 03:13:05.057602    2576 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 09 03:13:05.066621 ip-10-0-132-14 kubenswrapper[2576]: W0909 03:13:05.057605    2576 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 09 03:13:05.066621 ip-10-0-132-14 kubenswrapper[2576]: W0909 03:13:05.057607    2576 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 09 03:13:05.066621 ip-10-0-132-14 kubenswrapper[2576]: W0909 03:13:05.057610    2576 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 09 03:13:05.066621 ip-10-0-132-14 kubenswrapper[2576]: W0909 03:13:05.057612    2576 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 09 03:13:05.066621 ip-10-0-132-14 kubenswrapper[2576]: W0909 03:13:05.057615    2576 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 09 03:13:05.066621 ip-10-0-132-14 kubenswrapper[2576]: W0909 03:13:05.057618    2576 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 09 03:13:05.066621 ip-10-0-132-14 kubenswrapper[2576]: W0909 03:13:05.057621    2576 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 09 03:13:05.066621 ip-10-0-132-14 kubenswrapper[2576]: W0909 03:13:05.057639    2576 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 09 03:13:05.066621 ip-10-0-132-14 kubenswrapper[2576]: W0909 03:13:05.057642    2576 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 09 03:13:05.066621 ip-10-0-132-14 kubenswrapper[2576]: W0909 03:13:05.057644    2576 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 09 03:13:05.066621 ip-10-0-132-14 kubenswrapper[2576]: W0909 03:13:05.057647    2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 09 03:13:05.066621 ip-10-0-132-14 kubenswrapper[2576]: W0909 03:13:05.057649    2576 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 09 03:13:05.066621 ip-10-0-132-14 kubenswrapper[2576]: W0909 03:13:05.057652    2576 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 09 03:13:05.066621 ip-10-0-132-14 kubenswrapper[2576]: W0909 03:13:05.057654    2576 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 09 03:13:05.066621 ip-10-0-132-14 kubenswrapper[2576]: W0909 03:13:05.057657    2576 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 09 03:13:05.066621 ip-10-0-132-14 kubenswrapper[2576]: W0909 03:13:05.057660    2576 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 09 03:13:05.066621 ip-10-0-132-14 kubenswrapper[2576]: W0909 03:13:05.057669    2576 feature_gate.go:328] unrecognized feature gate: Example2
Sep 09 03:13:05.067304 ip-10-0-132-14 kubenswrapper[2576]: W0909 03:13:05.057672    2576 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 09 03:13:05.067304 ip-10-0-132-14 kubenswrapper[2576]: W0909 03:13:05.057675    2576 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 09 03:13:05.067304 ip-10-0-132-14 kubenswrapper[2576]: W0909 03:13:05.057677    2576 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 09 03:13:05.067304 ip-10-0-132-14 kubenswrapper[2576]: W0909 03:13:05.057680    2576 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 09 03:13:05.067304 ip-10-0-132-14 kubenswrapper[2576]: W0909 03:13:05.057683    2576 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 09 03:13:05.067304 ip-10-0-132-14 kubenswrapper[2576]: W0909 03:13:05.057685    2576 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 09 03:13:05.067304 ip-10-0-132-14 kubenswrapper[2576]: W0909 03:13:05.057687    2576 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 09 03:13:05.067304 ip-10-0-132-14 kubenswrapper[2576]: W0909 03:13:05.057690    2576 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 09 03:13:05.067304 ip-10-0-132-14 kubenswrapper[2576]: W0909 03:13:05.057693    2576 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 09 03:13:05.067304 ip-10-0-132-14 kubenswrapper[2576]: W0909 03:13:05.057695    2576 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 09 03:13:05.067304 ip-10-0-132-14 kubenswrapper[2576]: W0909 03:13:05.057703    2576 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 09 03:13:05.067304 ip-10-0-132-14 kubenswrapper[2576]: W0909 03:13:05.057707    2576 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 09 03:13:05.067304 ip-10-0-132-14 kubenswrapper[2576]: W0909 03:13:05.057711    2576 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 09 03:13:05.067304 ip-10-0-132-14 kubenswrapper[2576]: W0909 03:13:05.057714    2576 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 09 03:13:05.067304 ip-10-0-132-14 kubenswrapper[2576]: W0909 03:13:05.057716    2576 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 09 03:13:05.067304 ip-10-0-132-14 kubenswrapper[2576]: W0909 03:13:05.057720    2576 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 09 03:13:05.067304 ip-10-0-132-14 kubenswrapper[2576]: W0909 03:13:05.057724    2576 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 09 03:13:05.067304 ip-10-0-132-14 kubenswrapper[2576]: W0909 03:13:05.057726    2576 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 09 03:13:05.067304 ip-10-0-132-14 kubenswrapper[2576]: W0909 03:13:05.057729    2576 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 09 03:13:05.067793 ip-10-0-132-14 kubenswrapper[2576]: W0909 03:13:05.057731    2576 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 09 03:13:05.067793 ip-10-0-132-14 kubenswrapper[2576]: W0909 03:13:05.057736    2576 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 09 03:13:05.067793 ip-10-0-132-14 kubenswrapper[2576]: W0909 03:13:05.057738    2576 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 09 03:13:05.067793 ip-10-0-132-14 kubenswrapper[2576]: W0909 03:13:05.057741    2576 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 09 03:13:05.067793 ip-10-0-132-14 kubenswrapper[2576]: W0909 03:13:05.057743    2576 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 09 03:13:05.067793 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:13:05.058518    2576 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Sep 09 03:13:05.067793 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:13:05.067521    2576 server.go:529] "Kubelet version" kubeletVersion="v1.34.9"
Sep 09 03:13:05.067793 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:13:05.067669    2576 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK=""
Sep 09 03:13:05.067793 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:13:05.067691    2576 watchdog_linux.go:95] "Systemd watchdog is not enabled"
Sep 09 03:13:05.067793 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:13:05.067697    2576 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started."
Sep 09 03:13:05.067793 ip-10-0-132-14 kubenswrapper[2576]: W0909 03:13:05.067754    2576 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 09 03:13:05.067793 ip-10-0-132-14 kubenswrapper[2576]: W0909 03:13:05.067759    2576 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 09 03:13:05.067793 ip-10-0-132-14 kubenswrapper[2576]: W0909 03:13:05.067762    2576 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 09 03:13:05.067793 ip-10-0-132-14 kubenswrapper[2576]: W0909 03:13:05.067767    2576 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 09 03:13:05.068137 ip-10-0-132-14 kubenswrapper[2576]: W0909 03:13:05.067771    2576 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 09 03:13:05.068137 ip-10-0-132-14 kubenswrapper[2576]: W0909 03:13:05.067775    2576 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 09 03:13:05.068137 ip-10-0-132-14 kubenswrapper[2576]: W0909 03:13:05.067778    2576 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 09 03:13:05.068137 ip-10-0-132-14 kubenswrapper[2576]: W0909 03:13:05.067781    2576 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 09 03:13:05.068137 ip-10-0-132-14 kubenswrapper[2576]: W0909 03:13:05.067784    2576 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 09 03:13:05.068137 ip-10-0-132-14 kubenswrapper[2576]: W0909 03:13:05.067787    2576 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 09 03:13:05.068137 ip-10-0-132-14 kubenswrapper[2576]: W0909 03:13:05.067790    2576 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 09 03:13:05.068137 ip-10-0-132-14 kubenswrapper[2576]: W0909 03:13:05.067792    2576 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 09 03:13:05.068137 ip-10-0-132-14 kubenswrapper[2576]: W0909 03:13:05.067795    2576 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 09 03:13:05.068137 ip-10-0-132-14 kubenswrapper[2576]: W0909 03:13:05.067798    2576 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 09 03:13:05.068137 ip-10-0-132-14 kubenswrapper[2576]: W0909 03:13:05.067802    2576 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 09 03:13:05.068137 ip-10-0-132-14 kubenswrapper[2576]: W0909 03:13:05.067806    2576 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 09 03:13:05.068137 ip-10-0-132-14 kubenswrapper[2576]: W0909 03:13:05.067809    2576 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 09 03:13:05.068137 ip-10-0-132-14 kubenswrapper[2576]: W0909 03:13:05.067812    2576 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 09 03:13:05.068137 ip-10-0-132-14 kubenswrapper[2576]: W0909 03:13:05.067815    2576 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 09 03:13:05.068137 ip-10-0-132-14 kubenswrapper[2576]: W0909 03:13:05.067817    2576 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 09 03:13:05.068137 ip-10-0-132-14 kubenswrapper[2576]: W0909 03:13:05.067820    2576 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 09 03:13:05.068137 ip-10-0-132-14 kubenswrapper[2576]: W0909 03:13:05.067823    2576 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 09 03:13:05.068137 ip-10-0-132-14 kubenswrapper[2576]: W0909 03:13:05.067825    2576 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 09 03:13:05.068601 ip-10-0-132-14 kubenswrapper[2576]: W0909 03:13:05.067828    2576 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 09 03:13:05.068601 ip-10-0-132-14 kubenswrapper[2576]: W0909 03:13:05.067830    2576 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 09 03:13:05.068601 ip-10-0-132-14 kubenswrapper[2576]: W0909 03:13:05.067833    2576 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 09 03:13:05.068601 ip-10-0-132-14 kubenswrapper[2576]: W0909 03:13:05.067835    2576 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 09 03:13:05.068601 ip-10-0-132-14 kubenswrapper[2576]: W0909 03:13:05.067838    2576 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 09 03:13:05.068601 ip-10-0-132-14 kubenswrapper[2576]: W0909 03:13:05.067841    2576 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 09 03:13:05.068601 ip-10-0-132-14 kubenswrapper[2576]: W0909 03:13:05.067844    2576 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 09 03:13:05.068601 ip-10-0-132-14 kubenswrapper[2576]: W0909 03:13:05.067846    2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 09 03:13:05.068601 ip-10-0-132-14 kubenswrapper[2576]: W0909 03:13:05.067849    2576 feature_gate.go:328] unrecognized feature gate: Example2
Sep 09 03:13:05.068601 ip-10-0-132-14 kubenswrapper[2576]: W0909 03:13:05.067851    2576 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 09 03:13:05.068601 ip-10-0-132-14 kubenswrapper[2576]: W0909 03:13:05.067854    2576 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 09 03:13:05.068601 ip-10-0-132-14 kubenswrapper[2576]: W0909 03:13:05.067856    2576 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 09 03:13:05.068601 ip-10-0-132-14 kubenswrapper[2576]: W0909 03:13:05.067858    2576 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 09 03:13:05.068601 ip-10-0-132-14 kubenswrapper[2576]: W0909 03:13:05.067861    2576 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 09 03:13:05.068601 ip-10-0-132-14 kubenswrapper[2576]: W0909 03:13:05.067864    2576 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 09 03:13:05.068601 ip-10-0-132-14 kubenswrapper[2576]: W0909 03:13:05.067867    2576 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 09 03:13:05.068601 ip-10-0-132-14 kubenswrapper[2576]: W0909 03:13:05.067869    2576 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 09 03:13:05.068601 ip-10-0-132-14 kubenswrapper[2576]: W0909 03:13:05.067871    2576 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 09 03:13:05.068601 ip-10-0-132-14 kubenswrapper[2576]: W0909 03:13:05.067874    2576 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 09 03:13:05.068601 ip-10-0-132-14 kubenswrapper[2576]: W0909 03:13:05.067876    2576 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 09 03:13:05.069142 ip-10-0-132-14 kubenswrapper[2576]: W0909 03:13:05.067879    2576 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 09 03:13:05.069142 ip-10-0-132-14 kubenswrapper[2576]: W0909 03:13:05.067882    2576 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 09 03:13:05.069142 ip-10-0-132-14 kubenswrapper[2576]: W0909 03:13:05.067884    2576 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 09 03:13:05.069142 ip-10-0-132-14 kubenswrapper[2576]: W0909 03:13:05.067887    2576 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 09 03:13:05.069142 ip-10-0-132-14 kubenswrapper[2576]: W0909 03:13:05.067889    2576 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 09 03:13:05.069142 ip-10-0-132-14 kubenswrapper[2576]: W0909 03:13:05.067892    2576 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 09 03:13:05.069142 ip-10-0-132-14 kubenswrapper[2576]: W0909 03:13:05.067895    2576 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 09 03:13:05.069142 ip-10-0-132-14 kubenswrapper[2576]: W0909 03:13:05.067897    2576 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 09 03:13:05.069142 ip-10-0-132-14 kubenswrapper[2576]: W0909 03:13:05.067900    2576 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 09 03:13:05.069142 ip-10-0-132-14 kubenswrapper[2576]: W0909 03:13:05.067902    2576 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 09 03:13:05.069142 ip-10-0-132-14 kubenswrapper[2576]: W0909 03:13:05.067904    2576 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 09 03:13:05.069142 ip-10-0-132-14 kubenswrapper[2576]: W0909 03:13:05.067907    2576 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 09 03:13:05.069142 ip-10-0-132-14 kubenswrapper[2576]: W0909 03:13:05.067910    2576 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 09 03:13:05.069142 ip-10-0-132-14 kubenswrapper[2576]: W0909 03:13:05.067912    2576 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 09 03:13:05.069142 ip-10-0-132-14 kubenswrapper[2576]: W0909 03:13:05.067914    2576 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 09 03:13:05.069142 ip-10-0-132-14 kubenswrapper[2576]: W0909 03:13:05.067917    2576 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 09 03:13:05.069142 ip-10-0-132-14 kubenswrapper[2576]: W0909 03:13:05.067920    2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 09 03:13:05.069142 ip-10-0-132-14 kubenswrapper[2576]: W0909 03:13:05.067922    2576 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 09 03:13:05.069142 ip-10-0-132-14 kubenswrapper[2576]: W0909 03:13:05.067925    2576 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 09 03:13:05.069585 ip-10-0-132-14 kubenswrapper[2576]: W0909 03:13:05.067927    2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 09 03:13:05.069585 ip-10-0-132-14 kubenswrapper[2576]: W0909 03:13:05.067930    2576 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 09 03:13:05.069585 ip-10-0-132-14 kubenswrapper[2576]: W0909 03:13:05.067932    2576 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 09 03:13:05.069585 ip-10-0-132-14 kubenswrapper[2576]: W0909 03:13:05.067935    2576 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 09 03:13:05.069585 ip-10-0-132-14 kubenswrapper[2576]: W0909 03:13:05.067937    2576 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 09 03:13:05.069585 ip-10-0-132-14 kubenswrapper[2576]: W0909 03:13:05.067940    2576 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 09 03:13:05.069585 ip-10-0-132-14 kubenswrapper[2576]: W0909 03:13:05.067942    2576 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 09 03:13:05.069585 ip-10-0-132-14 kubenswrapper[2576]: W0909 03:13:05.067945    2576 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 09 03:13:05.069585 ip-10-0-132-14 kubenswrapper[2576]: W0909 03:13:05.067948    2576 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 09 03:13:05.069585 ip-10-0-132-14 kubenswrapper[2576]: W0909 03:13:05.067950    2576 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 09 03:13:05.069585 ip-10-0-132-14 kubenswrapper[2576]: W0909 03:13:05.067953    2576 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 09 03:13:05.069585 ip-10-0-132-14 kubenswrapper[2576]: W0909 03:13:05.067955    2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 09 03:13:05.069585 ip-10-0-132-14 kubenswrapper[2576]: W0909 03:13:05.067957    2576 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 09 03:13:05.069585 ip-10-0-132-14 kubenswrapper[2576]: W0909 03:13:05.067960    2576 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 09 03:13:05.069585 ip-10-0-132-14 kubenswrapper[2576]: W0909 03:13:05.067963    2576 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 09 03:13:05.069585 ip-10-0-132-14 kubenswrapper[2576]: W0909 03:13:05.067965    2576 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 09 03:13:05.069585 ip-10-0-132-14 kubenswrapper[2576]: W0909 03:13:05.067967    2576 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 09 03:13:05.069585 ip-10-0-132-14 kubenswrapper[2576]: W0909 03:13:05.067971    2576 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 09 03:13:05.069585 ip-10-0-132-14 kubenswrapper[2576]: W0909 03:13:05.067974    2576 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 09 03:13:05.069585 ip-10-0-132-14 kubenswrapper[2576]: W0909 03:13:05.067976    2576 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 09 03:13:05.070076 ip-10-0-132-14 kubenswrapper[2576]: W0909 03:13:05.067979    2576 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 09 03:13:05.070076 ip-10-0-132-14 kubenswrapper[2576]: W0909 03:13:05.067981    2576 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 09 03:13:05.070076 ip-10-0-132-14 kubenswrapper[2576]: W0909 03:13:05.067985    2576 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 09 03:13:05.070076 ip-10-0-132-14 kubenswrapper[2576]: W0909 03:13:05.067988    2576 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 09 03:13:05.070076 ip-10-0-132-14 kubenswrapper[2576]: W0909 03:13:05.067991    2576 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 09 03:13:05.070076 ip-10-0-132-14 kubenswrapper[2576]: W0909 03:13:05.067994    2576 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 09 03:13:05.070076 ip-10-0-132-14 kubenswrapper[2576]: W0909 03:13:05.067997    2576 feature_gate.go:328] unrecognized feature gate: Example
Sep 09 03:13:05.070076 ip-10-0-132-14 kubenswrapper[2576]: W0909 03:13:05.067999    2576 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 09 03:13:05.070076 ip-10-0-132-14 kubenswrapper[2576]: W0909 03:13:05.068002    2576 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 09 03:13:05.070076 ip-10-0-132-14 kubenswrapper[2576]: W0909 03:13:05.068004    2576 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 09 03:13:05.070076 ip-10-0-132-14 kubenswrapper[2576]: W0909 03:13:05.068007    2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 09 03:13:05.070076 ip-10-0-132-14 kubenswrapper[2576]: W0909 03:13:05.068009    2576 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 09 03:13:05.070076 ip-10-0-132-14 kubenswrapper[2576]: W0909 03:13:05.068011    2576 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 09 03:13:05.070076 ip-10-0-132-14 kubenswrapper[2576]: W0909 03:13:05.068014    2576 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 09 03:13:05.070076 ip-10-0-132-14 kubenswrapper[2576]: W0909 03:13:05.068016    2576 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 09 03:13:05.070076 ip-10-0-132-14 kubenswrapper[2576]: W0909 03:13:05.068019    2576 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 09 03:13:05.070076 ip-10-0-132-14 kubenswrapper[2576]: W0909 03:13:05.068021    2576 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 09 03:13:05.070491 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:13:05.068026    2576 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Sep 09 03:13:05.070491 ip-10-0-132-14 kubenswrapper[2576]: W0909 03:13:05.068144    2576 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 09 03:13:05.070491 ip-10-0-132-14 kubenswrapper[2576]: W0909 03:13:05.068150    2576 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 09 03:13:05.070491 ip-10-0-132-14 kubenswrapper[2576]: W0909 03:13:05.068154    2576 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 09 03:13:05.070491 ip-10-0-132-14 kubenswrapper[2576]: W0909 03:13:05.068157    2576 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 09 03:13:05.070491 ip-10-0-132-14 kubenswrapper[2576]: W0909 03:13:05.068160    2576 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 09 03:13:05.070491 ip-10-0-132-14 kubenswrapper[2576]: W0909 03:13:05.068162    2576 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 09 03:13:05.070491 ip-10-0-132-14 kubenswrapper[2576]: W0909 03:13:05.068165    2576 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 09 03:13:05.070491 ip-10-0-132-14 kubenswrapper[2576]: W0909 03:13:05.068167    2576 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 09 03:13:05.070491 ip-10-0-132-14 kubenswrapper[2576]: W0909 03:13:05.068170    2576 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 09 03:13:05.070491 ip-10-0-132-14 kubenswrapper[2576]: W0909 03:13:05.068172    2576 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 09 03:13:05.070491 ip-10-0-132-14 kubenswrapper[2576]: W0909 03:13:05.068175    2576 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 09 03:13:05.070491 ip-10-0-132-14 kubenswrapper[2576]: W0909 03:13:05.068178    2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 09 03:13:05.070491 ip-10-0-132-14 kubenswrapper[2576]: W0909 03:13:05.068181    2576 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 09 03:13:05.070491 ip-10-0-132-14 kubenswrapper[2576]: W0909 03:13:05.068183    2576 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 09 03:13:05.070876 ip-10-0-132-14 kubenswrapper[2576]: W0909 03:13:05.068186    2576 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 09 03:13:05.070876 ip-10-0-132-14 kubenswrapper[2576]: W0909 03:13:05.068189    2576 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 09 03:13:05.070876 ip-10-0-132-14 kubenswrapper[2576]: W0909 03:13:05.068191    2576 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 09 03:13:05.070876 ip-10-0-132-14 kubenswrapper[2576]: W0909 03:13:05.068193    2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 09 03:13:05.070876 ip-10-0-132-14 kubenswrapper[2576]: W0909 03:13:05.068196    2576 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 09 03:13:05.070876 ip-10-0-132-14 kubenswrapper[2576]: W0909 03:13:05.068198    2576 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 09 03:13:05.070876 ip-10-0-132-14 kubenswrapper[2576]: W0909 03:13:05.068201    2576 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 09 03:13:05.070876 ip-10-0-132-14 kubenswrapper[2576]: W0909 03:13:05.068203    2576 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 09 03:13:05.070876 ip-10-0-132-14 kubenswrapper[2576]: W0909 03:13:05.068205    2576 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 09 03:13:05.070876 ip-10-0-132-14 kubenswrapper[2576]: W0909 03:13:05.068208    2576 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 09 03:13:05.070876 ip-10-0-132-14 kubenswrapper[2576]: W0909 03:13:05.068210    2576 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 09 03:13:05.070876 ip-10-0-132-14 kubenswrapper[2576]: W0909 03:13:05.068213    2576 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 09 03:13:05.070876 ip-10-0-132-14 kubenswrapper[2576]: W0909 03:13:05.068216    2576 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 09 03:13:05.070876 ip-10-0-132-14 kubenswrapper[2576]: W0909 03:13:05.068218    2576 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 09 03:13:05.070876 ip-10-0-132-14 kubenswrapper[2576]: W0909 03:13:05.068220    2576 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 09 03:13:05.070876 ip-10-0-132-14 kubenswrapper[2576]: W0909 03:13:05.068223    2576 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 09 03:13:05.070876 ip-10-0-132-14 kubenswrapper[2576]: W0909 03:13:05.068225    2576 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 09 03:13:05.070876 ip-10-0-132-14 kubenswrapper[2576]: W0909 03:13:05.068228    2576 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 09 03:13:05.070876 ip-10-0-132-14 kubenswrapper[2576]: W0909 03:13:05.068230    2576 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 09 03:13:05.070876 ip-10-0-132-14 kubenswrapper[2576]: W0909 03:13:05.068233    2576 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 09 03:13:05.071422 ip-10-0-132-14 kubenswrapper[2576]: W0909 03:13:05.068236    2576 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 09 03:13:05.071422 ip-10-0-132-14 kubenswrapper[2576]: W0909 03:13:05.068239    2576 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 09 03:13:05.071422 ip-10-0-132-14 kubenswrapper[2576]: W0909 03:13:05.068241    2576 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 09 03:13:05.071422 ip-10-0-132-14 kubenswrapper[2576]: W0909 03:13:05.068244    2576 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 09 03:13:05.071422 ip-10-0-132-14 kubenswrapper[2576]: W0909 03:13:05.068247    2576 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 09 03:13:05.071422 ip-10-0-132-14 kubenswrapper[2576]: W0909 03:13:05.068252    2576 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 09 03:13:05.071422 ip-10-0-132-14 kubenswrapper[2576]: W0909 03:13:05.068254    2576 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 09 03:13:05.071422 ip-10-0-132-14 kubenswrapper[2576]: W0909 03:13:05.068257    2576 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 09 03:13:05.071422 ip-10-0-132-14 kubenswrapper[2576]: W0909 03:13:05.068260    2576 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 09 03:13:05.071422 ip-10-0-132-14 kubenswrapper[2576]: W0909 03:13:05.068262    2576 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 09 03:13:05.071422 ip-10-0-132-14 kubenswrapper[2576]: W0909 03:13:05.068265    2576 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 09 03:13:05.071422 ip-10-0-132-14 kubenswrapper[2576]: W0909 03:13:05.068268    2576 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 09 03:13:05.071422 ip-10-0-132-14 kubenswrapper[2576]: W0909 03:13:05.068270    2576 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 09 03:13:05.071422 ip-10-0-132-14 kubenswrapper[2576]: W0909 03:13:05.068274    2576 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 09 03:13:05.071422 ip-10-0-132-14 kubenswrapper[2576]: W0909 03:13:05.068277    2576 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 09 03:13:05.071422 ip-10-0-132-14 kubenswrapper[2576]: W0909 03:13:05.068280    2576 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 09 03:13:05.071422 ip-10-0-132-14 kubenswrapper[2576]: W0909 03:13:05.068282    2576 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 09 03:13:05.071422 ip-10-0-132-14 kubenswrapper[2576]: W0909 03:13:05.068285    2576 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 09 03:13:05.071422 ip-10-0-132-14 kubenswrapper[2576]: W0909 03:13:05.068288    2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 09 03:13:05.071915 ip-10-0-132-14 kubenswrapper[2576]: W0909 03:13:05.068290    2576 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 09 03:13:05.071915 ip-10-0-132-14 kubenswrapper[2576]: W0909 03:13:05.068293    2576 feature_gate.go:328] unrecognized feature gate: Example2
Sep 09 03:13:05.071915 ip-10-0-132-14 kubenswrapper[2576]: W0909 03:13:05.068295    2576 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 09 03:13:05.071915 ip-10-0-132-14 kubenswrapper[2576]: W0909 03:13:05.068297    2576 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 09 03:13:05.071915 ip-10-0-132-14 kubenswrapper[2576]: W0909 03:13:05.068300    2576 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 09 03:13:05.071915 ip-10-0-132-14 kubenswrapper[2576]: W0909 03:13:05.068302    2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 09 03:13:05.071915 ip-10-0-132-14 kubenswrapper[2576]: W0909 03:13:05.068305    2576 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 09 03:13:05.071915 ip-10-0-132-14 kubenswrapper[2576]: W0909 03:13:05.068308    2576 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 09 03:13:05.071915 ip-10-0-132-14 kubenswrapper[2576]: W0909 03:13:05.068310    2576 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 09 03:13:05.071915 ip-10-0-132-14 kubenswrapper[2576]: W0909 03:13:05.068313    2576 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 09 03:13:05.071915 ip-10-0-132-14 kubenswrapper[2576]: W0909 03:13:05.068315    2576 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 09 03:13:05.071915 ip-10-0-132-14 kubenswrapper[2576]: W0909 03:13:05.068317    2576 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 09 03:13:05.071915 ip-10-0-132-14 kubenswrapper[2576]: W0909 03:13:05.068320    2576 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 09 03:13:05.071915 ip-10-0-132-14 kubenswrapper[2576]: W0909 03:13:05.068323    2576 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 09 03:13:05.071915 ip-10-0-132-14 kubenswrapper[2576]: W0909 03:13:05.068326    2576 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 09 03:13:05.071915 ip-10-0-132-14 kubenswrapper[2576]: W0909 03:13:05.068328    2576 feature_gate.go:328] unrecognized feature gate: Example
Sep 09 03:13:05.071915 ip-10-0-132-14 kubenswrapper[2576]: W0909 03:13:05.068331    2576 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 09 03:13:05.071915 ip-10-0-132-14 kubenswrapper[2576]: W0909 03:13:05.068333    2576 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 09 03:13:05.071915 ip-10-0-132-14 kubenswrapper[2576]: W0909 03:13:05.068335    2576 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 09 03:13:05.071915 ip-10-0-132-14 kubenswrapper[2576]: W0909 03:13:05.068338    2576 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 09 03:13:05.071915 ip-10-0-132-14 kubenswrapper[2576]: W0909 03:13:05.068340    2576 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 09 03:13:05.072436 ip-10-0-132-14 kubenswrapper[2576]: W0909 03:13:05.068343    2576 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 09 03:13:05.072436 ip-10-0-132-14 kubenswrapper[2576]: W0909 03:13:05.068345    2576 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 09 03:13:05.072436 ip-10-0-132-14 kubenswrapper[2576]: W0909 03:13:05.068348    2576 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 09 03:13:05.072436 ip-10-0-132-14 kubenswrapper[2576]: W0909 03:13:05.068350    2576 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 09 03:13:05.072436 ip-10-0-132-14 kubenswrapper[2576]: W0909 03:13:05.068353    2576 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 09 03:13:05.072436 ip-10-0-132-14 kubenswrapper[2576]: W0909 03:13:05.068355    2576 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 09 03:13:05.072436 ip-10-0-132-14 kubenswrapper[2576]: W0909 03:13:05.068358    2576 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 09 03:13:05.072436 ip-10-0-132-14 kubenswrapper[2576]: W0909 03:13:05.068360    2576 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 09 03:13:05.072436 ip-10-0-132-14 kubenswrapper[2576]: W0909 03:13:05.068362    2576 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 09 03:13:05.072436 ip-10-0-132-14 kubenswrapper[2576]: W0909 03:13:05.068365    2576 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 09 03:13:05.072436 ip-10-0-132-14 kubenswrapper[2576]: W0909 03:13:05.068367    2576 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 09 03:13:05.072436 ip-10-0-132-14 kubenswrapper[2576]: W0909 03:13:05.068369    2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 09 03:13:05.072436 ip-10-0-132-14 kubenswrapper[2576]: W0909 03:13:05.068372    2576 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 09 03:13:05.072436 ip-10-0-132-14 kubenswrapper[2576]: W0909 03:13:05.068374    2576 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 09 03:13:05.072436 ip-10-0-132-14 kubenswrapper[2576]: W0909 03:13:05.068377    2576 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 09 03:13:05.072436 ip-10-0-132-14 kubenswrapper[2576]: W0909 03:13:05.068379    2576 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 09 03:13:05.072436 ip-10-0-132-14 kubenswrapper[2576]: W0909 03:13:05.068381    2576 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 09 03:13:05.072436 ip-10-0-132-14 kubenswrapper[2576]: W0909 03:13:05.068384    2576 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 09 03:13:05.072436 ip-10-0-132-14 kubenswrapper[2576]: W0909 03:13:05.068386    2576 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 09 03:13:05.072924 ip-10-0-132-14 kubenswrapper[2576]: W0909 03:13:05.068388    2576 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 09 03:13:05.072924 ip-10-0-132-14 kubenswrapper[2576]: W0909 03:13:05.068392    2576 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 09 03:13:05.072924 ip-10-0-132-14 kubenswrapper[2576]: W0909 03:13:05.068395    2576 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 09 03:13:05.072924 ip-10-0-132-14 kubenswrapper[2576]: W0909 03:13:05.068398    2576 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 09 03:13:05.072924 ip-10-0-132-14 kubenswrapper[2576]: W0909 03:13:05.068401    2576 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 09 03:13:05.072924 ip-10-0-132-14 kubenswrapper[2576]: W0909 03:13:05.068404    2576 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 09 03:13:05.072924 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:13:05.068409    2576 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Sep 09 03:13:05.072924 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:13:05.068560    2576 server.go:962] "Client rotation is on, will bootstrap in background"
Sep 09 03:13:05.072924 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:13:05.071027    2576 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir"
Sep 09 03:13:05.072924 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:13:05.072155    2576 server.go:1019] "Starting client certificate rotation"
Sep 09 03:13:05.072924 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:13:05.072249    2576 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet"
Sep 09 03:13:05.072924 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:13:05.072282    2576 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet"
Sep 09 03:13:05.075193 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:13:05.075178    2576 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Sep 09 03:13:05.079639 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:13:05.079605    2576 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Sep 09 03:13:05.089020 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:13:05.089003    2576 log.go:25] "Validated CRI v1 runtime API"
Sep 09 03:13:05.094000 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:13:05.093981    2576 log.go:25] "Validated CRI v1 image API"
Sep 09 03:13:05.094891 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:13:05.094858    2576 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd"
Sep 09 03:13:05.097471 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:13:05.097451    2576 fs.go:135] Filesystem UUIDs: map[7B77-95E7:/dev/nvme0n1p2 e5b828da-8ec2-4702-8a2a-3aeb26a22fbf:/dev/nvme0n1p3 e6385661-80be-4eb1-b5a8-938dc3641132:/dev/nvme0n1p4]
Sep 09 03:13:05.097538 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:13:05.097477    2576 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}]
Sep 09 03:13:05.103220 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:13:05.103102    2576 manager.go:216] Machine: {Timestamp:2026-09-09 03:13:05.101658872 +0000 UTC m=+0.381318752 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3134756 MemoryCapacity:32812158976 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec298566d2e4ca087c73677031d982b3 SystemUUID:ec298566-d2e4-ca08-7c73-677031d982b3 BootID:6d209ae0-10ab-42af-8aaa-2cca5f81d03d Filesystems:[{Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16406077440 Type:vfs Inodes:4005390 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6562435072 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6111232 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16406081536 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:4c:e4:05:ed:61 Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:4c:e4:05:ed:61 Speed:0 Mtu:9001} {Name:ovs-system MacAddress:9e:63:66:8c:64:0b Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:32812158976 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:34603008 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None}
Sep 09 03:13:05.103220 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:13:05.103217    2576 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available.
Sep 09 03:13:05.103352 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:13:05.103340    2576 manager.go:232] Version: {KernelVersion:5.14.0-570.138.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260901-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:}
Sep 09 03:13:05.106021 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:13:05.106002    2576 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Sep 09 03:13:05.107014 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:13:05.106986    2576 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[]
Sep 09 03:13:05.107162 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:13:05.107018    2576 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-132-14.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2}
Sep 09 03:13:05.107909 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:13:05.107898    2576 topology_manager.go:138] "Creating topology manager with none policy"
Sep 09 03:13:05.107943 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:13:05.107911    2576 container_manager_linux.go:309] "Creating device plugin manager"
Sep 09 03:13:05.107943 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:13:05.107919    2576 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock"
Sep 09 03:13:05.107943 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:13:05.107933    2576 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock"
Sep 09 03:13:05.109184 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:13:05.109173    2576 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager"
Sep 09 03:13:05.113778 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:13:05.113767    2576 state_mem.go:36] "Initialized new in-memory state store"
Sep 09 03:13:05.114111 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:13:05.114059    2576 server.go:1240] "Using root directory" path="/var/lib/kubelet"
Sep 09 03:13:05.114173 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:13:05.114155    2576 kubelet.go:486] "Attempting to sync node with API server"
Sep 09 03:13:05.114214 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:13:05.114172    2576 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests"
Sep 09 03:13:05.114214 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:13:05.114193    2576 file.go:69] "Watching path" path="/etc/kubernetes/manifests"
Sep 09 03:13:05.114214 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:13:05.114206    2576 kubelet.go:398] "Adding apiserver pod source"
Sep 09 03:13:05.114348 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:13:05.114217    2576 apiserver.go:42] "Waiting for node sync before watching apiserver pods"
Sep 09 03:13:05.114586 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:13:05.114567    2576 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Sep 09 03:13:05.114715 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:13:05.114595    2576 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Sep 09 03:13:05.117414 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:13:05.117394    2576 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.12-5.rhaos4.21.gitd1ec63c.el9" apiVersion="v1"
Sep 09 03:13:05.120459 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:13:05.120442    2576 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled"
Sep 09 03:13:05.120544 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:13:05.120482    2576 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled"
Sep 09 03:13:05.121976 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:13:05.121962    2576 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume"
Sep 09 03:13:05.122059 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:13:05.121983    2576 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir"
Sep 09 03:13:05.122059 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:13:05.121992    2576 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo"
Sep 09 03:13:05.122059 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:13:05.122000    2576 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path"
Sep 09 03:13:05.122059 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:13:05.122011    2576 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs"
Sep 09 03:13:05.122059 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:13:05.122019    2576 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret"
Sep 09 03:13:05.122059 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:13:05.122038    2576 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi"
Sep 09 03:13:05.122059 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:13:05.122046    2576 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api"
Sep 09 03:13:05.122059 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:13:05.122055    2576 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc"
Sep 09 03:13:05.122059 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:13:05.122063    2576 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap"
Sep 09 03:13:05.122352 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:13:05.122075    2576 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected"
Sep 09 03:13:05.122352 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:13:05.122083    2576 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume"
Sep 09 03:13:05.122981 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:13:05.122969    2576 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi"
Sep 09 03:13:05.123028 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:13:05.122984    2576 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image"
Sep 09 03:13:05.123917 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:13:05.123897    2576 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-zjrr4"
Sep 09 03:13:05.124369 ip-10-0-132-14 kubenswrapper[2576]: E0909 03:13:05.124343    2576 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-132-14.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node"
Sep 09 03:13:05.124369 ip-10-0-132-14 kubenswrapper[2576]: E0909 03:13:05.124344    2576 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service"
Sep 09 03:13:05.124532 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:13:05.124514    2576 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-132-14.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Sep 09 03:13:05.127153 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:13:05.127136    2576 server.go:1268] "Started kubelet"
Sep 09 03:13:05.127257 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:13:05.127229    2576 server.go:180] "Starting to listen" address="0.0.0.0" port=10250
Sep 09 03:13:05.127306 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:13:05.127226    2576 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10
Sep 09 03:13:05.127306 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:13:05.127301    2576 server_v1.go:49] "podresources" method="list" useActivePods=true
Sep 09 03:13:05.127906 ip-10-0-132-14 systemd[1]: Started Kubernetes Kubelet.
Sep 09 03:13:05.128692 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:13:05.128678    2576 server.go:310] "Adding debug handlers to kubelet server"
Sep 09 03:13:05.128869 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:13:05.128810    2576 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock"
Sep 09 03:13:05.131603 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:13:05.131586    2576 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-zjrr4"
Sep 09 03:13:05.132960 ip-10-0-132-14 kubenswrapper[2576]: E0909 03:13:05.132939    2576 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache"
Sep 09 03:13:05.136782 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:13:05.136769    2576 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving"
Sep 09 03:13:05.137491 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:13:05.137474    2576 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer"
Sep 09 03:13:05.137704 ip-10-0-132-14 kubenswrapper[2576]: E0909 03:13:05.137680    2576 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-132-14.ec2.internal\" not found"
Sep 09 03:13:05.138335 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:13:05.138315    2576 factory.go:55] Registering systemd factory
Sep 09 03:13:05.138426 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:13:05.138350    2576 desired_state_of_world_populator.go:146] "Desired state populator starts to run"
Sep 09 03:13:05.138426 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:13:05.138352    2576 volume_manager.go:311] "The desired_state_of_world populator starts"
Sep 09 03:13:05.138426 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:13:05.138372    2576 volume_manager.go:313] "Starting Kubelet Volume Manager"
Sep 09 03:13:05.138426 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:13:05.138355    2576 factory.go:223] Registration of the systemd container factory successfully
Sep 09 03:13:05.138595 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:13:05.138475    2576 reconstruct.go:97] "Volume reconstruction finished"
Sep 09 03:13:05.138595 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:13:05.138484    2576 reconciler.go:29] "Reconciler: start to sync state"
Sep 09 03:13:05.138696 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:13:05.138654    2576 factory.go:153] Registering CRI-O factory
Sep 09 03:13:05.138696 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:13:05.138667    2576 factory.go:223] Registration of the crio container factory successfully
Sep 09 03:13:05.138764 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:13:05.138726    2576 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory
Sep 09 03:13:05.138764 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:13:05.138748    2576 factory.go:103] Registering Raw factory
Sep 09 03:13:05.138764 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:13:05.138764    2576 manager.go:1196] Started watching for new ooms in manager
Sep 09 03:13:05.139457 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:13:05.139132    2576 manager.go:349] Starting recovery of all containers
Sep 09 03:13:05.139804 ip-10-0-132-14 kubenswrapper[2576]: E0909 03:13:05.138572    2576 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-132-14.ec2.internal.18d38885c5d1d16b  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-132-14.ec2.internal,UID:ip-10-0-132-14.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-132-14.ec2.internal,},FirstTimestamp:2026-09-09 03:13:05.127108971 +0000 UTC m=+0.406768851,LastTimestamp:2026-09-09 03:13:05.127108971 +0000 UTC m=+0.406768851,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-132-14.ec2.internal,}"
Sep 09 03:13:05.148105 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:13:05.148082    2576 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 09 03:13:05.149963 ip-10-0-132-14 kubenswrapper[2576]: E0909 03:13:05.149936    2576 nodelease.go:49] "Failed to get node when trying to set owner ref to the node lease" err="nodes \"ip-10-0-132-14.ec2.internal\" not found" node="ip-10-0-132-14.ec2.internal"
Sep 09 03:13:05.150371 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:13:05.150333    2576 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6"
Sep 09 03:13:05.151772 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:13:05.151757    2576 manager.go:354] Recovery completed
Sep 09 03:13:05.155758 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:13:05.155745    2576 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 09 03:13:05.159595 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:13:05.159578    2576 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-14.ec2.internal" event="NodeHasSufficientMemory"
Sep 09 03:13:05.159684 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:13:05.159608    2576 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-14.ec2.internal" event="NodeHasNoDiskPressure"
Sep 09 03:13:05.159684 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:13:05.159623    2576 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-14.ec2.internal" event="NodeHasSufficientPID"
Sep 09 03:13:05.160187 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:13:05.160170    2576 cpu_manager.go:222] "Starting CPU manager" policy="none"
Sep 09 03:13:05.160187 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:13:05.160185    2576 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s"
Sep 09 03:13:05.160298 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:13:05.160189    2576 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 09 03:13:05.160298 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:13:05.160201    2576 state_mem.go:36] "Initialized new in-memory state store"
Sep 09 03:13:05.160942 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:13:05.160927    2576 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-14.ec2.internal" event="NodeHasSufficientMemory"
Sep 09 03:13:05.161032 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:13:05.160959    2576 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-14.ec2.internal" event="NodeHasNoDiskPressure"
Sep 09 03:13:05.161032 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:13:05.160977    2576 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-14.ec2.internal" event="NodeHasSufficientPID"
Sep 09 03:13:05.162278 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:13:05.162266    2576 policy_none.go:49] "None policy: Start"
Sep 09 03:13:05.162326 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:13:05.162283    2576 memory_manager.go:187] "Starting memorymanager" policy="None"
Sep 09 03:13:05.162326 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:13:05.162294    2576 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint"
Sep 09 03:13:05.164288 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:13:05.164276    2576 policy_none.go:47] "Start"
Sep 09 03:13:05.206213 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:13:05.206199    2576 manager.go:337] "Starting Device Plugin manager"
Sep 09 03:13:05.225061 ip-10-0-132-14 kubenswrapper[2576]: E0909 03:13:05.206236    2576 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint"
Sep 09 03:13:05.225061 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:13:05.206250    2576 server.go:87] "Starting device plugin registration server"
Sep 09 03:13:05.225061 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:13:05.206506    2576 eviction_manager.go:189] "Eviction manager: starting control loop"
Sep 09 03:13:05.225061 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:13:05.206517    2576 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s"
Sep 09 03:13:05.225061 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:13:05.206621    2576 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry"
Sep 09 03:13:05.225061 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:13:05.206716    2576 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts"
Sep 09 03:13:05.225061 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:13:05.206725    2576 plugin_manager.go:118] "Starting Kubelet Plugin Manager"
Sep 09 03:13:05.225061 ip-10-0-132-14 kubenswrapper[2576]: E0909 03:13:05.207275    2576 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\""
Sep 09 03:13:05.225061 ip-10-0-132-14 kubenswrapper[2576]: E0909 03:13:05.207321    2576 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-132-14.ec2.internal\" not found"
Sep 09 03:13:05.225061 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:13:05.215447    2576 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4"
Sep 09 03:13:05.225061 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:13:05.215494    2576 status_manager.go:244] "Starting to sync pod status with apiserver"
Sep 09 03:13:05.225061 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:13:05.215514    2576 kubelet.go:2448] "Starting kubelet main sync loop"
Sep 09 03:13:05.225061 ip-10-0-132-14 kubenswrapper[2576]: E0909 03:13:05.215554    2576 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful"
Sep 09 03:13:05.225061 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:13:05.217105    2576 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 09 03:13:05.306980 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:13:05.306865    2576 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 09 03:13:05.307864 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:13:05.307845    2576 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-14.ec2.internal" event="NodeHasSufficientMemory"
Sep 09 03:13:05.307955 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:13:05.307886    2576 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-14.ec2.internal" event="NodeHasNoDiskPressure"
Sep 09 03:13:05.307955 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:13:05.307898    2576 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-14.ec2.internal" event="NodeHasSufficientPID"
Sep 09 03:13:05.307955 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:13:05.307921    2576 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-132-14.ec2.internal"
Sep 09 03:13:05.316396 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:13:05.316377    2576 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-132-14.ec2.internal"
Sep 09 03:13:05.316498 ip-10-0-132-14 kubenswrapper[2576]: E0909 03:13:05.316402    2576 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-132-14.ec2.internal\": node \"ip-10-0-132-14.ec2.internal\" not found"
Sep 09 03:13:05.316498 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:13:05.316402    2576 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-14.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-132-14.ec2.internal"]
Sep 09 03:13:05.316498 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:13:05.316485    2576 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 09 03:13:05.318127 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:13:05.318113    2576 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-14.ec2.internal" event="NodeHasSufficientMemory"
Sep 09 03:13:05.318208 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:13:05.318133    2576 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-14.ec2.internal" event="NodeHasNoDiskPressure"
Sep 09 03:13:05.318208 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:13:05.318145    2576 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-14.ec2.internal" event="NodeHasSufficientPID"
Sep 09 03:13:05.320428 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:13:05.320415    2576 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 09 03:13:05.320571 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:13:05.320545    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-14.ec2.internal"
Sep 09 03:13:05.320605 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:13:05.320587    2576 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 09 03:13:05.321159 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:13:05.321137    2576 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-14.ec2.internal" event="NodeHasSufficientMemory"
Sep 09 03:13:05.321159 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:13:05.321138    2576 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-14.ec2.internal" event="NodeHasSufficientMemory"
Sep 09 03:13:05.321282 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:13:05.321187    2576 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-14.ec2.internal" event="NodeHasNoDiskPressure"
Sep 09 03:13:05.321282 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:13:05.321208    2576 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-14.ec2.internal" event="NodeHasSufficientPID"
Sep 09 03:13:05.321282 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:13:05.321162    2576 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-14.ec2.internal" event="NodeHasNoDiskPressure"
Sep 09 03:13:05.321282 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:13:05.321247    2576 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-14.ec2.internal" event="NodeHasSufficientPID"
Sep 09 03:13:05.323450 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:13:05.323432    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-132-14.ec2.internal"
Sep 09 03:13:05.323543 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:13:05.323458    2576 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 09 03:13:05.324070 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:13:05.324054    2576 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-14.ec2.internal" event="NodeHasSufficientMemory"
Sep 09 03:13:05.324155 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:13:05.324083    2576 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-14.ec2.internal" event="NodeHasNoDiskPressure"
Sep 09 03:13:05.324155 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:13:05.324098    2576 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-14.ec2.internal" event="NodeHasSufficientPID"
Sep 09 03:13:05.334384 ip-10-0-132-14 kubenswrapper[2576]: E0909 03:13:05.334368    2576 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-132-14.ec2.internal\" not found"
Sep 09 03:13:05.337254 ip-10-0-132-14 kubenswrapper[2576]: E0909 03:13:05.337237    2576 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-132-14.ec2.internal\" not found" node="ip-10-0-132-14.ec2.internal"
Sep 09 03:13:05.338733 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:13:05.338716    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/3882ef2c4b5f745a591c85452d6b7dae-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-132-14.ec2.internal\" (UID: \"3882ef2c4b5f745a591c85452d6b7dae\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-14.ec2.internal"
Sep 09 03:13:05.338794 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:13:05.338740    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/3882ef2c4b5f745a591c85452d6b7dae-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-132-14.ec2.internal\" (UID: \"3882ef2c4b5f745a591c85452d6b7dae\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-14.ec2.internal"
Sep 09 03:13:05.338794 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:13:05.338755    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/f91cf12d2f3ddb8526c92708693ce21c-config\") pod \"kube-apiserver-proxy-ip-10-0-132-14.ec2.internal\" (UID: \"f91cf12d2f3ddb8526c92708693ce21c\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-132-14.ec2.internal"
Sep 09 03:13:05.340722 ip-10-0-132-14 kubenswrapper[2576]: E0909 03:13:05.340708    2576 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-132-14.ec2.internal\" not found" node="ip-10-0-132-14.ec2.internal"
Sep 09 03:13:05.435054 ip-10-0-132-14 kubenswrapper[2576]: E0909 03:13:05.435028    2576 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-132-14.ec2.internal\" not found"
Sep 09 03:13:05.439309 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:13:05.439291    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/3882ef2c4b5f745a591c85452d6b7dae-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-132-14.ec2.internal\" (UID: \"3882ef2c4b5f745a591c85452d6b7dae\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-14.ec2.internal"
Sep 09 03:13:05.439367 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:13:05.439319    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/3882ef2c4b5f745a591c85452d6b7dae-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-132-14.ec2.internal\" (UID: \"3882ef2c4b5f745a591c85452d6b7dae\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-14.ec2.internal"
Sep 09 03:13:05.439367 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:13:05.439337    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/f91cf12d2f3ddb8526c92708693ce21c-config\") pod \"kube-apiserver-proxy-ip-10-0-132-14.ec2.internal\" (UID: \"f91cf12d2f3ddb8526c92708693ce21c\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-132-14.ec2.internal"
Sep 09 03:13:05.439428 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:13:05.439378    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/3882ef2c4b5f745a591c85452d6b7dae-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-132-14.ec2.internal\" (UID: \"3882ef2c4b5f745a591c85452d6b7dae\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-14.ec2.internal"
Sep 09 03:13:05.439464 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:13:05.439429    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/3882ef2c4b5f745a591c85452d6b7dae-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-132-14.ec2.internal\" (UID: \"3882ef2c4b5f745a591c85452d6b7dae\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-14.ec2.internal"
Sep 09 03:13:05.439464 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:13:05.439455    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/f91cf12d2f3ddb8526c92708693ce21c-config\") pod \"kube-apiserver-proxy-ip-10-0-132-14.ec2.internal\" (UID: \"f91cf12d2f3ddb8526c92708693ce21c\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-132-14.ec2.internal"
Sep 09 03:13:05.535591 ip-10-0-132-14 kubenswrapper[2576]: E0909 03:13:05.535548    2576 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-132-14.ec2.internal\" not found"
Sep 09 03:13:05.636315 ip-10-0-132-14 kubenswrapper[2576]: E0909 03:13:05.636245    2576 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-132-14.ec2.internal\" not found"
Sep 09 03:13:05.640543 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:13:05.640529    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-14.ec2.internal"
Sep 09 03:13:05.646463 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:13:05.646012    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-132-14.ec2.internal"
Sep 09 03:13:05.737360 ip-10-0-132-14 kubenswrapper[2576]: E0909 03:13:05.737325    2576 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-132-14.ec2.internal\" not found"
Sep 09 03:13:05.837788 ip-10-0-132-14 kubenswrapper[2576]: E0909 03:13:05.837760    2576 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-132-14.ec2.internal\" not found"
Sep 09 03:13:05.938277 ip-10-0-132-14 kubenswrapper[2576]: E0909 03:13:05.938181    2576 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-132-14.ec2.internal\" not found"
Sep 09 03:13:06.020175 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:13:06.020147    2576 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 09 03:13:06.038721 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:13:06.038696    2576 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-14.ec2.internal"
Sep 09 03:13:06.048856 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:13:06.048836    2576 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Sep 09 03:13:06.050759 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:13:06.050743    2576 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-132-14.ec2.internal"
Sep 09 03:13:06.060563 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:13:06.060548    2576 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Sep 09 03:13:06.071690 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:13:06.071665    2576 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials"
Sep 09 03:13:06.071824 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:13:06.071806    2576 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received"
Sep 09 03:13:06.071873 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:13:06.071808    2576 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received"
Sep 09 03:13:06.071873 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:13:06.071808    2576 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received"
Sep 09 03:13:06.116015 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:13:06.115978    2576 apiserver.go:52] "Watching apiserver"
Sep 09 03:13:06.135434 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:13:06.135402    2576 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66"
Sep 09 03:13:06.135792 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:13:06.135763    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-cluster-node-tuning-operator/tuned-4b7pm","openshift-multus/network-metrics-daemon-99qzh","openshift-ovn-kubernetes/ovnkube-node-9t27t","kube-system/konnectivity-agent-c4rmf","openshift-dns/node-resolver-cqhff","openshift-image-registry/node-ca-2zp8k","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-14.ec2.internal","openshift-multus/multus-5rpmm","openshift-multus/multus-additional-cni-plugins-qpxhx","openshift-network-diagnostics/network-check-target-s2pb7","openshift-network-operator/iptables-alerter-j5fzd","kube-system/global-pull-secret-syncer-t9bbm","kube-system/kube-apiserver-proxy-ip-10-0-132-14.ec2.internal","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rhxhj"]
Sep 09 03:13:06.137438 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:13:06.137416    2576 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving"
Sep 09 03:13:06.138090 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:13:06.138072    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-s2pb7"
Sep 09 03:13:06.138173 ip-10-0-132-14 kubenswrapper[2576]: E0909 03:13:06.138153    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-s2pb7" podUID="344533be-8249-42e9-9abc-c0bc68e18132"
Sep 09 03:13:06.140162 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:13:06.140130    2576 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-09-08 03:08:06 +0000 UTC" deadline="2028-06-24 11:50:57.910347044 +0000 UTC"
Sep 09 03:13:06.140162 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:13:06.140156    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-j5fzd"
Sep 09 03:13:06.140276 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:13:06.140162    2576 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="15704h37m51.770189023s"
Sep 09 03:13:06.142329 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:13:06.142310    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-9t27t"
Sep 09 03:13:06.143312 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:13:06.143287    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4dddl\" (UniqueName: \"kubernetes.io/projected/344533be-8249-42e9-9abc-c0bc68e18132-kube-api-access-4dddl\") pod \"network-check-target-s2pb7\" (UID: \"344533be-8249-42e9-9abc-c0bc68e18132\") " pod="openshift-network-diagnostics/network-check-target-s2pb7"
Sep 09 03:13:06.143435 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:13:06.143325    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/8c6c6ac7-ea75-45fd-ade0-a1767724b340-iptables-alerter-script\") pod \"iptables-alerter-j5fzd\" (UID: \"8c6c6ac7-ea75-45fd-ade0-a1767724b340\") " pod="openshift-network-operator/iptables-alerter-j5fzd"
Sep 09 03:13:06.143435 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:13:06.143357    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/8c6c6ac7-ea75-45fd-ade0-a1767724b340-host-slash\") pod \"iptables-alerter-j5fzd\" (UID: \"8c6c6ac7-ea75-45fd-ade0-a1767724b340\") " pod="openshift-network-operator/iptables-alerter-j5fzd"
Sep 09 03:13:06.143435 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:13:06.143387    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mk9h5\" (UniqueName: \"kubernetes.io/projected/8c6c6ac7-ea75-45fd-ade0-a1767724b340-kube-api-access-mk9h5\") pod \"iptables-alerter-j5fzd\" (UID: \"8c6c6ac7-ea75-45fd-ade0-a1767724b340\") " pod="openshift-network-operator/iptables-alerter-j5fzd"
Sep 09 03:13:06.144029 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:13:06.144009    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\""
Sep 09 03:13:06.144245 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:13:06.144226    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\""
Sep 09 03:13:06.144460 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:13:06.144436    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\""
Sep 09 03:13:06.145006 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:13:06.144986    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-k476f\""
Sep 09 03:13:06.145118 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:13:06.145087    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-c4rmf"
Sep 09 03:13:06.146142 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:13:06.146122    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-z4h5g\""
Sep 09 03:13:06.146261 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:13:06.146243    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\""
Sep 09 03:13:06.146359 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:13:06.146324    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\""
Sep 09 03:13:06.146406 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:13:06.146382    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\""
Sep 09 03:13:06.146521 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:13:06.146505    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\""
Sep 09 03:13:06.146557 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:13:06.146508    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\""
Sep 09 03:13:06.146614 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:13:06.146515    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\""
Sep 09 03:13:06.147449 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:13:06.147434    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-b9mlp\""
Sep 09 03:13:06.147519 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:13:06.147461    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\""
Sep 09 03:13:06.147576 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:13:06.147516    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\""
Sep 09 03:13:06.147692 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:13:06.147679    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-cqhff"
Sep 09 03:13:06.149040 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:13:06.149026    2576 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Sep 09 03:13:06.149710 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:13:06.149695    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-2zp8k"
Sep 09 03:13:06.149790 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:13:06.149739    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\""
Sep 09 03:13:06.149958 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:13:06.149929    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-hrwmn\""
Sep 09 03:13:06.150153 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:13:06.150138    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\""
Sep 09 03:13:06.151831 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:13:06.151814    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\""
Sep 09 03:13:06.152097 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:13:06.152077    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-qpxhx"
Sep 09 03:13:06.152203 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:13:06.152108    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\""
Sep 09 03:13:06.152203 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:13:06.152119    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\""
Sep 09 03:13:06.152460 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:13:06.152442    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-q7lsd\""
Sep 09 03:13:06.154553 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:13:06.154405    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rhxhj"
Sep 09 03:13:06.154621 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:13:06.154580    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-kfxjx\""
Sep 09 03:13:06.154830 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:13:06.154815    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\""
Sep 09 03:13:06.154903 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:13:06.154893    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\""
Sep 09 03:13:06.154991 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:13:06.154893    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\""
Sep 09 03:13:06.155377 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:13:06.155345    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\""
Sep 09 03:13:06.155455 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:13:06.155438    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\""
Sep 09 03:13:06.156854 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:13:06.156838    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-99qzh"
Sep 09 03:13:06.156948 ip-10-0-132-14 kubenswrapper[2576]: E0909 03:13:06.156901    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-99qzh" podUID="443a2987-5246-4a8c-8631-1b751e02101b"
Sep 09 03:13:06.156948 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:13:06.156926    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\""
Sep 09 03:13:06.157061 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:13:06.156961    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-tt6wv\""
Sep 09 03:13:06.157165 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:13:06.157144    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\""
Sep 09 03:13:06.157277 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:13:06.157264    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\""
Sep 09 03:13:06.159547 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:13:06.159530    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-4b7pm"
Sep 09 03:13:06.161874 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:13:06.161857    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-5rpmm"
Sep 09 03:13:06.161984 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:13:06.161963    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-kdfzf\""
Sep 09 03:13:06.162085 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:13:06.162044    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\""
Sep 09 03:13:06.162146 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:13:06.162088    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\""
Sep 09 03:13:06.164277 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:13:06.164263    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\""
Sep 09 03:13:06.164379 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:13:06.164365    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-tz2vp\""
Sep 09 03:13:06.164982 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:13:06.164970    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-t9bbm"
Sep 09 03:13:06.165030 ip-10-0-132-14 kubenswrapper[2576]: E0909 03:13:06.165019    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-t9bbm" podUID="2f43cc28-7345-4c06-9160-4245b99697b4"
Sep 09 03:13:06.171144 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:13:06.171123    2576 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-f6md9"
Sep 09 03:13:06.178995 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:13:06.178974    2576 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-f6md9"
Sep 09 03:13:06.217901 ip-10-0-132-14 kubenswrapper[2576]: W0909 03:13:06.217738    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3882ef2c4b5f745a591c85452d6b7dae.slice/crio-9bc9187c771c2bb7bcac98dd608596b8a101f5f5b1b0218a7042d4c7d2ae242e WatchSource:0}: Error finding container 9bc9187c771c2bb7bcac98dd608596b8a101f5f5b1b0218a7042d4c7d2ae242e: Status 404 returned error can't find the container with id 9bc9187c771c2bb7bcac98dd608596b8a101f5f5b1b0218a7042d4c7d2ae242e
Sep 09 03:13:06.221832 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:13:06.221813    2576 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 09 03:13:06.239753 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:13:06.239734    2576 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world"
Sep 09 03:13:06.243517 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:13:06.243501    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/849f0c91-81ec-458a-a73d-c1922a7f5bc5-etc-kubernetes\") pod \"tuned-4b7pm\" (UID: \"849f0c91-81ec-458a-a73d-c1922a7f5bc5\") " pod="openshift-cluster-node-tuning-operator/tuned-4b7pm"
Sep 09 03:13:06.243600 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:13:06.243526    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c9e03e08-c803-4ca3-8f43-aa3390269c0a-tmp-dir\") pod \"node-resolver-cqhff\" (UID: \"c9e03e08-c803-4ca3-8f43-aa3390269c0a\") " pod="openshift-dns/node-resolver-cqhff"
Sep 09 03:13:06.243600 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:13:06.243543    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bjld7\" (UniqueName: \"kubernetes.io/projected/c9e03e08-c803-4ca3-8f43-aa3390269c0a-kube-api-access-bjld7\") pod \"node-resolver-cqhff\" (UID: \"c9e03e08-c803-4ca3-8f43-aa3390269c0a\") " pod="openshift-dns/node-resolver-cqhff"
Sep 09 03:13:06.243600 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:13:06.243562    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-t6jd8\" (UniqueName: \"kubernetes.io/projected/334f1652-d04c-4382-bbf3-367d96ba0b5c-kube-api-access-t6jd8\") pod \"node-ca-2zp8k\" (UID: \"334f1652-d04c-4382-bbf3-367d96ba0b5c\") " pod="openshift-image-registry/node-ca-2zp8k"
Sep 09 03:13:06.243600 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:13:06.243596    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/f0fa1f03-0dd2-4015-a21a-02fc04073778-host-cni-bin\") pod \"ovnkube-node-9t27t\" (UID: \"f0fa1f03-0dd2-4015-a21a-02fc04073778\") " pod="openshift-ovn-kubernetes/ovnkube-node-9t27t"
Sep 09 03:13:06.243786 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:13:06.243657    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/aaf8534a-72e4-4c88-972e-26eacf111fa2-cni-binary-copy\") pod \"multus-additional-cni-plugins-qpxhx\" (UID: \"aaf8534a-72e4-4c88-972e-26eacf111fa2\") " pod="openshift-multus/multus-additional-cni-plugins-qpxhx"
Sep 09 03:13:06.243786 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:13:06.243691    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/61140997-9a63-4a79-ae4d-368d92a92960-kubelet-dir\") pod \"aws-ebs-csi-driver-node-rhxhj\" (UID: \"61140997-9a63-4a79-ae4d-368d92a92960\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rhxhj"
Sep 09 03:13:06.243786 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:13:06.243717    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/9ed6dfe0-8780-4a3f-bbeb-9a012722f6e6-multus-cni-dir\") pod \"multus-5rpmm\" (UID: \"9ed6dfe0-8780-4a3f-bbeb-9a012722f6e6\") " pod="openshift-multus/multus-5rpmm"
Sep 09 03:13:06.243786 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:13:06.243740    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/9ed6dfe0-8780-4a3f-bbeb-9a012722f6e6-host-run-netns\") pod \"multus-5rpmm\" (UID: \"9ed6dfe0-8780-4a3f-bbeb-9a012722f6e6\") " pod="openshift-multus/multus-5rpmm"
Sep 09 03:13:06.243786 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:13:06.243774    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/9ed6dfe0-8780-4a3f-bbeb-9a012722f6e6-host-var-lib-kubelet\") pod \"multus-5rpmm\" (UID: \"9ed6dfe0-8780-4a3f-bbeb-9a012722f6e6\") " pod="openshift-multus/multus-5rpmm"
Sep 09 03:13:06.243951 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:13:06.243816    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/9ed6dfe0-8780-4a3f-bbeb-9a012722f6e6-host-run-multus-certs\") pod \"multus-5rpmm\" (UID: \"9ed6dfe0-8780-4a3f-bbeb-9a012722f6e6\") " pod="openshift-multus/multus-5rpmm"
Sep 09 03:13:06.243951 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:13:06.243836    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/f0fa1f03-0dd2-4015-a21a-02fc04073778-host-run-netns\") pod \"ovnkube-node-9t27t\" (UID: \"f0fa1f03-0dd2-4015-a21a-02fc04073778\") " pod="openshift-ovn-kubernetes/ovnkube-node-9t27t"
Sep 09 03:13:06.243951 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:13:06.243850    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/849f0c91-81ec-458a-a73d-c1922a7f5bc5-var-lib-kubelet\") pod \"tuned-4b7pm\" (UID: \"849f0c91-81ec-458a-a73d-c1922a7f5bc5\") " pod="openshift-cluster-node-tuning-operator/tuned-4b7pm"
Sep 09 03:13:06.243951 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:13:06.243870    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/9ed6dfe0-8780-4a3f-bbeb-9a012722f6e6-multus-daemon-config\") pod \"multus-5rpmm\" (UID: \"9ed6dfe0-8780-4a3f-bbeb-9a012722f6e6\") " pod="openshift-multus/multus-5rpmm"
Sep 09 03:13:06.243951 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:13:06.243886    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/f0fa1f03-0dd2-4015-a21a-02fc04073778-host-slash\") pod \"ovnkube-node-9t27t\" (UID: \"f0fa1f03-0dd2-4015-a21a-02fc04073778\") " pod="openshift-ovn-kubernetes/ovnkube-node-9t27t"
Sep 09 03:13:06.243951 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:13:06.243900    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/f0fa1f03-0dd2-4015-a21a-02fc04073778-var-lib-openvswitch\") pod \"ovnkube-node-9t27t\" (UID: \"f0fa1f03-0dd2-4015-a21a-02fc04073778\") " pod="openshift-ovn-kubernetes/ovnkube-node-9t27t"
Sep 09 03:13:06.243951 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:13:06.243915    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/f0fa1f03-0dd2-4015-a21a-02fc04073778-ovnkube-script-lib\") pod \"ovnkube-node-9t27t\" (UID: \"f0fa1f03-0dd2-4015-a21a-02fc04073778\") " pod="openshift-ovn-kubernetes/ovnkube-node-9t27t"
Sep 09 03:13:06.243951 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:13:06.243937    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/849f0c91-81ec-458a-a73d-c1922a7f5bc5-etc-sysctl-d\") pod \"tuned-4b7pm\" (UID: \"849f0c91-81ec-458a-a73d-c1922a7f5bc5\") " pod="openshift-cluster-node-tuning-operator/tuned-4b7pm"
Sep 09 03:13:06.244257 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:13:06.243957    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/849f0c91-81ec-458a-a73d-c1922a7f5bc5-host\") pod \"tuned-4b7pm\" (UID: \"849f0c91-81ec-458a-a73d-c1922a7f5bc5\") " pod="openshift-cluster-node-tuning-operator/tuned-4b7pm"
Sep 09 03:13:06.244257 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:13:06.243983    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/2f43cc28-7345-4c06-9160-4245b99697b4-original-pull-secret\") pod \"global-pull-secret-syncer-t9bbm\" (UID: \"2f43cc28-7345-4c06-9160-4245b99697b4\") " pod="kube-system/global-pull-secret-syncer-t9bbm"
Sep 09 03:13:06.244257 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:13:06.243998    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/c9e03e08-c803-4ca3-8f43-aa3390269c0a-hosts-file\") pod \"node-resolver-cqhff\" (UID: \"c9e03e08-c803-4ca3-8f43-aa3390269c0a\") " pod="openshift-dns/node-resolver-cqhff"
Sep 09 03:13:06.244257 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:13:06.244023    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/8c6c6ac7-ea75-45fd-ade0-a1767724b340-iptables-alerter-script\") pod \"iptables-alerter-j5fzd\" (UID: \"8c6c6ac7-ea75-45fd-ade0-a1767724b340\") " pod="openshift-network-operator/iptables-alerter-j5fzd"
Sep 09 03:13:06.244257 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:13:06.244037    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/aaf8534a-72e4-4c88-972e-26eacf111fa2-tuning-conf-dir\") pod \"multus-additional-cni-plugins-qpxhx\" (UID: \"aaf8534a-72e4-4c88-972e-26eacf111fa2\") " pod="openshift-multus/multus-additional-cni-plugins-qpxhx"
Sep 09 03:13:06.244257 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:13:06.244051    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/9ed6dfe0-8780-4a3f-bbeb-9a012722f6e6-host-var-lib-cni-multus\") pod \"multus-5rpmm\" (UID: \"9ed6dfe0-8780-4a3f-bbeb-9a012722f6e6\") " pod="openshift-multus/multus-5rpmm"
Sep 09 03:13:06.244257 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:13:06.244069    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/f0fa1f03-0dd2-4015-a21a-02fc04073778-log-socket\") pod \"ovnkube-node-9t27t\" (UID: \"f0fa1f03-0dd2-4015-a21a-02fc04073778\") " pod="openshift-ovn-kubernetes/ovnkube-node-9t27t"
Sep 09 03:13:06.244257 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:13:06.244097    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/aaf8534a-72e4-4c88-972e-26eacf111fa2-cnibin\") pod \"multus-additional-cni-plugins-qpxhx\" (UID: \"aaf8534a-72e4-4c88-972e-26eacf111fa2\") " pod="openshift-multus/multus-additional-cni-plugins-qpxhx"
Sep 09 03:13:06.244257 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:13:06.244141    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/443a2987-5246-4a8c-8631-1b751e02101b-metrics-certs\") pod \"network-metrics-daemon-99qzh\" (UID: \"443a2987-5246-4a8c-8631-1b751e02101b\") " pod="openshift-multus/network-metrics-daemon-99qzh"
Sep 09 03:13:06.244257 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:13:06.244182    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/f0fa1f03-0dd2-4015-a21a-02fc04073778-etc-openvswitch\") pod \"ovnkube-node-9t27t\" (UID: \"f0fa1f03-0dd2-4015-a21a-02fc04073778\") " pod="openshift-ovn-kubernetes/ovnkube-node-9t27t"
Sep 09 03:13:06.244257 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:13:06.244210    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/f0fa1f03-0dd2-4015-a21a-02fc04073778-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-9t27t\" (UID: \"f0fa1f03-0dd2-4015-a21a-02fc04073778\") " pod="openshift-ovn-kubernetes/ovnkube-node-9t27t"
Sep 09 03:13:06.244257 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:13:06.244240    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/f0fa1f03-0dd2-4015-a21a-02fc04073778-ovn-node-metrics-cert\") pod \"ovnkube-node-9t27t\" (UID: \"f0fa1f03-0dd2-4015-a21a-02fc04073778\") " pod="openshift-ovn-kubernetes/ovnkube-node-9t27t"
Sep 09 03:13:06.244819 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:13:06.244270    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/61140997-9a63-4a79-ae4d-368d92a92960-sys-fs\") pod \"aws-ebs-csi-driver-node-rhxhj\" (UID: \"61140997-9a63-4a79-ae4d-368d92a92960\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rhxhj"
Sep 09 03:13:06.244819 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:13:06.244293    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-72wt9\" (UniqueName: \"kubernetes.io/projected/443a2987-5246-4a8c-8631-1b751e02101b-kube-api-access-72wt9\") pod \"network-metrics-daemon-99qzh\" (UID: \"443a2987-5246-4a8c-8631-1b751e02101b\") " pod="openshift-multus/network-metrics-daemon-99qzh"
Sep 09 03:13:06.244819 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:13:06.244322    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/849f0c91-81ec-458a-a73d-c1922a7f5bc5-lib-modules\") pod \"tuned-4b7pm\" (UID: \"849f0c91-81ec-458a-a73d-c1922a7f5bc5\") " pod="openshift-cluster-node-tuning-operator/tuned-4b7pm"
Sep 09 03:13:06.244819 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:13:06.244356    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4dddl\" (UniqueName: \"kubernetes.io/projected/344533be-8249-42e9-9abc-c0bc68e18132-kube-api-access-4dddl\") pod \"network-check-target-s2pb7\" (UID: \"344533be-8249-42e9-9abc-c0bc68e18132\") " pod="openshift-network-diagnostics/network-check-target-s2pb7"
Sep 09 03:13:06.244819 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:13:06.244373    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/f0fa1f03-0dd2-4015-a21a-02fc04073778-systemd-units\") pod \"ovnkube-node-9t27t\" (UID: \"f0fa1f03-0dd2-4015-a21a-02fc04073778\") " pod="openshift-ovn-kubernetes/ovnkube-node-9t27t"
Sep 09 03:13:06.244819 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:13:06.244394    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/f0fa1f03-0dd2-4015-a21a-02fc04073778-host-cni-netd\") pod \"ovnkube-node-9t27t\" (UID: \"f0fa1f03-0dd2-4015-a21a-02fc04073778\") " pod="openshift-ovn-kubernetes/ovnkube-node-9t27t"
Sep 09 03:13:06.244819 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:13:06.244409    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/aaf8534a-72e4-4c88-972e-26eacf111fa2-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-qpxhx\" (UID: \"aaf8534a-72e4-4c88-972e-26eacf111fa2\") " pod="openshift-multus/multus-additional-cni-plugins-qpxhx"
Sep 09 03:13:06.244819 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:13:06.244426    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-59qk7\" (UniqueName: \"kubernetes.io/projected/849f0c91-81ec-458a-a73d-c1922a7f5bc5-kube-api-access-59qk7\") pod \"tuned-4b7pm\" (UID: \"849f0c91-81ec-458a-a73d-c1922a7f5bc5\") " pod="openshift-cluster-node-tuning-operator/tuned-4b7pm"
Sep 09 03:13:06.244819 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:13:06.244443    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/9ed6dfe0-8780-4a3f-bbeb-9a012722f6e6-etc-kubernetes\") pod \"multus-5rpmm\" (UID: \"9ed6dfe0-8780-4a3f-bbeb-9a012722f6e6\") " pod="openshift-multus/multus-5rpmm"
Sep 09 03:13:06.244819 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:13:06.244465    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/f0fa1f03-0dd2-4015-a21a-02fc04073778-host-run-ovn-kubernetes\") pod \"ovnkube-node-9t27t\" (UID: \"f0fa1f03-0dd2-4015-a21a-02fc04073778\") " pod="openshift-ovn-kubernetes/ovnkube-node-9t27t"
Sep 09 03:13:06.244819 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:13:06.244483    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5x42s\" (UniqueName: \"kubernetes.io/projected/f0fa1f03-0dd2-4015-a21a-02fc04073778-kube-api-access-5x42s\") pod \"ovnkube-node-9t27t\" (UID: \"f0fa1f03-0dd2-4015-a21a-02fc04073778\") " pod="openshift-ovn-kubernetes/ovnkube-node-9t27t"
Sep 09 03:13:06.244819 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:13:06.244522    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qtf7c\" (UniqueName: \"kubernetes.io/projected/61140997-9a63-4a79-ae4d-368d92a92960-kube-api-access-qtf7c\") pod \"aws-ebs-csi-driver-node-rhxhj\" (UID: \"61140997-9a63-4a79-ae4d-368d92a92960\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rhxhj"
Sep 09 03:13:06.244819 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:13:06.244556    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/849f0c91-81ec-458a-a73d-c1922a7f5bc5-etc-modprobe-d\") pod \"tuned-4b7pm\" (UID: \"849f0c91-81ec-458a-a73d-c1922a7f5bc5\") " pod="openshift-cluster-node-tuning-operator/tuned-4b7pm"
Sep 09 03:13:06.244819 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:13:06.244584    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/849f0c91-81ec-458a-a73d-c1922a7f5bc5-run\") pod \"tuned-4b7pm\" (UID: \"849f0c91-81ec-458a-a73d-c1922a7f5bc5\") " pod="openshift-cluster-node-tuning-operator/tuned-4b7pm"
Sep 09 03:13:06.244819 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:13:06.244603    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/9ed6dfe0-8780-4a3f-bbeb-9a012722f6e6-hostroot\") pod \"multus-5rpmm\" (UID: \"9ed6dfe0-8780-4a3f-bbeb-9a012722f6e6\") " pod="openshift-multus/multus-5rpmm"
Sep 09 03:13:06.244819 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:13:06.244617    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/849f0c91-81ec-458a-a73d-c1922a7f5bc5-sys\") pod \"tuned-4b7pm\" (UID: \"849f0c91-81ec-458a-a73d-c1922a7f5bc5\") " pod="openshift-cluster-node-tuning-operator/tuned-4b7pm"
Sep 09 03:13:06.245360 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:13:06.244620    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/8c6c6ac7-ea75-45fd-ade0-a1767724b340-iptables-alerter-script\") pod \"iptables-alerter-j5fzd\" (UID: \"8c6c6ac7-ea75-45fd-ade0-a1767724b340\") " pod="openshift-network-operator/iptables-alerter-j5fzd"
Sep 09 03:13:06.245360 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:13:06.244647    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/9ed6dfe0-8780-4a3f-bbeb-9a012722f6e6-multus-socket-dir-parent\") pod \"multus-5rpmm\" (UID: \"9ed6dfe0-8780-4a3f-bbeb-9a012722f6e6\") " pod="openshift-multus/multus-5rpmm"
Sep 09 03:13:06.245360 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:13:06.244691    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/9ed6dfe0-8780-4a3f-bbeb-9a012722f6e6-host-var-lib-cni-bin\") pod \"multus-5rpmm\" (UID: \"9ed6dfe0-8780-4a3f-bbeb-9a012722f6e6\") " pod="openshift-multus/multus-5rpmm"
Sep 09 03:13:06.245360 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:13:06.244709    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/f0fa1f03-0dd2-4015-a21a-02fc04073778-env-overrides\") pod \"ovnkube-node-9t27t\" (UID: \"f0fa1f03-0dd2-4015-a21a-02fc04073778\") " pod="openshift-ovn-kubernetes/ovnkube-node-9t27t"
Sep 09 03:13:06.245360 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:13:06.244745    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/9ed6dfe0-8780-4a3f-bbeb-9a012722f6e6-cnibin\") pod \"multus-5rpmm\" (UID: \"9ed6dfe0-8780-4a3f-bbeb-9a012722f6e6\") " pod="openshift-multus/multus-5rpmm"
Sep 09 03:13:06.245360 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:13:06.244759    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/9ed6dfe0-8780-4a3f-bbeb-9a012722f6e6-cni-binary-copy\") pod \"multus-5rpmm\" (UID: \"9ed6dfe0-8780-4a3f-bbeb-9a012722f6e6\") " pod="openshift-multus/multus-5rpmm"
Sep 09 03:13:06.245360 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:13:06.244773    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/9ed6dfe0-8780-4a3f-bbeb-9a012722f6e6-system-cni-dir\") pod \"multus-5rpmm\" (UID: \"9ed6dfe0-8780-4a3f-bbeb-9a012722f6e6\") " pod="openshift-multus/multus-5rpmm"
Sep 09 03:13:06.245360 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:13:06.244788    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/f0fa1f03-0dd2-4015-a21a-02fc04073778-run-systemd\") pod \"ovnkube-node-9t27t\" (UID: \"f0fa1f03-0dd2-4015-a21a-02fc04073778\") " pod="openshift-ovn-kubernetes/ovnkube-node-9t27t"
Sep 09 03:13:06.245360 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:13:06.244814    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/f0fa1f03-0dd2-4015-a21a-02fc04073778-run-openvswitch\") pod \"ovnkube-node-9t27t\" (UID: \"f0fa1f03-0dd2-4015-a21a-02fc04073778\") " pod="openshift-ovn-kubernetes/ovnkube-node-9t27t"
Sep 09 03:13:06.245360 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:13:06.244831    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/f0fa1f03-0dd2-4015-a21a-02fc04073778-ovnkube-config\") pod \"ovnkube-node-9t27t\" (UID: \"f0fa1f03-0dd2-4015-a21a-02fc04073778\") " pod="openshift-ovn-kubernetes/ovnkube-node-9t27t"
Sep 09 03:13:06.245360 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:13:06.244844    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/849f0c91-81ec-458a-a73d-c1922a7f5bc5-etc-systemd\") pod \"tuned-4b7pm\" (UID: \"849f0c91-81ec-458a-a73d-c1922a7f5bc5\") " pod="openshift-cluster-node-tuning-operator/tuned-4b7pm"
Sep 09 03:13:06.245360 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:13:06.244858    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/2f43cc28-7345-4c06-9160-4245b99697b4-dbus\") pod \"global-pull-secret-syncer-t9bbm\" (UID: \"2f43cc28-7345-4c06-9160-4245b99697b4\") " pod="kube-system/global-pull-secret-syncer-t9bbm"
Sep 09 03:13:06.245360 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:13:06.244871    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/472d1b86-78f9-45a8-a782-f4274b90a158-konnectivity-ca\") pod \"konnectivity-agent-c4rmf\" (UID: \"472d1b86-78f9-45a8-a782-f4274b90a158\") " pod="kube-system/konnectivity-agent-c4rmf"
Sep 09 03:13:06.245360 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:13:06.244893    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mk9h5\" (UniqueName: \"kubernetes.io/projected/8c6c6ac7-ea75-45fd-ade0-a1767724b340-kube-api-access-mk9h5\") pod \"iptables-alerter-j5fzd\" (UID: \"8c6c6ac7-ea75-45fd-ade0-a1767724b340\") " pod="openshift-network-operator/iptables-alerter-j5fzd"
Sep 09 03:13:06.245360 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:13:06.244906    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/f0fa1f03-0dd2-4015-a21a-02fc04073778-host-kubelet\") pod \"ovnkube-node-9t27t\" (UID: \"f0fa1f03-0dd2-4015-a21a-02fc04073778\") " pod="openshift-ovn-kubernetes/ovnkube-node-9t27t"
Sep 09 03:13:06.245360 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:13:06.244920    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/f0fa1f03-0dd2-4015-a21a-02fc04073778-run-ovn\") pod \"ovnkube-node-9t27t\" (UID: \"f0fa1f03-0dd2-4015-a21a-02fc04073778\") " pod="openshift-ovn-kubernetes/ovnkube-node-9t27t"
Sep 09 03:13:06.245360 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:13:06.244948    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/61140997-9a63-4a79-ae4d-368d92a92960-registration-dir\") pod \"aws-ebs-csi-driver-node-rhxhj\" (UID: \"61140997-9a63-4a79-ae4d-368d92a92960\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rhxhj"
Sep 09 03:13:06.245927 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:13:06.245000    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/61140997-9a63-4a79-ae4d-368d92a92960-device-dir\") pod \"aws-ebs-csi-driver-node-rhxhj\" (UID: \"61140997-9a63-4a79-ae4d-368d92a92960\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rhxhj"
Sep 09 03:13:06.245927 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:13:06.245016    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/849f0c91-81ec-458a-a73d-c1922a7f5bc5-etc-sysconfig\") pod \"tuned-4b7pm\" (UID: \"849f0c91-81ec-458a-a73d-c1922a7f5bc5\") " pod="openshift-cluster-node-tuning-operator/tuned-4b7pm"
Sep 09 03:13:06.245927 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:13:06.245033    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/334f1652-d04c-4382-bbf3-367d96ba0b5c-host\") pod \"node-ca-2zp8k\" (UID: \"334f1652-d04c-4382-bbf3-367d96ba0b5c\") " pod="openshift-image-registry/node-ca-2zp8k"
Sep 09 03:13:06.245927 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:13:06.245048    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/8c6c6ac7-ea75-45fd-ade0-a1767724b340-host-slash\") pod \"iptables-alerter-j5fzd\" (UID: \"8c6c6ac7-ea75-45fd-ade0-a1767724b340\") " pod="openshift-network-operator/iptables-alerter-j5fzd"
Sep 09 03:13:06.245927 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:13:06.245062    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/f0fa1f03-0dd2-4015-a21a-02fc04073778-node-log\") pod \"ovnkube-node-9t27t\" (UID: \"f0fa1f03-0dd2-4015-a21a-02fc04073778\") " pod="openshift-ovn-kubernetes/ovnkube-node-9t27t"
Sep 09 03:13:06.245927 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:13:06.245111    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/aaf8534a-72e4-4c88-972e-26eacf111fa2-system-cni-dir\") pod \"multus-additional-cni-plugins-qpxhx\" (UID: \"aaf8534a-72e4-4c88-972e-26eacf111fa2\") " pod="openshift-multus/multus-additional-cni-plugins-qpxhx"
Sep 09 03:13:06.245927 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:13:06.245108    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/8c6c6ac7-ea75-45fd-ade0-a1767724b340-host-slash\") pod \"iptables-alerter-j5fzd\" (UID: \"8c6c6ac7-ea75-45fd-ade0-a1767724b340\") " pod="openshift-network-operator/iptables-alerter-j5fzd"
Sep 09 03:13:06.245927 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:13:06.245127    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/61140997-9a63-4a79-ae4d-368d92a92960-socket-dir\") pod \"aws-ebs-csi-driver-node-rhxhj\" (UID: \"61140997-9a63-4a79-ae4d-368d92a92960\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rhxhj"
Sep 09 03:13:06.245927 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:13:06.245143    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/849f0c91-81ec-458a-a73d-c1922a7f5bc5-tmp\") pod \"tuned-4b7pm\" (UID: \"849f0c91-81ec-458a-a73d-c1922a7f5bc5\") " pod="openshift-cluster-node-tuning-operator/tuned-4b7pm"
Sep 09 03:13:06.245927 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:13:06.245157    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/9ed6dfe0-8780-4a3f-bbeb-9a012722f6e6-host-run-k8s-cni-cncf-io\") pod \"multus-5rpmm\" (UID: \"9ed6dfe0-8780-4a3f-bbeb-9a012722f6e6\") " pod="openshift-multus/multus-5rpmm"
Sep 09 03:13:06.245927 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:13:06.245171    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-r79cr\" (UniqueName: \"kubernetes.io/projected/9ed6dfe0-8780-4a3f-bbeb-9a012722f6e6-kube-api-access-r79cr\") pod \"multus-5rpmm\" (UID: \"9ed6dfe0-8780-4a3f-bbeb-9a012722f6e6\") " pod="openshift-multus/multus-5rpmm"
Sep 09 03:13:06.245927 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:13:06.245185    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/2f43cc28-7345-4c06-9160-4245b99697b4-kubelet-config\") pod \"global-pull-secret-syncer-t9bbm\" (UID: \"2f43cc28-7345-4c06-9160-4245b99697b4\") " pod="kube-system/global-pull-secret-syncer-t9bbm"
Sep 09 03:13:06.245927 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:13:06.245208    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-25gnh\" (UniqueName: \"kubernetes.io/projected/aaf8534a-72e4-4c88-972e-26eacf111fa2-kube-api-access-25gnh\") pod \"multus-additional-cni-plugins-qpxhx\" (UID: \"aaf8534a-72e4-4c88-972e-26eacf111fa2\") " pod="openshift-multus/multus-additional-cni-plugins-qpxhx"
Sep 09 03:13:06.245927 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:13:06.245235    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/849f0c91-81ec-458a-a73d-c1922a7f5bc5-etc-sysctl-conf\") pod \"tuned-4b7pm\" (UID: \"849f0c91-81ec-458a-a73d-c1922a7f5bc5\") " pod="openshift-cluster-node-tuning-operator/tuned-4b7pm"
Sep 09 03:13:06.245927 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:13:06.245251    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/849f0c91-81ec-458a-a73d-c1922a7f5bc5-etc-tuned\") pod \"tuned-4b7pm\" (UID: \"849f0c91-81ec-458a-a73d-c1922a7f5bc5\") " pod="openshift-cluster-node-tuning-operator/tuned-4b7pm"
Sep 09 03:13:06.245927 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:13:06.245264    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/9ed6dfe0-8780-4a3f-bbeb-9a012722f6e6-multus-conf-dir\") pod \"multus-5rpmm\" (UID: \"9ed6dfe0-8780-4a3f-bbeb-9a012722f6e6\") " pod="openshift-multus/multus-5rpmm"
Sep 09 03:13:06.246351 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:13:06.245282    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/472d1b86-78f9-45a8-a782-f4274b90a158-agent-certs\") pod \"konnectivity-agent-c4rmf\" (UID: \"472d1b86-78f9-45a8-a782-f4274b90a158\") " pod="kube-system/konnectivity-agent-c4rmf"
Sep 09 03:13:06.246351 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:13:06.245302    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/334f1652-d04c-4382-bbf3-367d96ba0b5c-serviceca\") pod \"node-ca-2zp8k\" (UID: \"334f1652-d04c-4382-bbf3-367d96ba0b5c\") " pod="openshift-image-registry/node-ca-2zp8k"
Sep 09 03:13:06.246351 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:13:06.245325    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/aaf8534a-72e4-4c88-972e-26eacf111fa2-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-qpxhx\" (UID: \"aaf8534a-72e4-4c88-972e-26eacf111fa2\") " pod="openshift-multus/multus-additional-cni-plugins-qpxhx"
Sep 09 03:13:06.246351 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:13:06.245343    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/61140997-9a63-4a79-ae4d-368d92a92960-etc-selinux\") pod \"aws-ebs-csi-driver-node-rhxhj\" (UID: \"61140997-9a63-4a79-ae4d-368d92a92960\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rhxhj"
Sep 09 03:13:06.250170 ip-10-0-132-14 kubenswrapper[2576]: E0909 03:13:06.250152    2576 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 09 03:13:06.250260 ip-10-0-132-14 kubenswrapper[2576]: E0909 03:13:06.250175    2576 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 09 03:13:06.250260 ip-10-0-132-14 kubenswrapper[2576]: E0909 03:13:06.250188    2576 projected.go:196] Error preparing data for projected volume kube-api-access-4dddl for pod openshift-network-diagnostics/network-check-target-s2pb7: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 09 03:13:06.250365 ip-10-0-132-14 kubenswrapper[2576]: E0909 03:13:06.250261    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/344533be-8249-42e9-9abc-c0bc68e18132-kube-api-access-4dddl podName:344533be-8249-42e9-9abc-c0bc68e18132 nodeName:}" failed. No retries permitted until 2026-09-09 03:13:06.750240419 +0000 UTC m=+2.029900303 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-4dddl" (UniqueName: "kubernetes.io/projected/344533be-8249-42e9-9abc-c0bc68e18132-kube-api-access-4dddl") pod "network-check-target-s2pb7" (UID: "344533be-8249-42e9-9abc-c0bc68e18132") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 09 03:13:06.253355 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:13:06.253328    2576 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory"
Sep 09 03:13:06.256907 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:13:06.256892    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mk9h5\" (UniqueName: \"kubernetes.io/projected/8c6c6ac7-ea75-45fd-ade0-a1767724b340-kube-api-access-mk9h5\") pod \"iptables-alerter-j5fzd\" (UID: \"8c6c6ac7-ea75-45fd-ade0-a1767724b340\") " pod="openshift-network-operator/iptables-alerter-j5fzd"
Sep 09 03:13:06.346547 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:13:06.346517    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/9ed6dfe0-8780-4a3f-bbeb-9a012722f6e6-host-run-netns\") pod \"multus-5rpmm\" (UID: \"9ed6dfe0-8780-4a3f-bbeb-9a012722f6e6\") " pod="openshift-multus/multus-5rpmm"
Sep 09 03:13:06.346547 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:13:06.346548    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/9ed6dfe0-8780-4a3f-bbeb-9a012722f6e6-host-var-lib-kubelet\") pod \"multus-5rpmm\" (UID: \"9ed6dfe0-8780-4a3f-bbeb-9a012722f6e6\") " pod="openshift-multus/multus-5rpmm"
Sep 09 03:13:06.346792 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:13:06.346570    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/9ed6dfe0-8780-4a3f-bbeb-9a012722f6e6-host-run-multus-certs\") pod \"multus-5rpmm\" (UID: \"9ed6dfe0-8780-4a3f-bbeb-9a012722f6e6\") " pod="openshift-multus/multus-5rpmm"
Sep 09 03:13:06.346792 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:13:06.346593    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/f0fa1f03-0dd2-4015-a21a-02fc04073778-host-run-netns\") pod \"ovnkube-node-9t27t\" (UID: \"f0fa1f03-0dd2-4015-a21a-02fc04073778\") " pod="openshift-ovn-kubernetes/ovnkube-node-9t27t"
Sep 09 03:13:06.346792 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:13:06.346615    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/849f0c91-81ec-458a-a73d-c1922a7f5bc5-var-lib-kubelet\") pod \"tuned-4b7pm\" (UID: \"849f0c91-81ec-458a-a73d-c1922a7f5bc5\") " pod="openshift-cluster-node-tuning-operator/tuned-4b7pm"
Sep 09 03:13:06.346792 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:13:06.346647    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/9ed6dfe0-8780-4a3f-bbeb-9a012722f6e6-host-run-netns\") pod \"multus-5rpmm\" (UID: \"9ed6dfe0-8780-4a3f-bbeb-9a012722f6e6\") " pod="openshift-multus/multus-5rpmm"
Sep 09 03:13:06.346792 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:13:06.346653    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/9ed6dfe0-8780-4a3f-bbeb-9a012722f6e6-multus-daemon-config\") pod \"multus-5rpmm\" (UID: \"9ed6dfe0-8780-4a3f-bbeb-9a012722f6e6\") " pod="openshift-multus/multus-5rpmm"
Sep 09 03:13:06.346792 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:13:06.346663    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/9ed6dfe0-8780-4a3f-bbeb-9a012722f6e6-host-var-lib-kubelet\") pod \"multus-5rpmm\" (UID: \"9ed6dfe0-8780-4a3f-bbeb-9a012722f6e6\") " pod="openshift-multus/multus-5rpmm"
Sep 09 03:13:06.346792 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:13:06.346683    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/f0fa1f03-0dd2-4015-a21a-02fc04073778-host-slash\") pod \"ovnkube-node-9t27t\" (UID: \"f0fa1f03-0dd2-4015-a21a-02fc04073778\") " pod="openshift-ovn-kubernetes/ovnkube-node-9t27t"
Sep 09 03:13:06.346792 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:13:06.346689    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/9ed6dfe0-8780-4a3f-bbeb-9a012722f6e6-host-run-multus-certs\") pod \"multus-5rpmm\" (UID: \"9ed6dfe0-8780-4a3f-bbeb-9a012722f6e6\") " pod="openshift-multus/multus-5rpmm"
Sep 09 03:13:06.346792 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:13:06.346698    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/f0fa1f03-0dd2-4015-a21a-02fc04073778-host-run-netns\") pod \"ovnkube-node-9t27t\" (UID: \"f0fa1f03-0dd2-4015-a21a-02fc04073778\") " pod="openshift-ovn-kubernetes/ovnkube-node-9t27t"
Sep 09 03:13:06.346792 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:13:06.346707    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/f0fa1f03-0dd2-4015-a21a-02fc04073778-var-lib-openvswitch\") pod \"ovnkube-node-9t27t\" (UID: \"f0fa1f03-0dd2-4015-a21a-02fc04073778\") " pod="openshift-ovn-kubernetes/ovnkube-node-9t27t"
Sep 09 03:13:06.346792 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:13:06.346723    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/849f0c91-81ec-458a-a73d-c1922a7f5bc5-var-lib-kubelet\") pod \"tuned-4b7pm\" (UID: \"849f0c91-81ec-458a-a73d-c1922a7f5bc5\") " pod="openshift-cluster-node-tuning-operator/tuned-4b7pm"
Sep 09 03:13:06.346792 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:13:06.346747    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/f0fa1f03-0dd2-4015-a21a-02fc04073778-host-slash\") pod \"ovnkube-node-9t27t\" (UID: \"f0fa1f03-0dd2-4015-a21a-02fc04073778\") " pod="openshift-ovn-kubernetes/ovnkube-node-9t27t"
Sep 09 03:13:06.346792 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:13:06.346747    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/f0fa1f03-0dd2-4015-a21a-02fc04073778-ovnkube-script-lib\") pod \"ovnkube-node-9t27t\" (UID: \"f0fa1f03-0dd2-4015-a21a-02fc04073778\") " pod="openshift-ovn-kubernetes/ovnkube-node-9t27t"
Sep 09 03:13:06.346792 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:13:06.346748    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/f0fa1f03-0dd2-4015-a21a-02fc04073778-var-lib-openvswitch\") pod \"ovnkube-node-9t27t\" (UID: \"f0fa1f03-0dd2-4015-a21a-02fc04073778\") " pod="openshift-ovn-kubernetes/ovnkube-node-9t27t"
Sep 09 03:13:06.347468 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:13:06.346828    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/849f0c91-81ec-458a-a73d-c1922a7f5bc5-etc-sysctl-d\") pod \"tuned-4b7pm\" (UID: \"849f0c91-81ec-458a-a73d-c1922a7f5bc5\") " pod="openshift-cluster-node-tuning-operator/tuned-4b7pm"
Sep 09 03:13:06.347468 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:13:06.346856    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/849f0c91-81ec-458a-a73d-c1922a7f5bc5-host\") pod \"tuned-4b7pm\" (UID: \"849f0c91-81ec-458a-a73d-c1922a7f5bc5\") " pod="openshift-cluster-node-tuning-operator/tuned-4b7pm"
Sep 09 03:13:06.347468 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:13:06.346876    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/2f43cc28-7345-4c06-9160-4245b99697b4-original-pull-secret\") pod \"global-pull-secret-syncer-t9bbm\" (UID: \"2f43cc28-7345-4c06-9160-4245b99697b4\") " pod="kube-system/global-pull-secret-syncer-t9bbm"
Sep 09 03:13:06.347468 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:13:06.346899    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/c9e03e08-c803-4ca3-8f43-aa3390269c0a-hosts-file\") pod \"node-resolver-cqhff\" (UID: \"c9e03e08-c803-4ca3-8f43-aa3390269c0a\") " pod="openshift-dns/node-resolver-cqhff"
Sep 09 03:13:06.347468 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:13:06.346936    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/aaf8534a-72e4-4c88-972e-26eacf111fa2-tuning-conf-dir\") pod \"multus-additional-cni-plugins-qpxhx\" (UID: \"aaf8534a-72e4-4c88-972e-26eacf111fa2\") " pod="openshift-multus/multus-additional-cni-plugins-qpxhx"
Sep 09 03:13:06.347468 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:13:06.346946    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/849f0c91-81ec-458a-a73d-c1922a7f5bc5-host\") pod \"tuned-4b7pm\" (UID: \"849f0c91-81ec-458a-a73d-c1922a7f5bc5\") " pod="openshift-cluster-node-tuning-operator/tuned-4b7pm"
Sep 09 03:13:06.347468 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:13:06.346954    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/c9e03e08-c803-4ca3-8f43-aa3390269c0a-hosts-file\") pod \"node-resolver-cqhff\" (UID: \"c9e03e08-c803-4ca3-8f43-aa3390269c0a\") " pod="openshift-dns/node-resolver-cqhff"
Sep 09 03:13:06.347468 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:13:06.346963    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/849f0c91-81ec-458a-a73d-c1922a7f5bc5-etc-sysctl-d\") pod \"tuned-4b7pm\" (UID: \"849f0c91-81ec-458a-a73d-c1922a7f5bc5\") " pod="openshift-cluster-node-tuning-operator/tuned-4b7pm"
Sep 09 03:13:06.347468 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:13:06.347011    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/9ed6dfe0-8780-4a3f-bbeb-9a012722f6e6-host-var-lib-cni-multus\") pod \"multus-5rpmm\" (UID: \"9ed6dfe0-8780-4a3f-bbeb-9a012722f6e6\") " pod="openshift-multus/multus-5rpmm"
Sep 09 03:13:06.347468 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:13:06.347038    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/f0fa1f03-0dd2-4015-a21a-02fc04073778-log-socket\") pod \"ovnkube-node-9t27t\" (UID: \"f0fa1f03-0dd2-4015-a21a-02fc04073778\") " pod="openshift-ovn-kubernetes/ovnkube-node-9t27t"
Sep 09 03:13:06.347468 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:13:06.347062    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/aaf8534a-72e4-4c88-972e-26eacf111fa2-cnibin\") pod \"multus-additional-cni-plugins-qpxhx\" (UID: \"aaf8534a-72e4-4c88-972e-26eacf111fa2\") " pod="openshift-multus/multus-additional-cni-plugins-qpxhx"
Sep 09 03:13:06.347468 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:13:06.347063    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/9ed6dfe0-8780-4a3f-bbeb-9a012722f6e6-host-var-lib-cni-multus\") pod \"multus-5rpmm\" (UID: \"9ed6dfe0-8780-4a3f-bbeb-9a012722f6e6\") " pod="openshift-multus/multus-5rpmm"
Sep 09 03:13:06.347468 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:13:06.347085    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/443a2987-5246-4a8c-8631-1b751e02101b-metrics-certs\") pod \"network-metrics-daemon-99qzh\" (UID: \"443a2987-5246-4a8c-8631-1b751e02101b\") " pod="openshift-multus/network-metrics-daemon-99qzh"
Sep 09 03:13:06.347468 ip-10-0-132-14 kubenswrapper[2576]: E0909 03:13:06.347087    2576 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 09 03:13:06.347468 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:13:06.347119    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/f0fa1f03-0dd2-4015-a21a-02fc04073778-etc-openvswitch\") pod \"ovnkube-node-9t27t\" (UID: \"f0fa1f03-0dd2-4015-a21a-02fc04073778\") " pod="openshift-ovn-kubernetes/ovnkube-node-9t27t"
Sep 09 03:13:06.347468 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:13:06.347124    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/aaf8534a-72e4-4c88-972e-26eacf111fa2-cnibin\") pod \"multus-additional-cni-plugins-qpxhx\" (UID: \"aaf8534a-72e4-4c88-972e-26eacf111fa2\") " pod="openshift-multus/multus-additional-cni-plugins-qpxhx"
Sep 09 03:13:06.347468 ip-10-0-132-14 kubenswrapper[2576]: E0909 03:13:06.347153    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2f43cc28-7345-4c06-9160-4245b99697b4-original-pull-secret podName:2f43cc28-7345-4c06-9160-4245b99697b4 nodeName:}" failed. No retries permitted until 2026-09-09 03:13:06.847133215 +0000 UTC m=+2.126793085 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/2f43cc28-7345-4c06-9160-4245b99697b4-original-pull-secret") pod "global-pull-secret-syncer-t9bbm" (UID: "2f43cc28-7345-4c06-9160-4245b99697b4") : object "kube-system"/"original-pull-secret" not registered
Sep 09 03:13:06.348393 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:13:06.347163    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/f0fa1f03-0dd2-4015-a21a-02fc04073778-etc-openvswitch\") pod \"ovnkube-node-9t27t\" (UID: \"f0fa1f03-0dd2-4015-a21a-02fc04073778\") " pod="openshift-ovn-kubernetes/ovnkube-node-9t27t"
Sep 09 03:13:06.348393 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:13:06.347168    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/f0fa1f03-0dd2-4015-a21a-02fc04073778-log-socket\") pod \"ovnkube-node-9t27t\" (UID: \"f0fa1f03-0dd2-4015-a21a-02fc04073778\") " pod="openshift-ovn-kubernetes/ovnkube-node-9t27t"
Sep 09 03:13:06.348393 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:13:06.347170    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/f0fa1f03-0dd2-4015-a21a-02fc04073778-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-9t27t\" (UID: \"f0fa1f03-0dd2-4015-a21a-02fc04073778\") " pod="openshift-ovn-kubernetes/ovnkube-node-9t27t"
Sep 09 03:13:06.348393 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:13:06.347170    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/aaf8534a-72e4-4c88-972e-26eacf111fa2-tuning-conf-dir\") pod \"multus-additional-cni-plugins-qpxhx\" (UID: \"aaf8534a-72e4-4c88-972e-26eacf111fa2\") " pod="openshift-multus/multus-additional-cni-plugins-qpxhx"
Sep 09 03:13:06.348393 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:13:06.347190    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/9ed6dfe0-8780-4a3f-bbeb-9a012722f6e6-multus-daemon-config\") pod \"multus-5rpmm\" (UID: \"9ed6dfe0-8780-4a3f-bbeb-9a012722f6e6\") " pod="openshift-multus/multus-5rpmm"
Sep 09 03:13:06.348393 ip-10-0-132-14 kubenswrapper[2576]: E0909 03:13:06.347198    2576 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 09 03:13:06.348393 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:13:06.347203    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/f0fa1f03-0dd2-4015-a21a-02fc04073778-ovn-node-metrics-cert\") pod \"ovnkube-node-9t27t\" (UID: \"f0fa1f03-0dd2-4015-a21a-02fc04073778\") " pod="openshift-ovn-kubernetes/ovnkube-node-9t27t"
Sep 09 03:13:06.348393 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:13:06.347206    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/f0fa1f03-0dd2-4015-a21a-02fc04073778-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-9t27t\" (UID: \"f0fa1f03-0dd2-4015-a21a-02fc04073778\") " pod="openshift-ovn-kubernetes/ovnkube-node-9t27t"
Sep 09 03:13:06.348393 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:13:06.347230    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/61140997-9a63-4a79-ae4d-368d92a92960-sys-fs\") pod \"aws-ebs-csi-driver-node-rhxhj\" (UID: \"61140997-9a63-4a79-ae4d-368d92a92960\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rhxhj"
Sep 09 03:13:06.348393 ip-10-0-132-14 kubenswrapper[2576]: E0909 03:13:06.347244    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/443a2987-5246-4a8c-8631-1b751e02101b-metrics-certs podName:443a2987-5246-4a8c-8631-1b751e02101b nodeName:}" failed. No retries permitted until 2026-09-09 03:13:06.847230765 +0000 UTC m=+2.126890657 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/443a2987-5246-4a8c-8631-1b751e02101b-metrics-certs") pod "network-metrics-daemon-99qzh" (UID: "443a2987-5246-4a8c-8631-1b751e02101b") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 09 03:13:06.348393 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:13:06.347270    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-72wt9\" (UniqueName: \"kubernetes.io/projected/443a2987-5246-4a8c-8631-1b751e02101b-kube-api-access-72wt9\") pod \"network-metrics-daemon-99qzh\" (UID: \"443a2987-5246-4a8c-8631-1b751e02101b\") " pod="openshift-multus/network-metrics-daemon-99qzh"
Sep 09 03:13:06.348393 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:13:06.347292    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/849f0c91-81ec-458a-a73d-c1922a7f5bc5-lib-modules\") pod \"tuned-4b7pm\" (UID: \"849f0c91-81ec-458a-a73d-c1922a7f5bc5\") " pod="openshift-cluster-node-tuning-operator/tuned-4b7pm"
Sep 09 03:13:06.348393 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:13:06.347302    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/61140997-9a63-4a79-ae4d-368d92a92960-sys-fs\") pod \"aws-ebs-csi-driver-node-rhxhj\" (UID: \"61140997-9a63-4a79-ae4d-368d92a92960\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rhxhj"
Sep 09 03:13:06.348393 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:13:06.347324    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/f0fa1f03-0dd2-4015-a21a-02fc04073778-systemd-units\") pod \"ovnkube-node-9t27t\" (UID: \"f0fa1f03-0dd2-4015-a21a-02fc04073778\") " pod="openshift-ovn-kubernetes/ovnkube-node-9t27t"
Sep 09 03:13:06.348393 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:13:06.347325    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/f0fa1f03-0dd2-4015-a21a-02fc04073778-ovnkube-script-lib\") pod \"ovnkube-node-9t27t\" (UID: \"f0fa1f03-0dd2-4015-a21a-02fc04073778\") " pod="openshift-ovn-kubernetes/ovnkube-node-9t27t"
Sep 09 03:13:06.348393 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:13:06.347346    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/f0fa1f03-0dd2-4015-a21a-02fc04073778-host-cni-netd\") pod \"ovnkube-node-9t27t\" (UID: \"f0fa1f03-0dd2-4015-a21a-02fc04073778\") " pod="openshift-ovn-kubernetes/ovnkube-node-9t27t"
Sep 09 03:13:06.348393 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:13:06.347371    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/aaf8534a-72e4-4c88-972e-26eacf111fa2-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-qpxhx\" (UID: \"aaf8534a-72e4-4c88-972e-26eacf111fa2\") " pod="openshift-multus/multus-additional-cni-plugins-qpxhx"
Sep 09 03:13:06.350195 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:13:06.347395    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-59qk7\" (UniqueName: \"kubernetes.io/projected/849f0c91-81ec-458a-a73d-c1922a7f5bc5-kube-api-access-59qk7\") pod \"tuned-4b7pm\" (UID: \"849f0c91-81ec-458a-a73d-c1922a7f5bc5\") " pod="openshift-cluster-node-tuning-operator/tuned-4b7pm"
Sep 09 03:13:06.350195 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:13:06.347397    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/f0fa1f03-0dd2-4015-a21a-02fc04073778-host-cni-netd\") pod \"ovnkube-node-9t27t\" (UID: \"f0fa1f03-0dd2-4015-a21a-02fc04073778\") " pod="openshift-ovn-kubernetes/ovnkube-node-9t27t"
Sep 09 03:13:06.350195 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:13:06.347409    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/9ed6dfe0-8780-4a3f-bbeb-9a012722f6e6-etc-kubernetes\") pod \"multus-5rpmm\" (UID: \"9ed6dfe0-8780-4a3f-bbeb-9a012722f6e6\") " pod="openshift-multus/multus-5rpmm"
Sep 09 03:13:06.350195 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:13:06.347425    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/f0fa1f03-0dd2-4015-a21a-02fc04073778-host-run-ovn-kubernetes\") pod \"ovnkube-node-9t27t\" (UID: \"f0fa1f03-0dd2-4015-a21a-02fc04073778\") " pod="openshift-ovn-kubernetes/ovnkube-node-9t27t"
Sep 09 03:13:06.350195 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:13:06.347449    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5x42s\" (UniqueName: \"kubernetes.io/projected/f0fa1f03-0dd2-4015-a21a-02fc04073778-kube-api-access-5x42s\") pod \"ovnkube-node-9t27t\" (UID: \"f0fa1f03-0dd2-4015-a21a-02fc04073778\") " pod="openshift-ovn-kubernetes/ovnkube-node-9t27t"
Sep 09 03:13:06.350195 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:13:06.347469    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/9ed6dfe0-8780-4a3f-bbeb-9a012722f6e6-etc-kubernetes\") pod \"multus-5rpmm\" (UID: \"9ed6dfe0-8780-4a3f-bbeb-9a012722f6e6\") " pod="openshift-multus/multus-5rpmm"
Sep 09 03:13:06.350195 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:13:06.347473    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qtf7c\" (UniqueName: \"kubernetes.io/projected/61140997-9a63-4a79-ae4d-368d92a92960-kube-api-access-qtf7c\") pod \"aws-ebs-csi-driver-node-rhxhj\" (UID: \"61140997-9a63-4a79-ae4d-368d92a92960\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rhxhj"
Sep 09 03:13:06.350195 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:13:06.347501    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/849f0c91-81ec-458a-a73d-c1922a7f5bc5-etc-modprobe-d\") pod \"tuned-4b7pm\" (UID: \"849f0c91-81ec-458a-a73d-c1922a7f5bc5\") " pod="openshift-cluster-node-tuning-operator/tuned-4b7pm"
Sep 09 03:13:06.350195 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:13:06.347518    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/849f0c91-81ec-458a-a73d-c1922a7f5bc5-lib-modules\") pod \"tuned-4b7pm\" (UID: \"849f0c91-81ec-458a-a73d-c1922a7f5bc5\") " pod="openshift-cluster-node-tuning-operator/tuned-4b7pm"
Sep 09 03:13:06.350195 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:13:06.347525    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/849f0c91-81ec-458a-a73d-c1922a7f5bc5-run\") pod \"tuned-4b7pm\" (UID: \"849f0c91-81ec-458a-a73d-c1922a7f5bc5\") " pod="openshift-cluster-node-tuning-operator/tuned-4b7pm"
Sep 09 03:13:06.350195 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:13:06.347555    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/9ed6dfe0-8780-4a3f-bbeb-9a012722f6e6-hostroot\") pod \"multus-5rpmm\" (UID: \"9ed6dfe0-8780-4a3f-bbeb-9a012722f6e6\") " pod="openshift-multus/multus-5rpmm"
Sep 09 03:13:06.350195 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:13:06.347572    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/849f0c91-81ec-458a-a73d-c1922a7f5bc5-sys\") pod \"tuned-4b7pm\" (UID: \"849f0c91-81ec-458a-a73d-c1922a7f5bc5\") " pod="openshift-cluster-node-tuning-operator/tuned-4b7pm"
Sep 09 03:13:06.350195 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:13:06.347591    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/9ed6dfe0-8780-4a3f-bbeb-9a012722f6e6-multus-socket-dir-parent\") pod \"multus-5rpmm\" (UID: \"9ed6dfe0-8780-4a3f-bbeb-9a012722f6e6\") " pod="openshift-multus/multus-5rpmm"
Sep 09 03:13:06.350195 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:13:06.347609    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/849f0c91-81ec-458a-a73d-c1922a7f5bc5-run\") pod \"tuned-4b7pm\" (UID: \"849f0c91-81ec-458a-a73d-c1922a7f5bc5\") " pod="openshift-cluster-node-tuning-operator/tuned-4b7pm"
Sep 09 03:13:06.350195 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:13:06.347621    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/f0fa1f03-0dd2-4015-a21a-02fc04073778-systemd-units\") pod \"ovnkube-node-9t27t\" (UID: \"f0fa1f03-0dd2-4015-a21a-02fc04073778\") " pod="openshift-ovn-kubernetes/ovnkube-node-9t27t"
Sep 09 03:13:06.350195 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:13:06.347622    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/f0fa1f03-0dd2-4015-a21a-02fc04073778-host-run-ovn-kubernetes\") pod \"ovnkube-node-9t27t\" (UID: \"f0fa1f03-0dd2-4015-a21a-02fc04073778\") " pod="openshift-ovn-kubernetes/ovnkube-node-9t27t"
Sep 09 03:13:06.350195 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:13:06.347623    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/9ed6dfe0-8780-4a3f-bbeb-9a012722f6e6-host-var-lib-cni-bin\") pod \"multus-5rpmm\" (UID: \"9ed6dfe0-8780-4a3f-bbeb-9a012722f6e6\") " pod="openshift-multus/multus-5rpmm"
Sep 09 03:13:06.350195 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:13:06.347681    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/9ed6dfe0-8780-4a3f-bbeb-9a012722f6e6-host-var-lib-cni-bin\") pod \"multus-5rpmm\" (UID: \"9ed6dfe0-8780-4a3f-bbeb-9a012722f6e6\") " pod="openshift-multus/multus-5rpmm"
Sep 09 03:13:06.351052 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:13:06.347701    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/f0fa1f03-0dd2-4015-a21a-02fc04073778-env-overrides\") pod \"ovnkube-node-9t27t\" (UID: \"f0fa1f03-0dd2-4015-a21a-02fc04073778\") " pod="openshift-ovn-kubernetes/ovnkube-node-9t27t"
Sep 09 03:13:06.351052 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:13:06.347722    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/849f0c91-81ec-458a-a73d-c1922a7f5bc5-sys\") pod \"tuned-4b7pm\" (UID: \"849f0c91-81ec-458a-a73d-c1922a7f5bc5\") " pod="openshift-cluster-node-tuning-operator/tuned-4b7pm"
Sep 09 03:13:06.351052 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:13:06.347735    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/9ed6dfe0-8780-4a3f-bbeb-9a012722f6e6-multus-socket-dir-parent\") pod \"multus-5rpmm\" (UID: \"9ed6dfe0-8780-4a3f-bbeb-9a012722f6e6\") " pod="openshift-multus/multus-5rpmm"
Sep 09 03:13:06.351052 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:13:06.347724    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/849f0c91-81ec-458a-a73d-c1922a7f5bc5-etc-modprobe-d\") pod \"tuned-4b7pm\" (UID: \"849f0c91-81ec-458a-a73d-c1922a7f5bc5\") " pod="openshift-cluster-node-tuning-operator/tuned-4b7pm"
Sep 09 03:13:06.351052 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:13:06.347727    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/9ed6dfe0-8780-4a3f-bbeb-9a012722f6e6-cnibin\") pod \"multus-5rpmm\" (UID: \"9ed6dfe0-8780-4a3f-bbeb-9a012722f6e6\") " pod="openshift-multus/multus-5rpmm"
Sep 09 03:13:06.351052 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:13:06.347773    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/9ed6dfe0-8780-4a3f-bbeb-9a012722f6e6-cni-binary-copy\") pod \"multus-5rpmm\" (UID: \"9ed6dfe0-8780-4a3f-bbeb-9a012722f6e6\") " pod="openshift-multus/multus-5rpmm"
Sep 09 03:13:06.351052 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:13:06.347780    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/9ed6dfe0-8780-4a3f-bbeb-9a012722f6e6-hostroot\") pod \"multus-5rpmm\" (UID: \"9ed6dfe0-8780-4a3f-bbeb-9a012722f6e6\") " pod="openshift-multus/multus-5rpmm"
Sep 09 03:13:06.351052 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:13:06.347768    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/9ed6dfe0-8780-4a3f-bbeb-9a012722f6e6-cnibin\") pod \"multus-5rpmm\" (UID: \"9ed6dfe0-8780-4a3f-bbeb-9a012722f6e6\") " pod="openshift-multus/multus-5rpmm"
Sep 09 03:13:06.351052 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:13:06.347795    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/9ed6dfe0-8780-4a3f-bbeb-9a012722f6e6-system-cni-dir\") pod \"multus-5rpmm\" (UID: \"9ed6dfe0-8780-4a3f-bbeb-9a012722f6e6\") " pod="openshift-multus/multus-5rpmm"
Sep 09 03:13:06.351052 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:13:06.347816    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/f0fa1f03-0dd2-4015-a21a-02fc04073778-run-systemd\") pod \"ovnkube-node-9t27t\" (UID: \"f0fa1f03-0dd2-4015-a21a-02fc04073778\") " pod="openshift-ovn-kubernetes/ovnkube-node-9t27t"
Sep 09 03:13:06.351052 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:13:06.347837    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/f0fa1f03-0dd2-4015-a21a-02fc04073778-run-openvswitch\") pod \"ovnkube-node-9t27t\" (UID: \"f0fa1f03-0dd2-4015-a21a-02fc04073778\") " pod="openshift-ovn-kubernetes/ovnkube-node-9t27t"
Sep 09 03:13:06.351052 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:13:06.347860    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/f0fa1f03-0dd2-4015-a21a-02fc04073778-ovnkube-config\") pod \"ovnkube-node-9t27t\" (UID: \"f0fa1f03-0dd2-4015-a21a-02fc04073778\") " pod="openshift-ovn-kubernetes/ovnkube-node-9t27t"
Sep 09 03:13:06.351052 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:13:06.347864    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/f0fa1f03-0dd2-4015-a21a-02fc04073778-run-systemd\") pod \"ovnkube-node-9t27t\" (UID: \"f0fa1f03-0dd2-4015-a21a-02fc04073778\") " pod="openshift-ovn-kubernetes/ovnkube-node-9t27t"
Sep 09 03:13:06.351052 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:13:06.347881    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/849f0c91-81ec-458a-a73d-c1922a7f5bc5-etc-systemd\") pod \"tuned-4b7pm\" (UID: \"849f0c91-81ec-458a-a73d-c1922a7f5bc5\") " pod="openshift-cluster-node-tuning-operator/tuned-4b7pm"
Sep 09 03:13:06.351052 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:13:06.347904    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/2f43cc28-7345-4c06-9160-4245b99697b4-dbus\") pod \"global-pull-secret-syncer-t9bbm\" (UID: \"2f43cc28-7345-4c06-9160-4245b99697b4\") " pod="kube-system/global-pull-secret-syncer-t9bbm"
Sep 09 03:13:06.351052 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:13:06.347913    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/f0fa1f03-0dd2-4015-a21a-02fc04073778-run-openvswitch\") pod \"ovnkube-node-9t27t\" (UID: \"f0fa1f03-0dd2-4015-a21a-02fc04073778\") " pod="openshift-ovn-kubernetes/ovnkube-node-9t27t"
Sep 09 03:13:06.351052 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:13:06.347927    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/472d1b86-78f9-45a8-a782-f4274b90a158-konnectivity-ca\") pod \"konnectivity-agent-c4rmf\" (UID: \"472d1b86-78f9-45a8-a782-f4274b90a158\") " pod="kube-system/konnectivity-agent-c4rmf"
Sep 09 03:13:06.351052 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:13:06.347951    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/aaf8534a-72e4-4c88-972e-26eacf111fa2-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-qpxhx\" (UID: \"aaf8534a-72e4-4c88-972e-26eacf111fa2\") " pod="openshift-multus/multus-additional-cni-plugins-qpxhx"
Sep 09 03:13:06.351549 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:13:06.347952    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/f0fa1f03-0dd2-4015-a21a-02fc04073778-host-kubelet\") pod \"ovnkube-node-9t27t\" (UID: \"f0fa1f03-0dd2-4015-a21a-02fc04073778\") " pod="openshift-ovn-kubernetes/ovnkube-node-9t27t"
Sep 09 03:13:06.351549 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:13:06.347995    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/f0fa1f03-0dd2-4015-a21a-02fc04073778-run-ovn\") pod \"ovnkube-node-9t27t\" (UID: \"f0fa1f03-0dd2-4015-a21a-02fc04073778\") " pod="openshift-ovn-kubernetes/ovnkube-node-9t27t"
Sep 09 03:13:06.351549 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:13:06.347968    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/849f0c91-81ec-458a-a73d-c1922a7f5bc5-etc-systemd\") pod \"tuned-4b7pm\" (UID: \"849f0c91-81ec-458a-a73d-c1922a7f5bc5\") " pod="openshift-cluster-node-tuning-operator/tuned-4b7pm"
Sep 09 03:13:06.351549 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:13:06.348037    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/f0fa1f03-0dd2-4015-a21a-02fc04073778-host-kubelet\") pod \"ovnkube-node-9t27t\" (UID: \"f0fa1f03-0dd2-4015-a21a-02fc04073778\") " pod="openshift-ovn-kubernetes/ovnkube-node-9t27t"
Sep 09 03:13:06.351549 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:13:06.348046    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/2f43cc28-7345-4c06-9160-4245b99697b4-dbus\") pod \"global-pull-secret-syncer-t9bbm\" (UID: \"2f43cc28-7345-4c06-9160-4245b99697b4\") " pod="kube-system/global-pull-secret-syncer-t9bbm"
Sep 09 03:13:06.351549 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:13:06.348067    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/61140997-9a63-4a79-ae4d-368d92a92960-registration-dir\") pod \"aws-ebs-csi-driver-node-rhxhj\" (UID: \"61140997-9a63-4a79-ae4d-368d92a92960\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rhxhj"
Sep 09 03:13:06.351549 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:13:06.347901    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/9ed6dfe0-8780-4a3f-bbeb-9a012722f6e6-system-cni-dir\") pod \"multus-5rpmm\" (UID: \"9ed6dfe0-8780-4a3f-bbeb-9a012722f6e6\") " pod="openshift-multus/multus-5rpmm"
Sep 09 03:13:06.351549 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:13:06.348076    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/f0fa1f03-0dd2-4015-a21a-02fc04073778-run-ovn\") pod \"ovnkube-node-9t27t\" (UID: \"f0fa1f03-0dd2-4015-a21a-02fc04073778\") " pod="openshift-ovn-kubernetes/ovnkube-node-9t27t"
Sep 09 03:13:06.351549 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:13:06.348109    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/61140997-9a63-4a79-ae4d-368d92a92960-device-dir\") pod \"aws-ebs-csi-driver-node-rhxhj\" (UID: \"61140997-9a63-4a79-ae4d-368d92a92960\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rhxhj"
Sep 09 03:13:06.351549 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:13:06.348127    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/849f0c91-81ec-458a-a73d-c1922a7f5bc5-etc-sysconfig\") pod \"tuned-4b7pm\" (UID: \"849f0c91-81ec-458a-a73d-c1922a7f5bc5\") " pod="openshift-cluster-node-tuning-operator/tuned-4b7pm"
Sep 09 03:13:06.351549 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:13:06.348149    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/334f1652-d04c-4382-bbf3-367d96ba0b5c-host\") pod \"node-ca-2zp8k\" (UID: \"334f1652-d04c-4382-bbf3-367d96ba0b5c\") " pod="openshift-image-registry/node-ca-2zp8k"
Sep 09 03:13:06.351549 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:13:06.348159    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/f0fa1f03-0dd2-4015-a21a-02fc04073778-env-overrides\") pod \"ovnkube-node-9t27t\" (UID: \"f0fa1f03-0dd2-4015-a21a-02fc04073778\") " pod="openshift-ovn-kubernetes/ovnkube-node-9t27t"
Sep 09 03:13:06.351549 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:13:06.348173    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/f0fa1f03-0dd2-4015-a21a-02fc04073778-node-log\") pod \"ovnkube-node-9t27t\" (UID: \"f0fa1f03-0dd2-4015-a21a-02fc04073778\") " pod="openshift-ovn-kubernetes/ovnkube-node-9t27t"
Sep 09 03:13:06.351549 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:13:06.348228    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/f0fa1f03-0dd2-4015-a21a-02fc04073778-node-log\") pod \"ovnkube-node-9t27t\" (UID: \"f0fa1f03-0dd2-4015-a21a-02fc04073778\") " pod="openshift-ovn-kubernetes/ovnkube-node-9t27t"
Sep 09 03:13:06.351549 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:13:06.348237    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/334f1652-d04c-4382-bbf3-367d96ba0b5c-host\") pod \"node-ca-2zp8k\" (UID: \"334f1652-d04c-4382-bbf3-367d96ba0b5c\") " pod="openshift-image-registry/node-ca-2zp8k"
Sep 09 03:13:06.351549 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:13:06.348263    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/aaf8534a-72e4-4c88-972e-26eacf111fa2-system-cni-dir\") pod \"multus-additional-cni-plugins-qpxhx\" (UID: \"aaf8534a-72e4-4c88-972e-26eacf111fa2\") " pod="openshift-multus/multus-additional-cni-plugins-qpxhx"
Sep 09 03:13:06.351549 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:13:06.348272    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/61140997-9a63-4a79-ae4d-368d92a92960-registration-dir\") pod \"aws-ebs-csi-driver-node-rhxhj\" (UID: \"61140997-9a63-4a79-ae4d-368d92a92960\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rhxhj"
Sep 09 03:13:06.351549 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:13:06.348326    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/9ed6dfe0-8780-4a3f-bbeb-9a012722f6e6-cni-binary-copy\") pod \"multus-5rpmm\" (UID: \"9ed6dfe0-8780-4a3f-bbeb-9a012722f6e6\") " pod="openshift-multus/multus-5rpmm"
Sep 09 03:13:06.352080 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:13:06.348294    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/849f0c91-81ec-458a-a73d-c1922a7f5bc5-etc-sysconfig\") pod \"tuned-4b7pm\" (UID: \"849f0c91-81ec-458a-a73d-c1922a7f5bc5\") " pod="openshift-cluster-node-tuning-operator/tuned-4b7pm"
Sep 09 03:13:06.352080 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:13:06.348378    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/61140997-9a63-4a79-ae4d-368d92a92960-socket-dir\") pod \"aws-ebs-csi-driver-node-rhxhj\" (UID: \"61140997-9a63-4a79-ae4d-368d92a92960\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rhxhj"
Sep 09 03:13:06.352080 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:13:06.348400    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/aaf8534a-72e4-4c88-972e-26eacf111fa2-system-cni-dir\") pod \"multus-additional-cni-plugins-qpxhx\" (UID: \"aaf8534a-72e4-4c88-972e-26eacf111fa2\") " pod="openshift-multus/multus-additional-cni-plugins-qpxhx"
Sep 09 03:13:06.352080 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:13:06.348442    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/849f0c91-81ec-458a-a73d-c1922a7f5bc5-tmp\") pod \"tuned-4b7pm\" (UID: \"849f0c91-81ec-458a-a73d-c1922a7f5bc5\") " pod="openshift-cluster-node-tuning-operator/tuned-4b7pm"
Sep 09 03:13:06.352080 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:13:06.348457    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/f0fa1f03-0dd2-4015-a21a-02fc04073778-ovnkube-config\") pod \"ovnkube-node-9t27t\" (UID: \"f0fa1f03-0dd2-4015-a21a-02fc04073778\") " pod="openshift-ovn-kubernetes/ovnkube-node-9t27t"
Sep 09 03:13:06.352080 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:13:06.348494    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/9ed6dfe0-8780-4a3f-bbeb-9a012722f6e6-host-run-k8s-cni-cncf-io\") pod \"multus-5rpmm\" (UID: \"9ed6dfe0-8780-4a3f-bbeb-9a012722f6e6\") " pod="openshift-multus/multus-5rpmm"
Sep 09 03:13:06.352080 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:13:06.348520    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-r79cr\" (UniqueName: \"kubernetes.io/projected/9ed6dfe0-8780-4a3f-bbeb-9a012722f6e6-kube-api-access-r79cr\") pod \"multus-5rpmm\" (UID: \"9ed6dfe0-8780-4a3f-bbeb-9a012722f6e6\") " pod="openshift-multus/multus-5rpmm"
Sep 09 03:13:06.352080 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:13:06.348536    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/61140997-9a63-4a79-ae4d-368d92a92960-socket-dir\") pod \"aws-ebs-csi-driver-node-rhxhj\" (UID: \"61140997-9a63-4a79-ae4d-368d92a92960\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rhxhj"
Sep 09 03:13:06.352080 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:13:06.348539    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/472d1b86-78f9-45a8-a782-f4274b90a158-konnectivity-ca\") pod \"konnectivity-agent-c4rmf\" (UID: \"472d1b86-78f9-45a8-a782-f4274b90a158\") " pod="kube-system/konnectivity-agent-c4rmf"
Sep 09 03:13:06.352080 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:13:06.348539    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/2f43cc28-7345-4c06-9160-4245b99697b4-kubelet-config\") pod \"global-pull-secret-syncer-t9bbm\" (UID: \"2f43cc28-7345-4c06-9160-4245b99697b4\") " pod="kube-system/global-pull-secret-syncer-t9bbm"
Sep 09 03:13:06.352080 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:13:06.348572    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-25gnh\" (UniqueName: \"kubernetes.io/projected/aaf8534a-72e4-4c88-972e-26eacf111fa2-kube-api-access-25gnh\") pod \"multus-additional-cni-plugins-qpxhx\" (UID: \"aaf8534a-72e4-4c88-972e-26eacf111fa2\") " pod="openshift-multus/multus-additional-cni-plugins-qpxhx"
Sep 09 03:13:06.352080 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:13:06.348580    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/2f43cc28-7345-4c06-9160-4245b99697b4-kubelet-config\") pod \"global-pull-secret-syncer-t9bbm\" (UID: \"2f43cc28-7345-4c06-9160-4245b99697b4\") " pod="kube-system/global-pull-secret-syncer-t9bbm"
Sep 09 03:13:06.352080 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:13:06.348599    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/849f0c91-81ec-458a-a73d-c1922a7f5bc5-etc-sysctl-conf\") pod \"tuned-4b7pm\" (UID: \"849f0c91-81ec-458a-a73d-c1922a7f5bc5\") " pod="openshift-cluster-node-tuning-operator/tuned-4b7pm"
Sep 09 03:13:06.352080 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:13:06.348619    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/9ed6dfe0-8780-4a3f-bbeb-9a012722f6e6-host-run-k8s-cni-cncf-io\") pod \"multus-5rpmm\" (UID: \"9ed6dfe0-8780-4a3f-bbeb-9a012722f6e6\") " pod="openshift-multus/multus-5rpmm"
Sep 09 03:13:06.352080 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:13:06.348654    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/849f0c91-81ec-458a-a73d-c1922a7f5bc5-etc-tuned\") pod \"tuned-4b7pm\" (UID: \"849f0c91-81ec-458a-a73d-c1922a7f5bc5\") " pod="openshift-cluster-node-tuning-operator/tuned-4b7pm"
Sep 09 03:13:06.352080 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:13:06.348688    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/9ed6dfe0-8780-4a3f-bbeb-9a012722f6e6-multus-conf-dir\") pod \"multus-5rpmm\" (UID: \"9ed6dfe0-8780-4a3f-bbeb-9a012722f6e6\") " pod="openshift-multus/multus-5rpmm"
Sep 09 03:13:06.352080 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:13:06.348712    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/472d1b86-78f9-45a8-a782-f4274b90a158-agent-certs\") pod \"konnectivity-agent-c4rmf\" (UID: \"472d1b86-78f9-45a8-a782-f4274b90a158\") " pod="kube-system/konnectivity-agent-c4rmf"
Sep 09 03:13:06.352656 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:13:06.348737    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/334f1652-d04c-4382-bbf3-367d96ba0b5c-serviceca\") pod \"node-ca-2zp8k\" (UID: \"334f1652-d04c-4382-bbf3-367d96ba0b5c\") " pod="openshift-image-registry/node-ca-2zp8k"
Sep 09 03:13:06.352656 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:13:06.348757    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/849f0c91-81ec-458a-a73d-c1922a7f5bc5-etc-sysctl-conf\") pod \"tuned-4b7pm\" (UID: \"849f0c91-81ec-458a-a73d-c1922a7f5bc5\") " pod="openshift-cluster-node-tuning-operator/tuned-4b7pm"
Sep 09 03:13:06.352656 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:13:06.348766    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/aaf8534a-72e4-4c88-972e-26eacf111fa2-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-qpxhx\" (UID: \"aaf8534a-72e4-4c88-972e-26eacf111fa2\") " pod="openshift-multus/multus-additional-cni-plugins-qpxhx"
Sep 09 03:13:06.352656 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:13:06.348793    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/61140997-9a63-4a79-ae4d-368d92a92960-etc-selinux\") pod \"aws-ebs-csi-driver-node-rhxhj\" (UID: \"61140997-9a63-4a79-ae4d-368d92a92960\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rhxhj"
Sep 09 03:13:06.352656 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:13:06.348797    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/9ed6dfe0-8780-4a3f-bbeb-9a012722f6e6-multus-conf-dir\") pod \"multus-5rpmm\" (UID: \"9ed6dfe0-8780-4a3f-bbeb-9a012722f6e6\") " pod="openshift-multus/multus-5rpmm"
Sep 09 03:13:06.352656 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:13:06.348816    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/849f0c91-81ec-458a-a73d-c1922a7f5bc5-etc-kubernetes\") pod \"tuned-4b7pm\" (UID: \"849f0c91-81ec-458a-a73d-c1922a7f5bc5\") " pod="openshift-cluster-node-tuning-operator/tuned-4b7pm"
Sep 09 03:13:06.352656 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:13:06.348842    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c9e03e08-c803-4ca3-8f43-aa3390269c0a-tmp-dir\") pod \"node-resolver-cqhff\" (UID: \"c9e03e08-c803-4ca3-8f43-aa3390269c0a\") " pod="openshift-dns/node-resolver-cqhff"
Sep 09 03:13:06.352656 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:13:06.348866    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bjld7\" (UniqueName: \"kubernetes.io/projected/c9e03e08-c803-4ca3-8f43-aa3390269c0a-kube-api-access-bjld7\") pod \"node-resolver-cqhff\" (UID: \"c9e03e08-c803-4ca3-8f43-aa3390269c0a\") " pod="openshift-dns/node-resolver-cqhff"
Sep 09 03:13:06.352656 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:13:06.348889    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-t6jd8\" (UniqueName: \"kubernetes.io/projected/334f1652-d04c-4382-bbf3-367d96ba0b5c-kube-api-access-t6jd8\") pod \"node-ca-2zp8k\" (UID: \"334f1652-d04c-4382-bbf3-367d96ba0b5c\") " pod="openshift-image-registry/node-ca-2zp8k"
Sep 09 03:13:06.352656 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:13:06.348917    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/f0fa1f03-0dd2-4015-a21a-02fc04073778-host-cni-bin\") pod \"ovnkube-node-9t27t\" (UID: \"f0fa1f03-0dd2-4015-a21a-02fc04073778\") " pod="openshift-ovn-kubernetes/ovnkube-node-9t27t"
Sep 09 03:13:06.352656 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:13:06.348943    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/aaf8534a-72e4-4c88-972e-26eacf111fa2-cni-binary-copy\") pod \"multus-additional-cni-plugins-qpxhx\" (UID: \"aaf8534a-72e4-4c88-972e-26eacf111fa2\") " pod="openshift-multus/multus-additional-cni-plugins-qpxhx"
Sep 09 03:13:06.352656 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:13:06.348969    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/61140997-9a63-4a79-ae4d-368d92a92960-kubelet-dir\") pod \"aws-ebs-csi-driver-node-rhxhj\" (UID: \"61140997-9a63-4a79-ae4d-368d92a92960\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rhxhj"
Sep 09 03:13:06.352656 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:13:06.348991    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/9ed6dfe0-8780-4a3f-bbeb-9a012722f6e6-multus-cni-dir\") pod \"multus-5rpmm\" (UID: \"9ed6dfe0-8780-4a3f-bbeb-9a012722f6e6\") " pod="openshift-multus/multus-5rpmm"
Sep 09 03:13:06.352656 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:13:06.349075    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/9ed6dfe0-8780-4a3f-bbeb-9a012722f6e6-multus-cni-dir\") pod \"multus-5rpmm\" (UID: \"9ed6dfe0-8780-4a3f-bbeb-9a012722f6e6\") " pod="openshift-multus/multus-5rpmm"
Sep 09 03:13:06.352656 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:13:06.349161    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/61140997-9a63-4a79-ae4d-368d92a92960-etc-selinux\") pod \"aws-ebs-csi-driver-node-rhxhj\" (UID: \"61140997-9a63-4a79-ae4d-368d92a92960\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rhxhj"
Sep 09 03:13:06.352656 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:13:06.349209    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/61140997-9a63-4a79-ae4d-368d92a92960-kubelet-dir\") pod \"aws-ebs-csi-driver-node-rhxhj\" (UID: \"61140997-9a63-4a79-ae4d-368d92a92960\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rhxhj"
Sep 09 03:13:06.352656 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:13:06.349248    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/f0fa1f03-0dd2-4015-a21a-02fc04073778-host-cni-bin\") pod \"ovnkube-node-9t27t\" (UID: \"f0fa1f03-0dd2-4015-a21a-02fc04073778\") " pod="openshift-ovn-kubernetes/ovnkube-node-9t27t"
Sep 09 03:13:06.353134 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:13:06.349353    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/849f0c91-81ec-458a-a73d-c1922a7f5bc5-etc-kubernetes\") pod \"tuned-4b7pm\" (UID: \"849f0c91-81ec-458a-a73d-c1922a7f5bc5\") " pod="openshift-cluster-node-tuning-operator/tuned-4b7pm"
Sep 09 03:13:06.353134 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:13:06.349503    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c9e03e08-c803-4ca3-8f43-aa3390269c0a-tmp-dir\") pod \"node-resolver-cqhff\" (UID: \"c9e03e08-c803-4ca3-8f43-aa3390269c0a\") " pod="openshift-dns/node-resolver-cqhff"
Sep 09 03:13:06.353134 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:13:06.349511    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/aaf8534a-72e4-4c88-972e-26eacf111fa2-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-qpxhx\" (UID: \"aaf8534a-72e4-4c88-972e-26eacf111fa2\") " pod="openshift-multus/multus-additional-cni-plugins-qpxhx"
Sep 09 03:13:06.353134 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:13:06.349545    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/61140997-9a63-4a79-ae4d-368d92a92960-device-dir\") pod \"aws-ebs-csi-driver-node-rhxhj\" (UID: \"61140997-9a63-4a79-ae4d-368d92a92960\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rhxhj"
Sep 09 03:13:06.353134 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:13:06.349590    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/aaf8534a-72e4-4c88-972e-26eacf111fa2-cni-binary-copy\") pod \"multus-additional-cni-plugins-qpxhx\" (UID: \"aaf8534a-72e4-4c88-972e-26eacf111fa2\") " pod="openshift-multus/multus-additional-cni-plugins-qpxhx"
Sep 09 03:13:06.353134 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:13:06.349952    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/334f1652-d04c-4382-bbf3-367d96ba0b5c-serviceca\") pod \"node-ca-2zp8k\" (UID: \"334f1652-d04c-4382-bbf3-367d96ba0b5c\") " pod="openshift-image-registry/node-ca-2zp8k"
Sep 09 03:13:06.353134 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:13:06.350443    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/f0fa1f03-0dd2-4015-a21a-02fc04073778-ovn-node-metrics-cert\") pod \"ovnkube-node-9t27t\" (UID: \"f0fa1f03-0dd2-4015-a21a-02fc04073778\") " pod="openshift-ovn-kubernetes/ovnkube-node-9t27t"
Sep 09 03:13:06.353134 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:13:06.351219    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/849f0c91-81ec-458a-a73d-c1922a7f5bc5-tmp\") pod \"tuned-4b7pm\" (UID: \"849f0c91-81ec-458a-a73d-c1922a7f5bc5\") " pod="openshift-cluster-node-tuning-operator/tuned-4b7pm"
Sep 09 03:13:06.353134 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:13:06.351217    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/472d1b86-78f9-45a8-a782-f4274b90a158-agent-certs\") pod \"konnectivity-agent-c4rmf\" (UID: \"472d1b86-78f9-45a8-a782-f4274b90a158\") " pod="kube-system/konnectivity-agent-c4rmf"
Sep 09 03:13:06.353134 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:13:06.351404    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/849f0c91-81ec-458a-a73d-c1922a7f5bc5-etc-tuned\") pod \"tuned-4b7pm\" (UID: \"849f0c91-81ec-458a-a73d-c1922a7f5bc5\") " pod="openshift-cluster-node-tuning-operator/tuned-4b7pm"
Sep 09 03:13:06.360198 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:13:06.360172    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-59qk7\" (UniqueName: \"kubernetes.io/projected/849f0c91-81ec-458a-a73d-c1922a7f5bc5-kube-api-access-59qk7\") pod \"tuned-4b7pm\" (UID: \"849f0c91-81ec-458a-a73d-c1922a7f5bc5\") " pod="openshift-cluster-node-tuning-operator/tuned-4b7pm"
Sep 09 03:13:06.360552 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:13:06.360534    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-r79cr\" (UniqueName: \"kubernetes.io/projected/9ed6dfe0-8780-4a3f-bbeb-9a012722f6e6-kube-api-access-r79cr\") pod \"multus-5rpmm\" (UID: \"9ed6dfe0-8780-4a3f-bbeb-9a012722f6e6\") " pod="openshift-multus/multus-5rpmm"
Sep 09 03:13:06.360690 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:13:06.360669    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qtf7c\" (UniqueName: \"kubernetes.io/projected/61140997-9a63-4a79-ae4d-368d92a92960-kube-api-access-qtf7c\") pod \"aws-ebs-csi-driver-node-rhxhj\" (UID: \"61140997-9a63-4a79-ae4d-368d92a92960\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rhxhj"
Sep 09 03:13:06.360757 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:13:06.360738    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-72wt9\" (UniqueName: \"kubernetes.io/projected/443a2987-5246-4a8c-8631-1b751e02101b-kube-api-access-72wt9\") pod \"network-metrics-daemon-99qzh\" (UID: \"443a2987-5246-4a8c-8631-1b751e02101b\") " pod="openshift-multus/network-metrics-daemon-99qzh"
Sep 09 03:13:06.360802 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:13:06.360738    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bjld7\" (UniqueName: \"kubernetes.io/projected/c9e03e08-c803-4ca3-8f43-aa3390269c0a-kube-api-access-bjld7\") pod \"node-resolver-cqhff\" (UID: \"c9e03e08-c803-4ca3-8f43-aa3390269c0a\") " pod="openshift-dns/node-resolver-cqhff"
Sep 09 03:13:06.360983 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:13:06.360967    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-t6jd8\" (UniqueName: \"kubernetes.io/projected/334f1652-d04c-4382-bbf3-367d96ba0b5c-kube-api-access-t6jd8\") pod \"node-ca-2zp8k\" (UID: \"334f1652-d04c-4382-bbf3-367d96ba0b5c\") " pod="openshift-image-registry/node-ca-2zp8k"
Sep 09 03:13:06.361138 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:13:06.361124    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-25gnh\" (UniqueName: \"kubernetes.io/projected/aaf8534a-72e4-4c88-972e-26eacf111fa2-kube-api-access-25gnh\") pod \"multus-additional-cni-plugins-qpxhx\" (UID: \"aaf8534a-72e4-4c88-972e-26eacf111fa2\") " pod="openshift-multus/multus-additional-cni-plugins-qpxhx"
Sep 09 03:13:06.361535 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:13:06.361522    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5x42s\" (UniqueName: \"kubernetes.io/projected/f0fa1f03-0dd2-4015-a21a-02fc04073778-kube-api-access-5x42s\") pod \"ovnkube-node-9t27t\" (UID: \"f0fa1f03-0dd2-4015-a21a-02fc04073778\") " pod="openshift-ovn-kubernetes/ovnkube-node-9t27t"
Sep 09 03:13:06.391818 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:13:06.391798    2576 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 09 03:13:06.458272 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:13:06.458197    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-j5fzd"
Sep 09 03:13:06.468582 ip-10-0-132-14 kubenswrapper[2576]: W0909 03:13:06.468559    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8c6c6ac7_ea75_45fd_ade0_a1767724b340.slice/crio-2d62332aa0acc77bfcebb562e1473cfc23a752473de62eb31a73eb8b0df4e22e WatchSource:0}: Error finding container 2d62332aa0acc77bfcebb562e1473cfc23a752473de62eb31a73eb8b0df4e22e: Status 404 returned error can't find the container with id 2d62332aa0acc77bfcebb562e1473cfc23a752473de62eb31a73eb8b0df4e22e
Sep 09 03:13:06.491548 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:13:06.491527    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-9t27t"
Sep 09 03:13:06.498406 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:13:06.498359    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-c4rmf"
Sep 09 03:13:06.502005 ip-10-0-132-14 kubenswrapper[2576]: W0909 03:13:06.501980    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf0fa1f03_0dd2_4015_a21a_02fc04073778.slice/crio-01154734702f9f18110216e513859d823e472763d20b70535355920283c02f84 WatchSource:0}: Error finding container 01154734702f9f18110216e513859d823e472763d20b70535355920283c02f84: Status 404 returned error can't find the container with id 01154734702f9f18110216e513859d823e472763d20b70535355920283c02f84
Sep 09 03:13:06.506457 ip-10-0-132-14 kubenswrapper[2576]: W0909 03:13:06.506436    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod472d1b86_78f9_45a8_a782_f4274b90a158.slice/crio-411409de19bacfaceef4f56bfca45d52a120c320e0663cf4b4a32b995bf75b37 WatchSource:0}: Error finding container 411409de19bacfaceef4f56bfca45d52a120c320e0663cf4b4a32b995bf75b37: Status 404 returned error can't find the container with id 411409de19bacfaceef4f56bfca45d52a120c320e0663cf4b4a32b995bf75b37
Sep 09 03:13:06.523437 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:13:06.523414    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-cqhff"
Sep 09 03:13:06.528078 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:13:06.528062    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-2zp8k"
Sep 09 03:13:06.532311 ip-10-0-132-14 kubenswrapper[2576]: W0909 03:13:06.532288    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc9e03e08_c803_4ca3_8f43_aa3390269c0a.slice/crio-4f23f77b98ea62adad5e06baaec2d21e1dc444b7a9060edeac7ab383c27a3dbd WatchSource:0}: Error finding container 4f23f77b98ea62adad5e06baaec2d21e1dc444b7a9060edeac7ab383c27a3dbd: Status 404 returned error can't find the container with id 4f23f77b98ea62adad5e06baaec2d21e1dc444b7a9060edeac7ab383c27a3dbd
Sep 09 03:13:06.533867 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:13:06.533851    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-qpxhx"
Sep 09 03:13:06.536738 ip-10-0-132-14 kubenswrapper[2576]: W0909 03:13:06.536718    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod334f1652_d04c_4382_bbf3_367d96ba0b5c.slice/crio-fd4c667487c487aba98fef9f9c057e4cdf4881ba445d5b6023eef146f0f31c15 WatchSource:0}: Error finding container fd4c667487c487aba98fef9f9c057e4cdf4881ba445d5b6023eef146f0f31c15: Status 404 returned error can't find the container with id fd4c667487c487aba98fef9f9c057e4cdf4881ba445d5b6023eef146f0f31c15
Sep 09 03:13:06.540198 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:13:06.540170    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rhxhj"
Sep 09 03:13:06.541681 ip-10-0-132-14 kubenswrapper[2576]: W0909 03:13:06.541661    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podaaf8534a_72e4_4c88_972e_26eacf111fa2.slice/crio-91064c453d07910f2968e227a523053a42904411b388cc9465781cc421eede74 WatchSource:0}: Error finding container 91064c453d07910f2968e227a523053a42904411b388cc9465781cc421eede74: Status 404 returned error can't find the container with id 91064c453d07910f2968e227a523053a42904411b388cc9465781cc421eede74
Sep 09 03:13:06.545820 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:13:06.545799    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-4b7pm"
Sep 09 03:13:06.549223 ip-10-0-132-14 kubenswrapper[2576]: W0909 03:13:06.549200    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod61140997_9a63_4a79_ae4d_368d92a92960.slice/crio-913b5968fc859836750160d3fd9289fa1cb55ee42370f0a94c514dad08f7a85d WatchSource:0}: Error finding container 913b5968fc859836750160d3fd9289fa1cb55ee42370f0a94c514dad08f7a85d: Status 404 returned error can't find the container with id 913b5968fc859836750160d3fd9289fa1cb55ee42370f0a94c514dad08f7a85d
Sep 09 03:13:06.550061 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:13:06.550042    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-5rpmm"
Sep 09 03:13:06.554831 ip-10-0-132-14 kubenswrapper[2576]: W0909 03:13:06.554804    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod849f0c91_81ec_458a_a73d_c1922a7f5bc5.slice/crio-4bc0e53f1bd973bfdd1df37520e8a3070fc2ffdb3a167ba764099584198d1ace WatchSource:0}: Error finding container 4bc0e53f1bd973bfdd1df37520e8a3070fc2ffdb3a167ba764099584198d1ace: Status 404 returned error can't find the container with id 4bc0e53f1bd973bfdd1df37520e8a3070fc2ffdb3a167ba764099584198d1ace
Sep 09 03:13:06.560881 ip-10-0-132-14 kubenswrapper[2576]: W0909 03:13:06.560858    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9ed6dfe0_8780_4a3f_bbeb_9a012722f6e6.slice/crio-5d2670662386b668c564e04b331e900c576c81a9f51103dc02e1331630c224ea WatchSource:0}: Error finding container 5d2670662386b668c564e04b331e900c576c81a9f51103dc02e1331630c224ea: Status 404 returned error can't find the container with id 5d2670662386b668c564e04b331e900c576c81a9f51103dc02e1331630c224ea
Sep 09 03:13:07.890641 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:13:07.890550    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4dddl\" (UniqueName: \"kubernetes.io/projected/344533be-8249-42e9-9abc-c0bc68e18132-kube-api-access-4dddl\") pod \"network-check-target-s2pb7\" (UID: \"344533be-8249-42e9-9abc-c0bc68e18132\") " pod="openshift-network-diagnostics/network-check-target-s2pb7"
Sep 09 03:13:07.890778 ip-10-0-132-14 kubenswrapper[2576]: E0909 03:13:07.890694    2576 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 09 03:13:07.890778 ip-10-0-132-14 kubenswrapper[2576]: E0909 03:13:07.890709    2576 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 09 03:13:07.890778 ip-10-0-132-14 kubenswrapper[2576]: E0909 03:13:07.890718    2576 projected.go:196] Error preparing data for projected volume kube-api-access-4dddl for pod openshift-network-diagnostics/network-check-target-s2pb7: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 09 03:13:07.890778 ip-10-0-132-14 kubenswrapper[2576]: E0909 03:13:07.890766    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/344533be-8249-42e9-9abc-c0bc68e18132-kube-api-access-4dddl podName:344533be-8249-42e9-9abc-c0bc68e18132 nodeName:}" failed. No retries permitted until 2026-09-09 03:13:08.890752779 +0000 UTC m=+3.032238839 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-4dddl" (UniqueName: "kubernetes.io/projected/344533be-8249-42e9-9abc-c0bc68e18132-kube-api-access-4dddl") pod "network-check-target-s2pb7" (UID: "344533be-8249-42e9-9abc-c0bc68e18132") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 09 03:13:07.992343 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:13:07.991665    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/2f43cc28-7345-4c06-9160-4245b99697b4-original-pull-secret\") pod \"global-pull-secret-syncer-t9bbm\" (UID: \"2f43cc28-7345-4c06-9160-4245b99697b4\") " pod="kube-system/global-pull-secret-syncer-t9bbm"
Sep 09 03:13:07.992343 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:13:07.991713    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/443a2987-5246-4a8c-8631-1b751e02101b-metrics-certs\") pod \"network-metrics-daemon-99qzh\" (UID: \"443a2987-5246-4a8c-8631-1b751e02101b\") " pod="openshift-multus/network-metrics-daemon-99qzh"
Sep 09 03:13:07.992343 ip-10-0-132-14 kubenswrapper[2576]: E0909 03:13:07.991861    2576 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 09 03:13:07.992343 ip-10-0-132-14 kubenswrapper[2576]: E0909 03:13:07.991923    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/443a2987-5246-4a8c-8631-1b751e02101b-metrics-certs podName:443a2987-5246-4a8c-8631-1b751e02101b nodeName:}" failed. No retries permitted until 2026-09-09 03:13:08.991903298 +0000 UTC m=+3.133389369 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/443a2987-5246-4a8c-8631-1b751e02101b-metrics-certs") pod "network-metrics-daemon-99qzh" (UID: "443a2987-5246-4a8c-8631-1b751e02101b") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 09 03:13:07.992343 ip-10-0-132-14 kubenswrapper[2576]: E0909 03:13:07.992270    2576 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 09 03:13:07.992343 ip-10-0-132-14 kubenswrapper[2576]: E0909 03:13:07.992319    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2f43cc28-7345-4c06-9160-4245b99697b4-original-pull-secret podName:2f43cc28-7345-4c06-9160-4245b99697b4 nodeName:}" failed. No retries permitted until 2026-09-09 03:13:08.992301393 +0000 UTC m=+3.133787463 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/2f43cc28-7345-4c06-9160-4245b99697b4-original-pull-secret") pod "global-pull-secret-syncer-t9bbm" (UID: "2f43cc28-7345-4c06-9160-4245b99697b4") : object "kube-system"/"original-pull-secret" not registered
Sep 09 03:13:08.218300 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:13:08.218222    2576 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 09 03:13:08.319942 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:13:08.319882    2576 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-09-08 03:08:07 +0000 UTC" deadline="2028-06-14 11:17:55.885191688 +0000 UTC"
Sep 09 03:13:08.319942 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:13:08.319939    2576 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="15464h4m47.565256108s"
Sep 09 03:13:08.356049 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:13:08.356016    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-s2pb7"
Sep 09 03:13:08.356211 ip-10-0-132-14 kubenswrapper[2576]: E0909 03:13:08.356140    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-s2pb7" podUID="344533be-8249-42e9-9abc-c0bc68e18132"
Sep 09 03:13:08.356211 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:13:08.356182    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-99qzh"
Sep 09 03:13:08.356322 ip-10-0-132-14 kubenswrapper[2576]: E0909 03:13:08.356265    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-99qzh" podUID="443a2987-5246-4a8c-8631-1b751e02101b"
Sep 09 03:13:08.379139 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:13:08.379076    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rhxhj" event={"ID":"61140997-9a63-4a79-ae4d-368d92a92960","Type":"ContainerStarted","Data":"913b5968fc859836750160d3fd9289fa1cb55ee42370f0a94c514dad08f7a85d"}
Sep 09 03:13:08.385089 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:13:08.385052    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-2zp8k" event={"ID":"334f1652-d04c-4382-bbf3-367d96ba0b5c","Type":"ContainerStarted","Data":"fd4c667487c487aba98fef9f9c057e4cdf4881ba445d5b6023eef146f0f31c15"}
Sep 09 03:13:08.395076 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:13:08.394832    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-cqhff" event={"ID":"c9e03e08-c803-4ca3-8f43-aa3390269c0a","Type":"ContainerStarted","Data":"4f23f77b98ea62adad5e06baaec2d21e1dc444b7a9060edeac7ab383c27a3dbd"}
Sep 09 03:13:08.401129 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:13:08.401093    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-c4rmf" event={"ID":"472d1b86-78f9-45a8-a782-f4274b90a158","Type":"ContainerStarted","Data":"411409de19bacfaceef4f56bfca45d52a120c320e0663cf4b4a32b995bf75b37"}
Sep 09 03:13:08.403358 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:13:08.403328    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-9t27t" event={"ID":"f0fa1f03-0dd2-4015-a21a-02fc04073778","Type":"ContainerStarted","Data":"01154734702f9f18110216e513859d823e472763d20b70535355920283c02f84"}
Sep 09 03:13:08.407034 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:13:08.407004    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-j5fzd" event={"ID":"8c6c6ac7-ea75-45fd-ade0-a1767724b340","Type":"ContainerStarted","Data":"2d62332aa0acc77bfcebb562e1473cfc23a752473de62eb31a73eb8b0df4e22e"}
Sep 09 03:13:08.431127 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:13:08.431088    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-132-14.ec2.internal" event={"ID":"f91cf12d2f3ddb8526c92708693ce21c","Type":"ContainerStarted","Data":"092beb102984d640b934da59480798822a7c2f3ec4db80d75d8862059a66a513"}
Sep 09 03:13:08.440285 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:13:08.440234    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-5rpmm" event={"ID":"9ed6dfe0-8780-4a3f-bbeb-9a012722f6e6","Type":"ContainerStarted","Data":"5d2670662386b668c564e04b331e900c576c81a9f51103dc02e1331630c224ea"}
Sep 09 03:13:08.445350 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:13:08.445316    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-4b7pm" event={"ID":"849f0c91-81ec-458a-a73d-c1922a7f5bc5","Type":"ContainerStarted","Data":"4bc0e53f1bd973bfdd1df37520e8a3070fc2ffdb3a167ba764099584198d1ace"}
Sep 09 03:13:08.456539 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:13:08.456445    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-qpxhx" event={"ID":"aaf8534a-72e4-4c88-972e-26eacf111fa2","Type":"ContainerStarted","Data":"91064c453d07910f2968e227a523053a42904411b388cc9465781cc421eede74"}
Sep 09 03:13:08.459550 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:13:08.459517    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-14.ec2.internal" event={"ID":"3882ef2c4b5f745a591c85452d6b7dae","Type":"ContainerStarted","Data":"9bc9187c771c2bb7bcac98dd608596b8a101f5f5b1b0218a7042d4c7d2ae242e"}
Sep 09 03:13:08.695517 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:13:08.695436    2576 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 09 03:13:08.899573 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:13:08.899534    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4dddl\" (UniqueName: \"kubernetes.io/projected/344533be-8249-42e9-9abc-c0bc68e18132-kube-api-access-4dddl\") pod \"network-check-target-s2pb7\" (UID: \"344533be-8249-42e9-9abc-c0bc68e18132\") " pod="openshift-network-diagnostics/network-check-target-s2pb7"
Sep 09 03:13:08.899749 ip-10-0-132-14 kubenswrapper[2576]: E0909 03:13:08.899732    2576 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 09 03:13:08.899833 ip-10-0-132-14 kubenswrapper[2576]: E0909 03:13:08.899758    2576 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 09 03:13:08.899833 ip-10-0-132-14 kubenswrapper[2576]: E0909 03:13:08.899771    2576 projected.go:196] Error preparing data for projected volume kube-api-access-4dddl for pod openshift-network-diagnostics/network-check-target-s2pb7: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 09 03:13:08.899952 ip-10-0-132-14 kubenswrapper[2576]: E0909 03:13:08.899849    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/344533be-8249-42e9-9abc-c0bc68e18132-kube-api-access-4dddl podName:344533be-8249-42e9-9abc-c0bc68e18132 nodeName:}" failed. No retries permitted until 2026-09-09 03:13:10.899829545 +0000 UTC m=+5.041315620 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-4dddl" (UniqueName: "kubernetes.io/projected/344533be-8249-42e9-9abc-c0bc68e18132-kube-api-access-4dddl") pod "network-check-target-s2pb7" (UID: "344533be-8249-42e9-9abc-c0bc68e18132") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 09 03:13:09.000688 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:13:09.000598    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/443a2987-5246-4a8c-8631-1b751e02101b-metrics-certs\") pod \"network-metrics-daemon-99qzh\" (UID: \"443a2987-5246-4a8c-8631-1b751e02101b\") " pod="openshift-multus/network-metrics-daemon-99qzh"
Sep 09 03:13:09.000866 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:13:09.000703    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/2f43cc28-7345-4c06-9160-4245b99697b4-original-pull-secret\") pod \"global-pull-secret-syncer-t9bbm\" (UID: \"2f43cc28-7345-4c06-9160-4245b99697b4\") " pod="kube-system/global-pull-secret-syncer-t9bbm"
Sep 09 03:13:09.000866 ip-10-0-132-14 kubenswrapper[2576]: E0909 03:13:09.000838    2576 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 09 03:13:09.000990 ip-10-0-132-14 kubenswrapper[2576]: E0909 03:13:09.000902    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2f43cc28-7345-4c06-9160-4245b99697b4-original-pull-secret podName:2f43cc28-7345-4c06-9160-4245b99697b4 nodeName:}" failed. No retries permitted until 2026-09-09 03:13:11.000881773 +0000 UTC m=+5.142367835 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/2f43cc28-7345-4c06-9160-4245b99697b4-original-pull-secret") pod "global-pull-secret-syncer-t9bbm" (UID: "2f43cc28-7345-4c06-9160-4245b99697b4") : object "kube-system"/"original-pull-secret" not registered
Sep 09 03:13:09.001337 ip-10-0-132-14 kubenswrapper[2576]: E0909 03:13:09.001317    2576 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 09 03:13:09.001420 ip-10-0-132-14 kubenswrapper[2576]: E0909 03:13:09.001378    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/443a2987-5246-4a8c-8631-1b751e02101b-metrics-certs podName:443a2987-5246-4a8c-8631-1b751e02101b nodeName:}" failed. No retries permitted until 2026-09-09 03:13:11.001360961 +0000 UTC m=+5.142847016 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/443a2987-5246-4a8c-8631-1b751e02101b-metrics-certs") pod "network-metrics-daemon-99qzh" (UID: "443a2987-5246-4a8c-8631-1b751e02101b") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 09 03:13:09.320601 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:13:09.320486    2576 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-09-08 03:08:07 +0000 UTC" deadline="2028-03-20 16:14:49.744993139 +0000 UTC"
Sep 09 03:13:09.320601 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:13:09.320549    2576 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="13405h1m40.424449035s"
Sep 09 03:13:09.354381 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:13:09.354346    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-t9bbm"
Sep 09 03:13:09.354568 ip-10-0-132-14 kubenswrapper[2576]: E0909 03:13:09.354480    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-t9bbm" podUID="2f43cc28-7345-4c06-9160-4245b99697b4"
Sep 09 03:13:10.250476 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:13:10.250440    2576 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 09 03:13:10.356457 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:13:10.356427    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-s2pb7"
Sep 09 03:13:10.356900 ip-10-0-132-14 kubenswrapper[2576]: E0909 03:13:10.356555    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-s2pb7" podUID="344533be-8249-42e9-9abc-c0bc68e18132"
Sep 09 03:13:10.357045 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:13:10.357024    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-99qzh"
Sep 09 03:13:10.357166 ip-10-0-132-14 kubenswrapper[2576]: E0909 03:13:10.357144    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-99qzh" podUID="443a2987-5246-4a8c-8631-1b751e02101b"
Sep 09 03:13:10.917979 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:13:10.917942    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4dddl\" (UniqueName: \"kubernetes.io/projected/344533be-8249-42e9-9abc-c0bc68e18132-kube-api-access-4dddl\") pod \"network-check-target-s2pb7\" (UID: \"344533be-8249-42e9-9abc-c0bc68e18132\") " pod="openshift-network-diagnostics/network-check-target-s2pb7"
Sep 09 03:13:10.918185 ip-10-0-132-14 kubenswrapper[2576]: E0909 03:13:10.918098    2576 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 09 03:13:10.918185 ip-10-0-132-14 kubenswrapper[2576]: E0909 03:13:10.918121    2576 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 09 03:13:10.918185 ip-10-0-132-14 kubenswrapper[2576]: E0909 03:13:10.918133    2576 projected.go:196] Error preparing data for projected volume kube-api-access-4dddl for pod openshift-network-diagnostics/network-check-target-s2pb7: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 09 03:13:10.918185 ip-10-0-132-14 kubenswrapper[2576]: E0909 03:13:10.918186    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/344533be-8249-42e9-9abc-c0bc68e18132-kube-api-access-4dddl podName:344533be-8249-42e9-9abc-c0bc68e18132 nodeName:}" failed. No retries permitted until 2026-09-09 03:13:14.918168769 +0000 UTC m=+9.059654840 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-4dddl" (UniqueName: "kubernetes.io/projected/344533be-8249-42e9-9abc-c0bc68e18132-kube-api-access-4dddl") pod "network-check-target-s2pb7" (UID: "344533be-8249-42e9-9abc-c0bc68e18132") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 09 03:13:11.019180 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:13:11.019140    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/2f43cc28-7345-4c06-9160-4245b99697b4-original-pull-secret\") pod \"global-pull-secret-syncer-t9bbm\" (UID: \"2f43cc28-7345-4c06-9160-4245b99697b4\") " pod="kube-system/global-pull-secret-syncer-t9bbm"
Sep 09 03:13:11.019370 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:13:11.019194    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/443a2987-5246-4a8c-8631-1b751e02101b-metrics-certs\") pod \"network-metrics-daemon-99qzh\" (UID: \"443a2987-5246-4a8c-8631-1b751e02101b\") " pod="openshift-multus/network-metrics-daemon-99qzh"
Sep 09 03:13:11.019370 ip-10-0-132-14 kubenswrapper[2576]: E0909 03:13:11.019316    2576 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 09 03:13:11.019370 ip-10-0-132-14 kubenswrapper[2576]: E0909 03:13:11.019317    2576 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 09 03:13:11.019538 ip-10-0-132-14 kubenswrapper[2576]: E0909 03:13:11.019385    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/443a2987-5246-4a8c-8631-1b751e02101b-metrics-certs podName:443a2987-5246-4a8c-8631-1b751e02101b nodeName:}" failed. No retries permitted until 2026-09-09 03:13:15.019365232 +0000 UTC m=+9.160851307 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/443a2987-5246-4a8c-8631-1b751e02101b-metrics-certs") pod "network-metrics-daemon-99qzh" (UID: "443a2987-5246-4a8c-8631-1b751e02101b") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 09 03:13:11.019538 ip-10-0-132-14 kubenswrapper[2576]: E0909 03:13:11.019403    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2f43cc28-7345-4c06-9160-4245b99697b4-original-pull-secret podName:2f43cc28-7345-4c06-9160-4245b99697b4 nodeName:}" failed. No retries permitted until 2026-09-09 03:13:15.0193945 +0000 UTC m=+9.160880556 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/2f43cc28-7345-4c06-9160-4245b99697b4-original-pull-secret") pod "global-pull-secret-syncer-t9bbm" (UID: "2f43cc28-7345-4c06-9160-4245b99697b4") : object "kube-system"/"original-pull-secret" not registered
Sep 09 03:13:11.354674 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:13:11.354596    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-t9bbm"
Sep 09 03:13:11.354870 ip-10-0-132-14 kubenswrapper[2576]: E0909 03:13:11.354718    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-t9bbm" podUID="2f43cc28-7345-4c06-9160-4245b99697b4"
Sep 09 03:13:12.356347 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:13:12.355943    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-s2pb7"
Sep 09 03:13:12.356347 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:13:12.355981    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-99qzh"
Sep 09 03:13:12.356347 ip-10-0-132-14 kubenswrapper[2576]: E0909 03:13:12.356065    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-s2pb7" podUID="344533be-8249-42e9-9abc-c0bc68e18132"
Sep 09 03:13:12.356347 ip-10-0-132-14 kubenswrapper[2576]: E0909 03:13:12.356223    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-99qzh" podUID="443a2987-5246-4a8c-8631-1b751e02101b"
Sep 09 03:13:13.354889 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:13:13.354376    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-t9bbm"
Sep 09 03:13:13.354889 ip-10-0-132-14 kubenswrapper[2576]: E0909 03:13:13.354502    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-t9bbm" podUID="2f43cc28-7345-4c06-9160-4245b99697b4"
Sep 09 03:13:14.355391 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:13:14.354929    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-99qzh"
Sep 09 03:13:14.355391 ip-10-0-132-14 kubenswrapper[2576]: E0909 03:13:14.355072    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-99qzh" podUID="443a2987-5246-4a8c-8631-1b751e02101b"
Sep 09 03:13:14.355391 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:13:14.355129    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-s2pb7"
Sep 09 03:13:14.355391 ip-10-0-132-14 kubenswrapper[2576]: E0909 03:13:14.355252    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-s2pb7" podUID="344533be-8249-42e9-9abc-c0bc68e18132"
Sep 09 03:13:14.952127 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:13:14.952089    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4dddl\" (UniqueName: \"kubernetes.io/projected/344533be-8249-42e9-9abc-c0bc68e18132-kube-api-access-4dddl\") pod \"network-check-target-s2pb7\" (UID: \"344533be-8249-42e9-9abc-c0bc68e18132\") " pod="openshift-network-diagnostics/network-check-target-s2pb7"
Sep 09 03:13:14.952315 ip-10-0-132-14 kubenswrapper[2576]: E0909 03:13:14.952250    2576 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 09 03:13:14.952315 ip-10-0-132-14 kubenswrapper[2576]: E0909 03:13:14.952266    2576 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 09 03:13:14.952315 ip-10-0-132-14 kubenswrapper[2576]: E0909 03:13:14.952276    2576 projected.go:196] Error preparing data for projected volume kube-api-access-4dddl for pod openshift-network-diagnostics/network-check-target-s2pb7: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 09 03:13:14.952475 ip-10-0-132-14 kubenswrapper[2576]: E0909 03:13:14.952323    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/344533be-8249-42e9-9abc-c0bc68e18132-kube-api-access-4dddl podName:344533be-8249-42e9-9abc-c0bc68e18132 nodeName:}" failed. No retries permitted until 2026-09-09 03:13:22.952310349 +0000 UTC m=+17.093796404 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-4dddl" (UniqueName: "kubernetes.io/projected/344533be-8249-42e9-9abc-c0bc68e18132-kube-api-access-4dddl") pod "network-check-target-s2pb7" (UID: "344533be-8249-42e9-9abc-c0bc68e18132") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 09 03:13:15.052714 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:13:15.052678    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/2f43cc28-7345-4c06-9160-4245b99697b4-original-pull-secret\") pod \"global-pull-secret-syncer-t9bbm\" (UID: \"2f43cc28-7345-4c06-9160-4245b99697b4\") " pod="kube-system/global-pull-secret-syncer-t9bbm"
Sep 09 03:13:15.052714 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:13:15.052727    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/443a2987-5246-4a8c-8631-1b751e02101b-metrics-certs\") pod \"network-metrics-daemon-99qzh\" (UID: \"443a2987-5246-4a8c-8631-1b751e02101b\") " pod="openshift-multus/network-metrics-daemon-99qzh"
Sep 09 03:13:15.052983 ip-10-0-132-14 kubenswrapper[2576]: E0909 03:13:15.052879    2576 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 09 03:13:15.052983 ip-10-0-132-14 kubenswrapper[2576]: E0909 03:13:15.052943    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/443a2987-5246-4a8c-8631-1b751e02101b-metrics-certs podName:443a2987-5246-4a8c-8631-1b751e02101b nodeName:}" failed. No retries permitted until 2026-09-09 03:13:23.052922266 +0000 UTC m=+17.194408333 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/443a2987-5246-4a8c-8631-1b751e02101b-metrics-certs") pod "network-metrics-daemon-99qzh" (UID: "443a2987-5246-4a8c-8631-1b751e02101b") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 09 03:13:15.053213 ip-10-0-132-14 kubenswrapper[2576]: E0909 03:13:15.053188    2576 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 09 03:13:15.053342 ip-10-0-132-14 kubenswrapper[2576]: E0909 03:13:15.053247    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2f43cc28-7345-4c06-9160-4245b99697b4-original-pull-secret podName:2f43cc28-7345-4c06-9160-4245b99697b4 nodeName:}" failed. No retries permitted until 2026-09-09 03:13:23.05323385 +0000 UTC m=+17.194719905 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/2f43cc28-7345-4c06-9160-4245b99697b4-original-pull-secret") pod "global-pull-secret-syncer-t9bbm" (UID: "2f43cc28-7345-4c06-9160-4245b99697b4") : object "kube-system"/"original-pull-secret" not registered
Sep 09 03:13:15.353924 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:13:15.353884    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-t9bbm"
Sep 09 03:13:15.354083 ip-10-0-132-14 kubenswrapper[2576]: E0909 03:13:15.354023    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-t9bbm" podUID="2f43cc28-7345-4c06-9160-4245b99697b4"
Sep 09 03:13:16.354941 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:13:16.354891    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-s2pb7"
Sep 09 03:13:16.355624 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:13:16.354953    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-99qzh"
Sep 09 03:13:16.355624 ip-10-0-132-14 kubenswrapper[2576]: E0909 03:13:16.355030    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-s2pb7" podUID="344533be-8249-42e9-9abc-c0bc68e18132"
Sep 09 03:13:16.355624 ip-10-0-132-14 kubenswrapper[2576]: E0909 03:13:16.355147    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-99qzh" podUID="443a2987-5246-4a8c-8631-1b751e02101b"
Sep 09 03:13:17.354434 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:13:17.354403    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-t9bbm"
Sep 09 03:13:17.354595 ip-10-0-132-14 kubenswrapper[2576]: E0909 03:13:17.354513    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-t9bbm" podUID="2f43cc28-7345-4c06-9160-4245b99697b4"
Sep 09 03:13:18.354277 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:13:18.354243    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-s2pb7"
Sep 09 03:13:18.354698 ip-10-0-132-14 kubenswrapper[2576]: E0909 03:13:18.354372    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-s2pb7" podUID="344533be-8249-42e9-9abc-c0bc68e18132"
Sep 09 03:13:18.354698 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:13:18.354409    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-99qzh"
Sep 09 03:13:18.354698 ip-10-0-132-14 kubenswrapper[2576]: E0909 03:13:18.354528    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-99qzh" podUID="443a2987-5246-4a8c-8631-1b751e02101b"
Sep 09 03:13:19.354603 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:13:19.354555    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-t9bbm"
Sep 09 03:13:19.355101 ip-10-0-132-14 kubenswrapper[2576]: E0909 03:13:19.354698    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-t9bbm" podUID="2f43cc28-7345-4c06-9160-4245b99697b4"
Sep 09 03:13:20.354536 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:13:20.354499    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-s2pb7"
Sep 09 03:13:20.354724 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:13:20.354503    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-99qzh"
Sep 09 03:13:20.354724 ip-10-0-132-14 kubenswrapper[2576]: E0909 03:13:20.354633    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-s2pb7" podUID="344533be-8249-42e9-9abc-c0bc68e18132"
Sep 09 03:13:20.354724 ip-10-0-132-14 kubenswrapper[2576]: E0909 03:13:20.354713    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-99qzh" podUID="443a2987-5246-4a8c-8631-1b751e02101b"
Sep 09 03:13:21.354384 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:13:21.354351    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-t9bbm"
Sep 09 03:13:21.354562 ip-10-0-132-14 kubenswrapper[2576]: E0909 03:13:21.354496    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-t9bbm" podUID="2f43cc28-7345-4c06-9160-4245b99697b4"
Sep 09 03:13:22.354639 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:13:22.354606    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-99qzh"
Sep 09 03:13:22.355278 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:13:22.354606    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-s2pb7"
Sep 09 03:13:22.355278 ip-10-0-132-14 kubenswrapper[2576]: E0909 03:13:22.354760    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-99qzh" podUID="443a2987-5246-4a8c-8631-1b751e02101b"
Sep 09 03:13:22.355278 ip-10-0-132-14 kubenswrapper[2576]: E0909 03:13:22.354849    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-s2pb7" podUID="344533be-8249-42e9-9abc-c0bc68e18132"
Sep 09 03:13:23.009930 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:13:23.009872    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4dddl\" (UniqueName: \"kubernetes.io/projected/344533be-8249-42e9-9abc-c0bc68e18132-kube-api-access-4dddl\") pod \"network-check-target-s2pb7\" (UID: \"344533be-8249-42e9-9abc-c0bc68e18132\") " pod="openshift-network-diagnostics/network-check-target-s2pb7"
Sep 09 03:13:23.010140 ip-10-0-132-14 kubenswrapper[2576]: E0909 03:13:23.010043    2576 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 09 03:13:23.010140 ip-10-0-132-14 kubenswrapper[2576]: E0909 03:13:23.010072    2576 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 09 03:13:23.010140 ip-10-0-132-14 kubenswrapper[2576]: E0909 03:13:23.010085    2576 projected.go:196] Error preparing data for projected volume kube-api-access-4dddl for pod openshift-network-diagnostics/network-check-target-s2pb7: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 09 03:13:23.010269 ip-10-0-132-14 kubenswrapper[2576]: E0909 03:13:23.010142    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/344533be-8249-42e9-9abc-c0bc68e18132-kube-api-access-4dddl podName:344533be-8249-42e9-9abc-c0bc68e18132 nodeName:}" failed. No retries permitted until 2026-09-09 03:13:39.01012741 +0000 UTC m=+33.151613468 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-4dddl" (UniqueName: "kubernetes.io/projected/344533be-8249-42e9-9abc-c0bc68e18132-kube-api-access-4dddl") pod "network-check-target-s2pb7" (UID: "344533be-8249-42e9-9abc-c0bc68e18132") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 09 03:13:23.110528 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:13:23.110485    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/2f43cc28-7345-4c06-9160-4245b99697b4-original-pull-secret\") pod \"global-pull-secret-syncer-t9bbm\" (UID: \"2f43cc28-7345-4c06-9160-4245b99697b4\") " pod="kube-system/global-pull-secret-syncer-t9bbm"
Sep 09 03:13:23.110727 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:13:23.110540    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/443a2987-5246-4a8c-8631-1b751e02101b-metrics-certs\") pod \"network-metrics-daemon-99qzh\" (UID: \"443a2987-5246-4a8c-8631-1b751e02101b\") " pod="openshift-multus/network-metrics-daemon-99qzh"
Sep 09 03:13:23.110727 ip-10-0-132-14 kubenswrapper[2576]: E0909 03:13:23.110668    2576 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 09 03:13:23.110727 ip-10-0-132-14 kubenswrapper[2576]: E0909 03:13:23.110695    2576 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 09 03:13:23.110924 ip-10-0-132-14 kubenswrapper[2576]: E0909 03:13:23.110750    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2f43cc28-7345-4c06-9160-4245b99697b4-original-pull-secret podName:2f43cc28-7345-4c06-9160-4245b99697b4 nodeName:}" failed. No retries permitted until 2026-09-09 03:13:39.110726636 +0000 UTC m=+33.252212708 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/2f43cc28-7345-4c06-9160-4245b99697b4-original-pull-secret") pod "global-pull-secret-syncer-t9bbm" (UID: "2f43cc28-7345-4c06-9160-4245b99697b4") : object "kube-system"/"original-pull-secret" not registered
Sep 09 03:13:23.110924 ip-10-0-132-14 kubenswrapper[2576]: E0909 03:13:23.110772    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/443a2987-5246-4a8c-8631-1b751e02101b-metrics-certs podName:443a2987-5246-4a8c-8631-1b751e02101b nodeName:}" failed. No retries permitted until 2026-09-09 03:13:39.110761859 +0000 UTC m=+33.252247914 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/443a2987-5246-4a8c-8631-1b751e02101b-metrics-certs") pod "network-metrics-daemon-99qzh" (UID: "443a2987-5246-4a8c-8631-1b751e02101b") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 09 03:13:23.354084 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:13:23.354011    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-t9bbm"
Sep 09 03:13:23.354216 ip-10-0-132-14 kubenswrapper[2576]: E0909 03:13:23.354138    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-t9bbm" podUID="2f43cc28-7345-4c06-9160-4245b99697b4"
Sep 09 03:13:24.354737 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:13:24.354706    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-99qzh"
Sep 09 03:13:24.354737 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:13:24.354737    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-s2pb7"
Sep 09 03:13:24.355293 ip-10-0-132-14 kubenswrapper[2576]: E0909 03:13:24.354885    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-99qzh" podUID="443a2987-5246-4a8c-8631-1b751e02101b"
Sep 09 03:13:24.355293 ip-10-0-132-14 kubenswrapper[2576]: E0909 03:13:24.355001    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-s2pb7" podUID="344533be-8249-42e9-9abc-c0bc68e18132"
Sep 09 03:13:25.354819 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:13:25.354780    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-t9bbm"
Sep 09 03:13:25.355264 ip-10-0-132-14 kubenswrapper[2576]: E0909 03:13:25.354916    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-t9bbm" podUID="2f43cc28-7345-4c06-9160-4245b99697b4"
Sep 09 03:13:26.355241 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:13:26.355014    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-s2pb7"
Sep 09 03:13:26.355702 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:13:26.355083    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-99qzh"
Sep 09 03:13:26.355702 ip-10-0-132-14 kubenswrapper[2576]: E0909 03:13:26.355327    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-s2pb7" podUID="344533be-8249-42e9-9abc-c0bc68e18132"
Sep 09 03:13:26.355702 ip-10-0-132-14 kubenswrapper[2576]: E0909 03:13:26.355433    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-99qzh" podUID="443a2987-5246-4a8c-8631-1b751e02101b"
Sep 09 03:13:26.503997 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:13:26.503178    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-4b7pm" event={"ID":"849f0c91-81ec-458a-a73d-c1922a7f5bc5","Type":"ContainerStarted","Data":"3e92b117a9f03c5b9580146bb88df450224e122708e10c3c506b315c46edb0ec"}
Sep 09 03:13:26.506388 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:13:26.506226    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-9t27t" event={"ID":"f0fa1f03-0dd2-4015-a21a-02fc04073778","Type":"ContainerStarted","Data":"147619abb7b91da63609c7ed8e6680008462bc862b70e325bc9941d25875c1d7"}
Sep 09 03:13:26.518384 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:13:26.517467    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-132-14.ec2.internal" event={"ID":"f91cf12d2f3ddb8526c92708693ce21c","Type":"ContainerStarted","Data":"0ad496b9945f8cad71ee2b8c8ece5c77dd02012399a0b6c1f6a7c7ade1ba3b61"}
Sep 09 03:13:26.528957 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:13:26.527021    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-4b7pm" podStartSLOduration=2.141465987 podStartE2EDuration="20.527000964s" totalImagesPullingTime="18.385534977s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-09 03:13:06 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 03:13:26.52546483 +0000 UTC m=+20.666950906" watchObservedRunningTime="2026-09-09 03:13:26.527000964 +0000 UTC m=+20.668487042"
Sep 09 03:13:26.528957 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:13:26.528586    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-5rpmm" event={"ID":"9ed6dfe0-8780-4a3f-bbeb-9a012722f6e6","Type":"ContainerStarted","Data":"cc2b66319e2575c6fbb8b1203defe0f9fd9c04ad3cbbf2cc3c08b689980d2235"}
Sep 09 03:13:26.564255 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:13:26.562888    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-132-14.ec2.internal" podStartSLOduration=19.56286758 podStartE2EDuration="19.56286758s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-09 03:13:07 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 03:13:26.545201267 +0000 UTC m=+20.686687346" watchObservedRunningTime="2026-09-09 03:13:26.56286758 +0000 UTC m=+20.704353658"
Sep 09 03:13:26.564255 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:13:26.563855    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-5rpmm" podStartSLOduration=2.079294519 podStartE2EDuration="20.56383585s" totalImagesPullingTime="18.484541331s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-09 03:13:06 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 03:13:26.563633325 +0000 UTC m=+20.705119414" watchObservedRunningTime="2026-09-09 03:13:26.56383585 +0000 UTC m=+20.705321924"
Sep 09 03:13:27.354404 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:13:27.354337    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-t9bbm"
Sep 09 03:13:27.354516 ip-10-0-132-14 kubenswrapper[2576]: E0909 03:13:27.354467    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-t9bbm" podUID="2f43cc28-7345-4c06-9160-4245b99697b4"
Sep 09 03:13:27.532494 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:13:27.532458    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rhxhj" event={"ID":"61140997-9a63-4a79-ae4d-368d92a92960","Type":"ContainerStarted","Data":"3748fe0e93e3cdca87abe18f4a9806714d7993c94f6c40ee7dae0917cfb3c746"}
Sep 09 03:13:27.534006 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:13:27.533980    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-2zp8k" event={"ID":"334f1652-d04c-4382-bbf3-367d96ba0b5c","Type":"ContainerStarted","Data":"c4bedc44224e8539ac5e1bb2278077fb4b07c97ca0993ac04051bed509f2fd49"}
Sep 09 03:13:27.535375 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:13:27.535352    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-cqhff" event={"ID":"c9e03e08-c803-4ca3-8f43-aa3390269c0a","Type":"ContainerStarted","Data":"90af9ee331523a14d8b3ea5c5ab60130a652df09381033d60d128ee946899218"}
Sep 09 03:13:27.536697 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:13:27.536678    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-c4rmf" event={"ID":"472d1b86-78f9-45a8-a782-f4274b90a158","Type":"ContainerStarted","Data":"c3536c98c6536db403ea823709beb37ea3cf172ed492380bb9cb343bd2b4d6da"}
Sep 09 03:13:27.539532 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:13:27.539513    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-9t27t" event={"ID":"f0fa1f03-0dd2-4015-a21a-02fc04073778","Type":"ContainerStarted","Data":"8021c6643402512bbec3b4508bc87234381bfa060810926629f7e99edcd3ef8e"}
Sep 09 03:13:27.539605 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:13:27.539572    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-9t27t" event={"ID":"f0fa1f03-0dd2-4015-a21a-02fc04073778","Type":"ContainerStarted","Data":"5e8df0bd75bc66f5c454ad2c0a868c29300b6dc28de50a57b287b4bdd8530d16"}
Sep 09 03:13:27.539605 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:13:27.539602    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-9t27t" event={"ID":"f0fa1f03-0dd2-4015-a21a-02fc04073778","Type":"ContainerStarted","Data":"5f173a5a3761e85a451f540335fb24084d6de9f18245ce1d46aaa8ffd699783c"}
Sep 09 03:13:27.539685 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:13:27.539637    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-9t27t" event={"ID":"f0fa1f03-0dd2-4015-a21a-02fc04073778","Type":"ContainerStarted","Data":"3769dad22b1967c7aa992f139c17d1c09511807d8ce3d5d25b13bbf320b8533b"}
Sep 09 03:13:27.539685 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:13:27.539665    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-9t27t" event={"ID":"f0fa1f03-0dd2-4015-a21a-02fc04073778","Type":"ContainerStarted","Data":"8f1be88d67b5260fe2b8399fecc333b4d20d4ee4ec57f9b364091ba69f8e5465"}
Sep 09 03:13:27.540944 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:13:27.540925    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-j5fzd" event={"ID":"8c6c6ac7-ea75-45fd-ade0-a1767724b340","Type":"ContainerStarted","Data":"952986a298def9580cca68530e0cfdff0ad06e4b1dd1e98d34f4f582b2b69ad1"}
Sep 09 03:13:27.542327 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:13:27.542305    2576 generic.go:356] "Generic (PLEG): container finished" podID="aaf8534a-72e4-4c88-972e-26eacf111fa2" containerID="9f4dc5691a6f9b01d4a678283a117cf2f116b0f11fe659c7ea2bcdfd7ae5faf9" exitCode=0
Sep 09 03:13:27.542418 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:13:27.542381    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-qpxhx" event={"ID":"aaf8534a-72e4-4c88-972e-26eacf111fa2","Type":"ContainerDied","Data":"9f4dc5691a6f9b01d4a678283a117cf2f116b0f11fe659c7ea2bcdfd7ae5faf9"}
Sep 09 03:13:27.543839 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:13:27.543818    2576 generic.go:356] "Generic (PLEG): container finished" podID="3882ef2c4b5f745a591c85452d6b7dae" containerID="c262d9bf5a22a2977aa6fa1d5d498d1f9dc3591ced4575a0be6cb14a7e467c8d" exitCode=0
Sep 09 03:13:27.543925 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:13:27.543907    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-14.ec2.internal" event={"ID":"3882ef2c4b5f745a591c85452d6b7dae","Type":"ContainerDied","Data":"c262d9bf5a22a2977aa6fa1d5d498d1f9dc3591ced4575a0be6cb14a7e467c8d"}
Sep 09 03:13:27.549983 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:13:27.549944    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-2zp8k" podStartSLOduration=3.173338167 podStartE2EDuration="21.549931654s" totalImagesPullingTime="18.376593487s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-09 03:13:06 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 03:13:27.549564487 +0000 UTC m=+21.691050563" watchObservedRunningTime="2026-09-09 03:13:27.549931654 +0000 UTC m=+21.691417732"
Sep 09 03:13:27.580175 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:13:27.580130    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-j5fzd" podStartSLOduration=3.135041262 podStartE2EDuration="21.580114556s" totalImagesPullingTime="18.445073294s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-09 03:13:06 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 03:13:27.580073308 +0000 UTC m=+21.721559387" watchObservedRunningTime="2026-09-09 03:13:27.580114556 +0000 UTC m=+21.721600633"
Sep 09 03:13:27.596630 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:13:27.596586    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-c4rmf" podStartSLOduration=3.1742184509999998 podStartE2EDuration="21.596571176s" totalImagesPullingTime="18.422352725s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 03:13:06 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 03:13:27.596247362 +0000 UTC m=+21.737733440" watchObservedRunningTime="2026-09-09 03:13:27.596571176 +0000 UTC m=+21.738057253"
Sep 09 03:13:27.609566 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:13:27.609525    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-cqhff" podStartSLOduration=3.198505173 podStartE2EDuration="21.609512454s" totalImagesPullingTime="18.411007281s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-09 03:13:06 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 03:13:27.609100944 +0000 UTC m=+21.750587021" watchObservedRunningTime="2026-09-09 03:13:27.609512454 +0000 UTC m=+21.750998530"
Sep 09 03:13:27.657693 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:13:27.657536    2576 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock"
Sep 09 03:13:28.354900 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:13:28.354834    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-99qzh"
Sep 09 03:13:28.355080 ip-10-0-132-14 kubenswrapper[2576]: E0909 03:13:28.355007    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-99qzh" podUID="443a2987-5246-4a8c-8631-1b751e02101b"
Sep 09 03:13:28.355187 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:13:28.355166    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-s2pb7"
Sep 09 03:13:28.355291 ip-10-0-132-14 kubenswrapper[2576]: E0909 03:13:28.355272    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-s2pb7" podUID="344533be-8249-42e9-9abc-c0bc68e18132"
Sep 09 03:13:28.355494 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:13:28.355411    2576 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-09-09T03:13:27.657686925Z","UUID":"f86ee600-5609-4ad9-a1f2-5840db4ac92b","Handler":null,"Name":"","Endpoint":""}
Sep 09 03:13:28.357168 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:13:28.357146    2576 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0
Sep 09 03:13:28.357277 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:13:28.357181    2576 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock
Sep 09 03:13:28.549922 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:13:28.549848    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rhxhj" event={"ID":"61140997-9a63-4a79-ae4d-368d92a92960","Type":"ContainerStarted","Data":"5c72380b48a05e4bbb3a295a73d6a2f153999657559434cc8d9368d607d6f22e"}
Sep 09 03:13:28.552350 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:13:28.552318    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-14.ec2.internal" event={"ID":"3882ef2c4b5f745a591c85452d6b7dae","Type":"ContainerStarted","Data":"a5531b8801527a3c95344daea67030bb647b2e48eb238768822d38828a9f80df"}
Sep 09 03:13:28.572134 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:13:28.572075    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-14.ec2.internal" podStartSLOduration=21.57205787 podStartE2EDuration="21.57205787s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-09 03:13:07 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 03:13:28.571724628 +0000 UTC m=+22.713210704" watchObservedRunningTime="2026-09-09 03:13:28.57205787 +0000 UTC m=+22.713543949"
Sep 09 03:13:29.354498 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:13:29.354045    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-t9bbm"
Sep 09 03:13:29.354498 ip-10-0-132-14 kubenswrapper[2576]: E0909 03:13:29.354167    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-t9bbm" podUID="2f43cc28-7345-4c06-9160-4245b99697b4"
Sep 09 03:13:29.557645 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:13:29.557598    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rhxhj" event={"ID":"61140997-9a63-4a79-ae4d-368d92a92960","Type":"ContainerStarted","Data":"b5d9f20695f2cfb543abadd62d280454fa70bc3ff206204502e7d0382582c2f2"}
Sep 09 03:13:29.561567 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:13:29.561537    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-9t27t" event={"ID":"f0fa1f03-0dd2-4015-a21a-02fc04073778","Type":"ContainerStarted","Data":"aad9a60edd78836bd7a4fe74c9eb24efcb5f39c88419fa8ee0235204b69a29b9"}
Sep 09 03:13:29.575753 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:13:29.575699    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rhxhj" podStartSLOduration=3.422419611 podStartE2EDuration="23.575678198s" totalImagesPullingTime="20.153258587s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-09 03:13:06 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 03:13:29.575034349 +0000 UTC m=+23.716520426" watchObservedRunningTime="2026-09-09 03:13:29.575678198 +0000 UTC m=+23.717164275"
Sep 09 03:13:30.354369 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:13:30.354337    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-99qzh"
Sep 09 03:13:30.354570 ip-10-0-132-14 kubenswrapper[2576]: E0909 03:13:30.354491    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-99qzh" podUID="443a2987-5246-4a8c-8631-1b751e02101b"
Sep 09 03:13:30.354570 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:13:30.354513    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-s2pb7"
Sep 09 03:13:30.354710 ip-10-0-132-14 kubenswrapper[2576]: E0909 03:13:30.354686    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-s2pb7" podUID="344533be-8249-42e9-9abc-c0bc68e18132"
Sep 09 03:13:30.513227 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:13:30.513189    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-c4rmf"
Sep 09 03:13:30.513929 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:13:30.513907    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-c4rmf"
Sep 09 03:13:31.354546 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:13:31.354516    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-t9bbm"
Sep 09 03:13:31.354977 ip-10-0-132-14 kubenswrapper[2576]: E0909 03:13:31.354625    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-t9bbm" podUID="2f43cc28-7345-4c06-9160-4245b99697b4"
Sep 09 03:13:32.354999 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:13:32.354819    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-99qzh"
Sep 09 03:13:32.355368 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:13:32.354857    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-s2pb7"
Sep 09 03:13:32.355368 ip-10-0-132-14 kubenswrapper[2576]: E0909 03:13:32.355131    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-99qzh" podUID="443a2987-5246-4a8c-8631-1b751e02101b"
Sep 09 03:13:32.355368 ip-10-0-132-14 kubenswrapper[2576]: E0909 03:13:32.355229    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-s2pb7" podUID="344533be-8249-42e9-9abc-c0bc68e18132"
Sep 09 03:13:32.572355 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:13:32.572263    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-9t27t" event={"ID":"f0fa1f03-0dd2-4015-a21a-02fc04073778","Type":"ContainerStarted","Data":"2554c506b3ee30173b210703aaf3b712231c22c23d614399c825e1eda9c37b52"}
Sep 09 03:13:32.572587 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:13:32.572557    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-9t27t"
Sep 09 03:13:32.574182 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:13:32.574159    2576 generic.go:356] "Generic (PLEG): container finished" podID="aaf8534a-72e4-4c88-972e-26eacf111fa2" containerID="a2fa3c564090832d33888a3db0bb8dc67cff5af2ac2c8dd1454dff56351da88a" exitCode=0
Sep 09 03:13:32.574299 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:13:32.574192    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-qpxhx" event={"ID":"aaf8534a-72e4-4c88-972e-26eacf111fa2","Type":"ContainerDied","Data":"a2fa3c564090832d33888a3db0bb8dc67cff5af2ac2c8dd1454dff56351da88a"}
Sep 09 03:13:32.574668 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:13:32.574633    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-c4rmf"
Sep 09 03:13:32.574785 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:13:32.574730    2576 prober_manager.go:312] "Failed to trigger a manual run" probe="Readiness"
Sep 09 03:13:32.575367 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:13:32.575351    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-c4rmf"
Sep 09 03:13:32.588827 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:13:32.588784    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-9t27t"
Sep 09 03:13:32.600258 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:13:32.600182    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-9t27t" podStartSLOduration=8.152515987 podStartE2EDuration="26.600158526s" totalImagesPullingTime="18.447642539s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-09 03:13:06 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 03:13:32.600119236 +0000 UTC m=+26.741605312" watchObservedRunningTime="2026-09-09 03:13:32.600158526 +0000 UTC m=+26.741644596"
Sep 09 03:13:33.354645 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:13:33.354602    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-t9bbm"
Sep 09 03:13:33.354828 ip-10-0-132-14 kubenswrapper[2576]: E0909 03:13:33.354733    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-t9bbm" podUID="2f43cc28-7345-4c06-9160-4245b99697b4"
Sep 09 03:13:33.577261 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:13:33.576934    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-9t27t"
Sep 09 03:13:33.577261 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:13:33.577024    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-9t27t"
Sep 09 03:13:33.594401 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:13:33.594373    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-9t27t"
Sep 09 03:13:33.854097 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:13:33.854061    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-t9bbm"]
Sep 09 03:13:33.854250 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:13:33.854193    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-t9bbm"
Sep 09 03:13:33.854338 ip-10-0-132-14 kubenswrapper[2576]: E0909 03:13:33.854312    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-t9bbm" podUID="2f43cc28-7345-4c06-9160-4245b99697b4"
Sep 09 03:13:33.857669 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:13:33.857641    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-99qzh"]
Sep 09 03:13:33.857797 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:13:33.857697    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-s2pb7"]
Sep 09 03:13:33.857797 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:13:33.857769    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-s2pb7"
Sep 09 03:13:33.857931 ip-10-0-132-14 kubenswrapper[2576]: E0909 03:13:33.857878    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-s2pb7" podUID="344533be-8249-42e9-9abc-c0bc68e18132"
Sep 09 03:13:33.857931 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:13:33.857912    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-99qzh"
Sep 09 03:13:33.858027 ip-10-0-132-14 kubenswrapper[2576]: E0909 03:13:33.857999    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-99qzh" podUID="443a2987-5246-4a8c-8631-1b751e02101b"
Sep 09 03:13:34.580961 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:13:34.580927    2576 generic.go:356] "Generic (PLEG): container finished" podID="aaf8534a-72e4-4c88-972e-26eacf111fa2" containerID="eca0b86dcb11a598ff66b8babd115507bac5b61f60e9a673981d0e02157dd21e" exitCode=0
Sep 09 03:13:34.581435 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:13:34.581009    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-qpxhx" event={"ID":"aaf8534a-72e4-4c88-972e-26eacf111fa2","Type":"ContainerDied","Data":"eca0b86dcb11a598ff66b8babd115507bac5b61f60e9a673981d0e02157dd21e"}
Sep 09 03:13:35.354148 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:13:35.353947    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-s2pb7"
Sep 09 03:13:35.354316 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:13:35.353955    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-t9bbm"
Sep 09 03:13:35.354316 ip-10-0-132-14 kubenswrapper[2576]: E0909 03:13:35.354239    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-s2pb7" podUID="344533be-8249-42e9-9abc-c0bc68e18132"
Sep 09 03:13:35.354418 ip-10-0-132-14 kubenswrapper[2576]: E0909 03:13:35.354340    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-t9bbm" podUID="2f43cc28-7345-4c06-9160-4245b99697b4"
Sep 09 03:13:35.354418 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:13:35.354012    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-99qzh"
Sep 09 03:13:35.354519 ip-10-0-132-14 kubenswrapper[2576]: E0909 03:13:35.354452    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-99qzh" podUID="443a2987-5246-4a8c-8631-1b751e02101b"
Sep 09 03:13:35.585705 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:13:35.585672    2576 generic.go:356] "Generic (PLEG): container finished" podID="aaf8534a-72e4-4c88-972e-26eacf111fa2" containerID="86b5c8e86401abf5c4e4aa93907626124675a9b08a1cc46edaa12e876ea10896" exitCode=0
Sep 09 03:13:35.586090 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:13:35.585754    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-qpxhx" event={"ID":"aaf8534a-72e4-4c88-972e-26eacf111fa2","Type":"ContainerDied","Data":"86b5c8e86401abf5c4e4aa93907626124675a9b08a1cc46edaa12e876ea10896"}
Sep 09 03:13:37.354947 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:13:37.354908    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-s2pb7"
Sep 09 03:13:37.354947 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:13:37.354931    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-99qzh"
Sep 09 03:13:37.355491 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:13:37.354931    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-t9bbm"
Sep 09 03:13:37.355491 ip-10-0-132-14 kubenswrapper[2576]: E0909 03:13:37.355031    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-s2pb7" podUID="344533be-8249-42e9-9abc-c0bc68e18132"
Sep 09 03:13:37.355491 ip-10-0-132-14 kubenswrapper[2576]: E0909 03:13:37.355097    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-99qzh" podUID="443a2987-5246-4a8c-8631-1b751e02101b"
Sep 09 03:13:37.355491 ip-10-0-132-14 kubenswrapper[2576]: E0909 03:13:37.355152    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-t9bbm" podUID="2f43cc28-7345-4c06-9160-4245b99697b4"
Sep 09 03:13:39.023881 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:13:39.023825    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4dddl\" (UniqueName: \"kubernetes.io/projected/344533be-8249-42e9-9abc-c0bc68e18132-kube-api-access-4dddl\") pod \"network-check-target-s2pb7\" (UID: \"344533be-8249-42e9-9abc-c0bc68e18132\") " pod="openshift-network-diagnostics/network-check-target-s2pb7"
Sep 09 03:13:39.024346 ip-10-0-132-14 kubenswrapper[2576]: E0909 03:13:39.024022    2576 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 09 03:13:39.024346 ip-10-0-132-14 kubenswrapper[2576]: E0909 03:13:39.024047    2576 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 09 03:13:39.024346 ip-10-0-132-14 kubenswrapper[2576]: E0909 03:13:39.024060    2576 projected.go:196] Error preparing data for projected volume kube-api-access-4dddl for pod openshift-network-diagnostics/network-check-target-s2pb7: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 09 03:13:39.024346 ip-10-0-132-14 kubenswrapper[2576]: E0909 03:13:39.024131    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/344533be-8249-42e9-9abc-c0bc68e18132-kube-api-access-4dddl podName:344533be-8249-42e9-9abc-c0bc68e18132 nodeName:}" failed. No retries permitted until 2026-09-09 03:14:11.024109606 +0000 UTC m=+65.165595662 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-4dddl" (UniqueName: "kubernetes.io/projected/344533be-8249-42e9-9abc-c0bc68e18132-kube-api-access-4dddl") pod "network-check-target-s2pb7" (UID: "344533be-8249-42e9-9abc-c0bc68e18132") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 09 03:13:39.125189 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:13:39.125098    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/2f43cc28-7345-4c06-9160-4245b99697b4-original-pull-secret\") pod \"global-pull-secret-syncer-t9bbm\" (UID: \"2f43cc28-7345-4c06-9160-4245b99697b4\") " pod="kube-system/global-pull-secret-syncer-t9bbm"
Sep 09 03:13:39.125189 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:13:39.125144    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/443a2987-5246-4a8c-8631-1b751e02101b-metrics-certs\") pod \"network-metrics-daemon-99qzh\" (UID: \"443a2987-5246-4a8c-8631-1b751e02101b\") " pod="openshift-multus/network-metrics-daemon-99qzh"
Sep 09 03:13:39.125412 ip-10-0-132-14 kubenswrapper[2576]: E0909 03:13:39.125283    2576 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 09 03:13:39.125412 ip-10-0-132-14 kubenswrapper[2576]: E0909 03:13:39.125285    2576 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 09 03:13:39.125412 ip-10-0-132-14 kubenswrapper[2576]: E0909 03:13:39.125355    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/443a2987-5246-4a8c-8631-1b751e02101b-metrics-certs podName:443a2987-5246-4a8c-8631-1b751e02101b nodeName:}" failed. No retries permitted until 2026-09-09 03:14:11.125335683 +0000 UTC m=+65.266821743 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/443a2987-5246-4a8c-8631-1b751e02101b-metrics-certs") pod "network-metrics-daemon-99qzh" (UID: "443a2987-5246-4a8c-8631-1b751e02101b") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 09 03:13:39.125412 ip-10-0-132-14 kubenswrapper[2576]: E0909 03:13:39.125375    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2f43cc28-7345-4c06-9160-4245b99697b4-original-pull-secret podName:2f43cc28-7345-4c06-9160-4245b99697b4 nodeName:}" failed. No retries permitted until 2026-09-09 03:14:11.125364778 +0000 UTC m=+65.266850839 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/2f43cc28-7345-4c06-9160-4245b99697b4-original-pull-secret") pod "global-pull-secret-syncer-t9bbm" (UID: "2f43cc28-7345-4c06-9160-4245b99697b4") : object "kube-system"/"original-pull-secret" not registered
Sep 09 03:13:39.186337 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:13:39.186302    2576 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-14.ec2.internal" event="NodeReady"
Sep 09 03:13:39.186501 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:13:39.186463    2576 kubelet_node_status.go:489] "Fast updating node status as it just became ready"
Sep 09 03:13:39.242230 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:13:39.242191    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-6z6x2"]
Sep 09 03:13:39.271975 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:13:39.271946    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-227kg"]
Sep 09 03:13:39.272140 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:13:39.272052    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-6z6x2"
Sep 09 03:13:39.275693 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:13:39.275659    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-pxbsq\""
Sep 09 03:13:39.275855 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:13:39.275773    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\""
Sep 09 03:13:39.275855 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:13:39.275789    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\""
Sep 09 03:13:39.287791 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:13:39.287769    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-227kg"]
Sep 09 03:13:39.287918 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:13:39.287823    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-6z6x2"]
Sep 09 03:13:39.287918 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:13:39.287903    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-227kg"
Sep 09 03:13:39.291043 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:13:39.291019    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\""
Sep 09 03:13:39.291640 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:13:39.291622    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\""
Sep 09 03:13:39.291795 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:13:39.291778    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-5kdc9\""
Sep 09 03:13:39.291889 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:13:39.291874    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\""
Sep 09 03:13:39.326151 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:13:39.326121    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pssmc\" (UniqueName: \"kubernetes.io/projected/fd790ee2-f462-4e4f-ae65-f09b4d36d5bc-kube-api-access-pssmc\") pod \"dns-default-6z6x2\" (UID: \"fd790ee2-f462-4e4f-ae65-f09b4d36d5bc\") " pod="openshift-dns/dns-default-6z6x2"
Sep 09 03:13:39.326310 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:13:39.326190    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/fd790ee2-f462-4e4f-ae65-f09b4d36d5bc-metrics-tls\") pod \"dns-default-6z6x2\" (UID: \"fd790ee2-f462-4e4f-ae65-f09b4d36d5bc\") " pod="openshift-dns/dns-default-6z6x2"
Sep 09 03:13:39.326366 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:13:39.326299    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/fd790ee2-f462-4e4f-ae65-f09b4d36d5bc-config-volume\") pod \"dns-default-6z6x2\" (UID: \"fd790ee2-f462-4e4f-ae65-f09b4d36d5bc\") " pod="openshift-dns/dns-default-6z6x2"
Sep 09 03:13:39.326366 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:13:39.326335    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/fd790ee2-f462-4e4f-ae65-f09b4d36d5bc-tmp-dir\") pod \"dns-default-6z6x2\" (UID: \"fd790ee2-f462-4e4f-ae65-f09b4d36d5bc\") " pod="openshift-dns/dns-default-6z6x2"
Sep 09 03:13:39.354824 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:13:39.354759    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-t9bbm"
Sep 09 03:13:39.354824 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:13:39.354825    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-s2pb7"
Sep 09 03:13:39.355142 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:13:39.354907    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-99qzh"
Sep 09 03:13:39.357873 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:13:39.357848    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Sep 09 03:13:39.357873 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:13:39.357870    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Sep 09 03:13:39.358267 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:13:39.358246    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Sep 09 03:13:39.358392 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:13:39.358366    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Sep 09 03:13:39.358504 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:13:39.358424    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-g4cct\""
Sep 09 03:13:39.358727 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:13:39.358708    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-rkcrq\""
Sep 09 03:13:39.427566 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:13:39.427529    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/fd790ee2-f462-4e4f-ae65-f09b4d36d5bc-config-volume\") pod \"dns-default-6z6x2\" (UID: \"fd790ee2-f462-4e4f-ae65-f09b4d36d5bc\") " pod="openshift-dns/dns-default-6z6x2"
Sep 09 03:13:39.427749 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:13:39.427574    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/fd790ee2-f462-4e4f-ae65-f09b4d36d5bc-tmp-dir\") pod \"dns-default-6z6x2\" (UID: \"fd790ee2-f462-4e4f-ae65-f09b4d36d5bc\") " pod="openshift-dns/dns-default-6z6x2"
Sep 09 03:13:39.427749 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:13:39.427605    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/3862bef2-07ad-41d7-978c-3f6f5c264854-cert\") pod \"ingress-canary-227kg\" (UID: \"3862bef2-07ad-41d7-978c-3f6f5c264854\") " pod="openshift-ingress-canary/ingress-canary-227kg"
Sep 09 03:13:39.427749 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:13:39.427632    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4n6xp\" (UniqueName: \"kubernetes.io/projected/3862bef2-07ad-41d7-978c-3f6f5c264854-kube-api-access-4n6xp\") pod \"ingress-canary-227kg\" (UID: \"3862bef2-07ad-41d7-978c-3f6f5c264854\") " pod="openshift-ingress-canary/ingress-canary-227kg"
Sep 09 03:13:39.427749 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:13:39.427715    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pssmc\" (UniqueName: \"kubernetes.io/projected/fd790ee2-f462-4e4f-ae65-f09b4d36d5bc-kube-api-access-pssmc\") pod \"dns-default-6z6x2\" (UID: \"fd790ee2-f462-4e4f-ae65-f09b4d36d5bc\") " pod="openshift-dns/dns-default-6z6x2"
Sep 09 03:13:39.428035 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:13:39.427768    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/fd790ee2-f462-4e4f-ae65-f09b4d36d5bc-metrics-tls\") pod \"dns-default-6z6x2\" (UID: \"fd790ee2-f462-4e4f-ae65-f09b4d36d5bc\") " pod="openshift-dns/dns-default-6z6x2"
Sep 09 03:13:39.428035 ip-10-0-132-14 kubenswrapper[2576]: E0909 03:13:39.427892    2576 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 09 03:13:39.428035 ip-10-0-132-14 kubenswrapper[2576]: E0909 03:13:39.427965    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/fd790ee2-f462-4e4f-ae65-f09b4d36d5bc-metrics-tls podName:fd790ee2-f462-4e4f-ae65-f09b4d36d5bc nodeName:}" failed. No retries permitted until 2026-09-09 03:13:39.927944348 +0000 UTC m=+34.069430416 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/fd790ee2-f462-4e4f-ae65-f09b4d36d5bc-metrics-tls") pod "dns-default-6z6x2" (UID: "fd790ee2-f462-4e4f-ae65-f09b4d36d5bc") : secret "dns-default-metrics-tls" not found
Sep 09 03:13:39.428035 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:13:39.427969    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/fd790ee2-f462-4e4f-ae65-f09b4d36d5bc-tmp-dir\") pod \"dns-default-6z6x2\" (UID: \"fd790ee2-f462-4e4f-ae65-f09b4d36d5bc\") " pod="openshift-dns/dns-default-6z6x2"
Sep 09 03:13:39.428221 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:13:39.428172    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/fd790ee2-f462-4e4f-ae65-f09b4d36d5bc-config-volume\") pod \"dns-default-6z6x2\" (UID: \"fd790ee2-f462-4e4f-ae65-f09b4d36d5bc\") " pod="openshift-dns/dns-default-6z6x2"
Sep 09 03:13:39.440524 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:13:39.440501    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pssmc\" (UniqueName: \"kubernetes.io/projected/fd790ee2-f462-4e4f-ae65-f09b4d36d5bc-kube-api-access-pssmc\") pod \"dns-default-6z6x2\" (UID: \"fd790ee2-f462-4e4f-ae65-f09b4d36d5bc\") " pod="openshift-dns/dns-default-6z6x2"
Sep 09 03:13:39.528733 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:13:39.528680    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/3862bef2-07ad-41d7-978c-3f6f5c264854-cert\") pod \"ingress-canary-227kg\" (UID: \"3862bef2-07ad-41d7-978c-3f6f5c264854\") " pod="openshift-ingress-canary/ingress-canary-227kg"
Sep 09 03:13:39.528945 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:13:39.528843    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4n6xp\" (UniqueName: \"kubernetes.io/projected/3862bef2-07ad-41d7-978c-3f6f5c264854-kube-api-access-4n6xp\") pod \"ingress-canary-227kg\" (UID: \"3862bef2-07ad-41d7-978c-3f6f5c264854\") " pod="openshift-ingress-canary/ingress-canary-227kg"
Sep 09 03:13:39.531401 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:13:39.531379    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/3862bef2-07ad-41d7-978c-3f6f5c264854-cert\") pod \"ingress-canary-227kg\" (UID: \"3862bef2-07ad-41d7-978c-3f6f5c264854\") " pod="openshift-ingress-canary/ingress-canary-227kg"
Sep 09 03:13:39.537501 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:13:39.537480    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4n6xp\" (UniqueName: \"kubernetes.io/projected/3862bef2-07ad-41d7-978c-3f6f5c264854-kube-api-access-4n6xp\") pod \"ingress-canary-227kg\" (UID: \"3862bef2-07ad-41d7-978c-3f6f5c264854\") " pod="openshift-ingress-canary/ingress-canary-227kg"
Sep 09 03:13:39.598931 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:13:39.598899    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-227kg"
Sep 09 03:13:39.931791 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:13:39.931751    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/fd790ee2-f462-4e4f-ae65-f09b4d36d5bc-metrics-tls\") pod \"dns-default-6z6x2\" (UID: \"fd790ee2-f462-4e4f-ae65-f09b4d36d5bc\") " pod="openshift-dns/dns-default-6z6x2"
Sep 09 03:13:39.932019 ip-10-0-132-14 kubenswrapper[2576]: E0909 03:13:39.931915    2576 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 09 03:13:39.932019 ip-10-0-132-14 kubenswrapper[2576]: E0909 03:13:39.932007    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/fd790ee2-f462-4e4f-ae65-f09b4d36d5bc-metrics-tls podName:fd790ee2-f462-4e4f-ae65-f09b4d36d5bc nodeName:}" failed. No retries permitted until 2026-09-09 03:13:40.931984918 +0000 UTC m=+35.073470974 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/fd790ee2-f462-4e4f-ae65-f09b4d36d5bc-metrics-tls") pod "dns-default-6z6x2" (UID: "fd790ee2-f462-4e4f-ae65-f09b4d36d5bc") : secret "dns-default-metrics-tls" not found
Sep 09 03:13:40.020449 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:13:40.020213    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-227kg"]
Sep 09 03:13:40.023103 ip-10-0-132-14 kubenswrapper[2576]: W0909 03:13:40.023082    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3862bef2_07ad_41d7_978c_3f6f5c264854.slice/crio-c3eb4a2c42073f9dac8220e4e3a90899e5139db8c205939fae0952c91488fe31 WatchSource:0}: Error finding container c3eb4a2c42073f9dac8220e4e3a90899e5139db8c205939fae0952c91488fe31: Status 404 returned error can't find the container with id c3eb4a2c42073f9dac8220e4e3a90899e5139db8c205939fae0952c91488fe31
Sep 09 03:13:40.601099 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:13:40.601061    2576 generic.go:356] "Generic (PLEG): container finished" podID="aaf8534a-72e4-4c88-972e-26eacf111fa2" containerID="de1fc24c8bf22589ae445239f4a190015b605ddde3b0947d6fcccc9647821c60" exitCode=0
Sep 09 03:13:40.601594 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:13:40.601148    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-qpxhx" event={"ID":"aaf8534a-72e4-4c88-972e-26eacf111fa2","Type":"ContainerDied","Data":"de1fc24c8bf22589ae445239f4a190015b605ddde3b0947d6fcccc9647821c60"}
Sep 09 03:13:40.602918 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:13:40.602882    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-227kg" event={"ID":"3862bef2-07ad-41d7-978c-3f6f5c264854","Type":"ContainerStarted","Data":"c3eb4a2c42073f9dac8220e4e3a90899e5139db8c205939fae0952c91488fe31"}
Sep 09 03:13:40.940701 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:13:40.940603    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/fd790ee2-f462-4e4f-ae65-f09b4d36d5bc-metrics-tls\") pod \"dns-default-6z6x2\" (UID: \"fd790ee2-f462-4e4f-ae65-f09b4d36d5bc\") " pod="openshift-dns/dns-default-6z6x2"
Sep 09 03:13:40.940903 ip-10-0-132-14 kubenswrapper[2576]: E0909 03:13:40.940777    2576 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 09 03:13:40.940977 ip-10-0-132-14 kubenswrapper[2576]: E0909 03:13:40.940938    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/fd790ee2-f462-4e4f-ae65-f09b4d36d5bc-metrics-tls podName:fd790ee2-f462-4e4f-ae65-f09b4d36d5bc nodeName:}" failed. No retries permitted until 2026-09-09 03:13:42.940912623 +0000 UTC m=+37.082398689 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/fd790ee2-f462-4e4f-ae65-f09b4d36d5bc-metrics-tls") pod "dns-default-6z6x2" (UID: "fd790ee2-f462-4e4f-ae65-f09b4d36d5bc") : secret "dns-default-metrics-tls" not found
Sep 09 03:13:41.609742 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:13:41.609698    2576 generic.go:356] "Generic (PLEG): container finished" podID="aaf8534a-72e4-4c88-972e-26eacf111fa2" containerID="1812814334c48ea4361a538727f361b391ec316e7f162e2bddc4a02c516810c4" exitCode=0
Sep 09 03:13:41.610328 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:13:41.609772    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-qpxhx" event={"ID":"aaf8534a-72e4-4c88-972e-26eacf111fa2","Type":"ContainerDied","Data":"1812814334c48ea4361a538727f361b391ec316e7f162e2bddc4a02c516810c4"}
Sep 09 03:13:42.615063 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:13:42.615032    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-qpxhx" event={"ID":"aaf8534a-72e4-4c88-972e-26eacf111fa2","Type":"ContainerStarted","Data":"44a8e890ce702df5979d1ff0c57348045a9ccf1b357ed23cbf99c26b3370e05a"}
Sep 09 03:13:42.638870 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:13:42.638794    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-qpxhx" podStartSLOduration=7.496644028 podStartE2EDuration="36.638773376s" totalImagesPullingTime="29.142129348s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-09 03:13:06 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 03:13:42.638347762 +0000 UTC m=+36.779833840" watchObservedRunningTime="2026-09-09 03:13:42.638773376 +0000 UTC m=+36.780259455"
Sep 09 03:13:42.957463 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:13:42.957379    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/fd790ee2-f462-4e4f-ae65-f09b4d36d5bc-metrics-tls\") pod \"dns-default-6z6x2\" (UID: \"fd790ee2-f462-4e4f-ae65-f09b4d36d5bc\") " pod="openshift-dns/dns-default-6z6x2"
Sep 09 03:13:42.957605 ip-10-0-132-14 kubenswrapper[2576]: E0909 03:13:42.957509    2576 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 09 03:13:42.957605 ip-10-0-132-14 kubenswrapper[2576]: E0909 03:13:42.957565    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/fd790ee2-f462-4e4f-ae65-f09b4d36d5bc-metrics-tls podName:fd790ee2-f462-4e4f-ae65-f09b4d36d5bc nodeName:}" failed. No retries permitted until 2026-09-09 03:13:46.957551558 +0000 UTC m=+41.099037614 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/fd790ee2-f462-4e4f-ae65-f09b4d36d5bc-metrics-tls") pod "dns-default-6z6x2" (UID: "fd790ee2-f462-4e4f-ae65-f09b4d36d5bc") : secret "dns-default-metrics-tls" not found
Sep 09 03:13:43.619162 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:13:43.619122    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-227kg" event={"ID":"3862bef2-07ad-41d7-978c-3f6f5c264854","Type":"ContainerStarted","Data":"3f7a6b65d538b08f452d7a27496177de2e1bf3d15f38dc4142ff71a6a24a0d18"}
Sep 09 03:13:43.634367 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:13:43.634300    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-227kg" podStartSLOduration=2.131002872 podStartE2EDuration="4.63427882s" totalImagesPullingTime="2.503275948s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 03:13:39 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 03:13:43.633978143 +0000 UTC m=+37.775464220" watchObservedRunningTime="2026-09-09 03:13:43.63427882 +0000 UTC m=+37.775764898"
Sep 09 03:13:46.985975 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:13:46.985933    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/fd790ee2-f462-4e4f-ae65-f09b4d36d5bc-metrics-tls\") pod \"dns-default-6z6x2\" (UID: \"fd790ee2-f462-4e4f-ae65-f09b4d36d5bc\") " pod="openshift-dns/dns-default-6z6x2"
Sep 09 03:13:46.986361 ip-10-0-132-14 kubenswrapper[2576]: E0909 03:13:46.986085    2576 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 09 03:13:46.986361 ip-10-0-132-14 kubenswrapper[2576]: E0909 03:13:46.986155    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/fd790ee2-f462-4e4f-ae65-f09b4d36d5bc-metrics-tls podName:fd790ee2-f462-4e4f-ae65-f09b4d36d5bc nodeName:}" failed. No retries permitted until 2026-09-09 03:13:54.986138219 +0000 UTC m=+49.127624274 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/fd790ee2-f462-4e4f-ae65-f09b4d36d5bc-metrics-tls") pod "dns-default-6z6x2" (UID: "fd790ee2-f462-4e4f-ae65-f09b4d36d5bc") : secret "dns-default-metrics-tls" not found
Sep 09 03:13:55.042308 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:13:55.042270    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/fd790ee2-f462-4e4f-ae65-f09b4d36d5bc-metrics-tls\") pod \"dns-default-6z6x2\" (UID: \"fd790ee2-f462-4e4f-ae65-f09b4d36d5bc\") " pod="openshift-dns/dns-default-6z6x2"
Sep 09 03:13:55.042722 ip-10-0-132-14 kubenswrapper[2576]: E0909 03:13:55.042420    2576 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 09 03:13:55.042722 ip-10-0-132-14 kubenswrapper[2576]: E0909 03:13:55.042505    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/fd790ee2-f462-4e4f-ae65-f09b4d36d5bc-metrics-tls podName:fd790ee2-f462-4e4f-ae65-f09b4d36d5bc nodeName:}" failed. No retries permitted until 2026-09-09 03:14:11.042488723 +0000 UTC m=+65.183974779 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/fd790ee2-f462-4e4f-ae65-f09b4d36d5bc-metrics-tls") pod "dns-default-6z6x2" (UID: "fd790ee2-f462-4e4f-ae65-f09b4d36d5bc") : secret "dns-default-metrics-tls" not found
Sep 09 03:14:05.597165 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:14:05.597131    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-9t27t"
Sep 09 03:14:11.058840 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:14:11.058782    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/fd790ee2-f462-4e4f-ae65-f09b4d36d5bc-metrics-tls\") pod \"dns-default-6z6x2\" (UID: \"fd790ee2-f462-4e4f-ae65-f09b4d36d5bc\") " pod="openshift-dns/dns-default-6z6x2"
Sep 09 03:14:11.059285 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:14:11.058853    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4dddl\" (UniqueName: \"kubernetes.io/projected/344533be-8249-42e9-9abc-c0bc68e18132-kube-api-access-4dddl\") pod \"network-check-target-s2pb7\" (UID: \"344533be-8249-42e9-9abc-c0bc68e18132\") " pod="openshift-network-diagnostics/network-check-target-s2pb7"
Sep 09 03:14:11.059285 ip-10-0-132-14 kubenswrapper[2576]: E0909 03:14:11.058950    2576 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 09 03:14:11.059285 ip-10-0-132-14 kubenswrapper[2576]: E0909 03:14:11.059016    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/fd790ee2-f462-4e4f-ae65-f09b4d36d5bc-metrics-tls podName:fd790ee2-f462-4e4f-ae65-f09b4d36d5bc nodeName:}" failed. No retries permitted until 2026-09-09 03:14:43.058996057 +0000 UTC m=+97.200482111 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/fd790ee2-f462-4e4f-ae65-f09b4d36d5bc-metrics-tls") pod "dns-default-6z6x2" (UID: "fd790ee2-f462-4e4f-ae65-f09b4d36d5bc") : secret "dns-default-metrics-tls" not found
Sep 09 03:14:11.062497 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:14:11.062478    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Sep 09 03:14:11.071972 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:14:11.071952    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Sep 09 03:14:11.083436 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:14:11.083413    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4dddl\" (UniqueName: \"kubernetes.io/projected/344533be-8249-42e9-9abc-c0bc68e18132-kube-api-access-4dddl\") pod \"network-check-target-s2pb7\" (UID: \"344533be-8249-42e9-9abc-c0bc68e18132\") " pod="openshift-network-diagnostics/network-check-target-s2pb7"
Sep 09 03:14:11.159509 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:14:11.159458    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/2f43cc28-7345-4c06-9160-4245b99697b4-original-pull-secret\") pod \"global-pull-secret-syncer-t9bbm\" (UID: \"2f43cc28-7345-4c06-9160-4245b99697b4\") " pod="kube-system/global-pull-secret-syncer-t9bbm"
Sep 09 03:14:11.159656 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:14:11.159519    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/443a2987-5246-4a8c-8631-1b751e02101b-metrics-certs\") pod \"network-metrics-daemon-99qzh\" (UID: \"443a2987-5246-4a8c-8631-1b751e02101b\") " pod="openshift-multus/network-metrics-daemon-99qzh"
Sep 09 03:14:11.162410 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:14:11.162388    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Sep 09 03:14:11.162486 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:14:11.162396    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Sep 09 03:14:11.169966 ip-10-0-132-14 kubenswrapper[2576]: E0909 03:14:11.169945    2576 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found
Sep 09 03:14:11.170040 ip-10-0-132-14 kubenswrapper[2576]: E0909 03:14:11.170024    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/443a2987-5246-4a8c-8631-1b751e02101b-metrics-certs podName:443a2987-5246-4a8c-8631-1b751e02101b nodeName:}" failed. No retries permitted until 2026-09-09 03:15:15.1700058 +0000 UTC m=+129.311491867 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/443a2987-5246-4a8c-8631-1b751e02101b-metrics-certs") pod "network-metrics-daemon-99qzh" (UID: "443a2987-5246-4a8c-8631-1b751e02101b") : secret "metrics-daemon-secret" not found
Sep 09 03:14:11.171124 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:14:11.171095    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-g4cct\""
Sep 09 03:14:11.172234 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:14:11.172216    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/2f43cc28-7345-4c06-9160-4245b99697b4-original-pull-secret\") pod \"global-pull-secret-syncer-t9bbm\" (UID: \"2f43cc28-7345-4c06-9160-4245b99697b4\") " pod="kube-system/global-pull-secret-syncer-t9bbm"
Sep 09 03:14:11.173431 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:14:11.173416    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-t9bbm"
Sep 09 03:14:11.179069 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:14:11.179052    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-s2pb7"
Sep 09 03:14:11.447595 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:14:11.447520    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-t9bbm"]
Sep 09 03:14:11.450636 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:14:11.450607    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-s2pb7"]
Sep 09 03:14:11.451092 ip-10-0-132-14 kubenswrapper[2576]: W0909 03:14:11.451060    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2f43cc28_7345_4c06_9160_4245b99697b4.slice/crio-aea34732d8b7da1605b5a243a1e92bfb2ddb6fdae75042fb1a285c6ad4a5b1e1 WatchSource:0}: Error finding container aea34732d8b7da1605b5a243a1e92bfb2ddb6fdae75042fb1a285c6ad4a5b1e1: Status 404 returned error can't find the container with id aea34732d8b7da1605b5a243a1e92bfb2ddb6fdae75042fb1a285c6ad4a5b1e1
Sep 09 03:14:11.453561 ip-10-0-132-14 kubenswrapper[2576]: W0909 03:14:11.453537    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod344533be_8249_42e9_9abc_c0bc68e18132.slice/crio-13d5ecd1fbd859fe1f87a7493f90f0627de41a229c6be35b9a8913df4313fed4 WatchSource:0}: Error finding container 13d5ecd1fbd859fe1f87a7493f90f0627de41a229c6be35b9a8913df4313fed4: Status 404 returned error can't find the container with id 13d5ecd1fbd859fe1f87a7493f90f0627de41a229c6be35b9a8913df4313fed4
Sep 09 03:14:11.685192 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:14:11.685154    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-s2pb7" event={"ID":"344533be-8249-42e9-9abc-c0bc68e18132","Type":"ContainerStarted","Data":"13d5ecd1fbd859fe1f87a7493f90f0627de41a229c6be35b9a8913df4313fed4"}
Sep 09 03:14:11.686249 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:14:11.686226    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-t9bbm" event={"ID":"2f43cc28-7345-4c06-9160-4245b99697b4","Type":"ContainerStarted","Data":"aea34732d8b7da1605b5a243a1e92bfb2ddb6fdae75042fb1a285c6ad4a5b1e1"}
Sep 09 03:14:16.701996 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:14:16.701886    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-s2pb7" event={"ID":"344533be-8249-42e9-9abc-c0bc68e18132","Type":"ContainerStarted","Data":"0e635f92d81a1a6eebb749c67781f6dc0894c074253e9b7d28433f2338ba2597"}
Sep 09 03:14:16.701996 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:14:16.701959    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-s2pb7"
Sep 09 03:14:16.703371 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:14:16.703346    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-t9bbm" event={"ID":"2f43cc28-7345-4c06-9160-4245b99697b4","Type":"ContainerStarted","Data":"6cd4792a069eda03a5e69256f723af54777e1cf87f8adaade191a0e205b53dbc"}
Sep 09 03:14:16.759146 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:14:16.759093    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-s2pb7" podStartSLOduration=65.914945809 podStartE2EDuration="1m10.759077991s" totalImagesPullingTime="4.844132182s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 03:13:06 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 03:14:16.732426162 +0000 UTC m=+70.873912238" watchObservedRunningTime="2026-09-09 03:14:16.759077991 +0000 UTC m=+70.900564067"
Sep 09 03:14:16.759582 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:14:16.759560    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-t9bbm" podStartSLOduration=65.901889968 podStartE2EDuration="1m10.759553731s" totalImagesPullingTime="4.857663763s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-09 03:13:06 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 03:14:16.75890749 +0000 UTC m=+70.900393569" watchObservedRunningTime="2026-09-09 03:14:16.759553731 +0000 UTC m=+70.901039807"
Sep 09 03:14:43.064907 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:14:43.064849    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/fd790ee2-f462-4e4f-ae65-f09b4d36d5bc-metrics-tls\") pod \"dns-default-6z6x2\" (UID: \"fd790ee2-f462-4e4f-ae65-f09b4d36d5bc\") " pod="openshift-dns/dns-default-6z6x2"
Sep 09 03:14:43.065392 ip-10-0-132-14 kubenswrapper[2576]: E0909 03:14:43.064997    2576 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 09 03:14:43.065392 ip-10-0-132-14 kubenswrapper[2576]: E0909 03:14:43.065078    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/fd790ee2-f462-4e4f-ae65-f09b4d36d5bc-metrics-tls podName:fd790ee2-f462-4e4f-ae65-f09b4d36d5bc nodeName:}" failed. No retries permitted until 2026-09-09 03:15:47.065058939 +0000 UTC m=+161.206544994 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/fd790ee2-f462-4e4f-ae65-f09b4d36d5bc-metrics-tls") pod "dns-default-6z6x2" (UID: "fd790ee2-f462-4e4f-ae65-f09b4d36d5bc") : secret "dns-default-metrics-tls" not found
Sep 09 03:14:47.708590 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:14:47.708555    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-s2pb7"
Sep 09 03:15:00.993693 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:15:00.993659    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-775666786-94lvs"]
Sep 09 03:15:00.996360 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:15:00.996340    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-7df6889b48-4bl8z"]
Sep 09 03:15:00.996512 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:15:00.996491    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-775666786-94lvs"
Sep 09 03:15:00.999014 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:15:00.998994    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-7df6889b48-4bl8z"
Sep 09 03:15:00.999482 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:15:00.999465    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-open-cluster-management.io-proxy-agent-signer-client-cert\""
Sep 09 03:15:00.999584 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:15:00.999565    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"kube-root-ca.crt\""
Sep 09 03:15:00.999686 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:15:00.999672    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-hub-kubeconfig\""
Sep 09 03:15:01.000651 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:15:01.000634    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-ca\""
Sep 09 03:15:01.000732 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:15:01.000634    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"openshift-service-ca.crt\""
Sep 09 03:15:01.000732 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:15:01.000638    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"open-cluster-management-image-pull-credentials\""
Sep 09 03:15:01.000732 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:15:01.000680    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-service-proxy-server-certificates\""
Sep 09 03:15:01.001428 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:15:01.001410    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"work-manager-hub-kubeconfig\""
Sep 09 03:15:01.007490 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:15:01.007473    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-7df6889b48-4bl8z"]
Sep 09 03:15:01.010090 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:15:01.010071    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-775666786-94lvs"]
Sep 09 03:15:01.076983 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:15:01.076950    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub\" (UniqueName: \"kubernetes.io/secret/33582a33-efef-482b-95f3-aabe6c109fb6-hub\") pod \"cluster-proxy-proxy-agent-775666786-94lvs\" (UID: \"33582a33-efef-482b-95f3-aabe6c109fb6\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-775666786-94lvs"
Sep 09 03:15:01.077135 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:15:01.076991    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/33582a33-efef-482b-95f3-aabe6c109fb6-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-775666786-94lvs\" (UID: \"33582a33-efef-482b-95f3-aabe6c109fb6\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-775666786-94lvs"
Sep 09 03:15:01.077135 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:15:01.077033    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca\" (UniqueName: \"kubernetes.io/secret/33582a33-efef-482b-95f3-aabe6c109fb6-ca\") pod \"cluster-proxy-proxy-agent-775666786-94lvs\" (UID: \"33582a33-efef-482b-95f3-aabe6c109fb6\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-775666786-94lvs"
Sep 09 03:15:01.077135 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:15:01.077048    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-w4vg2\" (UniqueName: \"kubernetes.io/projected/33582a33-efef-482b-95f3-aabe6c109fb6-kube-api-access-w4vg2\") pod \"cluster-proxy-proxy-agent-775666786-94lvs\" (UID: \"33582a33-efef-482b-95f3-aabe6c109fb6\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-775666786-94lvs"
Sep 09 03:15:01.077135 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:15:01.077071    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cjq28\" (UniqueName: \"kubernetes.io/projected/225fd920-7010-4b6d-b7c1-ae84d877bb1d-kube-api-access-cjq28\") pod \"klusterlet-addon-workmgr-7df6889b48-4bl8z\" (UID: \"225fd920-7010-4b6d-b7c1-ae84d877bb1d\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-7df6889b48-4bl8z"
Sep 09 03:15:01.077135 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:15:01.077090    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/33582a33-efef-482b-95f3-aabe6c109fb6-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-775666786-94lvs\" (UID: \"33582a33-efef-482b-95f3-aabe6c109fb6\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-775666786-94lvs"
Sep 09 03:15:01.077135 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:15:01.077107    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/225fd920-7010-4b6d-b7c1-ae84d877bb1d-klusterlet-config\") pod \"klusterlet-addon-workmgr-7df6889b48-4bl8z\" (UID: \"225fd920-7010-4b6d-b7c1-ae84d877bb1d\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-7df6889b48-4bl8z"
Sep 09 03:15:01.077135 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:15:01.077121    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/225fd920-7010-4b6d-b7c1-ae84d877bb1d-tmp\") pod \"klusterlet-addon-workmgr-7df6889b48-4bl8z\" (UID: \"225fd920-7010-4b6d-b7c1-ae84d877bb1d\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-7df6889b48-4bl8z"
Sep 09 03:15:01.077135 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:15:01.077134    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"additional-service-ca\" (UniqueName: \"kubernetes.io/configmap/33582a33-efef-482b-95f3-aabe6c109fb6-additional-service-ca\") pod \"cluster-proxy-proxy-agent-775666786-94lvs\" (UID: \"33582a33-efef-482b-95f3-aabe6c109fb6\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-775666786-94lvs"
Sep 09 03:15:01.177731 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:15:01.177702    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cjq28\" (UniqueName: \"kubernetes.io/projected/225fd920-7010-4b6d-b7c1-ae84d877bb1d-kube-api-access-cjq28\") pod \"klusterlet-addon-workmgr-7df6889b48-4bl8z\" (UID: \"225fd920-7010-4b6d-b7c1-ae84d877bb1d\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-7df6889b48-4bl8z"
Sep 09 03:15:01.177829 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:15:01.177737    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/33582a33-efef-482b-95f3-aabe6c109fb6-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-775666786-94lvs\" (UID: \"33582a33-efef-482b-95f3-aabe6c109fb6\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-775666786-94lvs"
Sep 09 03:15:01.177829 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:15:01.177756    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/225fd920-7010-4b6d-b7c1-ae84d877bb1d-klusterlet-config\") pod \"klusterlet-addon-workmgr-7df6889b48-4bl8z\" (UID: \"225fd920-7010-4b6d-b7c1-ae84d877bb1d\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-7df6889b48-4bl8z"
Sep 09 03:15:01.177908 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:15:01.177881    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/225fd920-7010-4b6d-b7c1-ae84d877bb1d-tmp\") pod \"klusterlet-addon-workmgr-7df6889b48-4bl8z\" (UID: \"225fd920-7010-4b6d-b7c1-ae84d877bb1d\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-7df6889b48-4bl8z"
Sep 09 03:15:01.177950 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:15:01.177909    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"additional-service-ca\" (UniqueName: \"kubernetes.io/configmap/33582a33-efef-482b-95f3-aabe6c109fb6-additional-service-ca\") pod \"cluster-proxy-proxy-agent-775666786-94lvs\" (UID: \"33582a33-efef-482b-95f3-aabe6c109fb6\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-775666786-94lvs"
Sep 09 03:15:01.177950 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:15:01.177939    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub\" (UniqueName: \"kubernetes.io/secret/33582a33-efef-482b-95f3-aabe6c109fb6-hub\") pod \"cluster-proxy-proxy-agent-775666786-94lvs\" (UID: \"33582a33-efef-482b-95f3-aabe6c109fb6\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-775666786-94lvs"
Sep 09 03:15:01.178041 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:15:01.177986    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/33582a33-efef-482b-95f3-aabe6c109fb6-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-775666786-94lvs\" (UID: \"33582a33-efef-482b-95f3-aabe6c109fb6\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-775666786-94lvs"
Sep 09 03:15:01.178096 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:15:01.178052    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca\" (UniqueName: \"kubernetes.io/secret/33582a33-efef-482b-95f3-aabe6c109fb6-ca\") pod \"cluster-proxy-proxy-agent-775666786-94lvs\" (UID: \"33582a33-efef-482b-95f3-aabe6c109fb6\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-775666786-94lvs"
Sep 09 03:15:01.178357 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:15:01.178198    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-w4vg2\" (UniqueName: \"kubernetes.io/projected/33582a33-efef-482b-95f3-aabe6c109fb6-kube-api-access-w4vg2\") pod \"cluster-proxy-proxy-agent-775666786-94lvs\" (UID: \"33582a33-efef-482b-95f3-aabe6c109fb6\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-775666786-94lvs"
Sep 09 03:15:01.178357 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:15:01.178308    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/225fd920-7010-4b6d-b7c1-ae84d877bb1d-tmp\") pod \"klusterlet-addon-workmgr-7df6889b48-4bl8z\" (UID: \"225fd920-7010-4b6d-b7c1-ae84d877bb1d\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-7df6889b48-4bl8z"
Sep 09 03:15:01.178852 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:15:01.178828    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"additional-service-ca\" (UniqueName: \"kubernetes.io/configmap/33582a33-efef-482b-95f3-aabe6c109fb6-additional-service-ca\") pod \"cluster-proxy-proxy-agent-775666786-94lvs\" (UID: \"33582a33-efef-482b-95f3-aabe6c109fb6\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-775666786-94lvs"
Sep 09 03:15:01.180366 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:15:01.180347    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/33582a33-efef-482b-95f3-aabe6c109fb6-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-775666786-94lvs\" (UID: \"33582a33-efef-482b-95f3-aabe6c109fb6\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-775666786-94lvs"
Sep 09 03:15:01.181030 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:15:01.181013    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca\" (UniqueName: \"kubernetes.io/secret/33582a33-efef-482b-95f3-aabe6c109fb6-ca\") pod \"cluster-proxy-proxy-agent-775666786-94lvs\" (UID: \"33582a33-efef-482b-95f3-aabe6c109fb6\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-775666786-94lvs"
Sep 09 03:15:01.181113 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:15:01.181075    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/33582a33-efef-482b-95f3-aabe6c109fb6-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-775666786-94lvs\" (UID: \"33582a33-efef-482b-95f3-aabe6c109fb6\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-775666786-94lvs"
Sep 09 03:15:01.181113 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:15:01.181086    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/225fd920-7010-4b6d-b7c1-ae84d877bb1d-klusterlet-config\") pod \"klusterlet-addon-workmgr-7df6889b48-4bl8z\" (UID: \"225fd920-7010-4b6d-b7c1-ae84d877bb1d\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-7df6889b48-4bl8z"
Sep 09 03:15:01.181113 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:15:01.181092    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hub\" (UniqueName: \"kubernetes.io/secret/33582a33-efef-482b-95f3-aabe6c109fb6-hub\") pod \"cluster-proxy-proxy-agent-775666786-94lvs\" (UID: \"33582a33-efef-482b-95f3-aabe6c109fb6\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-775666786-94lvs"
Sep 09 03:15:01.185508 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:15:01.185481    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-w4vg2\" (UniqueName: \"kubernetes.io/projected/33582a33-efef-482b-95f3-aabe6c109fb6-kube-api-access-w4vg2\") pod \"cluster-proxy-proxy-agent-775666786-94lvs\" (UID: \"33582a33-efef-482b-95f3-aabe6c109fb6\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-775666786-94lvs"
Sep 09 03:15:01.185587 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:15:01.185504    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cjq28\" (UniqueName: \"kubernetes.io/projected/225fd920-7010-4b6d-b7c1-ae84d877bb1d-kube-api-access-cjq28\") pod \"klusterlet-addon-workmgr-7df6889b48-4bl8z\" (UID: \"225fd920-7010-4b6d-b7c1-ae84d877bb1d\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-7df6889b48-4bl8z"
Sep 09 03:15:01.318763 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:15:01.318725    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-775666786-94lvs"
Sep 09 03:15:01.322509 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:15:01.322490    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-7df6889b48-4bl8z"
Sep 09 03:15:01.594416 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:15:01.594244    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-7df6889b48-4bl8z"]
Sep 09 03:15:01.597738 ip-10-0-132-14 kubenswrapper[2576]: W0909 03:15:01.597690    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod225fd920_7010_4b6d_b7c1_ae84d877bb1d.slice/crio-c6d2817432cff5b9c05720ad4363454168f8a06458f6035cbbb9a8bfe787bb1d WatchSource:0}: Error finding container c6d2817432cff5b9c05720ad4363454168f8a06458f6035cbbb9a8bfe787bb1d: Status 404 returned error can't find the container with id c6d2817432cff5b9c05720ad4363454168f8a06458f6035cbbb9a8bfe787bb1d
Sep 09 03:15:01.599736 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:15:01.598984    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-775666786-94lvs"]
Sep 09 03:15:01.601736 ip-10-0-132-14 kubenswrapper[2576]: W0909 03:15:01.601704    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod33582a33_efef_482b_95f3_aabe6c109fb6.slice/crio-86d294d1d7c6d41eac067a9a620862a9ddc989b35db25e4af0c0c9b46221ea40 WatchSource:0}: Error finding container 86d294d1d7c6d41eac067a9a620862a9ddc989b35db25e4af0c0c9b46221ea40: Status 404 returned error can't find the container with id 86d294d1d7c6d41eac067a9a620862a9ddc989b35db25e4af0c0c9b46221ea40
Sep 09 03:15:01.815016 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:15:01.814959    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-775666786-94lvs" event={"ID":"33582a33-efef-482b-95f3-aabe6c109fb6","Type":"ContainerStarted","Data":"86d294d1d7c6d41eac067a9a620862a9ddc989b35db25e4af0c0c9b46221ea40"}
Sep 09 03:15:01.816027 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:15:01.815997    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-7df6889b48-4bl8z" event={"ID":"225fd920-7010-4b6d-b7c1-ae84d877bb1d","Type":"ContainerStarted","Data":"c6d2817432cff5b9c05720ad4363454168f8a06458f6035cbbb9a8bfe787bb1d"}
Sep 09 03:15:08.837578 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:15:08.837544    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-775666786-94lvs" event={"ID":"33582a33-efef-482b-95f3-aabe6c109fb6","Type":"ContainerStarted","Data":"f55b48c538d6cf575cf72443ed39baa642ec5a0fb6cfd46e82d7b04d2acb6bb6"}
Sep 09 03:15:08.838048 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:15:08.837590    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-775666786-94lvs" event={"ID":"33582a33-efef-482b-95f3-aabe6c109fb6","Type":"ContainerStarted","Data":"6b57bae21a98a6ab18f2dd10c241ac5bcb92c5e1e6190631b51b3aec16dc8bd5"}
Sep 09 03:15:08.838048 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:15:08.837602    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-775666786-94lvs" event={"ID":"33582a33-efef-482b-95f3-aabe6c109fb6","Type":"ContainerStarted","Data":"8d5f5b0132430a1c30f5efd218d40d1a1a45c782be11b3b281e688e1213f9b92"}
Sep 09 03:15:08.839040 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:15:08.839015    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-7df6889b48-4bl8z" event={"ID":"225fd920-7010-4b6d-b7c1-ae84d877bb1d","Type":"ContainerStarted","Data":"7c1a816f9eb537db5680b958da7a39fd48d464221dd53e310eccb35058e5eb9f"}
Sep 09 03:15:08.839239 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:15:08.839222    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-7df6889b48-4bl8z"
Sep 09 03:15:08.840899 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:15:08.840882    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-7df6889b48-4bl8z"
Sep 09 03:15:08.854628 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:15:08.854581    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-775666786-94lvs" podStartSLOduration=2.234331239 podStartE2EDuration="8.854569588s" totalImagesPullingTime="6.620238349s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 03:15:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 03:15:08.853855956 +0000 UTC m=+122.995342033" watchObservedRunningTime="2026-09-09 03:15:08.854569588 +0000 UTC m=+122.996055685"
Sep 09 03:15:08.868456 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:15:08.868414    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-7df6889b48-4bl8z" podStartSLOduration=2.264005792 podStartE2EDuration="8.868399845s" totalImagesPullingTime="6.604394053s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 03:15:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 03:15:08.867640277 +0000 UTC m=+123.009126357" watchObservedRunningTime="2026-09-09 03:15:08.868399845 +0000 UTC m=+123.009885921"
Sep 09 03:15:15.182300 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:15:15.182259    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/443a2987-5246-4a8c-8631-1b751e02101b-metrics-certs\") pod \"network-metrics-daemon-99qzh\" (UID: \"443a2987-5246-4a8c-8631-1b751e02101b\") " pod="openshift-multus/network-metrics-daemon-99qzh"
Sep 09 03:15:15.182789 ip-10-0-132-14 kubenswrapper[2576]: E0909 03:15:15.182383    2576 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found
Sep 09 03:15:15.182789 ip-10-0-132-14 kubenswrapper[2576]: E0909 03:15:15.182438    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/443a2987-5246-4a8c-8631-1b751e02101b-metrics-certs podName:443a2987-5246-4a8c-8631-1b751e02101b nodeName:}" failed. No retries permitted until 2026-09-09 03:17:17.182424236 +0000 UTC m=+251.323910290 (durationBeforeRetry 2m2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/443a2987-5246-4a8c-8631-1b751e02101b-metrics-certs") pod "network-metrics-daemon-99qzh" (UID: "443a2987-5246-4a8c-8631-1b751e02101b") : secret "metrics-daemon-secret" not found
Sep 09 03:15:42.284351 ip-10-0-132-14 kubenswrapper[2576]: E0909 03:15:42.284287    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[metrics-tls], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-dns/dns-default-6z6x2" podUID="fd790ee2-f462-4e4f-ae65-f09b4d36d5bc"
Sep 09 03:15:42.378908 ip-10-0-132-14 kubenswrapper[2576]: E0909 03:15:42.378863    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[metrics-certs], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-multus/network-metrics-daemon-99qzh" podUID="443a2987-5246-4a8c-8631-1b751e02101b"
Sep 09 03:15:42.931559 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:15:42.931532    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-6z6x2"
Sep 09 03:15:47.111847 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:15:47.111783    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/fd790ee2-f462-4e4f-ae65-f09b4d36d5bc-metrics-tls\") pod \"dns-default-6z6x2\" (UID: \"fd790ee2-f462-4e4f-ae65-f09b4d36d5bc\") " pod="openshift-dns/dns-default-6z6x2"
Sep 09 03:15:47.112333 ip-10-0-132-14 kubenswrapper[2576]: E0909 03:15:47.111945    2576 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 09 03:15:47.112333 ip-10-0-132-14 kubenswrapper[2576]: E0909 03:15:47.112039    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/fd790ee2-f462-4e4f-ae65-f09b4d36d5bc-metrics-tls podName:fd790ee2-f462-4e4f-ae65-f09b4d36d5bc nodeName:}" failed. No retries permitted until 2026-09-09 03:17:49.112021577 +0000 UTC m=+283.253507637 (durationBeforeRetry 2m2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/fd790ee2-f462-4e4f-ae65-f09b4d36d5bc-metrics-tls") pod "dns-default-6z6x2" (UID: "fd790ee2-f462-4e4f-ae65-f09b4d36d5bc") : secret "dns-default-metrics-tls" not found
Sep 09 03:15:55.354694 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:15:55.354650    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-99qzh"
Sep 09 03:17:08.840434 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:17:08.840386    2576 prober.go:120] "Probe failed" probeType="Readiness" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-7df6889b48-4bl8z" podUID="225fd920-7010-4b6d-b7c1-ae84d877bb1d" containerName="acm-agent" probeResult="failure" output="Get \"http://10.133.0.9:8000/readyz\": dial tcp 10.133.0.9:8000: connect: connection refused"
Sep 09 03:17:09.179835 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:17:09.179721    2576 generic.go:356] "Generic (PLEG): container finished" podID="225fd920-7010-4b6d-b7c1-ae84d877bb1d" containerID="7c1a816f9eb537db5680b958da7a39fd48d464221dd53e310eccb35058e5eb9f" exitCode=1
Sep 09 03:17:09.179835 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:17:09.179787    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-7df6889b48-4bl8z" event={"ID":"225fd920-7010-4b6d-b7c1-ae84d877bb1d","Type":"ContainerDied","Data":"7c1a816f9eb537db5680b958da7a39fd48d464221dd53e310eccb35058e5eb9f"}
Sep 09 03:17:09.180163 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:17:09.180146    2576 scope.go:117] "RemoveContainer" containerID="7c1a816f9eb537db5680b958da7a39fd48d464221dd53e310eccb35058e5eb9f"
Sep 09 03:17:10.184093 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:17:10.184057    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-7df6889b48-4bl8z" event={"ID":"225fd920-7010-4b6d-b7c1-ae84d877bb1d","Type":"ContainerStarted","Data":"79d34fd533e89ab316a7349fce71114516b1285f41297f53811b258a850507ff"}
Sep 09 03:17:10.184536 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:17:10.184416    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-7df6889b48-4bl8z"
Sep 09 03:17:10.185027 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:17:10.185009    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-7df6889b48-4bl8z"
Sep 09 03:17:17.235664 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:17:17.235623    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/443a2987-5246-4a8c-8631-1b751e02101b-metrics-certs\") pod \"network-metrics-daemon-99qzh\" (UID: \"443a2987-5246-4a8c-8631-1b751e02101b\") " pod="openshift-multus/network-metrics-daemon-99qzh"
Sep 09 03:17:17.236088 ip-10-0-132-14 kubenswrapper[2576]: E0909 03:17:17.235749    2576 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found
Sep 09 03:17:17.236088 ip-10-0-132-14 kubenswrapper[2576]: E0909 03:17:17.235834    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/443a2987-5246-4a8c-8631-1b751e02101b-metrics-certs podName:443a2987-5246-4a8c-8631-1b751e02101b nodeName:}" failed. No retries permitted until 2026-09-09 03:19:19.235796497 +0000 UTC m=+373.377282553 (durationBeforeRetry 2m2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/443a2987-5246-4a8c-8631-1b751e02101b-metrics-certs") pod "network-metrics-daemon-99qzh" (UID: "443a2987-5246-4a8c-8631-1b751e02101b") : secret "metrics-daemon-secret" not found
Sep 09 03:17:29.084602 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:17:29.084517    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-zgdg6"]
Sep 09 03:17:29.087629 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:17:29.087608    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-zgdg6"
Sep 09 03:17:29.090140 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:17:29.090120    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\""
Sep 09 03:17:29.090256 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:17:29.090127    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\""
Sep 09 03:17:29.090256 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:17:29.090212    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\""
Sep 09 03:17:29.090368 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:17:29.090353    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-h5g47\""
Sep 09 03:17:29.091291 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:17:29.091275    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\""
Sep 09 03:17:29.097633 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:17:29.097614    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-zgdg6"]
Sep 09 03:17:29.215261 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:17:29.215229    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-h22wx\" (UniqueName: \"kubernetes.io/projected/ca7eb27b-cd8e-45f0-bbce-6b1f4d8e22fb-kube-api-access-h22wx\") pod \"insights-runtime-extractor-zgdg6\" (UID: \"ca7eb27b-cd8e-45f0-bbce-6b1f4d8e22fb\") " pod="openshift-insights/insights-runtime-extractor-zgdg6"
Sep 09 03:17:29.215261 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:17:29.215267    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/ca7eb27b-cd8e-45f0-bbce-6b1f4d8e22fb-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-zgdg6\" (UID: \"ca7eb27b-cd8e-45f0-bbce-6b1f4d8e22fb\") " pod="openshift-insights/insights-runtime-extractor-zgdg6"
Sep 09 03:17:29.215518 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:17:29.215313    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/ca7eb27b-cd8e-45f0-bbce-6b1f4d8e22fb-crio-socket\") pod \"insights-runtime-extractor-zgdg6\" (UID: \"ca7eb27b-cd8e-45f0-bbce-6b1f4d8e22fb\") " pod="openshift-insights/insights-runtime-extractor-zgdg6"
Sep 09 03:17:29.215518 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:17:29.215363    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/ca7eb27b-cd8e-45f0-bbce-6b1f4d8e22fb-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-zgdg6\" (UID: \"ca7eb27b-cd8e-45f0-bbce-6b1f4d8e22fb\") " pod="openshift-insights/insights-runtime-extractor-zgdg6"
Sep 09 03:17:29.215518 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:17:29.215395    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/ca7eb27b-cd8e-45f0-bbce-6b1f4d8e22fb-data-volume\") pod \"insights-runtime-extractor-zgdg6\" (UID: \"ca7eb27b-cd8e-45f0-bbce-6b1f4d8e22fb\") " pod="openshift-insights/insights-runtime-extractor-zgdg6"
Sep 09 03:17:29.316615 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:17:29.316575    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/ca7eb27b-cd8e-45f0-bbce-6b1f4d8e22fb-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-zgdg6\" (UID: \"ca7eb27b-cd8e-45f0-bbce-6b1f4d8e22fb\") " pod="openshift-insights/insights-runtime-extractor-zgdg6"
Sep 09 03:17:29.316787 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:17:29.316635    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/ca7eb27b-cd8e-45f0-bbce-6b1f4d8e22fb-crio-socket\") pod \"insights-runtime-extractor-zgdg6\" (UID: \"ca7eb27b-cd8e-45f0-bbce-6b1f4d8e22fb\") " pod="openshift-insights/insights-runtime-extractor-zgdg6"
Sep 09 03:17:29.316787 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:17:29.316665    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/ca7eb27b-cd8e-45f0-bbce-6b1f4d8e22fb-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-zgdg6\" (UID: \"ca7eb27b-cd8e-45f0-bbce-6b1f4d8e22fb\") " pod="openshift-insights/insights-runtime-extractor-zgdg6"
Sep 09 03:17:29.316787 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:17:29.316684    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/ca7eb27b-cd8e-45f0-bbce-6b1f4d8e22fb-data-volume\") pod \"insights-runtime-extractor-zgdg6\" (UID: \"ca7eb27b-cd8e-45f0-bbce-6b1f4d8e22fb\") " pod="openshift-insights/insights-runtime-extractor-zgdg6"
Sep 09 03:17:29.316787 ip-10-0-132-14 kubenswrapper[2576]: E0909 03:17:29.316704    2576 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Sep 09 03:17:29.316787 ip-10-0-132-14 kubenswrapper[2576]: E0909 03:17:29.316787    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/ca7eb27b-cd8e-45f0-bbce-6b1f4d8e22fb-kube-rbac-proxy-cm podName:ca7eb27b-cd8e-45f0-bbce-6b1f4d8e22fb nodeName:}" failed. No retries permitted until 2026-09-09 03:17:29.816770234 +0000 UTC m=+263.958256305 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/ca7eb27b-cd8e-45f0-bbce-6b1f4d8e22fb-kube-rbac-proxy-cm") pod "insights-runtime-extractor-zgdg6" (UID: "ca7eb27b-cd8e-45f0-bbce-6b1f4d8e22fb") : configmap "kube-rbac-proxy" not found
Sep 09 03:17:29.317057 ip-10-0-132-14 kubenswrapper[2576]: E0909 03:17:29.316795    2576 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Sep 09 03:17:29.317057 ip-10-0-132-14 kubenswrapper[2576]: E0909 03:17:29.316876    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ca7eb27b-cd8e-45f0-bbce-6b1f4d8e22fb-insights-runtime-extractor-tls podName:ca7eb27b-cd8e-45f0-bbce-6b1f4d8e22fb nodeName:}" failed. No retries permitted until 2026-09-09 03:17:29.816862798 +0000 UTC m=+263.958348852 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/ca7eb27b-cd8e-45f0-bbce-6b1f4d8e22fb-insights-runtime-extractor-tls") pod "insights-runtime-extractor-zgdg6" (UID: "ca7eb27b-cd8e-45f0-bbce-6b1f4d8e22fb") : secret "insights-runtime-extractor-tls" not found
Sep 09 03:17:29.317057 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:17:29.316708    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-h22wx\" (UniqueName: \"kubernetes.io/projected/ca7eb27b-cd8e-45f0-bbce-6b1f4d8e22fb-kube-api-access-h22wx\") pod \"insights-runtime-extractor-zgdg6\" (UID: \"ca7eb27b-cd8e-45f0-bbce-6b1f4d8e22fb\") " pod="openshift-insights/insights-runtime-extractor-zgdg6"
Sep 09 03:17:29.317057 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:17:29.316794    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/ca7eb27b-cd8e-45f0-bbce-6b1f4d8e22fb-crio-socket\") pod \"insights-runtime-extractor-zgdg6\" (UID: \"ca7eb27b-cd8e-45f0-bbce-6b1f4d8e22fb\") " pod="openshift-insights/insights-runtime-extractor-zgdg6"
Sep 09 03:17:29.317057 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:17:29.317030    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/ca7eb27b-cd8e-45f0-bbce-6b1f4d8e22fb-data-volume\") pod \"insights-runtime-extractor-zgdg6\" (UID: \"ca7eb27b-cd8e-45f0-bbce-6b1f4d8e22fb\") " pod="openshift-insights/insights-runtime-extractor-zgdg6"
Sep 09 03:17:29.326384 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:17:29.326349    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-h22wx\" (UniqueName: \"kubernetes.io/projected/ca7eb27b-cd8e-45f0-bbce-6b1f4d8e22fb-kube-api-access-h22wx\") pod \"insights-runtime-extractor-zgdg6\" (UID: \"ca7eb27b-cd8e-45f0-bbce-6b1f4d8e22fb\") " pod="openshift-insights/insights-runtime-extractor-zgdg6"
Sep 09 03:17:29.819643 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:17:29.819605    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/ca7eb27b-cd8e-45f0-bbce-6b1f4d8e22fb-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-zgdg6\" (UID: \"ca7eb27b-cd8e-45f0-bbce-6b1f4d8e22fb\") " pod="openshift-insights/insights-runtime-extractor-zgdg6"
Sep 09 03:17:29.819845 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:17:29.819655    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/ca7eb27b-cd8e-45f0-bbce-6b1f4d8e22fb-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-zgdg6\" (UID: \"ca7eb27b-cd8e-45f0-bbce-6b1f4d8e22fb\") " pod="openshift-insights/insights-runtime-extractor-zgdg6"
Sep 09 03:17:29.819845 ip-10-0-132-14 kubenswrapper[2576]: E0909 03:17:29.819748    2576 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Sep 09 03:17:29.819845 ip-10-0-132-14 kubenswrapper[2576]: E0909 03:17:29.819759    2576 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Sep 09 03:17:29.819845 ip-10-0-132-14 kubenswrapper[2576]: E0909 03:17:29.819832    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/ca7eb27b-cd8e-45f0-bbce-6b1f4d8e22fb-kube-rbac-proxy-cm podName:ca7eb27b-cd8e-45f0-bbce-6b1f4d8e22fb nodeName:}" failed. No retries permitted until 2026-09-09 03:17:30.819798123 +0000 UTC m=+264.961284179 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/ca7eb27b-cd8e-45f0-bbce-6b1f4d8e22fb-kube-rbac-proxy-cm") pod "insights-runtime-extractor-zgdg6" (UID: "ca7eb27b-cd8e-45f0-bbce-6b1f4d8e22fb") : configmap "kube-rbac-proxy" not found
Sep 09 03:17:29.819845 ip-10-0-132-14 kubenswrapper[2576]: E0909 03:17:29.819845    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ca7eb27b-cd8e-45f0-bbce-6b1f4d8e22fb-insights-runtime-extractor-tls podName:ca7eb27b-cd8e-45f0-bbce-6b1f4d8e22fb nodeName:}" failed. No retries permitted until 2026-09-09 03:17:30.819839725 +0000 UTC m=+264.961325779 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/ca7eb27b-cd8e-45f0-bbce-6b1f4d8e22fb-insights-runtime-extractor-tls") pod "insights-runtime-extractor-zgdg6" (UID: "ca7eb27b-cd8e-45f0-bbce-6b1f4d8e22fb") : secret "insights-runtime-extractor-tls" not found
Sep 09 03:17:30.826599 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:17:30.826554    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/ca7eb27b-cd8e-45f0-bbce-6b1f4d8e22fb-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-zgdg6\" (UID: \"ca7eb27b-cd8e-45f0-bbce-6b1f4d8e22fb\") " pod="openshift-insights/insights-runtime-extractor-zgdg6"
Sep 09 03:17:30.826599 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:17:30.826613    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/ca7eb27b-cd8e-45f0-bbce-6b1f4d8e22fb-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-zgdg6\" (UID: \"ca7eb27b-cd8e-45f0-bbce-6b1f4d8e22fb\") " pod="openshift-insights/insights-runtime-extractor-zgdg6"
Sep 09 03:17:30.827069 ip-10-0-132-14 kubenswrapper[2576]: E0909 03:17:30.826677    2576 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Sep 09 03:17:30.827069 ip-10-0-132-14 kubenswrapper[2576]: E0909 03:17:30.826705    2576 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Sep 09 03:17:30.827069 ip-10-0-132-14 kubenswrapper[2576]: E0909 03:17:30.826730    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/ca7eb27b-cd8e-45f0-bbce-6b1f4d8e22fb-kube-rbac-proxy-cm podName:ca7eb27b-cd8e-45f0-bbce-6b1f4d8e22fb nodeName:}" failed. No retries permitted until 2026-09-09 03:17:32.826716676 +0000 UTC m=+266.968202732 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/ca7eb27b-cd8e-45f0-bbce-6b1f4d8e22fb-kube-rbac-proxy-cm") pod "insights-runtime-extractor-zgdg6" (UID: "ca7eb27b-cd8e-45f0-bbce-6b1f4d8e22fb") : configmap "kube-rbac-proxy" not found
Sep 09 03:17:30.827069 ip-10-0-132-14 kubenswrapper[2576]: E0909 03:17:30.826755    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ca7eb27b-cd8e-45f0-bbce-6b1f4d8e22fb-insights-runtime-extractor-tls podName:ca7eb27b-cd8e-45f0-bbce-6b1f4d8e22fb nodeName:}" failed. No retries permitted until 2026-09-09 03:17:32.826741975 +0000 UTC m=+266.968228030 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/ca7eb27b-cd8e-45f0-bbce-6b1f4d8e22fb-insights-runtime-extractor-tls") pod "insights-runtime-extractor-zgdg6" (UID: "ca7eb27b-cd8e-45f0-bbce-6b1f4d8e22fb") : secret "insights-runtime-extractor-tls" not found
Sep 09 03:17:30.860558 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:17:30.860532    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-cqhff_c9e03e08-c803-4ca3-8f43-aa3390269c0a/dns-node-resolver/0.log"
Sep 09 03:17:31.860347 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:17:31.860316    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-2zp8k_334f1652-d04c-4382-bbf3-367d96ba0b5c/node-ca/0.log"
Sep 09 03:17:32.660549 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:17:32.660511    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-227kg_3862bef2-07ad-41d7-978c-3f6f5c264854/serve-healthcheck-canary/0.log"
Sep 09 03:17:32.843363 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:17:32.843330    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/ca7eb27b-cd8e-45f0-bbce-6b1f4d8e22fb-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-zgdg6\" (UID: \"ca7eb27b-cd8e-45f0-bbce-6b1f4d8e22fb\") " pod="openshift-insights/insights-runtime-extractor-zgdg6"
Sep 09 03:17:32.843530 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:17:32.843380    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/ca7eb27b-cd8e-45f0-bbce-6b1f4d8e22fb-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-zgdg6\" (UID: \"ca7eb27b-cd8e-45f0-bbce-6b1f4d8e22fb\") " pod="openshift-insights/insights-runtime-extractor-zgdg6"
Sep 09 03:17:32.843530 ip-10-0-132-14 kubenswrapper[2576]: E0909 03:17:32.843459    2576 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Sep 09 03:17:32.843530 ip-10-0-132-14 kubenswrapper[2576]: E0909 03:17:32.843492    2576 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Sep 09 03:17:32.843530 ip-10-0-132-14 kubenswrapper[2576]: E0909 03:17:32.843520    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/ca7eb27b-cd8e-45f0-bbce-6b1f4d8e22fb-kube-rbac-proxy-cm podName:ca7eb27b-cd8e-45f0-bbce-6b1f4d8e22fb nodeName:}" failed. No retries permitted until 2026-09-09 03:17:36.843502821 +0000 UTC m=+270.984988893 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/ca7eb27b-cd8e-45f0-bbce-6b1f4d8e22fb-kube-rbac-proxy-cm") pod "insights-runtime-extractor-zgdg6" (UID: "ca7eb27b-cd8e-45f0-bbce-6b1f4d8e22fb") : configmap "kube-rbac-proxy" not found
Sep 09 03:17:32.843660 ip-10-0-132-14 kubenswrapper[2576]: E0909 03:17:32.843545    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ca7eb27b-cd8e-45f0-bbce-6b1f4d8e22fb-insights-runtime-extractor-tls podName:ca7eb27b-cd8e-45f0-bbce-6b1f4d8e22fb nodeName:}" failed. No retries permitted until 2026-09-09 03:17:36.843531881 +0000 UTC m=+270.985017936 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/ca7eb27b-cd8e-45f0-bbce-6b1f4d8e22fb-insights-runtime-extractor-tls") pod "insights-runtime-extractor-zgdg6" (UID: "ca7eb27b-cd8e-45f0-bbce-6b1f4d8e22fb") : secret "insights-runtime-extractor-tls" not found
Sep 09 03:17:36.871599 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:17:36.871548    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/ca7eb27b-cd8e-45f0-bbce-6b1f4d8e22fb-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-zgdg6\" (UID: \"ca7eb27b-cd8e-45f0-bbce-6b1f4d8e22fb\") " pod="openshift-insights/insights-runtime-extractor-zgdg6"
Sep 09 03:17:36.872011 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:17:36.871615    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/ca7eb27b-cd8e-45f0-bbce-6b1f4d8e22fb-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-zgdg6\" (UID: \"ca7eb27b-cd8e-45f0-bbce-6b1f4d8e22fb\") " pod="openshift-insights/insights-runtime-extractor-zgdg6"
Sep 09 03:17:36.872011 ip-10-0-132-14 kubenswrapper[2576]: E0909 03:17:36.871709    2576 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Sep 09 03:17:36.872011 ip-10-0-132-14 kubenswrapper[2576]: E0909 03:17:36.871710    2576 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Sep 09 03:17:36.872011 ip-10-0-132-14 kubenswrapper[2576]: E0909 03:17:36.871767    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/ca7eb27b-cd8e-45f0-bbce-6b1f4d8e22fb-kube-rbac-proxy-cm podName:ca7eb27b-cd8e-45f0-bbce-6b1f4d8e22fb nodeName:}" failed. No retries permitted until 2026-09-09 03:17:44.871751644 +0000 UTC m=+279.013237699 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/ca7eb27b-cd8e-45f0-bbce-6b1f4d8e22fb-kube-rbac-proxy-cm") pod "insights-runtime-extractor-zgdg6" (UID: "ca7eb27b-cd8e-45f0-bbce-6b1f4d8e22fb") : configmap "kube-rbac-proxy" not found
Sep 09 03:17:36.872011 ip-10-0-132-14 kubenswrapper[2576]: E0909 03:17:36.871780    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ca7eb27b-cd8e-45f0-bbce-6b1f4d8e22fb-insights-runtime-extractor-tls podName:ca7eb27b-cd8e-45f0-bbce-6b1f4d8e22fb nodeName:}" failed. No retries permitted until 2026-09-09 03:17:44.871774811 +0000 UTC m=+279.013260866 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/ca7eb27b-cd8e-45f0-bbce-6b1f4d8e22fb-insights-runtime-extractor-tls") pod "insights-runtime-extractor-zgdg6" (UID: "ca7eb27b-cd8e-45f0-bbce-6b1f4d8e22fb") : secret "insights-runtime-extractor-tls" not found
Sep 09 03:17:44.929022 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:17:44.928982    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/ca7eb27b-cd8e-45f0-bbce-6b1f4d8e22fb-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-zgdg6\" (UID: \"ca7eb27b-cd8e-45f0-bbce-6b1f4d8e22fb\") " pod="openshift-insights/insights-runtime-extractor-zgdg6"
Sep 09 03:17:44.929408 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:17:44.929036    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/ca7eb27b-cd8e-45f0-bbce-6b1f4d8e22fb-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-zgdg6\" (UID: \"ca7eb27b-cd8e-45f0-bbce-6b1f4d8e22fb\") " pod="openshift-insights/insights-runtime-extractor-zgdg6"
Sep 09 03:17:44.929520 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:17:44.929502    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/ca7eb27b-cd8e-45f0-bbce-6b1f4d8e22fb-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-zgdg6\" (UID: \"ca7eb27b-cd8e-45f0-bbce-6b1f4d8e22fb\") " pod="openshift-insights/insights-runtime-extractor-zgdg6"
Sep 09 03:17:44.931413 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:17:44.931391    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/ca7eb27b-cd8e-45f0-bbce-6b1f4d8e22fb-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-zgdg6\" (UID: \"ca7eb27b-cd8e-45f0-bbce-6b1f4d8e22fb\") " pod="openshift-insights/insights-runtime-extractor-zgdg6"
Sep 09 03:17:44.996597 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:17:44.996557    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-zgdg6"
Sep 09 03:17:45.262760 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:17:45.262729    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-zgdg6"]
Sep 09 03:17:45.266908 ip-10-0-132-14 kubenswrapper[2576]: W0909 03:17:45.266881    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podca7eb27b_cd8e_45f0_bbce_6b1f4d8e22fb.slice/crio-fa64974561f91df92f96b23858c0b47708abeb7e71e42f6d20cec83415116ba8 WatchSource:0}: Error finding container fa64974561f91df92f96b23858c0b47708abeb7e71e42f6d20cec83415116ba8: Status 404 returned error can't find the container with id fa64974561f91df92f96b23858c0b47708abeb7e71e42f6d20cec83415116ba8
Sep 09 03:17:45.285614 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:17:45.285572    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-zgdg6" event={"ID":"ca7eb27b-cd8e-45f0-bbce-6b1f4d8e22fb","Type":"ContainerStarted","Data":"fa64974561f91df92f96b23858c0b47708abeb7e71e42f6d20cec83415116ba8"}
Sep 09 03:17:45.932326 ip-10-0-132-14 kubenswrapper[2576]: E0909 03:17:45.932278    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[metrics-tls], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-dns/dns-default-6z6x2" podUID="fd790ee2-f462-4e4f-ae65-f09b4d36d5bc"
Sep 09 03:17:46.290608 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:17:46.290577    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-6z6x2"
Sep 09 03:17:46.290775 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:17:46.290610    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-zgdg6" event={"ID":"ca7eb27b-cd8e-45f0-bbce-6b1f4d8e22fb","Type":"ContainerStarted","Data":"2c726552c2070ad520624c5a55adba43c0bfbf1866111bd5dbf918251df9d6b8"}
Sep 09 03:17:46.290775 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:17:46.290665    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-zgdg6" event={"ID":"ca7eb27b-cd8e-45f0-bbce-6b1f4d8e22fb","Type":"ContainerStarted","Data":"f9f7fcd985408ece6b4fd5facb957d3d1459ffcac307112788382c628ae17619"}
Sep 09 03:17:48.299686 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:17:48.299647    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-zgdg6" event={"ID":"ca7eb27b-cd8e-45f0-bbce-6b1f4d8e22fb","Type":"ContainerStarted","Data":"e79499dcf3eaef15fc816c563756ae5cd70f47c752d02e3d78a56abad25e7730"}
Sep 09 03:17:48.318602 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:17:48.318555    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-zgdg6" podStartSLOduration=16.867036782 podStartE2EDuration="19.3185386s" totalImagesPullingTime="2.451501818s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-09 03:17:29 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 03:17:48.318372399 +0000 UTC m=+282.459858495" watchObservedRunningTime="2026-09-09 03:17:48.3185386 +0000 UTC m=+282.460024716"
Sep 09 03:17:49.160012 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:17:49.159963    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/fd790ee2-f462-4e4f-ae65-f09b4d36d5bc-metrics-tls\") pod \"dns-default-6z6x2\" (UID: \"fd790ee2-f462-4e4f-ae65-f09b4d36d5bc\") " pod="openshift-dns/dns-default-6z6x2"
Sep 09 03:17:49.162399 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:17:49.162365    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/fd790ee2-f462-4e4f-ae65-f09b4d36d5bc-metrics-tls\") pod \"dns-default-6z6x2\" (UID: \"fd790ee2-f462-4e4f-ae65-f09b4d36d5bc\") " pod="openshift-dns/dns-default-6z6x2"
Sep 09 03:17:49.294482 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:17:49.294452    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-pxbsq\""
Sep 09 03:17:49.302340 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:17:49.302318    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-6z6x2"
Sep 09 03:17:49.569161 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:17:49.568976    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-6z6x2"]
Sep 09 03:17:49.571482 ip-10-0-132-14 kubenswrapper[2576]: W0909 03:17:49.571456    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfd790ee2_f462_4e4f_ae65_f09b4d36d5bc.slice/crio-7419c7ac58661ef46c32f62b75da03c60d2ccfe8149da774da4a87a4bda32d62 WatchSource:0}: Error finding container 7419c7ac58661ef46c32f62b75da03c60d2ccfe8149da774da4a87a4bda32d62: Status 404 returned error can't find the container with id 7419c7ac58661ef46c32f62b75da03c60d2ccfe8149da774da4a87a4bda32d62
Sep 09 03:17:50.308511 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:17:50.308473    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-6z6x2" event={"ID":"fd790ee2-f462-4e4f-ae65-f09b4d36d5bc","Type":"ContainerStarted","Data":"7419c7ac58661ef46c32f62b75da03c60d2ccfe8149da774da4a87a4bda32d62"}
Sep 09 03:17:51.313151 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:17:51.313119    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-6z6x2" event={"ID":"fd790ee2-f462-4e4f-ae65-f09b4d36d5bc","Type":"ContainerStarted","Data":"b35fc4618d8357fa47bf3091dda11713581458b106a7f5c260e03a21f38b18b3"}
Sep 09 03:17:51.313519 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:17:51.313170    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-6z6x2" event={"ID":"fd790ee2-f462-4e4f-ae65-f09b4d36d5bc","Type":"ContainerStarted","Data":"d53d64434abbbfe482f23937c7087f746ff01f4923fa6222ae11fec348508974"}
Sep 09 03:17:51.313519 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:17:51.313199    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-6z6x2"
Sep 09 03:17:51.330919 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:17:51.330876    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-6z6x2" podStartSLOduration=250.819977674 podStartE2EDuration="4m12.330850633s" totalImagesPullingTime="1.510872959s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 03:13:39 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 03:17:51.328815771 +0000 UTC m=+285.470301840" watchObservedRunningTime="2026-09-09 03:17:51.330850633 +0000 UTC m=+285.472336710"
Sep 09 03:17:52.011510 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:17:52.011473    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-668c7d5bd8-6trbs"]
Sep 09 03:17:52.014336 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:17:52.014318    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-668c7d5bd8-6trbs"
Sep 09 03:17:52.017700 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:17:52.017673    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\""
Sep 09 03:17:52.017924 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:17:52.017907    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\""
Sep 09 03:17:52.017978 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:17:52.017907    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-dmprq\""
Sep 09 03:17:52.017978 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:17:52.017963    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\""
Sep 09 03:17:52.023115 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:17:52.023086    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\""
Sep 09 03:17:52.027920 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:17:52.027892    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-668c7d5bd8-6trbs"]
Sep 09 03:17:52.085069 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:17:52.085038    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/f26dea10-3e1c-45af-8c3c-ea95592593b4-image-registry-private-configuration\") pod \"image-registry-668c7d5bd8-6trbs\" (UID: \"f26dea10-3e1c-45af-8c3c-ea95592593b4\") " pod="openshift-image-registry/image-registry-668c7d5bd8-6trbs"
Sep 09 03:17:52.085224 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:17:52.085079    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/f26dea10-3e1c-45af-8c3c-ea95592593b4-trusted-ca\") pod \"image-registry-668c7d5bd8-6trbs\" (UID: \"f26dea10-3e1c-45af-8c3c-ea95592593b4\") " pod="openshift-image-registry/image-registry-668c7d5bd8-6trbs"
Sep 09 03:17:52.085224 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:17:52.085098    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/f26dea10-3e1c-45af-8c3c-ea95592593b4-bound-sa-token\") pod \"image-registry-668c7d5bd8-6trbs\" (UID: \"f26dea10-3e1c-45af-8c3c-ea95592593b4\") " pod="openshift-image-registry/image-registry-668c7d5bd8-6trbs"
Sep 09 03:17:52.085224 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:17:52.085129    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/f26dea10-3e1c-45af-8c3c-ea95592593b4-ca-trust-extracted\") pod \"image-registry-668c7d5bd8-6trbs\" (UID: \"f26dea10-3e1c-45af-8c3c-ea95592593b4\") " pod="openshift-image-registry/image-registry-668c7d5bd8-6trbs"
Sep 09 03:17:52.085224 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:17:52.085176    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/f26dea10-3e1c-45af-8c3c-ea95592593b4-registry-tls\") pod \"image-registry-668c7d5bd8-6trbs\" (UID: \"f26dea10-3e1c-45af-8c3c-ea95592593b4\") " pod="openshift-image-registry/image-registry-668c7d5bd8-6trbs"
Sep 09 03:17:52.085224 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:17:52.085191    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/f26dea10-3e1c-45af-8c3c-ea95592593b4-registry-certificates\") pod \"image-registry-668c7d5bd8-6trbs\" (UID: \"f26dea10-3e1c-45af-8c3c-ea95592593b4\") " pod="openshift-image-registry/image-registry-668c7d5bd8-6trbs"
Sep 09 03:17:52.085402 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:17:52.085251    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2fnt2\" (UniqueName: \"kubernetes.io/projected/f26dea10-3e1c-45af-8c3c-ea95592593b4-kube-api-access-2fnt2\") pod \"image-registry-668c7d5bd8-6trbs\" (UID: \"f26dea10-3e1c-45af-8c3c-ea95592593b4\") " pod="openshift-image-registry/image-registry-668c7d5bd8-6trbs"
Sep 09 03:17:52.085402 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:17:52.085299    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/f26dea10-3e1c-45af-8c3c-ea95592593b4-installation-pull-secrets\") pod \"image-registry-668c7d5bd8-6trbs\" (UID: \"f26dea10-3e1c-45af-8c3c-ea95592593b4\") " pod="openshift-image-registry/image-registry-668c7d5bd8-6trbs"
Sep 09 03:17:52.186266 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:17:52.186229    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/f26dea10-3e1c-45af-8c3c-ea95592593b4-registry-tls\") pod \"image-registry-668c7d5bd8-6trbs\" (UID: \"f26dea10-3e1c-45af-8c3c-ea95592593b4\") " pod="openshift-image-registry/image-registry-668c7d5bd8-6trbs"
Sep 09 03:17:52.186266 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:17:52.186262    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/f26dea10-3e1c-45af-8c3c-ea95592593b4-registry-certificates\") pod \"image-registry-668c7d5bd8-6trbs\" (UID: \"f26dea10-3e1c-45af-8c3c-ea95592593b4\") " pod="openshift-image-registry/image-registry-668c7d5bd8-6trbs"
Sep 09 03:17:52.186486 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:17:52.186281    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2fnt2\" (UniqueName: \"kubernetes.io/projected/f26dea10-3e1c-45af-8c3c-ea95592593b4-kube-api-access-2fnt2\") pod \"image-registry-668c7d5bd8-6trbs\" (UID: \"f26dea10-3e1c-45af-8c3c-ea95592593b4\") " pod="openshift-image-registry/image-registry-668c7d5bd8-6trbs"
Sep 09 03:17:52.186486 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:17:52.186303    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/f26dea10-3e1c-45af-8c3c-ea95592593b4-installation-pull-secrets\") pod \"image-registry-668c7d5bd8-6trbs\" (UID: \"f26dea10-3e1c-45af-8c3c-ea95592593b4\") " pod="openshift-image-registry/image-registry-668c7d5bd8-6trbs"
Sep 09 03:17:52.186486 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:17:52.186335    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/f26dea10-3e1c-45af-8c3c-ea95592593b4-image-registry-private-configuration\") pod \"image-registry-668c7d5bd8-6trbs\" (UID: \"f26dea10-3e1c-45af-8c3c-ea95592593b4\") " pod="openshift-image-registry/image-registry-668c7d5bd8-6trbs"
Sep 09 03:17:52.186486 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:17:52.186359    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/f26dea10-3e1c-45af-8c3c-ea95592593b4-trusted-ca\") pod \"image-registry-668c7d5bd8-6trbs\" (UID: \"f26dea10-3e1c-45af-8c3c-ea95592593b4\") " pod="openshift-image-registry/image-registry-668c7d5bd8-6trbs"
Sep 09 03:17:52.186486 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:17:52.186373    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/f26dea10-3e1c-45af-8c3c-ea95592593b4-bound-sa-token\") pod \"image-registry-668c7d5bd8-6trbs\" (UID: \"f26dea10-3e1c-45af-8c3c-ea95592593b4\") " pod="openshift-image-registry/image-registry-668c7d5bd8-6trbs"
Sep 09 03:17:52.186486 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:17:52.186404    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/f26dea10-3e1c-45af-8c3c-ea95592593b4-ca-trust-extracted\") pod \"image-registry-668c7d5bd8-6trbs\" (UID: \"f26dea10-3e1c-45af-8c3c-ea95592593b4\") " pod="openshift-image-registry/image-registry-668c7d5bd8-6trbs"
Sep 09 03:17:52.186990 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:17:52.186966    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/f26dea10-3e1c-45af-8c3c-ea95592593b4-ca-trust-extracted\") pod \"image-registry-668c7d5bd8-6trbs\" (UID: \"f26dea10-3e1c-45af-8c3c-ea95592593b4\") " pod="openshift-image-registry/image-registry-668c7d5bd8-6trbs"
Sep 09 03:17:52.187367 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:17:52.187346    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/f26dea10-3e1c-45af-8c3c-ea95592593b4-trusted-ca\") pod \"image-registry-668c7d5bd8-6trbs\" (UID: \"f26dea10-3e1c-45af-8c3c-ea95592593b4\") " pod="openshift-image-registry/image-registry-668c7d5bd8-6trbs"
Sep 09 03:17:52.187457 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:17:52.187405    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/f26dea10-3e1c-45af-8c3c-ea95592593b4-registry-certificates\") pod \"image-registry-668c7d5bd8-6trbs\" (UID: \"f26dea10-3e1c-45af-8c3c-ea95592593b4\") " pod="openshift-image-registry/image-registry-668c7d5bd8-6trbs"
Sep 09 03:17:52.189029 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:17:52.189000    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/f26dea10-3e1c-45af-8c3c-ea95592593b4-installation-pull-secrets\") pod \"image-registry-668c7d5bd8-6trbs\" (UID: \"f26dea10-3e1c-45af-8c3c-ea95592593b4\") " pod="openshift-image-registry/image-registry-668c7d5bd8-6trbs"
Sep 09 03:17:52.189029 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:17:52.189022    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/f26dea10-3e1c-45af-8c3c-ea95592593b4-registry-tls\") pod \"image-registry-668c7d5bd8-6trbs\" (UID: \"f26dea10-3e1c-45af-8c3c-ea95592593b4\") " pod="openshift-image-registry/image-registry-668c7d5bd8-6trbs"
Sep 09 03:17:52.189148 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:17:52.189074    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/f26dea10-3e1c-45af-8c3c-ea95592593b4-image-registry-private-configuration\") pod \"image-registry-668c7d5bd8-6trbs\" (UID: \"f26dea10-3e1c-45af-8c3c-ea95592593b4\") " pod="openshift-image-registry/image-registry-668c7d5bd8-6trbs"
Sep 09 03:17:52.201928 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:17:52.201904    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2fnt2\" (UniqueName: \"kubernetes.io/projected/f26dea10-3e1c-45af-8c3c-ea95592593b4-kube-api-access-2fnt2\") pod \"image-registry-668c7d5bd8-6trbs\" (UID: \"f26dea10-3e1c-45af-8c3c-ea95592593b4\") " pod="openshift-image-registry/image-registry-668c7d5bd8-6trbs"
Sep 09 03:17:52.218900 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:17:52.218878    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/f26dea10-3e1c-45af-8c3c-ea95592593b4-bound-sa-token\") pod \"image-registry-668c7d5bd8-6trbs\" (UID: \"f26dea10-3e1c-45af-8c3c-ea95592593b4\") " pod="openshift-image-registry/image-registry-668c7d5bd8-6trbs"
Sep 09 03:17:52.323049 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:17:52.322965    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-668c7d5bd8-6trbs"
Sep 09 03:17:52.597786 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:17:52.597715    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-668c7d5bd8-6trbs"]
Sep 09 03:17:52.600653 ip-10-0-132-14 kubenswrapper[2576]: W0909 03:17:52.600622    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf26dea10_3e1c_45af_8c3c_ea95592593b4.slice/crio-e3d9e12699b58fea2fce15766a1209e6f0f8748d03c015b1590a5c950f209a73 WatchSource:0}: Error finding container e3d9e12699b58fea2fce15766a1209e6f0f8748d03c015b1590a5c950f209a73: Status 404 returned error can't find the container with id e3d9e12699b58fea2fce15766a1209e6f0f8748d03c015b1590a5c950f209a73
Sep 09 03:17:53.320781 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:17:53.320742    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-668c7d5bd8-6trbs" event={"ID":"f26dea10-3e1c-45af-8c3c-ea95592593b4","Type":"ContainerStarted","Data":"34f2d98cd700b194db05351613b8357d687931061a351b57ae1161e3256c2742"}
Sep 09 03:17:53.320996 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:17:53.320791    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-668c7d5bd8-6trbs" event={"ID":"f26dea10-3e1c-45af-8c3c-ea95592593b4","Type":"ContainerStarted","Data":"e3d9e12699b58fea2fce15766a1209e6f0f8748d03c015b1590a5c950f209a73"}
Sep 09 03:17:53.320996 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:17:53.320837    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-668c7d5bd8-6trbs"
Sep 09 03:17:53.339757 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:17:53.339710    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/image-registry-668c7d5bd8-6trbs" podStartSLOduration=2.339694305 podStartE2EDuration="2.339694305s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 03:17:51 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 03:17:53.338555642 +0000 UTC m=+287.480041734" watchObservedRunningTime="2026-09-09 03:17:53.339694305 +0000 UTC m=+287.481180382"
Sep 09 03:17:58.355111 ip-10-0-132-14 kubenswrapper[2576]: E0909 03:17:58.355069    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[metrics-certs], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-multus/network-metrics-daemon-99qzh" podUID="443a2987-5246-4a8c-8631-1b751e02101b"
Sep 09 03:18:01.318919 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:18:01.318878    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-6z6x2"
Sep 09 03:18:06.291884 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:18:06.291859    2576 kubelet.go:1648] "Image garbage collection succeeded"
Sep 09 03:18:09.354264 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:18:09.354229    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-99qzh"
Sep 09 03:18:14.329076 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:18:14.329028    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-668c7d5bd8-6trbs"
Sep 09 03:18:14.696866 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:18:14.696770    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/openshift-state-metrics-657fd6c54-5jhlq"]
Sep 09 03:18:14.699876 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:18:14.699860    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/openshift-state-metrics-657fd6c54-5jhlq"
Sep 09 03:18:14.704680 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:18:14.704661    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-dockercfg-dztwk\""
Sep 09 03:18:14.704830 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:18:14.704685    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\""
Sep 09 03:18:14.706171 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:18:14.705894    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-tls\""
Sep 09 03:18:14.706171 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:18:14.705907    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-kube-rbac-proxy-config\""
Sep 09 03:18:14.706171 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:18:14.705995    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\""
Sep 09 03:18:14.706171 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:18:14.706120    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\""
Sep 09 03:18:14.713667 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:18:14.713648    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/openshift-state-metrics-657fd6c54-5jhlq"]
Sep 09 03:18:14.749665 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:18:14.749636    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-dts6n"]
Sep 09 03:18:14.752637 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:18:14.752619    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-dts6n"
Sep 09 03:18:14.755454 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:18:14.755437    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\""
Sep 09 03:18:14.755533 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:18:14.755455    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-z8kcn\""
Sep 09 03:18:14.755757 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:18:14.755725    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\""
Sep 09 03:18:14.755926 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:18:14.755908    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\""
Sep 09 03:18:14.839528 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:18:14.839494    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/5f6c90ad-5940-4380-8c65-ec0e6e1df53b-metrics-client-ca\") pod \"openshift-state-metrics-657fd6c54-5jhlq\" (UID: \"5f6c90ad-5940-4380-8c65-ec0e6e1df53b\") " pod="openshift-monitoring/openshift-state-metrics-657fd6c54-5jhlq"
Sep 09 03:18:14.839528 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:18:14.839530    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/5f6c90ad-5940-4380-8c65-ec0e6e1df53b-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-657fd6c54-5jhlq\" (UID: \"5f6c90ad-5940-4380-8c65-ec0e6e1df53b\") " pod="openshift-monitoring/openshift-state-metrics-657fd6c54-5jhlq"
Sep 09 03:18:14.839712 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:18:14.839571    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/5f6c90ad-5940-4380-8c65-ec0e6e1df53b-openshift-state-metrics-tls\") pod \"openshift-state-metrics-657fd6c54-5jhlq\" (UID: \"5f6c90ad-5940-4380-8c65-ec0e6e1df53b\") " pod="openshift-monitoring/openshift-state-metrics-657fd6c54-5jhlq"
Sep 09 03:18:14.839712 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:18:14.839630    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-h6l6l\" (UniqueName: \"kubernetes.io/projected/5f6c90ad-5940-4380-8c65-ec0e6e1df53b-kube-api-access-h6l6l\") pod \"openshift-state-metrics-657fd6c54-5jhlq\" (UID: \"5f6c90ad-5940-4380-8c65-ec0e6e1df53b\") " pod="openshift-monitoring/openshift-state-metrics-657fd6c54-5jhlq"
Sep 09 03:18:14.940011 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:18:14.939971    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/8845a405-bcfc-4690-bfc8-7868d670f68c-node-exporter-textfile\") pod \"node-exporter-dts6n\" (UID: \"8845a405-bcfc-4690-bfc8-7868d670f68c\") " pod="openshift-monitoring/node-exporter-dts6n"
Sep 09 03:18:14.940011 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:18:14.940009    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/8845a405-bcfc-4690-bfc8-7868d670f68c-node-exporter-accelerators-collector-config\") pod \"node-exporter-dts6n\" (UID: \"8845a405-bcfc-4690-bfc8-7868d670f68c\") " pod="openshift-monitoring/node-exporter-dts6n"
Sep 09 03:18:14.940244 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:18:14.940038    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/5f6c90ad-5940-4380-8c65-ec0e6e1df53b-openshift-state-metrics-tls\") pod \"openshift-state-metrics-657fd6c54-5jhlq\" (UID: \"5f6c90ad-5940-4380-8c65-ec0e6e1df53b\") " pod="openshift-monitoring/openshift-state-metrics-657fd6c54-5jhlq"
Sep 09 03:18:14.940244 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:18:14.940102    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-h6l6l\" (UniqueName: \"kubernetes.io/projected/5f6c90ad-5940-4380-8c65-ec0e6e1df53b-kube-api-access-h6l6l\") pod \"openshift-state-metrics-657fd6c54-5jhlq\" (UID: \"5f6c90ad-5940-4380-8c65-ec0e6e1df53b\") " pod="openshift-monitoring/openshift-state-metrics-657fd6c54-5jhlq"
Sep 09 03:18:14.940244 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:18:14.940140    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/8845a405-bcfc-4690-bfc8-7868d670f68c-root\") pod \"node-exporter-dts6n\" (UID: \"8845a405-bcfc-4690-bfc8-7868d670f68c\") " pod="openshift-monitoring/node-exporter-dts6n"
Sep 09 03:18:14.940244 ip-10-0-132-14 kubenswrapper[2576]: E0909 03:18:14.940145    2576 secret.go:189] Couldn't get secret openshift-monitoring/openshift-state-metrics-tls: secret "openshift-state-metrics-tls" not found
Sep 09 03:18:14.940244 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:18:14.940167    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/8845a405-bcfc-4690-bfc8-7868d670f68c-sys\") pod \"node-exporter-dts6n\" (UID: \"8845a405-bcfc-4690-bfc8-7868d670f68c\") " pod="openshift-monitoring/node-exporter-dts6n"
Sep 09 03:18:14.940244 ip-10-0-132-14 kubenswrapper[2576]: E0909 03:18:14.940214    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5f6c90ad-5940-4380-8c65-ec0e6e1df53b-openshift-state-metrics-tls podName:5f6c90ad-5940-4380-8c65-ec0e6e1df53b nodeName:}" failed. No retries permitted until 2026-09-09 03:18:15.440194722 +0000 UTC m=+309.581680791 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "openshift-state-metrics-tls" (UniqueName: "kubernetes.io/secret/5f6c90ad-5940-4380-8c65-ec0e6e1df53b-openshift-state-metrics-tls") pod "openshift-state-metrics-657fd6c54-5jhlq" (UID: "5f6c90ad-5940-4380-8c65-ec0e6e1df53b") : secret "openshift-state-metrics-tls" not found
Sep 09 03:18:14.940244 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:18:14.940229    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/8845a405-bcfc-4690-bfc8-7868d670f68c-node-exporter-tls\") pod \"node-exporter-dts6n\" (UID: \"8845a405-bcfc-4690-bfc8-7868d670f68c\") " pod="openshift-monitoring/node-exporter-dts6n"
Sep 09 03:18:14.940610 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:18:14.940265    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/5f6c90ad-5940-4380-8c65-ec0e6e1df53b-metrics-client-ca\") pod \"openshift-state-metrics-657fd6c54-5jhlq\" (UID: \"5f6c90ad-5940-4380-8c65-ec0e6e1df53b\") " pod="openshift-monitoring/openshift-state-metrics-657fd6c54-5jhlq"
Sep 09 03:18:14.940610 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:18:14.940318    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/5f6c90ad-5940-4380-8c65-ec0e6e1df53b-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-657fd6c54-5jhlq\" (UID: \"5f6c90ad-5940-4380-8c65-ec0e6e1df53b\") " pod="openshift-monitoring/openshift-state-metrics-657fd6c54-5jhlq"
Sep 09 03:18:14.940610 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:18:14.940344    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/8845a405-bcfc-4690-bfc8-7868d670f68c-metrics-client-ca\") pod \"node-exporter-dts6n\" (UID: \"8845a405-bcfc-4690-bfc8-7868d670f68c\") " pod="openshift-monitoring/node-exporter-dts6n"
Sep 09 03:18:14.940610 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:18:14.940362    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tkz78\" (UniqueName: \"kubernetes.io/projected/8845a405-bcfc-4690-bfc8-7868d670f68c-kube-api-access-tkz78\") pod \"node-exporter-dts6n\" (UID: \"8845a405-bcfc-4690-bfc8-7868d670f68c\") " pod="openshift-monitoring/node-exporter-dts6n"
Sep 09 03:18:14.940610 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:18:14.940392    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/8845a405-bcfc-4690-bfc8-7868d670f68c-node-exporter-wtmp\") pod \"node-exporter-dts6n\" (UID: \"8845a405-bcfc-4690-bfc8-7868d670f68c\") " pod="openshift-monitoring/node-exporter-dts6n"
Sep 09 03:18:14.940610 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:18:14.940418    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/8845a405-bcfc-4690-bfc8-7868d670f68c-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-dts6n\" (UID: \"8845a405-bcfc-4690-bfc8-7868d670f68c\") " pod="openshift-monitoring/node-exporter-dts6n"
Sep 09 03:18:14.940934 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:18:14.940852    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/5f6c90ad-5940-4380-8c65-ec0e6e1df53b-metrics-client-ca\") pod \"openshift-state-metrics-657fd6c54-5jhlq\" (UID: \"5f6c90ad-5940-4380-8c65-ec0e6e1df53b\") " pod="openshift-monitoring/openshift-state-metrics-657fd6c54-5jhlq"
Sep 09 03:18:14.942945 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:18:14.942918    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/5f6c90ad-5940-4380-8c65-ec0e6e1df53b-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-657fd6c54-5jhlq\" (UID: \"5f6c90ad-5940-4380-8c65-ec0e6e1df53b\") " pod="openshift-monitoring/openshift-state-metrics-657fd6c54-5jhlq"
Sep 09 03:18:14.949337 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:18:14.949285    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-h6l6l\" (UniqueName: \"kubernetes.io/projected/5f6c90ad-5940-4380-8c65-ec0e6e1df53b-kube-api-access-h6l6l\") pod \"openshift-state-metrics-657fd6c54-5jhlq\" (UID: \"5f6c90ad-5940-4380-8c65-ec0e6e1df53b\") " pod="openshift-monitoring/openshift-state-metrics-657fd6c54-5jhlq"
Sep 09 03:18:15.041424 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:18:15.041389    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/8845a405-bcfc-4690-bfc8-7868d670f68c-node-exporter-tls\") pod \"node-exporter-dts6n\" (UID: \"8845a405-bcfc-4690-bfc8-7868d670f68c\") " pod="openshift-monitoring/node-exporter-dts6n"
Sep 09 03:18:15.041604 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:18:15.041443    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/8845a405-bcfc-4690-bfc8-7868d670f68c-metrics-client-ca\") pod \"node-exporter-dts6n\" (UID: \"8845a405-bcfc-4690-bfc8-7868d670f68c\") " pod="openshift-monitoring/node-exporter-dts6n"
Sep 09 03:18:15.041604 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:18:15.041465    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tkz78\" (UniqueName: \"kubernetes.io/projected/8845a405-bcfc-4690-bfc8-7868d670f68c-kube-api-access-tkz78\") pod \"node-exporter-dts6n\" (UID: \"8845a405-bcfc-4690-bfc8-7868d670f68c\") " pod="openshift-monitoring/node-exporter-dts6n"
Sep 09 03:18:15.041604 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:18:15.041491    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/8845a405-bcfc-4690-bfc8-7868d670f68c-node-exporter-wtmp\") pod \"node-exporter-dts6n\" (UID: \"8845a405-bcfc-4690-bfc8-7868d670f68c\") " pod="openshift-monitoring/node-exporter-dts6n"
Sep 09 03:18:15.041604 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:18:15.041519    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/8845a405-bcfc-4690-bfc8-7868d670f68c-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-dts6n\" (UID: \"8845a405-bcfc-4690-bfc8-7868d670f68c\") " pod="openshift-monitoring/node-exporter-dts6n"
Sep 09 03:18:15.041604 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:18:15.041556    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/8845a405-bcfc-4690-bfc8-7868d670f68c-node-exporter-textfile\") pod \"node-exporter-dts6n\" (UID: \"8845a405-bcfc-4690-bfc8-7868d670f68c\") " pod="openshift-monitoring/node-exporter-dts6n"
Sep 09 03:18:15.041604 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:18:15.041579    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/8845a405-bcfc-4690-bfc8-7868d670f68c-node-exporter-accelerators-collector-config\") pod \"node-exporter-dts6n\" (UID: \"8845a405-bcfc-4690-bfc8-7868d670f68c\") " pod="openshift-monitoring/node-exporter-dts6n"
Sep 09 03:18:15.041930 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:18:15.041683    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/8845a405-bcfc-4690-bfc8-7868d670f68c-node-exporter-wtmp\") pod \"node-exporter-dts6n\" (UID: \"8845a405-bcfc-4690-bfc8-7868d670f68c\") " pod="openshift-monitoring/node-exporter-dts6n"
Sep 09 03:18:15.041930 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:18:15.041687    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/8845a405-bcfc-4690-bfc8-7868d670f68c-root\") pod \"node-exporter-dts6n\" (UID: \"8845a405-bcfc-4690-bfc8-7868d670f68c\") " pod="openshift-monitoring/node-exporter-dts6n"
Sep 09 03:18:15.041930 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:18:15.041728    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/8845a405-bcfc-4690-bfc8-7868d670f68c-root\") pod \"node-exporter-dts6n\" (UID: \"8845a405-bcfc-4690-bfc8-7868d670f68c\") " pod="openshift-monitoring/node-exporter-dts6n"
Sep 09 03:18:15.041930 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:18:15.041743    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/8845a405-bcfc-4690-bfc8-7868d670f68c-sys\") pod \"node-exporter-dts6n\" (UID: \"8845a405-bcfc-4690-bfc8-7868d670f68c\") " pod="openshift-monitoring/node-exporter-dts6n"
Sep 09 03:18:15.041930 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:18:15.041776    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/8845a405-bcfc-4690-bfc8-7868d670f68c-sys\") pod \"node-exporter-dts6n\" (UID: \"8845a405-bcfc-4690-bfc8-7868d670f68c\") " pod="openshift-monitoring/node-exporter-dts6n"
Sep 09 03:18:15.042122 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:18:15.041959    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/8845a405-bcfc-4690-bfc8-7868d670f68c-node-exporter-textfile\") pod \"node-exporter-dts6n\" (UID: \"8845a405-bcfc-4690-bfc8-7868d670f68c\") " pod="openshift-monitoring/node-exporter-dts6n"
Sep 09 03:18:15.042258 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:18:15.042229    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/8845a405-bcfc-4690-bfc8-7868d670f68c-node-exporter-accelerators-collector-config\") pod \"node-exporter-dts6n\" (UID: \"8845a405-bcfc-4690-bfc8-7868d670f68c\") " pod="openshift-monitoring/node-exporter-dts6n"
Sep 09 03:18:15.042350 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:18:15.042334    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/8845a405-bcfc-4690-bfc8-7868d670f68c-metrics-client-ca\") pod \"node-exporter-dts6n\" (UID: \"8845a405-bcfc-4690-bfc8-7868d670f68c\") " pod="openshift-monitoring/node-exporter-dts6n"
Sep 09 03:18:15.043856 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:18:15.043840    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/8845a405-bcfc-4690-bfc8-7868d670f68c-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-dts6n\" (UID: \"8845a405-bcfc-4690-bfc8-7868d670f68c\") " pod="openshift-monitoring/node-exporter-dts6n"
Sep 09 03:18:15.043955 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:18:15.043936    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/8845a405-bcfc-4690-bfc8-7868d670f68c-node-exporter-tls\") pod \"node-exporter-dts6n\" (UID: \"8845a405-bcfc-4690-bfc8-7868d670f68c\") " pod="openshift-monitoring/node-exporter-dts6n"
Sep 09 03:18:15.049920 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:18:15.049895    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tkz78\" (UniqueName: \"kubernetes.io/projected/8845a405-bcfc-4690-bfc8-7868d670f68c-kube-api-access-tkz78\") pod \"node-exporter-dts6n\" (UID: \"8845a405-bcfc-4690-bfc8-7868d670f68c\") " pod="openshift-monitoring/node-exporter-dts6n"
Sep 09 03:18:15.061441 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:18:15.061420    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-dts6n"
Sep 09 03:18:15.072656 ip-10-0-132-14 kubenswrapper[2576]: W0909 03:18:15.072634    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8845a405_bcfc_4690_bfc8_7868d670f68c.slice/crio-097889f166db669128fa7bbbe80a1c1ab0c4da51059897294cbe94a7694890fb WatchSource:0}: Error finding container 097889f166db669128fa7bbbe80a1c1ab0c4da51059897294cbe94a7694890fb: Status 404 returned error can't find the container with id 097889f166db669128fa7bbbe80a1c1ab0c4da51059897294cbe94a7694890fb
Sep 09 03:18:15.074298 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:18:15.074284    2576 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 09 03:18:15.386734 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:18:15.386694    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-dts6n" event={"ID":"8845a405-bcfc-4690-bfc8-7868d670f68c","Type":"ContainerStarted","Data":"097889f166db669128fa7bbbe80a1c1ab0c4da51059897294cbe94a7694890fb"}
Sep 09 03:18:15.446198 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:18:15.446159    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/5f6c90ad-5940-4380-8c65-ec0e6e1df53b-openshift-state-metrics-tls\") pod \"openshift-state-metrics-657fd6c54-5jhlq\" (UID: \"5f6c90ad-5940-4380-8c65-ec0e6e1df53b\") " pod="openshift-monitoring/openshift-state-metrics-657fd6c54-5jhlq"
Sep 09 03:18:15.448667 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:18:15.448640    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/5f6c90ad-5940-4380-8c65-ec0e6e1df53b-openshift-state-metrics-tls\") pod \"openshift-state-metrics-657fd6c54-5jhlq\" (UID: \"5f6c90ad-5940-4380-8c65-ec0e6e1df53b\") " pod="openshift-monitoring/openshift-state-metrics-657fd6c54-5jhlq"
Sep 09 03:18:15.609012 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:18:15.608979    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/openshift-state-metrics-657fd6c54-5jhlq"
Sep 09 03:18:15.894545 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:18:15.894525    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/openshift-state-metrics-657fd6c54-5jhlq"]
Sep 09 03:18:15.934775 ip-10-0-132-14 kubenswrapper[2576]: W0909 03:18:15.934702    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5f6c90ad_5940_4380_8c65_ec0e6e1df53b.slice/crio-eaac0d9d2beef9becc6a7d11b318381188317087f7c3d4d746d352080065ba9e WatchSource:0}: Error finding container eaac0d9d2beef9becc6a7d11b318381188317087f7c3d4d746d352080065ba9e: Status 404 returned error can't find the container with id eaac0d9d2beef9becc6a7d11b318381188317087f7c3d4d746d352080065ba9e
Sep 09 03:18:16.392236 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:18:16.392197    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-657fd6c54-5jhlq" event={"ID":"5f6c90ad-5940-4380-8c65-ec0e6e1df53b","Type":"ContainerStarted","Data":"8127eb4712f3c8800fe18752fb4b4f2fa2cff0717880c5fa5ac1e156a8a89676"}
Sep 09 03:18:16.392660 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:18:16.392259    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-657fd6c54-5jhlq" event={"ID":"5f6c90ad-5940-4380-8c65-ec0e6e1df53b","Type":"ContainerStarted","Data":"39720182b96dc8f4f17f580bffa01172b9fd6621e73c877f4e300ce295e4b9ac"}
Sep 09 03:18:16.392660 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:18:16.392287    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-657fd6c54-5jhlq" event={"ID":"5f6c90ad-5940-4380-8c65-ec0e6e1df53b","Type":"ContainerStarted","Data":"eaac0d9d2beef9becc6a7d11b318381188317087f7c3d4d746d352080065ba9e"}
Sep 09 03:18:16.393704 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:18:16.393672    2576 generic.go:356] "Generic (PLEG): container finished" podID="8845a405-bcfc-4690-bfc8-7868d670f68c" containerID="c76301c92fbd047afe20ec9822a184a92532e7bb3a16023bb4ce44acf273e51c" exitCode=0
Sep 09 03:18:16.393834 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:18:16.393716    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-dts6n" event={"ID":"8845a405-bcfc-4690-bfc8-7868d670f68c","Type":"ContainerDied","Data":"c76301c92fbd047afe20ec9822a184a92532e7bb3a16023bb4ce44acf273e51c"}
Sep 09 03:18:17.399267 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:18:17.399220    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-657fd6c54-5jhlq" event={"ID":"5f6c90ad-5940-4380-8c65-ec0e6e1df53b","Type":"ContainerStarted","Data":"6da8ffb3254ba81e16fe90c203b3f66d7245a3aed7c5f1b5df9157219999bc08"}
Sep 09 03:18:17.401341 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:18:17.401315    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-dts6n" event={"ID":"8845a405-bcfc-4690-bfc8-7868d670f68c","Type":"ContainerStarted","Data":"ece931b12dfff58ce126c349474c645ce9488898151d83e1c1e67695c3d9ee6e"}
Sep 09 03:18:17.401449 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:18:17.401366    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-dts6n" event={"ID":"8845a405-bcfc-4690-bfc8-7868d670f68c","Type":"ContainerStarted","Data":"d4dc18181bcaeb8e280082a1fc3751529d2ec876243f57a57688dd08042fb8e3"}
Sep 09 03:18:17.423597 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:18:17.423549    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/openshift-state-metrics-657fd6c54-5jhlq" podStartSLOduration=2.594614209 podStartE2EDuration="3.423534437s" totalImagesPullingTime="828.920228ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 03:18:14 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 03:18:17.422990801 +0000 UTC m=+311.564476877" watchObservedRunningTime="2026-09-09 03:18:17.423534437 +0000 UTC m=+311.565020515"
Sep 09 03:18:17.444075 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:18:17.444030    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-dts6n" podStartSLOduration=2.467973973 podStartE2EDuration="3.444015822s" totalImagesPullingTime="904.693002ms" totalInitContainerRuntime="71.348847ms" isStatefulPod=true podCreationTimestamp="2026-09-09 03:18:14 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 03:18:17.443827672 +0000 UTC m=+311.585313743" watchObservedRunningTime="2026-09-09 03:18:17.444015822 +0000 UTC m=+311.585501899"
Sep 09 03:18:41.320411 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:18:41.320373    2576 prober.go:120] "Probe failed" probeType="Liveness" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-775666786-94lvs" podUID="33582a33-efef-482b-95f3-aabe6c109fb6" containerName="service-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 500"
Sep 09 03:18:51.320593 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:18:51.320536    2576 prober.go:120] "Probe failed" probeType="Liveness" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-775666786-94lvs" podUID="33582a33-efef-482b-95f3-aabe6c109fb6" containerName="service-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 500"
Sep 09 03:19:01.319761 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:19:01.319670    2576 prober.go:120] "Probe failed" probeType="Liveness" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-775666786-94lvs" podUID="33582a33-efef-482b-95f3-aabe6c109fb6" containerName="service-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 500"
Sep 09 03:19:01.319761 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:19:01.319751    2576 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-775666786-94lvs"
Sep 09 03:19:01.320403 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:19:01.320368    2576 kuberuntime_manager.go:1192] "Message for Container of pod" containerName="service-proxy" containerStatusID={"Type":"cri-o","ID":"f55b48c538d6cf575cf72443ed39baa642ec5a0fb6cfd46e82d7b04d2acb6bb6"} pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-775666786-94lvs" containerMessage="Container service-proxy failed liveness probe, will be restarted"
Sep 09 03:19:01.320484 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:19:01.320431    2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-775666786-94lvs" podUID="33582a33-efef-482b-95f3-aabe6c109fb6" containerName="service-proxy" containerID="cri-o://f55b48c538d6cf575cf72443ed39baa642ec5a0fb6cfd46e82d7b04d2acb6bb6" gracePeriod=30
Sep 09 03:19:01.546772 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:19:01.546740    2576 generic.go:356] "Generic (PLEG): container finished" podID="33582a33-efef-482b-95f3-aabe6c109fb6" containerID="f55b48c538d6cf575cf72443ed39baa642ec5a0fb6cfd46e82d7b04d2acb6bb6" exitCode=2
Sep 09 03:19:01.546969 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:19:01.546788    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-775666786-94lvs" event={"ID":"33582a33-efef-482b-95f3-aabe6c109fb6","Type":"ContainerDied","Data":"f55b48c538d6cf575cf72443ed39baa642ec5a0fb6cfd46e82d7b04d2acb6bb6"}
Sep 09 03:19:01.546969 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:19:01.546836    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-775666786-94lvs" event={"ID":"33582a33-efef-482b-95f3-aabe6c109fb6","Type":"ContainerStarted","Data":"42c3cecdb71991801ae6c0d743e258e80b9bee9f97a37477f8acdfe914dcc053"}
Sep 09 03:19:19.332518 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:19:19.332478    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/443a2987-5246-4a8c-8631-1b751e02101b-metrics-certs\") pod \"network-metrics-daemon-99qzh\" (UID: \"443a2987-5246-4a8c-8631-1b751e02101b\") " pod="openshift-multus/network-metrics-daemon-99qzh"
Sep 09 03:19:19.334984 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:19:19.334960    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/443a2987-5246-4a8c-8631-1b751e02101b-metrics-certs\") pod \"network-metrics-daemon-99qzh\" (UID: \"443a2987-5246-4a8c-8631-1b751e02101b\") " pod="openshift-multus/network-metrics-daemon-99qzh"
Sep 09 03:19:19.557992 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:19:19.557957    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-rkcrq\""
Sep 09 03:19:19.565355 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:19:19.565335    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-99qzh"
Sep 09 03:19:19.839879 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:19:19.839855    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-99qzh"]
Sep 09 03:19:19.842112 ip-10-0-132-14 kubenswrapper[2576]: W0909 03:19:19.842080    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod443a2987_5246_4a8c_8631_1b751e02101b.slice/crio-dba5360d0a9ba53974e5222114a26ce4235ca3fde24d9a4ffd7579ef66541c51 WatchSource:0}: Error finding container dba5360d0a9ba53974e5222114a26ce4235ca3fde24d9a4ffd7579ef66541c51: Status 404 returned error can't find the container with id dba5360d0a9ba53974e5222114a26ce4235ca3fde24d9a4ffd7579ef66541c51
Sep 09 03:19:20.609533 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:19:20.609497    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-99qzh" event={"ID":"443a2987-5246-4a8c-8631-1b751e02101b","Type":"ContainerStarted","Data":"dba5360d0a9ba53974e5222114a26ce4235ca3fde24d9a4ffd7579ef66541c51"}
Sep 09 03:19:21.614990 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:19:21.614954    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-99qzh" event={"ID":"443a2987-5246-4a8c-8631-1b751e02101b","Type":"ContainerStarted","Data":"09d6c03c41e44157f7280d08e7af2c39dffe941a237e999422af85ddc047bd58"}
Sep 09 03:19:21.615322 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:19:21.615037    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-99qzh" event={"ID":"443a2987-5246-4a8c-8631-1b751e02101b","Type":"ContainerStarted","Data":"e8ce1f802e20dab7a808e62466d639a594cfcac0fbd75660146e97010a8aa470"}
Sep 09 03:19:21.633781 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:19:21.633612    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-99qzh" podStartSLOduration=374.085450957 podStartE2EDuration="6m15.633593074s" totalImagesPullingTime="1.548142117s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 03:13:06 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 03:19:21.633240934 +0000 UTC m=+375.774727012" watchObservedRunningTime="2026-09-09 03:19:21.633593074 +0000 UTC m=+375.775079150"
Sep 09 03:20:55.495367 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:20:55.495332    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-4rzmd"]
Sep 09 03:20:55.497202 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:20:55.497186    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-4rzmd"
Sep 09 03:20:55.499972 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:20:55.499948    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager-operator\"/\"openshift-service-ca.crt\""
Sep 09 03:20:55.500085 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:20:55.499948    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager-operator\"/\"cert-manager-operator-controller-manager-dockercfg-g2fls\""
Sep 09 03:20:55.500145 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:20:55.500130    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager-operator\"/\"kube-root-ca.crt\""
Sep 09 03:20:55.511226 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:20:55.511197    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-4rzmd"]
Sep 09 03:20:55.577691 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:20:55.574160    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/323d1e32-49ae-40ed-bac9-621b2bf419de-tmp\") pod \"cert-manager-operator-controller-manager-55c444bf86-4rzmd\" (UID: \"323d1e32-49ae-40ed-bac9-621b2bf419de\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-4rzmd"
Sep 09 03:20:55.577691 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:20:55.574236    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bzm8p\" (UniqueName: \"kubernetes.io/projected/323d1e32-49ae-40ed-bac9-621b2bf419de-kube-api-access-bzm8p\") pod \"cert-manager-operator-controller-manager-55c444bf86-4rzmd\" (UID: \"323d1e32-49ae-40ed-bac9-621b2bf419de\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-4rzmd"
Sep 09 03:20:55.675359 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:20:55.675314    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/323d1e32-49ae-40ed-bac9-621b2bf419de-tmp\") pod \"cert-manager-operator-controller-manager-55c444bf86-4rzmd\" (UID: \"323d1e32-49ae-40ed-bac9-621b2bf419de\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-4rzmd"
Sep 09 03:20:55.675557 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:20:55.675374    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bzm8p\" (UniqueName: \"kubernetes.io/projected/323d1e32-49ae-40ed-bac9-621b2bf419de-kube-api-access-bzm8p\") pod \"cert-manager-operator-controller-manager-55c444bf86-4rzmd\" (UID: \"323d1e32-49ae-40ed-bac9-621b2bf419de\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-4rzmd"
Sep 09 03:20:55.675711 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:20:55.675687    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/323d1e32-49ae-40ed-bac9-621b2bf419de-tmp\") pod \"cert-manager-operator-controller-manager-55c444bf86-4rzmd\" (UID: \"323d1e32-49ae-40ed-bac9-621b2bf419de\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-4rzmd"
Sep 09 03:20:55.684847 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:20:55.684796    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bzm8p\" (UniqueName: \"kubernetes.io/projected/323d1e32-49ae-40ed-bac9-621b2bf419de-kube-api-access-bzm8p\") pod \"cert-manager-operator-controller-manager-55c444bf86-4rzmd\" (UID: \"323d1e32-49ae-40ed-bac9-621b2bf419de\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-4rzmd"
Sep 09 03:20:55.806396 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:20:55.806356    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-4rzmd"
Sep 09 03:20:56.088561 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:20:56.088489    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-4rzmd"]
Sep 09 03:20:56.092018 ip-10-0-132-14 kubenswrapper[2576]: W0909 03:20:56.091982    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod323d1e32_49ae_40ed_bac9_621b2bf419de.slice/crio-5bbb901b16b6032da8d1a07441ed1950f2c7e1a318fe345949f47fe43382b96e WatchSource:0}: Error finding container 5bbb901b16b6032da8d1a07441ed1950f2c7e1a318fe345949f47fe43382b96e: Status 404 returned error can't find the container with id 5bbb901b16b6032da8d1a07441ed1950f2c7e1a318fe345949f47fe43382b96e
Sep 09 03:20:56.942758 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:20:56.942715    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-4rzmd" event={"ID":"323d1e32-49ae-40ed-bac9-621b2bf419de","Type":"ContainerStarted","Data":"5bbb901b16b6032da8d1a07441ed1950f2c7e1a318fe345949f47fe43382b96e"}
Sep 09 03:20:58.954023 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:20:58.953984    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-4rzmd" event={"ID":"323d1e32-49ae-40ed-bac9-621b2bf419de","Type":"ContainerStarted","Data":"e663c38c20e741bf01efedc1ab5a709c09f70d06c540100cf96805aca952863e"}
Sep 09 03:20:58.977543 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:20:58.977482    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-4rzmd" podStartSLOduration=1.770018707 podStartE2EDuration="3.97746575s" totalImagesPullingTime="2.207447043s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 03:20:55 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 03:20:58.975109904 +0000 UTC m=+473.116595981" watchObservedRunningTime="2026-09-09 03:20:58.97746575 +0000 UTC m=+473.118951827"
Sep 09 03:21:37.252453 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:21:37.252416    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-ff49c877-z6z99"]
Sep 09 03:21:37.255795 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:21:37.255778    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-ff49c877-z6z99"
Sep 09 03:21:37.258551 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:21:37.258502    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\""
Sep 09 03:21:37.258695 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:21:37.258567    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-jcqpg\""
Sep 09 03:21:37.258695 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:21:37.258581    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\""
Sep 09 03:21:37.258865 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:21:37.258697    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\""
Sep 09 03:21:37.259058 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:21:37.259042    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\""
Sep 09 03:21:37.280414 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:21:37.280374    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-ff49c877-z6z99"]
Sep 09 03:21:37.396119 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:21:37.396087    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/57afb6b9-bdb8-45ce-8a10-2d8b722bb37d-manifests\") pod \"opendatahub-operator-controller-manager-ff49c877-z6z99\" (UID: \"57afb6b9-bdb8-45ce-8a10-2d8b722bb37d\") " pod="opendatahub/opendatahub-operator-controller-manager-ff49c877-z6z99"
Sep 09 03:21:37.396299 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:21:37.396130    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qkxdp\" (UniqueName: \"kubernetes.io/projected/57afb6b9-bdb8-45ce-8a10-2d8b722bb37d-kube-api-access-qkxdp\") pod \"opendatahub-operator-controller-manager-ff49c877-z6z99\" (UID: \"57afb6b9-bdb8-45ce-8a10-2d8b722bb37d\") " pod="opendatahub/opendatahub-operator-controller-manager-ff49c877-z6z99"
Sep 09 03:21:37.396299 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:21:37.396172    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/57afb6b9-bdb8-45ce-8a10-2d8b722bb37d-apiservice-cert\") pod \"opendatahub-operator-controller-manager-ff49c877-z6z99\" (UID: \"57afb6b9-bdb8-45ce-8a10-2d8b722bb37d\") " pod="opendatahub/opendatahub-operator-controller-manager-ff49c877-z6z99"
Sep 09 03:21:37.396299 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:21:37.396203    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/57afb6b9-bdb8-45ce-8a10-2d8b722bb37d-webhook-cert\") pod \"opendatahub-operator-controller-manager-ff49c877-z6z99\" (UID: \"57afb6b9-bdb8-45ce-8a10-2d8b722bb37d\") " pod="opendatahub/opendatahub-operator-controller-manager-ff49c877-z6z99"
Sep 09 03:21:37.396299 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:21:37.396222    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/57afb6b9-bdb8-45ce-8a10-2d8b722bb37d-tmp\") pod \"opendatahub-operator-controller-manager-ff49c877-z6z99\" (UID: \"57afb6b9-bdb8-45ce-8a10-2d8b722bb37d\") " pod="opendatahub/opendatahub-operator-controller-manager-ff49c877-z6z99"
Sep 09 03:21:37.497540 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:21:37.497512    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/57afb6b9-bdb8-45ce-8a10-2d8b722bb37d-manifests\") pod \"opendatahub-operator-controller-manager-ff49c877-z6z99\" (UID: \"57afb6b9-bdb8-45ce-8a10-2d8b722bb37d\") " pod="opendatahub/opendatahub-operator-controller-manager-ff49c877-z6z99"
Sep 09 03:21:37.497716 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:21:37.497555    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qkxdp\" (UniqueName: \"kubernetes.io/projected/57afb6b9-bdb8-45ce-8a10-2d8b722bb37d-kube-api-access-qkxdp\") pod \"opendatahub-operator-controller-manager-ff49c877-z6z99\" (UID: \"57afb6b9-bdb8-45ce-8a10-2d8b722bb37d\") " pod="opendatahub/opendatahub-operator-controller-manager-ff49c877-z6z99"
Sep 09 03:21:37.497716 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:21:37.497585    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/57afb6b9-bdb8-45ce-8a10-2d8b722bb37d-apiservice-cert\") pod \"opendatahub-operator-controller-manager-ff49c877-z6z99\" (UID: \"57afb6b9-bdb8-45ce-8a10-2d8b722bb37d\") " pod="opendatahub/opendatahub-operator-controller-manager-ff49c877-z6z99"
Sep 09 03:21:37.497786 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:21:37.497758    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/57afb6b9-bdb8-45ce-8a10-2d8b722bb37d-webhook-cert\") pod \"opendatahub-operator-controller-manager-ff49c877-z6z99\" (UID: \"57afb6b9-bdb8-45ce-8a10-2d8b722bb37d\") " pod="opendatahub/opendatahub-operator-controller-manager-ff49c877-z6z99"
Sep 09 03:21:37.497850 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:21:37.497791    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/57afb6b9-bdb8-45ce-8a10-2d8b722bb37d-tmp\") pod \"opendatahub-operator-controller-manager-ff49c877-z6z99\" (UID: \"57afb6b9-bdb8-45ce-8a10-2d8b722bb37d\") " pod="opendatahub/opendatahub-operator-controller-manager-ff49c877-z6z99"
Sep 09 03:21:37.497912 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:21:37.497885    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/57afb6b9-bdb8-45ce-8a10-2d8b722bb37d-manifests\") pod \"opendatahub-operator-controller-manager-ff49c877-z6z99\" (UID: \"57afb6b9-bdb8-45ce-8a10-2d8b722bb37d\") " pod="opendatahub/opendatahub-operator-controller-manager-ff49c877-z6z99"
Sep 09 03:21:37.500027 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:21:37.500003    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/57afb6b9-bdb8-45ce-8a10-2d8b722bb37d-tmp\") pod \"opendatahub-operator-controller-manager-ff49c877-z6z99\" (UID: \"57afb6b9-bdb8-45ce-8a10-2d8b722bb37d\") " pod="opendatahub/opendatahub-operator-controller-manager-ff49c877-z6z99"
Sep 09 03:21:37.500132 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:21:37.500096    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/57afb6b9-bdb8-45ce-8a10-2d8b722bb37d-apiservice-cert\") pod \"opendatahub-operator-controller-manager-ff49c877-z6z99\" (UID: \"57afb6b9-bdb8-45ce-8a10-2d8b722bb37d\") " pod="opendatahub/opendatahub-operator-controller-manager-ff49c877-z6z99"
Sep 09 03:21:37.500211 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:21:37.500194    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/57afb6b9-bdb8-45ce-8a10-2d8b722bb37d-webhook-cert\") pod \"opendatahub-operator-controller-manager-ff49c877-z6z99\" (UID: \"57afb6b9-bdb8-45ce-8a10-2d8b722bb37d\") " pod="opendatahub/opendatahub-operator-controller-manager-ff49c877-z6z99"
Sep 09 03:21:37.506663 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:21:37.506607    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qkxdp\" (UniqueName: \"kubernetes.io/projected/57afb6b9-bdb8-45ce-8a10-2d8b722bb37d-kube-api-access-qkxdp\") pod \"opendatahub-operator-controller-manager-ff49c877-z6z99\" (UID: \"57afb6b9-bdb8-45ce-8a10-2d8b722bb37d\") " pod="opendatahub/opendatahub-operator-controller-manager-ff49c877-z6z99"
Sep 09 03:21:37.566051 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:21:37.566010    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-ff49c877-z6z99"
Sep 09 03:21:37.843508 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:21:37.843437    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-ff49c877-z6z99"]
Sep 09 03:21:37.846693 ip-10-0-132-14 kubenswrapper[2576]: W0909 03:21:37.846670    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod57afb6b9_bdb8_45ce_8a10_2d8b722bb37d.slice/crio-1035e2213339020d64312a87e80265a7cc9aa32242b38114f387fd5e8ba4ddcd WatchSource:0}: Error finding container 1035e2213339020d64312a87e80265a7cc9aa32242b38114f387fd5e8ba4ddcd: Status 404 returned error can't find the container with id 1035e2213339020d64312a87e80265a7cc9aa32242b38114f387fd5e8ba4ddcd
Sep 09 03:21:38.090102 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:21:38.090067    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-ff49c877-z6z99" event={"ID":"57afb6b9-bdb8-45ce-8a10-2d8b722bb37d","Type":"ContainerStarted","Data":"1035e2213339020d64312a87e80265a7cc9aa32242b38114f387fd5e8ba4ddcd"}
Sep 09 03:21:41.105551 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:21:41.105457    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-ff49c877-z6z99" event={"ID":"57afb6b9-bdb8-45ce-8a10-2d8b722bb37d","Type":"ContainerStarted","Data":"b12cb2487fc1ab5e307a54fc94e49c685eb72b40dc97f93cd4a9a1a7e45a4c5b"}
Sep 09 03:21:42.110592 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:21:42.110556    2576 generic.go:356] "Generic (PLEG): container finished" podID="57afb6b9-bdb8-45ce-8a10-2d8b722bb37d" containerID="b12cb2487fc1ab5e307a54fc94e49c685eb72b40dc97f93cd4a9a1a7e45a4c5b" exitCode=0
Sep 09 03:21:42.111016 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:21:42.110650    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-ff49c877-z6z99" event={"ID":"57afb6b9-bdb8-45ce-8a10-2d8b722bb37d","Type":"ContainerDied","Data":"b12cb2487fc1ab5e307a54fc94e49c685eb72b40dc97f93cd4a9a1a7e45a4c5b"}
Sep 09 03:21:43.116410 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:21:43.116376    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-ff49c877-z6z99" event={"ID":"57afb6b9-bdb8-45ce-8a10-2d8b722bb37d","Type":"ContainerStarted","Data":"73e54aa42294e840e7b7d36b9ef451c2b4311f57c051363589a56c1d464b9f3a"}
Sep 09 03:21:43.116844 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:21:43.116578    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-ff49c877-z6z99"
Sep 09 03:21:43.141729 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:21:43.141666    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-ff49c877-z6z99" podStartSLOduration=1.640602879 podStartE2EDuration="6.141647292s" totalImagesPullingTime="3.464187633s" totalInitContainerRuntime="1.03685678s" isStatefulPod=false podCreationTimestamp="2026-09-09 03:21:37 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 03:21:43.139234815 +0000 UTC m=+517.280720893" watchObservedRunningTime="2026-09-09 03:21:43.141647292 +0000 UTC m=+517.283133369"
Sep 09 03:21:54.121922 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:21:54.121885    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-ff49c877-z6z99"
Sep 09 03:21:54.995305 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:21:54.995268    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/lws-controller-manager-6f5d5c9c64-vkprt"]
Sep 09 03:21:55.008347 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:21:55.008317    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-6f5d5c9c64-vkprt"]
Sep 09 03:21:55.008497 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:21:55.008419    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-6f5d5c9c64-vkprt"
Sep 09 03:21:55.012542 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:21:55.012520    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"lws-manager-config\""
Sep 09 03:21:55.012674 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:21:55.012546    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"lws-controller-manager-dockercfg-r7mfk\""
Sep 09 03:21:55.012674 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:21:55.012575    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"metrics-server-cert\""
Sep 09 03:21:55.012674 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:21:55.012584    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"openshift-service-ca.crt\""
Sep 09 03:21:55.012931 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:21:55.012912    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"kube-root-ca.crt\""
Sep 09 03:21:55.013140 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:21:55.013125    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"webhook-server-cert\""
Sep 09 03:21:55.025778 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:21:55.025753    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/5df2bb00-24b1-45ad-a4e7-20a0f086ffce-manager-config\") pod \"lws-controller-manager-6f5d5c9c64-vkprt\" (UID: \"5df2bb00-24b1-45ad-a4e7-20a0f086ffce\") " pod="openshift-lws-operator/lws-controller-manager-6f5d5c9c64-vkprt"
Sep 09 03:21:55.025913 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:21:55.025782    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/5df2bb00-24b1-45ad-a4e7-20a0f086ffce-cert\") pod \"lws-controller-manager-6f5d5c9c64-vkprt\" (UID: \"5df2bb00-24b1-45ad-a4e7-20a0f086ffce\") " pod="openshift-lws-operator/lws-controller-manager-6f5d5c9c64-vkprt"
Sep 09 03:21:55.025913 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:21:55.025816    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mt46b\" (UniqueName: \"kubernetes.io/projected/5df2bb00-24b1-45ad-a4e7-20a0f086ffce-kube-api-access-mt46b\") pod \"lws-controller-manager-6f5d5c9c64-vkprt\" (UID: \"5df2bb00-24b1-45ad-a4e7-20a0f086ffce\") " pod="openshift-lws-operator/lws-controller-manager-6f5d5c9c64-vkprt"
Sep 09 03:21:55.025913 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:21:55.025835    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/5df2bb00-24b1-45ad-a4e7-20a0f086ffce-metrics-cert\") pod \"lws-controller-manager-6f5d5c9c64-vkprt\" (UID: \"5df2bb00-24b1-45ad-a4e7-20a0f086ffce\") " pod="openshift-lws-operator/lws-controller-manager-6f5d5c9c64-vkprt"
Sep 09 03:21:55.126857 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:21:55.126822    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/5df2bb00-24b1-45ad-a4e7-20a0f086ffce-manager-config\") pod \"lws-controller-manager-6f5d5c9c64-vkprt\" (UID: \"5df2bb00-24b1-45ad-a4e7-20a0f086ffce\") " pod="openshift-lws-operator/lws-controller-manager-6f5d5c9c64-vkprt"
Sep 09 03:21:55.126857 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:21:55.126859    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/5df2bb00-24b1-45ad-a4e7-20a0f086ffce-cert\") pod \"lws-controller-manager-6f5d5c9c64-vkprt\" (UID: \"5df2bb00-24b1-45ad-a4e7-20a0f086ffce\") " pod="openshift-lws-operator/lws-controller-manager-6f5d5c9c64-vkprt"
Sep 09 03:21:55.127301 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:21:55.126887    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mt46b\" (UniqueName: \"kubernetes.io/projected/5df2bb00-24b1-45ad-a4e7-20a0f086ffce-kube-api-access-mt46b\") pod \"lws-controller-manager-6f5d5c9c64-vkprt\" (UID: \"5df2bb00-24b1-45ad-a4e7-20a0f086ffce\") " pod="openshift-lws-operator/lws-controller-manager-6f5d5c9c64-vkprt"
Sep 09 03:21:55.127301 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:21:55.126919    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/5df2bb00-24b1-45ad-a4e7-20a0f086ffce-metrics-cert\") pod \"lws-controller-manager-6f5d5c9c64-vkprt\" (UID: \"5df2bb00-24b1-45ad-a4e7-20a0f086ffce\") " pod="openshift-lws-operator/lws-controller-manager-6f5d5c9c64-vkprt"
Sep 09 03:21:55.127472 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:21:55.127439    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/5df2bb00-24b1-45ad-a4e7-20a0f086ffce-manager-config\") pod \"lws-controller-manager-6f5d5c9c64-vkprt\" (UID: \"5df2bb00-24b1-45ad-a4e7-20a0f086ffce\") " pod="openshift-lws-operator/lws-controller-manager-6f5d5c9c64-vkprt"
Sep 09 03:21:55.129494 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:21:55.129471    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/5df2bb00-24b1-45ad-a4e7-20a0f086ffce-cert\") pod \"lws-controller-manager-6f5d5c9c64-vkprt\" (UID: \"5df2bb00-24b1-45ad-a4e7-20a0f086ffce\") " pod="openshift-lws-operator/lws-controller-manager-6f5d5c9c64-vkprt"
Sep 09 03:21:55.129578 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:21:55.129549    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/5df2bb00-24b1-45ad-a4e7-20a0f086ffce-metrics-cert\") pod \"lws-controller-manager-6f5d5c9c64-vkprt\" (UID: \"5df2bb00-24b1-45ad-a4e7-20a0f086ffce\") " pod="openshift-lws-operator/lws-controller-manager-6f5d5c9c64-vkprt"
Sep 09 03:21:55.136470 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:21:55.136451    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mt46b\" (UniqueName: \"kubernetes.io/projected/5df2bb00-24b1-45ad-a4e7-20a0f086ffce-kube-api-access-mt46b\") pod \"lws-controller-manager-6f5d5c9c64-vkprt\" (UID: \"5df2bb00-24b1-45ad-a4e7-20a0f086ffce\") " pod="openshift-lws-operator/lws-controller-manager-6f5d5c9c64-vkprt"
Sep 09 03:21:55.318600 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:21:55.318571    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-6f5d5c9c64-vkprt"
Sep 09 03:21:55.592759 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:21:55.592691    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-6f5d5c9c64-vkprt"]
Sep 09 03:21:55.595023 ip-10-0-132-14 kubenswrapper[2576]: W0909 03:21:55.594997    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5df2bb00_24b1_45ad_a4e7_20a0f086ffce.slice/crio-8b71c5554148920282a1af5012190bee0038958b9a73185e1df73bd72b2ed32d WatchSource:0}: Error finding container 8b71c5554148920282a1af5012190bee0038958b9a73185e1df73bd72b2ed32d: Status 404 returned error can't find the container with id 8b71c5554148920282a1af5012190bee0038958b9a73185e1df73bd72b2ed32d
Sep 09 03:21:56.166843 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:21:56.166792    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-6f5d5c9c64-vkprt" event={"ID":"5df2bb00-24b1-45ad-a4e7-20a0f086ffce","Type":"ContainerStarted","Data":"8b71c5554148920282a1af5012190bee0038958b9a73185e1df73bd72b2ed32d"}
Sep 09 03:21:59.179686 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:21:59.179644    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-6f5d5c9c64-vkprt" event={"ID":"5df2bb00-24b1-45ad-a4e7-20a0f086ffce","Type":"ContainerStarted","Data":"83f42c921b4378189dce4f4c24098e25c8d4786f31dfddd823a900cf196ca5ee"}
Sep 09 03:21:59.180195 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:21:59.179768    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-lws-operator/lws-controller-manager-6f5d5c9c64-vkprt"
Sep 09 03:21:59.196787 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:21:59.196741    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-lws-operator/lws-controller-manager-6f5d5c9c64-vkprt" podStartSLOduration=2.649650956 podStartE2EDuration="5.196727573s" totalImagesPullingTime="2.547076617s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 03:21:54 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 03:21:59.194925714 +0000 UTC m=+533.336411791" watchObservedRunningTime="2026-09-09 03:21:59.196727573 +0000 UTC m=+533.338213650"
Sep 09 03:22:10.186480 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:22:10.186449    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-lws-operator/lws-controller-manager-6f5d5c9c64-vkprt"
Sep 09 03:22:15.171759 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:22:15.171727    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/istiod-openshift-gateway-5677ccb49b-rtcp7"]
Sep 09 03:22:15.176388 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:22:15.176370    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-rtcp7"
Sep 09 03:22:15.186059 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:22:15.186038    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"cacerts\""
Sep 09 03:22:15.186059 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:22:15.186056    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istio-kubeconfig\""
Sep 09 03:22:15.186236 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:22:15.186218    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\""
Sep 09 03:22:15.186324 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:22:15.186309    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\""
Sep 09 03:22:15.186389 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:22:15.186324    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-openshift-gateway-dockercfg-h2zjk\""
Sep 09 03:22:15.187258 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:22:15.187241    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-tls\""
Sep 09 03:22:15.187309 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:22:15.187247    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\""
Sep 09 03:22:15.207622 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:22:15.207584    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-5677ccb49b-rtcp7"]
Sep 09 03:22:15.291216 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:22:15.291189    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/8eb57cdf-33f8-483f-a943-d16317fad3bd-local-certs\") pod \"istiod-openshift-gateway-5677ccb49b-rtcp7\" (UID: \"8eb57cdf-33f8-483f-a943-d16317fad3bd\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-rtcp7"
Sep 09 03:22:15.291377 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:22:15.291230    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/8eb57cdf-33f8-483f-a943-d16317fad3bd-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-5677ccb49b-rtcp7\" (UID: \"8eb57cdf-33f8-483f-a943-d16317fad3bd\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-rtcp7"
Sep 09 03:22:15.291377 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:22:15.291254    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/8eb57cdf-33f8-483f-a943-d16317fad3bd-istio-kubeconfig\") pod \"istiod-openshift-gateway-5677ccb49b-rtcp7\" (UID: \"8eb57cdf-33f8-483f-a943-d16317fad3bd\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-rtcp7"
Sep 09 03:22:15.291377 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:22:15.291272    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/8eb57cdf-33f8-483f-a943-d16317fad3bd-cacerts\") pod \"istiod-openshift-gateway-5677ccb49b-rtcp7\" (UID: \"8eb57cdf-33f8-483f-a943-d16317fad3bd\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-rtcp7"
Sep 09 03:22:15.291377 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:22:15.291313    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-q7cjj\" (UniqueName: \"kubernetes.io/projected/8eb57cdf-33f8-483f-a943-d16317fad3bd-kube-api-access-q7cjj\") pod \"istiod-openshift-gateway-5677ccb49b-rtcp7\" (UID: \"8eb57cdf-33f8-483f-a943-d16317fad3bd\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-rtcp7"
Sep 09 03:22:15.291377 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:22:15.291357    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/8eb57cdf-33f8-483f-a943-d16317fad3bd-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-5677ccb49b-rtcp7\" (UID: \"8eb57cdf-33f8-483f-a943-d16317fad3bd\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-rtcp7"
Sep 09 03:22:15.291542 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:22:15.291381    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8eb57cdf-33f8-483f-a943-d16317fad3bd-istio-token\") pod \"istiod-openshift-gateway-5677ccb49b-rtcp7\" (UID: \"8eb57cdf-33f8-483f-a943-d16317fad3bd\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-rtcp7"
Sep 09 03:22:15.392087 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:22:15.392055    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/8eb57cdf-33f8-483f-a943-d16317fad3bd-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-5677ccb49b-rtcp7\" (UID: \"8eb57cdf-33f8-483f-a943-d16317fad3bd\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-rtcp7"
Sep 09 03:22:15.392272 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:22:15.392093    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/8eb57cdf-33f8-483f-a943-d16317fad3bd-istio-kubeconfig\") pod \"istiod-openshift-gateway-5677ccb49b-rtcp7\" (UID: \"8eb57cdf-33f8-483f-a943-d16317fad3bd\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-rtcp7"
Sep 09 03:22:15.392272 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:22:15.392110    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/8eb57cdf-33f8-483f-a943-d16317fad3bd-cacerts\") pod \"istiod-openshift-gateway-5677ccb49b-rtcp7\" (UID: \"8eb57cdf-33f8-483f-a943-d16317fad3bd\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-rtcp7"
Sep 09 03:22:15.392272 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:22:15.392133    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-q7cjj\" (UniqueName: \"kubernetes.io/projected/8eb57cdf-33f8-483f-a943-d16317fad3bd-kube-api-access-q7cjj\") pod \"istiod-openshift-gateway-5677ccb49b-rtcp7\" (UID: \"8eb57cdf-33f8-483f-a943-d16317fad3bd\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-rtcp7"
Sep 09 03:22:15.392272 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:22:15.392163    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/8eb57cdf-33f8-483f-a943-d16317fad3bd-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-5677ccb49b-rtcp7\" (UID: \"8eb57cdf-33f8-483f-a943-d16317fad3bd\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-rtcp7"
Sep 09 03:22:15.392486 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:22:15.392366    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8eb57cdf-33f8-483f-a943-d16317fad3bd-istio-token\") pod \"istiod-openshift-gateway-5677ccb49b-rtcp7\" (UID: \"8eb57cdf-33f8-483f-a943-d16317fad3bd\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-rtcp7"
Sep 09 03:22:15.392486 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:22:15.392436    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/8eb57cdf-33f8-483f-a943-d16317fad3bd-local-certs\") pod \"istiod-openshift-gateway-5677ccb49b-rtcp7\" (UID: \"8eb57cdf-33f8-483f-a943-d16317fad3bd\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-rtcp7"
Sep 09 03:22:15.392921 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:22:15.392845    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/8eb57cdf-33f8-483f-a943-d16317fad3bd-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-5677ccb49b-rtcp7\" (UID: \"8eb57cdf-33f8-483f-a943-d16317fad3bd\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-rtcp7"
Sep 09 03:22:15.394627 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:22:15.394604    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/8eb57cdf-33f8-483f-a943-d16317fad3bd-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-5677ccb49b-rtcp7\" (UID: \"8eb57cdf-33f8-483f-a943-d16317fad3bd\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-rtcp7"
Sep 09 03:22:15.394725 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:22:15.394693    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/8eb57cdf-33f8-483f-a943-d16317fad3bd-istio-kubeconfig\") pod \"istiod-openshift-gateway-5677ccb49b-rtcp7\" (UID: \"8eb57cdf-33f8-483f-a943-d16317fad3bd\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-rtcp7"
Sep 09 03:22:15.394725 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:22:15.394712    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/8eb57cdf-33f8-483f-a943-d16317fad3bd-local-certs\") pod \"istiod-openshift-gateway-5677ccb49b-rtcp7\" (UID: \"8eb57cdf-33f8-483f-a943-d16317fad3bd\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-rtcp7"
Sep 09 03:22:15.394793 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:22:15.394740    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/8eb57cdf-33f8-483f-a943-d16317fad3bd-cacerts\") pod \"istiod-openshift-gateway-5677ccb49b-rtcp7\" (UID: \"8eb57cdf-33f8-483f-a943-d16317fad3bd\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-rtcp7"
Sep 09 03:22:15.400589 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:22:15.400566    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-q7cjj\" (UniqueName: \"kubernetes.io/projected/8eb57cdf-33f8-483f-a943-d16317fad3bd-kube-api-access-q7cjj\") pod \"istiod-openshift-gateway-5677ccb49b-rtcp7\" (UID: \"8eb57cdf-33f8-483f-a943-d16317fad3bd\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-rtcp7"
Sep 09 03:22:15.400966 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:22:15.400946    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8eb57cdf-33f8-483f-a943-d16317fad3bd-istio-token\") pod \"istiod-openshift-gateway-5677ccb49b-rtcp7\" (UID: \"8eb57cdf-33f8-483f-a943-d16317fad3bd\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-rtcp7"
Sep 09 03:22:15.485515 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:22:15.485441    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-rtcp7"
Sep 09 03:22:15.766909 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:22:15.766878    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-5677ccb49b-rtcp7"]
Sep 09 03:22:15.769789 ip-10-0-132-14 kubenswrapper[2576]: W0909 03:22:15.769737    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8eb57cdf_33f8_483f_a943_d16317fad3bd.slice/crio-3584c5ccd3084c65012f30239c900161f8e63f1fcbf8d2c57ef559f45468c56d WatchSource:0}: Error finding container 3584c5ccd3084c65012f30239c900161f8e63f1fcbf8d2c57ef559f45468c56d: Status 404 returned error can't find the container with id 3584c5ccd3084c65012f30239c900161f8e63f1fcbf8d2c57ef559f45468c56d
Sep 09 03:22:16.247432 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:22:16.247115    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-rtcp7" event={"ID":"8eb57cdf-33f8-483f-a943-d16317fad3bd","Type":"ContainerStarted","Data":"3584c5ccd3084c65012f30239c900161f8e63f1fcbf8d2c57ef559f45468c56d"}
Sep 09 03:22:17.939291 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:22:17.939253    2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892148Ki","pods":"250"}
Sep 09 03:22:17.939565 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:22:17.939331    2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892148Ki","pods":"250"}
Sep 09 03:22:18.257303 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:22:18.257266    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-rtcp7" event={"ID":"8eb57cdf-33f8-483f-a943-d16317fad3bd","Type":"ContainerStarted","Data":"3101ad00cd3cfd59fd3b55f3dcc6333e9c947d723ef07f848a3add8030b494e3"}
Sep 09 03:22:18.257504 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:22:18.257486    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-rtcp7"
Sep 09 03:22:18.259172 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:22:18.259145    2576 patch_prober.go:28] interesting pod/istiod-openshift-gateway-5677ccb49b-rtcp7 container/discovery namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Sep 09 03:22:18.259285 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:22:18.259198    2576 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-rtcp7" podUID="8eb57cdf-33f8-483f-a943-d16317fad3bd" containerName="discovery" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Sep 09 03:22:18.311572 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:22:18.311512    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-rtcp7" podStartSLOduration=1.144408606 podStartE2EDuration="3.311493635s" totalImagesPullingTime="2.167085029s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 03:22:15 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 03:22:18.30944489 +0000 UTC m=+552.450930966" watchObservedRunningTime="2026-09-09 03:22:18.311493635 +0000 UTC m=+552.452979713"
Sep 09 03:22:19.262123 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:22:19.262097    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-rtcp7"
Sep 09 03:24:32.999071 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:24:32.999034    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-limitador-8588749977-lsqkx"]
Sep 09 03:24:33.002354 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:24:33.002325    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-8588749977-lsqkx"
Sep 09 03:24:33.011727 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:24:33.011701    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\""
Sep 09 03:24:33.012276 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:24:33.012260    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\""
Sep 09 03:24:33.015045 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:24:33.015027    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"limitador-limits-config-limitador\""
Sep 09 03:24:33.018577 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:24:33.018559    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-9kbrl\""
Sep 09 03:24:33.113088 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:24:33.113052    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/26aab628-36a2-4e57-933f-6309f10e1e4e-config-file\") pod \"limitador-limitador-8588749977-lsqkx\" (UID: \"26aab628-36a2-4e57-933f-6309f10e1e4e\") " pod="kuadrant-system/limitador-limitador-8588749977-lsqkx"
Sep 09 03:24:33.113265 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:24:33.113112    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vtcgx\" (UniqueName: \"kubernetes.io/projected/26aab628-36a2-4e57-933f-6309f10e1e4e-kube-api-access-vtcgx\") pod \"limitador-limitador-8588749977-lsqkx\" (UID: \"26aab628-36a2-4e57-933f-6309f10e1e4e\") " pod="kuadrant-system/limitador-limitador-8588749977-lsqkx"
Sep 09 03:24:33.131922 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:24:33.131889    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-8588749977-lsqkx"]
Sep 09 03:24:33.174027 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:24:33.173997    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-8588749977-lsqkx"]
Sep 09 03:24:33.213881 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:24:33.213845    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/26aab628-36a2-4e57-933f-6309f10e1e4e-config-file\") pod \"limitador-limitador-8588749977-lsqkx\" (UID: \"26aab628-36a2-4e57-933f-6309f10e1e4e\") " pod="kuadrant-system/limitador-limitador-8588749977-lsqkx"
Sep 09 03:24:33.214095 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:24:33.213903    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vtcgx\" (UniqueName: \"kubernetes.io/projected/26aab628-36a2-4e57-933f-6309f10e1e4e-kube-api-access-vtcgx\") pod \"limitador-limitador-8588749977-lsqkx\" (UID: \"26aab628-36a2-4e57-933f-6309f10e1e4e\") " pod="kuadrant-system/limitador-limitador-8588749977-lsqkx"
Sep 09 03:24:33.214533 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:24:33.214507    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/26aab628-36a2-4e57-933f-6309f10e1e4e-config-file\") pod \"limitador-limitador-8588749977-lsqkx\" (UID: \"26aab628-36a2-4e57-933f-6309f10e1e4e\") " pod="kuadrant-system/limitador-limitador-8588749977-lsqkx"
Sep 09 03:24:33.225842 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:24:33.225797    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vtcgx\" (UniqueName: \"kubernetes.io/projected/26aab628-36a2-4e57-933f-6309f10e1e4e-kube-api-access-vtcgx\") pod \"limitador-limitador-8588749977-lsqkx\" (UID: \"26aab628-36a2-4e57-933f-6309f10e1e4e\") " pod="kuadrant-system/limitador-limitador-8588749977-lsqkx"
Sep 09 03:24:33.313187 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:24:33.313152    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-8588749977-lsqkx"
Sep 09 03:24:33.625676 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:24:33.625589    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-8588749977-lsqkx"]
Sep 09 03:24:33.631267 ip-10-0-132-14 kubenswrapper[2576]: W0909 03:24:33.631240    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod26aab628_36a2_4e57_933f_6309f10e1e4e.slice/crio-f2272ceef6d9adfab1209cb1853cc0a8cb720d7c4e87d90891fa62ef2e59b305 WatchSource:0}: Error finding container f2272ceef6d9adfab1209cb1853cc0a8cb720d7c4e87d90891fa62ef2e59b305: Status 404 returned error can't find the container with id f2272ceef6d9adfab1209cb1853cc0a8cb720d7c4e87d90891fa62ef2e59b305
Sep 09 03:24:33.633379 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:24:33.633362    2576 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 09 03:24:33.798323 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:24:33.798287    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-8588749977-lsqkx" event={"ID":"26aab628-36a2-4e57-933f-6309f10e1e4e","Type":"ContainerStarted","Data":"f2272ceef6d9adfab1209cb1853cc0a8cb720d7c4e87d90891fa62ef2e59b305"}
Sep 09 03:24:45.848545 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:24:45.848504    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-8588749977-lsqkx" event={"ID":"26aab628-36a2-4e57-933f-6309f10e1e4e","Type":"ContainerStarted","Data":"da6e7273290e1f510394a69cce6abf25b067e5788936ba7620d43003fa909ef1"}
Sep 09 03:24:45.848955 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:24:45.848711    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-limitador-8588749977-lsqkx"
Sep 09 03:24:45.890346 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:24:45.890289    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-limitador-8588749977-lsqkx" podStartSLOduration=2.160980738 podStartE2EDuration="13.890272228s" totalImagesPullingTime="11.72929149s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 03:24:32 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 03:24:45.88935122 +0000 UTC m=+700.030837299" watchObservedRunningTime="2026-09-09 03:24:45.890272228 +0000 UTC m=+700.031758301"
Sep 09 03:24:48.380115 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:24:48.380083    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-limitador-8588749977-lsqkx"]
Sep 09 03:24:48.380503 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:24:48.380252    2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/limitador-limitador-8588749977-lsqkx" podUID="26aab628-36a2-4e57-933f-6309f10e1e4e" containerName="limitador" containerID="cri-o://da6e7273290e1f510394a69cce6abf25b067e5788936ba7620d43003fa909ef1" gracePeriod=30
Sep 09 03:24:48.861412 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:24:48.861377    2576 generic.go:356] "Generic (PLEG): container finished" podID="26aab628-36a2-4e57-933f-6309f10e1e4e" containerID="da6e7273290e1f510394a69cce6abf25b067e5788936ba7620d43003fa909ef1" exitCode=0
Sep 09 03:24:48.861626 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:24:48.861445    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-8588749977-lsqkx" event={"ID":"26aab628-36a2-4e57-933f-6309f10e1e4e","Type":"ContainerDied","Data":"da6e7273290e1f510394a69cce6abf25b067e5788936ba7620d43003fa909ef1"}
Sep 09 03:24:49.316157 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:24:49.316132    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-8588749977-lsqkx"
Sep 09 03:24:49.453139 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:24:49.453039    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-vtcgx\" (UniqueName: \"kubernetes.io/projected/26aab628-36a2-4e57-933f-6309f10e1e4e-kube-api-access-vtcgx\") pod \"26aab628-36a2-4e57-933f-6309f10e1e4e\" (UID: \"26aab628-36a2-4e57-933f-6309f10e1e4e\") "
Sep 09 03:24:49.453139 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:24:49.453106    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/26aab628-36a2-4e57-933f-6309f10e1e4e-config-file\") pod \"26aab628-36a2-4e57-933f-6309f10e1e4e\" (UID: \"26aab628-36a2-4e57-933f-6309f10e1e4e\") "
Sep 09 03:24:49.453635 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:24:49.453471    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/26aab628-36a2-4e57-933f-6309f10e1e4e-config-file" (OuterVolumeSpecName: "config-file") pod "26aab628-36a2-4e57-933f-6309f10e1e4e" (UID: "26aab628-36a2-4e57-933f-6309f10e1e4e"). InnerVolumeSpecName "config-file". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 09 03:24:49.455426 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:24:49.455406    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/26aab628-36a2-4e57-933f-6309f10e1e4e-kube-api-access-vtcgx" (OuterVolumeSpecName: "kube-api-access-vtcgx") pod "26aab628-36a2-4e57-933f-6309f10e1e4e" (UID: "26aab628-36a2-4e57-933f-6309f10e1e4e"). InnerVolumeSpecName "kube-api-access-vtcgx". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 09 03:24:49.554087 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:24:49.554049    2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vtcgx\" (UniqueName: \"kubernetes.io/projected/26aab628-36a2-4e57-933f-6309f10e1e4e-kube-api-access-vtcgx\") on node \"ip-10-0-132-14.ec2.internal\" DevicePath \"\""
Sep 09 03:24:49.554087 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:24:49.554081    2576 reconciler_common.go:299] "Volume detached for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/26aab628-36a2-4e57-933f-6309f10e1e4e-config-file\") on node \"ip-10-0-132-14.ec2.internal\" DevicePath \"\""
Sep 09 03:24:49.866902 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:24:49.866871    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-8588749977-lsqkx"
Sep 09 03:24:49.867075 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:24:49.866871    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-8588749977-lsqkx" event={"ID":"26aab628-36a2-4e57-933f-6309f10e1e4e","Type":"ContainerDied","Data":"f2272ceef6d9adfab1209cb1853cc0a8cb720d7c4e87d90891fa62ef2e59b305"}
Sep 09 03:24:49.867075 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:24:49.867010    2576 scope.go:117] "RemoveContainer" containerID="da6e7273290e1f510394a69cce6abf25b067e5788936ba7620d43003fa909ef1"
Sep 09 03:24:49.927987 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:24:49.927949    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-limitador-8588749977-lsqkx"]
Sep 09 03:24:49.937121 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:24:49.937097    2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/limitador-limitador-8588749977-lsqkx"]
Sep 09 03:24:50.359580 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:24:50.359543    2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="26aab628-36a2-4e57-933f-6309f10e1e4e" path="/var/lib/kubelet/pods/26aab628-36a2-4e57-933f-6309f10e1e4e/volumes"
Sep 09 03:24:54.762114 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:24:54.762078    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/postgres-69fb4594cf-5qpj2"]
Sep 09 03:24:54.762504 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:24:54.762486    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="26aab628-36a2-4e57-933f-6309f10e1e4e" containerName="limitador"
Sep 09 03:24:54.762546 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:24:54.762509    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="26aab628-36a2-4e57-933f-6309f10e1e4e" containerName="limitador"
Sep 09 03:24:54.762593 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:24:54.762581    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="26aab628-36a2-4e57-933f-6309f10e1e4e" containerName="limitador"
Sep 09 03:24:54.767258 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:24:54.767237    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/postgres-69fb4594cf-5qpj2"
Sep 09 03:24:54.771105 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:24:54.771083    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\""
Sep 09 03:24:54.772047 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:24:54.772030    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 09 03:24:54.772134 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:24:54.772056    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"postgres-creds\""
Sep 09 03:24:54.772134 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:24:54.772110    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"default-dockercfg-6kxgw\""
Sep 09 03:24:54.784631 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:24:54.784611    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/postgres-69fb4594cf-5qpj2"]
Sep 09 03:24:54.900647 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:24:54.900609    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zqlzt\" (UniqueName: \"kubernetes.io/projected/ff17b5e7-f11d-4612-a3ba-1b55922c8a4d-kube-api-access-zqlzt\") pod \"postgres-69fb4594cf-5qpj2\" (UID: \"ff17b5e7-f11d-4612-a3ba-1b55922c8a4d\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-5qpj2"
Sep 09 03:24:54.900826 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:24:54.900671    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/ff17b5e7-f11d-4612-a3ba-1b55922c8a4d-data\") pod \"postgres-69fb4594cf-5qpj2\" (UID: \"ff17b5e7-f11d-4612-a3ba-1b55922c8a4d\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-5qpj2"
Sep 09 03:24:55.001864 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:24:55.001789    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/ff17b5e7-f11d-4612-a3ba-1b55922c8a4d-data\") pod \"postgres-69fb4594cf-5qpj2\" (UID: \"ff17b5e7-f11d-4612-a3ba-1b55922c8a4d\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-5qpj2"
Sep 09 03:24:55.002059 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:24:55.001896    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zqlzt\" (UniqueName: \"kubernetes.io/projected/ff17b5e7-f11d-4612-a3ba-1b55922c8a4d-kube-api-access-zqlzt\") pod \"postgres-69fb4594cf-5qpj2\" (UID: \"ff17b5e7-f11d-4612-a3ba-1b55922c8a4d\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-5qpj2"
Sep 09 03:24:55.002244 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:24:55.002221    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/ff17b5e7-f11d-4612-a3ba-1b55922c8a4d-data\") pod \"postgres-69fb4594cf-5qpj2\" (UID: \"ff17b5e7-f11d-4612-a3ba-1b55922c8a4d\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-5qpj2"
Sep 09 03:24:55.014588 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:24:55.014533    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zqlzt\" (UniqueName: \"kubernetes.io/projected/ff17b5e7-f11d-4612-a3ba-1b55922c8a4d-kube-api-access-zqlzt\") pod \"postgres-69fb4594cf-5qpj2\" (UID: \"ff17b5e7-f11d-4612-a3ba-1b55922c8a4d\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-5qpj2"
Sep 09 03:24:55.077111 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:24:55.077081    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/postgres-69fb4594cf-5qpj2"
Sep 09 03:24:55.391697 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:24:55.391614    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/postgres-69fb4594cf-5qpj2"]
Sep 09 03:24:55.395274 ip-10-0-132-14 kubenswrapper[2576]: W0909 03:24:55.395243    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podff17b5e7_f11d_4612_a3ba_1b55922c8a4d.slice/crio-b3ef57f67351656f3d52b47ef6d05fe1a42e8b728af29913f05f76691333cabc WatchSource:0}: Error finding container b3ef57f67351656f3d52b47ef6d05fe1a42e8b728af29913f05f76691333cabc: Status 404 returned error can't find the container with id b3ef57f67351656f3d52b47ef6d05fe1a42e8b728af29913f05f76691333cabc
Sep 09 03:24:55.898174 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:24:55.898131    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/postgres-69fb4594cf-5qpj2" event={"ID":"ff17b5e7-f11d-4612-a3ba-1b55922c8a4d","Type":"ContainerStarted","Data":"b3ef57f67351656f3d52b47ef6d05fe1a42e8b728af29913f05f76691333cabc"}
Sep 09 03:25:00.925124 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:25:00.925086    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/postgres-69fb4594cf-5qpj2" event={"ID":"ff17b5e7-f11d-4612-a3ba-1b55922c8a4d","Type":"ContainerStarted","Data":"aa43ea098a5e42667367b3830c9f3523d0a11a12a557edfee42c6e5e53dd95d7"}
Sep 09 03:25:00.925503 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:25:00.925203    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/postgres-69fb4594cf-5qpj2"
Sep 09 03:25:00.955754 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:25:00.955701    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/postgres-69fb4594cf-5qpj2" podStartSLOduration=2.009163956 podStartE2EDuration="6.955683413s" totalImagesPullingTime="4.946519457s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 03:24:54 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 03:25:00.954703631 +0000 UTC m=+715.096189702" watchObservedRunningTime="2026-09-09 03:25:00.955683413 +0000 UTC m=+715.097169490"
Sep 09 03:25:06.958683 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:25:06.958654    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/postgres-69fb4594cf-5qpj2"
Sep 09 03:25:15.214431 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:25:15.214399    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-6fcf75c44b-5l46n"]
Sep 09 03:25:15.240606 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:25:15.240573    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-6fcf75c44b-5l46n"
Sep 09 03:25:15.241743 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:25:15.241694    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-6fcf75c44b-5l46n"]
Sep 09 03:25:15.247984 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:25:15.247600    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\""
Sep 09 03:25:15.248383 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:25:15.248364    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-nrndt\""
Sep 09 03:25:15.249845 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:25:15.249825    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\""
Sep 09 03:25:15.249926 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:25:15.249893    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-metrics-tls\""
Sep 09 03:25:15.262348 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:25:15.262317    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/f0cb5e5f-181b-457b-9878-8be206048751-webhook-cert\") pod \"maas-controller-6fcf75c44b-5l46n\" (UID: \"f0cb5e5f-181b-457b-9878-8be206048751\") " pod="opendatahub/maas-controller-6fcf75c44b-5l46n"
Sep 09 03:25:15.262475 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:25:15.262362    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f0cb5e5f-181b-457b-9878-8be206048751-metrics-certs\") pod \"maas-controller-6fcf75c44b-5l46n\" (UID: \"f0cb5e5f-181b-457b-9878-8be206048751\") " pod="opendatahub/maas-controller-6fcf75c44b-5l46n"
Sep 09 03:25:15.262475 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:25:15.262419    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nktgw\" (UniqueName: \"kubernetes.io/projected/f0cb5e5f-181b-457b-9878-8be206048751-kube-api-access-nktgw\") pod \"maas-controller-6fcf75c44b-5l46n\" (UID: \"f0cb5e5f-181b-457b-9878-8be206048751\") " pod="opendatahub/maas-controller-6fcf75c44b-5l46n"
Sep 09 03:25:15.362960 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:25:15.362931    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f0cb5e5f-181b-457b-9878-8be206048751-metrics-certs\") pod \"maas-controller-6fcf75c44b-5l46n\" (UID: \"f0cb5e5f-181b-457b-9878-8be206048751\") " pod="opendatahub/maas-controller-6fcf75c44b-5l46n"
Sep 09 03:25:15.363141 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:25:15.362999    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nktgw\" (UniqueName: \"kubernetes.io/projected/f0cb5e5f-181b-457b-9878-8be206048751-kube-api-access-nktgw\") pod \"maas-controller-6fcf75c44b-5l46n\" (UID: \"f0cb5e5f-181b-457b-9878-8be206048751\") " pod="opendatahub/maas-controller-6fcf75c44b-5l46n"
Sep 09 03:25:15.363212 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:25:15.363164    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/f0cb5e5f-181b-457b-9878-8be206048751-webhook-cert\") pod \"maas-controller-6fcf75c44b-5l46n\" (UID: \"f0cb5e5f-181b-457b-9878-8be206048751\") " pod="opendatahub/maas-controller-6fcf75c44b-5l46n"
Sep 09 03:25:15.365698 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:25:15.365673    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f0cb5e5f-181b-457b-9878-8be206048751-metrics-certs\") pod \"maas-controller-6fcf75c44b-5l46n\" (UID: \"f0cb5e5f-181b-457b-9878-8be206048751\") " pod="opendatahub/maas-controller-6fcf75c44b-5l46n"
Sep 09 03:25:15.365698 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:25:15.365685    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/f0cb5e5f-181b-457b-9878-8be206048751-webhook-cert\") pod \"maas-controller-6fcf75c44b-5l46n\" (UID: \"f0cb5e5f-181b-457b-9878-8be206048751\") " pod="opendatahub/maas-controller-6fcf75c44b-5l46n"
Sep 09 03:25:15.377596 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:25:15.377571    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nktgw\" (UniqueName: \"kubernetes.io/projected/f0cb5e5f-181b-457b-9878-8be206048751-kube-api-access-nktgw\") pod \"maas-controller-6fcf75c44b-5l46n\" (UID: \"f0cb5e5f-181b-457b-9878-8be206048751\") " pod="opendatahub/maas-controller-6fcf75c44b-5l46n"
Sep 09 03:25:15.426916 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:25:15.426878    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-69d6c4f5bf-khxz4"]
Sep 09 03:25:15.430268 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:25:15.430250    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-69d6c4f5bf-khxz4"
Sep 09 03:25:15.455705 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:25:15.455670    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-69d6c4f5bf-khxz4"]
Sep 09 03:25:15.464537 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:25:15.464472    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/5ff59eb2-7e81-4885-a896-d74f3eabb514-webhook-cert\") pod \"maas-controller-69d6c4f5bf-khxz4\" (UID: \"5ff59eb2-7e81-4885-a896-d74f3eabb514\") " pod="opendatahub/maas-controller-69d6c4f5bf-khxz4"
Sep 09 03:25:15.464537 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:25:15.464515    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rdhsr\" (UniqueName: \"kubernetes.io/projected/5ff59eb2-7e81-4885-a896-d74f3eabb514-kube-api-access-rdhsr\") pod \"maas-controller-69d6c4f5bf-khxz4\" (UID: \"5ff59eb2-7e81-4885-a896-d74f3eabb514\") " pod="opendatahub/maas-controller-69d6c4f5bf-khxz4"
Sep 09 03:25:15.464701 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:25:15.464576    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/5ff59eb2-7e81-4885-a896-d74f3eabb514-metrics-certs\") pod \"maas-controller-69d6c4f5bf-khxz4\" (UID: \"5ff59eb2-7e81-4885-a896-d74f3eabb514\") " pod="opendatahub/maas-controller-69d6c4f5bf-khxz4"
Sep 09 03:25:15.551755 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:25:15.551720    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-6fcf75c44b-5l46n"
Sep 09 03:25:15.565171 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:25:15.565139    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/5ff59eb2-7e81-4885-a896-d74f3eabb514-webhook-cert\") pod \"maas-controller-69d6c4f5bf-khxz4\" (UID: \"5ff59eb2-7e81-4885-a896-d74f3eabb514\") " pod="opendatahub/maas-controller-69d6c4f5bf-khxz4"
Sep 09 03:25:15.565297 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:25:15.565219    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rdhsr\" (UniqueName: \"kubernetes.io/projected/5ff59eb2-7e81-4885-a896-d74f3eabb514-kube-api-access-rdhsr\") pod \"maas-controller-69d6c4f5bf-khxz4\" (UID: \"5ff59eb2-7e81-4885-a896-d74f3eabb514\") " pod="opendatahub/maas-controller-69d6c4f5bf-khxz4"
Sep 09 03:25:15.565297 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:25:15.565288    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/5ff59eb2-7e81-4885-a896-d74f3eabb514-metrics-certs\") pod \"maas-controller-69d6c4f5bf-khxz4\" (UID: \"5ff59eb2-7e81-4885-a896-d74f3eabb514\") " pod="opendatahub/maas-controller-69d6c4f5bf-khxz4"
Sep 09 03:25:15.567921 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:25:15.567896    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/5ff59eb2-7e81-4885-a896-d74f3eabb514-metrics-certs\") pod \"maas-controller-69d6c4f5bf-khxz4\" (UID: \"5ff59eb2-7e81-4885-a896-d74f3eabb514\") " pod="opendatahub/maas-controller-69d6c4f5bf-khxz4"
Sep 09 03:25:15.568028 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:25:15.567898    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/5ff59eb2-7e81-4885-a896-d74f3eabb514-webhook-cert\") pod \"maas-controller-69d6c4f5bf-khxz4\" (UID: \"5ff59eb2-7e81-4885-a896-d74f3eabb514\") " pod="opendatahub/maas-controller-69d6c4f5bf-khxz4"
Sep 09 03:25:15.576087 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:25:15.576061    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rdhsr\" (UniqueName: \"kubernetes.io/projected/5ff59eb2-7e81-4885-a896-d74f3eabb514-kube-api-access-rdhsr\") pod \"maas-controller-69d6c4f5bf-khxz4\" (UID: \"5ff59eb2-7e81-4885-a896-d74f3eabb514\") " pod="opendatahub/maas-controller-69d6c4f5bf-khxz4"
Sep 09 03:25:15.740843 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:25:15.740731    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-69d6c4f5bf-khxz4"
Sep 09 03:25:15.835411 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:25:15.835381    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-6fcf75c44b-5l46n"]
Sep 09 03:25:15.840929 ip-10-0-132-14 kubenswrapper[2576]: W0909 03:25:15.840904    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf0cb5e5f_181b_457b_9878_8be206048751.slice/crio-dea1dda6cd41d6a11bd45cff410751fe9926a97c1380038ae2588229a216831a WatchSource:0}: Error finding container dea1dda6cd41d6a11bd45cff410751fe9926a97c1380038ae2588229a216831a: Status 404 returned error can't find the container with id dea1dda6cd41d6a11bd45cff410751fe9926a97c1380038ae2588229a216831a
Sep 09 03:25:15.985230 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:25:15.985199    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-6fcf75c44b-5l46n" event={"ID":"f0cb5e5f-181b-457b-9878-8be206048751","Type":"ContainerStarted","Data":"dea1dda6cd41d6a11bd45cff410751fe9926a97c1380038ae2588229a216831a"}
Sep 09 03:25:16.024547 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:25:16.024418    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-69d6c4f5bf-khxz4"]
Sep 09 03:25:16.027560 ip-10-0-132-14 kubenswrapper[2576]: W0909 03:25:16.027529    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5ff59eb2_7e81_4885_a896_d74f3eabb514.slice/crio-807cf953768acae0d8fb678a6b1274b958f52cf8b3a9111cc3ab21ab264f774f WatchSource:0}: Error finding container 807cf953768acae0d8fb678a6b1274b958f52cf8b3a9111cc3ab21ab264f774f: Status 404 returned error can't find the container with id 807cf953768acae0d8fb678a6b1274b958f52cf8b3a9111cc3ab21ab264f774f
Sep 09 03:25:16.990586 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:25:16.990552    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-69d6c4f5bf-khxz4" event={"ID":"5ff59eb2-7e81-4885-a896-d74f3eabb514","Type":"ContainerStarted","Data":"807cf953768acae0d8fb678a6b1274b958f52cf8b3a9111cc3ab21ab264f774f"}
Sep 09 03:25:17.996557 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:25:17.996447    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-6fcf75c44b-5l46n" event={"ID":"f0cb5e5f-181b-457b-9878-8be206048751","Type":"ContainerStarted","Data":"08b26d369a3175b86142783d8865e10e6510960917e891e4add22206420a1c43"}
Sep 09 03:25:17.996557 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:25:17.996524    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-6fcf75c44b-5l46n"
Sep 09 03:25:17.998039 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:25:17.998012    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-69d6c4f5bf-khxz4" event={"ID":"5ff59eb2-7e81-4885-a896-d74f3eabb514","Type":"ContainerStarted","Data":"690e697c6d0658496f4ae08b572f756bfc33a596e37b77c9f482e440d63e658c"}
Sep 09 03:25:17.998173 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:25:17.998071    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-69d6c4f5bf-khxz4"
Sep 09 03:25:18.023425 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:25:18.023374    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-6fcf75c44b-5l46n" podStartSLOduration=1.262658044 podStartE2EDuration="3.023356069s" totalImagesPullingTime="1.760698025s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 03:25:15 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 03:25:18.021940801 +0000 UTC m=+732.163426868" watchObservedRunningTime="2026-09-09 03:25:18.023356069 +0000 UTC m=+732.164842145"
Sep 09 03:25:18.061863 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:25:18.061715    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-69d6c4f5bf-khxz4" podStartSLOduration=1.491322812 podStartE2EDuration="3.061692894s" totalImagesPullingTime="1.570370082s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 03:25:15 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 03:25:18.061324554 +0000 UTC m=+732.202810625" watchObservedRunningTime="2026-09-09 03:25:18.061692894 +0000 UTC m=+732.203178972"
Sep 09 03:25:29.003945 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:25:29.003910    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-6fcf75c44b-5l46n"
Sep 09 03:25:29.004323 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:25:29.003997    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-69d6c4f5bf-khxz4"
Sep 09 03:25:29.139029 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:25:29.138990    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-6fcf75c44b-5l46n"]
Sep 09 03:25:29.139205 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:25:29.139182    2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/maas-controller-6fcf75c44b-5l46n" podUID="f0cb5e5f-181b-457b-9878-8be206048751" containerName="manager" containerID="cri-o://08b26d369a3175b86142783d8865e10e6510960917e891e4add22206420a1c43" gracePeriod=30
Sep 09 03:25:29.384971 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:25:29.384948    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-6fcf75c44b-5l46n"
Sep 09 03:25:29.477106 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:25:29.477076    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-nktgw\" (UniqueName: \"kubernetes.io/projected/f0cb5e5f-181b-457b-9878-8be206048751-kube-api-access-nktgw\") pod \"f0cb5e5f-181b-457b-9878-8be206048751\" (UID: \"f0cb5e5f-181b-457b-9878-8be206048751\") "
Sep 09 03:25:29.477272 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:25:29.477127    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f0cb5e5f-181b-457b-9878-8be206048751-metrics-certs\") pod \"f0cb5e5f-181b-457b-9878-8be206048751\" (UID: \"f0cb5e5f-181b-457b-9878-8be206048751\") "
Sep 09 03:25:29.477272 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:25:29.477187    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/f0cb5e5f-181b-457b-9878-8be206048751-webhook-cert\") pod \"f0cb5e5f-181b-457b-9878-8be206048751\" (UID: \"f0cb5e5f-181b-457b-9878-8be206048751\") "
Sep 09 03:25:29.479506 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:25:29.479473    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f0cb5e5f-181b-457b-9878-8be206048751-kube-api-access-nktgw" (OuterVolumeSpecName: "kube-api-access-nktgw") pod "f0cb5e5f-181b-457b-9878-8be206048751" (UID: "f0cb5e5f-181b-457b-9878-8be206048751"). InnerVolumeSpecName "kube-api-access-nktgw". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 09 03:25:29.479506 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:25:29.479490    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f0cb5e5f-181b-457b-9878-8be206048751-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "f0cb5e5f-181b-457b-9878-8be206048751" (UID: "f0cb5e5f-181b-457b-9878-8be206048751"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 09 03:25:29.479657 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:25:29.479512    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f0cb5e5f-181b-457b-9878-8be206048751-webhook-cert" (OuterVolumeSpecName: "webhook-cert") pod "f0cb5e5f-181b-457b-9878-8be206048751" (UID: "f0cb5e5f-181b-457b-9878-8be206048751"). InnerVolumeSpecName "webhook-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 09 03:25:29.578256 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:25:29.578159    2576 reconciler_common.go:299] "Volume detached for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/f0cb5e5f-181b-457b-9878-8be206048751-webhook-cert\") on node \"ip-10-0-132-14.ec2.internal\" DevicePath \"\""
Sep 09 03:25:29.578256 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:25:29.578195    2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-nktgw\" (UniqueName: \"kubernetes.io/projected/f0cb5e5f-181b-457b-9878-8be206048751-kube-api-access-nktgw\") on node \"ip-10-0-132-14.ec2.internal\" DevicePath \"\""
Sep 09 03:25:29.578256 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:25:29.578208    2576 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f0cb5e5f-181b-457b-9878-8be206048751-metrics-certs\") on node \"ip-10-0-132-14.ec2.internal\" DevicePath \"\""
Sep 09 03:25:30.045327 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:25:30.045290    2576 generic.go:356] "Generic (PLEG): container finished" podID="f0cb5e5f-181b-457b-9878-8be206048751" containerID="08b26d369a3175b86142783d8865e10e6510960917e891e4add22206420a1c43" exitCode=0
Sep 09 03:25:30.045715 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:25:30.045338    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-6fcf75c44b-5l46n"
Sep 09 03:25:30.045715 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:25:30.045347    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-6fcf75c44b-5l46n" event={"ID":"f0cb5e5f-181b-457b-9878-8be206048751","Type":"ContainerDied","Data":"08b26d369a3175b86142783d8865e10e6510960917e891e4add22206420a1c43"}
Sep 09 03:25:30.045715 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:25:30.045390    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-6fcf75c44b-5l46n" event={"ID":"f0cb5e5f-181b-457b-9878-8be206048751","Type":"ContainerDied","Data":"dea1dda6cd41d6a11bd45cff410751fe9926a97c1380038ae2588229a216831a"}
Sep 09 03:25:30.045715 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:25:30.045412    2576 scope.go:117] "RemoveContainer" containerID="08b26d369a3175b86142783d8865e10e6510960917e891e4add22206420a1c43"
Sep 09 03:25:30.054592 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:25:30.054575    2576 scope.go:117] "RemoveContainer" containerID="08b26d369a3175b86142783d8865e10e6510960917e891e4add22206420a1c43"
Sep 09 03:25:30.054898 ip-10-0-132-14 kubenswrapper[2576]: E0909 03:25:30.054876    2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"08b26d369a3175b86142783d8865e10e6510960917e891e4add22206420a1c43\": container with ID starting with 08b26d369a3175b86142783d8865e10e6510960917e891e4add22206420a1c43 not found: ID does not exist" containerID="08b26d369a3175b86142783d8865e10e6510960917e891e4add22206420a1c43"
Sep 09 03:25:30.054985 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:25:30.054907    2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"08b26d369a3175b86142783d8865e10e6510960917e891e4add22206420a1c43"} err="failed to get container status \"08b26d369a3175b86142783d8865e10e6510960917e891e4add22206420a1c43\": rpc error: code = NotFound desc = could not find container \"08b26d369a3175b86142783d8865e10e6510960917e891e4add22206420a1c43\": container with ID starting with 08b26d369a3175b86142783d8865e10e6510960917e891e4add22206420a1c43 not found: ID does not exist"
Sep 09 03:25:30.074953 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:25:30.074925    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-6fcf75c44b-5l46n"]
Sep 09 03:25:30.080631 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:25:30.080610    2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-6fcf75c44b-5l46n"]
Sep 09 03:25:30.359631 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:25:30.359551    2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f0cb5e5f-181b-457b-9878-8be206048751" path="/var/lib/kubelet/pods/f0cb5e5f-181b-457b-9878-8be206048751/volumes"
Sep 09 03:25:51.563630 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:25:51.563596    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-5b54b7fbb7-x6t9q"]
Sep 09 03:25:51.564145 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:25:51.563880    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f0cb5e5f-181b-457b-9878-8be206048751" containerName="manager"
Sep 09 03:25:51.564145 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:25:51.563891    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="f0cb5e5f-181b-457b-9878-8be206048751" containerName="manager"
Sep 09 03:25:51.564145 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:25:51.563949    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="f0cb5e5f-181b-457b-9878-8be206048751" containerName="manager"
Sep 09 03:25:51.569394 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:25:51.569374    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-5b54b7fbb7-x6t9q"
Sep 09 03:25:51.575152 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:25:51.575123    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins\""
Sep 09 03:25:51.575285 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:25:51.575239    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-dockercfg-btgsq\""
Sep 09 03:25:51.599620 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:25:51.599595    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-5b54b7fbb7-x6t9q"]
Sep 09 03:25:51.663531 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:25:51.663496    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-59fbc47bf8-8j4v2"]
Sep 09 03:25:51.666496 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:25:51.666476    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-p4dj9\" (UniqueName: \"kubernetes.io/projected/701775d1-64f8-4057-b7ee-5aab7fb98ac1-kube-api-access-p4dj9\") pod \"payload-processing-5b54b7fbb7-x6t9q\" (UID: \"701775d1-64f8-4057-b7ee-5aab7fb98ac1\") " pod="openshift-ingress/payload-processing-5b54b7fbb7-x6t9q"
Sep 09 03:25:51.666551 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:25:51.666526    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/701775d1-64f8-4057-b7ee-5aab7fb98ac1-plugins-config-volume\") pod \"payload-processing-5b54b7fbb7-x6t9q\" (UID: \"701775d1-64f8-4057-b7ee-5aab7fb98ac1\") " pod="openshift-ingress/payload-processing-5b54b7fbb7-x6t9q"
Sep 09 03:25:51.667392 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:25:51.667378    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-59fbc47bf8-8j4v2"
Sep 09 03:25:51.705519 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:25:51.705488    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-59fbc47bf8-8j4v2"]
Sep 09 03:25:51.767111 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:25:51.767070    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/0215d837-84cb-49f2-8652-2c182506b0e9-plugins-config-volume\") pod \"payload-pre-processing-59fbc47bf8-8j4v2\" (UID: \"0215d837-84cb-49f2-8652-2c182506b0e9\") " pod="openshift-ingress/payload-pre-processing-59fbc47bf8-8j4v2"
Sep 09 03:25:51.767282 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:25:51.767164    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-p4dj9\" (UniqueName: \"kubernetes.io/projected/701775d1-64f8-4057-b7ee-5aab7fb98ac1-kube-api-access-p4dj9\") pod \"payload-processing-5b54b7fbb7-x6t9q\" (UID: \"701775d1-64f8-4057-b7ee-5aab7fb98ac1\") " pod="openshift-ingress/payload-processing-5b54b7fbb7-x6t9q"
Sep 09 03:25:51.767282 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:25:51.767196    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-57q6f\" (UniqueName: \"kubernetes.io/projected/0215d837-84cb-49f2-8652-2c182506b0e9-kube-api-access-57q6f\") pod \"payload-pre-processing-59fbc47bf8-8j4v2\" (UID: \"0215d837-84cb-49f2-8652-2c182506b0e9\") " pod="openshift-ingress/payload-pre-processing-59fbc47bf8-8j4v2"
Sep 09 03:25:51.767282 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:25:51.767253    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/701775d1-64f8-4057-b7ee-5aab7fb98ac1-plugins-config-volume\") pod \"payload-processing-5b54b7fbb7-x6t9q\" (UID: \"701775d1-64f8-4057-b7ee-5aab7fb98ac1\") " pod="openshift-ingress/payload-processing-5b54b7fbb7-x6t9q"
Sep 09 03:25:51.767967 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:25:51.767947    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/701775d1-64f8-4057-b7ee-5aab7fb98ac1-plugins-config-volume\") pod \"payload-processing-5b54b7fbb7-x6t9q\" (UID: \"701775d1-64f8-4057-b7ee-5aab7fb98ac1\") " pod="openshift-ingress/payload-processing-5b54b7fbb7-x6t9q"
Sep 09 03:25:51.819335 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:25:51.819258    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-p4dj9\" (UniqueName: \"kubernetes.io/projected/701775d1-64f8-4057-b7ee-5aab7fb98ac1-kube-api-access-p4dj9\") pod \"payload-processing-5b54b7fbb7-x6t9q\" (UID: \"701775d1-64f8-4057-b7ee-5aab7fb98ac1\") " pod="openshift-ingress/payload-processing-5b54b7fbb7-x6t9q"
Sep 09 03:25:51.868453 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:25:51.868415    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/0215d837-84cb-49f2-8652-2c182506b0e9-plugins-config-volume\") pod \"payload-pre-processing-59fbc47bf8-8j4v2\" (UID: \"0215d837-84cb-49f2-8652-2c182506b0e9\") " pod="openshift-ingress/payload-pre-processing-59fbc47bf8-8j4v2"
Sep 09 03:25:51.868623 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:25:51.868485    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-57q6f\" (UniqueName: \"kubernetes.io/projected/0215d837-84cb-49f2-8652-2c182506b0e9-kube-api-access-57q6f\") pod \"payload-pre-processing-59fbc47bf8-8j4v2\" (UID: \"0215d837-84cb-49f2-8652-2c182506b0e9\") " pod="openshift-ingress/payload-pre-processing-59fbc47bf8-8j4v2"
Sep 09 03:25:51.869266 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:25:51.869246    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/0215d837-84cb-49f2-8652-2c182506b0e9-plugins-config-volume\") pod \"payload-pre-processing-59fbc47bf8-8j4v2\" (UID: \"0215d837-84cb-49f2-8652-2c182506b0e9\") " pod="openshift-ingress/payload-pre-processing-59fbc47bf8-8j4v2"
Sep 09 03:25:51.879389 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:25:51.879365    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-5b54b7fbb7-x6t9q"
Sep 09 03:25:51.916883 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:25:51.916790    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-57q6f\" (UniqueName: \"kubernetes.io/projected/0215d837-84cb-49f2-8652-2c182506b0e9-kube-api-access-57q6f\") pod \"payload-pre-processing-59fbc47bf8-8j4v2\" (UID: \"0215d837-84cb-49f2-8652-2c182506b0e9\") " pod="openshift-ingress/payload-pre-processing-59fbc47bf8-8j4v2"
Sep 09 03:25:51.977418 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:25:51.977389    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-59fbc47bf8-8j4v2"
Sep 09 03:25:52.250314 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:25:52.250237    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-5b54b7fbb7-x6t9q"]
Sep 09 03:25:52.256080 ip-10-0-132-14 kubenswrapper[2576]: W0909 03:25:52.256054    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod701775d1_64f8_4057_b7ee_5aab7fb98ac1.slice/crio-f957e0277ddc5e497f22fe6f108aaa73649f17cfc532a2d36aad913b86788476 WatchSource:0}: Error finding container f957e0277ddc5e497f22fe6f108aaa73649f17cfc532a2d36aad913b86788476: Status 404 returned error can't find the container with id f957e0277ddc5e497f22fe6f108aaa73649f17cfc532a2d36aad913b86788476
Sep 09 03:25:52.280830 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:25:52.280773    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-59fbc47bf8-8j4v2"]
Sep 09 03:25:52.284750 ip-10-0-132-14 kubenswrapper[2576]: W0909 03:25:52.284724    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0215d837_84cb_49f2_8652_2c182506b0e9.slice/crio-50192e2b07d720a6e162c9f8587672aa5c08f3cffab0dbef11e4980c52b57b50 WatchSource:0}: Error finding container 50192e2b07d720a6e162c9f8587672aa5c08f3cffab0dbef11e4980c52b57b50: Status 404 returned error can't find the container with id 50192e2b07d720a6e162c9f8587672aa5c08f3cffab0dbef11e4980c52b57b50
Sep 09 03:25:53.148371 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:25:53.148334    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-59fbc47bf8-8j4v2" event={"ID":"0215d837-84cb-49f2-8652-2c182506b0e9","Type":"ContainerStarted","Data":"50192e2b07d720a6e162c9f8587672aa5c08f3cffab0dbef11e4980c52b57b50"}
Sep 09 03:25:53.152571 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:25:53.152536    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-5b54b7fbb7-x6t9q" event={"ID":"701775d1-64f8-4057-b7ee-5aab7fb98ac1","Type":"ContainerStarted","Data":"f957e0277ddc5e497f22fe6f108aaa73649f17cfc532a2d36aad913b86788476"}
Sep 09 03:25:54.159661 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:25:54.159627    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-59fbc47bf8-8j4v2" event={"ID":"0215d837-84cb-49f2-8652-2c182506b0e9","Type":"ContainerStarted","Data":"45405c7d8d12f01b0543bdf2979ae0a749fecb13b3c15241bb1cf498a69512d0"}
Sep 09 03:25:54.160115 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:25:54.159702    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-59fbc47bf8-8j4v2"
Sep 09 03:25:54.161097 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:25:54.161074    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-5b54b7fbb7-x6t9q" event={"ID":"701775d1-64f8-4057-b7ee-5aab7fb98ac1","Type":"ContainerStarted","Data":"055e283a5d95d6b2b4fbf4415268c79743a9148d557ded39c4e4c262c146d7e1"}
Sep 09 03:25:54.161233 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:25:54.161203    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-5b54b7fbb7-x6t9q"
Sep 09 03:25:54.218012 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:25:54.217953    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-59fbc47bf8-8j4v2" podStartSLOduration=1.462733667 podStartE2EDuration="3.217937407s" totalImagesPullingTime="1.75520374s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 03:25:51 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 03:25:54.216898469 +0000 UTC m=+768.358384546" watchObservedRunningTime="2026-09-09 03:25:54.217937407 +0000 UTC m=+768.359423478"
Sep 09 03:25:54.260128 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:25:54.260070    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-5b54b7fbb7-x6t9q" podStartSLOduration=1.470046372 podStartE2EDuration="3.260053073s" totalImagesPullingTime="1.790006701s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 03:25:51 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 03:25:54.259520812 +0000 UTC m=+768.401006890" watchObservedRunningTime="2026-09-09 03:25:54.260053073 +0000 UTC m=+768.401539149"
Sep 09 03:26:05.168501 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:26:05.168467    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-5b54b7fbb7-x6t9q"
Sep 09 03:26:05.168984 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:26:05.168549    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-59fbc47bf8-8j4v2"
Sep 09 03:26:30.835199 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:26:30.835117    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-distinct-simulated-kserve-59975d69f9-vmtpt"]
Sep 09 03:26:30.841022 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:26:30.841000    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-vmtpt"
Sep 09 03:26:30.868002 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:26:30.867974    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-simulated-kserve-59975d69f9-vmtpt"]
Sep 09 03:26:30.884340 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:26:30.884311    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\""
Sep 09 03:26:30.884790 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:26:30.884773    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-9xhf2\""
Sep 09 03:26:30.885859 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:26:30.885842    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-distinct-simulated-kserve-self-signed-certs\""
Sep 09 03:26:30.886105 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:26:30.886090    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\""
Sep 09 03:26:30.897009 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:26:30.896982    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/253c7999-6ade-4740-821f-d41834527d2e-model-cache\") pod \"e2e-distinct-simulated-kserve-59975d69f9-vmtpt\" (UID: \"253c7999-6ade-4740-821f-d41834527d2e\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-vmtpt"
Sep 09 03:26:30.897117 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:26:30.897020    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/253c7999-6ade-4740-821f-d41834527d2e-home\") pod \"e2e-distinct-simulated-kserve-59975d69f9-vmtpt\" (UID: \"253c7999-6ade-4740-821f-d41834527d2e\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-vmtpt"
Sep 09 03:26:30.897117 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:26:30.897048    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xt6kt\" (UniqueName: \"kubernetes.io/projected/253c7999-6ade-4740-821f-d41834527d2e-kube-api-access-xt6kt\") pod \"e2e-distinct-simulated-kserve-59975d69f9-vmtpt\" (UID: \"253c7999-6ade-4740-821f-d41834527d2e\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-vmtpt"
Sep 09 03:26:30.897192 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:26:30.897121    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/253c7999-6ade-4740-821f-d41834527d2e-tmp-dir\") pod \"e2e-distinct-simulated-kserve-59975d69f9-vmtpt\" (UID: \"253c7999-6ade-4740-821f-d41834527d2e\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-vmtpt"
Sep 09 03:26:30.897192 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:26:30.897163    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/253c7999-6ade-4740-821f-d41834527d2e-dshm\") pod \"e2e-distinct-simulated-kserve-59975d69f9-vmtpt\" (UID: \"253c7999-6ade-4740-821f-d41834527d2e\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-vmtpt"
Sep 09 03:26:30.897192 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:26:30.897180    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/253c7999-6ade-4740-821f-d41834527d2e-tls-certs\") pod \"e2e-distinct-simulated-kserve-59975d69f9-vmtpt\" (UID: \"253c7999-6ade-4740-821f-d41834527d2e\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-vmtpt"
Sep 09 03:26:30.998555 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:26:30.998521    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/253c7999-6ade-4740-821f-d41834527d2e-tmp-dir\") pod \"e2e-distinct-simulated-kserve-59975d69f9-vmtpt\" (UID: \"253c7999-6ade-4740-821f-d41834527d2e\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-vmtpt"
Sep 09 03:26:30.998555 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:26:30.998568    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/253c7999-6ade-4740-821f-d41834527d2e-dshm\") pod \"e2e-distinct-simulated-kserve-59975d69f9-vmtpt\" (UID: \"253c7999-6ade-4740-821f-d41834527d2e\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-vmtpt"
Sep 09 03:26:30.998840 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:26:30.998584    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/253c7999-6ade-4740-821f-d41834527d2e-tls-certs\") pod \"e2e-distinct-simulated-kserve-59975d69f9-vmtpt\" (UID: \"253c7999-6ade-4740-821f-d41834527d2e\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-vmtpt"
Sep 09 03:26:30.998840 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:26:30.998617    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/253c7999-6ade-4740-821f-d41834527d2e-model-cache\") pod \"e2e-distinct-simulated-kserve-59975d69f9-vmtpt\" (UID: \"253c7999-6ade-4740-821f-d41834527d2e\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-vmtpt"
Sep 09 03:26:30.998840 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:26:30.998657    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/253c7999-6ade-4740-821f-d41834527d2e-home\") pod \"e2e-distinct-simulated-kserve-59975d69f9-vmtpt\" (UID: \"253c7999-6ade-4740-821f-d41834527d2e\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-vmtpt"
Sep 09 03:26:30.998840 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:26:30.998703    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xt6kt\" (UniqueName: \"kubernetes.io/projected/253c7999-6ade-4740-821f-d41834527d2e-kube-api-access-xt6kt\") pod \"e2e-distinct-simulated-kserve-59975d69f9-vmtpt\" (UID: \"253c7999-6ade-4740-821f-d41834527d2e\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-vmtpt"
Sep 09 03:26:30.999077 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:26:30.999052    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/253c7999-6ade-4740-821f-d41834527d2e-tmp-dir\") pod \"e2e-distinct-simulated-kserve-59975d69f9-vmtpt\" (UID: \"253c7999-6ade-4740-821f-d41834527d2e\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-vmtpt"
Sep 09 03:26:30.999124 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:26:30.999106    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/253c7999-6ade-4740-821f-d41834527d2e-home\") pod \"e2e-distinct-simulated-kserve-59975d69f9-vmtpt\" (UID: \"253c7999-6ade-4740-821f-d41834527d2e\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-vmtpt"
Sep 09 03:26:30.999212 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:26:30.999164    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/253c7999-6ade-4740-821f-d41834527d2e-model-cache\") pod \"e2e-distinct-simulated-kserve-59975d69f9-vmtpt\" (UID: \"253c7999-6ade-4740-821f-d41834527d2e\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-vmtpt"
Sep 09 03:26:31.001100 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:26:31.001081    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/253c7999-6ade-4740-821f-d41834527d2e-dshm\") pod \"e2e-distinct-simulated-kserve-59975d69f9-vmtpt\" (UID: \"253c7999-6ade-4740-821f-d41834527d2e\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-vmtpt"
Sep 09 03:26:31.001265 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:26:31.001237    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/253c7999-6ade-4740-821f-d41834527d2e-tls-certs\") pod \"e2e-distinct-simulated-kserve-59975d69f9-vmtpt\" (UID: \"253c7999-6ade-4740-821f-d41834527d2e\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-vmtpt"
Sep 09 03:26:31.021569 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:26:31.021530    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xt6kt\" (UniqueName: \"kubernetes.io/projected/253c7999-6ade-4740-821f-d41834527d2e-kube-api-access-xt6kt\") pod \"e2e-distinct-simulated-kserve-59975d69f9-vmtpt\" (UID: \"253c7999-6ade-4740-821f-d41834527d2e\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-vmtpt"
Sep 09 03:26:31.150996 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:26:31.150905    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-vmtpt"
Sep 09 03:26:31.443977 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:26:31.443950    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-simulated-kserve-59975d69f9-vmtpt"]
Sep 09 03:26:31.446123 ip-10-0-132-14 kubenswrapper[2576]: W0909 03:26:31.446092    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod253c7999_6ade_4740_821f_d41834527d2e.slice/crio-6b0c591cd372fb0a8248b8b9170d54053b63183c09185b4b9ac94b2e6392202b WatchSource:0}: Error finding container 6b0c591cd372fb0a8248b8b9170d54053b63183c09185b4b9ac94b2e6392202b: Status 404 returned error can't find the container with id 6b0c591cd372fb0a8248b8b9170d54053b63183c09185b4b9ac94b2e6392202b
Sep 09 03:26:32.326117 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:26:32.326081    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-vmtpt" event={"ID":"253c7999-6ade-4740-821f-d41834527d2e","Type":"ContainerStarted","Data":"6b0c591cd372fb0a8248b8b9170d54053b63183c09185b4b9ac94b2e6392202b"}
Sep 09 03:26:33.335025 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:26:33.334978    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-vmtpt" event={"ID":"253c7999-6ade-4740-821f-d41834527d2e","Type":"ContainerStarted","Data":"2112ccf1f3a143fa878d625bf05b77a065c3f9e3e066bbb3b7306dd3d803b912"}
Sep 09 03:26:41.151596 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:26:41.151551    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-vmtpt"
Sep 09 03:26:41.152043 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:26:41.151626    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-vmtpt"
Sep 09 03:26:41.164053 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:26:41.164029    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-vmtpt"
Sep 09 03:26:41.253199 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:26:41.253136    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-vmtpt" podStartSLOduration=10.291679574 podStartE2EDuration="11.253118733s" totalImagesPullingTime="961.439159ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 03:26:30 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 03:26:33.367566626 +0000 UTC m=+807.509052704" watchObservedRunningTime="2026-09-09 03:26:41.253118733 +0000 UTC m=+815.394604831"
Sep 09 03:26:41.379115 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:26:41.379085    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-vmtpt"
Sep 09 03:26:48.335273 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:26:48.335239    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-598466548d-fj4l4"]
Sep 09 03:26:48.338546 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:26:48.338529    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-fj4l4"
Sep 09 03:26:48.342547 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:26:48.342527    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-trlp-test-simulated-kserve-self-signed-certs\""
Sep 09 03:26:48.372242 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:26:48.372217    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-598466548d-fj4l4"]
Sep 09 03:26:48.444914 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:26:48.444878    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/0f9c7c36-a8ef-4f4d-acd4-56e395ac6230-home\") pod \"e2e-trlp-test-simulated-kserve-598466548d-fj4l4\" (UID: \"0f9c7c36-a8ef-4f4d-acd4-56e395ac6230\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-fj4l4"
Sep 09 03:26:48.444914 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:26:48.444926    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/0f9c7c36-a8ef-4f4d-acd4-56e395ac6230-model-cache\") pod \"e2e-trlp-test-simulated-kserve-598466548d-fj4l4\" (UID: \"0f9c7c36-a8ef-4f4d-acd4-56e395ac6230\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-fj4l4"
Sep 09 03:26:48.445162 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:26:48.444941    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0f9c7c36-a8ef-4f4d-acd4-56e395ac6230-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-598466548d-fj4l4\" (UID: \"0f9c7c36-a8ef-4f4d-acd4-56e395ac6230\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-fj4l4"
Sep 09 03:26:48.445162 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:26:48.445084    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nrbqk\" (UniqueName: \"kubernetes.io/projected/0f9c7c36-a8ef-4f4d-acd4-56e395ac6230-kube-api-access-nrbqk\") pod \"e2e-trlp-test-simulated-kserve-598466548d-fj4l4\" (UID: \"0f9c7c36-a8ef-4f4d-acd4-56e395ac6230\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-fj4l4"
Sep 09 03:26:48.445271 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:26:48.445189    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0f9c7c36-a8ef-4f4d-acd4-56e395ac6230-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-598466548d-fj4l4\" (UID: \"0f9c7c36-a8ef-4f4d-acd4-56e395ac6230\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-fj4l4"
Sep 09 03:26:48.445271 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:26:48.445221    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/0f9c7c36-a8ef-4f4d-acd4-56e395ac6230-dshm\") pod \"e2e-trlp-test-simulated-kserve-598466548d-fj4l4\" (UID: \"0f9c7c36-a8ef-4f4d-acd4-56e395ac6230\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-fj4l4"
Sep 09 03:26:48.546616 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:26:48.546583    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/0f9c7c36-a8ef-4f4d-acd4-56e395ac6230-dshm\") pod \"e2e-trlp-test-simulated-kserve-598466548d-fj4l4\" (UID: \"0f9c7c36-a8ef-4f4d-acd4-56e395ac6230\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-fj4l4"
Sep 09 03:26:48.546825 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:26:48.546658    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/0f9c7c36-a8ef-4f4d-acd4-56e395ac6230-home\") pod \"e2e-trlp-test-simulated-kserve-598466548d-fj4l4\" (UID: \"0f9c7c36-a8ef-4f4d-acd4-56e395ac6230\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-fj4l4"
Sep 09 03:26:48.546825 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:26:48.546695    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/0f9c7c36-a8ef-4f4d-acd4-56e395ac6230-model-cache\") pod \"e2e-trlp-test-simulated-kserve-598466548d-fj4l4\" (UID: \"0f9c7c36-a8ef-4f4d-acd4-56e395ac6230\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-fj4l4"
Sep 09 03:26:48.546825 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:26:48.546717    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0f9c7c36-a8ef-4f4d-acd4-56e395ac6230-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-598466548d-fj4l4\" (UID: \"0f9c7c36-a8ef-4f4d-acd4-56e395ac6230\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-fj4l4"
Sep 09 03:26:48.546825 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:26:48.546752    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nrbqk\" (UniqueName: \"kubernetes.io/projected/0f9c7c36-a8ef-4f4d-acd4-56e395ac6230-kube-api-access-nrbqk\") pod \"e2e-trlp-test-simulated-kserve-598466548d-fj4l4\" (UID: \"0f9c7c36-a8ef-4f4d-acd4-56e395ac6230\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-fj4l4"
Sep 09 03:26:48.547026 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:26:48.546797    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0f9c7c36-a8ef-4f4d-acd4-56e395ac6230-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-598466548d-fj4l4\" (UID: \"0f9c7c36-a8ef-4f4d-acd4-56e395ac6230\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-fj4l4"
Sep 09 03:26:48.547127 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:26:48.547103    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/0f9c7c36-a8ef-4f4d-acd4-56e395ac6230-home\") pod \"e2e-trlp-test-simulated-kserve-598466548d-fj4l4\" (UID: \"0f9c7c36-a8ef-4f4d-acd4-56e395ac6230\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-fj4l4"
Sep 09 03:26:48.547193 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:26:48.547128    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/0f9c7c36-a8ef-4f4d-acd4-56e395ac6230-model-cache\") pod \"e2e-trlp-test-simulated-kserve-598466548d-fj4l4\" (UID: \"0f9c7c36-a8ef-4f4d-acd4-56e395ac6230\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-fj4l4"
Sep 09 03:26:48.547193 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:26:48.547166    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0f9c7c36-a8ef-4f4d-acd4-56e395ac6230-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-598466548d-fj4l4\" (UID: \"0f9c7c36-a8ef-4f4d-acd4-56e395ac6230\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-fj4l4"
Sep 09 03:26:48.549158 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:26:48.549138    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/0f9c7c36-a8ef-4f4d-acd4-56e395ac6230-dshm\") pod \"e2e-trlp-test-simulated-kserve-598466548d-fj4l4\" (UID: \"0f9c7c36-a8ef-4f4d-acd4-56e395ac6230\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-fj4l4"
Sep 09 03:26:48.549318 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:26:48.549302    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0f9c7c36-a8ef-4f4d-acd4-56e395ac6230-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-598466548d-fj4l4\" (UID: \"0f9c7c36-a8ef-4f4d-acd4-56e395ac6230\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-fj4l4"
Sep 09 03:26:48.558697 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:26:48.558674    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nrbqk\" (UniqueName: \"kubernetes.io/projected/0f9c7c36-a8ef-4f4d-acd4-56e395ac6230-kube-api-access-nrbqk\") pod \"e2e-trlp-test-simulated-kserve-598466548d-fj4l4\" (UID: \"0f9c7c36-a8ef-4f4d-acd4-56e395ac6230\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-fj4l4"
Sep 09 03:26:48.647713 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:26:48.647622    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-fj4l4"
Sep 09 03:26:48.934182 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:26:48.933972    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-598466548d-fj4l4"]
Sep 09 03:26:48.936649 ip-10-0-132-14 kubenswrapper[2576]: W0909 03:26:48.936617    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0f9c7c36_a8ef_4f4d_acd4_56e395ac6230.slice/crio-69a42122a61d4c6f89f54a35573ddc53d1ea98c22ac202b6480b893d526d44d4 WatchSource:0}: Error finding container 69a42122a61d4c6f89f54a35573ddc53d1ea98c22ac202b6480b893d526d44d4: Status 404 returned error can't find the container with id 69a42122a61d4c6f89f54a35573ddc53d1ea98c22ac202b6480b893d526d44d4
Sep 09 03:26:49.400712 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:26:49.400678    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-fj4l4" event={"ID":"0f9c7c36-a8ef-4f4d-acd4-56e395ac6230","Type":"ContainerStarted","Data":"382903d0330fe26dc8e8c847c6e0ed992fdbfca62649df984785e4c0034edc8a"}
Sep 09 03:26:49.401150 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:26:49.400726    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-fj4l4" event={"ID":"0f9c7c36-a8ef-4f4d-acd4-56e395ac6230","Type":"ContainerStarted","Data":"69a42122a61d4c6f89f54a35573ddc53d1ea98c22ac202b6480b893d526d44d4"}
Sep 09 03:26:49.432444 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:26:49.432383    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-fj4l4" podStartSLOduration=1.237649349 podStartE2EDuration="1.432364649s" totalImagesPullingTime="194.7153ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 03:26:48 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 03:26:49.432187657 +0000 UTC m=+823.573673735" watchObservedRunningTime="2026-09-09 03:26:49.432364649 +0000 UTC m=+823.573850727"
Sep 09 03:26:58.648422 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:26:58.648380    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-fj4l4"
Sep 09 03:26:58.648874 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:26:58.648443    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-fj4l4"
Sep 09 03:26:58.661220 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:26:58.661196    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-fj4l4"
Sep 09 03:26:59.455166 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:26:59.455136    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-fj4l4"
Sep 09 03:29:21.728868 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:29:21.728831    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-a-40a1c9-7475b65c4-x5f5l"]
Sep 09 03:29:21.732163 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:29:21.732148    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-40a1c9-7475b65c4-x5f5l"
Sep 09 03:29:21.739071 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:29:21.739053    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-inf-a-40a1c9\""
Sep 09 03:29:21.739366 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:29:21.739347    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-inf-a-40a1c9-dockercfg-xs6sp\""
Sep 09 03:29:21.754400 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:29:21.754372    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-a-40a1c9-7475b65c4-x5f5l"]
Sep 09 03:29:21.774926 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:29:21.774891    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/0dd1fa3c-7ce6-46ce-acb7-5da3e43ea653-plugins-config-volume\") pod \"payload-pre-processing-e2e-inf-a-40a1c9-7475b65c4-x5f5l\" (UID: \"0dd1fa3c-7ce6-46ce-acb7-5da3e43ea653\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-a-40a1c9-7475b65c4-x5f5l"
Sep 09 03:29:21.775031 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:29:21.774948    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-78gdh\" (UniqueName: \"kubernetes.io/projected/0dd1fa3c-7ce6-46ce-acb7-5da3e43ea653-kube-api-access-78gdh\") pod \"payload-pre-processing-e2e-inf-a-40a1c9-7475b65c4-x5f5l\" (UID: \"0dd1fa3c-7ce6-46ce-acb7-5da3e43ea653\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-a-40a1c9-7475b65c4-x5f5l"
Sep 09 03:29:21.798112 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:29:21.798084    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-a-40a1c9-5d8b764ddf-rwn6f"]
Sep 09 03:29:21.801436 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:29:21.801415    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-a-40a1c9-5d8b764ddf-rwn6f"
Sep 09 03:29:21.816542 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:29:21.816471    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-a-40a1c9-5d8b764ddf-rwn6f"]
Sep 09 03:29:21.875972 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:29:21.875943    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7st6n\" (UniqueName: \"kubernetes.io/projected/dd6e63da-0e5d-49c1-89b3-8df1f5d52a19-kube-api-access-7st6n\") pod \"payload-processing-e2e-inf-a-40a1c9-5d8b764ddf-rwn6f\" (UID: \"dd6e63da-0e5d-49c1-89b3-8df1f5d52a19\") " pod="openshift-ingress/payload-processing-e2e-inf-a-40a1c9-5d8b764ddf-rwn6f"
Sep 09 03:29:21.876135 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:29:21.875977    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/dd6e63da-0e5d-49c1-89b3-8df1f5d52a19-plugins-config-volume\") pod \"payload-processing-e2e-inf-a-40a1c9-5d8b764ddf-rwn6f\" (UID: \"dd6e63da-0e5d-49c1-89b3-8df1f5d52a19\") " pod="openshift-ingress/payload-processing-e2e-inf-a-40a1c9-5d8b764ddf-rwn6f"
Sep 09 03:29:21.876135 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:29:21.876026    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/0dd1fa3c-7ce6-46ce-acb7-5da3e43ea653-plugins-config-volume\") pod \"payload-pre-processing-e2e-inf-a-40a1c9-7475b65c4-x5f5l\" (UID: \"0dd1fa3c-7ce6-46ce-acb7-5da3e43ea653\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-a-40a1c9-7475b65c4-x5f5l"
Sep 09 03:29:21.876135 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:29:21.876103    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-78gdh\" (UniqueName: \"kubernetes.io/projected/0dd1fa3c-7ce6-46ce-acb7-5da3e43ea653-kube-api-access-78gdh\") pod \"payload-pre-processing-e2e-inf-a-40a1c9-7475b65c4-x5f5l\" (UID: \"0dd1fa3c-7ce6-46ce-acb7-5da3e43ea653\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-a-40a1c9-7475b65c4-x5f5l"
Sep 09 03:29:21.876719 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:29:21.876699    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/0dd1fa3c-7ce6-46ce-acb7-5da3e43ea653-plugins-config-volume\") pod \"payload-pre-processing-e2e-inf-a-40a1c9-7475b65c4-x5f5l\" (UID: \"0dd1fa3c-7ce6-46ce-acb7-5da3e43ea653\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-a-40a1c9-7475b65c4-x5f5l"
Sep 09 03:29:21.889129 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:29:21.889107    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-78gdh\" (UniqueName: \"kubernetes.io/projected/0dd1fa3c-7ce6-46ce-acb7-5da3e43ea653-kube-api-access-78gdh\") pod \"payload-pre-processing-e2e-inf-a-40a1c9-7475b65c4-x5f5l\" (UID: \"0dd1fa3c-7ce6-46ce-acb7-5da3e43ea653\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-a-40a1c9-7475b65c4-x5f5l"
Sep 09 03:29:21.976687 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:29:21.976652    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7st6n\" (UniqueName: \"kubernetes.io/projected/dd6e63da-0e5d-49c1-89b3-8df1f5d52a19-kube-api-access-7st6n\") pod \"payload-processing-e2e-inf-a-40a1c9-5d8b764ddf-rwn6f\" (UID: \"dd6e63da-0e5d-49c1-89b3-8df1f5d52a19\") " pod="openshift-ingress/payload-processing-e2e-inf-a-40a1c9-5d8b764ddf-rwn6f"
Sep 09 03:29:21.976687 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:29:21.976700    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/dd6e63da-0e5d-49c1-89b3-8df1f5d52a19-plugins-config-volume\") pod \"payload-processing-e2e-inf-a-40a1c9-5d8b764ddf-rwn6f\" (UID: \"dd6e63da-0e5d-49c1-89b3-8df1f5d52a19\") " pod="openshift-ingress/payload-processing-e2e-inf-a-40a1c9-5d8b764ddf-rwn6f"
Sep 09 03:29:21.977394 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:29:21.977374    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/dd6e63da-0e5d-49c1-89b3-8df1f5d52a19-plugins-config-volume\") pod \"payload-processing-e2e-inf-a-40a1c9-5d8b764ddf-rwn6f\" (UID: \"dd6e63da-0e5d-49c1-89b3-8df1f5d52a19\") " pod="openshift-ingress/payload-processing-e2e-inf-a-40a1c9-5d8b764ddf-rwn6f"
Sep 09 03:29:21.988129 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:29:21.988070    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7st6n\" (UniqueName: \"kubernetes.io/projected/dd6e63da-0e5d-49c1-89b3-8df1f5d52a19-kube-api-access-7st6n\") pod \"payload-processing-e2e-inf-a-40a1c9-5d8b764ddf-rwn6f\" (UID: \"dd6e63da-0e5d-49c1-89b3-8df1f5d52a19\") " pod="openshift-ingress/payload-processing-e2e-inf-a-40a1c9-5d8b764ddf-rwn6f"
Sep 09 03:29:22.041916 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:29:22.041887    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-40a1c9-7475b65c4-x5f5l"
Sep 09 03:29:22.112208 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:29:22.112180    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-a-40a1c9-5d8b764ddf-rwn6f"
Sep 09 03:29:22.334389 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:29:22.334362    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-a-40a1c9-7475b65c4-x5f5l"]
Sep 09 03:29:22.337040 ip-10-0-132-14 kubenswrapper[2576]: W0909 03:29:22.337012    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0dd1fa3c_7ce6_46ce_acb7_5da3e43ea653.slice/crio-43402e52bbb6a574754623165703e9fac885d68e10846783a5f89938ea2f63f2 WatchSource:0}: Error finding container 43402e52bbb6a574754623165703e9fac885d68e10846783a5f89938ea2f63f2: Status 404 returned error can't find the container with id 43402e52bbb6a574754623165703e9fac885d68e10846783a5f89938ea2f63f2
Sep 09 03:29:22.391905 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:29:22.391882    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-a-40a1c9-5d8b764ddf-rwn6f"]
Sep 09 03:29:22.393497 ip-10-0-132-14 kubenswrapper[2576]: W0909 03:29:22.393470    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddd6e63da_0e5d_49c1_89b3_8df1f5d52a19.slice/crio-6da6b7dbc265e8f150d2a0a85e3f671d78314c8c1706765739f2c3c0adf7a60f WatchSource:0}: Error finding container 6da6b7dbc265e8f150d2a0a85e3f671d78314c8c1706765739f2c3c0adf7a60f: Status 404 returned error can't find the container with id 6da6b7dbc265e8f150d2a0a85e3f671d78314c8c1706765739f2c3c0adf7a60f
Sep 09 03:29:23.031915 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:29:23.031877    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-40a1c9-7475b65c4-x5f5l" event={"ID":"0dd1fa3c-7ce6-46ce-acb7-5da3e43ea653","Type":"ContainerStarted","Data":"3d16fe803556134574d8de6ad4cd6b605fdffdad6472ef7338f2144e51d386e7"}
Sep 09 03:29:23.032317 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:29:23.031940    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-40a1c9-7475b65c4-x5f5l" event={"ID":"0dd1fa3c-7ce6-46ce-acb7-5da3e43ea653","Type":"ContainerStarted","Data":"43402e52bbb6a574754623165703e9fac885d68e10846783a5f89938ea2f63f2"}
Sep 09 03:29:23.032317 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:29:23.031968    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-40a1c9-7475b65c4-x5f5l"
Sep 09 03:29:23.033752 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:29:23.033724    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-a-40a1c9-5d8b764ddf-rwn6f" event={"ID":"dd6e63da-0e5d-49c1-89b3-8df1f5d52a19","Type":"ContainerStarted","Data":"1298772a45eab162c86395165ce9f37eb731ff43f5f5216476583898ee363421"}
Sep 09 03:29:23.033897 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:29:23.033777    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-inf-a-40a1c9-5d8b764ddf-rwn6f"
Sep 09 03:29:23.033897 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:29:23.033824    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-a-40a1c9-5d8b764ddf-rwn6f" event={"ID":"dd6e63da-0e5d-49c1-89b3-8df1f5d52a19","Type":"ContainerStarted","Data":"6da6b7dbc265e8f150d2a0a85e3f671d78314c8c1706765739f2c3c0adf7a60f"}
Sep 09 03:29:23.081477 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:29:23.081431    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-40a1c9-7475b65c4-x5f5l" podStartSLOduration=2.081416398 podStartE2EDuration="2.081416398s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 03:29:21 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 03:29:23.081069605 +0000 UTC m=+977.222555687" watchObservedRunningTime="2026-09-09 03:29:23.081416398 +0000 UTC m=+977.222902475"
Sep 09 03:29:23.126325 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:29:23.126281    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-inf-a-40a1c9-5d8b764ddf-rwn6f" podStartSLOduration=2.126265159 podStartE2EDuration="2.126265159s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 03:29:21 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 03:29:23.123864086 +0000 UTC m=+977.265350163" watchObservedRunningTime="2026-09-09 03:29:23.126265159 +0000 UTC m=+977.267751235"
Sep 09 03:29:30.887694 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:29:30.887614    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-autores-a2fc47-57857d79d5-vbw6n"]
Sep 09 03:29:30.891209 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:29:30.891191    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-autores-a2fc47-57857d79d5-vbw6n"
Sep 09 03:29:30.896178 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:29:30.896161    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-phvzt\""
Sep 09 03:29:30.897608 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:29:30.897578    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-metrics-tls\""
Sep 09 03:29:30.897830 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:29:30.897794    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-autores-a2fc47-serving-cert\""
Sep 09 03:29:30.910226 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:29:30.910204    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-autores-a2fc47-57857d79d5-vbw6n"]
Sep 09 03:29:30.954201 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:29:30.954172    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/03281339-40a3-4b4e-9440-18a8008be8fa-maas-api-tls\") pod \"maas-api-e2e-autores-a2fc47-57857d79d5-vbw6n\" (UID: \"03281339-40a3-4b4e-9440-18a8008be8fa\") " pod="odh-ai-gateway-infra/maas-api-e2e-autores-a2fc47-57857d79d5-vbw6n"
Sep 09 03:29:30.954201 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:29:30.954203    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/03281339-40a3-4b4e-9440-18a8008be8fa-openshift-service-ca\") pod \"maas-api-e2e-autores-a2fc47-57857d79d5-vbw6n\" (UID: \"03281339-40a3-4b4e-9440-18a8008be8fa\") " pod="odh-ai-gateway-infra/maas-api-e2e-autores-a2fc47-57857d79d5-vbw6n"
Sep 09 03:29:30.954375 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:29:30.954228    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2t2kn\" (UniqueName: \"kubernetes.io/projected/03281339-40a3-4b4e-9440-18a8008be8fa-kube-api-access-2t2kn\") pod \"maas-api-e2e-autores-a2fc47-57857d79d5-vbw6n\" (UID: \"03281339-40a3-4b4e-9440-18a8008be8fa\") " pod="odh-ai-gateway-infra/maas-api-e2e-autores-a2fc47-57857d79d5-vbw6n"
Sep 09 03:29:30.954375 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:29:30.954336    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/03281339-40a3-4b4e-9440-18a8008be8fa-metrics-certs\") pod \"maas-api-e2e-autores-a2fc47-57857d79d5-vbw6n\" (UID: \"03281339-40a3-4b4e-9440-18a8008be8fa\") " pod="odh-ai-gateway-infra/maas-api-e2e-autores-a2fc47-57857d79d5-vbw6n"
Sep 09 03:29:31.055551 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:29:31.055510    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/03281339-40a3-4b4e-9440-18a8008be8fa-metrics-certs\") pod \"maas-api-e2e-autores-a2fc47-57857d79d5-vbw6n\" (UID: \"03281339-40a3-4b4e-9440-18a8008be8fa\") " pod="odh-ai-gateway-infra/maas-api-e2e-autores-a2fc47-57857d79d5-vbw6n"
Sep 09 03:29:31.055736 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:29:31.055576    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/03281339-40a3-4b4e-9440-18a8008be8fa-maas-api-tls\") pod \"maas-api-e2e-autores-a2fc47-57857d79d5-vbw6n\" (UID: \"03281339-40a3-4b4e-9440-18a8008be8fa\") " pod="odh-ai-gateway-infra/maas-api-e2e-autores-a2fc47-57857d79d5-vbw6n"
Sep 09 03:29:31.055736 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:29:31.055596    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/03281339-40a3-4b4e-9440-18a8008be8fa-openshift-service-ca\") pod \"maas-api-e2e-autores-a2fc47-57857d79d5-vbw6n\" (UID: \"03281339-40a3-4b4e-9440-18a8008be8fa\") " pod="odh-ai-gateway-infra/maas-api-e2e-autores-a2fc47-57857d79d5-vbw6n"
Sep 09 03:29:31.055736 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:29:31.055621    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2t2kn\" (UniqueName: \"kubernetes.io/projected/03281339-40a3-4b4e-9440-18a8008be8fa-kube-api-access-2t2kn\") pod \"maas-api-e2e-autores-a2fc47-57857d79d5-vbw6n\" (UID: \"03281339-40a3-4b4e-9440-18a8008be8fa\") " pod="odh-ai-gateway-infra/maas-api-e2e-autores-a2fc47-57857d79d5-vbw6n"
Sep 09 03:29:31.055736 ip-10-0-132-14 kubenswrapper[2576]: E0909 03:29:31.055726    2576 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-autores-a2fc47-serving-cert: secret "maas-api-e2e-autores-a2fc47-serving-cert" not found
Sep 09 03:29:31.055990 ip-10-0-132-14 kubenswrapper[2576]: E0909 03:29:31.055795    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/03281339-40a3-4b4e-9440-18a8008be8fa-maas-api-tls podName:03281339-40a3-4b4e-9440-18a8008be8fa nodeName:}" failed. No retries permitted until 2026-09-09 03:29:31.555778264 +0000 UTC m=+985.697264320 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/03281339-40a3-4b4e-9440-18a8008be8fa-maas-api-tls") pod "maas-api-e2e-autores-a2fc47-57857d79d5-vbw6n" (UID: "03281339-40a3-4b4e-9440-18a8008be8fa") : secret "maas-api-e2e-autores-a2fc47-serving-cert" not found
Sep 09 03:29:31.056309 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:29:31.056290    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/03281339-40a3-4b4e-9440-18a8008be8fa-openshift-service-ca\") pod \"maas-api-e2e-autores-a2fc47-57857d79d5-vbw6n\" (UID: \"03281339-40a3-4b4e-9440-18a8008be8fa\") " pod="odh-ai-gateway-infra/maas-api-e2e-autores-a2fc47-57857d79d5-vbw6n"
Sep 09 03:29:31.058198 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:29:31.058174    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/03281339-40a3-4b4e-9440-18a8008be8fa-metrics-certs\") pod \"maas-api-e2e-autores-a2fc47-57857d79d5-vbw6n\" (UID: \"03281339-40a3-4b4e-9440-18a8008be8fa\") " pod="odh-ai-gateway-infra/maas-api-e2e-autores-a2fc47-57857d79d5-vbw6n"
Sep 09 03:29:31.065589 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:29:31.065543    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2t2kn\" (UniqueName: \"kubernetes.io/projected/03281339-40a3-4b4e-9440-18a8008be8fa-kube-api-access-2t2kn\") pod \"maas-api-e2e-autores-a2fc47-57857d79d5-vbw6n\" (UID: \"03281339-40a3-4b4e-9440-18a8008be8fa\") " pod="odh-ai-gateway-infra/maas-api-e2e-autores-a2fc47-57857d79d5-vbw6n"
Sep 09 03:29:31.559322 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:29:31.559291    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/03281339-40a3-4b4e-9440-18a8008be8fa-maas-api-tls\") pod \"maas-api-e2e-autores-a2fc47-57857d79d5-vbw6n\" (UID: \"03281339-40a3-4b4e-9440-18a8008be8fa\") " pod="odh-ai-gateway-infra/maas-api-e2e-autores-a2fc47-57857d79d5-vbw6n"
Sep 09 03:29:31.561831 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:29:31.561779    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/03281339-40a3-4b4e-9440-18a8008be8fa-maas-api-tls\") pod \"maas-api-e2e-autores-a2fc47-57857d79d5-vbw6n\" (UID: \"03281339-40a3-4b4e-9440-18a8008be8fa\") " pod="odh-ai-gateway-infra/maas-api-e2e-autores-a2fc47-57857d79d5-vbw6n"
Sep 09 03:29:31.754532 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:29:31.754500    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-autores-a2fc47-8fdd95594-hxg7w"]
Sep 09 03:29:31.758006 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:29:31.757989    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-autores-a2fc47-8fdd95594-hxg7w"
Sep 09 03:29:31.761001 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:29:31.760979    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-autores-a2fc47\""
Sep 09 03:29:31.770065 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:29:31.770042    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-autores-a2fc47-8fdd95594-hxg7w"]
Sep 09 03:29:31.801403 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:29:31.801379    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-autores-a2fc47-57857d79d5-vbw6n"
Sep 09 03:29:31.861673 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:29:31.861636    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jz8vx\" (UniqueName: \"kubernetes.io/projected/1047ee3d-c020-44c2-948c-719a1e85ff72-kube-api-access-jz8vx\") pod \"payload-pre-processing-e2e-autores-a2fc47-8fdd95594-hxg7w\" (UID: \"1047ee3d-c020-44c2-948c-719a1e85ff72\") " pod="openshift-ingress/payload-pre-processing-e2e-autores-a2fc47-8fdd95594-hxg7w"
Sep 09 03:29:31.861843 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:29:31.861719    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/1047ee3d-c020-44c2-948c-719a1e85ff72-plugins-config-volume\") pod \"payload-pre-processing-e2e-autores-a2fc47-8fdd95594-hxg7w\" (UID: \"1047ee3d-c020-44c2-948c-719a1e85ff72\") " pod="openshift-ingress/payload-pre-processing-e2e-autores-a2fc47-8fdd95594-hxg7w"
Sep 09 03:29:31.884431 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:29:31.884404    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-autores-a2fc47-69fdd48ccf-wjj48"]
Sep 09 03:29:31.888145 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:29:31.888122    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-autores-a2fc47-69fdd48ccf-wjj48"
Sep 09 03:29:31.891326 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:29:31.891305    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-autores-a2fc47-dockercfg-b2w5m\""
Sep 09 03:29:31.902291 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:29:31.902266    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-autores-a2fc47-69fdd48ccf-wjj48"]
Sep 09 03:29:31.962344 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:29:31.962313    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/82d8fbaa-1300-4bcc-be5d-77bf70dbed9c-plugins-config-volume\") pod \"payload-processing-e2e-autores-a2fc47-69fdd48ccf-wjj48\" (UID: \"82d8fbaa-1300-4bcc-be5d-77bf70dbed9c\") " pod="openshift-ingress/payload-processing-e2e-autores-a2fc47-69fdd48ccf-wjj48"
Sep 09 03:29:31.962518 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:29:31.962359    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jz8vx\" (UniqueName: \"kubernetes.io/projected/1047ee3d-c020-44c2-948c-719a1e85ff72-kube-api-access-jz8vx\") pod \"payload-pre-processing-e2e-autores-a2fc47-8fdd95594-hxg7w\" (UID: \"1047ee3d-c020-44c2-948c-719a1e85ff72\") " pod="openshift-ingress/payload-pre-processing-e2e-autores-a2fc47-8fdd95594-hxg7w"
Sep 09 03:29:31.962518 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:29:31.962420    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/1047ee3d-c020-44c2-948c-719a1e85ff72-plugins-config-volume\") pod \"payload-pre-processing-e2e-autores-a2fc47-8fdd95594-hxg7w\" (UID: \"1047ee3d-c020-44c2-948c-719a1e85ff72\") " pod="openshift-ingress/payload-pre-processing-e2e-autores-a2fc47-8fdd95594-hxg7w"
Sep 09 03:29:31.962518 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:29:31.962436    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-j4wbf\" (UniqueName: \"kubernetes.io/projected/82d8fbaa-1300-4bcc-be5d-77bf70dbed9c-kube-api-access-j4wbf\") pod \"payload-processing-e2e-autores-a2fc47-69fdd48ccf-wjj48\" (UID: \"82d8fbaa-1300-4bcc-be5d-77bf70dbed9c\") " pod="openshift-ingress/payload-processing-e2e-autores-a2fc47-69fdd48ccf-wjj48"
Sep 09 03:29:31.963172 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:29:31.963148    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/1047ee3d-c020-44c2-948c-719a1e85ff72-plugins-config-volume\") pod \"payload-pre-processing-e2e-autores-a2fc47-8fdd95594-hxg7w\" (UID: \"1047ee3d-c020-44c2-948c-719a1e85ff72\") " pod="openshift-ingress/payload-pre-processing-e2e-autores-a2fc47-8fdd95594-hxg7w"
Sep 09 03:29:31.972216 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:29:31.972191    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jz8vx\" (UniqueName: \"kubernetes.io/projected/1047ee3d-c020-44c2-948c-719a1e85ff72-kube-api-access-jz8vx\") pod \"payload-pre-processing-e2e-autores-a2fc47-8fdd95594-hxg7w\" (UID: \"1047ee3d-c020-44c2-948c-719a1e85ff72\") " pod="openshift-ingress/payload-pre-processing-e2e-autores-a2fc47-8fdd95594-hxg7w"
Sep 09 03:29:32.063204 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:29:32.063121    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/82d8fbaa-1300-4bcc-be5d-77bf70dbed9c-plugins-config-volume\") pod \"payload-processing-e2e-autores-a2fc47-69fdd48ccf-wjj48\" (UID: \"82d8fbaa-1300-4bcc-be5d-77bf70dbed9c\") " pod="openshift-ingress/payload-processing-e2e-autores-a2fc47-69fdd48ccf-wjj48"
Sep 09 03:29:32.063204 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:29:32.063198    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-j4wbf\" (UniqueName: \"kubernetes.io/projected/82d8fbaa-1300-4bcc-be5d-77bf70dbed9c-kube-api-access-j4wbf\") pod \"payload-processing-e2e-autores-a2fc47-69fdd48ccf-wjj48\" (UID: \"82d8fbaa-1300-4bcc-be5d-77bf70dbed9c\") " pod="openshift-ingress/payload-processing-e2e-autores-a2fc47-69fdd48ccf-wjj48"
Sep 09 03:29:32.063829 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:29:32.063792    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/82d8fbaa-1300-4bcc-be5d-77bf70dbed9c-plugins-config-volume\") pod \"payload-processing-e2e-autores-a2fc47-69fdd48ccf-wjj48\" (UID: \"82d8fbaa-1300-4bcc-be5d-77bf70dbed9c\") " pod="openshift-ingress/payload-processing-e2e-autores-a2fc47-69fdd48ccf-wjj48"
Sep 09 03:29:32.068572 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:29:32.068546    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-autores-a2fc47-8fdd95594-hxg7w"
Sep 09 03:29:32.074696 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:29:32.074678    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-autores-a2fc47-57857d79d5-vbw6n"]
Sep 09 03:29:32.078346 ip-10-0-132-14 kubenswrapper[2576]: W0909 03:29:32.078323    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod03281339_40a3_4b4e_9440_18a8008be8fa.slice/crio-1b8601bb23aca731514e50aeb5041f012beeff193005f8cb49cb0570b9804bf8 WatchSource:0}: Error finding container 1b8601bb23aca731514e50aeb5041f012beeff193005f8cb49cb0570b9804bf8: Status 404 returned error can't find the container with id 1b8601bb23aca731514e50aeb5041f012beeff193005f8cb49cb0570b9804bf8
Sep 09 03:29:32.080945 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:29:32.080920    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-j4wbf\" (UniqueName: \"kubernetes.io/projected/82d8fbaa-1300-4bcc-be5d-77bf70dbed9c-kube-api-access-j4wbf\") pod \"payload-processing-e2e-autores-a2fc47-69fdd48ccf-wjj48\" (UID: \"82d8fbaa-1300-4bcc-be5d-77bf70dbed9c\") " pod="openshift-ingress/payload-processing-e2e-autores-a2fc47-69fdd48ccf-wjj48"
Sep 09 03:29:32.202240 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:29:32.202211    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-autores-a2fc47-69fdd48ccf-wjj48"
Sep 09 03:29:32.350907 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:29:32.350873    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-autores-a2fc47-8fdd95594-hxg7w"]
Sep 09 03:29:32.351841 ip-10-0-132-14 kubenswrapper[2576]: W0909 03:29:32.351793    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1047ee3d_c020_44c2_948c_719a1e85ff72.slice/crio-248197bba51920178e0106063f841a1511e935e524a57dc6225dc0226a92a92f WatchSource:0}: Error finding container 248197bba51920178e0106063f841a1511e935e524a57dc6225dc0226a92a92f: Status 404 returned error can't find the container with id 248197bba51920178e0106063f841a1511e935e524a57dc6225dc0226a92a92f
Sep 09 03:29:32.483967 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:29:32.483944    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-autores-a2fc47-69fdd48ccf-wjj48"]
Sep 09 03:29:32.485632 ip-10-0-132-14 kubenswrapper[2576]: W0909 03:29:32.485610    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod82d8fbaa_1300_4bcc_be5d_77bf70dbed9c.slice/crio-6df7a86d73b6c044fcdb3e5ffb696d41c89a8ec83b8834e0a5fece1fe24fb979 WatchSource:0}: Error finding container 6df7a86d73b6c044fcdb3e5ffb696d41c89a8ec83b8834e0a5fece1fe24fb979: Status 404 returned error can't find the container with id 6df7a86d73b6c044fcdb3e5ffb696d41c89a8ec83b8834e0a5fece1fe24fb979
Sep 09 03:29:32.945841 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:29:32.945788    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-9c982818-8487d9d978-2smf9"]
Sep 09 03:29:32.949623 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:29:32.949591    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-9c982818-8487d9d978-2smf9"
Sep 09 03:29:32.953203 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:29:32.953177    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-ait-9c982818\""
Sep 09 03:29:32.970128 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:29:32.970044    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-9c982818-8487d9d978-2smf9"]
Sep 09 03:29:33.082513 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:29:33.081831    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-autores-a2fc47-69fdd48ccf-wjj48" event={"ID":"82d8fbaa-1300-4bcc-be5d-77bf70dbed9c","Type":"ContainerStarted","Data":"f0b0badd73f452a7093302bd917882275647cfc46cb764b351abea8742c0368c"}
Sep 09 03:29:33.082513 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:29:33.081901    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-autores-a2fc47-69fdd48ccf-wjj48" event={"ID":"82d8fbaa-1300-4bcc-be5d-77bf70dbed9c","Type":"ContainerStarted","Data":"6df7a86d73b6c044fcdb3e5ffb696d41c89a8ec83b8834e0a5fece1fe24fb979"}
Sep 09 03:29:33.082513 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:29:33.082004    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-r8d9j\" (UniqueName: \"kubernetes.io/projected/64b0e07d-79c1-48b3-ae6f-f7236fcc6e47-kube-api-access-r8d9j\") pod \"payload-pre-processing-e2e-ait-9c982818-8487d9d978-2smf9\" (UID: \"64b0e07d-79c1-48b3-ae6f-f7236fcc6e47\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-9c982818-8487d9d978-2smf9"
Sep 09 03:29:33.082513 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:29:33.082060    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/64b0e07d-79c1-48b3-ae6f-f7236fcc6e47-plugins-config-volume\") pod \"payload-pre-processing-e2e-ait-9c982818-8487d9d978-2smf9\" (UID: \"64b0e07d-79c1-48b3-ae6f-f7236fcc6e47\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-9c982818-8487d9d978-2smf9"
Sep 09 03:29:33.082513 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:29:33.082172    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-autores-a2fc47-69fdd48ccf-wjj48"
Sep 09 03:29:33.084618 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:29:33.084593    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-autores-a2fc47-57857d79d5-vbw6n" event={"ID":"03281339-40a3-4b4e-9440-18a8008be8fa","Type":"ContainerStarted","Data":"1b8601bb23aca731514e50aeb5041f012beeff193005f8cb49cb0570b9804bf8"}
Sep 09 03:29:33.086446 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:29:33.086425    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-autores-a2fc47-8fdd95594-hxg7w" event={"ID":"1047ee3d-c020-44c2-948c-719a1e85ff72","Type":"ContainerStarted","Data":"174b04dd483193fe2b17dbf2702846a3a303faf8b839d934a77c4e5e380688a0"}
Sep 09 03:29:33.086548 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:29:33.086465    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-autores-a2fc47-8fdd95594-hxg7w" event={"ID":"1047ee3d-c020-44c2-948c-719a1e85ff72","Type":"ContainerStarted","Data":"248197bba51920178e0106063f841a1511e935e524a57dc6225dc0226a92a92f"}
Sep 09 03:29:33.086609 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:29:33.086596    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-autores-a2fc47-8fdd95594-hxg7w"
Sep 09 03:29:33.111150 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:29:33.111060    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-autores-a2fc47-69fdd48ccf-wjj48" podStartSLOduration=2.111041315 podStartE2EDuration="2.111041315s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 03:29:31 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 03:29:33.110251635 +0000 UTC m=+987.251737712" watchObservedRunningTime="2026-09-09 03:29:33.111041315 +0000 UTC m=+987.252527393"
Sep 09 03:29:33.133765 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:29:33.133684    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-autores-a2fc47-8fdd95594-hxg7w" podStartSLOduration=2.133646211 podStartE2EDuration="2.133646211s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 03:29:31 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 03:29:33.132180775 +0000 UTC m=+987.273666851" watchObservedRunningTime="2026-09-09 03:29:33.133646211 +0000 UTC m=+987.275132291"
Sep 09 03:29:33.183958 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:29:33.183544    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-r8d9j\" (UniqueName: \"kubernetes.io/projected/64b0e07d-79c1-48b3-ae6f-f7236fcc6e47-kube-api-access-r8d9j\") pod \"payload-pre-processing-e2e-ait-9c982818-8487d9d978-2smf9\" (UID: \"64b0e07d-79c1-48b3-ae6f-f7236fcc6e47\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-9c982818-8487d9d978-2smf9"
Sep 09 03:29:33.183958 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:29:33.183600    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/64b0e07d-79c1-48b3-ae6f-f7236fcc6e47-plugins-config-volume\") pod \"payload-pre-processing-e2e-ait-9c982818-8487d9d978-2smf9\" (UID: \"64b0e07d-79c1-48b3-ae6f-f7236fcc6e47\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-9c982818-8487d9d978-2smf9"
Sep 09 03:29:33.184432 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:29:33.184403    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/64b0e07d-79c1-48b3-ae6f-f7236fcc6e47-plugins-config-volume\") pod \"payload-pre-processing-e2e-ait-9c982818-8487d9d978-2smf9\" (UID: \"64b0e07d-79c1-48b3-ae6f-f7236fcc6e47\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-9c982818-8487d9d978-2smf9"
Sep 09 03:29:33.193381 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:29:33.193355    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-r8d9j\" (UniqueName: \"kubernetes.io/projected/64b0e07d-79c1-48b3-ae6f-f7236fcc6e47-kube-api-access-r8d9j\") pod \"payload-pre-processing-e2e-ait-9c982818-8487d9d978-2smf9\" (UID: \"64b0e07d-79c1-48b3-ae6f-f7236fcc6e47\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-9c982818-8487d9d978-2smf9"
Sep 09 03:29:33.262868 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:29:33.262832    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-9c982818-8487d9d978-2smf9"
Sep 09 03:29:33.551547 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:29:33.551466    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-9c982818-8487d9d978-2smf9"]
Sep 09 03:29:33.739534 ip-10-0-132-14 kubenswrapper[2576]: W0909 03:29:33.739497    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod64b0e07d_79c1_48b3_ae6f_f7236fcc6e47.slice/crio-74fe19f70330468a56c8519000b9adb59de26cb5114ec98742c3ec8667f21279 WatchSource:0}: Error finding container 74fe19f70330468a56c8519000b9adb59de26cb5114ec98742c3ec8667f21279: Status 404 returned error can't find the container with id 74fe19f70330468a56c8519000b9adb59de26cb5114ec98742c3ec8667f21279
Sep 09 03:29:34.040414 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:29:34.040381    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-40a1c9-7475b65c4-x5f5l"
Sep 09 03:29:34.040904 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:29:34.040462    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-inf-a-40a1c9-5d8b764ddf-rwn6f"
Sep 09 03:29:34.094544 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:29:34.094428    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-9c982818-8487d9d978-2smf9" event={"ID":"64b0e07d-79c1-48b3-ae6f-f7236fcc6e47","Type":"ContainerStarted","Data":"2e39d96c2ea111b14f2802e85d4ffe46fe530b101f3a47293b1a8e9d59ac598e"}
Sep 09 03:29:34.094544 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:29:34.094490    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-9c982818-8487d9d978-2smf9" event={"ID":"64b0e07d-79c1-48b3-ae6f-f7236fcc6e47","Type":"ContainerStarted","Data":"74fe19f70330468a56c8519000b9adb59de26cb5114ec98742c3ec8667f21279"}
Sep 09 03:29:34.094544 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:29:34.094522    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-ait-9c982818-8487d9d978-2smf9"
Sep 09 03:29:34.102628 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:29:34.102589    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-autores-a2fc47-57857d79d5-vbw6n" event={"ID":"03281339-40a3-4b4e-9440-18a8008be8fa","Type":"ContainerStarted","Data":"a59ef5365878d0afa2fd61977eaf8d735444f3a960dae0f573868a07ba5b0a7e"}
Sep 09 03:29:34.125628 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:29:34.125576    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-ait-9c982818-8487d9d978-2smf9" podStartSLOduration=2.12556023 podStartE2EDuration="2.12556023s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 03:29:32 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 03:29:34.124308413 +0000 UTC m=+988.265794491" watchObservedRunningTime="2026-09-09 03:29:34.12556023 +0000 UTC m=+988.267046361"
Sep 09 03:29:34.146257 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:29:34.146205    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-autores-a2fc47-57857d79d5-vbw6n" podStartSLOduration=2.440970824 podStartE2EDuration="4.146183218s" totalImagesPullingTime="1.705212394s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 03:29:30 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 03:29:34.145203174 +0000 UTC m=+988.286689253" watchObservedRunningTime="2026-09-09 03:29:34.146183218 +0000 UTC m=+988.287669298"
Sep 09 03:29:35.106527 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:29:35.106498    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-autores-a2fc47-57857d79d5-vbw6n"
Sep 09 03:29:41.116356 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:29:41.116327    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-autores-a2fc47-57857d79d5-vbw6n"
Sep 09 03:29:44.104967 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:29:44.104939    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-autores-a2fc47-69fdd48ccf-wjj48"
Sep 09 03:29:44.105324 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:29:44.105002    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-autores-a2fc47-8fdd95594-hxg7w"
Sep 09 03:29:45.108868 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:29:45.108836    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-ait-9c982818-8487d9d978-2smf9"
Sep 09 03:29:55.928296 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:29:55.928260    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-9c982818-spb9z"]
Sep 09 03:29:55.933073 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:29:55.933053    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-9c982818-spb9z"
Sep 09 03:29:55.935999 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:29:55.935974    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-w7q9f\""
Sep 09 03:29:55.941294 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:29:55.941271    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-9c982818-spb9z"]
Sep 09 03:29:56.079428 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:29:56.079402    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/ead9312f-d248-4d9b-aee4-9706a8e2f478-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-9c982818-spb9z\" (UID: \"ead9312f-d248-4d9b-aee4-9706a8e2f478\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-9c982818-spb9z"
Sep 09 03:29:56.180460 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:29:56.180389    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/ead9312f-d248-4d9b-aee4-9706a8e2f478-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-9c982818-spb9z\" (UID: \"ead9312f-d248-4d9b-aee4-9706a8e2f478\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-9c982818-spb9z"
Sep 09 03:29:56.181014 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:29:56.180994    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/ead9312f-d248-4d9b-aee4-9706a8e2f478-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-9c982818-spb9z\" (UID: \"ead9312f-d248-4d9b-aee4-9706a8e2f478\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-9c982818-spb9z"
Sep 09 03:29:56.244665 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:29:56.244636    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-9c982818-spb9z"
Sep 09 03:29:56.515203 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:29:56.515179    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-9c982818-spb9z"]
Sep 09 03:29:56.517215 ip-10-0-132-14 kubenswrapper[2576]: W0909 03:29:56.517188    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podead9312f_d248_4d9b_aee4_9706a8e2f478.slice/crio-10614351edc9c4874aec65297110a7937e6e30ad31ed73ce2a65f807847586bc WatchSource:0}: Error finding container 10614351edc9c4874aec65297110a7937e6e30ad31ed73ce2a65f807847586bc: Status 404 returned error can't find the container with id 10614351edc9c4874aec65297110a7937e6e30ad31ed73ce2a65f807847586bc
Sep 09 03:29:56.519288 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:29:56.519267    2576 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 09 03:29:57.217221 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:29:57.217179    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-9c982818-spb9z" event={"ID":"ead9312f-d248-4d9b-aee4-9706a8e2f478","Type":"ContainerStarted","Data":"10614351edc9c4874aec65297110a7937e6e30ad31ed73ce2a65f807847586bc"}
Sep 09 03:29:57.296510 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:29:57.296475    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-b-4bd6b8-6f98d797d-h8b2x"]
Sep 09 03:29:57.301096 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:29:57.301080    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-4bd6b8-6f98d797d-h8b2x"
Sep 09 03:29:57.316863 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:29:57.316842    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-inf-b-4bd6b8\""
Sep 09 03:29:57.337961 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:29:57.337940    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-b-4bd6b8-6f98d797d-h8b2x"]
Sep 09 03:29:57.388844 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:29:57.388816    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-b-4bd6b8-6458c89d79-jt8fk"]
Sep 09 03:29:57.389191 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:29:57.389170    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2grgt\" (UniqueName: \"kubernetes.io/projected/a2041f45-6ca8-4207-9604-1c8890c9e9ea-kube-api-access-2grgt\") pod \"payload-pre-processing-e2e-inf-b-4bd6b8-6f98d797d-h8b2x\" (UID: \"a2041f45-6ca8-4207-9604-1c8890c9e9ea\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-b-4bd6b8-6f98d797d-h8b2x"
Sep 09 03:29:57.389276 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:29:57.389244    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a2041f45-6ca8-4207-9604-1c8890c9e9ea-plugins-config-volume\") pod \"payload-pre-processing-e2e-inf-b-4bd6b8-6f98d797d-h8b2x\" (UID: \"a2041f45-6ca8-4207-9604-1c8890c9e9ea\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-b-4bd6b8-6f98d797d-h8b2x"
Sep 09 03:29:57.392265 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:29:57.392251    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-b-4bd6b8-6458c89d79-jt8fk"
Sep 09 03:29:57.412586 ip-10-0-132-14 kubenswrapper[2576]: E0909 03:29:57.412554    2576 reflector.go:205] "Failed to watch" err="failed to list *v1.Secret: secrets \"payload-processing-e2e-inf-b-4bd6b8-dockercfg-kh6nw\" is forbidden: User \"system:node:ip-10-0-132-14.ec2.internal\" cannot list resource \"secrets\" in API group \"\" in the namespace \"openshift-ingress\": no relationship found between node 'ip-10-0-132-14.ec2.internal' and this object" logger="UnhandledError" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-inf-b-4bd6b8-dockercfg-kh6nw\"" type="*v1.Secret"
Sep 09 03:29:57.461848 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:29:57.461817    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-b-4bd6b8-6458c89d79-jt8fk"]
Sep 09 03:29:57.490726 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:29:57.490650    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2grgt\" (UniqueName: \"kubernetes.io/projected/a2041f45-6ca8-4207-9604-1c8890c9e9ea-kube-api-access-2grgt\") pod \"payload-pre-processing-e2e-inf-b-4bd6b8-6f98d797d-h8b2x\" (UID: \"a2041f45-6ca8-4207-9604-1c8890c9e9ea\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-b-4bd6b8-6f98d797d-h8b2x"
Sep 09 03:29:57.490726 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:29:57.490681    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-b88q7\" (UniqueName: \"kubernetes.io/projected/766d00dc-b659-4682-b44b-4e42a6c4e844-kube-api-access-b88q7\") pod \"payload-processing-e2e-inf-b-4bd6b8-6458c89d79-jt8fk\" (UID: \"766d00dc-b659-4682-b44b-4e42a6c4e844\") " pod="openshift-ingress/payload-processing-e2e-inf-b-4bd6b8-6458c89d79-jt8fk"
Sep 09 03:29:57.490930 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:29:57.490844    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a2041f45-6ca8-4207-9604-1c8890c9e9ea-plugins-config-volume\") pod \"payload-pre-processing-e2e-inf-b-4bd6b8-6f98d797d-h8b2x\" (UID: \"a2041f45-6ca8-4207-9604-1c8890c9e9ea\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-b-4bd6b8-6f98d797d-h8b2x"
Sep 09 03:29:57.490930 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:29:57.490865    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/766d00dc-b659-4682-b44b-4e42a6c4e844-plugins-config-volume\") pod \"payload-processing-e2e-inf-b-4bd6b8-6458c89d79-jt8fk\" (UID: \"766d00dc-b659-4682-b44b-4e42a6c4e844\") " pod="openshift-ingress/payload-processing-e2e-inf-b-4bd6b8-6458c89d79-jt8fk"
Sep 09 03:29:57.491554 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:29:57.491535    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a2041f45-6ca8-4207-9604-1c8890c9e9ea-plugins-config-volume\") pod \"payload-pre-processing-e2e-inf-b-4bd6b8-6f98d797d-h8b2x\" (UID: \"a2041f45-6ca8-4207-9604-1c8890c9e9ea\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-b-4bd6b8-6f98d797d-h8b2x"
Sep 09 03:29:57.506279 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:29:57.506248    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2grgt\" (UniqueName: \"kubernetes.io/projected/a2041f45-6ca8-4207-9604-1c8890c9e9ea-kube-api-access-2grgt\") pod \"payload-pre-processing-e2e-inf-b-4bd6b8-6f98d797d-h8b2x\" (UID: \"a2041f45-6ca8-4207-9604-1c8890c9e9ea\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-b-4bd6b8-6f98d797d-h8b2x"
Sep 09 03:29:57.591455 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:29:57.591424    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-b88q7\" (UniqueName: \"kubernetes.io/projected/766d00dc-b659-4682-b44b-4e42a6c4e844-kube-api-access-b88q7\") pod \"payload-processing-e2e-inf-b-4bd6b8-6458c89d79-jt8fk\" (UID: \"766d00dc-b659-4682-b44b-4e42a6c4e844\") " pod="openshift-ingress/payload-processing-e2e-inf-b-4bd6b8-6458c89d79-jt8fk"
Sep 09 03:29:57.591575 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:29:57.591516    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/766d00dc-b659-4682-b44b-4e42a6c4e844-plugins-config-volume\") pod \"payload-processing-e2e-inf-b-4bd6b8-6458c89d79-jt8fk\" (UID: \"766d00dc-b659-4682-b44b-4e42a6c4e844\") " pod="openshift-ingress/payload-processing-e2e-inf-b-4bd6b8-6458c89d79-jt8fk"
Sep 09 03:29:57.592245 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:29:57.592224    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/766d00dc-b659-4682-b44b-4e42a6c4e844-plugins-config-volume\") pod \"payload-processing-e2e-inf-b-4bd6b8-6458c89d79-jt8fk\" (UID: \"766d00dc-b659-4682-b44b-4e42a6c4e844\") " pod="openshift-ingress/payload-processing-e2e-inf-b-4bd6b8-6458c89d79-jt8fk"
Sep 09 03:29:57.611099 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:29:57.611076    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-4bd6b8-6f98d797d-h8b2x"
Sep 09 03:29:57.615648 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:29:57.615609    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-b88q7\" (UniqueName: \"kubernetes.io/projected/766d00dc-b659-4682-b44b-4e42a6c4e844-kube-api-access-b88q7\") pod \"payload-processing-e2e-inf-b-4bd6b8-6458c89d79-jt8fk\" (UID: \"766d00dc-b659-4682-b44b-4e42a6c4e844\") " pod="openshift-ingress/payload-processing-e2e-inf-b-4bd6b8-6458c89d79-jt8fk"
Sep 09 03:29:57.898775 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:29:57.898727    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-b-4bd6b8-6f98d797d-h8b2x"]
Sep 09 03:29:57.901388 ip-10-0-132-14 kubenswrapper[2576]: W0909 03:29:57.901355    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda2041f45_6ca8_4207_9604_1c8890c9e9ea.slice/crio-d401d6afb958ba6627af05d15daf7ed48aab0a31d2deb50f1ccb10d89dd6b683 WatchSource:0}: Error finding container d401d6afb958ba6627af05d15daf7ed48aab0a31d2deb50f1ccb10d89dd6b683: Status 404 returned error can't find the container with id d401d6afb958ba6627af05d15daf7ed48aab0a31d2deb50f1ccb10d89dd6b683
Sep 09 03:29:58.224005 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:29:58.223936    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-4bd6b8-6f98d797d-h8b2x" event={"ID":"a2041f45-6ca8-4207-9604-1c8890c9e9ea","Type":"ContainerStarted","Data":"2aad3eaa40285cefc7ec738749512a3382b4d829046c254d6716c8bb0f9078c3"}
Sep 09 03:29:58.224005 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:29:58.223988    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-4bd6b8-6f98d797d-h8b2x" event={"ID":"a2041f45-6ca8-4207-9604-1c8890c9e9ea","Type":"ContainerStarted","Data":"d401d6afb958ba6627af05d15daf7ed48aab0a31d2deb50f1ccb10d89dd6b683"}
Sep 09 03:29:58.224370 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:29:58.224053    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-4bd6b8-6f98d797d-h8b2x"
Sep 09 03:29:58.243975 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:29:58.243921    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-4bd6b8-6f98d797d-h8b2x" podStartSLOduration=1.243906765 podStartE2EDuration="1.243906765s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 03:29:57 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 03:29:58.241975688 +0000 UTC m=+1012.383461765" watchObservedRunningTime="2026-09-09 03:29:58.243906765 +0000 UTC m=+1012.385392863"
Sep 09 03:29:58.455177 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:29:58.455136    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-inf-b-4bd6b8-dockercfg-kh6nw\""
Sep 09 03:29:58.463881 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:29:58.463857    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-b-4bd6b8-6458c89d79-jt8fk"
Sep 09 03:29:59.229294 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:29:59.229269    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-b-4bd6b8-6458c89d79-jt8fk"]
Sep 09 03:29:59.231477 ip-10-0-132-14 kubenswrapper[2576]: W0909 03:29:59.230942    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod766d00dc_b659_4682_b44b_4e42a6c4e844.slice/crio-eeb93b70d0206356698014d6a6b2384014065b304e3b92e05d86eff9a69ce1a3 WatchSource:0}: Error finding container eeb93b70d0206356698014d6a6b2384014065b304e3b92e05d86eff9a69ce1a3: Status 404 returned error can't find the container with id eeb93b70d0206356698014d6a6b2384014065b304e3b92e05d86eff9a69ce1a3
Sep 09 03:30:00.147387 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:30:00.147353    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ait-9c982818-29815410-kb9f4"]
Sep 09 03:30:00.150750 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:30:00.150734    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ait-9c982818-29815410-kb9f4"
Sep 09 03:30:00.161026 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:30:00.161006    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ait-9c982818-29815410-kb9f4"
Sep 09 03:30:00.168157 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:30:00.168136    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ait-9c982818-29815410-kb9f4"]
Sep 09 03:30:00.238548 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:30:00.238497    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-b-4bd6b8-6458c89d79-jt8fk" event={"ID":"766d00dc-b659-4682-b44b-4e42a6c4e844","Type":"ContainerStarted","Data":"0ec19459f242ce4cd865d15e9983f2e0a775f18244f99aaf76759ceb5972b29d"}
Sep 09 03:30:00.238939 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:30:00.238563    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-b-4bd6b8-6458c89d79-jt8fk" event={"ID":"766d00dc-b659-4682-b44b-4e42a6c4e844","Type":"ContainerStarted","Data":"eeb93b70d0206356698014d6a6b2384014065b304e3b92e05d86eff9a69ce1a3"}
Sep 09 03:30:00.238939 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:30:00.238592    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-inf-b-4bd6b8-6458c89d79-jt8fk"
Sep 09 03:30:00.240161 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:30:00.240136    2576 generic.go:356] "Generic (PLEG): container finished" podID="ead9312f-d248-4d9b-aee4-9706a8e2f478" containerID="fcb132ca92af3fd30bf19bd57cd18033767b0d95bbac801b9317f9a9bb725472" exitCode=0
Sep 09 03:30:00.240272 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:30:00.240188    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-9c982818-spb9z" event={"ID":"ead9312f-d248-4d9b-aee4-9706a8e2f478","Type":"ContainerDied","Data":"fcb132ca92af3fd30bf19bd57cd18033767b0d95bbac801b9317f9a9bb725472"}
Sep 09 03:30:00.258660 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:30:00.258605    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-inf-b-4bd6b8-6458c89d79-jt8fk" podStartSLOduration=3.258588704 podStartE2EDuration="3.258588704s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 03:29:57 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 03:30:00.257675184 +0000 UTC m=+1014.399161263" watchObservedRunningTime="2026-09-09 03:30:00.258588704 +0000 UTC m=+1014.400074781"
Sep 09 03:30:00.436560 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:30:00.436537    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ait-9c982818-29815410-kb9f4"]
Sep 09 03:30:00.438412 ip-10-0-132-14 kubenswrapper[2576]: W0909 03:30:00.438383    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7f2ffd18_66ef_40ca_a37e_0d55931f79ea.slice/crio-2e58e6171ca6261136cf0ff4c5cdce0cb55a7319961e104ac08936d4ab913124 WatchSource:0}: Error finding container 2e58e6171ca6261136cf0ff4c5cdce0cb55a7319961e104ac08936d4ab913124: Status 404 returned error can't find the container with id 2e58e6171ca6261136cf0ff4c5cdce0cb55a7319961e104ac08936d4ab913124
Sep 09 03:30:01.246706 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:30:01.246675    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ait-9c982818-29815410-kb9f4" event={"ID":"7f2ffd18-66ef-40ca-a37e-0d55931f79ea","Type":"ContainerStarted","Data":"b58704fd7565a402683bde797d506e50cd49e9b9171ab7f17b98f1959ef36e10"}
Sep 09 03:30:01.247099 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:30:01.246735    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ait-9c982818-29815410-kb9f4" event={"ID":"7f2ffd18-66ef-40ca-a37e-0d55931f79ea","Type":"ContainerStarted","Data":"2e58e6171ca6261136cf0ff4c5cdce0cb55a7319961e104ac08936d4ab913124"}
Sep 09 03:30:01.379041 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:30:01.379015    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-9c982818-spb9z"
Sep 09 03:30:01.417867 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:30:01.417843    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/ead9312f-d248-4d9b-aee4-9706a8e2f478-maas-api-service-ca\") pod \"ead9312f-d248-4d9b-aee4-9706a8e2f478\" (UID: \"ead9312f-d248-4d9b-aee4-9706a8e2f478\") "
Sep 09 03:30:01.418193 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:30:01.418170    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ead9312f-d248-4d9b-aee4-9706a8e2f478-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "ead9312f-d248-4d9b-aee4-9706a8e2f478" (UID: "ead9312f-d248-4d9b-aee4-9706a8e2f478"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 09 03:30:01.518635 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:30:01.518603    2576 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/ead9312f-d248-4d9b-aee4-9706a8e2f478-maas-api-service-ca\") on node \"ip-10-0-132-14.ec2.internal\" DevicePath \"\""
Sep 09 03:30:02.252633 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:30:02.252600    2576 generic.go:356] "Generic (PLEG): container finished" podID="7f2ffd18-66ef-40ca-a37e-0d55931f79ea" containerID="b58704fd7565a402683bde797d506e50cd49e9b9171ab7f17b98f1959ef36e10" exitCode=0
Sep 09 03:30:02.253146 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:30:02.252684    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ait-9c982818-29815410-kb9f4" event={"ID":"7f2ffd18-66ef-40ca-a37e-0d55931f79ea","Type":"ContainerDied","Data":"b58704fd7565a402683bde797d506e50cd49e9b9171ab7f17b98f1959ef36e10"}
Sep 09 03:30:02.254126 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:30:02.254103    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-9c982818-spb9z" event={"ID":"ead9312f-d248-4d9b-aee4-9706a8e2f478","Type":"ContainerDied","Data":"10614351edc9c4874aec65297110a7937e6e30ad31ed73ce2a65f807847586bc"}
Sep 09 03:30:02.254197 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:30:02.254153    2576 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="10614351edc9c4874aec65297110a7937e6e30ad31ed73ce2a65f807847586bc"
Sep 09 03:30:02.254233 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:30:02.254219    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-9c982818-spb9z"
Sep 09 03:30:02.387507 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:30:02.387487    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ait-9c982818-29815410-kb9f4"
Sep 09 03:30:03.260139 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:30:03.260106    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ait-9c982818-29815410-kb9f4"
Sep 09 03:30:03.260139 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:30:03.260133    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ait-9c982818-29815410-kb9f4" event={"ID":"7f2ffd18-66ef-40ca-a37e-0d55931f79ea","Type":"ContainerDied","Data":"2e58e6171ca6261136cf0ff4c5cdce0cb55a7319961e104ac08936d4ab913124"}
Sep 09 03:30:03.260585 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:30:03.260208    2576 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="2e58e6171ca6261136cf0ff4c5cdce0cb55a7319961e104ac08936d4ab913124"
Sep 09 03:30:09.233950 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:30:09.233917    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-4bd6b8-6f98d797d-h8b2x"
Sep 09 03:30:11.248547 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:30:11.248517    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-inf-b-4bd6b8-6458c89d79-jt8fk"
Sep 09 03:30:12.280682 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:30:12.280643    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ait-9c982818-29815410-kb9f4"]
Sep 09 03:30:12.282913 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:30:12.282884    2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ait-9c982818-29815410-kb9f4"]
Sep 09 03:30:12.323659 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:30:12.323627    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-9c982818-8487d9d978-2smf9"]
Sep 09 03:30:12.323989 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:30:12.323930    2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-ait-9c982818-8487d9d978-2smf9" podUID="64b0e07d-79c1-48b3-ae6f-f7236fcc6e47" containerName="payload-pre-processing" containerID="cri-o://2e39d96c2ea111b14f2802e85d4ffe46fe530b101f3a47293b1a8e9d59ac598e" gracePeriod=30
Sep 09 03:30:12.361189 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:30:12.361163    2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7f2ffd18-66ef-40ca-a37e-0d55931f79ea" path="/var/lib/kubelet/pods/7f2ffd18-66ef-40ca-a37e-0d55931f79ea/volumes"
Sep 09 03:30:12.494418 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:30:12.494387    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-9c982818-spb9z"]
Sep 09 03:30:12.500643 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:30:12.500618    2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-9c982818-spb9z"]
Sep 09 03:30:12.569845 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:30:12.569795    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-9c982818-8487d9d978-2smf9"
Sep 09 03:30:12.602934 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:30:12.602881    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-r8d9j\" (UniqueName: \"kubernetes.io/projected/64b0e07d-79c1-48b3-ae6f-f7236fcc6e47-kube-api-access-r8d9j\") pod \"64b0e07d-79c1-48b3-ae6f-f7236fcc6e47\" (UID: \"64b0e07d-79c1-48b3-ae6f-f7236fcc6e47\") "
Sep 09 03:30:12.602934 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:30:12.602937    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/64b0e07d-79c1-48b3-ae6f-f7236fcc6e47-plugins-config-volume\") pod \"64b0e07d-79c1-48b3-ae6f-f7236fcc6e47\" (UID: \"64b0e07d-79c1-48b3-ae6f-f7236fcc6e47\") "
Sep 09 03:30:12.603633 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:30:12.603595    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/64b0e07d-79c1-48b3-ae6f-f7236fcc6e47-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "64b0e07d-79c1-48b3-ae6f-f7236fcc6e47" (UID: "64b0e07d-79c1-48b3-ae6f-f7236fcc6e47"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 09 03:30:12.605986 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:30:12.605947    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/64b0e07d-79c1-48b3-ae6f-f7236fcc6e47-kube-api-access-r8d9j" (OuterVolumeSpecName: "kube-api-access-r8d9j") pod "64b0e07d-79c1-48b3-ae6f-f7236fcc6e47" (UID: "64b0e07d-79c1-48b3-ae6f-f7236fcc6e47"). InnerVolumeSpecName "kube-api-access-r8d9j". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 09 03:30:12.703890 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:30:12.703854    2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-r8d9j\" (UniqueName: \"kubernetes.io/projected/64b0e07d-79c1-48b3-ae6f-f7236fcc6e47-kube-api-access-r8d9j\") on node \"ip-10-0-132-14.ec2.internal\" DevicePath \"\""
Sep 09 03:30:12.703890 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:30:12.703882    2576 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/64b0e07d-79c1-48b3-ae6f-f7236fcc6e47-plugins-config-volume\") on node \"ip-10-0-132-14.ec2.internal\" DevicePath \"\""
Sep 09 03:30:13.306037 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:30:13.306003    2576 generic.go:356] "Generic (PLEG): container finished" podID="64b0e07d-79c1-48b3-ae6f-f7236fcc6e47" containerID="2e39d96c2ea111b14f2802e85d4ffe46fe530b101f3a47293b1a8e9d59ac598e" exitCode=0
Sep 09 03:30:13.306492 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:30:13.306055    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-9c982818-8487d9d978-2smf9"
Sep 09 03:30:13.306492 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:30:13.306069    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-9c982818-8487d9d978-2smf9" event={"ID":"64b0e07d-79c1-48b3-ae6f-f7236fcc6e47","Type":"ContainerDied","Data":"2e39d96c2ea111b14f2802e85d4ffe46fe530b101f3a47293b1a8e9d59ac598e"}
Sep 09 03:30:13.306492 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:30:13.306111    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-9c982818-8487d9d978-2smf9" event={"ID":"64b0e07d-79c1-48b3-ae6f-f7236fcc6e47","Type":"ContainerDied","Data":"74fe19f70330468a56c8519000b9adb59de26cb5114ec98742c3ec8667f21279"}
Sep 09 03:30:13.306492 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:30:13.306133    2576 scope.go:117] "RemoveContainer" containerID="2e39d96c2ea111b14f2802e85d4ffe46fe530b101f3a47293b1a8e9d59ac598e"
Sep 09 03:30:13.314972 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:30:13.314955    2576 scope.go:117] "RemoveContainer" containerID="2e39d96c2ea111b14f2802e85d4ffe46fe530b101f3a47293b1a8e9d59ac598e"
Sep 09 03:30:13.315221 ip-10-0-132-14 kubenswrapper[2576]: E0909 03:30:13.315203    2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2e39d96c2ea111b14f2802e85d4ffe46fe530b101f3a47293b1a8e9d59ac598e\": container with ID starting with 2e39d96c2ea111b14f2802e85d4ffe46fe530b101f3a47293b1a8e9d59ac598e not found: ID does not exist" containerID="2e39d96c2ea111b14f2802e85d4ffe46fe530b101f3a47293b1a8e9d59ac598e"
Sep 09 03:30:13.315282 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:30:13.315227    2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2e39d96c2ea111b14f2802e85d4ffe46fe530b101f3a47293b1a8e9d59ac598e"} err="failed to get container status \"2e39d96c2ea111b14f2802e85d4ffe46fe530b101f3a47293b1a8e9d59ac598e\": rpc error: code = NotFound desc = could not find container \"2e39d96c2ea111b14f2802e85d4ffe46fe530b101f3a47293b1a8e9d59ac598e\": container with ID starting with 2e39d96c2ea111b14f2802e85d4ffe46fe530b101f3a47293b1a8e9d59ac598e not found: ID does not exist"
Sep 09 03:30:13.328538 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:30:13.328512    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-9c982818-8487d9d978-2smf9"]
Sep 09 03:30:13.332575 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:30:13.332557    2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-9c982818-8487d9d978-2smf9"]
Sep 09 03:30:14.360696 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:30:14.360658    2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="64b0e07d-79c1-48b3-ae6f-f7236fcc6e47" path="/var/lib/kubelet/pods/64b0e07d-79c1-48b3-ae6f-f7236fcc6e47/volumes"
Sep 09 03:30:14.361222 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:30:14.361199    2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ead9312f-d248-4d9b-aee4-9706a8e2f478" path="/var/lib/kubelet/pods/ead9312f-d248-4d9b-aee4-9706a8e2f478/volumes"
Sep 09 03:30:17.680460 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:30:17.680427    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-migrate-5f9aae59-7cd98b4b9d-ctpvg"]
Sep 09 03:30:17.680832 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:30:17.680785    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="64b0e07d-79c1-48b3-ae6f-f7236fcc6e47" containerName="payload-pre-processing"
Sep 09 03:30:17.680832 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:30:17.680796    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="64b0e07d-79c1-48b3-ae6f-f7236fcc6e47" containerName="payload-pre-processing"
Sep 09 03:30:17.680832 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:30:17.680831    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7f2ffd18-66ef-40ca-a37e-0d55931f79ea" containerName="cleanup"
Sep 09 03:30:17.680946 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:30:17.680837    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="7f2ffd18-66ef-40ca-a37e-0d55931f79ea" containerName="cleanup"
Sep 09 03:30:17.680946 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:30:17.680846    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ead9312f-d248-4d9b-aee4-9706a8e2f478" containerName="revoke-keys"
Sep 09 03:30:17.680946 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:30:17.680851    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="ead9312f-d248-4d9b-aee4-9706a8e2f478" containerName="revoke-keys"
Sep 09 03:30:17.680946 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:30:17.680907    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="64b0e07d-79c1-48b3-ae6f-f7236fcc6e47" containerName="payload-pre-processing"
Sep 09 03:30:17.680946 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:30:17.680914    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="ead9312f-d248-4d9b-aee4-9706a8e2f478" containerName="revoke-keys"
Sep 09 03:30:17.680946 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:30:17.680921    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="7f2ffd18-66ef-40ca-a37e-0d55931f79ea" containerName="cleanup"
Sep 09 03:30:17.683951 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:30:17.683934    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-migrate-5f9aae59-7cd98b4b9d-ctpvg"
Sep 09 03:30:17.686951 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:30:17.686930    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-migrate-5f9aae59-dockercfg-mjm6l\""
Sep 09 03:30:17.687063 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:30:17.686958    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-migrate-5f9aae59\""
Sep 09 03:30:17.703739 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:30:17.703710    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-migrate-5f9aae59-7cd98b4b9d-ctpvg"]
Sep 09 03:30:17.744323 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:30:17.744289    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-d2lwc\" (UniqueName: \"kubernetes.io/projected/6fb03975-e94b-4887-91ac-7e680abaddd7-kube-api-access-d2lwc\") pod \"payload-pre-processing-e2e-migrate-5f9aae59-7cd98b4b9d-ctpvg\" (UID: \"6fb03975-e94b-4887-91ac-7e680abaddd7\") " pod="openshift-ingress/payload-pre-processing-e2e-migrate-5f9aae59-7cd98b4b9d-ctpvg"
Sep 09 03:30:17.744480 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:30:17.744331    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/6fb03975-e94b-4887-91ac-7e680abaddd7-plugins-config-volume\") pod \"payload-pre-processing-e2e-migrate-5f9aae59-7cd98b4b9d-ctpvg\" (UID: \"6fb03975-e94b-4887-91ac-7e680abaddd7\") " pod="openshift-ingress/payload-pre-processing-e2e-migrate-5f9aae59-7cd98b4b9d-ctpvg"
Sep 09 03:30:17.752122 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:30:17.752097    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-migrate-5f9aae59-75c57dc66-snrpb"]
Sep 09 03:30:17.755234 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:30:17.755213    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-migrate-5f9aae59-75c57dc66-snrpb"
Sep 09 03:30:17.772728 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:30:17.772704    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-migrate-5f9aae59-75c57dc66-snrpb"]
Sep 09 03:30:17.845637 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:30:17.845607    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-s4wz7\" (UniqueName: \"kubernetes.io/projected/93f78cdf-c11c-4106-bf64-b940788c7b16-kube-api-access-s4wz7\") pod \"payload-processing-e2e-migrate-5f9aae59-75c57dc66-snrpb\" (UID: \"93f78cdf-c11c-4106-bf64-b940788c7b16\") " pod="openshift-ingress/payload-processing-e2e-migrate-5f9aae59-75c57dc66-snrpb"
Sep 09 03:30:17.845796 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:30:17.845658    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-d2lwc\" (UniqueName: \"kubernetes.io/projected/6fb03975-e94b-4887-91ac-7e680abaddd7-kube-api-access-d2lwc\") pod \"payload-pre-processing-e2e-migrate-5f9aae59-7cd98b4b9d-ctpvg\" (UID: \"6fb03975-e94b-4887-91ac-7e680abaddd7\") " pod="openshift-ingress/payload-pre-processing-e2e-migrate-5f9aae59-7cd98b4b9d-ctpvg"
Sep 09 03:30:17.845796 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:30:17.845680    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/93f78cdf-c11c-4106-bf64-b940788c7b16-plugins-config-volume\") pod \"payload-processing-e2e-migrate-5f9aae59-75c57dc66-snrpb\" (UID: \"93f78cdf-c11c-4106-bf64-b940788c7b16\") " pod="openshift-ingress/payload-processing-e2e-migrate-5f9aae59-75c57dc66-snrpb"
Sep 09 03:30:17.845796 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:30:17.845718    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/6fb03975-e94b-4887-91ac-7e680abaddd7-plugins-config-volume\") pod \"payload-pre-processing-e2e-migrate-5f9aae59-7cd98b4b9d-ctpvg\" (UID: \"6fb03975-e94b-4887-91ac-7e680abaddd7\") " pod="openshift-ingress/payload-pre-processing-e2e-migrate-5f9aae59-7cd98b4b9d-ctpvg"
Sep 09 03:30:17.846423 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:30:17.846405    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/6fb03975-e94b-4887-91ac-7e680abaddd7-plugins-config-volume\") pod \"payload-pre-processing-e2e-migrate-5f9aae59-7cd98b4b9d-ctpvg\" (UID: \"6fb03975-e94b-4887-91ac-7e680abaddd7\") " pod="openshift-ingress/payload-pre-processing-e2e-migrate-5f9aae59-7cd98b4b9d-ctpvg"
Sep 09 03:30:17.856577 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:30:17.856549    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-d2lwc\" (UniqueName: \"kubernetes.io/projected/6fb03975-e94b-4887-91ac-7e680abaddd7-kube-api-access-d2lwc\") pod \"payload-pre-processing-e2e-migrate-5f9aae59-7cd98b4b9d-ctpvg\" (UID: \"6fb03975-e94b-4887-91ac-7e680abaddd7\") " pod="openshift-ingress/payload-pre-processing-e2e-migrate-5f9aae59-7cd98b4b9d-ctpvg"
Sep 09 03:30:17.946498 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:30:17.946424    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/93f78cdf-c11c-4106-bf64-b940788c7b16-plugins-config-volume\") pod \"payload-processing-e2e-migrate-5f9aae59-75c57dc66-snrpb\" (UID: \"93f78cdf-c11c-4106-bf64-b940788c7b16\") " pod="openshift-ingress/payload-processing-e2e-migrate-5f9aae59-75c57dc66-snrpb"
Sep 09 03:30:17.946643 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:30:17.946539    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-s4wz7\" (UniqueName: \"kubernetes.io/projected/93f78cdf-c11c-4106-bf64-b940788c7b16-kube-api-access-s4wz7\") pod \"payload-processing-e2e-migrate-5f9aae59-75c57dc66-snrpb\" (UID: \"93f78cdf-c11c-4106-bf64-b940788c7b16\") " pod="openshift-ingress/payload-processing-e2e-migrate-5f9aae59-75c57dc66-snrpb"
Sep 09 03:30:17.947234 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:30:17.947212    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/93f78cdf-c11c-4106-bf64-b940788c7b16-plugins-config-volume\") pod \"payload-processing-e2e-migrate-5f9aae59-75c57dc66-snrpb\" (UID: \"93f78cdf-c11c-4106-bf64-b940788c7b16\") " pod="openshift-ingress/payload-processing-e2e-migrate-5f9aae59-75c57dc66-snrpb"
Sep 09 03:30:17.966230 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:30:17.966205    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-s4wz7\" (UniqueName: \"kubernetes.io/projected/93f78cdf-c11c-4106-bf64-b940788c7b16-kube-api-access-s4wz7\") pod \"payload-processing-e2e-migrate-5f9aae59-75c57dc66-snrpb\" (UID: \"93f78cdf-c11c-4106-bf64-b940788c7b16\") " pod="openshift-ingress/payload-processing-e2e-migrate-5f9aae59-75c57dc66-snrpb"
Sep 09 03:30:17.993224 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:30:17.993194    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-migrate-5f9aae59-7cd98b4b9d-ctpvg"
Sep 09 03:30:18.068267 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:30:18.068237    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-migrate-5f9aae59-75c57dc66-snrpb"
Sep 09 03:30:18.299332 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:30:18.299308    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-migrate-5f9aae59-7cd98b4b9d-ctpvg"]
Sep 09 03:30:18.301887 ip-10-0-132-14 kubenswrapper[2576]: W0909 03:30:18.301851    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6fb03975_e94b_4887_91ac_7e680abaddd7.slice/crio-671152cd988846425cac84e4bbe995d038b20f70a5341f44ccf9bd8174163d12 WatchSource:0}: Error finding container 671152cd988846425cac84e4bbe995d038b20f70a5341f44ccf9bd8174163d12: Status 404 returned error can't find the container with id 671152cd988846425cac84e4bbe995d038b20f70a5341f44ccf9bd8174163d12
Sep 09 03:30:18.331364 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:30:18.331337    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-migrate-5f9aae59-7cd98b4b9d-ctpvg" event={"ID":"6fb03975-e94b-4887-91ac-7e680abaddd7","Type":"ContainerStarted","Data":"671152cd988846425cac84e4bbe995d038b20f70a5341f44ccf9bd8174163d12"}
Sep 09 03:30:18.356290 ip-10-0-132-14 kubenswrapper[2576]: W0909 03:30:18.356257    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod93f78cdf_c11c_4106_bf64_b940788c7b16.slice/crio-6c4b4b832a4bca2eb4e811a2ad91af2bc605dcabd5bc989b8ff70afa0cfbac21 WatchSource:0}: Error finding container 6c4b4b832a4bca2eb4e811a2ad91af2bc605dcabd5bc989b8ff70afa0cfbac21: Status 404 returned error can't find the container with id 6c4b4b832a4bca2eb4e811a2ad91af2bc605dcabd5bc989b8ff70afa0cfbac21
Sep 09 03:30:18.360793 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:30:18.360760    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-migrate-5f9aae59-75c57dc66-snrpb"]
Sep 09 03:30:19.338215 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:30:19.338175    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-migrate-5f9aae59-7cd98b4b9d-ctpvg" event={"ID":"6fb03975-e94b-4887-91ac-7e680abaddd7","Type":"ContainerStarted","Data":"02008d0cdacf556d551384be6d0c67600d3994eba7b49b46638b053133e55b9a"}
Sep 09 03:30:19.338697 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:30:19.338285    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-migrate-5f9aae59-7cd98b4b9d-ctpvg"
Sep 09 03:30:19.339959 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:30:19.339930    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-migrate-5f9aae59-75c57dc66-snrpb" event={"ID":"93f78cdf-c11c-4106-bf64-b940788c7b16","Type":"ContainerStarted","Data":"7be1d0d93ff5a3f07a5ae937795c4e5f18a899d7588a9f4446bcb7ccc51c3c6d"}
Sep 09 03:30:19.340083 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:30:19.339981    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-migrate-5f9aae59-75c57dc66-snrpb" event={"ID":"93f78cdf-c11c-4106-bf64-b940788c7b16","Type":"ContainerStarted","Data":"6c4b4b832a4bca2eb4e811a2ad91af2bc605dcabd5bc989b8ff70afa0cfbac21"}
Sep 09 03:30:19.340083 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:30:19.340003    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-migrate-5f9aae59-75c57dc66-snrpb"
Sep 09 03:30:19.360447 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:30:19.360387    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-migrate-5f9aae59-7cd98b4b9d-ctpvg" podStartSLOduration=2.360368406 podStartE2EDuration="2.360368406s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 03:30:17 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 03:30:19.358767028 +0000 UTC m=+1033.500253109" watchObservedRunningTime="2026-09-09 03:30:19.360368406 +0000 UTC m=+1033.501854485"
Sep 09 03:30:19.379355 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:30:19.379306    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-migrate-5f9aae59-75c57dc66-snrpb" podStartSLOduration=2.379293983 podStartE2EDuration="2.379293983s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 03:30:17 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 03:30:19.377986839 +0000 UTC m=+1033.519472914" watchObservedRunningTime="2026-09-09 03:30:19.379293983 +0000 UTC m=+1033.520780063"
Sep 09 03:30:30.347525 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:30:30.347495    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-migrate-5f9aae59-75c57dc66-snrpb"
Sep 09 03:30:30.347971 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:30:30.347857    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-migrate-5f9aae59-7cd98b4b9d-ctpvg"
Sep 09 03:30:41.516470 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:30:41.516430    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-5f9aae59-f9n9d"]
Sep 09 03:30:41.520080 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:30:41.520062    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-5f9aae59-f9n9d"
Sep 09 03:30:41.522824 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:30:41.522770    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-w7q9f\""
Sep 09 03:30:41.528355 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:30:41.528326    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-5f9aae59-f9n9d"]
Sep 09 03:30:41.646228 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:30:41.646193    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/6b9078de-fb76-4f52-9060-85db83dd8888-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-5f9aae59-f9n9d\" (UID: \"6b9078de-fb76-4f52-9060-85db83dd8888\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-5f9aae59-f9n9d"
Sep 09 03:30:41.746878 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:30:41.746842    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/6b9078de-fb76-4f52-9060-85db83dd8888-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-5f9aae59-f9n9d\" (UID: \"6b9078de-fb76-4f52-9060-85db83dd8888\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-5f9aae59-f9n9d"
Sep 09 03:30:41.747412 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:30:41.747394    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/6b9078de-fb76-4f52-9060-85db83dd8888-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-5f9aae59-f9n9d\" (UID: \"6b9078de-fb76-4f52-9060-85db83dd8888\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-5f9aae59-f9n9d"
Sep 09 03:30:41.833002 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:30:41.832917    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-5f9aae59-f9n9d"
Sep 09 03:30:42.104642 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:30:42.104619    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-5f9aae59-f9n9d"]
Sep 09 03:30:42.106333 ip-10-0-132-14 kubenswrapper[2576]: W0909 03:30:42.106305    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6b9078de_fb76_4f52_9060_85db83dd8888.slice/crio-ea299d57f3859ca5f4f4b846de26659224da386326dabbe784f294ef72535354 WatchSource:0}: Error finding container ea299d57f3859ca5f4f4b846de26659224da386326dabbe784f294ef72535354: Status 404 returned error can't find the container with id ea299d57f3859ca5f4f4b846de26659224da386326dabbe784f294ef72535354
Sep 09 03:30:42.453329 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:30:42.453284    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-5f9aae59-f9n9d" event={"ID":"6b9078de-fb76-4f52-9060-85db83dd8888","Type":"ContainerStarted","Data":"8e49b3437f0552d5e9efedb83a853777c8f8964bb5545c0a02d5089e21179855"}
Sep 09 03:30:42.453600 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:30:42.453346    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-5f9aae59-f9n9d" event={"ID":"6b9078de-fb76-4f52-9060-85db83dd8888","Type":"ContainerStarted","Data":"ea299d57f3859ca5f4f4b846de26659224da386326dabbe784f294ef72535354"}
Sep 09 03:30:42.469773 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:30:42.469467    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-5f9aae59-f9n9d" podStartSLOduration=1.469449666 podStartE2EDuration="1.469449666s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 03:30:41 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 03:30:42.468790843 +0000 UTC m=+1056.610276921" watchObservedRunningTime="2026-09-09 03:30:42.469449666 +0000 UTC m=+1056.610935742"
Sep 09 03:30:43.459092 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:30:43.459060    2576 generic.go:356] "Generic (PLEG): container finished" podID="6b9078de-fb76-4f52-9060-85db83dd8888" containerID="8e49b3437f0552d5e9efedb83a853777c8f8964bb5545c0a02d5089e21179855" exitCode=0
Sep 09 03:30:43.459460 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:30:43.459114    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-5f9aae59-f9n9d" event={"ID":"6b9078de-fb76-4f52-9060-85db83dd8888","Type":"ContainerDied","Data":"8e49b3437f0552d5e9efedb83a853777c8f8964bb5545c0a02d5089e21179855"}
Sep 09 03:30:44.596962 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:30:44.596939    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-5f9aae59-f9n9d"
Sep 09 03:30:44.675669 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:30:44.675636    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/6b9078de-fb76-4f52-9060-85db83dd8888-maas-api-service-ca\") pod \"6b9078de-fb76-4f52-9060-85db83dd8888\" (UID: \"6b9078de-fb76-4f52-9060-85db83dd8888\") "
Sep 09 03:30:44.676037 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:30:44.676013    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6b9078de-fb76-4f52-9060-85db83dd8888-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "6b9078de-fb76-4f52-9060-85db83dd8888" (UID: "6b9078de-fb76-4f52-9060-85db83dd8888"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 09 03:30:44.776860 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:30:44.776831    2576 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/6b9078de-fb76-4f52-9060-85db83dd8888-maas-api-service-ca\") on node \"ip-10-0-132-14.ec2.internal\" DevicePath \"\""
Sep 09 03:30:45.470272 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:30:45.470238    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-5f9aae59-f9n9d"
Sep 09 03:30:45.470436 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:30:45.470238    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-5f9aae59-f9n9d" event={"ID":"6b9078de-fb76-4f52-9060-85db83dd8888","Type":"ContainerDied","Data":"ea299d57f3859ca5f4f4b846de26659224da386326dabbe784f294ef72535354"}
Sep 09 03:30:45.470436 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:30:45.470362    2576 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="ea299d57f3859ca5f4f4b846de26659224da386326dabbe784f294ef72535354"
Sep 09 03:30:48.113795 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:30:48.113758    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-migrate-5f9aae59-75c57dc66-snrpb"]
Sep 09 03:30:48.114205 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:30:48.114043    2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-migrate-5f9aae59-75c57dc66-snrpb" podUID="93f78cdf-c11c-4106-bf64-b940788c7b16" containerName="payload-processing" containerID="cri-o://7be1d0d93ff5a3f07a5ae937795c4e5f18a899d7588a9f4446bcb7ccc51c3c6d" gracePeriod=30
Sep 09 03:30:48.119566 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:30:48.119537    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-migrate-5f9aae59-7cd98b4b9d-ctpvg"]
Sep 09 03:30:48.119817 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:30:48.119777    2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-migrate-5f9aae59-7cd98b4b9d-ctpvg" podUID="6fb03975-e94b-4887-91ac-7e680abaddd7" containerName="payload-pre-processing" containerID="cri-o://02008d0cdacf556d551384be6d0c67600d3994eba7b49b46638b053133e55b9a" gracePeriod=30
Sep 09 03:30:48.343249 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:30:48.343186    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-5f9aae59-f9n9d"]
Sep 09 03:30:48.346542 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:30:48.346510    2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-5f9aae59-f9n9d"]
Sep 09 03:30:48.361641 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:30:48.361612    2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6b9078de-fb76-4f52-9060-85db83dd8888" path="/var/lib/kubelet/pods/6b9078de-fb76-4f52-9060-85db83dd8888/volumes"
Sep 09 03:30:48.398272 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:30:48.398243    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-migrate-5f9aae59-75c57dc66-snrpb"
Sep 09 03:30:48.400930 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:30:48.400912    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-migrate-5f9aae59-7cd98b4b9d-ctpvg"
Sep 09 03:30:48.486427 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:30:48.486391    2576 generic.go:356] "Generic (PLEG): container finished" podID="6fb03975-e94b-4887-91ac-7e680abaddd7" containerID="02008d0cdacf556d551384be6d0c67600d3994eba7b49b46638b053133e55b9a" exitCode=0
Sep 09 03:30:48.486604 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:30:48.486426    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-migrate-5f9aae59-7cd98b4b9d-ctpvg" event={"ID":"6fb03975-e94b-4887-91ac-7e680abaddd7","Type":"ContainerDied","Data":"02008d0cdacf556d551384be6d0c67600d3994eba7b49b46638b053133e55b9a"}
Sep 09 03:30:48.486604 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:30:48.486439    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-migrate-5f9aae59-7cd98b4b9d-ctpvg"
Sep 09 03:30:48.486604 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:30:48.486479    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-migrate-5f9aae59-7cd98b4b9d-ctpvg" event={"ID":"6fb03975-e94b-4887-91ac-7e680abaddd7","Type":"ContainerDied","Data":"671152cd988846425cac84e4bbe995d038b20f70a5341f44ccf9bd8174163d12"}
Sep 09 03:30:48.486604 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:30:48.486500    2576 scope.go:117] "RemoveContainer" containerID="02008d0cdacf556d551384be6d0c67600d3994eba7b49b46638b053133e55b9a"
Sep 09 03:30:48.488029 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:30:48.488008    2576 generic.go:356] "Generic (PLEG): container finished" podID="93f78cdf-c11c-4106-bf64-b940788c7b16" containerID="7be1d0d93ff5a3f07a5ae937795c4e5f18a899d7588a9f4446bcb7ccc51c3c6d" exitCode=0
Sep 09 03:30:48.488317 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:30:48.488050    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-migrate-5f9aae59-75c57dc66-snrpb"
Sep 09 03:30:48.488317 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:30:48.488067    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-migrate-5f9aae59-75c57dc66-snrpb" event={"ID":"93f78cdf-c11c-4106-bf64-b940788c7b16","Type":"ContainerDied","Data":"7be1d0d93ff5a3f07a5ae937795c4e5f18a899d7588a9f4446bcb7ccc51c3c6d"}
Sep 09 03:30:48.488317 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:30:48.488102    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-migrate-5f9aae59-75c57dc66-snrpb" event={"ID":"93f78cdf-c11c-4106-bf64-b940788c7b16","Type":"ContainerDied","Data":"6c4b4b832a4bca2eb4e811a2ad91af2bc605dcabd5bc989b8ff70afa0cfbac21"}
Sep 09 03:30:48.495489 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:30:48.495472    2576 scope.go:117] "RemoveContainer" containerID="02008d0cdacf556d551384be6d0c67600d3994eba7b49b46638b053133e55b9a"
Sep 09 03:30:48.495758 ip-10-0-132-14 kubenswrapper[2576]: E0909 03:30:48.495739    2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"02008d0cdacf556d551384be6d0c67600d3994eba7b49b46638b053133e55b9a\": container with ID starting with 02008d0cdacf556d551384be6d0c67600d3994eba7b49b46638b053133e55b9a not found: ID does not exist" containerID="02008d0cdacf556d551384be6d0c67600d3994eba7b49b46638b053133e55b9a"
Sep 09 03:30:48.495831 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:30:48.495766    2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"02008d0cdacf556d551384be6d0c67600d3994eba7b49b46638b053133e55b9a"} err="failed to get container status \"02008d0cdacf556d551384be6d0c67600d3994eba7b49b46638b053133e55b9a\": rpc error: code = NotFound desc = could not find container \"02008d0cdacf556d551384be6d0c67600d3994eba7b49b46638b053133e55b9a\": container with ID starting with 02008d0cdacf556d551384be6d0c67600d3994eba7b49b46638b053133e55b9a not found: ID does not exist"
Sep 09 03:30:48.495831 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:30:48.495780    2576 scope.go:117] "RemoveContainer" containerID="7be1d0d93ff5a3f07a5ae937795c4e5f18a899d7588a9f4446bcb7ccc51c3c6d"
Sep 09 03:30:48.503523 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:30:48.503508    2576 scope.go:117] "RemoveContainer" containerID="7be1d0d93ff5a3f07a5ae937795c4e5f18a899d7588a9f4446bcb7ccc51c3c6d"
Sep 09 03:30:48.503793 ip-10-0-132-14 kubenswrapper[2576]: E0909 03:30:48.503769    2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7be1d0d93ff5a3f07a5ae937795c4e5f18a899d7588a9f4446bcb7ccc51c3c6d\": container with ID starting with 7be1d0d93ff5a3f07a5ae937795c4e5f18a899d7588a9f4446bcb7ccc51c3c6d not found: ID does not exist" containerID="7be1d0d93ff5a3f07a5ae937795c4e5f18a899d7588a9f4446bcb7ccc51c3c6d"
Sep 09 03:30:48.503881 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:30:48.503821    2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7be1d0d93ff5a3f07a5ae937795c4e5f18a899d7588a9f4446bcb7ccc51c3c6d"} err="failed to get container status \"7be1d0d93ff5a3f07a5ae937795c4e5f18a899d7588a9f4446bcb7ccc51c3c6d\": rpc error: code = NotFound desc = could not find container \"7be1d0d93ff5a3f07a5ae937795c4e5f18a899d7588a9f4446bcb7ccc51c3c6d\": container with ID starting with 7be1d0d93ff5a3f07a5ae937795c4e5f18a899d7588a9f4446bcb7ccc51c3c6d not found: ID does not exist"
Sep 09 03:30:48.506100 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:30:48.506072    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-d2lwc\" (UniqueName: \"kubernetes.io/projected/6fb03975-e94b-4887-91ac-7e680abaddd7-kube-api-access-d2lwc\") pod \"6fb03975-e94b-4887-91ac-7e680abaddd7\" (UID: \"6fb03975-e94b-4887-91ac-7e680abaddd7\") "
Sep 09 03:30:48.506183 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:30:48.506115    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/6fb03975-e94b-4887-91ac-7e680abaddd7-plugins-config-volume\") pod \"6fb03975-e94b-4887-91ac-7e680abaddd7\" (UID: \"6fb03975-e94b-4887-91ac-7e680abaddd7\") "
Sep 09 03:30:48.506183 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:30:48.506154    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/93f78cdf-c11c-4106-bf64-b940788c7b16-plugins-config-volume\") pod \"93f78cdf-c11c-4106-bf64-b940788c7b16\" (UID: \"93f78cdf-c11c-4106-bf64-b940788c7b16\") "
Sep 09 03:30:48.506287 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:30:48.506214    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-s4wz7\" (UniqueName: \"kubernetes.io/projected/93f78cdf-c11c-4106-bf64-b940788c7b16-kube-api-access-s4wz7\") pod \"93f78cdf-c11c-4106-bf64-b940788c7b16\" (UID: \"93f78cdf-c11c-4106-bf64-b940788c7b16\") "
Sep 09 03:30:48.506491 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:30:48.506466    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6fb03975-e94b-4887-91ac-7e680abaddd7-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "6fb03975-e94b-4887-91ac-7e680abaddd7" (UID: "6fb03975-e94b-4887-91ac-7e680abaddd7"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 09 03:30:48.506625 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:30:48.506542    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/93f78cdf-c11c-4106-bf64-b940788c7b16-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "93f78cdf-c11c-4106-bf64-b940788c7b16" (UID: "93f78cdf-c11c-4106-bf64-b940788c7b16"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 09 03:30:48.508179 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:30:48.508160    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6fb03975-e94b-4887-91ac-7e680abaddd7-kube-api-access-d2lwc" (OuterVolumeSpecName: "kube-api-access-d2lwc") pod "6fb03975-e94b-4887-91ac-7e680abaddd7" (UID: "6fb03975-e94b-4887-91ac-7e680abaddd7"). InnerVolumeSpecName "kube-api-access-d2lwc". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 09 03:30:48.508375 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:30:48.508356    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/93f78cdf-c11c-4106-bf64-b940788c7b16-kube-api-access-s4wz7" (OuterVolumeSpecName: "kube-api-access-s4wz7") pod "93f78cdf-c11c-4106-bf64-b940788c7b16" (UID: "93f78cdf-c11c-4106-bf64-b940788c7b16"). InnerVolumeSpecName "kube-api-access-s4wz7". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 09 03:30:48.607372 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:30:48.607339    2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-d2lwc\" (UniqueName: \"kubernetes.io/projected/6fb03975-e94b-4887-91ac-7e680abaddd7-kube-api-access-d2lwc\") on node \"ip-10-0-132-14.ec2.internal\" DevicePath \"\""
Sep 09 03:30:48.607372 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:30:48.607371    2576 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/6fb03975-e94b-4887-91ac-7e680abaddd7-plugins-config-volume\") on node \"ip-10-0-132-14.ec2.internal\" DevicePath \"\""
Sep 09 03:30:48.607576 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:30:48.607385    2576 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/93f78cdf-c11c-4106-bf64-b940788c7b16-plugins-config-volume\") on node \"ip-10-0-132-14.ec2.internal\" DevicePath \"\""
Sep 09 03:30:48.607576 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:30:48.607394    2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-s4wz7\" (UniqueName: \"kubernetes.io/projected/93f78cdf-c11c-4106-bf64-b940788c7b16-kube-api-access-s4wz7\") on node \"ip-10-0-132-14.ec2.internal\" DevicePath \"\""
Sep 09 03:30:48.809061 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:30:48.809029    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-migrate-5f9aae59-7cd98b4b9d-ctpvg"]
Sep 09 03:30:48.812907 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:30:48.812884    2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-migrate-5f9aae59-7cd98b4b9d-ctpvg"]
Sep 09 03:30:48.824129 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:30:48.824105    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-migrate-5f9aae59-75c57dc66-snrpb"]
Sep 09 03:30:48.828080 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:30:48.828059    2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-migrate-5f9aae59-75c57dc66-snrpb"]
Sep 09 03:30:50.366417 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:30:50.366375    2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6fb03975-e94b-4887-91ac-7e680abaddd7" path="/var/lib/kubelet/pods/6fb03975-e94b-4887-91ac-7e680abaddd7/volumes"
Sep 09 03:30:50.366833 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:30:50.366820    2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="93f78cdf-c11c-4106-bf64-b940788c7b16" path="/var/lib/kubelet/pods/93f78cdf-c11c-4106-bf64-b940788c7b16/volumes"
Sep 09 03:30:53.403924 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:30:53.403892    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-b0f5d040-59b87d7d78-9j8cq"]
Sep 09 03:30:53.404402 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:30:53.404234    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6b9078de-fb76-4f52-9060-85db83dd8888" containerName="revoke-keys"
Sep 09 03:30:53.404402 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:30:53.404245    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="6b9078de-fb76-4f52-9060-85db83dd8888" containerName="revoke-keys"
Sep 09 03:30:53.404402 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:30:53.404259    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="93f78cdf-c11c-4106-bf64-b940788c7b16" containerName="payload-processing"
Sep 09 03:30:53.404402 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:30:53.404265    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="93f78cdf-c11c-4106-bf64-b940788c7b16" containerName="payload-processing"
Sep 09 03:30:53.404402 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:30:53.404277    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6fb03975-e94b-4887-91ac-7e680abaddd7" containerName="payload-pre-processing"
Sep 09 03:30:53.404402 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:30:53.404282    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="6fb03975-e94b-4887-91ac-7e680abaddd7" containerName="payload-pre-processing"
Sep 09 03:30:53.404402 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:30:53.404341    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="93f78cdf-c11c-4106-bf64-b940788c7b16" containerName="payload-processing"
Sep 09 03:30:53.404402 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:30:53.404350    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="6fb03975-e94b-4887-91ac-7e680abaddd7" containerName="payload-pre-processing"
Sep 09 03:30:53.404402 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:30:53.404358    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="6b9078de-fb76-4f52-9060-85db83dd8888" containerName="revoke-keys"
Sep 09 03:30:53.409123 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:30:53.409101    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-b0f5d040-59b87d7d78-9j8cq"
Sep 09 03:30:53.412482 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:30:53.412463    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-ait-b0f5d040\""
Sep 09 03:30:53.421446 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:30:53.421423    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-b0f5d040-59b87d7d78-9j8cq"]
Sep 09 03:30:53.435631 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:30:53.435610    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-b0f5d040-7979884f9c-fdmp9"]
Sep 09 03:30:53.438982 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:30:53.438960    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-b0f5d040-7979884f9c-fdmp9"
Sep 09 03:30:53.442074 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:30:53.442053    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-ait-b0f5d040-dockercfg-f7h8t\""
Sep 09 03:30:53.450325 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:30:53.450303    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-b0f5d040-7979884f9c-fdmp9"]
Sep 09 03:30:53.550432 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:30:53.550403    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-w97zf\" (UniqueName: \"kubernetes.io/projected/1734fd3e-15b7-4a03-aa48-3796c1762f7f-kube-api-access-w97zf\") pod \"payload-processing-e2e-ait-b0f5d040-7979884f9c-fdmp9\" (UID: \"1734fd3e-15b7-4a03-aa48-3796c1762f7f\") " pod="openshift-ingress/payload-processing-e2e-ait-b0f5d040-7979884f9c-fdmp9"
Sep 09 03:30:53.550599 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:30:53.550446    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/1734fd3e-15b7-4a03-aa48-3796c1762f7f-plugins-config-volume\") pod \"payload-processing-e2e-ait-b0f5d040-7979884f9c-fdmp9\" (UID: \"1734fd3e-15b7-4a03-aa48-3796c1762f7f\") " pod="openshift-ingress/payload-processing-e2e-ait-b0f5d040-7979884f9c-fdmp9"
Sep 09 03:30:53.550599 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:30:53.550552    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ghgrc\" (UniqueName: \"kubernetes.io/projected/21d13347-cb00-4c5f-9d7e-6f6cfb3498fe-kube-api-access-ghgrc\") pod \"payload-pre-processing-e2e-ait-b0f5d040-59b87d7d78-9j8cq\" (UID: \"21d13347-cb00-4c5f-9d7e-6f6cfb3498fe\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-b0f5d040-59b87d7d78-9j8cq"
Sep 09 03:30:53.550702 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:30:53.550605    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/21d13347-cb00-4c5f-9d7e-6f6cfb3498fe-plugins-config-volume\") pod \"payload-pre-processing-e2e-ait-b0f5d040-59b87d7d78-9j8cq\" (UID: \"21d13347-cb00-4c5f-9d7e-6f6cfb3498fe\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-b0f5d040-59b87d7d78-9j8cq"
Sep 09 03:30:53.651278 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:30:53.651238    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ghgrc\" (UniqueName: \"kubernetes.io/projected/21d13347-cb00-4c5f-9d7e-6f6cfb3498fe-kube-api-access-ghgrc\") pod \"payload-pre-processing-e2e-ait-b0f5d040-59b87d7d78-9j8cq\" (UID: \"21d13347-cb00-4c5f-9d7e-6f6cfb3498fe\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-b0f5d040-59b87d7d78-9j8cq"
Sep 09 03:30:53.651467 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:30:53.651311    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/21d13347-cb00-4c5f-9d7e-6f6cfb3498fe-plugins-config-volume\") pod \"payload-pre-processing-e2e-ait-b0f5d040-59b87d7d78-9j8cq\" (UID: \"21d13347-cb00-4c5f-9d7e-6f6cfb3498fe\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-b0f5d040-59b87d7d78-9j8cq"
Sep 09 03:30:53.651467 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:30:53.651359    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-w97zf\" (UniqueName: \"kubernetes.io/projected/1734fd3e-15b7-4a03-aa48-3796c1762f7f-kube-api-access-w97zf\") pod \"payload-processing-e2e-ait-b0f5d040-7979884f9c-fdmp9\" (UID: \"1734fd3e-15b7-4a03-aa48-3796c1762f7f\") " pod="openshift-ingress/payload-processing-e2e-ait-b0f5d040-7979884f9c-fdmp9"
Sep 09 03:30:53.651467 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:30:53.651409    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/1734fd3e-15b7-4a03-aa48-3796c1762f7f-plugins-config-volume\") pod \"payload-processing-e2e-ait-b0f5d040-7979884f9c-fdmp9\" (UID: \"1734fd3e-15b7-4a03-aa48-3796c1762f7f\") " pod="openshift-ingress/payload-processing-e2e-ait-b0f5d040-7979884f9c-fdmp9"
Sep 09 03:30:53.652132 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:30:53.652108    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/21d13347-cb00-4c5f-9d7e-6f6cfb3498fe-plugins-config-volume\") pod \"payload-pre-processing-e2e-ait-b0f5d040-59b87d7d78-9j8cq\" (UID: \"21d13347-cb00-4c5f-9d7e-6f6cfb3498fe\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-b0f5d040-59b87d7d78-9j8cq"
Sep 09 03:30:53.652291 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:30:53.652267    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/1734fd3e-15b7-4a03-aa48-3796c1762f7f-plugins-config-volume\") pod \"payload-processing-e2e-ait-b0f5d040-7979884f9c-fdmp9\" (UID: \"1734fd3e-15b7-4a03-aa48-3796c1762f7f\") " pod="openshift-ingress/payload-processing-e2e-ait-b0f5d040-7979884f9c-fdmp9"
Sep 09 03:30:53.662345 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:30:53.662284    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ghgrc\" (UniqueName: \"kubernetes.io/projected/21d13347-cb00-4c5f-9d7e-6f6cfb3498fe-kube-api-access-ghgrc\") pod \"payload-pre-processing-e2e-ait-b0f5d040-59b87d7d78-9j8cq\" (UID: \"21d13347-cb00-4c5f-9d7e-6f6cfb3498fe\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-b0f5d040-59b87d7d78-9j8cq"
Sep 09 03:30:53.662435 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:30:53.662411    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-w97zf\" (UniqueName: \"kubernetes.io/projected/1734fd3e-15b7-4a03-aa48-3796c1762f7f-kube-api-access-w97zf\") pod \"payload-processing-e2e-ait-b0f5d040-7979884f9c-fdmp9\" (UID: \"1734fd3e-15b7-4a03-aa48-3796c1762f7f\") " pod="openshift-ingress/payload-processing-e2e-ait-b0f5d040-7979884f9c-fdmp9"
Sep 09 03:30:53.719443 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:30:53.719402    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-b0f5d040-59b87d7d78-9j8cq"
Sep 09 03:30:53.750062 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:30:53.750027    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-b0f5d040-7979884f9c-fdmp9"
Sep 09 03:30:54.001317 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:30:54.001290    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-b0f5d040-59b87d7d78-9j8cq"]
Sep 09 03:30:54.003157 ip-10-0-132-14 kubenswrapper[2576]: W0909 03:30:54.003130    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod21d13347_cb00_4c5f_9d7e_6f6cfb3498fe.slice/crio-27ae738c49957831dde7b47e08b91e540bcdb00a1e2a8efad8ed43d757380e13 WatchSource:0}: Error finding container 27ae738c49957831dde7b47e08b91e540bcdb00a1e2a8efad8ed43d757380e13: Status 404 returned error can't find the container with id 27ae738c49957831dde7b47e08b91e540bcdb00a1e2a8efad8ed43d757380e13
Sep 09 03:30:54.038388 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:30:54.038361    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-b0f5d040-7979884f9c-fdmp9"]
Sep 09 03:30:54.039982 ip-10-0-132-14 kubenswrapper[2576]: W0909 03:30:54.039959    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1734fd3e_15b7_4a03_aa48_3796c1762f7f.slice/crio-d28a280b14ff0ef490e671de1f7217dfbb4e82a5e14b72085b03fb924789904c WatchSource:0}: Error finding container d28a280b14ff0ef490e671de1f7217dfbb4e82a5e14b72085b03fb924789904c: Status 404 returned error can't find the container with id d28a280b14ff0ef490e671de1f7217dfbb4e82a5e14b72085b03fb924789904c
Sep 09 03:30:54.523647 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:30:54.523609    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-b0f5d040-7979884f9c-fdmp9" event={"ID":"1734fd3e-15b7-4a03-aa48-3796c1762f7f","Type":"ContainerStarted","Data":"e45f9d183325110221b15a4181ab68f69efae8536d248d0c9cc1a68a615db0e1"}
Sep 09 03:30:54.524073 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:30:54.523672    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-b0f5d040-7979884f9c-fdmp9" event={"ID":"1734fd3e-15b7-4a03-aa48-3796c1762f7f","Type":"ContainerStarted","Data":"d28a280b14ff0ef490e671de1f7217dfbb4e82a5e14b72085b03fb924789904c"}
Sep 09 03:30:54.524073 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:30:54.523716    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-ait-b0f5d040-7979884f9c-fdmp9"
Sep 09 03:30:54.525395 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:30:54.525371    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-b0f5d040-59b87d7d78-9j8cq" event={"ID":"21d13347-cb00-4c5f-9d7e-6f6cfb3498fe","Type":"ContainerStarted","Data":"1b67eb78dbae447a8e939759042af1bf40eea52c8e08a028073b8689db4074aa"}
Sep 09 03:30:54.525547 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:30:54.525408    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-b0f5d040-59b87d7d78-9j8cq" event={"ID":"21d13347-cb00-4c5f-9d7e-6f6cfb3498fe","Type":"ContainerStarted","Data":"27ae738c49957831dde7b47e08b91e540bcdb00a1e2a8efad8ed43d757380e13"}
Sep 09 03:30:54.525547 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:30:54.525467    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-ait-b0f5d040-59b87d7d78-9j8cq"
Sep 09 03:30:54.543409 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:30:54.543355    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-ait-b0f5d040-7979884f9c-fdmp9" podStartSLOduration=1.54333322 podStartE2EDuration="1.54333322s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 03:30:53 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 03:30:54.541085399 +0000 UTC m=+1068.682571478" watchObservedRunningTime="2026-09-09 03:30:54.54333322 +0000 UTC m=+1068.684819301"
Sep 09 03:30:54.561104 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:30:54.561028    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-ait-b0f5d040-59b87d7d78-9j8cq" podStartSLOduration=1.561008986 podStartE2EDuration="1.561008986s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 03:30:53 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 03:30:54.560749739 +0000 UTC m=+1068.702235816" watchObservedRunningTime="2026-09-09 03:30:54.561008986 +0000 UTC m=+1068.702495064"
Sep 09 03:31:05.532044 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:31:05.532012    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-ait-b0f5d040-59b87d7d78-9j8cq"
Sep 09 03:31:05.532500 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:31:05.532336    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-ait-b0f5d040-7979884f9c-fdmp9"
Sep 09 03:31:17.477613 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:31:17.477575    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-b0f5d040-pxxkf"]
Sep 09 03:31:17.482362 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:31:17.482345    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-b0f5d040-pxxkf"
Sep 09 03:31:17.487520 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:31:17.487499    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-w7q9f\""
Sep 09 03:31:17.492417 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:31:17.492385    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-b0f5d040-pxxkf"]
Sep 09 03:31:17.554157 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:31:17.554124    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/06296c1b-9e7c-43db-967d-814b7f767b2e-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-b0f5d040-pxxkf\" (UID: \"06296c1b-9e7c-43db-967d-814b7f767b2e\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-b0f5d040-pxxkf"
Sep 09 03:31:17.655097 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:31:17.655061    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/06296c1b-9e7c-43db-967d-814b7f767b2e-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-b0f5d040-pxxkf\" (UID: \"06296c1b-9e7c-43db-967d-814b7f767b2e\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-b0f5d040-pxxkf"
Sep 09 03:31:17.655678 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:31:17.655660    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/06296c1b-9e7c-43db-967d-814b7f767b2e-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-b0f5d040-pxxkf\" (UID: \"06296c1b-9e7c-43db-967d-814b7f767b2e\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-b0f5d040-pxxkf"
Sep 09 03:31:17.793608 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:31:17.793578    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-b0f5d040-pxxkf"
Sep 09 03:31:18.084874 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:31:18.084845    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-b0f5d040-pxxkf"]
Sep 09 03:31:18.085788 ip-10-0-132-14 kubenswrapper[2576]: W0909 03:31:18.085768    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod06296c1b_9e7c_43db_967d_814b7f767b2e.slice/crio-8cfc6fb8e107aac3d38a964c6a95d2d21fab115f5ea0893b2def3136efbc4a1a WatchSource:0}: Error finding container 8cfc6fb8e107aac3d38a964c6a95d2d21fab115f5ea0893b2def3136efbc4a1a: Status 404 returned error can't find the container with id 8cfc6fb8e107aac3d38a964c6a95d2d21fab115f5ea0893b2def3136efbc4a1a
Sep 09 03:31:18.642527 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:31:18.642489    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-b0f5d040-pxxkf" event={"ID":"06296c1b-9e7c-43db-967d-814b7f767b2e","Type":"ContainerStarted","Data":"df24c0a849c5b86da97b1db4a054ea4072d10f94d330cbf789c607a73b921196"}
Sep 09 03:31:18.642980 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:31:18.642549    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-b0f5d040-pxxkf" event={"ID":"06296c1b-9e7c-43db-967d-814b7f767b2e","Type":"ContainerStarted","Data":"8cfc6fb8e107aac3d38a964c6a95d2d21fab115f5ea0893b2def3136efbc4a1a"}
Sep 09 03:31:19.648657 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:31:19.648616    2576 generic.go:356] "Generic (PLEG): container finished" podID="06296c1b-9e7c-43db-967d-814b7f767b2e" containerID="df24c0a849c5b86da97b1db4a054ea4072d10f94d330cbf789c607a73b921196" exitCode=0
Sep 09 03:31:19.649086 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:31:19.648682    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-b0f5d040-pxxkf" event={"ID":"06296c1b-9e7c-43db-967d-814b7f767b2e","Type":"ContainerDied","Data":"df24c0a849c5b86da97b1db4a054ea4072d10f94d330cbf789c607a73b921196"}
Sep 09 03:31:20.784935 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:31:20.784911    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-b0f5d040-pxxkf"
Sep 09 03:31:20.881059 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:31:20.881025    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/06296c1b-9e7c-43db-967d-814b7f767b2e-maas-api-service-ca\") pod \"06296c1b-9e7c-43db-967d-814b7f767b2e\" (UID: \"06296c1b-9e7c-43db-967d-814b7f767b2e\") "
Sep 09 03:31:20.881424 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:31:20.881402    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/06296c1b-9e7c-43db-967d-814b7f767b2e-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "06296c1b-9e7c-43db-967d-814b7f767b2e" (UID: "06296c1b-9e7c-43db-967d-814b7f767b2e"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 09 03:31:20.982756 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:31:20.982676    2576 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/06296c1b-9e7c-43db-967d-814b7f767b2e-maas-api-service-ca\") on node \"ip-10-0-132-14.ec2.internal\" DevicePath \"\""
Sep 09 03:31:21.661093 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:31:21.661058    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-b0f5d040-pxxkf"
Sep 09 03:31:21.661093 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:31:21.661064    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-b0f5d040-pxxkf" event={"ID":"06296c1b-9e7c-43db-967d-814b7f767b2e","Type":"ContainerDied","Data":"8cfc6fb8e107aac3d38a964c6a95d2d21fab115f5ea0893b2def3136efbc4a1a"}
Sep 09 03:31:21.661295 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:31:21.661110    2576 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="8cfc6fb8e107aac3d38a964c6a95d2d21fab115f5ea0893b2def3136efbc4a1a"
Sep 09 03:31:23.540166 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:31:23.540137    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-b0f5d040-7979884f9c-fdmp9"]
Sep 09 03:31:23.540623 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:31:23.540338    2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-ait-b0f5d040-7979884f9c-fdmp9" podUID="1734fd3e-15b7-4a03-aa48-3796c1762f7f" containerName="payload-processing" containerID="cri-o://e45f9d183325110221b15a4181ab68f69efae8536d248d0c9cc1a68a615db0e1" gracePeriod=30
Sep 09 03:31:23.550263 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:31:23.550234    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-b0f5d040-59b87d7d78-9j8cq"]
Sep 09 03:31:23.550517 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:31:23.550465    2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-ait-b0f5d040-59b87d7d78-9j8cq" podUID="21d13347-cb00-4c5f-9d7e-6f6cfb3498fe" containerName="payload-pre-processing" containerID="cri-o://1b67eb78dbae447a8e939759042af1bf40eea52c8e08a028073b8689db4074aa" gracePeriod=30
Sep 09 03:31:23.673930 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:31:23.673898    2576 generic.go:356] "Generic (PLEG): container finished" podID="1734fd3e-15b7-4a03-aa48-3796c1762f7f" containerID="e45f9d183325110221b15a4181ab68f69efae8536d248d0c9cc1a68a615db0e1" exitCode=0
Sep 09 03:31:23.674127 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:31:23.673938    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-b0f5d040-7979884f9c-fdmp9" event={"ID":"1734fd3e-15b7-4a03-aa48-3796c1762f7f","Type":"ContainerDied","Data":"e45f9d183325110221b15a4181ab68f69efae8536d248d0c9cc1a68a615db0e1"}
Sep 09 03:31:23.675865 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:31:23.675793    2576 generic.go:356] "Generic (PLEG): container finished" podID="21d13347-cb00-4c5f-9d7e-6f6cfb3498fe" containerID="1b67eb78dbae447a8e939759042af1bf40eea52c8e08a028073b8689db4074aa" exitCode=0
Sep 09 03:31:23.675969 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:31:23.675877    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-b0f5d040-59b87d7d78-9j8cq" event={"ID":"21d13347-cb00-4c5f-9d7e-6f6cfb3498fe","Type":"ContainerDied","Data":"1b67eb78dbae447a8e939759042af1bf40eea52c8e08a028073b8689db4074aa"}
Sep 09 03:31:23.731071 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:31:23.731041    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-b0f5d040-pxxkf"]
Sep 09 03:31:23.733719 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:31:23.733692    2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-b0f5d040-pxxkf"]
Sep 09 03:31:23.811992 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:31:23.811972    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-b0f5d040-7979884f9c-fdmp9"
Sep 09 03:31:23.824938 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:31:23.824918    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-b0f5d040-59b87d7d78-9j8cq"
Sep 09 03:31:23.911927 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:31:23.911890    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/21d13347-cb00-4c5f-9d7e-6f6cfb3498fe-plugins-config-volume\") pod \"21d13347-cb00-4c5f-9d7e-6f6cfb3498fe\" (UID: \"21d13347-cb00-4c5f-9d7e-6f6cfb3498fe\") "
Sep 09 03:31:23.912117 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:31:23.911948    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-ghgrc\" (UniqueName: \"kubernetes.io/projected/21d13347-cb00-4c5f-9d7e-6f6cfb3498fe-kube-api-access-ghgrc\") pod \"21d13347-cb00-4c5f-9d7e-6f6cfb3498fe\" (UID: \"21d13347-cb00-4c5f-9d7e-6f6cfb3498fe\") "
Sep 09 03:31:23.912117 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:31:23.911995    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-w97zf\" (UniqueName: \"kubernetes.io/projected/1734fd3e-15b7-4a03-aa48-3796c1762f7f-kube-api-access-w97zf\") pod \"1734fd3e-15b7-4a03-aa48-3796c1762f7f\" (UID: \"1734fd3e-15b7-4a03-aa48-3796c1762f7f\") "
Sep 09 03:31:23.912117 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:31:23.912074    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/1734fd3e-15b7-4a03-aa48-3796c1762f7f-plugins-config-volume\") pod \"1734fd3e-15b7-4a03-aa48-3796c1762f7f\" (UID: \"1734fd3e-15b7-4a03-aa48-3796c1762f7f\") "
Sep 09 03:31:23.912379 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:31:23.912346    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/21d13347-cb00-4c5f-9d7e-6f6cfb3498fe-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "21d13347-cb00-4c5f-9d7e-6f6cfb3498fe" (UID: "21d13347-cb00-4c5f-9d7e-6f6cfb3498fe"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 09 03:31:23.912452 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:31:23.912414    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1734fd3e-15b7-4a03-aa48-3796c1762f7f-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "1734fd3e-15b7-4a03-aa48-3796c1762f7f" (UID: "1734fd3e-15b7-4a03-aa48-3796c1762f7f"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 09 03:31:23.914240 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:31:23.914214    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/21d13347-cb00-4c5f-9d7e-6f6cfb3498fe-kube-api-access-ghgrc" (OuterVolumeSpecName: "kube-api-access-ghgrc") pod "21d13347-cb00-4c5f-9d7e-6f6cfb3498fe" (UID: "21d13347-cb00-4c5f-9d7e-6f6cfb3498fe"). InnerVolumeSpecName "kube-api-access-ghgrc". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 09 03:31:23.914323 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:31:23.914259    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1734fd3e-15b7-4a03-aa48-3796c1762f7f-kube-api-access-w97zf" (OuterVolumeSpecName: "kube-api-access-w97zf") pod "1734fd3e-15b7-4a03-aa48-3796c1762f7f" (UID: "1734fd3e-15b7-4a03-aa48-3796c1762f7f"). InnerVolumeSpecName "kube-api-access-w97zf". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 09 03:31:24.013426 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:31:24.013393    2576 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/1734fd3e-15b7-4a03-aa48-3796c1762f7f-plugins-config-volume\") on node \"ip-10-0-132-14.ec2.internal\" DevicePath \"\""
Sep 09 03:31:24.013426 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:31:24.013420    2576 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/21d13347-cb00-4c5f-9d7e-6f6cfb3498fe-plugins-config-volume\") on node \"ip-10-0-132-14.ec2.internal\" DevicePath \"\""
Sep 09 03:31:24.013426 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:31:24.013430    2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ghgrc\" (UniqueName: \"kubernetes.io/projected/21d13347-cb00-4c5f-9d7e-6f6cfb3498fe-kube-api-access-ghgrc\") on node \"ip-10-0-132-14.ec2.internal\" DevicePath \"\""
Sep 09 03:31:24.013667 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:31:24.013439    2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-w97zf\" (UniqueName: \"kubernetes.io/projected/1734fd3e-15b7-4a03-aa48-3796c1762f7f-kube-api-access-w97zf\") on node \"ip-10-0-132-14.ec2.internal\" DevicePath \"\""
Sep 09 03:31:24.360559 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:31:24.360526    2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="06296c1b-9e7c-43db-967d-814b7f767b2e" path="/var/lib/kubelet/pods/06296c1b-9e7c-43db-967d-814b7f767b2e/volumes"
Sep 09 03:31:24.685881 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:31:24.683063    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-b0f5d040-7979884f9c-fdmp9" event={"ID":"1734fd3e-15b7-4a03-aa48-3796c1762f7f","Type":"ContainerDied","Data":"d28a280b14ff0ef490e671de1f7217dfbb4e82a5e14b72085b03fb924789904c"}
Sep 09 03:31:24.685881 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:31:24.683178    2576 scope.go:117] "RemoveContainer" containerID="e45f9d183325110221b15a4181ab68f69efae8536d248d0c9cc1a68a615db0e1"
Sep 09 03:31:24.685881 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:31:24.683374    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-b0f5d040-7979884f9c-fdmp9"
Sep 09 03:31:24.690681 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:31:24.690654    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-b0f5d040-59b87d7d78-9j8cq" event={"ID":"21d13347-cb00-4c5f-9d7e-6f6cfb3498fe","Type":"ContainerDied","Data":"27ae738c49957831dde7b47e08b91e540bcdb00a1e2a8efad8ed43d757380e13"}
Sep 09 03:31:24.690759 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:31:24.690710    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-b0f5d040-59b87d7d78-9j8cq"
Sep 09 03:31:24.698379 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:31:24.698332    2576 scope.go:117] "RemoveContainer" containerID="1b67eb78dbae447a8e939759042af1bf40eea52c8e08a028073b8689db4074aa"
Sep 09 03:31:24.711929 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:31:24.711906    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-b0f5d040-7979884f9c-fdmp9"]
Sep 09 03:31:24.718929 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:31:24.718908    2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-b0f5d040-7979884f9c-fdmp9"]
Sep 09 03:31:24.733195 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:31:24.733175    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-b0f5d040-59b87d7d78-9j8cq"]
Sep 09 03:31:24.745485 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:31:24.745449    2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-b0f5d040-59b87d7d78-9j8cq"]
Sep 09 03:31:26.360408 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:31:26.360375    2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="1734fd3e-15b7-4a03-aa48-3796c1762f7f" path="/var/lib/kubelet/pods/1734fd3e-15b7-4a03-aa48-3796c1762f7f/volumes"
Sep 09 03:31:26.360783 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:31:26.360771    2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="21d13347-cb00-4c5f-9d7e-6f6cfb3498fe" path="/var/lib/kubelet/pods/21d13347-cb00-4c5f-9d7e-6f6cfb3498fe/volumes"
Sep 09 03:31:29.010259 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:31:29.010226    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-ad343832-7d9bb989cd-sq755"]
Sep 09 03:31:29.010707 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:31:29.010565    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="06296c1b-9e7c-43db-967d-814b7f767b2e" containerName="revoke-keys"
Sep 09 03:31:29.010707 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:31:29.010576    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="06296c1b-9e7c-43db-967d-814b7f767b2e" containerName="revoke-keys"
Sep 09 03:31:29.010707 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:31:29.010600    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1734fd3e-15b7-4a03-aa48-3796c1762f7f" containerName="payload-processing"
Sep 09 03:31:29.010707 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:31:29.010606    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="1734fd3e-15b7-4a03-aa48-3796c1762f7f" containerName="payload-processing"
Sep 09 03:31:29.010707 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:31:29.010615    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="21d13347-cb00-4c5f-9d7e-6f6cfb3498fe" containerName="payload-pre-processing"
Sep 09 03:31:29.010707 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:31:29.010620    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="21d13347-cb00-4c5f-9d7e-6f6cfb3498fe" containerName="payload-pre-processing"
Sep 09 03:31:29.010707 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:31:29.010674    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="06296c1b-9e7c-43db-967d-814b7f767b2e" containerName="revoke-keys"
Sep 09 03:31:29.010707 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:31:29.010683    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="1734fd3e-15b7-4a03-aa48-3796c1762f7f" containerName="payload-processing"
Sep 09 03:31:29.010707 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:31:29.010690    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="21d13347-cb00-4c5f-9d7e-6f6cfb3498fe" containerName="payload-pre-processing"
Sep 09 03:31:29.015368 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:31:29.015338    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-ad343832-7d9bb989cd-sq755"
Sep 09 03:31:29.018234 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:31:29.018213    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-derive-ad343832-serving-cert\""
Sep 09 03:31:29.027513 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:31:29.027492    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-ad343832-7d9bb989cd-sq755"]
Sep 09 03:31:29.160402 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:31:29.160361    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/1b31d5a7-3ba4-409d-8d2b-31b479dddbd7-openshift-service-ca\") pod \"maas-api-e2e-derive-ad343832-7d9bb989cd-sq755\" (UID: \"1b31d5a7-3ba4-409d-8d2b-31b479dddbd7\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-ad343832-7d9bb989cd-sq755"
Sep 09 03:31:29.160576 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:31:29.160411    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/1b31d5a7-3ba4-409d-8d2b-31b479dddbd7-maas-api-tls\") pod \"maas-api-e2e-derive-ad343832-7d9bb989cd-sq755\" (UID: \"1b31d5a7-3ba4-409d-8d2b-31b479dddbd7\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-ad343832-7d9bb989cd-sq755"
Sep 09 03:31:29.160576 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:31:29.160480    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/1b31d5a7-3ba4-409d-8d2b-31b479dddbd7-metrics-certs\") pod \"maas-api-e2e-derive-ad343832-7d9bb989cd-sq755\" (UID: \"1b31d5a7-3ba4-409d-8d2b-31b479dddbd7\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-ad343832-7d9bb989cd-sq755"
Sep 09 03:31:29.160576 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:31:29.160522    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sn8nb\" (UniqueName: \"kubernetes.io/projected/1b31d5a7-3ba4-409d-8d2b-31b479dddbd7-kube-api-access-sn8nb\") pod \"maas-api-e2e-derive-ad343832-7d9bb989cd-sq755\" (UID: \"1b31d5a7-3ba4-409d-8d2b-31b479dddbd7\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-ad343832-7d9bb989cd-sq755"
Sep 09 03:31:29.261264 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:31:29.261185    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sn8nb\" (UniqueName: \"kubernetes.io/projected/1b31d5a7-3ba4-409d-8d2b-31b479dddbd7-kube-api-access-sn8nb\") pod \"maas-api-e2e-derive-ad343832-7d9bb989cd-sq755\" (UID: \"1b31d5a7-3ba4-409d-8d2b-31b479dddbd7\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-ad343832-7d9bb989cd-sq755"
Sep 09 03:31:29.261264 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:31:29.261246    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/1b31d5a7-3ba4-409d-8d2b-31b479dddbd7-openshift-service-ca\") pod \"maas-api-e2e-derive-ad343832-7d9bb989cd-sq755\" (UID: \"1b31d5a7-3ba4-409d-8d2b-31b479dddbd7\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-ad343832-7d9bb989cd-sq755"
Sep 09 03:31:29.261448 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:31:29.261275    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/1b31d5a7-3ba4-409d-8d2b-31b479dddbd7-maas-api-tls\") pod \"maas-api-e2e-derive-ad343832-7d9bb989cd-sq755\" (UID: \"1b31d5a7-3ba4-409d-8d2b-31b479dddbd7\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-ad343832-7d9bb989cd-sq755"
Sep 09 03:31:29.261448 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:31:29.261309    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/1b31d5a7-3ba4-409d-8d2b-31b479dddbd7-metrics-certs\") pod \"maas-api-e2e-derive-ad343832-7d9bb989cd-sq755\" (UID: \"1b31d5a7-3ba4-409d-8d2b-31b479dddbd7\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-ad343832-7d9bb989cd-sq755"
Sep 09 03:31:29.261941 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:31:29.261921    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/1b31d5a7-3ba4-409d-8d2b-31b479dddbd7-openshift-service-ca\") pod \"maas-api-e2e-derive-ad343832-7d9bb989cd-sq755\" (UID: \"1b31d5a7-3ba4-409d-8d2b-31b479dddbd7\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-ad343832-7d9bb989cd-sq755"
Sep 09 03:31:29.264025 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:31:29.264005    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/1b31d5a7-3ba4-409d-8d2b-31b479dddbd7-metrics-certs\") pod \"maas-api-e2e-derive-ad343832-7d9bb989cd-sq755\" (UID: \"1b31d5a7-3ba4-409d-8d2b-31b479dddbd7\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-ad343832-7d9bb989cd-sq755"
Sep 09 03:31:29.264116 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:31:29.264056    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/1b31d5a7-3ba4-409d-8d2b-31b479dddbd7-maas-api-tls\") pod \"maas-api-e2e-derive-ad343832-7d9bb989cd-sq755\" (UID: \"1b31d5a7-3ba4-409d-8d2b-31b479dddbd7\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-ad343832-7d9bb989cd-sq755"
Sep 09 03:31:29.272141 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:31:29.272114    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-sn8nb\" (UniqueName: \"kubernetes.io/projected/1b31d5a7-3ba4-409d-8d2b-31b479dddbd7-kube-api-access-sn8nb\") pod \"maas-api-e2e-derive-ad343832-7d9bb989cd-sq755\" (UID: \"1b31d5a7-3ba4-409d-8d2b-31b479dddbd7\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-ad343832-7d9bb989cd-sq755"
Sep 09 03:31:29.326421 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:31:29.326389    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-ad343832-7d9bb989cd-sq755"
Sep 09 03:31:29.600073 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:31:29.600049    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-ad343832-7d9bb989cd-sq755"]
Sep 09 03:31:29.601863 ip-10-0-132-14 kubenswrapper[2576]: W0909 03:31:29.601840    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1b31d5a7_3ba4_409d_8d2b_31b479dddbd7.slice/crio-5b85884ef3a1e71b0e996c16114ff23840f3df188c7b7f222ba544150fcd8a30 WatchSource:0}: Error finding container 5b85884ef3a1e71b0e996c16114ff23840f3df188c7b7f222ba544150fcd8a30: Status 404 returned error can't find the container with id 5b85884ef3a1e71b0e996c16114ff23840f3df188c7b7f222ba544150fcd8a30
Sep 09 03:31:29.726366 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:31:29.726327    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-ad343832-7d9bb989cd-sq755" event={"ID":"1b31d5a7-3ba4-409d-8d2b-31b479dddbd7","Type":"ContainerStarted","Data":"5b85884ef3a1e71b0e996c16114ff23840f3df188c7b7f222ba544150fcd8a30"}
Sep 09 03:31:29.824379 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:31:29.824344    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-derive-ad343832-846957c78f-kx7px"]
Sep 09 03:31:29.829045 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:31:29.829024    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-derive-ad343832-846957c78f-kx7px"
Sep 09 03:31:29.832053 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:31:29.832020    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-derive-ad343832\""
Sep 09 03:31:29.832273 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:31:29.832257    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-derive-ad343832-dockercfg-w4lrv\""
Sep 09 03:31:29.843192 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:31:29.843169    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-derive-ad343832-846957c78f-kx7px"]
Sep 09 03:31:29.967752 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:31:29.967668    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2fbtf\" (UniqueName: \"kubernetes.io/projected/4893ff8b-d74c-475e-ab0b-7cf86256fa72-kube-api-access-2fbtf\") pod \"payload-processing-e2e-derive-ad343832-846957c78f-kx7px\" (UID: \"4893ff8b-d74c-475e-ab0b-7cf86256fa72\") " pod="openshift-ingress/payload-processing-e2e-derive-ad343832-846957c78f-kx7px"
Sep 09 03:31:29.967752 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:31:29.967720    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/4893ff8b-d74c-475e-ab0b-7cf86256fa72-plugins-config-volume\") pod \"payload-processing-e2e-derive-ad343832-846957c78f-kx7px\" (UID: \"4893ff8b-d74c-475e-ab0b-7cf86256fa72\") " pod="openshift-ingress/payload-processing-e2e-derive-ad343832-846957c78f-kx7px"
Sep 09 03:31:30.068432 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:31:30.068403    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2fbtf\" (UniqueName: \"kubernetes.io/projected/4893ff8b-d74c-475e-ab0b-7cf86256fa72-kube-api-access-2fbtf\") pod \"payload-processing-e2e-derive-ad343832-846957c78f-kx7px\" (UID: \"4893ff8b-d74c-475e-ab0b-7cf86256fa72\") " pod="openshift-ingress/payload-processing-e2e-derive-ad343832-846957c78f-kx7px"
Sep 09 03:31:30.068859 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:31:30.068473    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/4893ff8b-d74c-475e-ab0b-7cf86256fa72-plugins-config-volume\") pod \"payload-processing-e2e-derive-ad343832-846957c78f-kx7px\" (UID: \"4893ff8b-d74c-475e-ab0b-7cf86256fa72\") " pod="openshift-ingress/payload-processing-e2e-derive-ad343832-846957c78f-kx7px"
Sep 09 03:31:30.069220 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:31:30.069197    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/4893ff8b-d74c-475e-ab0b-7cf86256fa72-plugins-config-volume\") pod \"payload-processing-e2e-derive-ad343832-846957c78f-kx7px\" (UID: \"4893ff8b-d74c-475e-ab0b-7cf86256fa72\") " pod="openshift-ingress/payload-processing-e2e-derive-ad343832-846957c78f-kx7px"
Sep 09 03:31:30.078708 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:31:30.078687    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2fbtf\" (UniqueName: \"kubernetes.io/projected/4893ff8b-d74c-475e-ab0b-7cf86256fa72-kube-api-access-2fbtf\") pod \"payload-processing-e2e-derive-ad343832-846957c78f-kx7px\" (UID: \"4893ff8b-d74c-475e-ab0b-7cf86256fa72\") " pod="openshift-ingress/payload-processing-e2e-derive-ad343832-846957c78f-kx7px"
Sep 09 03:31:30.139539 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:31:30.139509    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-derive-ad343832-846957c78f-kx7px"
Sep 09 03:31:30.415490 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:31:30.415463    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-derive-ad343832-846957c78f-kx7px"]
Sep 09 03:31:30.418178 ip-10-0-132-14 kubenswrapper[2576]: W0909 03:31:30.418150    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4893ff8b_d74c_475e_ab0b_7cf86256fa72.slice/crio-35e513d8e0f9da76b4912b6bc89e8c0e62e44d756d3b53259b9bf7f877c553b5 WatchSource:0}: Error finding container 35e513d8e0f9da76b4912b6bc89e8c0e62e44d756d3b53259b9bf7f877c553b5: Status 404 returned error can't find the container with id 35e513d8e0f9da76b4912b6bc89e8c0e62e44d756d3b53259b9bf7f877c553b5
Sep 09 03:31:30.733589 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:31:30.733502    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-ad343832-7d9bb989cd-sq755" event={"ID":"1b31d5a7-3ba4-409d-8d2b-31b479dddbd7","Type":"ContainerStarted","Data":"070013acfd999dbebb84bc2c1f5a06400f76d1f2e1ccb84dde96ceb312ed21dd"}
Sep 09 03:31:30.733734 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:31:30.733637    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-derive-ad343832-7d9bb989cd-sq755"
Sep 09 03:31:30.735404 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:31:30.735380    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-derive-ad343832-846957c78f-kx7px" event={"ID":"4893ff8b-d74c-475e-ab0b-7cf86256fa72","Type":"ContainerStarted","Data":"12f9f6f4ab68816fa44268c557d58a06e0dd8a1db1ab861deefd432d72ab7ced"}
Sep 09 03:31:30.735520 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:31:30.735424    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-derive-ad343832-846957c78f-kx7px" event={"ID":"4893ff8b-d74c-475e-ab0b-7cf86256fa72","Type":"ContainerStarted","Data":"35e513d8e0f9da76b4912b6bc89e8c0e62e44d756d3b53259b9bf7f877c553b5"}
Sep 09 03:31:30.735578 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:31:30.735532    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-derive-ad343832-846957c78f-kx7px"
Sep 09 03:31:30.753707 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:31:30.753663    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-derive-ad343832-7d9bb989cd-sq755" podStartSLOduration=2.361498001 podStartE2EDuration="2.753649047s" totalImagesPullingTime="392.151046ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 03:31:28 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 03:31:30.753287966 +0000 UTC m=+1104.894774046" watchObservedRunningTime="2026-09-09 03:31:30.753649047 +0000 UTC m=+1104.895135135"
Sep 09 03:31:30.774364 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:31:30.774316    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-derive-ad343832-846957c78f-kx7px" podStartSLOduration=1.774296517 podStartE2EDuration="1.774296517s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 03:31:29 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 03:31:30.773238162 +0000 UTC m=+1104.914724240" watchObservedRunningTime="2026-09-09 03:31:30.774296517 +0000 UTC m=+1104.915782599"
Sep 09 03:31:36.747411 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:31:36.747379    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-derive-ad343832-7d9bb989cd-sq755"
Sep 09 03:31:41.743987 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:31:41.743954    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-derive-ad343832-846957c78f-kx7px"
Sep 09 03:31:59.258425 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:31:59.258378    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-ad343832-7d9bb989cd-sq755"]
Sep 09 03:31:59.259010 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:31:59.258645    2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-derive-ad343832-7d9bb989cd-sq755" podUID="1b31d5a7-3ba4-409d-8d2b-31b479dddbd7" containerName="maas-api" containerID="cri-o://070013acfd999dbebb84bc2c1f5a06400f76d1f2e1ccb84dde96ceb312ed21dd" gracePeriod=30
Sep 09 03:31:59.322616 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:31:59.322579    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-derive-ad343832-846957c78f-kx7px"]
Sep 09 03:31:59.322841 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:31:59.322777    2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-derive-ad343832-846957c78f-kx7px" podUID="4893ff8b-d74c-475e-ab0b-7cf86256fa72" containerName="payload-processing" containerID="cri-o://12f9f6f4ab68816fa44268c557d58a06e0dd8a1db1ab861deefd432d72ab7ced" gracePeriod=30
Sep 09 03:31:59.449983 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:31:59.449934    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-autores-a2fc47-57857d79d5-vbw6n"]
Sep 09 03:31:59.450479 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:31:59.450397    2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-autores-a2fc47-57857d79d5-vbw6n" podUID="03281339-40a3-4b4e-9440-18a8008be8fa" containerName="maas-api" containerID="cri-o://a59ef5365878d0afa2fd61977eaf8d735444f3a960dae0f573868a07ba5b0a7e" gracePeriod=30
Sep 09 03:31:59.526213 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:31:59.526119    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-autores-a2fc47-69fdd48ccf-wjj48"]
Sep 09 03:31:59.526421 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:31:59.526371    2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-autores-a2fc47-69fdd48ccf-wjj48" podUID="82d8fbaa-1300-4bcc-be5d-77bf70dbed9c" containerName="payload-processing" containerID="cri-o://f0b0badd73f452a7093302bd917882275647cfc46cb764b351abea8742c0368c" gracePeriod=30
Sep 09 03:31:59.535375 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:31:59.535342    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-autores-a2fc47-8fdd95594-hxg7w"]
Sep 09 03:31:59.535759 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:31:59.535733    2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-autores-a2fc47-8fdd95594-hxg7w" podUID="1047ee3d-c020-44c2-948c-719a1e85ff72" containerName="payload-pre-processing" containerID="cri-o://174b04dd483193fe2b17dbf2702846a3a303faf8b839d934a77c4e5e380688a0" gracePeriod=30
Sep 09 03:31:59.677577 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:31:59.677328    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-derive-ad343832-846957c78f-kx7px"
Sep 09 03:31:59.681860 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:31:59.681129    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-ad343832-7d9bb989cd-sq755"
Sep 09 03:31:59.719055 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:31:59.719029    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/4893ff8b-d74c-475e-ab0b-7cf86256fa72-plugins-config-volume\") pod \"4893ff8b-d74c-475e-ab0b-7cf86256fa72\" (UID: \"4893ff8b-d74c-475e-ab0b-7cf86256fa72\") "
Sep 09 03:31:59.719250 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:31:59.719239    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-2fbtf\" (UniqueName: \"kubernetes.io/projected/4893ff8b-d74c-475e-ab0b-7cf86256fa72-kube-api-access-2fbtf\") pod \"4893ff8b-d74c-475e-ab0b-7cf86256fa72\" (UID: \"4893ff8b-d74c-475e-ab0b-7cf86256fa72\") "
Sep 09 03:31:59.719356 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:31:59.719347    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/1b31d5a7-3ba4-409d-8d2b-31b479dddbd7-metrics-certs\") pod \"1b31d5a7-3ba4-409d-8d2b-31b479dddbd7\" (UID: \"1b31d5a7-3ba4-409d-8d2b-31b479dddbd7\") "
Sep 09 03:31:59.719677 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:31:59.719401    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4893ff8b-d74c-475e-ab0b-7cf86256fa72-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "4893ff8b-d74c-475e-ab0b-7cf86256fa72" (UID: "4893ff8b-d74c-475e-ab0b-7cf86256fa72"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 09 03:31:59.719677 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:31:59.719466    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/1b31d5a7-3ba4-409d-8d2b-31b479dddbd7-openshift-service-ca\") pod \"1b31d5a7-3ba4-409d-8d2b-31b479dddbd7\" (UID: \"1b31d5a7-3ba4-409d-8d2b-31b479dddbd7\") "
Sep 09 03:31:59.719677 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:31:59.719498    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/1b31d5a7-3ba4-409d-8d2b-31b479dddbd7-maas-api-tls\") pod \"1b31d5a7-3ba4-409d-8d2b-31b479dddbd7\" (UID: \"1b31d5a7-3ba4-409d-8d2b-31b479dddbd7\") "
Sep 09 03:31:59.719677 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:31:59.719515    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-sn8nb\" (UniqueName: \"kubernetes.io/projected/1b31d5a7-3ba4-409d-8d2b-31b479dddbd7-kube-api-access-sn8nb\") pod \"1b31d5a7-3ba4-409d-8d2b-31b479dddbd7\" (UID: \"1b31d5a7-3ba4-409d-8d2b-31b479dddbd7\") "
Sep 09 03:31:59.720338 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:31:59.720054    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1b31d5a7-3ba4-409d-8d2b-31b479dddbd7-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "1b31d5a7-3ba4-409d-8d2b-31b479dddbd7" (UID: "1b31d5a7-3ba4-409d-8d2b-31b479dddbd7"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 09 03:31:59.720794 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:31:59.720764    2576 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/1b31d5a7-3ba4-409d-8d2b-31b479dddbd7-openshift-service-ca\") on node \"ip-10-0-132-14.ec2.internal\" DevicePath \"\""
Sep 09 03:31:59.720940 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:31:59.720926    2576 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/4893ff8b-d74c-475e-ab0b-7cf86256fa72-plugins-config-volume\") on node \"ip-10-0-132-14.ec2.internal\" DevicePath \"\""
Sep 09 03:31:59.722692 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:31:59.722625    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1b31d5a7-3ba4-409d-8d2b-31b479dddbd7-kube-api-access-sn8nb" (OuterVolumeSpecName: "kube-api-access-sn8nb") pod "1b31d5a7-3ba4-409d-8d2b-31b479dddbd7" (UID: "1b31d5a7-3ba4-409d-8d2b-31b479dddbd7"). InnerVolumeSpecName "kube-api-access-sn8nb". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 09 03:31:59.722692 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:31:59.722661    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1b31d5a7-3ba4-409d-8d2b-31b479dddbd7-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "1b31d5a7-3ba4-409d-8d2b-31b479dddbd7" (UID: "1b31d5a7-3ba4-409d-8d2b-31b479dddbd7"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 09 03:31:59.723012 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:31:59.722984    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1b31d5a7-3ba4-409d-8d2b-31b479dddbd7-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "1b31d5a7-3ba4-409d-8d2b-31b479dddbd7" (UID: "1b31d5a7-3ba4-409d-8d2b-31b479dddbd7"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 09 03:31:59.723781 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:31:59.723682    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4893ff8b-d74c-475e-ab0b-7cf86256fa72-kube-api-access-2fbtf" (OuterVolumeSpecName: "kube-api-access-2fbtf") pod "4893ff8b-d74c-475e-ab0b-7cf86256fa72" (UID: "4893ff8b-d74c-475e-ab0b-7cf86256fa72"). InnerVolumeSpecName "kube-api-access-2fbtf". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 09 03:31:59.821946 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:31:59.821870    2576 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/1b31d5a7-3ba4-409d-8d2b-31b479dddbd7-maas-api-tls\") on node \"ip-10-0-132-14.ec2.internal\" DevicePath \"\""
Sep 09 03:31:59.821946 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:31:59.821901    2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-sn8nb\" (UniqueName: \"kubernetes.io/projected/1b31d5a7-3ba4-409d-8d2b-31b479dddbd7-kube-api-access-sn8nb\") on node \"ip-10-0-132-14.ec2.internal\" DevicePath \"\""
Sep 09 03:31:59.821946 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:31:59.821912    2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2fbtf\" (UniqueName: \"kubernetes.io/projected/4893ff8b-d74c-475e-ab0b-7cf86256fa72-kube-api-access-2fbtf\") on node \"ip-10-0-132-14.ec2.internal\" DevicePath \"\""
Sep 09 03:31:59.821946 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:31:59.821921    2576 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/1b31d5a7-3ba4-409d-8d2b-31b479dddbd7-metrics-certs\") on node \"ip-10-0-132-14.ec2.internal\" DevicePath \"\""
Sep 09 03:31:59.834333 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:31:59.834308    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-autores-a2fc47-57857d79d5-vbw6n"
Sep 09 03:31:59.843963 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:31:59.843943    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-autores-a2fc47-69fdd48ccf-wjj48"
Sep 09 03:31:59.847077 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:31:59.847056    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-autores-a2fc47-8fdd95594-hxg7w"
Sep 09 03:31:59.885596 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:31:59.885564    2576 generic.go:356] "Generic (PLEG): container finished" podID="1047ee3d-c020-44c2-948c-719a1e85ff72" containerID="174b04dd483193fe2b17dbf2702846a3a303faf8b839d934a77c4e5e380688a0" exitCode=0
Sep 09 03:31:59.885763 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:31:59.885609    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-autores-a2fc47-8fdd95594-hxg7w"
Sep 09 03:31:59.885763 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:31:59.885653    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-autores-a2fc47-8fdd95594-hxg7w" event={"ID":"1047ee3d-c020-44c2-948c-719a1e85ff72","Type":"ContainerDied","Data":"174b04dd483193fe2b17dbf2702846a3a303faf8b839d934a77c4e5e380688a0"}
Sep 09 03:31:59.885763 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:31:59.885706    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-autores-a2fc47-8fdd95594-hxg7w" event={"ID":"1047ee3d-c020-44c2-948c-719a1e85ff72","Type":"ContainerDied","Data":"248197bba51920178e0106063f841a1511e935e524a57dc6225dc0226a92a92f"}
Sep 09 03:31:59.885763 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:31:59.885724    2576 scope.go:117] "RemoveContainer" containerID="174b04dd483193fe2b17dbf2702846a3a303faf8b839d934a77c4e5e380688a0"
Sep 09 03:31:59.887124 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:31:59.887100    2576 generic.go:356] "Generic (PLEG): container finished" podID="82d8fbaa-1300-4bcc-be5d-77bf70dbed9c" containerID="f0b0badd73f452a7093302bd917882275647cfc46cb764b351abea8742c0368c" exitCode=0
Sep 09 03:31:59.887245 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:31:59.887140    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-autores-a2fc47-69fdd48ccf-wjj48"
Sep 09 03:31:59.887245 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:31:59.887197    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-autores-a2fc47-69fdd48ccf-wjj48" event={"ID":"82d8fbaa-1300-4bcc-be5d-77bf70dbed9c","Type":"ContainerDied","Data":"f0b0badd73f452a7093302bd917882275647cfc46cb764b351abea8742c0368c"}
Sep 09 03:31:59.887373 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:31:59.887252    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-autores-a2fc47-69fdd48ccf-wjj48" event={"ID":"82d8fbaa-1300-4bcc-be5d-77bf70dbed9c","Type":"ContainerDied","Data":"6df7a86d73b6c044fcdb3e5ffb696d41c89a8ec83b8834e0a5fece1fe24fb979"}
Sep 09 03:31:59.888622 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:31:59.888605    2576 generic.go:356] "Generic (PLEG): container finished" podID="1b31d5a7-3ba4-409d-8d2b-31b479dddbd7" containerID="070013acfd999dbebb84bc2c1f5a06400f76d1f2e1ccb84dde96ceb312ed21dd" exitCode=0
Sep 09 03:31:59.888724 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:31:59.888658    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-ad343832-7d9bb989cd-sq755"
Sep 09 03:31:59.888724 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:31:59.888691    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-ad343832-7d9bb989cd-sq755" event={"ID":"1b31d5a7-3ba4-409d-8d2b-31b479dddbd7","Type":"ContainerDied","Data":"070013acfd999dbebb84bc2c1f5a06400f76d1f2e1ccb84dde96ceb312ed21dd"}
Sep 09 03:31:59.888859 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:31:59.888735    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-ad343832-7d9bb989cd-sq755" event={"ID":"1b31d5a7-3ba4-409d-8d2b-31b479dddbd7","Type":"ContainerDied","Data":"5b85884ef3a1e71b0e996c16114ff23840f3df188c7b7f222ba544150fcd8a30"}
Sep 09 03:31:59.890138 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:31:59.890022    2576 generic.go:356] "Generic (PLEG): container finished" podID="03281339-40a3-4b4e-9440-18a8008be8fa" containerID="a59ef5365878d0afa2fd61977eaf8d735444f3a960dae0f573868a07ba5b0a7e" exitCode=0
Sep 09 03:31:59.890138 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:31:59.890044    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-autores-a2fc47-57857d79d5-vbw6n" event={"ID":"03281339-40a3-4b4e-9440-18a8008be8fa","Type":"ContainerDied","Data":"a59ef5365878d0afa2fd61977eaf8d735444f3a960dae0f573868a07ba5b0a7e"}
Sep 09 03:31:59.890138 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:31:59.890070    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-autores-a2fc47-57857d79d5-vbw6n"
Sep 09 03:31:59.890138 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:31:59.890085    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-autores-a2fc47-57857d79d5-vbw6n" event={"ID":"03281339-40a3-4b4e-9440-18a8008be8fa","Type":"ContainerDied","Data":"1b8601bb23aca731514e50aeb5041f012beeff193005f8cb49cb0570b9804bf8"}
Sep 09 03:31:59.892163 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:31:59.892143    2576 generic.go:356] "Generic (PLEG): container finished" podID="4893ff8b-d74c-475e-ab0b-7cf86256fa72" containerID="12f9f6f4ab68816fa44268c557d58a06e0dd8a1db1ab861deefd432d72ab7ced" exitCode=0
Sep 09 03:31:59.892257 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:31:59.892174    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-derive-ad343832-846957c78f-kx7px" event={"ID":"4893ff8b-d74c-475e-ab0b-7cf86256fa72","Type":"ContainerDied","Data":"12f9f6f4ab68816fa44268c557d58a06e0dd8a1db1ab861deefd432d72ab7ced"}
Sep 09 03:31:59.892257 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:31:59.892210    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-derive-ad343832-846957c78f-kx7px" event={"ID":"4893ff8b-d74c-475e-ab0b-7cf86256fa72","Type":"ContainerDied","Data":"35e513d8e0f9da76b4912b6bc89e8c0e62e44d756d3b53259b9bf7f877c553b5"}
Sep 09 03:31:59.892370 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:31:59.892272    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-derive-ad343832-846957c78f-kx7px"
Sep 09 03:31:59.897786 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:31:59.897688    2576 scope.go:117] "RemoveContainer" containerID="174b04dd483193fe2b17dbf2702846a3a303faf8b839d934a77c4e5e380688a0"
Sep 09 03:31:59.898053 ip-10-0-132-14 kubenswrapper[2576]: E0909 03:31:59.898031    2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"174b04dd483193fe2b17dbf2702846a3a303faf8b839d934a77c4e5e380688a0\": container with ID starting with 174b04dd483193fe2b17dbf2702846a3a303faf8b839d934a77c4e5e380688a0 not found: ID does not exist" containerID="174b04dd483193fe2b17dbf2702846a3a303faf8b839d934a77c4e5e380688a0"
Sep 09 03:31:59.898131 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:31:59.898064    2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"174b04dd483193fe2b17dbf2702846a3a303faf8b839d934a77c4e5e380688a0"} err="failed to get container status \"174b04dd483193fe2b17dbf2702846a3a303faf8b839d934a77c4e5e380688a0\": rpc error: code = NotFound desc = could not find container \"174b04dd483193fe2b17dbf2702846a3a303faf8b839d934a77c4e5e380688a0\": container with ID starting with 174b04dd483193fe2b17dbf2702846a3a303faf8b839d934a77c4e5e380688a0 not found: ID does not exist"
Sep 09 03:31:59.898131 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:31:59.898086    2576 scope.go:117] "RemoveContainer" containerID="f0b0badd73f452a7093302bd917882275647cfc46cb764b351abea8742c0368c"
Sep 09 03:31:59.907263 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:31:59.907245    2576 scope.go:117] "RemoveContainer" containerID="f0b0badd73f452a7093302bd917882275647cfc46cb764b351abea8742c0368c"
Sep 09 03:31:59.907663 ip-10-0-132-14 kubenswrapper[2576]: E0909 03:31:59.907644    2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f0b0badd73f452a7093302bd917882275647cfc46cb764b351abea8742c0368c\": container with ID starting with f0b0badd73f452a7093302bd917882275647cfc46cb764b351abea8742c0368c not found: ID does not exist" containerID="f0b0badd73f452a7093302bd917882275647cfc46cb764b351abea8742c0368c"
Sep 09 03:31:59.907745 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:31:59.907669    2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f0b0badd73f452a7093302bd917882275647cfc46cb764b351abea8742c0368c"} err="failed to get container status \"f0b0badd73f452a7093302bd917882275647cfc46cb764b351abea8742c0368c\": rpc error: code = NotFound desc = could not find container \"f0b0badd73f452a7093302bd917882275647cfc46cb764b351abea8742c0368c\": container with ID starting with f0b0badd73f452a7093302bd917882275647cfc46cb764b351abea8742c0368c not found: ID does not exist"
Sep 09 03:31:59.907745 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:31:59.907687    2576 scope.go:117] "RemoveContainer" containerID="070013acfd999dbebb84bc2c1f5a06400f76d1f2e1ccb84dde96ceb312ed21dd"
Sep 09 03:31:59.917240 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:31:59.917223    2576 scope.go:117] "RemoveContainer" containerID="070013acfd999dbebb84bc2c1f5a06400f76d1f2e1ccb84dde96ceb312ed21dd"
Sep 09 03:31:59.917553 ip-10-0-132-14 kubenswrapper[2576]: E0909 03:31:59.917528    2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"070013acfd999dbebb84bc2c1f5a06400f76d1f2e1ccb84dde96ceb312ed21dd\": container with ID starting with 070013acfd999dbebb84bc2c1f5a06400f76d1f2e1ccb84dde96ceb312ed21dd not found: ID does not exist" containerID="070013acfd999dbebb84bc2c1f5a06400f76d1f2e1ccb84dde96ceb312ed21dd"
Sep 09 03:31:59.917610 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:31:59.917551    2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"070013acfd999dbebb84bc2c1f5a06400f76d1f2e1ccb84dde96ceb312ed21dd"} err="failed to get container status \"070013acfd999dbebb84bc2c1f5a06400f76d1f2e1ccb84dde96ceb312ed21dd\": rpc error: code = NotFound desc = could not find container \"070013acfd999dbebb84bc2c1f5a06400f76d1f2e1ccb84dde96ceb312ed21dd\": container with ID starting with 070013acfd999dbebb84bc2c1f5a06400f76d1f2e1ccb84dde96ceb312ed21dd not found: ID does not exist"
Sep 09 03:31:59.917610 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:31:59.917566    2576 scope.go:117] "RemoveContainer" containerID="a59ef5365878d0afa2fd61977eaf8d735444f3a960dae0f573868a07ba5b0a7e"
Sep 09 03:31:59.922053 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:31:59.922030    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-derive-ad343832-846957c78f-kx7px"]
Sep 09 03:31:59.922235 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:31:59.922216    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-jz8vx\" (UniqueName: \"kubernetes.io/projected/1047ee3d-c020-44c2-948c-719a1e85ff72-kube-api-access-jz8vx\") pod \"1047ee3d-c020-44c2-948c-719a1e85ff72\" (UID: \"1047ee3d-c020-44c2-948c-719a1e85ff72\") "
Sep 09 03:31:59.922318 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:31:59.922254    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/03281339-40a3-4b4e-9440-18a8008be8fa-openshift-service-ca\") pod \"03281339-40a3-4b4e-9440-18a8008be8fa\" (UID: \"03281339-40a3-4b4e-9440-18a8008be8fa\") "
Sep 09 03:31:59.922318 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:31:59.922276    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/03281339-40a3-4b4e-9440-18a8008be8fa-metrics-certs\") pod \"03281339-40a3-4b4e-9440-18a8008be8fa\" (UID: \"03281339-40a3-4b4e-9440-18a8008be8fa\") "
Sep 09 03:31:59.922428 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:31:59.922320    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-2t2kn\" (UniqueName: \"kubernetes.io/projected/03281339-40a3-4b4e-9440-18a8008be8fa-kube-api-access-2t2kn\") pod \"03281339-40a3-4b4e-9440-18a8008be8fa\" (UID: \"03281339-40a3-4b4e-9440-18a8008be8fa\") "
Sep 09 03:31:59.922428 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:31:59.922366    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/1047ee3d-c020-44c2-948c-719a1e85ff72-plugins-config-volume\") pod \"1047ee3d-c020-44c2-948c-719a1e85ff72\" (UID: \"1047ee3d-c020-44c2-948c-719a1e85ff72\") "
Sep 09 03:31:59.922428 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:31:59.922383    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/03281339-40a3-4b4e-9440-18a8008be8fa-maas-api-tls\") pod \"03281339-40a3-4b4e-9440-18a8008be8fa\" (UID: \"03281339-40a3-4b4e-9440-18a8008be8fa\") "
Sep 09 03:31:59.922575 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:31:59.922449    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/82d8fbaa-1300-4bcc-be5d-77bf70dbed9c-plugins-config-volume\") pod \"82d8fbaa-1300-4bcc-be5d-77bf70dbed9c\" (UID: \"82d8fbaa-1300-4bcc-be5d-77bf70dbed9c\") "
Sep 09 03:31:59.922575 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:31:59.922485    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-j4wbf\" (UniqueName: \"kubernetes.io/projected/82d8fbaa-1300-4bcc-be5d-77bf70dbed9c-kube-api-access-j4wbf\") pod \"82d8fbaa-1300-4bcc-be5d-77bf70dbed9c\" (UID: \"82d8fbaa-1300-4bcc-be5d-77bf70dbed9c\") "
Sep 09 03:31:59.922671 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:31:59.922587    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/03281339-40a3-4b4e-9440-18a8008be8fa-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "03281339-40a3-4b4e-9440-18a8008be8fa" (UID: "03281339-40a3-4b4e-9440-18a8008be8fa"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 09 03:31:59.922833 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:31:59.922789    2576 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/03281339-40a3-4b4e-9440-18a8008be8fa-openshift-service-ca\") on node \"ip-10-0-132-14.ec2.internal\" DevicePath \"\""
Sep 09 03:31:59.923157 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:31:59.923071    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/82d8fbaa-1300-4bcc-be5d-77bf70dbed9c-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "82d8fbaa-1300-4bcc-be5d-77bf70dbed9c" (UID: "82d8fbaa-1300-4bcc-be5d-77bf70dbed9c"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 09 03:31:59.923157 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:31:59.923118    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1047ee3d-c020-44c2-948c-719a1e85ff72-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "1047ee3d-c020-44c2-948c-719a1e85ff72" (UID: "1047ee3d-c020-44c2-948c-719a1e85ff72"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 09 03:31:59.925086 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:31:59.925050    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/03281339-40a3-4b4e-9440-18a8008be8fa-kube-api-access-2t2kn" (OuterVolumeSpecName: "kube-api-access-2t2kn") pod "03281339-40a3-4b4e-9440-18a8008be8fa" (UID: "03281339-40a3-4b4e-9440-18a8008be8fa"). InnerVolumeSpecName "kube-api-access-2t2kn". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 09 03:31:59.925182 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:31:59.925163    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/03281339-40a3-4b4e-9440-18a8008be8fa-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "03281339-40a3-4b4e-9440-18a8008be8fa" (UID: "03281339-40a3-4b4e-9440-18a8008be8fa"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 09 03:31:59.925325 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:31:59.925303    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/82d8fbaa-1300-4bcc-be5d-77bf70dbed9c-kube-api-access-j4wbf" (OuterVolumeSpecName: "kube-api-access-j4wbf") pod "82d8fbaa-1300-4bcc-be5d-77bf70dbed9c" (UID: "82d8fbaa-1300-4bcc-be5d-77bf70dbed9c"). InnerVolumeSpecName "kube-api-access-j4wbf". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 09 03:31:59.926144 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:31:59.926124    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/03281339-40a3-4b4e-9440-18a8008be8fa-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "03281339-40a3-4b4e-9440-18a8008be8fa" (UID: "03281339-40a3-4b4e-9440-18a8008be8fa"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 09 03:31:59.926340 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:31:59.926319    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1047ee3d-c020-44c2-948c-719a1e85ff72-kube-api-access-jz8vx" (OuterVolumeSpecName: "kube-api-access-jz8vx") pod "1047ee3d-c020-44c2-948c-719a1e85ff72" (UID: "1047ee3d-c020-44c2-948c-719a1e85ff72"). InnerVolumeSpecName "kube-api-access-jz8vx". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 09 03:31:59.927214 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:31:59.927153    2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-derive-ad343832-846957c78f-kx7px"]
Sep 09 03:31:59.927302 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:31:59.927290    2576 scope.go:117] "RemoveContainer" containerID="a59ef5365878d0afa2fd61977eaf8d735444f3a960dae0f573868a07ba5b0a7e"
Sep 09 03:31:59.927565 ip-10-0-132-14 kubenswrapper[2576]: E0909 03:31:59.927550    2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a59ef5365878d0afa2fd61977eaf8d735444f3a960dae0f573868a07ba5b0a7e\": container with ID starting with a59ef5365878d0afa2fd61977eaf8d735444f3a960dae0f573868a07ba5b0a7e not found: ID does not exist" containerID="a59ef5365878d0afa2fd61977eaf8d735444f3a960dae0f573868a07ba5b0a7e"
Sep 09 03:31:59.927601 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:31:59.927573    2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a59ef5365878d0afa2fd61977eaf8d735444f3a960dae0f573868a07ba5b0a7e"} err="failed to get container status \"a59ef5365878d0afa2fd61977eaf8d735444f3a960dae0f573868a07ba5b0a7e\": rpc error: code = NotFound desc = could not find container \"a59ef5365878d0afa2fd61977eaf8d735444f3a960dae0f573868a07ba5b0a7e\": container with ID starting with a59ef5365878d0afa2fd61977eaf8d735444f3a960dae0f573868a07ba5b0a7e not found: ID does not exist"
Sep 09 03:31:59.927601 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:31:59.927589    2576 scope.go:117] "RemoveContainer" containerID="12f9f6f4ab68816fa44268c557d58a06e0dd8a1db1ab861deefd432d72ab7ced"
Sep 09 03:31:59.935015 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:31:59.935000    2576 scope.go:117] "RemoveContainer" containerID="12f9f6f4ab68816fa44268c557d58a06e0dd8a1db1ab861deefd432d72ab7ced"
Sep 09 03:31:59.935276 ip-10-0-132-14 kubenswrapper[2576]: E0909 03:31:59.935259    2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"12f9f6f4ab68816fa44268c557d58a06e0dd8a1db1ab861deefd432d72ab7ced\": container with ID starting with 12f9f6f4ab68816fa44268c557d58a06e0dd8a1db1ab861deefd432d72ab7ced not found: ID does not exist" containerID="12f9f6f4ab68816fa44268c557d58a06e0dd8a1db1ab861deefd432d72ab7ced"
Sep 09 03:31:59.935323 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:31:59.935283    2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"12f9f6f4ab68816fa44268c557d58a06e0dd8a1db1ab861deefd432d72ab7ced"} err="failed to get container status \"12f9f6f4ab68816fa44268c557d58a06e0dd8a1db1ab861deefd432d72ab7ced\": rpc error: code = NotFound desc = could not find container \"12f9f6f4ab68816fa44268c557d58a06e0dd8a1db1ab861deefd432d72ab7ced\": container with ID starting with 12f9f6f4ab68816fa44268c557d58a06e0dd8a1db1ab861deefd432d72ab7ced not found: ID does not exist"
Sep 09 03:31:59.941987 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:31:59.941963    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-ad343832-7d9bb989cd-sq755"]
Sep 09 03:31:59.950287 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:31:59.950267    2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-ad343832-7d9bb989cd-sq755"]
Sep 09 03:32:00.023475 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:32:00.023443    2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jz8vx\" (UniqueName: \"kubernetes.io/projected/1047ee3d-c020-44c2-948c-719a1e85ff72-kube-api-access-jz8vx\") on node \"ip-10-0-132-14.ec2.internal\" DevicePath \"\""
Sep 09 03:32:00.023648 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:32:00.023590    2576 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/03281339-40a3-4b4e-9440-18a8008be8fa-metrics-certs\") on node \"ip-10-0-132-14.ec2.internal\" DevicePath \"\""
Sep 09 03:32:00.023648 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:32:00.023613    2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2t2kn\" (UniqueName: \"kubernetes.io/projected/03281339-40a3-4b4e-9440-18a8008be8fa-kube-api-access-2t2kn\") on node \"ip-10-0-132-14.ec2.internal\" DevicePath \"\""
Sep 09 03:32:00.023648 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:32:00.023624    2576 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/1047ee3d-c020-44c2-948c-719a1e85ff72-plugins-config-volume\") on node \"ip-10-0-132-14.ec2.internal\" DevicePath \"\""
Sep 09 03:32:00.023648 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:32:00.023633    2576 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/03281339-40a3-4b4e-9440-18a8008be8fa-maas-api-tls\") on node \"ip-10-0-132-14.ec2.internal\" DevicePath \"\""
Sep 09 03:32:00.023796 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:32:00.023671    2576 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/82d8fbaa-1300-4bcc-be5d-77bf70dbed9c-plugins-config-volume\") on node \"ip-10-0-132-14.ec2.internal\" DevicePath \"\""
Sep 09 03:32:00.023796 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:32:00.023686    2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-j4wbf\" (UniqueName: \"kubernetes.io/projected/82d8fbaa-1300-4bcc-be5d-77bf70dbed9c-kube-api-access-j4wbf\") on node \"ip-10-0-132-14.ec2.internal\" DevicePath \"\""
Sep 09 03:32:00.222872 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:32:00.222843    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-autores-a2fc47-8fdd95594-hxg7w"]
Sep 09 03:32:00.229624 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:32:00.229598    2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-autores-a2fc47-8fdd95594-hxg7w"]
Sep 09 03:32:00.240735 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:32:00.240713    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-autores-a2fc47-57857d79d5-vbw6n"]
Sep 09 03:32:00.245244 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:32:00.245226    2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-autores-a2fc47-57857d79d5-vbw6n"]
Sep 09 03:32:00.257590 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:32:00.257571    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-autores-a2fc47-69fdd48ccf-wjj48"]
Sep 09 03:32:00.264220 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:32:00.264203    2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-autores-a2fc47-69fdd48ccf-wjj48"]
Sep 09 03:32:00.362084 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:32:00.362060    2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="03281339-40a3-4b4e-9440-18a8008be8fa" path="/var/lib/kubelet/pods/03281339-40a3-4b4e-9440-18a8008be8fa/volumes"
Sep 09 03:32:00.362406 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:32:00.362393    2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="1047ee3d-c020-44c2-948c-719a1e85ff72" path="/var/lib/kubelet/pods/1047ee3d-c020-44c2-948c-719a1e85ff72/volumes"
Sep 09 03:32:00.362720 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:32:00.362708    2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="1b31d5a7-3ba4-409d-8d2b-31b479dddbd7" path="/var/lib/kubelet/pods/1b31d5a7-3ba4-409d-8d2b-31b479dddbd7/volumes"
Sep 09 03:32:00.363357 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:32:00.363341    2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4893ff8b-d74c-475e-ab0b-7cf86256fa72" path="/var/lib/kubelet/pods/4893ff8b-d74c-475e-ab0b-7cf86256fa72/volumes"
Sep 09 03:32:00.363654 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:32:00.363642    2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="82d8fbaa-1300-4bcc-be5d-77bf70dbed9c" path="/var/lib/kubelet/pods/82d8fbaa-1300-4bcc-be5d-77bf70dbed9c/volumes"
Sep 09 03:32:12.466337 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:32:12.466287    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-explicit-958a3f-openshift-default-54b68b49f6-9q84l"]
Sep 09 03:32:12.466821 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:32:12.466683    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1047ee3d-c020-44c2-948c-719a1e85ff72" containerName="payload-pre-processing"
Sep 09 03:32:12.466821 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:32:12.466697    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="1047ee3d-c020-44c2-948c-719a1e85ff72" containerName="payload-pre-processing"
Sep 09 03:32:12.466821 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:32:12.466707    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4893ff8b-d74c-475e-ab0b-7cf86256fa72" containerName="payload-processing"
Sep 09 03:32:12.466821 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:32:12.466714    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="4893ff8b-d74c-475e-ab0b-7cf86256fa72" containerName="payload-processing"
Sep 09 03:32:12.466821 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:32:12.466733    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="82d8fbaa-1300-4bcc-be5d-77bf70dbed9c" containerName="payload-processing"
Sep 09 03:32:12.466821 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:32:12.466741    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="82d8fbaa-1300-4bcc-be5d-77bf70dbed9c" containerName="payload-processing"
Sep 09 03:32:12.466821 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:32:12.466760    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="03281339-40a3-4b4e-9440-18a8008be8fa" containerName="maas-api"
Sep 09 03:32:12.466821 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:32:12.466769    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="03281339-40a3-4b4e-9440-18a8008be8fa" containerName="maas-api"
Sep 09 03:32:12.466821 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:32:12.466781    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1b31d5a7-3ba4-409d-8d2b-31b479dddbd7" containerName="maas-api"
Sep 09 03:32:12.466821 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:32:12.466789    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="1b31d5a7-3ba4-409d-8d2b-31b479dddbd7" containerName="maas-api"
Sep 09 03:32:12.467316 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:32:12.466912    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="82d8fbaa-1300-4bcc-be5d-77bf70dbed9c" containerName="payload-processing"
Sep 09 03:32:12.467316 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:32:12.466940    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="1b31d5a7-3ba4-409d-8d2b-31b479dddbd7" containerName="maas-api"
Sep 09 03:32:12.467316 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:32:12.466950    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="03281339-40a3-4b4e-9440-18a8008be8fa" containerName="maas-api"
Sep 09 03:32:12.467316 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:32:12.466963    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="1047ee3d-c020-44c2-948c-719a1e85ff72" containerName="payload-pre-processing"
Sep 09 03:32:12.467316 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:32:12.466973    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="4893ff8b-d74c-475e-ab0b-7cf86256fa72" containerName="payload-processing"
Sep 09 03:32:12.471518 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:32:12.471493    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-explicit-958a3f-openshift-default-54b68b49f6-9q84l"
Sep 09 03:32:12.474267 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:32:12.474249    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-explicit-958a3f-openshift-default-dockercfg-4vbd2\""
Sep 09 03:32:12.481104 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:32:12.481081    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-explicit-958a3f-openshift-default-54b68b49f6-9q84l"]
Sep 09 03:32:12.534845 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:32:12.534796    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/eb8e971b-0bc7-42f6-927e-1db93a64ff57-istio-podinfo\") pod \"e2e-explicit-958a3f-openshift-default-54b68b49f6-9q84l\" (UID: \"eb8e971b-0bc7-42f6-927e-1db93a64ff57\") " pod="openshift-ingress/e2e-explicit-958a3f-openshift-default-54b68b49f6-9q84l"
Sep 09 03:32:12.534954 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:32:12.534865    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/eb8e971b-0bc7-42f6-927e-1db93a64ff57-workload-socket\") pod \"e2e-explicit-958a3f-openshift-default-54b68b49f6-9q84l\" (UID: \"eb8e971b-0bc7-42f6-927e-1db93a64ff57\") " pod="openshift-ingress/e2e-explicit-958a3f-openshift-default-54b68b49f6-9q84l"
Sep 09 03:32:12.534954 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:32:12.534892    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/eb8e971b-0bc7-42f6-927e-1db93a64ff57-istiod-ca-cert\") pod \"e2e-explicit-958a3f-openshift-default-54b68b49f6-9q84l\" (UID: \"eb8e971b-0bc7-42f6-927e-1db93a64ff57\") " pod="openshift-ingress/e2e-explicit-958a3f-openshift-default-54b68b49f6-9q84l"
Sep 09 03:32:12.534954 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:32:12.534917    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/eb8e971b-0bc7-42f6-927e-1db93a64ff57-istio-envoy\") pod \"e2e-explicit-958a3f-openshift-default-54b68b49f6-9q84l\" (UID: \"eb8e971b-0bc7-42f6-927e-1db93a64ff57\") " pod="openshift-ingress/e2e-explicit-958a3f-openshift-default-54b68b49f6-9q84l"
Sep 09 03:32:12.534954 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:32:12.534934    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/eb8e971b-0bc7-42f6-927e-1db93a64ff57-istio-token\") pod \"e2e-explicit-958a3f-openshift-default-54b68b49f6-9q84l\" (UID: \"eb8e971b-0bc7-42f6-927e-1db93a64ff57\") " pod="openshift-ingress/e2e-explicit-958a3f-openshift-default-54b68b49f6-9q84l"
Sep 09 03:32:12.535134 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:32:12.534981    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/eb8e971b-0bc7-42f6-927e-1db93a64ff57-workload-certs\") pod \"e2e-explicit-958a3f-openshift-default-54b68b49f6-9q84l\" (UID: \"eb8e971b-0bc7-42f6-927e-1db93a64ff57\") " pod="openshift-ingress/e2e-explicit-958a3f-openshift-default-54b68b49f6-9q84l"
Sep 09 03:32:12.535134 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:32:12.535037    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mn6t7\" (UniqueName: \"kubernetes.io/projected/eb8e971b-0bc7-42f6-927e-1db93a64ff57-kube-api-access-mn6t7\") pod \"e2e-explicit-958a3f-openshift-default-54b68b49f6-9q84l\" (UID: \"eb8e971b-0bc7-42f6-927e-1db93a64ff57\") " pod="openshift-ingress/e2e-explicit-958a3f-openshift-default-54b68b49f6-9q84l"
Sep 09 03:32:12.535134 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:32:12.535097    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/eb8e971b-0bc7-42f6-927e-1db93a64ff57-credential-socket\") pod \"e2e-explicit-958a3f-openshift-default-54b68b49f6-9q84l\" (UID: \"eb8e971b-0bc7-42f6-927e-1db93a64ff57\") " pod="openshift-ingress/e2e-explicit-958a3f-openshift-default-54b68b49f6-9q84l"
Sep 09 03:32:12.535134 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:32:12.535121    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/eb8e971b-0bc7-42f6-927e-1db93a64ff57-istio-data\") pod \"e2e-explicit-958a3f-openshift-default-54b68b49f6-9q84l\" (UID: \"eb8e971b-0bc7-42f6-927e-1db93a64ff57\") " pod="openshift-ingress/e2e-explicit-958a3f-openshift-default-54b68b49f6-9q84l"
Sep 09 03:32:12.636161 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:32:12.636129    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/eb8e971b-0bc7-42f6-927e-1db93a64ff57-workload-certs\") pod \"e2e-explicit-958a3f-openshift-default-54b68b49f6-9q84l\" (UID: \"eb8e971b-0bc7-42f6-927e-1db93a64ff57\") " pod="openshift-ingress/e2e-explicit-958a3f-openshift-default-54b68b49f6-9q84l"
Sep 09 03:32:12.636161 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:32:12.636166    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mn6t7\" (UniqueName: \"kubernetes.io/projected/eb8e971b-0bc7-42f6-927e-1db93a64ff57-kube-api-access-mn6t7\") pod \"e2e-explicit-958a3f-openshift-default-54b68b49f6-9q84l\" (UID: \"eb8e971b-0bc7-42f6-927e-1db93a64ff57\") " pod="openshift-ingress/e2e-explicit-958a3f-openshift-default-54b68b49f6-9q84l"
Sep 09 03:32:12.636362 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:32:12.636280    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/eb8e971b-0bc7-42f6-927e-1db93a64ff57-credential-socket\") pod \"e2e-explicit-958a3f-openshift-default-54b68b49f6-9q84l\" (UID: \"eb8e971b-0bc7-42f6-927e-1db93a64ff57\") " pod="openshift-ingress/e2e-explicit-958a3f-openshift-default-54b68b49f6-9q84l"
Sep 09 03:32:12.636362 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:32:12.636324    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/eb8e971b-0bc7-42f6-927e-1db93a64ff57-istio-data\") pod \"e2e-explicit-958a3f-openshift-default-54b68b49f6-9q84l\" (UID: \"eb8e971b-0bc7-42f6-927e-1db93a64ff57\") " pod="openshift-ingress/e2e-explicit-958a3f-openshift-default-54b68b49f6-9q84l"
Sep 09 03:32:12.636441 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:32:12.636377    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/eb8e971b-0bc7-42f6-927e-1db93a64ff57-istio-podinfo\") pod \"e2e-explicit-958a3f-openshift-default-54b68b49f6-9q84l\" (UID: \"eb8e971b-0bc7-42f6-927e-1db93a64ff57\") " pod="openshift-ingress/e2e-explicit-958a3f-openshift-default-54b68b49f6-9q84l"
Sep 09 03:32:12.636441 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:32:12.636417    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/eb8e971b-0bc7-42f6-927e-1db93a64ff57-workload-socket\") pod \"e2e-explicit-958a3f-openshift-default-54b68b49f6-9q84l\" (UID: \"eb8e971b-0bc7-42f6-927e-1db93a64ff57\") " pod="openshift-ingress/e2e-explicit-958a3f-openshift-default-54b68b49f6-9q84l"
Sep 09 03:32:12.636553 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:32:12.636438    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/eb8e971b-0bc7-42f6-927e-1db93a64ff57-istiod-ca-cert\") pod \"e2e-explicit-958a3f-openshift-default-54b68b49f6-9q84l\" (UID: \"eb8e971b-0bc7-42f6-927e-1db93a64ff57\") " pod="openshift-ingress/e2e-explicit-958a3f-openshift-default-54b68b49f6-9q84l"
Sep 09 03:32:12.636553 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:32:12.636480    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/eb8e971b-0bc7-42f6-927e-1db93a64ff57-istio-envoy\") pod \"e2e-explicit-958a3f-openshift-default-54b68b49f6-9q84l\" (UID: \"eb8e971b-0bc7-42f6-927e-1db93a64ff57\") " pod="openshift-ingress/e2e-explicit-958a3f-openshift-default-54b68b49f6-9q84l"
Sep 09 03:32:12.636553 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:32:12.636488    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/eb8e971b-0bc7-42f6-927e-1db93a64ff57-workload-certs\") pod \"e2e-explicit-958a3f-openshift-default-54b68b49f6-9q84l\" (UID: \"eb8e971b-0bc7-42f6-927e-1db93a64ff57\") " pod="openshift-ingress/e2e-explicit-958a3f-openshift-default-54b68b49f6-9q84l"
Sep 09 03:32:12.636553 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:32:12.636508    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/eb8e971b-0bc7-42f6-927e-1db93a64ff57-istio-token\") pod \"e2e-explicit-958a3f-openshift-default-54b68b49f6-9q84l\" (UID: \"eb8e971b-0bc7-42f6-927e-1db93a64ff57\") " pod="openshift-ingress/e2e-explicit-958a3f-openshift-default-54b68b49f6-9q84l"
Sep 09 03:32:12.636751 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:32:12.636602    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/eb8e971b-0bc7-42f6-927e-1db93a64ff57-credential-socket\") pod \"e2e-explicit-958a3f-openshift-default-54b68b49f6-9q84l\" (UID: \"eb8e971b-0bc7-42f6-927e-1db93a64ff57\") " pod="openshift-ingress/e2e-explicit-958a3f-openshift-default-54b68b49f6-9q84l"
Sep 09 03:32:12.636751 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:32:12.636683    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/eb8e971b-0bc7-42f6-927e-1db93a64ff57-istio-data\") pod \"e2e-explicit-958a3f-openshift-default-54b68b49f6-9q84l\" (UID: \"eb8e971b-0bc7-42f6-927e-1db93a64ff57\") " pod="openshift-ingress/e2e-explicit-958a3f-openshift-default-54b68b49f6-9q84l"
Sep 09 03:32:12.636906 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:32:12.636882    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/eb8e971b-0bc7-42f6-927e-1db93a64ff57-workload-socket\") pod \"e2e-explicit-958a3f-openshift-default-54b68b49f6-9q84l\" (UID: \"eb8e971b-0bc7-42f6-927e-1db93a64ff57\") " pod="openshift-ingress/e2e-explicit-958a3f-openshift-default-54b68b49f6-9q84l"
Sep 09 03:32:12.637121 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:32:12.637103    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/eb8e971b-0bc7-42f6-927e-1db93a64ff57-istiod-ca-cert\") pod \"e2e-explicit-958a3f-openshift-default-54b68b49f6-9q84l\" (UID: \"eb8e971b-0bc7-42f6-927e-1db93a64ff57\") " pod="openshift-ingress/e2e-explicit-958a3f-openshift-default-54b68b49f6-9q84l"
Sep 09 03:32:12.638629 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:32:12.638604    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/eb8e971b-0bc7-42f6-927e-1db93a64ff57-istio-envoy\") pod \"e2e-explicit-958a3f-openshift-default-54b68b49f6-9q84l\" (UID: \"eb8e971b-0bc7-42f6-927e-1db93a64ff57\") " pod="openshift-ingress/e2e-explicit-958a3f-openshift-default-54b68b49f6-9q84l"
Sep 09 03:32:12.639109 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:32:12.639087    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/eb8e971b-0bc7-42f6-927e-1db93a64ff57-istio-podinfo\") pod \"e2e-explicit-958a3f-openshift-default-54b68b49f6-9q84l\" (UID: \"eb8e971b-0bc7-42f6-927e-1db93a64ff57\") " pod="openshift-ingress/e2e-explicit-958a3f-openshift-default-54b68b49f6-9q84l"
Sep 09 03:32:12.644835 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:32:12.644795    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mn6t7\" (UniqueName: \"kubernetes.io/projected/eb8e971b-0bc7-42f6-927e-1db93a64ff57-kube-api-access-mn6t7\") pod \"e2e-explicit-958a3f-openshift-default-54b68b49f6-9q84l\" (UID: \"eb8e971b-0bc7-42f6-927e-1db93a64ff57\") " pod="openshift-ingress/e2e-explicit-958a3f-openshift-default-54b68b49f6-9q84l"
Sep 09 03:32:12.645459 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:32:12.645432    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/eb8e971b-0bc7-42f6-927e-1db93a64ff57-istio-token\") pod \"e2e-explicit-958a3f-openshift-default-54b68b49f6-9q84l\" (UID: \"eb8e971b-0bc7-42f6-927e-1db93a64ff57\") " pod="openshift-ingress/e2e-explicit-958a3f-openshift-default-54b68b49f6-9q84l"
Sep 09 03:32:12.782553 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:32:12.782519    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-explicit-958a3f-openshift-default-54b68b49f6-9q84l"
Sep 09 03:32:13.058825 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:32:13.058787    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-explicit-958a3f-openshift-default-54b68b49f6-9q84l"]
Sep 09 03:32:13.060791 ip-10-0-132-14 kubenswrapper[2576]: W0909 03:32:13.060760    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podeb8e971b_0bc7_42f6_927e_1db93a64ff57.slice/crio-f7d4b7e53290a825cbcc938dbd7982c2c09a3e4578ceb0d146a4c497756aac1c WatchSource:0}: Error finding container f7d4b7e53290a825cbcc938dbd7982c2c09a3e4578ceb0d146a4c497756aac1c: Status 404 returned error can't find the container with id f7d4b7e53290a825cbcc938dbd7982c2c09a3e4578ceb0d146a4c497756aac1c
Sep 09 03:32:13.966586 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:32:13.966554    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-explicit-958a3f-openshift-default-54b68b49f6-9q84l" event={"ID":"eb8e971b-0bc7-42f6-927e-1db93a64ff57","Type":"ContainerStarted","Data":"f7d4b7e53290a825cbcc938dbd7982c2c09a3e4578ceb0d146a4c497756aac1c"}
Sep 09 03:32:17.442694 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:32:17.442658    2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892148Ki","pods":"250"}
Sep 09 03:32:17.442994 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:32:17.442730    2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892148Ki","pods":"250"}
Sep 09 03:32:17.442994 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:32:17.442760    2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892148Ki","pods":"250"}
Sep 09 03:32:17.987615 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:32:17.987572    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-explicit-958a3f-openshift-default-54b68b49f6-9q84l" event={"ID":"eb8e971b-0bc7-42f6-927e-1db93a64ff57","Type":"ContainerStarted","Data":"132098aa8f0bcf17cd1ab134e3a84d44621777ac16dac7dbc42f8505b0248aa3"}
Sep 09 03:32:18.012336 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:32:18.012289    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-explicit-958a3f-openshift-default-54b68b49f6-9q84l" podStartSLOduration=1.633137711 podStartE2EDuration="6.012272675s" totalImagesPullingTime="4.379134964s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 03:32:12 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 03:32:18.010888783 +0000 UTC m=+1152.152374841" watchObservedRunningTime="2026-09-09 03:32:18.012272675 +0000 UTC m=+1152.153758751"
Sep 09 03:32:18.783406 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:32:18.783377    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-explicit-958a3f-openshift-default-54b68b49f6-9q84l"
Sep 09 03:32:18.788912 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:32:18.788877    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-explicit-958a3f-openshift-default-54b68b49f6-9q84l"
Sep 09 03:32:18.992168 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:32:18.992138    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-explicit-958a3f-openshift-default-54b68b49f6-9q84l"
Sep 09 03:32:18.993117 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:32:18.993102    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-explicit-958a3f-openshift-default-54b68b49f6-9q84l"
Sep 09 03:32:21.843081 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:32:21.843049    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-explicit-958a3f-696775c758-lqms9"]
Sep 09 03:32:21.846757 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:32:21.846736    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-explicit-958a3f-696775c758-lqms9"
Sep 09 03:32:21.849523 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:32:21.849503    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-explicit-958a3f\""
Sep 09 03:32:21.862828 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:32:21.862783    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-explicit-958a3f-696775c758-lqms9"]
Sep 09 03:32:21.876517 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:32:21.876489    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-explicit-958a3f-7bb8558bcc-z4w2c"]
Sep 09 03:32:21.879759 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:32:21.879741    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-explicit-958a3f-7bb8558bcc-z4w2c"
Sep 09 03:32:21.884453 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:32:21.884436    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-explicit-958a3f-dockercfg-94kr4\""
Sep 09 03:32:21.894316 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:32:21.894297    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-explicit-958a3f-7bb8558bcc-z4w2c"]
Sep 09 03:32:21.914551 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:32:21.914524    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-997wc\" (UniqueName: \"kubernetes.io/projected/a2a8ac26-4f2b-4c51-8dc0-402daf9e072d-kube-api-access-997wc\") pod \"payload-pre-processing-e2e-explicit-958a3f-696775c758-lqms9\" (UID: \"a2a8ac26-4f2b-4c51-8dc0-402daf9e072d\") " pod="openshift-ingress/payload-pre-processing-e2e-explicit-958a3f-696775c758-lqms9"
Sep 09 03:32:21.914665 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:32:21.914582    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a2a8ac26-4f2b-4c51-8dc0-402daf9e072d-plugins-config-volume\") pod \"payload-pre-processing-e2e-explicit-958a3f-696775c758-lqms9\" (UID: \"a2a8ac26-4f2b-4c51-8dc0-402daf9e072d\") " pod="openshift-ingress/payload-pre-processing-e2e-explicit-958a3f-696775c758-lqms9"
Sep 09 03:32:22.016109 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:32:22.016075    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cpgxr\" (UniqueName: \"kubernetes.io/projected/b180cc10-3f98-4819-8563-87c151318982-kube-api-access-cpgxr\") pod \"payload-processing-e2e-explicit-958a3f-7bb8558bcc-z4w2c\" (UID: \"b180cc10-3f98-4819-8563-87c151318982\") " pod="openshift-ingress/payload-processing-e2e-explicit-958a3f-7bb8558bcc-z4w2c"
Sep 09 03:32:22.016299 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:32:22.016136    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a2a8ac26-4f2b-4c51-8dc0-402daf9e072d-plugins-config-volume\") pod \"payload-pre-processing-e2e-explicit-958a3f-696775c758-lqms9\" (UID: \"a2a8ac26-4f2b-4c51-8dc0-402daf9e072d\") " pod="openshift-ingress/payload-pre-processing-e2e-explicit-958a3f-696775c758-lqms9"
Sep 09 03:32:22.016299 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:32:22.016237    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b180cc10-3f98-4819-8563-87c151318982-plugins-config-volume\") pod \"payload-processing-e2e-explicit-958a3f-7bb8558bcc-z4w2c\" (UID: \"b180cc10-3f98-4819-8563-87c151318982\") " pod="openshift-ingress/payload-processing-e2e-explicit-958a3f-7bb8558bcc-z4w2c"
Sep 09 03:32:22.016403 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:32:22.016308    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-997wc\" (UniqueName: \"kubernetes.io/projected/a2a8ac26-4f2b-4c51-8dc0-402daf9e072d-kube-api-access-997wc\") pod \"payload-pre-processing-e2e-explicit-958a3f-696775c758-lqms9\" (UID: \"a2a8ac26-4f2b-4c51-8dc0-402daf9e072d\") " pod="openshift-ingress/payload-pre-processing-e2e-explicit-958a3f-696775c758-lqms9"
Sep 09 03:32:22.016766 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:32:22.016749    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a2a8ac26-4f2b-4c51-8dc0-402daf9e072d-plugins-config-volume\") pod \"payload-pre-processing-e2e-explicit-958a3f-696775c758-lqms9\" (UID: \"a2a8ac26-4f2b-4c51-8dc0-402daf9e072d\") " pod="openshift-ingress/payload-pre-processing-e2e-explicit-958a3f-696775c758-lqms9"
Sep 09 03:32:22.027172 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:32:22.027142    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-997wc\" (UniqueName: \"kubernetes.io/projected/a2a8ac26-4f2b-4c51-8dc0-402daf9e072d-kube-api-access-997wc\") pod \"payload-pre-processing-e2e-explicit-958a3f-696775c758-lqms9\" (UID: \"a2a8ac26-4f2b-4c51-8dc0-402daf9e072d\") " pod="openshift-ingress/payload-pre-processing-e2e-explicit-958a3f-696775c758-lqms9"
Sep 09 03:32:22.117771 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:32:22.117696    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cpgxr\" (UniqueName: \"kubernetes.io/projected/b180cc10-3f98-4819-8563-87c151318982-kube-api-access-cpgxr\") pod \"payload-processing-e2e-explicit-958a3f-7bb8558bcc-z4w2c\" (UID: \"b180cc10-3f98-4819-8563-87c151318982\") " pod="openshift-ingress/payload-processing-e2e-explicit-958a3f-7bb8558bcc-z4w2c"
Sep 09 03:32:22.117944 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:32:22.117778    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b180cc10-3f98-4819-8563-87c151318982-plugins-config-volume\") pod \"payload-processing-e2e-explicit-958a3f-7bb8558bcc-z4w2c\" (UID: \"b180cc10-3f98-4819-8563-87c151318982\") " pod="openshift-ingress/payload-processing-e2e-explicit-958a3f-7bb8558bcc-z4w2c"
Sep 09 03:32:22.118400 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:32:22.118380    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b180cc10-3f98-4819-8563-87c151318982-plugins-config-volume\") pod \"payload-processing-e2e-explicit-958a3f-7bb8558bcc-z4w2c\" (UID: \"b180cc10-3f98-4819-8563-87c151318982\") " pod="openshift-ingress/payload-processing-e2e-explicit-958a3f-7bb8558bcc-z4w2c"
Sep 09 03:32:22.127736 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:32:22.127712    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cpgxr\" (UniqueName: \"kubernetes.io/projected/b180cc10-3f98-4819-8563-87c151318982-kube-api-access-cpgxr\") pod \"payload-processing-e2e-explicit-958a3f-7bb8558bcc-z4w2c\" (UID: \"b180cc10-3f98-4819-8563-87c151318982\") " pod="openshift-ingress/payload-processing-e2e-explicit-958a3f-7bb8558bcc-z4w2c"
Sep 09 03:32:22.156516 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:32:22.156486    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-explicit-958a3f-696775c758-lqms9"
Sep 09 03:32:22.189787 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:32:22.189759    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-explicit-958a3f-7bb8558bcc-z4w2c"
Sep 09 03:32:22.432594 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:32:22.432573    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-explicit-958a3f-696775c758-lqms9"]
Sep 09 03:32:22.434219 ip-10-0-132-14 kubenswrapper[2576]: W0909 03:32:22.434188    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda2a8ac26_4f2b_4c51_8dc0_402daf9e072d.slice/crio-6336eda00354d4c7c7a156eb3627f4e388b18091907c8ed02f54bc695299497f WatchSource:0}: Error finding container 6336eda00354d4c7c7a156eb3627f4e388b18091907c8ed02f54bc695299497f: Status 404 returned error can't find the container with id 6336eda00354d4c7c7a156eb3627f4e388b18091907c8ed02f54bc695299497f
Sep 09 03:32:22.471457 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:32:22.471431    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-explicit-958a3f-7bb8558bcc-z4w2c"]
Sep 09 03:32:22.473457 ip-10-0-132-14 kubenswrapper[2576]: W0909 03:32:22.473420    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb180cc10_3f98_4819_8563_87c151318982.slice/crio-07754e407b4fa29184da70b0fc3c4ba6cfd481cc023fac2d810701aaea36afa4 WatchSource:0}: Error finding container 07754e407b4fa29184da70b0fc3c4ba6cfd481cc023fac2d810701aaea36afa4: Status 404 returned error can't find the container with id 07754e407b4fa29184da70b0fc3c4ba6cfd481cc023fac2d810701aaea36afa4
Sep 09 03:32:23.016531 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:32:23.016491    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-explicit-958a3f-696775c758-lqms9" event={"ID":"a2a8ac26-4f2b-4c51-8dc0-402daf9e072d","Type":"ContainerStarted","Data":"4ce0c35b20a2d461ab01fe758cb97155e2ea10ab237dcc28c480b0aee71b5c76"}
Sep 09 03:32:23.017038 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:32:23.016572    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-explicit-958a3f-696775c758-lqms9"
Sep 09 03:32:23.017038 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:32:23.016597    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-explicit-958a3f-696775c758-lqms9" event={"ID":"a2a8ac26-4f2b-4c51-8dc0-402daf9e072d","Type":"ContainerStarted","Data":"6336eda00354d4c7c7a156eb3627f4e388b18091907c8ed02f54bc695299497f"}
Sep 09 03:32:23.024676 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:32:23.024645    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-explicit-958a3f-7bb8558bcc-z4w2c" event={"ID":"b180cc10-3f98-4819-8563-87c151318982","Type":"ContainerStarted","Data":"85458fc597e599c86986dd3c9dcb603d4c171e7a2ac222c60cfaad8567f94948"}
Sep 09 03:32:23.024847 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:32:23.024735    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-explicit-958a3f-7bb8558bcc-z4w2c"
Sep 09 03:32:23.024847 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:32:23.024754    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-explicit-958a3f-7bb8558bcc-z4w2c" event={"ID":"b180cc10-3f98-4819-8563-87c151318982","Type":"ContainerStarted","Data":"07754e407b4fa29184da70b0fc3c4ba6cfd481cc023fac2d810701aaea36afa4"}
Sep 09 03:32:23.036191 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:32:23.036136    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-explicit-958a3f-696775c758-lqms9" podStartSLOduration=2.036118509 podStartE2EDuration="2.036118509s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 03:32:21 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 03:32:23.033763468 +0000 UTC m=+1157.175249547" watchObservedRunningTime="2026-09-09 03:32:23.036118509 +0000 UTC m=+1157.177604587"
Sep 09 03:32:23.052980 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:32:23.052943    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-explicit-958a3f-7bb8558bcc-z4w2c" podStartSLOduration=2.052930666 podStartE2EDuration="2.052930666s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 03:32:21 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 03:32:23.052192722 +0000 UTC m=+1157.193678793" watchObservedRunningTime="2026-09-09 03:32:23.052930666 +0000 UTC m=+1157.194416742"
Sep 09 03:32:34.031856 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:32:34.031792    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-explicit-958a3f-696775c758-lqms9"
Sep 09 03:32:34.032389 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:32:34.031896    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-explicit-958a3f-7bb8558bcc-z4w2c"
Sep 09 03:32:34.502792 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:32:34.501848    2576 patch_prober.go:28] interesting pod/e2e-explicit-958a3f-openshift-default-54b68b49f6-9q84l container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.44:15021/healthz/ready\": read tcp 10.133.0.2:45200->10.133.0.44:15021: read: connection reset by peer" start-of-body=
Sep 09 03:32:34.502792 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:32:34.501927    2576 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-explicit-958a3f-openshift-default-54b68b49f6-9q84l" podUID="eb8e971b-0bc7-42f6-927e-1db93a64ff57" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.44:15021/healthz/ready\": read tcp 10.133.0.2:45200->10.133.0.44:15021: read: connection reset by peer"
Sep 09 03:32:35.088117 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:32:35.088084    2576 generic.go:356] "Generic (PLEG): container finished" podID="eb8e971b-0bc7-42f6-927e-1db93a64ff57" containerID="132098aa8f0bcf17cd1ab134e3a84d44621777ac16dac7dbc42f8505b0248aa3" exitCode=0
Sep 09 03:32:35.088481 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:32:35.088161    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-explicit-958a3f-openshift-default-54b68b49f6-9q84l" event={"ID":"eb8e971b-0bc7-42f6-927e-1db93a64ff57","Type":"ContainerDied","Data":"132098aa8f0bcf17cd1ab134e3a84d44621777ac16dac7dbc42f8505b0248aa3"}
Sep 09 03:32:35.088611 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:32:35.088595    2576 scope.go:117] "RemoveContainer" containerID="132098aa8f0bcf17cd1ab134e3a84d44621777ac16dac7dbc42f8505b0248aa3"
Sep 09 03:32:35.090603 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:32:35.090574    2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892148Ki","pods":"250"}
Sep 09 03:32:35.090733 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:32:35.090669    2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892148Ki","pods":"250"}
Sep 09 03:32:35.090733 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:32:35.090718    2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892148Ki","pods":"250"}
Sep 09 03:32:36.095321 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:32:36.095290    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-explicit-958a3f-openshift-default-54b68b49f6-9q84l" event={"ID":"eb8e971b-0bc7-42f6-927e-1db93a64ff57","Type":"ContainerStarted","Data":"57afd9a78b830acd52ddb27c48c49b47305cde90c8ee0c29b58be0905745887e"}
Sep 09 03:32:36.782985 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:32:36.782948    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-explicit-958a3f-openshift-default-54b68b49f6-9q84l"
Sep 09 03:32:36.783215 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:32:36.783193    2576 patch_prober.go:28] interesting pod/e2e-explicit-958a3f-openshift-default-54b68b49f6-9q84l container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.44:15021/healthz/ready\": dial tcp 10.133.0.44:15021: connect: connection refused" start-of-body=
Sep 09 03:32:36.783274 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:32:36.783254    2576 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-explicit-958a3f-openshift-default-54b68b49f6-9q84l" podUID="eb8e971b-0bc7-42f6-927e-1db93a64ff57" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.44:15021/healthz/ready\": dial tcp 10.133.0.44:15021: connect: connection refused"
Sep 09 03:32:37.783425 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:32:37.783394    2576 patch_prober.go:28] interesting pod/e2e-explicit-958a3f-openshift-default-54b68b49f6-9q84l container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.44:15021/healthz/ready\": dial tcp 10.133.0.44:15021: connect: connection refused" start-of-body=
Sep 09 03:32:37.783829 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:32:37.783459    2576 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-explicit-958a3f-openshift-default-54b68b49f6-9q84l" podUID="eb8e971b-0bc7-42f6-927e-1db93a64ff57" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.44:15021/healthz/ready\": dial tcp 10.133.0.44:15021: connect: connection refused"
Sep 09 03:32:38.783513 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:32:38.783476    2576 patch_prober.go:28] interesting pod/e2e-explicit-958a3f-openshift-default-54b68b49f6-9q84l container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.44:15021/healthz/ready\": dial tcp 10.133.0.44:15021: connect: connection refused" start-of-body=
Sep 09 03:32:38.783939 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:32:38.783543    2576 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-explicit-958a3f-openshift-default-54b68b49f6-9q84l" podUID="eb8e971b-0bc7-42f6-927e-1db93a64ff57" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.44:15021/healthz/ready\": dial tcp 10.133.0.44:15021: connect: connection refused"
Sep 09 03:32:39.783165 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:32:39.783123    2576 patch_prober.go:28] interesting pod/e2e-explicit-958a3f-openshift-default-54b68b49f6-9q84l container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.44:15021/healthz/ready\": dial tcp 10.133.0.44:15021: connect: connection refused" start-of-body=
Sep 09 03:32:39.783338 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:32:39.783182    2576 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-explicit-958a3f-openshift-default-54b68b49f6-9q84l" podUID="eb8e971b-0bc7-42f6-927e-1db93a64ff57" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.44:15021/healthz/ready\": dial tcp 10.133.0.44:15021: connect: connection refused"
Sep 09 03:32:40.783191 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:32:40.783156    2576 patch_prober.go:28] interesting pod/e2e-explicit-958a3f-openshift-default-54b68b49f6-9q84l container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.44:15021/healthz/ready\": dial tcp 10.133.0.44:15021: connect: connection refused" start-of-body=
Sep 09 03:32:40.783564 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:32:40.783212    2576 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-explicit-958a3f-openshift-default-54b68b49f6-9q84l" podUID="eb8e971b-0bc7-42f6-927e-1db93a64ff57" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.44:15021/healthz/ready\": dial tcp 10.133.0.44:15021: connect: connection refused"
Sep 09 03:32:41.787328 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:32:41.787296    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-explicit-958a3f-openshift-default-54b68b49f6-9q84l"
Sep 09 03:32:41.787784 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:32:41.787713    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-explicit-958a3f-openshift-default-54b68b49f6-9q84l"
Sep 09 03:32:41.788511 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:32:41.788491    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-explicit-958a3f-openshift-default-54b68b49f6-9q84l"
Sep 09 03:32:53.074734 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:32:53.074695    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-explicit-958a3f-mwk98"]
Sep 09 03:32:53.082533 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:32:53.082507    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-explicit-958a3f-mwk98"
Sep 09 03:32:53.086065 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:32:53.086036    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-w7q9f\""
Sep 09 03:32:53.087630 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:32:53.087591    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-explicit-958a3f-mwk98"]
Sep 09 03:32:53.191966 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:32:53.191927    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/efa2010e-640e-433f-9915-438ab7785b2f-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-explicit-958a3f-mwk98\" (UID: \"efa2010e-640e-433f-9915-438ab7785b2f\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-explicit-958a3f-mwk98"
Sep 09 03:32:53.293062 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:32:53.293023    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/efa2010e-640e-433f-9915-438ab7785b2f-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-explicit-958a3f-mwk98\" (UID: \"efa2010e-640e-433f-9915-438ab7785b2f\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-explicit-958a3f-mwk98"
Sep 09 03:32:53.293702 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:32:53.293674    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/efa2010e-640e-433f-9915-438ab7785b2f-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-explicit-958a3f-mwk98\" (UID: \"efa2010e-640e-433f-9915-438ab7785b2f\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-explicit-958a3f-mwk98"
Sep 09 03:32:53.395457 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:32:53.395354    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-explicit-958a3f-mwk98"
Sep 09 03:32:53.672042 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:32:53.672008    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-explicit-958a3f-mwk98"]
Sep 09 03:32:53.673742 ip-10-0-132-14 kubenswrapper[2576]: W0909 03:32:53.673713    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podefa2010e_640e_433f_9915_438ab7785b2f.slice/crio-f40e60c4a084346df917f32bf5ad85eba49634dab4a826ae2ae381ffccccfa6e WatchSource:0}: Error finding container f40e60c4a084346df917f32bf5ad85eba49634dab4a826ae2ae381ffccccfa6e: Status 404 returned error can't find the container with id f40e60c4a084346df917f32bf5ad85eba49634dab4a826ae2ae381ffccccfa6e
Sep 09 03:32:54.183509 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:32:54.183460    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-explicit-958a3f-mwk98" event={"ID":"efa2010e-640e-433f-9915-438ab7785b2f","Type":"ContainerStarted","Data":"a6aca5da470c3be59442068b14ae9b1e3a884769ce75c312212ab0c6edf2f5b1"}
Sep 09 03:32:54.183988 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:32:54.183527    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-explicit-958a3f-mwk98" event={"ID":"efa2010e-640e-433f-9915-438ab7785b2f","Type":"ContainerStarted","Data":"f40e60c4a084346df917f32bf5ad85eba49634dab4a826ae2ae381ffccccfa6e"}
Sep 09 03:32:54.201135 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:32:54.201029    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-explicit-958a3f-mwk98" podStartSLOduration=1.201012449 podStartE2EDuration="1.201012449s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 03:32:53 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 03:32:54.199663283 +0000 UTC m=+1188.341149356" watchObservedRunningTime="2026-09-09 03:32:54.201012449 +0000 UTC m=+1188.342498526"
Sep 09 03:32:55.190499 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:32:55.190464    2576 generic.go:356] "Generic (PLEG): container finished" podID="efa2010e-640e-433f-9915-438ab7785b2f" containerID="a6aca5da470c3be59442068b14ae9b1e3a884769ce75c312212ab0c6edf2f5b1" exitCode=0
Sep 09 03:32:55.190924 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:32:55.190514    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-explicit-958a3f-mwk98" event={"ID":"efa2010e-640e-433f-9915-438ab7785b2f","Type":"ContainerDied","Data":"a6aca5da470c3be59442068b14ae9b1e3a884769ce75c312212ab0c6edf2f5b1"}
Sep 09 03:32:56.343771 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:32:56.343747    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-explicit-958a3f-mwk98"
Sep 09 03:32:56.421678 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:32:56.421650    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/efa2010e-640e-433f-9915-438ab7785b2f-maas-api-service-ca\") pod \"efa2010e-640e-433f-9915-438ab7785b2f\" (UID: \"efa2010e-640e-433f-9915-438ab7785b2f\") "
Sep 09 03:32:56.422056 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:32:56.422027    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/efa2010e-640e-433f-9915-438ab7785b2f-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "efa2010e-640e-433f-9915-438ab7785b2f" (UID: "efa2010e-640e-433f-9915-438ab7785b2f"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 09 03:32:56.523072 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:32:56.523038    2576 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/efa2010e-640e-433f-9915-438ab7785b2f-maas-api-service-ca\") on node \"ip-10-0-132-14.ec2.internal\" DevicePath \"\""
Sep 09 03:32:57.201165 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:32:57.201132    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-explicit-958a3f-mwk98"
Sep 09 03:32:57.201351 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:32:57.201134    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-explicit-958a3f-mwk98" event={"ID":"efa2010e-640e-433f-9915-438ab7785b2f","Type":"ContainerDied","Data":"f40e60c4a084346df917f32bf5ad85eba49634dab4a826ae2ae381ffccccfa6e"}
Sep 09 03:32:57.201351 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:32:57.201295    2576 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="f40e60c4a084346df917f32bf5ad85eba49634dab4a826ae2ae381ffccccfa6e"
Sep 09 03:33:03.741295 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:33:03.741260    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-explicit-958a3f-7bb8558bcc-z4w2c"]
Sep 09 03:33:03.741957 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:33:03.741467    2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-explicit-958a3f-7bb8558bcc-z4w2c" podUID="b180cc10-3f98-4819-8563-87c151318982" containerName="payload-processing" containerID="cri-o://85458fc597e599c86986dd3c9dcb603d4c171e7a2ac222c60cfaad8567f94948" gracePeriod=30
Sep 09 03:33:03.751291 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:33:03.751263    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-explicit-958a3f-696775c758-lqms9"]
Sep 09 03:33:03.751589 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:33:03.751524    2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-explicit-958a3f-696775c758-lqms9" podUID="a2a8ac26-4f2b-4c51-8dc0-402daf9e072d" containerName="payload-pre-processing" containerID="cri-o://4ce0c35b20a2d461ab01fe758cb97155e2ea10ab237dcc28c480b0aee71b5c76" gracePeriod=30
Sep 09 03:33:03.969574 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:33:03.969544    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-explicit-958a3f-mwk98"]
Sep 09 03:33:03.975695 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:33:03.975669    2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-explicit-958a3f-mwk98"]
Sep 09 03:33:04.023725 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:33:04.023701    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-explicit-958a3f-7bb8558bcc-z4w2c"
Sep 09 03:33:04.026915 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:33:04.026893    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-explicit-958a3f-696775c758-lqms9"
Sep 09 03:33:04.088392 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:33:04.084864    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b180cc10-3f98-4819-8563-87c151318982-plugins-config-volume\") pod \"b180cc10-3f98-4819-8563-87c151318982\" (UID: \"b180cc10-3f98-4819-8563-87c151318982\") "
Sep 09 03:33:04.088392 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:33:04.084999    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-997wc\" (UniqueName: \"kubernetes.io/projected/a2a8ac26-4f2b-4c51-8dc0-402daf9e072d-kube-api-access-997wc\") pod \"a2a8ac26-4f2b-4c51-8dc0-402daf9e072d\" (UID: \"a2a8ac26-4f2b-4c51-8dc0-402daf9e072d\") "
Sep 09 03:33:04.088392 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:33:04.085046    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-cpgxr\" (UniqueName: \"kubernetes.io/projected/b180cc10-3f98-4819-8563-87c151318982-kube-api-access-cpgxr\") pod \"b180cc10-3f98-4819-8563-87c151318982\" (UID: \"b180cc10-3f98-4819-8563-87c151318982\") "
Sep 09 03:33:04.088392 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:33:04.085159    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a2a8ac26-4f2b-4c51-8dc0-402daf9e072d-plugins-config-volume\") pod \"a2a8ac26-4f2b-4c51-8dc0-402daf9e072d\" (UID: \"a2a8ac26-4f2b-4c51-8dc0-402daf9e072d\") "
Sep 09 03:33:04.088392 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:33:04.086290    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a2a8ac26-4f2b-4c51-8dc0-402daf9e072d-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "a2a8ac26-4f2b-4c51-8dc0-402daf9e072d" (UID: "a2a8ac26-4f2b-4c51-8dc0-402daf9e072d"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 09 03:33:04.088392 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:33:04.086674    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b180cc10-3f98-4819-8563-87c151318982-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "b180cc10-3f98-4819-8563-87c151318982" (UID: "b180cc10-3f98-4819-8563-87c151318982"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 09 03:33:04.092108 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:33:04.091974    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a2a8ac26-4f2b-4c51-8dc0-402daf9e072d-kube-api-access-997wc" (OuterVolumeSpecName: "kube-api-access-997wc") pod "a2a8ac26-4f2b-4c51-8dc0-402daf9e072d" (UID: "a2a8ac26-4f2b-4c51-8dc0-402daf9e072d"). InnerVolumeSpecName "kube-api-access-997wc". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 09 03:33:04.094315 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:33:04.094285    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b180cc10-3f98-4819-8563-87c151318982-kube-api-access-cpgxr" (OuterVolumeSpecName: "kube-api-access-cpgxr") pod "b180cc10-3f98-4819-8563-87c151318982" (UID: "b180cc10-3f98-4819-8563-87c151318982"). InnerVolumeSpecName "kube-api-access-cpgxr". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 09 03:33:04.186373 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:33:04.186331    2576 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b180cc10-3f98-4819-8563-87c151318982-plugins-config-volume\") on node \"ip-10-0-132-14.ec2.internal\" DevicePath \"\""
Sep 09 03:33:04.186373 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:33:04.186366    2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-997wc\" (UniqueName: \"kubernetes.io/projected/a2a8ac26-4f2b-4c51-8dc0-402daf9e072d-kube-api-access-997wc\") on node \"ip-10-0-132-14.ec2.internal\" DevicePath \"\""
Sep 09 03:33:04.186373 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:33:04.186379    2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-cpgxr\" (UniqueName: \"kubernetes.io/projected/b180cc10-3f98-4819-8563-87c151318982-kube-api-access-cpgxr\") on node \"ip-10-0-132-14.ec2.internal\" DevicePath \"\""
Sep 09 03:33:04.186603 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:33:04.186391    2576 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a2a8ac26-4f2b-4c51-8dc0-402daf9e072d-plugins-config-volume\") on node \"ip-10-0-132-14.ec2.internal\" DevicePath \"\""
Sep 09 03:33:04.234998 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:33:04.234963    2576 generic.go:356] "Generic (PLEG): container finished" podID="b180cc10-3f98-4819-8563-87c151318982" containerID="85458fc597e599c86986dd3c9dcb603d4c171e7a2ac222c60cfaad8567f94948" exitCode=0
Sep 09 03:33:04.235186 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:33:04.235023    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-explicit-958a3f-7bb8558bcc-z4w2c"
Sep 09 03:33:04.235186 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:33:04.235051    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-explicit-958a3f-7bb8558bcc-z4w2c" event={"ID":"b180cc10-3f98-4819-8563-87c151318982","Type":"ContainerDied","Data":"85458fc597e599c86986dd3c9dcb603d4c171e7a2ac222c60cfaad8567f94948"}
Sep 09 03:33:04.235186 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:33:04.235155    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-explicit-958a3f-7bb8558bcc-z4w2c" event={"ID":"b180cc10-3f98-4819-8563-87c151318982","Type":"ContainerDied","Data":"07754e407b4fa29184da70b0fc3c4ba6cfd481cc023fac2d810701aaea36afa4"}
Sep 09 03:33:04.235361 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:33:04.235190    2576 scope.go:117] "RemoveContainer" containerID="85458fc597e599c86986dd3c9dcb603d4c171e7a2ac222c60cfaad8567f94948"
Sep 09 03:33:04.236489 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:33:04.236470    2576 generic.go:356] "Generic (PLEG): container finished" podID="a2a8ac26-4f2b-4c51-8dc0-402daf9e072d" containerID="4ce0c35b20a2d461ab01fe758cb97155e2ea10ab237dcc28c480b0aee71b5c76" exitCode=0
Sep 09 03:33:04.236572 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:33:04.236521    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-explicit-958a3f-696775c758-lqms9" event={"ID":"a2a8ac26-4f2b-4c51-8dc0-402daf9e072d","Type":"ContainerDied","Data":"4ce0c35b20a2d461ab01fe758cb97155e2ea10ab237dcc28c480b0aee71b5c76"}
Sep 09 03:33:04.236572 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:33:04.236528    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-explicit-958a3f-696775c758-lqms9"
Sep 09 03:33:04.236572 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:33:04.236560    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-explicit-958a3f-696775c758-lqms9" event={"ID":"a2a8ac26-4f2b-4c51-8dc0-402daf9e072d","Type":"ContainerDied","Data":"6336eda00354d4c7c7a156eb3627f4e388b18091907c8ed02f54bc695299497f"}
Sep 09 03:33:04.243814 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:33:04.243781    2576 scope.go:117] "RemoveContainer" containerID="85458fc597e599c86986dd3c9dcb603d4c171e7a2ac222c60cfaad8567f94948"
Sep 09 03:33:04.244110 ip-10-0-132-14 kubenswrapper[2576]: E0909 03:33:04.244089    2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"85458fc597e599c86986dd3c9dcb603d4c171e7a2ac222c60cfaad8567f94948\": container with ID starting with 85458fc597e599c86986dd3c9dcb603d4c171e7a2ac222c60cfaad8567f94948 not found: ID does not exist" containerID="85458fc597e599c86986dd3c9dcb603d4c171e7a2ac222c60cfaad8567f94948"
Sep 09 03:33:04.244207 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:33:04.244120    2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"85458fc597e599c86986dd3c9dcb603d4c171e7a2ac222c60cfaad8567f94948"} err="failed to get container status \"85458fc597e599c86986dd3c9dcb603d4c171e7a2ac222c60cfaad8567f94948\": rpc error: code = NotFound desc = could not find container \"85458fc597e599c86986dd3c9dcb603d4c171e7a2ac222c60cfaad8567f94948\": container with ID starting with 85458fc597e599c86986dd3c9dcb603d4c171e7a2ac222c60cfaad8567f94948 not found: ID does not exist"
Sep 09 03:33:04.244207 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:33:04.244143    2576 scope.go:117] "RemoveContainer" containerID="4ce0c35b20a2d461ab01fe758cb97155e2ea10ab237dcc28c480b0aee71b5c76"
Sep 09 03:33:04.252313 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:33:04.252298    2576 scope.go:117] "RemoveContainer" containerID="4ce0c35b20a2d461ab01fe758cb97155e2ea10ab237dcc28c480b0aee71b5c76"
Sep 09 03:33:04.252592 ip-10-0-132-14 kubenswrapper[2576]: E0909 03:33:04.252569    2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4ce0c35b20a2d461ab01fe758cb97155e2ea10ab237dcc28c480b0aee71b5c76\": container with ID starting with 4ce0c35b20a2d461ab01fe758cb97155e2ea10ab237dcc28c480b0aee71b5c76 not found: ID does not exist" containerID="4ce0c35b20a2d461ab01fe758cb97155e2ea10ab237dcc28c480b0aee71b5c76"
Sep 09 03:33:04.252650 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:33:04.252601    2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4ce0c35b20a2d461ab01fe758cb97155e2ea10ab237dcc28c480b0aee71b5c76"} err="failed to get container status \"4ce0c35b20a2d461ab01fe758cb97155e2ea10ab237dcc28c480b0aee71b5c76\": rpc error: code = NotFound desc = could not find container \"4ce0c35b20a2d461ab01fe758cb97155e2ea10ab237dcc28c480b0aee71b5c76\": container with ID starting with 4ce0c35b20a2d461ab01fe758cb97155e2ea10ab237dcc28c480b0aee71b5c76 not found: ID does not exist"
Sep 09 03:33:04.270896 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:33:04.270860    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-explicit-958a3f-696775c758-lqms9"]
Sep 09 03:33:04.278189 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:33:04.278170    2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-explicit-958a3f-696775c758-lqms9"]
Sep 09 03:33:04.296492 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:33:04.296469    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-explicit-958a3f-7bb8558bcc-z4w2c"]
Sep 09 03:33:04.303745 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:33:04.303720    2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-explicit-958a3f-7bb8558bcc-z4w2c"]
Sep 09 03:33:04.361558 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:33:04.361530    2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a2a8ac26-4f2b-4c51-8dc0-402daf9e072d" path="/var/lib/kubelet/pods/a2a8ac26-4f2b-4c51-8dc0-402daf9e072d/volumes"
Sep 09 03:33:04.361917 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:33:04.361904    2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b180cc10-3f98-4819-8563-87c151318982" path="/var/lib/kubelet/pods/b180cc10-3f98-4819-8563-87c151318982/volumes"
Sep 09 03:33:04.362173 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:33:04.362162    2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="efa2010e-640e-433f-9915-438ab7785b2f" path="/var/lib/kubelet/pods/efa2010e-640e-433f-9915-438ab7785b2f/volumes"
Sep 09 03:33:12.201606 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:33:12.201576    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-explicit-958a3f-openshift-default-54b68b49f6-9q84l"]
Sep 09 03:33:12.202051 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:33:12.201788    2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-explicit-958a3f-openshift-default-54b68b49f6-9q84l" podUID="eb8e971b-0bc7-42f6-927e-1db93a64ff57" containerName="istio-proxy" containerID="cri-o://57afd9a78b830acd52ddb27c48c49b47305cde90c8ee0c29b58be0905745887e" gracePeriod=30
Sep 09 03:33:13.109725 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:33:13.109695    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-noten-550c82-openshift-default-57d5b7d874-txrcc"]
Sep 09 03:33:13.110063 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:33:13.110050    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a2a8ac26-4f2b-4c51-8dc0-402daf9e072d" containerName="payload-pre-processing"
Sep 09 03:33:13.110109 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:33:13.110064    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="a2a8ac26-4f2b-4c51-8dc0-402daf9e072d" containerName="payload-pre-processing"
Sep 09 03:33:13.110109 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:33:13.110076    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b180cc10-3f98-4819-8563-87c151318982" containerName="payload-processing"
Sep 09 03:33:13.110109 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:33:13.110082    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="b180cc10-3f98-4819-8563-87c151318982" containerName="payload-processing"
Sep 09 03:33:13.110109 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:33:13.110094    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="efa2010e-640e-433f-9915-438ab7785b2f" containerName="revoke-keys"
Sep 09 03:33:13.110109 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:33:13.110099    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="efa2010e-640e-433f-9915-438ab7785b2f" containerName="revoke-keys"
Sep 09 03:33:13.110256 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:33:13.110154    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="a2a8ac26-4f2b-4c51-8dc0-402daf9e072d" containerName="payload-pre-processing"
Sep 09 03:33:13.110256 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:33:13.110163    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="efa2010e-640e-433f-9915-438ab7785b2f" containerName="revoke-keys"
Sep 09 03:33:13.110256 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:33:13.110170    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="b180cc10-3f98-4819-8563-87c151318982" containerName="payload-processing"
Sep 09 03:33:13.114150 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:33:13.114133    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-noten-550c82-openshift-default-57d5b7d874-txrcc"
Sep 09 03:33:13.117197 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:33:13.117180    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-noten-550c82-openshift-default-dockercfg-jzmjv\""
Sep 09 03:33:13.138691 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:33:13.138666    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-noten-550c82-openshift-default-57d5b7d874-txrcc"]
Sep 09 03:33:13.165341 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:33:13.165315    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b4695da1-d62f-4e9c-8bf7-41236e3f0814-istio-podinfo\") pod \"e2e-noten-550c82-openshift-default-57d5b7d874-txrcc\" (UID: \"b4695da1-d62f-4e9c-8bf7-41236e3f0814\") " pod="openshift-ingress/e2e-noten-550c82-openshift-default-57d5b7d874-txrcc"
Sep 09 03:33:13.165462 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:33:13.165353    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b4695da1-d62f-4e9c-8bf7-41236e3f0814-istio-envoy\") pod \"e2e-noten-550c82-openshift-default-57d5b7d874-txrcc\" (UID: \"b4695da1-d62f-4e9c-8bf7-41236e3f0814\") " pod="openshift-ingress/e2e-noten-550c82-openshift-default-57d5b7d874-txrcc"
Sep 09 03:33:13.165462 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:33:13.165382    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b4695da1-d62f-4e9c-8bf7-41236e3f0814-credential-socket\") pod \"e2e-noten-550c82-openshift-default-57d5b7d874-txrcc\" (UID: \"b4695da1-d62f-4e9c-8bf7-41236e3f0814\") " pod="openshift-ingress/e2e-noten-550c82-openshift-default-57d5b7d874-txrcc"
Sep 09 03:33:13.165462 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:33:13.165436    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b4695da1-d62f-4e9c-8bf7-41236e3f0814-workload-certs\") pod \"e2e-noten-550c82-openshift-default-57d5b7d874-txrcc\" (UID: \"b4695da1-d62f-4e9c-8bf7-41236e3f0814\") " pod="openshift-ingress/e2e-noten-550c82-openshift-default-57d5b7d874-txrcc"
Sep 09 03:33:13.165573 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:33:13.165464    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2hdgf\" (UniqueName: \"kubernetes.io/projected/b4695da1-d62f-4e9c-8bf7-41236e3f0814-kube-api-access-2hdgf\") pod \"e2e-noten-550c82-openshift-default-57d5b7d874-txrcc\" (UID: \"b4695da1-d62f-4e9c-8bf7-41236e3f0814\") " pod="openshift-ingress/e2e-noten-550c82-openshift-default-57d5b7d874-txrcc"
Sep 09 03:33:13.165573 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:33:13.165525    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b4695da1-d62f-4e9c-8bf7-41236e3f0814-istio-data\") pod \"e2e-noten-550c82-openshift-default-57d5b7d874-txrcc\" (UID: \"b4695da1-d62f-4e9c-8bf7-41236e3f0814\") " pod="openshift-ingress/e2e-noten-550c82-openshift-default-57d5b7d874-txrcc"
Sep 09 03:33:13.165573 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:33:13.165564    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b4695da1-d62f-4e9c-8bf7-41236e3f0814-workload-socket\") pod \"e2e-noten-550c82-openshift-default-57d5b7d874-txrcc\" (UID: \"b4695da1-d62f-4e9c-8bf7-41236e3f0814\") " pod="openshift-ingress/e2e-noten-550c82-openshift-default-57d5b7d874-txrcc"
Sep 09 03:33:13.165669 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:33:13.165618    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b4695da1-d62f-4e9c-8bf7-41236e3f0814-istiod-ca-cert\") pod \"e2e-noten-550c82-openshift-default-57d5b7d874-txrcc\" (UID: \"b4695da1-d62f-4e9c-8bf7-41236e3f0814\") " pod="openshift-ingress/e2e-noten-550c82-openshift-default-57d5b7d874-txrcc"
Sep 09 03:33:13.165702 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:33:13.165667    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b4695da1-d62f-4e9c-8bf7-41236e3f0814-istio-token\") pod \"e2e-noten-550c82-openshift-default-57d5b7d874-txrcc\" (UID: \"b4695da1-d62f-4e9c-8bf7-41236e3f0814\") " pod="openshift-ingress/e2e-noten-550c82-openshift-default-57d5b7d874-txrcc"
Sep 09 03:33:13.266826 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:33:13.266773    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b4695da1-d62f-4e9c-8bf7-41236e3f0814-istio-data\") pod \"e2e-noten-550c82-openshift-default-57d5b7d874-txrcc\" (UID: \"b4695da1-d62f-4e9c-8bf7-41236e3f0814\") " pod="openshift-ingress/e2e-noten-550c82-openshift-default-57d5b7d874-txrcc"
Sep 09 03:33:13.266826 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:33:13.266829    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b4695da1-d62f-4e9c-8bf7-41236e3f0814-workload-socket\") pod \"e2e-noten-550c82-openshift-default-57d5b7d874-txrcc\" (UID: \"b4695da1-d62f-4e9c-8bf7-41236e3f0814\") " pod="openshift-ingress/e2e-noten-550c82-openshift-default-57d5b7d874-txrcc"
Sep 09 03:33:13.267315 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:33:13.266857    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b4695da1-d62f-4e9c-8bf7-41236e3f0814-istiod-ca-cert\") pod \"e2e-noten-550c82-openshift-default-57d5b7d874-txrcc\" (UID: \"b4695da1-d62f-4e9c-8bf7-41236e3f0814\") " pod="openshift-ingress/e2e-noten-550c82-openshift-default-57d5b7d874-txrcc"
Sep 09 03:33:13.267315 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:33:13.266901    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b4695da1-d62f-4e9c-8bf7-41236e3f0814-istio-token\") pod \"e2e-noten-550c82-openshift-default-57d5b7d874-txrcc\" (UID: \"b4695da1-d62f-4e9c-8bf7-41236e3f0814\") " pod="openshift-ingress/e2e-noten-550c82-openshift-default-57d5b7d874-txrcc"
Sep 09 03:33:13.267315 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:33:13.266962    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b4695da1-d62f-4e9c-8bf7-41236e3f0814-istio-podinfo\") pod \"e2e-noten-550c82-openshift-default-57d5b7d874-txrcc\" (UID: \"b4695da1-d62f-4e9c-8bf7-41236e3f0814\") " pod="openshift-ingress/e2e-noten-550c82-openshift-default-57d5b7d874-txrcc"
Sep 09 03:33:13.267315 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:33:13.267013    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b4695da1-d62f-4e9c-8bf7-41236e3f0814-istio-envoy\") pod \"e2e-noten-550c82-openshift-default-57d5b7d874-txrcc\" (UID: \"b4695da1-d62f-4e9c-8bf7-41236e3f0814\") " pod="openshift-ingress/e2e-noten-550c82-openshift-default-57d5b7d874-txrcc"
Sep 09 03:33:13.267315 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:33:13.267047    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b4695da1-d62f-4e9c-8bf7-41236e3f0814-credential-socket\") pod \"e2e-noten-550c82-openshift-default-57d5b7d874-txrcc\" (UID: \"b4695da1-d62f-4e9c-8bf7-41236e3f0814\") " pod="openshift-ingress/e2e-noten-550c82-openshift-default-57d5b7d874-txrcc"
Sep 09 03:33:13.267315 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:33:13.267115    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b4695da1-d62f-4e9c-8bf7-41236e3f0814-workload-certs\") pod \"e2e-noten-550c82-openshift-default-57d5b7d874-txrcc\" (UID: \"b4695da1-d62f-4e9c-8bf7-41236e3f0814\") " pod="openshift-ingress/e2e-noten-550c82-openshift-default-57d5b7d874-txrcc"
Sep 09 03:33:13.267315 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:33:13.267154    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2hdgf\" (UniqueName: \"kubernetes.io/projected/b4695da1-d62f-4e9c-8bf7-41236e3f0814-kube-api-access-2hdgf\") pod \"e2e-noten-550c82-openshift-default-57d5b7d874-txrcc\" (UID: \"b4695da1-d62f-4e9c-8bf7-41236e3f0814\") " pod="openshift-ingress/e2e-noten-550c82-openshift-default-57d5b7d874-txrcc"
Sep 09 03:33:13.267315 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:33:13.267306    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b4695da1-d62f-4e9c-8bf7-41236e3f0814-istio-data\") pod \"e2e-noten-550c82-openshift-default-57d5b7d874-txrcc\" (UID: \"b4695da1-d62f-4e9c-8bf7-41236e3f0814\") " pod="openshift-ingress/e2e-noten-550c82-openshift-default-57d5b7d874-txrcc"
Sep 09 03:33:13.267628 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:33:13.267448    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b4695da1-d62f-4e9c-8bf7-41236e3f0814-workload-socket\") pod \"e2e-noten-550c82-openshift-default-57d5b7d874-txrcc\" (UID: \"b4695da1-d62f-4e9c-8bf7-41236e3f0814\") " pod="openshift-ingress/e2e-noten-550c82-openshift-default-57d5b7d874-txrcc"
Sep 09 03:33:13.267628 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:33:13.267519    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b4695da1-d62f-4e9c-8bf7-41236e3f0814-credential-socket\") pod \"e2e-noten-550c82-openshift-default-57d5b7d874-txrcc\" (UID: \"b4695da1-d62f-4e9c-8bf7-41236e3f0814\") " pod="openshift-ingress/e2e-noten-550c82-openshift-default-57d5b7d874-txrcc"
Sep 09 03:33:13.267628 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:33:13.267535    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b4695da1-d62f-4e9c-8bf7-41236e3f0814-workload-certs\") pod \"e2e-noten-550c82-openshift-default-57d5b7d874-txrcc\" (UID: \"b4695da1-d62f-4e9c-8bf7-41236e3f0814\") " pod="openshift-ingress/e2e-noten-550c82-openshift-default-57d5b7d874-txrcc"
Sep 09 03:33:13.267628 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:33:13.267568    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b4695da1-d62f-4e9c-8bf7-41236e3f0814-istiod-ca-cert\") pod \"e2e-noten-550c82-openshift-default-57d5b7d874-txrcc\" (UID: \"b4695da1-d62f-4e9c-8bf7-41236e3f0814\") " pod="openshift-ingress/e2e-noten-550c82-openshift-default-57d5b7d874-txrcc"
Sep 09 03:33:13.269304 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:33:13.269281    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b4695da1-d62f-4e9c-8bf7-41236e3f0814-istio-envoy\") pod \"e2e-noten-550c82-openshift-default-57d5b7d874-txrcc\" (UID: \"b4695da1-d62f-4e9c-8bf7-41236e3f0814\") " pod="openshift-ingress/e2e-noten-550c82-openshift-default-57d5b7d874-txrcc"
Sep 09 03:33:13.269622 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:33:13.269599    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b4695da1-d62f-4e9c-8bf7-41236e3f0814-istio-podinfo\") pod \"e2e-noten-550c82-openshift-default-57d5b7d874-txrcc\" (UID: \"b4695da1-d62f-4e9c-8bf7-41236e3f0814\") " pod="openshift-ingress/e2e-noten-550c82-openshift-default-57d5b7d874-txrcc"
Sep 09 03:33:13.279049 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:33:13.279023    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b4695da1-d62f-4e9c-8bf7-41236e3f0814-istio-token\") pod \"e2e-noten-550c82-openshift-default-57d5b7d874-txrcc\" (UID: \"b4695da1-d62f-4e9c-8bf7-41236e3f0814\") " pod="openshift-ingress/e2e-noten-550c82-openshift-default-57d5b7d874-txrcc"
Sep 09 03:33:13.289034 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:33:13.289012    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2hdgf\" (UniqueName: \"kubernetes.io/projected/b4695da1-d62f-4e9c-8bf7-41236e3f0814-kube-api-access-2hdgf\") pod \"e2e-noten-550c82-openshift-default-57d5b7d874-txrcc\" (UID: \"b4695da1-d62f-4e9c-8bf7-41236e3f0814\") " pod="openshift-ingress/e2e-noten-550c82-openshift-default-57d5b7d874-txrcc"
Sep 09 03:33:13.424111 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:33:13.424037    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-noten-550c82-openshift-default-57d5b7d874-txrcc"
Sep 09 03:33:13.708218 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:33:13.708192    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-noten-550c82-openshift-default-57d5b7d874-txrcc"]
Sep 09 03:33:13.709914 ip-10-0-132-14 kubenswrapper[2576]: W0909 03:33:13.709886    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb4695da1_d62f_4e9c_8bf7_41236e3f0814.slice/crio-2f876fed165ffac38d61460a4b3470f546aa0d372693342b8092203d5f8110ac WatchSource:0}: Error finding container 2f876fed165ffac38d61460a4b3470f546aa0d372693342b8092203d5f8110ac: Status 404 returned error can't find the container with id 2f876fed165ffac38d61460a4b3470f546aa0d372693342b8092203d5f8110ac
Sep 09 03:33:13.712032 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:33:13.712003    2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892148Ki","pods":"250"}
Sep 09 03:33:13.712089 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:33:13.712065    2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892148Ki","pods":"250"}
Sep 09 03:33:13.712132 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:33:13.712093    2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892148Ki","pods":"250"}
Sep 09 03:33:14.295779 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:33:14.295741    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-noten-550c82-openshift-default-57d5b7d874-txrcc" event={"ID":"b4695da1-d62f-4e9c-8bf7-41236e3f0814","Type":"ContainerStarted","Data":"1063fcbe72bd317c5c4b4994698a275176131cd4b6344b8f83a265708f46858a"}
Sep 09 03:33:14.296236 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:33:14.295826    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-noten-550c82-openshift-default-57d5b7d874-txrcc" event={"ID":"b4695da1-d62f-4e9c-8bf7-41236e3f0814","Type":"ContainerStarted","Data":"2f876fed165ffac38d61460a4b3470f546aa0d372693342b8092203d5f8110ac"}
Sep 09 03:33:14.337999 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:33:14.337944    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-noten-550c82-openshift-default-57d5b7d874-txrcc" podStartSLOduration=1.33792723 podStartE2EDuration="1.33792723s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 03:33:13 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 03:33:14.336598116 +0000 UTC m=+1208.478084192" watchObservedRunningTime="2026-09-09 03:33:14.33792723 +0000 UTC m=+1208.479413308"
Sep 09 03:33:14.424305 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:33:14.424273    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-noten-550c82-openshift-default-57d5b7d874-txrcc"
Sep 09 03:33:14.429950 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:33:14.429928    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-noten-550c82-openshift-default-57d5b7d874-txrcc"
Sep 09 03:33:15.301472 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:33:15.301441    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-noten-550c82-openshift-default-57d5b7d874-txrcc"
Sep 09 03:33:15.302466 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:33:15.302446    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-noten-550c82-openshift-default-57d5b7d874-txrcc"
Sep 09 03:33:17.447936 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:33:17.447912    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-explicit-958a3f-openshift-default-54b68b49f6-9q84l"
Sep 09 03:33:17.611861 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:33:17.611791    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-mn6t7\" (UniqueName: \"kubernetes.io/projected/eb8e971b-0bc7-42f6-927e-1db93a64ff57-kube-api-access-mn6t7\") pod \"eb8e971b-0bc7-42f6-927e-1db93a64ff57\" (UID: \"eb8e971b-0bc7-42f6-927e-1db93a64ff57\") "
Sep 09 03:33:17.612053 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:33:17.611882    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/eb8e971b-0bc7-42f6-927e-1db93a64ff57-credential-socket\") pod \"eb8e971b-0bc7-42f6-927e-1db93a64ff57\" (UID: \"eb8e971b-0bc7-42f6-927e-1db93a64ff57\") "
Sep 09 03:33:17.612053 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:33:17.611975    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/eb8e971b-0bc7-42f6-927e-1db93a64ff57-istiod-ca-cert\") pod \"eb8e971b-0bc7-42f6-927e-1db93a64ff57\" (UID: \"eb8e971b-0bc7-42f6-927e-1db93a64ff57\") "
Sep 09 03:33:17.612053 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:33:17.612013    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/eb8e971b-0bc7-42f6-927e-1db93a64ff57-workload-certs\") pod \"eb8e971b-0bc7-42f6-927e-1db93a64ff57\" (UID: \"eb8e971b-0bc7-42f6-927e-1db93a64ff57\") "
Sep 09 03:33:17.612211 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:33:17.612055    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/eb8e971b-0bc7-42f6-927e-1db93a64ff57-istio-podinfo\") pod \"eb8e971b-0bc7-42f6-927e-1db93a64ff57\" (UID: \"eb8e971b-0bc7-42f6-927e-1db93a64ff57\") "
Sep 09 03:33:17.612211 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:33:17.612088    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/eb8e971b-0bc7-42f6-927e-1db93a64ff57-istio-envoy\") pod \"eb8e971b-0bc7-42f6-927e-1db93a64ff57\" (UID: \"eb8e971b-0bc7-42f6-927e-1db93a64ff57\") "
Sep 09 03:33:17.612211 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:33:17.612104    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/eb8e971b-0bc7-42f6-927e-1db93a64ff57-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "eb8e971b-0bc7-42f6-927e-1db93a64ff57" (UID: "eb8e971b-0bc7-42f6-927e-1db93a64ff57"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 03:33:17.612211 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:33:17.612134    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/eb8e971b-0bc7-42f6-927e-1db93a64ff57-istio-token\") pod \"eb8e971b-0bc7-42f6-927e-1db93a64ff57\" (UID: \"eb8e971b-0bc7-42f6-927e-1db93a64ff57\") "
Sep 09 03:33:17.612211 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:33:17.612181    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/eb8e971b-0bc7-42f6-927e-1db93a64ff57-workload-socket\") pod \"eb8e971b-0bc7-42f6-927e-1db93a64ff57\" (UID: \"eb8e971b-0bc7-42f6-927e-1db93a64ff57\") "
Sep 09 03:33:17.612445 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:33:17.612225    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/eb8e971b-0bc7-42f6-927e-1db93a64ff57-istio-data\") pod \"eb8e971b-0bc7-42f6-927e-1db93a64ff57\" (UID: \"eb8e971b-0bc7-42f6-927e-1db93a64ff57\") "
Sep 09 03:33:17.612445 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:33:17.612334    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/eb8e971b-0bc7-42f6-927e-1db93a64ff57-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "eb8e971b-0bc7-42f6-927e-1db93a64ff57" (UID: "eb8e971b-0bc7-42f6-927e-1db93a64ff57"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 03:33:17.612445 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:33:17.612348    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/eb8e971b-0bc7-42f6-927e-1db93a64ff57-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "eb8e971b-0bc7-42f6-927e-1db93a64ff57" (UID: "eb8e971b-0bc7-42f6-927e-1db93a64ff57"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 09 03:33:17.612598 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:33:17.612532    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/eb8e971b-0bc7-42f6-927e-1db93a64ff57-istio-data" (OuterVolumeSpecName: "istio-data") pod "eb8e971b-0bc7-42f6-927e-1db93a64ff57" (UID: "eb8e971b-0bc7-42f6-927e-1db93a64ff57"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 03:33:17.612598 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:33:17.612588    2576 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/eb8e971b-0bc7-42f6-927e-1db93a64ff57-workload-certs\") on node \"ip-10-0-132-14.ec2.internal\" DevicePath \"\""
Sep 09 03:33:17.612598 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:33:17.612614    2576 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/eb8e971b-0bc7-42f6-927e-1db93a64ff57-credential-socket\") on node \"ip-10-0-132-14.ec2.internal\" DevicePath \"\""
Sep 09 03:33:17.612949 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:33:17.612631    2576 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/eb8e971b-0bc7-42f6-927e-1db93a64ff57-istiod-ca-cert\") on node \"ip-10-0-132-14.ec2.internal\" DevicePath \"\""
Sep 09 03:33:17.612949 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:33:17.612591    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/eb8e971b-0bc7-42f6-927e-1db93a64ff57-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "eb8e971b-0bc7-42f6-927e-1db93a64ff57" (UID: "eb8e971b-0bc7-42f6-927e-1db93a64ff57"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 03:33:17.614305 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:33:17.614274    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/eb8e971b-0bc7-42f6-927e-1db93a64ff57-kube-api-access-mn6t7" (OuterVolumeSpecName: "kube-api-access-mn6t7") pod "eb8e971b-0bc7-42f6-927e-1db93a64ff57" (UID: "eb8e971b-0bc7-42f6-927e-1db93a64ff57"). InnerVolumeSpecName "kube-api-access-mn6t7". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 09 03:33:17.614391 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:33:17.614371    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/eb8e971b-0bc7-42f6-927e-1db93a64ff57-istio-token" (OuterVolumeSpecName: "istio-token") pod "eb8e971b-0bc7-42f6-927e-1db93a64ff57" (UID: "eb8e971b-0bc7-42f6-927e-1db93a64ff57"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 09 03:33:17.614771 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:33:17.614751    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/eb8e971b-0bc7-42f6-927e-1db93a64ff57-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "eb8e971b-0bc7-42f6-927e-1db93a64ff57" (UID: "eb8e971b-0bc7-42f6-927e-1db93a64ff57"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 03:33:17.614771 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:33:17.614756    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/eb8e971b-0bc7-42f6-927e-1db93a64ff57-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "eb8e971b-0bc7-42f6-927e-1db93a64ff57" (UID: "eb8e971b-0bc7-42f6-927e-1db93a64ff57"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 09 03:33:17.713430 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:33:17.713397    2576 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/eb8e971b-0bc7-42f6-927e-1db93a64ff57-istio-podinfo\") on node \"ip-10-0-132-14.ec2.internal\" DevicePath \"\""
Sep 09 03:33:17.713430 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:33:17.713424    2576 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/eb8e971b-0bc7-42f6-927e-1db93a64ff57-istio-envoy\") on node \"ip-10-0-132-14.ec2.internal\" DevicePath \"\""
Sep 09 03:33:17.713430 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:33:17.713432    2576 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/eb8e971b-0bc7-42f6-927e-1db93a64ff57-istio-token\") on node \"ip-10-0-132-14.ec2.internal\" DevicePath \"\""
Sep 09 03:33:17.713430 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:33:17.713440    2576 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/eb8e971b-0bc7-42f6-927e-1db93a64ff57-workload-socket\") on node \"ip-10-0-132-14.ec2.internal\" DevicePath \"\""
Sep 09 03:33:17.713702 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:33:17.713448    2576 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/eb8e971b-0bc7-42f6-927e-1db93a64ff57-istio-data\") on node \"ip-10-0-132-14.ec2.internal\" DevicePath \"\""
Sep 09 03:33:17.713702 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:33:17.713456    2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mn6t7\" (UniqueName: \"kubernetes.io/projected/eb8e971b-0bc7-42f6-927e-1db93a64ff57-kube-api-access-mn6t7\") on node \"ip-10-0-132-14.ec2.internal\" DevicePath \"\""
Sep 09 03:33:18.318077 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:33:18.318042    2576 generic.go:356] "Generic (PLEG): container finished" podID="eb8e971b-0bc7-42f6-927e-1db93a64ff57" containerID="57afd9a78b830acd52ddb27c48c49b47305cde90c8ee0c29b58be0905745887e" exitCode=0
Sep 09 03:33:18.318324 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:33:18.318088    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-explicit-958a3f-openshift-default-54b68b49f6-9q84l"
Sep 09 03:33:18.318324 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:33:18.318122    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-explicit-958a3f-openshift-default-54b68b49f6-9q84l" event={"ID":"eb8e971b-0bc7-42f6-927e-1db93a64ff57","Type":"ContainerDied","Data":"57afd9a78b830acd52ddb27c48c49b47305cde90c8ee0c29b58be0905745887e"}
Sep 09 03:33:18.318324 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:33:18.318181    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-explicit-958a3f-openshift-default-54b68b49f6-9q84l" event={"ID":"eb8e971b-0bc7-42f6-927e-1db93a64ff57","Type":"ContainerDied","Data":"f7d4b7e53290a825cbcc938dbd7982c2c09a3e4578ceb0d146a4c497756aac1c"}
Sep 09 03:33:18.318324 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:33:18.318205    2576 scope.go:117] "RemoveContainer" containerID="57afd9a78b830acd52ddb27c48c49b47305cde90c8ee0c29b58be0905745887e"
Sep 09 03:33:18.329652 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:33:18.329430    2576 scope.go:117] "RemoveContainer" containerID="132098aa8f0bcf17cd1ab134e3a84d44621777ac16dac7dbc42f8505b0248aa3"
Sep 09 03:33:18.341950 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:33:18.341906    2576 scope.go:117] "RemoveContainer" containerID="57afd9a78b830acd52ddb27c48c49b47305cde90c8ee0c29b58be0905745887e"
Sep 09 03:33:18.342396 ip-10-0-132-14 kubenswrapper[2576]: E0909 03:33:18.342357    2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"57afd9a78b830acd52ddb27c48c49b47305cde90c8ee0c29b58be0905745887e\": container with ID starting with 57afd9a78b830acd52ddb27c48c49b47305cde90c8ee0c29b58be0905745887e not found: ID does not exist" containerID="57afd9a78b830acd52ddb27c48c49b47305cde90c8ee0c29b58be0905745887e"
Sep 09 03:33:18.342447 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:33:18.342414    2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"57afd9a78b830acd52ddb27c48c49b47305cde90c8ee0c29b58be0905745887e"} err="failed to get container status \"57afd9a78b830acd52ddb27c48c49b47305cde90c8ee0c29b58be0905745887e\": rpc error: code = NotFound desc = could not find container \"57afd9a78b830acd52ddb27c48c49b47305cde90c8ee0c29b58be0905745887e\": container with ID starting with 57afd9a78b830acd52ddb27c48c49b47305cde90c8ee0c29b58be0905745887e not found: ID does not exist"
Sep 09 03:33:18.342489 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:33:18.342448    2576 scope.go:117] "RemoveContainer" containerID="132098aa8f0bcf17cd1ab134e3a84d44621777ac16dac7dbc42f8505b0248aa3"
Sep 09 03:33:18.342813 ip-10-0-132-14 kubenswrapper[2576]: E0909 03:33:18.342767    2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"132098aa8f0bcf17cd1ab134e3a84d44621777ac16dac7dbc42f8505b0248aa3\": container with ID starting with 132098aa8f0bcf17cd1ab134e3a84d44621777ac16dac7dbc42f8505b0248aa3 not found: ID does not exist" containerID="132098aa8f0bcf17cd1ab134e3a84d44621777ac16dac7dbc42f8505b0248aa3"
Sep 09 03:33:18.342870 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:33:18.342830    2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"132098aa8f0bcf17cd1ab134e3a84d44621777ac16dac7dbc42f8505b0248aa3"} err="failed to get container status \"132098aa8f0bcf17cd1ab134e3a84d44621777ac16dac7dbc42f8505b0248aa3\": rpc error: code = NotFound desc = could not find container \"132098aa8f0bcf17cd1ab134e3a84d44621777ac16dac7dbc42f8505b0248aa3\": container with ID starting with 132098aa8f0bcf17cd1ab134e3a84d44621777ac16dac7dbc42f8505b0248aa3 not found: ID does not exist"
Sep 09 03:33:18.359877 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:33:18.359851    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-explicit-958a3f-openshift-default-54b68b49f6-9q84l"]
Sep 09 03:33:18.360029 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:33:18.359887    2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-explicit-958a3f-openshift-default-54b68b49f6-9q84l"]
Sep 09 03:33:20.359726 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:33:20.359697    2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="eb8e971b-0bc7-42f6-927e-1db93a64ff57" path="/var/lib/kubelet/pods/eb8e971b-0bc7-42f6-927e-1db93a64ff57/volumes"
Sep 09 03:33:36.902069 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:33:36.902025    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-noten-550c82-openshift-default-57d5b7d874-txrcc"]
Sep 09 03:33:36.902578 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:33:36.902290    2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-noten-550c82-openshift-default-57d5b7d874-txrcc" podUID="b4695da1-d62f-4e9c-8bf7-41236e3f0814" containerName="istio-proxy" containerID="cri-o://1063fcbe72bd317c5c4b4994698a275176131cd4b6344b8f83a265708f46858a" gracePeriod=30
Sep 09 03:33:41.546438 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:33:41.546402    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-40a1c9-s8fr8"]
Sep 09 03:33:41.546918 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:33:41.546766    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="eb8e971b-0bc7-42f6-927e-1db93a64ff57" containerName="istio-proxy"
Sep 09 03:33:41.546918 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:33:41.546779    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="eb8e971b-0bc7-42f6-927e-1db93a64ff57" containerName="istio-proxy"
Sep 09 03:33:41.546918 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:33:41.546796    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="eb8e971b-0bc7-42f6-927e-1db93a64ff57" containerName="istio-proxy"
Sep 09 03:33:41.546918 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:33:41.546816    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="eb8e971b-0bc7-42f6-927e-1db93a64ff57" containerName="istio-proxy"
Sep 09 03:33:41.546918 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:33:41.546869    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="eb8e971b-0bc7-42f6-927e-1db93a64ff57" containerName="istio-proxy"
Sep 09 03:33:41.546918 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:33:41.546875    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="eb8e971b-0bc7-42f6-927e-1db93a64ff57" containerName="istio-proxy"
Sep 09 03:33:41.550148 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:33:41.550122    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-40a1c9-s8fr8"
Sep 09 03:33:41.553548 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:33:41.553524    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-w7q9f\""
Sep 09 03:33:41.561963 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:33:41.561935    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-40a1c9-s8fr8"]
Sep 09 03:33:41.608881 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:33:41.608839    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/057e7b78-3925-47b5-8c44-cca91b377eb2-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-a-40a1c9-s8fr8\" (UID: \"057e7b78-3925-47b5-8c44-cca91b377eb2\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-40a1c9-s8fr8"
Sep 09 03:33:41.710018 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:33:41.709989    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/057e7b78-3925-47b5-8c44-cca91b377eb2-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-a-40a1c9-s8fr8\" (UID: \"057e7b78-3925-47b5-8c44-cca91b377eb2\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-40a1c9-s8fr8"
Sep 09 03:33:41.710600 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:33:41.710581    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/057e7b78-3925-47b5-8c44-cca91b377eb2-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-a-40a1c9-s8fr8\" (UID: \"057e7b78-3925-47b5-8c44-cca91b377eb2\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-40a1c9-s8fr8"
Sep 09 03:33:41.863212 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:33:41.863125    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-40a1c9-s8fr8"
Sep 09 03:33:42.135557 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:33:42.135525    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-40a1c9-s8fr8"]
Sep 09 03:33:42.137893 ip-10-0-132-14 kubenswrapper[2576]: W0909 03:33:42.137866    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod057e7b78_3925_47b5_8c44_cca91b377eb2.slice/crio-aaaaaed06439d745d8a2213b53f8e7b62556580eb2435ba21ac760a6003a3f53 WatchSource:0}: Error finding container aaaaaed06439d745d8a2213b53f8e7b62556580eb2435ba21ac760a6003a3f53: Status 404 returned error can't find the container with id aaaaaed06439d745d8a2213b53f8e7b62556580eb2435ba21ac760a6003a3f53
Sep 09 03:33:42.153485 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:33:42.153458    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-noten-550c82-openshift-default-57d5b7d874-txrcc"
Sep 09 03:33:42.214792 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:33:42.214753    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b4695da1-d62f-4e9c-8bf7-41236e3f0814-istio-token\") pod \"b4695da1-d62f-4e9c-8bf7-41236e3f0814\" (UID: \"b4695da1-d62f-4e9c-8bf7-41236e3f0814\") "
Sep 09 03:33:42.214977 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:33:42.214833    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b4695da1-d62f-4e9c-8bf7-41236e3f0814-workload-certs\") pod \"b4695da1-d62f-4e9c-8bf7-41236e3f0814\" (UID: \"b4695da1-d62f-4e9c-8bf7-41236e3f0814\") "
Sep 09 03:33:42.214977 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:33:42.214858    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b4695da1-d62f-4e9c-8bf7-41236e3f0814-istio-podinfo\") pod \"b4695da1-d62f-4e9c-8bf7-41236e3f0814\" (UID: \"b4695da1-d62f-4e9c-8bf7-41236e3f0814\") "
Sep 09 03:33:42.214977 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:33:42.214885    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b4695da1-d62f-4e9c-8bf7-41236e3f0814-istiod-ca-cert\") pod \"b4695da1-d62f-4e9c-8bf7-41236e3f0814\" (UID: \"b4695da1-d62f-4e9c-8bf7-41236e3f0814\") "
Sep 09 03:33:42.214977 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:33:42.214928    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b4695da1-d62f-4e9c-8bf7-41236e3f0814-workload-socket\") pod \"b4695da1-d62f-4e9c-8bf7-41236e3f0814\" (UID: \"b4695da1-d62f-4e9c-8bf7-41236e3f0814\") "
Sep 09 03:33:42.214977 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:33:42.214959    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b4695da1-d62f-4e9c-8bf7-41236e3f0814-istio-envoy\") pod \"b4695da1-d62f-4e9c-8bf7-41236e3f0814\" (UID: \"b4695da1-d62f-4e9c-8bf7-41236e3f0814\") "
Sep 09 03:33:42.215249 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:33:42.214993    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b4695da1-d62f-4e9c-8bf7-41236e3f0814-credential-socket\") pod \"b4695da1-d62f-4e9c-8bf7-41236e3f0814\" (UID: \"b4695da1-d62f-4e9c-8bf7-41236e3f0814\") "
Sep 09 03:33:42.215249 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:33:42.215028    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b4695da1-d62f-4e9c-8bf7-41236e3f0814-istio-data\") pod \"b4695da1-d62f-4e9c-8bf7-41236e3f0814\" (UID: \"b4695da1-d62f-4e9c-8bf7-41236e3f0814\") "
Sep 09 03:33:42.215249 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:33:42.215055    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-2hdgf\" (UniqueName: \"kubernetes.io/projected/b4695da1-d62f-4e9c-8bf7-41236e3f0814-kube-api-access-2hdgf\") pod \"b4695da1-d62f-4e9c-8bf7-41236e3f0814\" (UID: \"b4695da1-d62f-4e9c-8bf7-41236e3f0814\") "
Sep 09 03:33:42.215249 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:33:42.215134    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b4695da1-d62f-4e9c-8bf7-41236e3f0814-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "b4695da1-d62f-4e9c-8bf7-41236e3f0814" (UID: "b4695da1-d62f-4e9c-8bf7-41236e3f0814"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 03:33:42.215249 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:33:42.215182    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b4695da1-d62f-4e9c-8bf7-41236e3f0814-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "b4695da1-d62f-4e9c-8bf7-41236e3f0814" (UID: "b4695da1-d62f-4e9c-8bf7-41236e3f0814"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 03:33:42.215507 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:33:42.215306    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b4695da1-d62f-4e9c-8bf7-41236e3f0814-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "b4695da1-d62f-4e9c-8bf7-41236e3f0814" (UID: "b4695da1-d62f-4e9c-8bf7-41236e3f0814"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 09 03:33:42.215507 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:33:42.215401    2576 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b4695da1-d62f-4e9c-8bf7-41236e3f0814-workload-certs\") on node \"ip-10-0-132-14.ec2.internal\" DevicePath \"\""
Sep 09 03:33:42.215605 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:33:42.215507    2576 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b4695da1-d62f-4e9c-8bf7-41236e3f0814-istiod-ca-cert\") on node \"ip-10-0-132-14.ec2.internal\" DevicePath \"\""
Sep 09 03:33:42.215605 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:33:42.215508    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b4695da1-d62f-4e9c-8bf7-41236e3f0814-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "b4695da1-d62f-4e9c-8bf7-41236e3f0814" (UID: "b4695da1-d62f-4e9c-8bf7-41236e3f0814"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 03:33:42.215605 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:33:42.215524    2576 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b4695da1-d62f-4e9c-8bf7-41236e3f0814-workload-socket\") on node \"ip-10-0-132-14.ec2.internal\" DevicePath \"\""
Sep 09 03:33:42.215605 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:33:42.215522    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b4695da1-d62f-4e9c-8bf7-41236e3f0814-istio-data" (OuterVolumeSpecName: "istio-data") pod "b4695da1-d62f-4e9c-8bf7-41236e3f0814" (UID: "b4695da1-d62f-4e9c-8bf7-41236e3f0814"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 03:33:42.217300 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:33:42.217268    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b4695da1-d62f-4e9c-8bf7-41236e3f0814-istio-token" (OuterVolumeSpecName: "istio-token") pod "b4695da1-d62f-4e9c-8bf7-41236e3f0814" (UID: "b4695da1-d62f-4e9c-8bf7-41236e3f0814"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 09 03:33:42.217425 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:33:42.217315    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b4695da1-d62f-4e9c-8bf7-41236e3f0814-kube-api-access-2hdgf" (OuterVolumeSpecName: "kube-api-access-2hdgf") pod "b4695da1-d62f-4e9c-8bf7-41236e3f0814" (UID: "b4695da1-d62f-4e9c-8bf7-41236e3f0814"). InnerVolumeSpecName "kube-api-access-2hdgf". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 09 03:33:42.217425 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:33:42.217398    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b4695da1-d62f-4e9c-8bf7-41236e3f0814-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "b4695da1-d62f-4e9c-8bf7-41236e3f0814" (UID: "b4695da1-d62f-4e9c-8bf7-41236e3f0814"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 03:33:42.217599 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:33:42.217510    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/b4695da1-d62f-4e9c-8bf7-41236e3f0814-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "b4695da1-d62f-4e9c-8bf7-41236e3f0814" (UID: "b4695da1-d62f-4e9c-8bf7-41236e3f0814"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 09 03:33:42.316866 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:33:42.316836    2576 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b4695da1-d62f-4e9c-8bf7-41236e3f0814-istio-envoy\") on node \"ip-10-0-132-14.ec2.internal\" DevicePath \"\""
Sep 09 03:33:42.316866 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:33:42.316859    2576 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b4695da1-d62f-4e9c-8bf7-41236e3f0814-credential-socket\") on node \"ip-10-0-132-14.ec2.internal\" DevicePath \"\""
Sep 09 03:33:42.316866 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:33:42.316869    2576 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b4695da1-d62f-4e9c-8bf7-41236e3f0814-istio-data\") on node \"ip-10-0-132-14.ec2.internal\" DevicePath \"\""
Sep 09 03:33:42.317085 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:33:42.316877    2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2hdgf\" (UniqueName: \"kubernetes.io/projected/b4695da1-d62f-4e9c-8bf7-41236e3f0814-kube-api-access-2hdgf\") on node \"ip-10-0-132-14.ec2.internal\" DevicePath \"\""
Sep 09 03:33:42.317085 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:33:42.316886    2576 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b4695da1-d62f-4e9c-8bf7-41236e3f0814-istio-token\") on node \"ip-10-0-132-14.ec2.internal\" DevicePath \"\""
Sep 09 03:33:42.317085 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:33:42.316894    2576 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b4695da1-d62f-4e9c-8bf7-41236e3f0814-istio-podinfo\") on node \"ip-10-0-132-14.ec2.internal\" DevicePath \"\""
Sep 09 03:33:42.433157 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:33:42.433110    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-40a1c9-s8fr8" event={"ID":"057e7b78-3925-47b5-8c44-cca91b377eb2","Type":"ContainerStarted","Data":"aaaaaed06439d745d8a2213b53f8e7b62556580eb2435ba21ac760a6003a3f53"}
Sep 09 03:33:42.434584 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:33:42.434556    2576 generic.go:356] "Generic (PLEG): container finished" podID="b4695da1-d62f-4e9c-8bf7-41236e3f0814" containerID="1063fcbe72bd317c5c4b4994698a275176131cd4b6344b8f83a265708f46858a" exitCode=0
Sep 09 03:33:42.434706 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:33:42.434611    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-noten-550c82-openshift-default-57d5b7d874-txrcc"
Sep 09 03:33:42.434706 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:33:42.434625    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-noten-550c82-openshift-default-57d5b7d874-txrcc" event={"ID":"b4695da1-d62f-4e9c-8bf7-41236e3f0814","Type":"ContainerDied","Data":"1063fcbe72bd317c5c4b4994698a275176131cd4b6344b8f83a265708f46858a"}
Sep 09 03:33:42.434706 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:33:42.434680    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-noten-550c82-openshift-default-57d5b7d874-txrcc" event={"ID":"b4695da1-d62f-4e9c-8bf7-41236e3f0814","Type":"ContainerDied","Data":"2f876fed165ffac38d61460a4b3470f546aa0d372693342b8092203d5f8110ac"}
Sep 09 03:33:42.434904 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:33:42.434710    2576 scope.go:117] "RemoveContainer" containerID="1063fcbe72bd317c5c4b4994698a275176131cd4b6344b8f83a265708f46858a"
Sep 09 03:33:42.444053 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:33:42.444034    2576 scope.go:117] "RemoveContainer" containerID="1063fcbe72bd317c5c4b4994698a275176131cd4b6344b8f83a265708f46858a"
Sep 09 03:33:42.444417 ip-10-0-132-14 kubenswrapper[2576]: E0909 03:33:42.444391    2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1063fcbe72bd317c5c4b4994698a275176131cd4b6344b8f83a265708f46858a\": container with ID starting with 1063fcbe72bd317c5c4b4994698a275176131cd4b6344b8f83a265708f46858a not found: ID does not exist" containerID="1063fcbe72bd317c5c4b4994698a275176131cd4b6344b8f83a265708f46858a"
Sep 09 03:33:42.444511 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:33:42.444427    2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1063fcbe72bd317c5c4b4994698a275176131cd4b6344b8f83a265708f46858a"} err="failed to get container status \"1063fcbe72bd317c5c4b4994698a275176131cd4b6344b8f83a265708f46858a\": rpc error: code = NotFound desc = could not find container \"1063fcbe72bd317c5c4b4994698a275176131cd4b6344b8f83a265708f46858a\": container with ID starting with 1063fcbe72bd317c5c4b4994698a275176131cd4b6344b8f83a265708f46858a not found: ID does not exist"
Sep 09 03:33:42.455421 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:33:42.455397    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-noten-550c82-openshift-default-57d5b7d874-txrcc"]
Sep 09 03:33:42.466809 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:33:42.466784    2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-noten-550c82-openshift-default-57d5b7d874-txrcc"]
Sep 09 03:33:43.444028 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:33:43.443997    2576 generic.go:356] "Generic (PLEG): container finished" podID="057e7b78-3925-47b5-8c44-cca91b377eb2" containerID="2dd773dd7e7df95160d10800012c05d6b9c603a3c05aff67e12ef73aa1ab0d28" exitCode=0
Sep 09 03:33:43.444480 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:33:43.444088    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-40a1c9-s8fr8" event={"ID":"057e7b78-3925-47b5-8c44-cca91b377eb2","Type":"ContainerDied","Data":"2dd773dd7e7df95160d10800012c05d6b9c603a3c05aff67e12ef73aa1ab0d28"}
Sep 09 03:33:44.360249 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:33:44.360216    2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b4695da1-d62f-4e9c-8bf7-41236e3f0814" path="/var/lib/kubelet/pods/b4695da1-d62f-4e9c-8bf7-41236e3f0814/volumes"
Sep 09 03:33:44.580637 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:33:44.580605    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-40a1c9-s8fr8"
Sep 09 03:33:44.635659 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:33:44.635573    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/057e7b78-3925-47b5-8c44-cca91b377eb2-maas-api-service-ca\") pod \"057e7b78-3925-47b5-8c44-cca91b377eb2\" (UID: \"057e7b78-3925-47b5-8c44-cca91b377eb2\") "
Sep 09 03:33:44.635951 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:33:44.635927    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/057e7b78-3925-47b5-8c44-cca91b377eb2-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "057e7b78-3925-47b5-8c44-cca91b377eb2" (UID: "057e7b78-3925-47b5-8c44-cca91b377eb2"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 09 03:33:44.736873 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:33:44.736841    2576 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/057e7b78-3925-47b5-8c44-cca91b377eb2-maas-api-service-ca\") on node \"ip-10-0-132-14.ec2.internal\" DevicePath \"\""
Sep 09 03:33:45.458123 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:33:45.458088    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-40a1c9-s8fr8" event={"ID":"057e7b78-3925-47b5-8c44-cca91b377eb2","Type":"ContainerDied","Data":"aaaaaed06439d745d8a2213b53f8e7b62556580eb2435ba21ac760a6003a3f53"}
Sep 09 03:33:45.458123 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:33:45.458123    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-40a1c9-s8fr8"
Sep 09 03:33:45.458327 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:33:45.458140    2576 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="aaaaaed06439d745d8a2213b53f8e7b62556580eb2435ba21ac760a6003a3f53"
Sep 09 03:33:50.153654 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:33:50.153619    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-mt-c44cc65a-openshift-default-66c6b44c5b-cnpfj"]
Sep 09 03:33:50.154219 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:33:50.154117    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b4695da1-d62f-4e9c-8bf7-41236e3f0814" containerName="istio-proxy"
Sep 09 03:33:50.154219 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:33:50.154136    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="b4695da1-d62f-4e9c-8bf7-41236e3f0814" containerName="istio-proxy"
Sep 09 03:33:50.154219 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:33:50.154160    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="057e7b78-3925-47b5-8c44-cca91b377eb2" containerName="revoke-keys"
Sep 09 03:33:50.154219 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:33:50.154169    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="057e7b78-3925-47b5-8c44-cca91b377eb2" containerName="revoke-keys"
Sep 09 03:33:50.154490 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:33:50.154257    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="057e7b78-3925-47b5-8c44-cca91b377eb2" containerName="revoke-keys"
Sep 09 03:33:50.154490 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:33:50.154273    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="b4695da1-d62f-4e9c-8bf7-41236e3f0814" containerName="istio-proxy"
Sep 09 03:33:50.157340 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:33:50.157318    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-c44cc65a-openshift-default-66c6b44c5b-cnpfj"
Sep 09 03:33:50.161017 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:33:50.161000    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-mt-c44cc65a-openshift-default-dockercfg-x9j6h\""
Sep 09 03:33:50.173733 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:33:50.173706    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-c44cc65a-openshift-default-66c6b44c5b-cnpfj"]
Sep 09 03:33:50.182184 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:33:50.182159    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/31fc04d2-8016-4b58-ae28-6843df876bc9-istio-envoy\") pod \"e2e-mt-c44cc65a-openshift-default-66c6b44c5b-cnpfj\" (UID: \"31fc04d2-8016-4b58-ae28-6843df876bc9\") " pod="openshift-ingress/e2e-mt-c44cc65a-openshift-default-66c6b44c5b-cnpfj"
Sep 09 03:33:50.182282 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:33:50.182230    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/31fc04d2-8016-4b58-ae28-6843df876bc9-credential-socket\") pod \"e2e-mt-c44cc65a-openshift-default-66c6b44c5b-cnpfj\" (UID: \"31fc04d2-8016-4b58-ae28-6843df876bc9\") " pod="openshift-ingress/e2e-mt-c44cc65a-openshift-default-66c6b44c5b-cnpfj"
Sep 09 03:33:50.182348 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:33:50.182285    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/31fc04d2-8016-4b58-ae28-6843df876bc9-workload-socket\") pod \"e2e-mt-c44cc65a-openshift-default-66c6b44c5b-cnpfj\" (UID: \"31fc04d2-8016-4b58-ae28-6843df876bc9\") " pod="openshift-ingress/e2e-mt-c44cc65a-openshift-default-66c6b44c5b-cnpfj"
Sep 09 03:33:50.182348 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:33:50.182327    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/31fc04d2-8016-4b58-ae28-6843df876bc9-istio-token\") pod \"e2e-mt-c44cc65a-openshift-default-66c6b44c5b-cnpfj\" (UID: \"31fc04d2-8016-4b58-ae28-6843df876bc9\") " pod="openshift-ingress/e2e-mt-c44cc65a-openshift-default-66c6b44c5b-cnpfj"
Sep 09 03:33:50.182439 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:33:50.182371    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/31fc04d2-8016-4b58-ae28-6843df876bc9-workload-certs\") pod \"e2e-mt-c44cc65a-openshift-default-66c6b44c5b-cnpfj\" (UID: \"31fc04d2-8016-4b58-ae28-6843df876bc9\") " pod="openshift-ingress/e2e-mt-c44cc65a-openshift-default-66c6b44c5b-cnpfj"
Sep 09 03:33:50.182479 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:33:50.182436    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/31fc04d2-8016-4b58-ae28-6843df876bc9-istio-podinfo\") pod \"e2e-mt-c44cc65a-openshift-default-66c6b44c5b-cnpfj\" (UID: \"31fc04d2-8016-4b58-ae28-6843df876bc9\") " pod="openshift-ingress/e2e-mt-c44cc65a-openshift-default-66c6b44c5b-cnpfj"
Sep 09 03:33:50.182519 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:33:50.182489    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/31fc04d2-8016-4b58-ae28-6843df876bc9-istio-data\") pod \"e2e-mt-c44cc65a-openshift-default-66c6b44c5b-cnpfj\" (UID: \"31fc04d2-8016-4b58-ae28-6843df876bc9\") " pod="openshift-ingress/e2e-mt-c44cc65a-openshift-default-66c6b44c5b-cnpfj"
Sep 09 03:33:50.182553 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:33:50.182522    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/31fc04d2-8016-4b58-ae28-6843df876bc9-istiod-ca-cert\") pod \"e2e-mt-c44cc65a-openshift-default-66c6b44c5b-cnpfj\" (UID: \"31fc04d2-8016-4b58-ae28-6843df876bc9\") " pod="openshift-ingress/e2e-mt-c44cc65a-openshift-default-66c6b44c5b-cnpfj"
Sep 09 03:33:50.182597 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:33:50.182555    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xv9g2\" (UniqueName: \"kubernetes.io/projected/31fc04d2-8016-4b58-ae28-6843df876bc9-kube-api-access-xv9g2\") pod \"e2e-mt-c44cc65a-openshift-default-66c6b44c5b-cnpfj\" (UID: \"31fc04d2-8016-4b58-ae28-6843df876bc9\") " pod="openshift-ingress/e2e-mt-c44cc65a-openshift-default-66c6b44c5b-cnpfj"
Sep 09 03:33:50.283940 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:33:50.283905    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/31fc04d2-8016-4b58-ae28-6843df876bc9-workload-socket\") pod \"e2e-mt-c44cc65a-openshift-default-66c6b44c5b-cnpfj\" (UID: \"31fc04d2-8016-4b58-ae28-6843df876bc9\") " pod="openshift-ingress/e2e-mt-c44cc65a-openshift-default-66c6b44c5b-cnpfj"
Sep 09 03:33:50.284109 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:33:50.283991    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/31fc04d2-8016-4b58-ae28-6843df876bc9-istio-token\") pod \"e2e-mt-c44cc65a-openshift-default-66c6b44c5b-cnpfj\" (UID: \"31fc04d2-8016-4b58-ae28-6843df876bc9\") " pod="openshift-ingress/e2e-mt-c44cc65a-openshift-default-66c6b44c5b-cnpfj"
Sep 09 03:33:50.284109 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:33:50.284066    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/31fc04d2-8016-4b58-ae28-6843df876bc9-workload-certs\") pod \"e2e-mt-c44cc65a-openshift-default-66c6b44c5b-cnpfj\" (UID: \"31fc04d2-8016-4b58-ae28-6843df876bc9\") " pod="openshift-ingress/e2e-mt-c44cc65a-openshift-default-66c6b44c5b-cnpfj"
Sep 09 03:33:50.284109 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:33:50.284103    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/31fc04d2-8016-4b58-ae28-6843df876bc9-istio-podinfo\") pod \"e2e-mt-c44cc65a-openshift-default-66c6b44c5b-cnpfj\" (UID: \"31fc04d2-8016-4b58-ae28-6843df876bc9\") " pod="openshift-ingress/e2e-mt-c44cc65a-openshift-default-66c6b44c5b-cnpfj"
Sep 09 03:33:50.284281 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:33:50.284149    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/31fc04d2-8016-4b58-ae28-6843df876bc9-istio-data\") pod \"e2e-mt-c44cc65a-openshift-default-66c6b44c5b-cnpfj\" (UID: \"31fc04d2-8016-4b58-ae28-6843df876bc9\") " pod="openshift-ingress/e2e-mt-c44cc65a-openshift-default-66c6b44c5b-cnpfj"
Sep 09 03:33:50.284281 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:33:50.284183    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/31fc04d2-8016-4b58-ae28-6843df876bc9-istiod-ca-cert\") pod \"e2e-mt-c44cc65a-openshift-default-66c6b44c5b-cnpfj\" (UID: \"31fc04d2-8016-4b58-ae28-6843df876bc9\") " pod="openshift-ingress/e2e-mt-c44cc65a-openshift-default-66c6b44c5b-cnpfj"
Sep 09 03:33:50.284281 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:33:50.284220    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xv9g2\" (UniqueName: \"kubernetes.io/projected/31fc04d2-8016-4b58-ae28-6843df876bc9-kube-api-access-xv9g2\") pod \"e2e-mt-c44cc65a-openshift-default-66c6b44c5b-cnpfj\" (UID: \"31fc04d2-8016-4b58-ae28-6843df876bc9\") " pod="openshift-ingress/e2e-mt-c44cc65a-openshift-default-66c6b44c5b-cnpfj"
Sep 09 03:33:50.284281 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:33:50.284257    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/31fc04d2-8016-4b58-ae28-6843df876bc9-istio-envoy\") pod \"e2e-mt-c44cc65a-openshift-default-66c6b44c5b-cnpfj\" (UID: \"31fc04d2-8016-4b58-ae28-6843df876bc9\") " pod="openshift-ingress/e2e-mt-c44cc65a-openshift-default-66c6b44c5b-cnpfj"
Sep 09 03:33:50.284750 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:33:50.284320    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/31fc04d2-8016-4b58-ae28-6843df876bc9-credential-socket\") pod \"e2e-mt-c44cc65a-openshift-default-66c6b44c5b-cnpfj\" (UID: \"31fc04d2-8016-4b58-ae28-6843df876bc9\") " pod="openshift-ingress/e2e-mt-c44cc65a-openshift-default-66c6b44c5b-cnpfj"
Sep 09 03:33:50.284750 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:33:50.284323    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/31fc04d2-8016-4b58-ae28-6843df876bc9-workload-socket\") pod \"e2e-mt-c44cc65a-openshift-default-66c6b44c5b-cnpfj\" (UID: \"31fc04d2-8016-4b58-ae28-6843df876bc9\") " pod="openshift-ingress/e2e-mt-c44cc65a-openshift-default-66c6b44c5b-cnpfj"
Sep 09 03:33:50.284750 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:33:50.284419    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/31fc04d2-8016-4b58-ae28-6843df876bc9-workload-certs\") pod \"e2e-mt-c44cc65a-openshift-default-66c6b44c5b-cnpfj\" (UID: \"31fc04d2-8016-4b58-ae28-6843df876bc9\") " pod="openshift-ingress/e2e-mt-c44cc65a-openshift-default-66c6b44c5b-cnpfj"
Sep 09 03:33:50.284750 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:33:50.284539    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/31fc04d2-8016-4b58-ae28-6843df876bc9-istio-data\") pod \"e2e-mt-c44cc65a-openshift-default-66c6b44c5b-cnpfj\" (UID: \"31fc04d2-8016-4b58-ae28-6843df876bc9\") " pod="openshift-ingress/e2e-mt-c44cc65a-openshift-default-66c6b44c5b-cnpfj"
Sep 09 03:33:50.284989 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:33:50.284921    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/31fc04d2-8016-4b58-ae28-6843df876bc9-istiod-ca-cert\") pod \"e2e-mt-c44cc65a-openshift-default-66c6b44c5b-cnpfj\" (UID: \"31fc04d2-8016-4b58-ae28-6843df876bc9\") " pod="openshift-ingress/e2e-mt-c44cc65a-openshift-default-66c6b44c5b-cnpfj"
Sep 09 03:33:50.285064 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:33:50.285046    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/31fc04d2-8016-4b58-ae28-6843df876bc9-credential-socket\") pod \"e2e-mt-c44cc65a-openshift-default-66c6b44c5b-cnpfj\" (UID: \"31fc04d2-8016-4b58-ae28-6843df876bc9\") " pod="openshift-ingress/e2e-mt-c44cc65a-openshift-default-66c6b44c5b-cnpfj"
Sep 09 03:33:50.287021 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:33:50.286996    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/31fc04d2-8016-4b58-ae28-6843df876bc9-istio-envoy\") pod \"e2e-mt-c44cc65a-openshift-default-66c6b44c5b-cnpfj\" (UID: \"31fc04d2-8016-4b58-ae28-6843df876bc9\") " pod="openshift-ingress/e2e-mt-c44cc65a-openshift-default-66c6b44c5b-cnpfj"
Sep 09 03:33:50.287466 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:33:50.287448    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/31fc04d2-8016-4b58-ae28-6843df876bc9-istio-podinfo\") pod \"e2e-mt-c44cc65a-openshift-default-66c6b44c5b-cnpfj\" (UID: \"31fc04d2-8016-4b58-ae28-6843df876bc9\") " pod="openshift-ingress/e2e-mt-c44cc65a-openshift-default-66c6b44c5b-cnpfj"
Sep 09 03:33:50.295142 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:33:50.295120    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/31fc04d2-8016-4b58-ae28-6843df876bc9-istio-token\") pod \"e2e-mt-c44cc65a-openshift-default-66c6b44c5b-cnpfj\" (UID: \"31fc04d2-8016-4b58-ae28-6843df876bc9\") " pod="openshift-ingress/e2e-mt-c44cc65a-openshift-default-66c6b44c5b-cnpfj"
Sep 09 03:33:50.295653 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:33:50.295630    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xv9g2\" (UniqueName: \"kubernetes.io/projected/31fc04d2-8016-4b58-ae28-6843df876bc9-kube-api-access-xv9g2\") pod \"e2e-mt-c44cc65a-openshift-default-66c6b44c5b-cnpfj\" (UID: \"31fc04d2-8016-4b58-ae28-6843df876bc9\") " pod="openshift-ingress/e2e-mt-c44cc65a-openshift-default-66c6b44c5b-cnpfj"
Sep 09 03:33:50.467095 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:33:50.467020    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-c44cc65a-openshift-default-66c6b44c5b-cnpfj"
Sep 09 03:33:50.751160 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:33:50.751138    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-c44cc65a-openshift-default-66c6b44c5b-cnpfj"]
Sep 09 03:33:50.753344 ip-10-0-132-14 kubenswrapper[2576]: W0909 03:33:50.753316    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod31fc04d2_8016_4b58_ae28_6843df876bc9.slice/crio-276fef87a4f7015a0dddf03b9f7950f91fc5d2c964f42fa260921ef39e93635d WatchSource:0}: Error finding container 276fef87a4f7015a0dddf03b9f7950f91fc5d2c964f42fa260921ef39e93635d: Status 404 returned error can't find the container with id 276fef87a4f7015a0dddf03b9f7950f91fc5d2c964f42fa260921ef39e93635d
Sep 09 03:33:50.755388 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:33:50.755357    2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892148Ki","pods":"250"}
Sep 09 03:33:50.755464 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:33:50.755425    2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892148Ki","pods":"250"}
Sep 09 03:33:50.755464 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:33:50.755452    2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892148Ki","pods":"250"}
Sep 09 03:33:51.488439 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:33:51.488406    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-c44cc65a-openshift-default-66c6b44c5b-cnpfj" event={"ID":"31fc04d2-8016-4b58-ae28-6843df876bc9","Type":"ContainerStarted","Data":"8ad0663bdd18bb0f385601a5408910693aaa1787af0a11b8ad8cccf2029f34a7"}
Sep 09 03:33:51.488911 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:33:51.488462    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-c44cc65a-openshift-default-66c6b44c5b-cnpfj" event={"ID":"31fc04d2-8016-4b58-ae28-6843df876bc9","Type":"ContainerStarted","Data":"276fef87a4f7015a0dddf03b9f7950f91fc5d2c964f42fa260921ef39e93635d"}
Sep 09 03:33:51.512757 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:33:51.512697    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-mt-c44cc65a-openshift-default-66c6b44c5b-cnpfj" podStartSLOduration=1.512681568 podStartE2EDuration="1.512681568s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 03:33:50 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 03:33:51.510833684 +0000 UTC m=+1245.652319752" watchObservedRunningTime="2026-09-09 03:33:51.512681568 +0000 UTC m=+1245.654167636"
Sep 09 03:33:52.259755 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:33:52.259724    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-a-40a1c9-5d8b764ddf-rwn6f"]
Sep 09 03:33:52.260047 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:33:52.259998    2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-inf-a-40a1c9-5d8b764ddf-rwn6f" podUID="dd6e63da-0e5d-49c1-89b3-8df1f5d52a19" containerName="payload-processing" containerID="cri-o://1298772a45eab162c86395165ce9f37eb731ff43f5f5216476583898ee363421" gracePeriod=30
Sep 09 03:33:52.268183 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:33:52.268046    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-a-40a1c9-7475b65c4-x5f5l"]
Sep 09 03:33:52.268322 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:33:52.268272    2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-40a1c9-7475b65c4-x5f5l" podUID="0dd1fa3c-7ce6-46ce-acb7-5da3e43ea653" containerName="payload-pre-processing" containerID="cri-o://3d16fe803556134574d8de6ad4cd6b605fdffdad6472ef7338f2144e51d386e7" gracePeriod=30
Sep 09 03:33:52.467969 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:33:52.467921    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-c44cc65a-openshift-default-66c6b44c5b-cnpfj"
Sep 09 03:33:52.473199 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:33:52.473165    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-c44cc65a-openshift-default-66c6b44c5b-cnpfj"
Sep 09 03:33:52.496132 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:33:52.496097    2576 generic.go:356] "Generic (PLEG): container finished" podID="0dd1fa3c-7ce6-46ce-acb7-5da3e43ea653" containerID="3d16fe803556134574d8de6ad4cd6b605fdffdad6472ef7338f2144e51d386e7" exitCode=0
Sep 09 03:33:52.496554 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:33:52.496164    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-40a1c9-7475b65c4-x5f5l" event={"ID":"0dd1fa3c-7ce6-46ce-acb7-5da3e43ea653","Type":"ContainerDied","Data":"3d16fe803556134574d8de6ad4cd6b605fdffdad6472ef7338f2144e51d386e7"}
Sep 09 03:33:52.498234 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:33:52.498029    2576 generic.go:356] "Generic (PLEG): container finished" podID="dd6e63da-0e5d-49c1-89b3-8df1f5d52a19" containerID="1298772a45eab162c86395165ce9f37eb731ff43f5f5216476583898ee363421" exitCode=0
Sep 09 03:33:52.498234 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:33:52.498143    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-a-40a1c9-5d8b764ddf-rwn6f" event={"ID":"dd6e63da-0e5d-49c1-89b3-8df1f5d52a19","Type":"ContainerDied","Data":"1298772a45eab162c86395165ce9f37eb731ff43f5f5216476583898ee363421"}
Sep 09 03:33:52.498520 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:33:52.498497    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-c44cc65a-openshift-default-66c6b44c5b-cnpfj"
Sep 09 03:33:52.499428 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:33:52.499405    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-c44cc65a-openshift-default-66c6b44c5b-cnpfj"
Sep 09 03:33:52.536108 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:33:52.536079    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-40a1c9-s8fr8"]
Sep 09 03:33:52.545337 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:33:52.545292    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-a-40a1c9-5d8b764ddf-rwn6f"
Sep 09 03:33:52.546337 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:33:52.546277    2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-40a1c9-s8fr8"]
Sep 09 03:33:52.552738 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:33:52.552704    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-40a1c9-7475b65c4-x5f5l"
Sep 09 03:33:52.606430 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:33:52.606400    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/0dd1fa3c-7ce6-46ce-acb7-5da3e43ea653-plugins-config-volume\") pod \"0dd1fa3c-7ce6-46ce-acb7-5da3e43ea653\" (UID: \"0dd1fa3c-7ce6-46ce-acb7-5da3e43ea653\") "
Sep 09 03:33:52.606607 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:33:52.606437    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/dd6e63da-0e5d-49c1-89b3-8df1f5d52a19-plugins-config-volume\") pod \"dd6e63da-0e5d-49c1-89b3-8df1f5d52a19\" (UID: \"dd6e63da-0e5d-49c1-89b3-8df1f5d52a19\") "
Sep 09 03:33:52.606607 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:33:52.606459    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-78gdh\" (UniqueName: \"kubernetes.io/projected/0dd1fa3c-7ce6-46ce-acb7-5da3e43ea653-kube-api-access-78gdh\") pod \"0dd1fa3c-7ce6-46ce-acb7-5da3e43ea653\" (UID: \"0dd1fa3c-7ce6-46ce-acb7-5da3e43ea653\") "
Sep 09 03:33:52.606607 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:33:52.606539    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-7st6n\" (UniqueName: \"kubernetes.io/projected/dd6e63da-0e5d-49c1-89b3-8df1f5d52a19-kube-api-access-7st6n\") pod \"dd6e63da-0e5d-49c1-89b3-8df1f5d52a19\" (UID: \"dd6e63da-0e5d-49c1-89b3-8df1f5d52a19\") "
Sep 09 03:33:52.606912 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:33:52.606877    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0dd1fa3c-7ce6-46ce-acb7-5da3e43ea653-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "0dd1fa3c-7ce6-46ce-acb7-5da3e43ea653" (UID: "0dd1fa3c-7ce6-46ce-acb7-5da3e43ea653"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 09 03:33:52.607082 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:33:52.606883    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/dd6e63da-0e5d-49c1-89b3-8df1f5d52a19-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "dd6e63da-0e5d-49c1-89b3-8df1f5d52a19" (UID: "dd6e63da-0e5d-49c1-89b3-8df1f5d52a19"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 09 03:33:52.609116 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:33:52.609090    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0dd1fa3c-7ce6-46ce-acb7-5da3e43ea653-kube-api-access-78gdh" (OuterVolumeSpecName: "kube-api-access-78gdh") pod "0dd1fa3c-7ce6-46ce-acb7-5da3e43ea653" (UID: "0dd1fa3c-7ce6-46ce-acb7-5da3e43ea653"). InnerVolumeSpecName "kube-api-access-78gdh". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 09 03:33:52.609280 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:33:52.609261    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/dd6e63da-0e5d-49c1-89b3-8df1f5d52a19-kube-api-access-7st6n" (OuterVolumeSpecName: "kube-api-access-7st6n") pod "dd6e63da-0e5d-49c1-89b3-8df1f5d52a19" (UID: "dd6e63da-0e5d-49c1-89b3-8df1f5d52a19"). InnerVolumeSpecName "kube-api-access-7st6n". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 09 03:33:52.708203 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:33:52.708169    2576 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/dd6e63da-0e5d-49c1-89b3-8df1f5d52a19-plugins-config-volume\") on node \"ip-10-0-132-14.ec2.internal\" DevicePath \"\""
Sep 09 03:33:52.708203 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:33:52.708196    2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-78gdh\" (UniqueName: \"kubernetes.io/projected/0dd1fa3c-7ce6-46ce-acb7-5da3e43ea653-kube-api-access-78gdh\") on node \"ip-10-0-132-14.ec2.internal\" DevicePath \"\""
Sep 09 03:33:52.708203 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:33:52.708206    2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7st6n\" (UniqueName: \"kubernetes.io/projected/dd6e63da-0e5d-49c1-89b3-8df1f5d52a19-kube-api-access-7st6n\") on node \"ip-10-0-132-14.ec2.internal\" DevicePath \"\""
Sep 09 03:33:52.708437 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:33:52.708215    2576 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/0dd1fa3c-7ce6-46ce-acb7-5da3e43ea653-plugins-config-volume\") on node \"ip-10-0-132-14.ec2.internal\" DevicePath \"\""
Sep 09 03:33:53.504426 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:33:53.504391    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-40a1c9-7475b65c4-x5f5l"
Sep 09 03:33:53.504925 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:33:53.504434    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-40a1c9-7475b65c4-x5f5l" event={"ID":"0dd1fa3c-7ce6-46ce-acb7-5da3e43ea653","Type":"ContainerDied","Data":"43402e52bbb6a574754623165703e9fac885d68e10846783a5f89938ea2f63f2"}
Sep 09 03:33:53.504925 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:33:53.504505    2576 scope.go:117] "RemoveContainer" containerID="3d16fe803556134574d8de6ad4cd6b605fdffdad6472ef7338f2144e51d386e7"
Sep 09 03:33:53.505955 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:33:53.505875    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-a-40a1c9-5d8b764ddf-rwn6f"
Sep 09 03:33:53.505955 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:33:53.505914    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-a-40a1c9-5d8b764ddf-rwn6f" event={"ID":"dd6e63da-0e5d-49c1-89b3-8df1f5d52a19","Type":"ContainerDied","Data":"6da6b7dbc265e8f150d2a0a85e3f671d78314c8c1706765739f2c3c0adf7a60f"}
Sep 09 03:33:53.513660 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:33:53.513644    2576 scope.go:117] "RemoveContainer" containerID="1298772a45eab162c86395165ce9f37eb731ff43f5f5216476583898ee363421"
Sep 09 03:33:53.542655 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:33:53.542624    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-a-40a1c9-7475b65c4-x5f5l"]
Sep 09 03:33:53.549915 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:33:53.549895    2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-a-40a1c9-7475b65c4-x5f5l"]
Sep 09 03:33:53.563946 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:33:53.563926    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-a-40a1c9-5d8b764ddf-rwn6f"]
Sep 09 03:33:53.565257 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:33:53.565241    2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-a-40a1c9-5d8b764ddf-rwn6f"]
Sep 09 03:33:54.360415 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:33:54.360381    2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="057e7b78-3925-47b5-8c44-cca91b377eb2" path="/var/lib/kubelet/pods/057e7b78-3925-47b5-8c44-cca91b377eb2/volumes"
Sep 09 03:33:54.360732 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:33:54.360719    2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0dd1fa3c-7ce6-46ce-acb7-5da3e43ea653" path="/var/lib/kubelet/pods/0dd1fa3c-7ce6-46ce-acb7-5da3e43ea653/volumes"
Sep 09 03:33:54.361363 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:33:54.361310    2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="dd6e63da-0e5d-49c1-89b3-8df1f5d52a19" path="/var/lib/kubelet/pods/dd6e63da-0e5d-49c1-89b3-8df1f5d52a19/volumes"
Sep 09 03:33:55.921416 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:33:55.921385    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-c44cc65a-84bfb86d66-4bp7d"]
Sep 09 03:33:55.921837 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:33:55.921722    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0dd1fa3c-7ce6-46ce-acb7-5da3e43ea653" containerName="payload-pre-processing"
Sep 09 03:33:55.921837 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:33:55.921733    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="0dd1fa3c-7ce6-46ce-acb7-5da3e43ea653" containerName="payload-pre-processing"
Sep 09 03:33:55.921837 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:33:55.921748    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="dd6e63da-0e5d-49c1-89b3-8df1f5d52a19" containerName="payload-processing"
Sep 09 03:33:55.921837 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:33:55.921753    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="dd6e63da-0e5d-49c1-89b3-8df1f5d52a19" containerName="payload-processing"
Sep 09 03:33:55.921837 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:33:55.921821    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="dd6e63da-0e5d-49c1-89b3-8df1f5d52a19" containerName="payload-processing"
Sep 09 03:33:55.921837 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:33:55.921831    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="0dd1fa3c-7ce6-46ce-acb7-5da3e43ea653" containerName="payload-pre-processing"
Sep 09 03:33:55.926094 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:33:55.926076    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-c44cc65a-84bfb86d66-4bp7d"
Sep 09 03:33:55.929357 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:33:55.929340    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-mt-c44cc65a-dockercfg-x2td9\""
Sep 09 03:33:55.929733 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:33:55.929715    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-mt-c44cc65a\""
Sep 09 03:33:55.944050 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:33:55.944020    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-c44cc65a-84bfb86d66-4bp7d"]
Sep 09 03:33:55.984573 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:33:55.984542    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-c44cc65a-559789876f-24b8x"]
Sep 09 03:33:55.988511 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:33:55.988483    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-c44cc65a-559789876f-24b8x"
Sep 09 03:33:56.005584 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:33:56.005550    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-c44cc65a-559789876f-24b8x"]
Sep 09 03:33:56.036421 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:33:56.036394    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-n7pnq\" (UniqueName: \"kubernetes.io/projected/1e24840e-811b-4d59-8402-1586c3153e6d-kube-api-access-n7pnq\") pod \"payload-pre-processing-e2e-mt-c44cc65a-84bfb86d66-4bp7d\" (UID: \"1e24840e-811b-4d59-8402-1586c3153e6d\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-c44cc65a-84bfb86d66-4bp7d"
Sep 09 03:33:56.036575 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:33:56.036437    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/1e24840e-811b-4d59-8402-1586c3153e6d-plugins-config-volume\") pod \"payload-pre-processing-e2e-mt-c44cc65a-84bfb86d66-4bp7d\" (UID: \"1e24840e-811b-4d59-8402-1586c3153e6d\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-c44cc65a-84bfb86d66-4bp7d"
Sep 09 03:33:56.036575 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:33:56.036456    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mvvld\" (UniqueName: \"kubernetes.io/projected/7c7e63b7-e5da-4ca9-aa41-46ea78a3e2cb-kube-api-access-mvvld\") pod \"payload-processing-e2e-mt-c44cc65a-559789876f-24b8x\" (UID: \"7c7e63b7-e5da-4ca9-aa41-46ea78a3e2cb\") " pod="openshift-ingress/payload-processing-e2e-mt-c44cc65a-559789876f-24b8x"
Sep 09 03:33:56.036575 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:33:56.036491    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/7c7e63b7-e5da-4ca9-aa41-46ea78a3e2cb-plugins-config-volume\") pod \"payload-processing-e2e-mt-c44cc65a-559789876f-24b8x\" (UID: \"7c7e63b7-e5da-4ca9-aa41-46ea78a3e2cb\") " pod="openshift-ingress/payload-processing-e2e-mt-c44cc65a-559789876f-24b8x"
Sep 09 03:33:56.137697 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:33:56.137662    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-n7pnq\" (UniqueName: \"kubernetes.io/projected/1e24840e-811b-4d59-8402-1586c3153e6d-kube-api-access-n7pnq\") pod \"payload-pre-processing-e2e-mt-c44cc65a-84bfb86d66-4bp7d\" (UID: \"1e24840e-811b-4d59-8402-1586c3153e6d\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-c44cc65a-84bfb86d66-4bp7d"
Sep 09 03:33:56.137944 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:33:56.137714    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/1e24840e-811b-4d59-8402-1586c3153e6d-plugins-config-volume\") pod \"payload-pre-processing-e2e-mt-c44cc65a-84bfb86d66-4bp7d\" (UID: \"1e24840e-811b-4d59-8402-1586c3153e6d\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-c44cc65a-84bfb86d66-4bp7d"
Sep 09 03:33:56.137944 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:33:56.137734    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mvvld\" (UniqueName: \"kubernetes.io/projected/7c7e63b7-e5da-4ca9-aa41-46ea78a3e2cb-kube-api-access-mvvld\") pod \"payload-processing-e2e-mt-c44cc65a-559789876f-24b8x\" (UID: \"7c7e63b7-e5da-4ca9-aa41-46ea78a3e2cb\") " pod="openshift-ingress/payload-processing-e2e-mt-c44cc65a-559789876f-24b8x"
Sep 09 03:33:56.137944 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:33:56.137761    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/7c7e63b7-e5da-4ca9-aa41-46ea78a3e2cb-plugins-config-volume\") pod \"payload-processing-e2e-mt-c44cc65a-559789876f-24b8x\" (UID: \"7c7e63b7-e5da-4ca9-aa41-46ea78a3e2cb\") " pod="openshift-ingress/payload-processing-e2e-mt-c44cc65a-559789876f-24b8x"
Sep 09 03:33:56.138623 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:33:56.138596    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/7c7e63b7-e5da-4ca9-aa41-46ea78a3e2cb-plugins-config-volume\") pod \"payload-processing-e2e-mt-c44cc65a-559789876f-24b8x\" (UID: \"7c7e63b7-e5da-4ca9-aa41-46ea78a3e2cb\") " pod="openshift-ingress/payload-processing-e2e-mt-c44cc65a-559789876f-24b8x"
Sep 09 03:33:56.138720 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:33:56.138596    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/1e24840e-811b-4d59-8402-1586c3153e6d-plugins-config-volume\") pod \"payload-pre-processing-e2e-mt-c44cc65a-84bfb86d66-4bp7d\" (UID: \"1e24840e-811b-4d59-8402-1586c3153e6d\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-c44cc65a-84bfb86d66-4bp7d"
Sep 09 03:33:56.153268 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:33:56.153240    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mvvld\" (UniqueName: \"kubernetes.io/projected/7c7e63b7-e5da-4ca9-aa41-46ea78a3e2cb-kube-api-access-mvvld\") pod \"payload-processing-e2e-mt-c44cc65a-559789876f-24b8x\" (UID: \"7c7e63b7-e5da-4ca9-aa41-46ea78a3e2cb\") " pod="openshift-ingress/payload-processing-e2e-mt-c44cc65a-559789876f-24b8x"
Sep 09 03:33:56.153350 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:33:56.153333    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-n7pnq\" (UniqueName: \"kubernetes.io/projected/1e24840e-811b-4d59-8402-1586c3153e6d-kube-api-access-n7pnq\") pod \"payload-pre-processing-e2e-mt-c44cc65a-84bfb86d66-4bp7d\" (UID: \"1e24840e-811b-4d59-8402-1586c3153e6d\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-c44cc65a-84bfb86d66-4bp7d"
Sep 09 03:33:56.235704 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:33:56.235633    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-c44cc65a-84bfb86d66-4bp7d"
Sep 09 03:33:56.299605 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:33:56.299358    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-c44cc65a-559789876f-24b8x"
Sep 09 03:33:56.515320 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:33:56.515296    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-c44cc65a-84bfb86d66-4bp7d"]
Sep 09 03:33:56.517168 ip-10-0-132-14 kubenswrapper[2576]: W0909 03:33:56.517144    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1e24840e_811b_4d59_8402_1586c3153e6d.slice/crio-224aa78ee0dbe81567632477eb4dd030ddee7d94cd4c1abe51d9103a9e696532 WatchSource:0}: Error finding container 224aa78ee0dbe81567632477eb4dd030ddee7d94cd4c1abe51d9103a9e696532: Status 404 returned error can't find the container with id 224aa78ee0dbe81567632477eb4dd030ddee7d94cd4c1abe51d9103a9e696532
Sep 09 03:33:56.527402 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:33:56.527379    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-c44cc65a-84bfb86d66-4bp7d" event={"ID":"1e24840e-811b-4d59-8402-1586c3153e6d","Type":"ContainerStarted","Data":"224aa78ee0dbe81567632477eb4dd030ddee7d94cd4c1abe51d9103a9e696532"}
Sep 09 03:33:56.578324 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:33:56.578299    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-c44cc65a-559789876f-24b8x"]
Sep 09 03:33:56.579000 ip-10-0-132-14 kubenswrapper[2576]: W0909 03:33:56.578965    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7c7e63b7_e5da_4ca9_aa41_46ea78a3e2cb.slice/crio-43ae23f885bbf3549bf5f2563b62c7414d076c621f38afc653f3719a3e8ad46c WatchSource:0}: Error finding container 43ae23f885bbf3549bf5f2563b62c7414d076c621f38afc653f3719a3e8ad46c: Status 404 returned error can't find the container with id 43ae23f885bbf3549bf5f2563b62c7414d076c621f38afc653f3719a3e8ad46c
Sep 09 03:33:57.534502 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:33:57.534410    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-c44cc65a-559789876f-24b8x" event={"ID":"7c7e63b7-e5da-4ca9-aa41-46ea78a3e2cb","Type":"ContainerStarted","Data":"fe5e21ee3adaa188e5dec15c00340f9d2ee14ba20fcf4e95d028574ec587ddc1"}
Sep 09 03:33:57.534502 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:33:57.534485    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-c44cc65a-559789876f-24b8x" event={"ID":"7c7e63b7-e5da-4ca9-aa41-46ea78a3e2cb","Type":"ContainerStarted","Data":"43ae23f885bbf3549bf5f2563b62c7414d076c621f38afc653f3719a3e8ad46c"}
Sep 09 03:33:57.534980 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:33:57.534516    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-mt-c44cc65a-559789876f-24b8x"
Sep 09 03:33:57.536199 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:33:57.536175    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-c44cc65a-84bfb86d66-4bp7d" event={"ID":"1e24840e-811b-4d59-8402-1586c3153e6d","Type":"ContainerStarted","Data":"402ecbd96a66dfa6c5baf5ac2fbba7b3f45c96317e20cc667ae15169c70fe1e7"}
Sep 09 03:33:57.536322 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:33:57.536303    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-mt-c44cc65a-84bfb86d66-4bp7d"
Sep 09 03:33:57.571765 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:33:57.571723    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-mt-c44cc65a-559789876f-24b8x" podStartSLOduration=2.571709919 podStartE2EDuration="2.571709919s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 03:33:55 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 03:33:57.569520797 +0000 UTC m=+1251.711006873" watchObservedRunningTime="2026-09-09 03:33:57.571709919 +0000 UTC m=+1251.713195993"
Sep 09 03:33:57.610928 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:33:57.610884    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-mt-c44cc65a-84bfb86d66-4bp7d" podStartSLOduration=2.610870836 podStartE2EDuration="2.610870836s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 03:33:55 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 03:33:57.6096601 +0000 UTC m=+1251.751146179" watchObservedRunningTime="2026-09-09 03:33:57.610870836 +0000 UTC m=+1251.752356913"
Sep 09 03:33:59.736284 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:33:59.736244    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-4bd6b8-8zbc7"]
Sep 09 03:33:59.739456 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:33:59.739433    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-4bd6b8-8zbc7"
Sep 09 03:33:59.742335 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:33:59.742313    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-w7q9f\""
Sep 09 03:33:59.748140 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:33:59.748121    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-4bd6b8-8zbc7"]
Sep 09 03:33:59.873428 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:33:59.873395    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/0f29a419-a0b2-44a1-9846-3e911136dc54-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-b-4bd6b8-8zbc7\" (UID: \"0f29a419-a0b2-44a1-9846-3e911136dc54\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-4bd6b8-8zbc7"
Sep 09 03:33:59.974075 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:33:59.974041    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/0f29a419-a0b2-44a1-9846-3e911136dc54-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-b-4bd6b8-8zbc7\" (UID: \"0f29a419-a0b2-44a1-9846-3e911136dc54\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-4bd6b8-8zbc7"
Sep 09 03:33:59.974647 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:33:59.974628    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/0f29a419-a0b2-44a1-9846-3e911136dc54-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-b-4bd6b8-8zbc7\" (UID: \"0f29a419-a0b2-44a1-9846-3e911136dc54\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-4bd6b8-8zbc7"
Sep 09 03:34:00.052893 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:34:00.052857    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-4bd6b8-8zbc7"
Sep 09 03:34:00.354190 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:34:00.354154    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-4bd6b8-8zbc7"]
Sep 09 03:34:00.355136 ip-10-0-132-14 kubenswrapper[2576]: W0909 03:34:00.355106    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0f29a419_a0b2_44a1_9846_3e911136dc54.slice/crio-ab40f2c662dd4bbac5605b74f31c113cee0e8e62987828a958a4753c57bc3b78 WatchSource:0}: Error finding container ab40f2c662dd4bbac5605b74f31c113cee0e8e62987828a958a4753c57bc3b78: Status 404 returned error can't find the container with id ab40f2c662dd4bbac5605b74f31c113cee0e8e62987828a958a4753c57bc3b78
Sep 09 03:34:00.555738 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:34:00.555676    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-4bd6b8-8zbc7" event={"ID":"0f29a419-a0b2-44a1-9846-3e911136dc54","Type":"ContainerStarted","Data":"ab40f2c662dd4bbac5605b74f31c113cee0e8e62987828a958a4753c57bc3b78"}
Sep 09 03:34:01.562422 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:34:01.562386    2576 generic.go:356] "Generic (PLEG): container finished" podID="0f29a419-a0b2-44a1-9846-3e911136dc54" containerID="9fbb9eb408510f1b36d25704266cc94999b76154778616b65dfa2d94786f900f" exitCode=0
Sep 09 03:34:01.562822 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:34:01.562474    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-4bd6b8-8zbc7" event={"ID":"0f29a419-a0b2-44a1-9846-3e911136dc54","Type":"ContainerDied","Data":"9fbb9eb408510f1b36d25704266cc94999b76154778616b65dfa2d94786f900f"}
Sep 09 03:34:02.706581 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:34:02.706558    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-4bd6b8-8zbc7"
Sep 09 03:34:02.802394 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:34:02.802364    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/0f29a419-a0b2-44a1-9846-3e911136dc54-maas-api-service-ca\") pod \"0f29a419-a0b2-44a1-9846-3e911136dc54\" (UID: \"0f29a419-a0b2-44a1-9846-3e911136dc54\") "
Sep 09 03:34:02.802689 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:34:02.802667    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0f29a419-a0b2-44a1-9846-3e911136dc54-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "0f29a419-a0b2-44a1-9846-3e911136dc54" (UID: "0f29a419-a0b2-44a1-9846-3e911136dc54"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 09 03:34:02.903966 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:34:02.903879    2576 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/0f29a419-a0b2-44a1-9846-3e911136dc54-maas-api-service-ca\") on node \"ip-10-0-132-14.ec2.internal\" DevicePath \"\""
Sep 09 03:34:03.573793 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:34:03.573759    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-4bd6b8-8zbc7" event={"ID":"0f29a419-a0b2-44a1-9846-3e911136dc54","Type":"ContainerDied","Data":"ab40f2c662dd4bbac5605b74f31c113cee0e8e62987828a958a4753c57bc3b78"}
Sep 09 03:34:03.573793 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:34:03.573774    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-4bd6b8-8zbc7"
Sep 09 03:34:03.574063 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:34:03.573829    2576 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="ab40f2c662dd4bbac5605b74f31c113cee0e8e62987828a958a4753c57bc3b78"
Sep 09 03:34:08.539983 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:34:08.539943    2576 patch_prober.go:28] interesting pod/e2e-mt-c44cc65a-openshift-default-66c6b44c5b-cnpfj container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.50:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Sep 09 03:34:08.542264 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:34:08.540026    2576 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-mt-c44cc65a-openshift-default-66c6b44c5b-cnpfj" podUID="31fc04d2-8016-4b58-ae28-6843df876bc9" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.50:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Sep 09 03:34:08.542738 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:34:08.542715    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-mt-c44cc65a-559789876f-24b8x"
Sep 09 03:34:08.543034 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:34:08.543014    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-mt-c44cc65a-84bfb86d66-4bp7d"
Sep 09 03:34:08.603374 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:34:08.603347    2576 generic.go:356] "Generic (PLEG): container finished" podID="31fc04d2-8016-4b58-ae28-6843df876bc9" containerID="8ad0663bdd18bb0f385601a5408910693aaa1787af0a11b8ad8cccf2029f34a7" exitCode=0
Sep 09 03:34:08.603489 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:34:08.603424    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-c44cc65a-openshift-default-66c6b44c5b-cnpfj" event={"ID":"31fc04d2-8016-4b58-ae28-6843df876bc9","Type":"ContainerDied","Data":"8ad0663bdd18bb0f385601a5408910693aaa1787af0a11b8ad8cccf2029f34a7"}
Sep 09 03:34:08.603944 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:34:08.603924    2576 scope.go:117] "RemoveContainer" containerID="8ad0663bdd18bb0f385601a5408910693aaa1787af0a11b8ad8cccf2029f34a7"
Sep 09 03:34:08.605064 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:34:08.605036    2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892148Ki","pods":"250"}
Sep 09 03:34:08.605150 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:34:08.605106    2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892148Ki","pods":"250"}
Sep 09 03:34:08.605150 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:34:08.605137    2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892148Ki","pods":"250"}
Sep 09 03:34:09.610308 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:34:09.610272    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-c44cc65a-openshift-default-66c6b44c5b-cnpfj" event={"ID":"31fc04d2-8016-4b58-ae28-6843df876bc9","Type":"ContainerStarted","Data":"6b9af6686aa3d4fa2433c7a268ba319552ad78001e47e6f3dd1581dd8ae03fa1"}
Sep 09 03:34:10.467879 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:34:10.467838    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-c44cc65a-openshift-default-66c6b44c5b-cnpfj"
Sep 09 03:34:10.468083 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:34:10.468057    2576 patch_prober.go:28] interesting pod/e2e-mt-c44cc65a-openshift-default-66c6b44c5b-cnpfj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.50:15021/healthz/ready\": dial tcp 10.133.0.50:15021: connect: connection refused" start-of-body=
Sep 09 03:34:10.468158 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:34:10.468115    2576 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-c44cc65a-openshift-default-66c6b44c5b-cnpfj" podUID="31fc04d2-8016-4b58-ae28-6843df876bc9" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.50:15021/healthz/ready\": dial tcp 10.133.0.50:15021: connect: connection refused"
Sep 09 03:34:10.869225 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:34:10.869188    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-b-4bd6b8-6458c89d79-jt8fk"]
Sep 09 03:34:10.869640 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:34:10.869408    2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-inf-b-4bd6b8-6458c89d79-jt8fk" podUID="766d00dc-b659-4682-b44b-4e42a6c4e844" containerName="payload-processing" containerID="cri-o://0ec19459f242ce4cd865d15e9983f2e0a775f18244f99aaf76759ceb5972b29d" gracePeriod=30
Sep 09 03:34:10.885172 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:34:10.885113    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-b-4bd6b8-6f98d797d-h8b2x"]
Sep 09 03:34:10.885361 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:34:10.885338    2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-4bd6b8-6f98d797d-h8b2x" podUID="a2041f45-6ca8-4207-9604-1c8890c9e9ea" containerName="payload-pre-processing" containerID="cri-o://2aad3eaa40285cefc7ec738749512a3382b4d829046c254d6716c8bb0f9078c3" gracePeriod=30
Sep 09 03:34:11.190194 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:34:11.190172    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-b-4bd6b8-6458c89d79-jt8fk"
Sep 09 03:34:11.195241 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:34:11.195095    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-4bd6b8-6f98d797d-h8b2x"
Sep 09 03:34:11.276712 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:34:11.276670    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/766d00dc-b659-4682-b44b-4e42a6c4e844-plugins-config-volume\") pod \"766d00dc-b659-4682-b44b-4e42a6c4e844\" (UID: \"766d00dc-b659-4682-b44b-4e42a6c4e844\") "
Sep 09 03:34:11.276927 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:34:11.276785    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-2grgt\" (UniqueName: \"kubernetes.io/projected/a2041f45-6ca8-4207-9604-1c8890c9e9ea-kube-api-access-2grgt\") pod \"a2041f45-6ca8-4207-9604-1c8890c9e9ea\" (UID: \"a2041f45-6ca8-4207-9604-1c8890c9e9ea\") "
Sep 09 03:34:11.276927 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:34:11.276845    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-b88q7\" (UniqueName: \"kubernetes.io/projected/766d00dc-b659-4682-b44b-4e42a6c4e844-kube-api-access-b88q7\") pod \"766d00dc-b659-4682-b44b-4e42a6c4e844\" (UID: \"766d00dc-b659-4682-b44b-4e42a6c4e844\") "
Sep 09 03:34:11.276927 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:34:11.276874    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a2041f45-6ca8-4207-9604-1c8890c9e9ea-plugins-config-volume\") pod \"a2041f45-6ca8-4207-9604-1c8890c9e9ea\" (UID: \"a2041f45-6ca8-4207-9604-1c8890c9e9ea\") "
Sep 09 03:34:11.277158 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:34:11.277134    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/766d00dc-b659-4682-b44b-4e42a6c4e844-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "766d00dc-b659-4682-b44b-4e42a6c4e844" (UID: "766d00dc-b659-4682-b44b-4e42a6c4e844"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 09 03:34:11.277326 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:34:11.277265    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a2041f45-6ca8-4207-9604-1c8890c9e9ea-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "a2041f45-6ca8-4207-9604-1c8890c9e9ea" (UID: "a2041f45-6ca8-4207-9604-1c8890c9e9ea"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 09 03:34:11.279396 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:34:11.279359    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/766d00dc-b659-4682-b44b-4e42a6c4e844-kube-api-access-b88q7" (OuterVolumeSpecName: "kube-api-access-b88q7") pod "766d00dc-b659-4682-b44b-4e42a6c4e844" (UID: "766d00dc-b659-4682-b44b-4e42a6c4e844"). InnerVolumeSpecName "kube-api-access-b88q7". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 09 03:34:11.279396 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:34:11.279387    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a2041f45-6ca8-4207-9604-1c8890c9e9ea-kube-api-access-2grgt" (OuterVolumeSpecName: "kube-api-access-2grgt") pod "a2041f45-6ca8-4207-9604-1c8890c9e9ea" (UID: "a2041f45-6ca8-4207-9604-1c8890c9e9ea"). InnerVolumeSpecName "kube-api-access-2grgt". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 09 03:34:11.378070 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:34:11.378035    2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2grgt\" (UniqueName: \"kubernetes.io/projected/a2041f45-6ca8-4207-9604-1c8890c9e9ea-kube-api-access-2grgt\") on node \"ip-10-0-132-14.ec2.internal\" DevicePath \"\""
Sep 09 03:34:11.378070 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:34:11.378061    2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-b88q7\" (UniqueName: \"kubernetes.io/projected/766d00dc-b659-4682-b44b-4e42a6c4e844-kube-api-access-b88q7\") on node \"ip-10-0-132-14.ec2.internal\" DevicePath \"\""
Sep 09 03:34:11.378070 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:34:11.378070    2576 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a2041f45-6ca8-4207-9604-1c8890c9e9ea-plugins-config-volume\") on node \"ip-10-0-132-14.ec2.internal\" DevicePath \"\""
Sep 09 03:34:11.378070 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:34:11.378078    2576 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/766d00dc-b659-4682-b44b-4e42a6c4e844-plugins-config-volume\") on node \"ip-10-0-132-14.ec2.internal\" DevicePath \"\""
Sep 09 03:34:11.409180 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:34:11.409146    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-4bd6b8-8zbc7"]
Sep 09 03:34:11.415294 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:34:11.415271    2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-4bd6b8-8zbc7"]
Sep 09 03:34:11.468105 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:34:11.468022    2576 patch_prober.go:28] interesting pod/e2e-mt-c44cc65a-openshift-default-66c6b44c5b-cnpfj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.50:15021/healthz/ready\": dial tcp 10.133.0.50:15021: connect: connection refused" start-of-body=
Sep 09 03:34:11.468105 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:34:11.468086    2576 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-c44cc65a-openshift-default-66c6b44c5b-cnpfj" podUID="31fc04d2-8016-4b58-ae28-6843df876bc9" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.50:15021/healthz/ready\": dial tcp 10.133.0.50:15021: connect: connection refused"
Sep 09 03:34:11.620987 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:34:11.620949    2576 generic.go:356] "Generic (PLEG): container finished" podID="a2041f45-6ca8-4207-9604-1c8890c9e9ea" containerID="2aad3eaa40285cefc7ec738749512a3382b4d829046c254d6716c8bb0f9078c3" exitCode=0
Sep 09 03:34:11.620987 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:34:11.620990    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-4bd6b8-6f98d797d-h8b2x"
Sep 09 03:34:11.621217 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:34:11.621040    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-4bd6b8-6f98d797d-h8b2x" event={"ID":"a2041f45-6ca8-4207-9604-1c8890c9e9ea","Type":"ContainerDied","Data":"2aad3eaa40285cefc7ec738749512a3382b4d829046c254d6716c8bb0f9078c3"}
Sep 09 03:34:11.621217 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:34:11.621092    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-4bd6b8-6f98d797d-h8b2x" event={"ID":"a2041f45-6ca8-4207-9604-1c8890c9e9ea","Type":"ContainerDied","Data":"d401d6afb958ba6627af05d15daf7ed48aab0a31d2deb50f1ccb10d89dd6b683"}
Sep 09 03:34:11.621217 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:34:11.621122    2576 scope.go:117] "RemoveContainer" containerID="2aad3eaa40285cefc7ec738749512a3382b4d829046c254d6716c8bb0f9078c3"
Sep 09 03:34:11.622467 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:34:11.622444    2576 generic.go:356] "Generic (PLEG): container finished" podID="766d00dc-b659-4682-b44b-4e42a6c4e844" containerID="0ec19459f242ce4cd865d15e9983f2e0a775f18244f99aaf76759ceb5972b29d" exitCode=0
Sep 09 03:34:11.622542 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:34:11.622486    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-b-4bd6b8-6458c89d79-jt8fk"
Sep 09 03:34:11.622542 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:34:11.622531    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-b-4bd6b8-6458c89d79-jt8fk" event={"ID":"766d00dc-b659-4682-b44b-4e42a6c4e844","Type":"ContainerDied","Data":"0ec19459f242ce4cd865d15e9983f2e0a775f18244f99aaf76759ceb5972b29d"}
Sep 09 03:34:11.622632 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:34:11.622571    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-b-4bd6b8-6458c89d79-jt8fk" event={"ID":"766d00dc-b659-4682-b44b-4e42a6c4e844","Type":"ContainerDied","Data":"eeb93b70d0206356698014d6a6b2384014065b304e3b92e05d86eff9a69ce1a3"}
Sep 09 03:34:11.633229 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:34:11.633205    2576 scope.go:117] "RemoveContainer" containerID="2aad3eaa40285cefc7ec738749512a3382b4d829046c254d6716c8bb0f9078c3"
Sep 09 03:34:11.633521 ip-10-0-132-14 kubenswrapper[2576]: E0909 03:34:11.633493    2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2aad3eaa40285cefc7ec738749512a3382b4d829046c254d6716c8bb0f9078c3\": container with ID starting with 2aad3eaa40285cefc7ec738749512a3382b4d829046c254d6716c8bb0f9078c3 not found: ID does not exist" containerID="2aad3eaa40285cefc7ec738749512a3382b4d829046c254d6716c8bb0f9078c3"
Sep 09 03:34:11.633615 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:34:11.633533    2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2aad3eaa40285cefc7ec738749512a3382b4d829046c254d6716c8bb0f9078c3"} err="failed to get container status \"2aad3eaa40285cefc7ec738749512a3382b4d829046c254d6716c8bb0f9078c3\": rpc error: code = NotFound desc = could not find container \"2aad3eaa40285cefc7ec738749512a3382b4d829046c254d6716c8bb0f9078c3\": container with ID starting with 2aad3eaa40285cefc7ec738749512a3382b4d829046c254d6716c8bb0f9078c3 not found: ID does not exist"
Sep 09 03:34:11.633615 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:34:11.633558    2576 scope.go:117] "RemoveContainer" containerID="0ec19459f242ce4cd865d15e9983f2e0a775f18244f99aaf76759ceb5972b29d"
Sep 09 03:34:11.655862 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:34:11.654618    2576 scope.go:117] "RemoveContainer" containerID="0ec19459f242ce4cd865d15e9983f2e0a775f18244f99aaf76759ceb5972b29d"
Sep 09 03:34:11.655862 ip-10-0-132-14 kubenswrapper[2576]: E0909 03:34:11.654982    2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0ec19459f242ce4cd865d15e9983f2e0a775f18244f99aaf76759ceb5972b29d\": container with ID starting with 0ec19459f242ce4cd865d15e9983f2e0a775f18244f99aaf76759ceb5972b29d not found: ID does not exist" containerID="0ec19459f242ce4cd865d15e9983f2e0a775f18244f99aaf76759ceb5972b29d"
Sep 09 03:34:11.655862 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:34:11.655011    2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0ec19459f242ce4cd865d15e9983f2e0a775f18244f99aaf76759ceb5972b29d"} err="failed to get container status \"0ec19459f242ce4cd865d15e9983f2e0a775f18244f99aaf76759ceb5972b29d\": rpc error: code = NotFound desc = could not find container \"0ec19459f242ce4cd865d15e9983f2e0a775f18244f99aaf76759ceb5972b29d\": container with ID starting with 0ec19459f242ce4cd865d15e9983f2e0a775f18244f99aaf76759ceb5972b29d not found: ID does not exist"
Sep 09 03:34:11.677197 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:34:11.677165    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-b-4bd6b8-6f98d797d-h8b2x"]
Sep 09 03:34:11.682684 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:34:11.682658    2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-b-4bd6b8-6f98d797d-h8b2x"]
Sep 09 03:34:11.703980 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:34:11.703958    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-b-4bd6b8-6458c89d79-jt8fk"]
Sep 09 03:34:11.707653 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:34:11.707636    2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-b-4bd6b8-6458c89d79-jt8fk"]
Sep 09 03:34:12.360351 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:34:12.360322    2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0f29a419-a0b2-44a1-9846-3e911136dc54" path="/var/lib/kubelet/pods/0f29a419-a0b2-44a1-9846-3e911136dc54/volumes"
Sep 09 03:34:12.360725 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:34:12.360677    2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="766d00dc-b659-4682-b44b-4e42a6c4e844" path="/var/lib/kubelet/pods/766d00dc-b659-4682-b44b-4e42a6c4e844/volumes"
Sep 09 03:34:12.361028 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:34:12.361015    2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a2041f45-6ca8-4207-9604-1c8890c9e9ea" path="/var/lib/kubelet/pods/a2041f45-6ca8-4207-9604-1c8890c9e9ea/volumes"
Sep 09 03:34:12.468075 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:34:12.468037    2576 patch_prober.go:28] interesting pod/e2e-mt-c44cc65a-openshift-default-66c6b44c5b-cnpfj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.50:15021/healthz/ready\": dial tcp 10.133.0.50:15021: connect: connection refused" start-of-body=
Sep 09 03:34:12.468252 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:34:12.468099    2576 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-c44cc65a-openshift-default-66c6b44c5b-cnpfj" podUID="31fc04d2-8016-4b58-ae28-6843df876bc9" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.50:15021/healthz/ready\": dial tcp 10.133.0.50:15021: connect: connection refused"
Sep 09 03:34:13.467651 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:34:13.467613    2576 patch_prober.go:28] interesting pod/e2e-mt-c44cc65a-openshift-default-66c6b44c5b-cnpfj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.50:15021/healthz/ready\": dial tcp 10.133.0.50:15021: connect: connection refused" start-of-body=
Sep 09 03:34:13.468063 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:34:13.467673    2576 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-c44cc65a-openshift-default-66c6b44c5b-cnpfj" podUID="31fc04d2-8016-4b58-ae28-6843df876bc9" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.50:15021/healthz/ready\": dial tcp 10.133.0.50:15021: connect: connection refused"
Sep 09 03:34:14.468195 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:34:14.468164    2576 patch_prober.go:28] interesting pod/e2e-mt-c44cc65a-openshift-default-66c6b44c5b-cnpfj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.50:15021/healthz/ready\": dial tcp 10.133.0.50:15021: connect: connection refused" start-of-body=
Sep 09 03:34:14.468572 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:34:14.468215    2576 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-c44cc65a-openshift-default-66c6b44c5b-cnpfj" podUID="31fc04d2-8016-4b58-ae28-6843df876bc9" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.50:15021/healthz/ready\": dial tcp 10.133.0.50:15021: connect: connection refused"
Sep 09 03:34:15.471185 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:34:15.471158    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-c44cc65a-openshift-default-66c6b44c5b-cnpfj"
Sep 09 03:34:15.471543 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:34:15.471496    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-c44cc65a-openshift-default-66c6b44c5b-cnpfj"
Sep 09 03:34:15.472117 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:34:15.472101    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-c44cc65a-openshift-default-66c6b44c5b-cnpfj"
Sep 09 03:34:25.111875 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:34:25.111832    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w3-5d8e85-fb6bf859c-bd4jk"]
Sep 09 03:34:25.112259 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:34:25.112177    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0f29a419-a0b2-44a1-9846-3e911136dc54" containerName="revoke-keys"
Sep 09 03:34:25.112259 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:34:25.112189    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="0f29a419-a0b2-44a1-9846-3e911136dc54" containerName="revoke-keys"
Sep 09 03:34:25.112259 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:34:25.112197    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a2041f45-6ca8-4207-9604-1c8890c9e9ea" containerName="payload-pre-processing"
Sep 09 03:34:25.112259 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:34:25.112202    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="a2041f45-6ca8-4207-9604-1c8890c9e9ea" containerName="payload-pre-processing"
Sep 09 03:34:25.112259 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:34:25.112210    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="766d00dc-b659-4682-b44b-4e42a6c4e844" containerName="payload-processing"
Sep 09 03:34:25.112259 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:34:25.112216    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="766d00dc-b659-4682-b44b-4e42a6c4e844" containerName="payload-processing"
Sep 09 03:34:25.112448 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:34:25.112283    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="0f29a419-a0b2-44a1-9846-3e911136dc54" containerName="revoke-keys"
Sep 09 03:34:25.112448 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:34:25.112290    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="a2041f45-6ca8-4207-9604-1c8890c9e9ea" containerName="payload-pre-processing"
Sep 09 03:34:25.112448 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:34:25.112300    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="766d00dc-b659-4682-b44b-4e42a6c4e844" containerName="payload-processing"
Sep 09 03:34:25.116562 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:34:25.116542    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-a-w3-5d8e85-fb6bf859c-bd4jk"
Sep 09 03:34:25.119694 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:34:25.119654    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-shared-a-w3-5d8e85\""
Sep 09 03:34:25.119781 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:34:25.119763    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-shared-a-w3-5d8e85-dockercfg-dhfzh\""
Sep 09 03:34:25.129176 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:34:25.129157    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w3-5d8e85-fb6bf859c-bd4jk"]
Sep 09 03:34:25.196682 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:34:25.196653    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w3-5d8e85-59687cb69d-29wl6"]
Sep 09 03:34:25.199901 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:34:25.199886    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-5d8e85-59687cb69d-29wl6"
Sep 09 03:34:25.222719 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:34:25.222688    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w3-5d8e85-59687cb69d-29wl6"]
Sep 09 03:34:25.303399 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:34:25.303358    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b7eee332-40a1-4b1f-85e9-5e7200d80373-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-a-w3-5d8e85-59687cb69d-29wl6\" (UID: \"b7eee332-40a1-4b1f-85e9-5e7200d80373\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-5d8e85-59687cb69d-29wl6"
Sep 09 03:34:25.303399 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:34:25.303403    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d261d48b-863a-4bbd-9631-6bf66e695aea-plugins-config-volume\") pod \"payload-processing-e2e-shared-a-w3-5d8e85-fb6bf859c-bd4jk\" (UID: \"d261d48b-863a-4bbd-9631-6bf66e695aea\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w3-5d8e85-fb6bf859c-bd4jk"
Sep 09 03:34:25.303609 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:34:25.303506    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-d9ttb\" (UniqueName: \"kubernetes.io/projected/d261d48b-863a-4bbd-9631-6bf66e695aea-kube-api-access-d9ttb\") pod \"payload-processing-e2e-shared-a-w3-5d8e85-fb6bf859c-bd4jk\" (UID: \"d261d48b-863a-4bbd-9631-6bf66e695aea\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w3-5d8e85-fb6bf859c-bd4jk"
Sep 09 03:34:25.303609 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:34:25.303549    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2hlx8\" (UniqueName: \"kubernetes.io/projected/b7eee332-40a1-4b1f-85e9-5e7200d80373-kube-api-access-2hlx8\") pod \"payload-pre-processing-e2e-shared-a-w3-5d8e85-59687cb69d-29wl6\" (UID: \"b7eee332-40a1-4b1f-85e9-5e7200d80373\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-5d8e85-59687cb69d-29wl6"
Sep 09 03:34:25.404443 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:34:25.404363    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-d9ttb\" (UniqueName: \"kubernetes.io/projected/d261d48b-863a-4bbd-9631-6bf66e695aea-kube-api-access-d9ttb\") pod \"payload-processing-e2e-shared-a-w3-5d8e85-fb6bf859c-bd4jk\" (UID: \"d261d48b-863a-4bbd-9631-6bf66e695aea\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w3-5d8e85-fb6bf859c-bd4jk"
Sep 09 03:34:25.404443 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:34:25.404408    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2hlx8\" (UniqueName: \"kubernetes.io/projected/b7eee332-40a1-4b1f-85e9-5e7200d80373-kube-api-access-2hlx8\") pod \"payload-pre-processing-e2e-shared-a-w3-5d8e85-59687cb69d-29wl6\" (UID: \"b7eee332-40a1-4b1f-85e9-5e7200d80373\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-5d8e85-59687cb69d-29wl6"
Sep 09 03:34:25.404647 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:34:25.404451    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b7eee332-40a1-4b1f-85e9-5e7200d80373-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-a-w3-5d8e85-59687cb69d-29wl6\" (UID: \"b7eee332-40a1-4b1f-85e9-5e7200d80373\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-5d8e85-59687cb69d-29wl6"
Sep 09 03:34:25.404647 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:34:25.404477    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d261d48b-863a-4bbd-9631-6bf66e695aea-plugins-config-volume\") pod \"payload-processing-e2e-shared-a-w3-5d8e85-fb6bf859c-bd4jk\" (UID: \"d261d48b-863a-4bbd-9631-6bf66e695aea\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w3-5d8e85-fb6bf859c-bd4jk"
Sep 09 03:34:25.405174 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:34:25.405155    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b7eee332-40a1-4b1f-85e9-5e7200d80373-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-a-w3-5d8e85-59687cb69d-29wl6\" (UID: \"b7eee332-40a1-4b1f-85e9-5e7200d80373\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-5d8e85-59687cb69d-29wl6"
Sep 09 03:34:25.405219 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:34:25.405199    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d261d48b-863a-4bbd-9631-6bf66e695aea-plugins-config-volume\") pod \"payload-processing-e2e-shared-a-w3-5d8e85-fb6bf859c-bd4jk\" (UID: \"d261d48b-863a-4bbd-9631-6bf66e695aea\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w3-5d8e85-fb6bf859c-bd4jk"
Sep 09 03:34:25.413952 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:34:25.413924    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2hlx8\" (UniqueName: \"kubernetes.io/projected/b7eee332-40a1-4b1f-85e9-5e7200d80373-kube-api-access-2hlx8\") pod \"payload-pre-processing-e2e-shared-a-w3-5d8e85-59687cb69d-29wl6\" (UID: \"b7eee332-40a1-4b1f-85e9-5e7200d80373\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-5d8e85-59687cb69d-29wl6"
Sep 09 03:34:25.414335 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:34:25.414317    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-d9ttb\" (UniqueName: \"kubernetes.io/projected/d261d48b-863a-4bbd-9631-6bf66e695aea-kube-api-access-d9ttb\") pod \"payload-processing-e2e-shared-a-w3-5d8e85-fb6bf859c-bd4jk\" (UID: \"d261d48b-863a-4bbd-9631-6bf66e695aea\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w3-5d8e85-fb6bf859c-bd4jk"
Sep 09 03:34:25.426203 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:34:25.426183    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-a-w3-5d8e85-fb6bf859c-bd4jk"
Sep 09 03:34:25.510145 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:34:25.510089    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-5d8e85-59687cb69d-29wl6"
Sep 09 03:34:25.701322 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:34:25.701299    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w3-5d8e85-fb6bf859c-bd4jk"]
Sep 09 03:34:25.702507 ip-10-0-132-14 kubenswrapper[2576]: W0909 03:34:25.702483    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd261d48b_863a_4bbd_9631_6bf66e695aea.slice/crio-e6e03f21d6ed2cf7a49352356b13303c6ff7ac3bc95a46c4cebaf434be41db21 WatchSource:0}: Error finding container e6e03f21d6ed2cf7a49352356b13303c6ff7ac3bc95a46c4cebaf434be41db21: Status 404 returned error can't find the container with id e6e03f21d6ed2cf7a49352356b13303c6ff7ac3bc95a46c4cebaf434be41db21
Sep 09 03:34:25.792006 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:34:25.791980    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w3-5d8e85-59687cb69d-29wl6"]
Sep 09 03:34:25.793541 ip-10-0-132-14 kubenswrapper[2576]: W0909 03:34:25.793515    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb7eee332_40a1_4b1f_85e9_5e7200d80373.slice/crio-ce86f93b885de2451872c02f7e3d59e037c7c6a2591895a4f671a57519c94d10 WatchSource:0}: Error finding container ce86f93b885de2451872c02f7e3d59e037c7c6a2591895a4f671a57519c94d10: Status 404 returned error can't find the container with id ce86f93b885de2451872c02f7e3d59e037c7c6a2591895a4f671a57519c94d10
Sep 09 03:34:26.704420 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:34:26.704378    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-a-w3-5d8e85-fb6bf859c-bd4jk" event={"ID":"d261d48b-863a-4bbd-9631-6bf66e695aea","Type":"ContainerStarted","Data":"2c9fca7cc1d9b2909a1b0b94d4b6e74071b62e540051ee559fe07b60d81e4474"}
Sep 09 03:34:26.704885 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:34:26.704549    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-shared-a-w3-5d8e85-fb6bf859c-bd4jk"
Sep 09 03:34:26.704885 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:34:26.704581    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-a-w3-5d8e85-fb6bf859c-bd4jk" event={"ID":"d261d48b-863a-4bbd-9631-6bf66e695aea","Type":"ContainerStarted","Data":"e6e03f21d6ed2cf7a49352356b13303c6ff7ac3bc95a46c4cebaf434be41db21"}
Sep 09 03:34:26.705925 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:34:26.705901    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-5d8e85-59687cb69d-29wl6" event={"ID":"b7eee332-40a1-4b1f-85e9-5e7200d80373","Type":"ContainerStarted","Data":"b0801bae3ba8b8e221fb2b993bdfa5533d52bde66e3207899c5637321d624c06"}
Sep 09 03:34:26.705997 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:34:26.705950    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-5d8e85-59687cb69d-29wl6" event={"ID":"b7eee332-40a1-4b1f-85e9-5e7200d80373","Type":"ContainerStarted","Data":"ce86f93b885de2451872c02f7e3d59e037c7c6a2591895a4f671a57519c94d10"}
Sep 09 03:34:26.705997 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:34:26.705974    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-5d8e85-59687cb69d-29wl6"
Sep 09 03:34:26.728545 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:34:26.728495    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-shared-a-w3-5d8e85-fb6bf859c-bd4jk" podStartSLOduration=1.7284803819999999 podStartE2EDuration="1.728480382s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 03:34:25 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 03:34:26.72803035 +0000 UTC m=+1280.869516429" watchObservedRunningTime="2026-09-09 03:34:26.728480382 +0000 UTC m=+1280.869966459"
Sep 09 03:34:26.750276 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:34:26.750238    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-5d8e85-59687cb69d-29wl6" podStartSLOduration=1.750200098 podStartE2EDuration="1.750200098s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 03:34:25 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 03:34:26.749035713 +0000 UTC m=+1280.890521787" watchObservedRunningTime="2026-09-09 03:34:26.750200098 +0000 UTC m=+1280.891686174"
Sep 09 03:34:27.058503 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:34:27.058470    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-c44cc65a-rst5k"]
Sep 09 03:34:27.061771 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:34:27.061756    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-c44cc65a-rst5k"
Sep 09 03:34:27.064545 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:34:27.064521    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-w7q9f\""
Sep 09 03:34:27.074032 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:34:27.074010    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-c44cc65a-rst5k"]
Sep 09 03:34:27.220452 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:34:27.220414    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/01b9dff0-9854-4ecf-8733-81cc00955fc9-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-c44cc65a-rst5k\" (UID: \"01b9dff0-9854-4ecf-8733-81cc00955fc9\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-c44cc65a-rst5k"
Sep 09 03:34:27.320978 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:34:27.320888    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/01b9dff0-9854-4ecf-8733-81cc00955fc9-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-c44cc65a-rst5k\" (UID: \"01b9dff0-9854-4ecf-8733-81cc00955fc9\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-c44cc65a-rst5k"
Sep 09 03:34:27.321556 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:34:27.321527    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/01b9dff0-9854-4ecf-8733-81cc00955fc9-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-c44cc65a-rst5k\" (UID: \"01b9dff0-9854-4ecf-8733-81cc00955fc9\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-c44cc65a-rst5k"
Sep 09 03:34:27.372436 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:34:27.372409    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-c44cc65a-rst5k"
Sep 09 03:34:27.675491 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:34:27.675451    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-c44cc65a-rst5k"]
Sep 09 03:34:27.677577 ip-10-0-132-14 kubenswrapper[2576]: W0909 03:34:27.677550    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod01b9dff0_9854_4ecf_8733_81cc00955fc9.slice/crio-e9178ddaeb4ea1449a1f5be0fb5b771c83d6aae4ad939bf99e4242fad690bd49 WatchSource:0}: Error finding container e9178ddaeb4ea1449a1f5be0fb5b771c83d6aae4ad939bf99e4242fad690bd49: Status 404 returned error can't find the container with id e9178ddaeb4ea1449a1f5be0fb5b771c83d6aae4ad939bf99e4242fad690bd49
Sep 09 03:34:27.711514 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:34:27.711489    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-c44cc65a-rst5k" event={"ID":"01b9dff0-9854-4ecf-8733-81cc00955fc9","Type":"ContainerStarted","Data":"e9178ddaeb4ea1449a1f5be0fb5b771c83d6aae4ad939bf99e4242fad690bd49"}
Sep 09 03:34:28.717635 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:34:28.717548    2576 generic.go:356] "Generic (PLEG): container finished" podID="01b9dff0-9854-4ecf-8733-81cc00955fc9" containerID="6a7be77a144e103a7273ea3ccebec4fc1f28e0b8c21ce2383a2c94d010baa67e" exitCode=0
Sep 09 03:34:28.718018 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:34:28.717633    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-c44cc65a-rst5k" event={"ID":"01b9dff0-9854-4ecf-8733-81cc00955fc9","Type":"ContainerDied","Data":"6a7be77a144e103a7273ea3ccebec4fc1f28e0b8c21ce2383a2c94d010baa67e"}
Sep 09 03:34:29.847347 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:34:29.847321    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-c44cc65a-rst5k"
Sep 09 03:34:29.941482 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:34:29.941455    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/01b9dff0-9854-4ecf-8733-81cc00955fc9-maas-api-service-ca\") pod \"01b9dff0-9854-4ecf-8733-81cc00955fc9\" (UID: \"01b9dff0-9854-4ecf-8733-81cc00955fc9\") "
Sep 09 03:34:29.941794 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:34:29.941774    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/01b9dff0-9854-4ecf-8733-81cc00955fc9-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "01b9dff0-9854-4ecf-8733-81cc00955fc9" (UID: "01b9dff0-9854-4ecf-8733-81cc00955fc9"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 09 03:34:30.042207 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:34:30.042181    2576 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/01b9dff0-9854-4ecf-8733-81cc00955fc9-maas-api-service-ca\") on node \"ip-10-0-132-14.ec2.internal\" DevicePath \"\""
Sep 09 03:34:30.728392 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:34:30.728320    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-c44cc65a-rst5k"
Sep 09 03:34:30.728392 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:34:30.728364    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-c44cc65a-rst5k" event={"ID":"01b9dff0-9854-4ecf-8733-81cc00955fc9","Type":"ContainerDied","Data":"e9178ddaeb4ea1449a1f5be0fb5b771c83d6aae4ad939bf99e4242fad690bd49"}
Sep 09 03:34:30.728562 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:34:30.728409    2576 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="e9178ddaeb4ea1449a1f5be0fb5b771c83d6aae4ad939bf99e4242fad690bd49"
Sep 09 03:34:37.713916 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:34:37.713884    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-5d8e85-59687cb69d-29wl6"
Sep 09 03:34:37.714368 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:34:37.713963    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-shared-a-w3-5d8e85-fb6bf859c-bd4jk"
Sep 09 03:34:37.766508 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:34:37.766475    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-c44cc65a-559789876f-24b8x"]
Sep 09 03:34:37.766760 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:34:37.766713    2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-mt-c44cc65a-559789876f-24b8x" podUID="7c7e63b7-e5da-4ca9-aa41-46ea78a3e2cb" containerName="payload-processing" containerID="cri-o://fe5e21ee3adaa188e5dec15c00340f9d2ee14ba20fcf4e95d028574ec587ddc1" gracePeriod=30
Sep 09 03:34:37.780709 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:34:37.780682    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-c44cc65a-84bfb86d66-4bp7d"]
Sep 09 03:34:37.780982 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:34:37.780956    2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-mt-c44cc65a-84bfb86d66-4bp7d" podUID="1e24840e-811b-4d59-8402-1586c3153e6d" containerName="payload-pre-processing" containerID="cri-o://402ecbd96a66dfa6c5baf5ac2fbba7b3f45c96317e20cc667ae15169c70fe1e7" gracePeriod=30
Sep 09 03:34:37.971624 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:34:37.971548    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-c44cc65a-rst5k"]
Sep 09 03:34:37.977114 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:34:37.976661    2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-c44cc65a-rst5k"]
Sep 09 03:34:38.047196 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:34:38.047174    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-c44cc65a-559789876f-24b8x"
Sep 09 03:34:38.049728 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:34:38.049707    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-c44cc65a-84bfb86d66-4bp7d"
Sep 09 03:34:38.106394 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:34:38.106364    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/7c7e63b7-e5da-4ca9-aa41-46ea78a3e2cb-plugins-config-volume\") pod \"7c7e63b7-e5da-4ca9-aa41-46ea78a3e2cb\" (UID: \"7c7e63b7-e5da-4ca9-aa41-46ea78a3e2cb\") "
Sep 09 03:34:38.106575 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:34:38.106408    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-mvvld\" (UniqueName: \"kubernetes.io/projected/7c7e63b7-e5da-4ca9-aa41-46ea78a3e2cb-kube-api-access-mvvld\") pod \"7c7e63b7-e5da-4ca9-aa41-46ea78a3e2cb\" (UID: \"7c7e63b7-e5da-4ca9-aa41-46ea78a3e2cb\") "
Sep 09 03:34:38.106575 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:34:38.106453    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-n7pnq\" (UniqueName: \"kubernetes.io/projected/1e24840e-811b-4d59-8402-1586c3153e6d-kube-api-access-n7pnq\") pod \"1e24840e-811b-4d59-8402-1586c3153e6d\" (UID: \"1e24840e-811b-4d59-8402-1586c3153e6d\") "
Sep 09 03:34:38.106575 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:34:38.106530    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/1e24840e-811b-4d59-8402-1586c3153e6d-plugins-config-volume\") pod \"1e24840e-811b-4d59-8402-1586c3153e6d\" (UID: \"1e24840e-811b-4d59-8402-1586c3153e6d\") "
Sep 09 03:34:38.106989 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:34:38.106956    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1e24840e-811b-4d59-8402-1586c3153e6d-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "1e24840e-811b-4d59-8402-1586c3153e6d" (UID: "1e24840e-811b-4d59-8402-1586c3153e6d"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 09 03:34:38.107826 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:34:38.107447    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7c7e63b7-e5da-4ca9-aa41-46ea78a3e2cb-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "7c7e63b7-e5da-4ca9-aa41-46ea78a3e2cb" (UID: "7c7e63b7-e5da-4ca9-aa41-46ea78a3e2cb"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 09 03:34:38.109458 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:34:38.109435    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1e24840e-811b-4d59-8402-1586c3153e6d-kube-api-access-n7pnq" (OuterVolumeSpecName: "kube-api-access-n7pnq") pod "1e24840e-811b-4d59-8402-1586c3153e6d" (UID: "1e24840e-811b-4d59-8402-1586c3153e6d"). InnerVolumeSpecName "kube-api-access-n7pnq". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 09 03:34:38.109458 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:34:38.109443    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7c7e63b7-e5da-4ca9-aa41-46ea78a3e2cb-kube-api-access-mvvld" (OuterVolumeSpecName: "kube-api-access-mvvld") pod "7c7e63b7-e5da-4ca9-aa41-46ea78a3e2cb" (UID: "7c7e63b7-e5da-4ca9-aa41-46ea78a3e2cb"). InnerVolumeSpecName "kube-api-access-mvvld". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 09 03:34:38.207433 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:34:38.207405    2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mvvld\" (UniqueName: \"kubernetes.io/projected/7c7e63b7-e5da-4ca9-aa41-46ea78a3e2cb-kube-api-access-mvvld\") on node \"ip-10-0-132-14.ec2.internal\" DevicePath \"\""
Sep 09 03:34:38.207433 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:34:38.207429    2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-n7pnq\" (UniqueName: \"kubernetes.io/projected/1e24840e-811b-4d59-8402-1586c3153e6d-kube-api-access-n7pnq\") on node \"ip-10-0-132-14.ec2.internal\" DevicePath \"\""
Sep 09 03:34:38.207433 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:34:38.207438    2576 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/1e24840e-811b-4d59-8402-1586c3153e6d-plugins-config-volume\") on node \"ip-10-0-132-14.ec2.internal\" DevicePath \"\""
Sep 09 03:34:38.207636 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:34:38.207447    2576 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/7c7e63b7-e5da-4ca9-aa41-46ea78a3e2cb-plugins-config-volume\") on node \"ip-10-0-132-14.ec2.internal\" DevicePath \"\""
Sep 09 03:34:38.360190 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:34:38.360157    2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="01b9dff0-9854-4ecf-8733-81cc00955fc9" path="/var/lib/kubelet/pods/01b9dff0-9854-4ecf-8733-81cc00955fc9/volumes"
Sep 09 03:34:38.767504 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:34:38.767472    2576 generic.go:356] "Generic (PLEG): container finished" podID="1e24840e-811b-4d59-8402-1586c3153e6d" containerID="402ecbd96a66dfa6c5baf5ac2fbba7b3f45c96317e20cc667ae15169c70fe1e7" exitCode=0
Sep 09 03:34:38.768010 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:34:38.767510    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-c44cc65a-84bfb86d66-4bp7d"
Sep 09 03:34:38.768010 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:34:38.767564    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-c44cc65a-84bfb86d66-4bp7d" event={"ID":"1e24840e-811b-4d59-8402-1586c3153e6d","Type":"ContainerDied","Data":"402ecbd96a66dfa6c5baf5ac2fbba7b3f45c96317e20cc667ae15169c70fe1e7"}
Sep 09 03:34:38.768010 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:34:38.767614    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-c44cc65a-84bfb86d66-4bp7d" event={"ID":"1e24840e-811b-4d59-8402-1586c3153e6d","Type":"ContainerDied","Data":"224aa78ee0dbe81567632477eb4dd030ddee7d94cd4c1abe51d9103a9e696532"}
Sep 09 03:34:38.768010 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:34:38.767644    2576 scope.go:117] "RemoveContainer" containerID="402ecbd96a66dfa6c5baf5ac2fbba7b3f45c96317e20cc667ae15169c70fe1e7"
Sep 09 03:34:38.769037 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:34:38.768932    2576 generic.go:356] "Generic (PLEG): container finished" podID="7c7e63b7-e5da-4ca9-aa41-46ea78a3e2cb" containerID="fe5e21ee3adaa188e5dec15c00340f9d2ee14ba20fcf4e95d028574ec587ddc1" exitCode=0
Sep 09 03:34:38.769037 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:34:38.768987    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-c44cc65a-559789876f-24b8x"
Sep 09 03:34:38.769037 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:34:38.769001    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-c44cc65a-559789876f-24b8x" event={"ID":"7c7e63b7-e5da-4ca9-aa41-46ea78a3e2cb","Type":"ContainerDied","Data":"fe5e21ee3adaa188e5dec15c00340f9d2ee14ba20fcf4e95d028574ec587ddc1"}
Sep 09 03:34:38.769153 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:34:38.769039    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-c44cc65a-559789876f-24b8x" event={"ID":"7c7e63b7-e5da-4ca9-aa41-46ea78a3e2cb","Type":"ContainerDied","Data":"43ae23f885bbf3549bf5f2563b62c7414d076c621f38afc653f3719a3e8ad46c"}
Sep 09 03:34:38.776518 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:34:38.776502    2576 scope.go:117] "RemoveContainer" containerID="402ecbd96a66dfa6c5baf5ac2fbba7b3f45c96317e20cc667ae15169c70fe1e7"
Sep 09 03:34:38.776835 ip-10-0-132-14 kubenswrapper[2576]: E0909 03:34:38.776793    2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"402ecbd96a66dfa6c5baf5ac2fbba7b3f45c96317e20cc667ae15169c70fe1e7\": container with ID starting with 402ecbd96a66dfa6c5baf5ac2fbba7b3f45c96317e20cc667ae15169c70fe1e7 not found: ID does not exist" containerID="402ecbd96a66dfa6c5baf5ac2fbba7b3f45c96317e20cc667ae15169c70fe1e7"
Sep 09 03:34:38.776906 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:34:38.776836    2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"402ecbd96a66dfa6c5baf5ac2fbba7b3f45c96317e20cc667ae15169c70fe1e7"} err="failed to get container status \"402ecbd96a66dfa6c5baf5ac2fbba7b3f45c96317e20cc667ae15169c70fe1e7\": rpc error: code = NotFound desc = could not find container \"402ecbd96a66dfa6c5baf5ac2fbba7b3f45c96317e20cc667ae15169c70fe1e7\": container with ID starting with 402ecbd96a66dfa6c5baf5ac2fbba7b3f45c96317e20cc667ae15169c70fe1e7 not found: ID does not exist"
Sep 09 03:34:38.776906 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:34:38.776853    2576 scope.go:117] "RemoveContainer" containerID="fe5e21ee3adaa188e5dec15c00340f9d2ee14ba20fcf4e95d028574ec587ddc1"
Sep 09 03:34:38.787160 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:34:38.787144    2576 scope.go:117] "RemoveContainer" containerID="fe5e21ee3adaa188e5dec15c00340f9d2ee14ba20fcf4e95d028574ec587ddc1"
Sep 09 03:34:38.787421 ip-10-0-132-14 kubenswrapper[2576]: E0909 03:34:38.787405    2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"fe5e21ee3adaa188e5dec15c00340f9d2ee14ba20fcf4e95d028574ec587ddc1\": container with ID starting with fe5e21ee3adaa188e5dec15c00340f9d2ee14ba20fcf4e95d028574ec587ddc1 not found: ID does not exist" containerID="fe5e21ee3adaa188e5dec15c00340f9d2ee14ba20fcf4e95d028574ec587ddc1"
Sep 09 03:34:38.787478 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:34:38.787425    2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"fe5e21ee3adaa188e5dec15c00340f9d2ee14ba20fcf4e95d028574ec587ddc1"} err="failed to get container status \"fe5e21ee3adaa188e5dec15c00340f9d2ee14ba20fcf4e95d028574ec587ddc1\": rpc error: code = NotFound desc = could not find container \"fe5e21ee3adaa188e5dec15c00340f9d2ee14ba20fcf4e95d028574ec587ddc1\": container with ID starting with fe5e21ee3adaa188e5dec15c00340f9d2ee14ba20fcf4e95d028574ec587ddc1 not found: ID does not exist"
Sep 09 03:34:38.790744 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:34:38.790722    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-c44cc65a-84bfb86d66-4bp7d"]
Sep 09 03:34:38.794182 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:34:38.794161    2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-c44cc65a-84bfb86d66-4bp7d"]
Sep 09 03:34:38.814785 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:34:38.814763    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-c44cc65a-559789876f-24b8x"]
Sep 09 03:34:38.825539 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:34:38.825521    2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-c44cc65a-559789876f-24b8x"]
Sep 09 03:34:40.364730 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:34:40.361505    2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="1e24840e-811b-4d59-8402-1586c3153e6d" path="/var/lib/kubelet/pods/1e24840e-811b-4d59-8402-1586c3153e6d/volumes"
Sep 09 03:34:40.364730 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:34:40.362054    2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7c7e63b7-e5da-4ca9-aa41-46ea78a3e2cb" path="/var/lib/kubelet/pods/7c7e63b7-e5da-4ca9-aa41-46ea78a3e2cb/volumes"
Sep 09 03:34:44.827899 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:34:44.827867    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-c44cc65a-openshift-default-66c6b44c5b-cnpfj"]
Sep 09 03:34:44.828344 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:34:44.828121    2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-mt-c44cc65a-openshift-default-66c6b44c5b-cnpfj" podUID="31fc04d2-8016-4b58-ae28-6843df876bc9" containerName="istio-proxy" containerID="cri-o://6b9af6686aa3d4fa2433c7a268ba319552ad78001e47e6f3dd1581dd8ae03fa1" gracePeriod=30
Sep 09 03:34:45.474171 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:34:45.474131    2576 patch_prober.go:28] interesting pod/e2e-mt-c44cc65a-openshift-default-66c6b44c5b-cnpfj container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Sep 09 03:34:45.474344 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:34:45.474201    2576 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-mt-c44cc65a-openshift-default-66c6b44c5b-cnpfj" podUID="31fc04d2-8016-4b58-ae28-6843df876bc9" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Sep 09 03:34:50.075311 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:34:50.075286    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-c44cc65a-openshift-default-66c6b44c5b-cnpfj"
Sep 09 03:34:50.203465 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:34:50.203386    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/31fc04d2-8016-4b58-ae28-6843df876bc9-istio-data\") pod \"31fc04d2-8016-4b58-ae28-6843df876bc9\" (UID: \"31fc04d2-8016-4b58-ae28-6843df876bc9\") "
Sep 09 03:34:50.203465 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:34:50.203440    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/31fc04d2-8016-4b58-ae28-6843df876bc9-istiod-ca-cert\") pod \"31fc04d2-8016-4b58-ae28-6843df876bc9\" (UID: \"31fc04d2-8016-4b58-ae28-6843df876bc9\") "
Sep 09 03:34:50.203465 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:34:50.203464    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/31fc04d2-8016-4b58-ae28-6843df876bc9-istio-token\") pod \"31fc04d2-8016-4b58-ae28-6843df876bc9\" (UID: \"31fc04d2-8016-4b58-ae28-6843df876bc9\") "
Sep 09 03:34:50.203716 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:34:50.203506    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/31fc04d2-8016-4b58-ae28-6843df876bc9-workload-socket\") pod \"31fc04d2-8016-4b58-ae28-6843df876bc9\" (UID: \"31fc04d2-8016-4b58-ae28-6843df876bc9\") "
Sep 09 03:34:50.203716 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:34:50.203521    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-xv9g2\" (UniqueName: \"kubernetes.io/projected/31fc04d2-8016-4b58-ae28-6843df876bc9-kube-api-access-xv9g2\") pod \"31fc04d2-8016-4b58-ae28-6843df876bc9\" (UID: \"31fc04d2-8016-4b58-ae28-6843df876bc9\") "
Sep 09 03:34:50.203716 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:34:50.203644    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/31fc04d2-8016-4b58-ae28-6843df876bc9-istio-data" (OuterVolumeSpecName: "istio-data") pod "31fc04d2-8016-4b58-ae28-6843df876bc9" (UID: "31fc04d2-8016-4b58-ae28-6843df876bc9"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 03:34:50.203716 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:34:50.203663    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/31fc04d2-8016-4b58-ae28-6843df876bc9-istio-envoy\") pod \"31fc04d2-8016-4b58-ae28-6843df876bc9\" (UID: \"31fc04d2-8016-4b58-ae28-6843df876bc9\") "
Sep 09 03:34:50.203716 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:34:50.203696    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/31fc04d2-8016-4b58-ae28-6843df876bc9-credential-socket\") pod \"31fc04d2-8016-4b58-ae28-6843df876bc9\" (UID: \"31fc04d2-8016-4b58-ae28-6843df876bc9\") "
Sep 09 03:34:50.204021 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:34:50.203724    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/31fc04d2-8016-4b58-ae28-6843df876bc9-workload-certs\") pod \"31fc04d2-8016-4b58-ae28-6843df876bc9\" (UID: \"31fc04d2-8016-4b58-ae28-6843df876bc9\") "
Sep 09 03:34:50.204021 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:34:50.203767    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/31fc04d2-8016-4b58-ae28-6843df876bc9-istio-podinfo\") pod \"31fc04d2-8016-4b58-ae28-6843df876bc9\" (UID: \"31fc04d2-8016-4b58-ae28-6843df876bc9\") "
Sep 09 03:34:50.204021 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:34:50.203849    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/31fc04d2-8016-4b58-ae28-6843df876bc9-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "31fc04d2-8016-4b58-ae28-6843df876bc9" (UID: "31fc04d2-8016-4b58-ae28-6843df876bc9"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 03:34:50.204415 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:34:50.204019    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/31fc04d2-8016-4b58-ae28-6843df876bc9-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "31fc04d2-8016-4b58-ae28-6843df876bc9" (UID: "31fc04d2-8016-4b58-ae28-6843df876bc9"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 09 03:34:50.204415 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:34:50.204087    2576 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/31fc04d2-8016-4b58-ae28-6843df876bc9-workload-socket\") on node \"ip-10-0-132-14.ec2.internal\" DevicePath \"\""
Sep 09 03:34:50.204415 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:34:50.204106    2576 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/31fc04d2-8016-4b58-ae28-6843df876bc9-istio-data\") on node \"ip-10-0-132-14.ec2.internal\" DevicePath \"\""
Sep 09 03:34:50.204415 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:34:50.204122    2576 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/31fc04d2-8016-4b58-ae28-6843df876bc9-istiod-ca-cert\") on node \"ip-10-0-132-14.ec2.internal\" DevicePath \"\""
Sep 09 03:34:50.204415 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:34:50.204173    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/31fc04d2-8016-4b58-ae28-6843df876bc9-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "31fc04d2-8016-4b58-ae28-6843df876bc9" (UID: "31fc04d2-8016-4b58-ae28-6843df876bc9"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 03:34:50.204415 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:34:50.204401    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/31fc04d2-8016-4b58-ae28-6843df876bc9-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "31fc04d2-8016-4b58-ae28-6843df876bc9" (UID: "31fc04d2-8016-4b58-ae28-6843df876bc9"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 03:34:50.205887 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:34:50.205856    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/31fc04d2-8016-4b58-ae28-6843df876bc9-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "31fc04d2-8016-4b58-ae28-6843df876bc9" (UID: "31fc04d2-8016-4b58-ae28-6843df876bc9"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 09 03:34:50.206015 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:34:50.205925    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/31fc04d2-8016-4b58-ae28-6843df876bc9-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "31fc04d2-8016-4b58-ae28-6843df876bc9" (UID: "31fc04d2-8016-4b58-ae28-6843df876bc9"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 03:34:50.206015 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:34:50.205961    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/31fc04d2-8016-4b58-ae28-6843df876bc9-kube-api-access-xv9g2" (OuterVolumeSpecName: "kube-api-access-xv9g2") pod "31fc04d2-8016-4b58-ae28-6843df876bc9" (UID: "31fc04d2-8016-4b58-ae28-6843df876bc9"). InnerVolumeSpecName "kube-api-access-xv9g2". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 09 03:34:50.206015 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:34:50.206000    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/31fc04d2-8016-4b58-ae28-6843df876bc9-istio-token" (OuterVolumeSpecName: "istio-token") pod "31fc04d2-8016-4b58-ae28-6843df876bc9" (UID: "31fc04d2-8016-4b58-ae28-6843df876bc9"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 09 03:34:50.305300 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:34:50.305271    2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xv9g2\" (UniqueName: \"kubernetes.io/projected/31fc04d2-8016-4b58-ae28-6843df876bc9-kube-api-access-xv9g2\") on node \"ip-10-0-132-14.ec2.internal\" DevicePath \"\""
Sep 09 03:34:50.305300 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:34:50.305292    2576 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/31fc04d2-8016-4b58-ae28-6843df876bc9-istio-envoy\") on node \"ip-10-0-132-14.ec2.internal\" DevicePath \"\""
Sep 09 03:34:50.305300 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:34:50.305301    2576 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/31fc04d2-8016-4b58-ae28-6843df876bc9-credential-socket\") on node \"ip-10-0-132-14.ec2.internal\" DevicePath \"\""
Sep 09 03:34:50.305476 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:34:50.305309    2576 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/31fc04d2-8016-4b58-ae28-6843df876bc9-workload-certs\") on node \"ip-10-0-132-14.ec2.internal\" DevicePath \"\""
Sep 09 03:34:50.305476 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:34:50.305319    2576 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/31fc04d2-8016-4b58-ae28-6843df876bc9-istio-podinfo\") on node \"ip-10-0-132-14.ec2.internal\" DevicePath \"\""
Sep 09 03:34:50.305476 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:34:50.305326    2576 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/31fc04d2-8016-4b58-ae28-6843df876bc9-istio-token\") on node \"ip-10-0-132-14.ec2.internal\" DevicePath \"\""
Sep 09 03:34:50.829940 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:34:50.829902    2576 generic.go:356] "Generic (PLEG): container finished" podID="31fc04d2-8016-4b58-ae28-6843df876bc9" containerID="6b9af6686aa3d4fa2433c7a268ba319552ad78001e47e6f3dd1581dd8ae03fa1" exitCode=0
Sep 09 03:34:50.830112 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:34:50.829955    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-c44cc65a-openshift-default-66c6b44c5b-cnpfj"
Sep 09 03:34:50.830112 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:34:50.829969    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-c44cc65a-openshift-default-66c6b44c5b-cnpfj" event={"ID":"31fc04d2-8016-4b58-ae28-6843df876bc9","Type":"ContainerDied","Data":"6b9af6686aa3d4fa2433c7a268ba319552ad78001e47e6f3dd1581dd8ae03fa1"}
Sep 09 03:34:50.830112 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:34:50.830022    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-c44cc65a-openshift-default-66c6b44c5b-cnpfj" event={"ID":"31fc04d2-8016-4b58-ae28-6843df876bc9","Type":"ContainerDied","Data":"276fef87a4f7015a0dddf03b9f7950f91fc5d2c964f42fa260921ef39e93635d"}
Sep 09 03:34:50.830112 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:34:50.830053    2576 scope.go:117] "RemoveContainer" containerID="6b9af6686aa3d4fa2433c7a268ba319552ad78001e47e6f3dd1581dd8ae03fa1"
Sep 09 03:34:50.838603 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:34:50.838586    2576 scope.go:117] "RemoveContainer" containerID="8ad0663bdd18bb0f385601a5408910693aaa1787af0a11b8ad8cccf2029f34a7"
Sep 09 03:34:50.846167 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:34:50.846149    2576 scope.go:117] "RemoveContainer" containerID="6b9af6686aa3d4fa2433c7a268ba319552ad78001e47e6f3dd1581dd8ae03fa1"
Sep 09 03:34:50.846442 ip-10-0-132-14 kubenswrapper[2576]: E0909 03:34:50.846423    2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6b9af6686aa3d4fa2433c7a268ba319552ad78001e47e6f3dd1581dd8ae03fa1\": container with ID starting with 6b9af6686aa3d4fa2433c7a268ba319552ad78001e47e6f3dd1581dd8ae03fa1 not found: ID does not exist" containerID="6b9af6686aa3d4fa2433c7a268ba319552ad78001e47e6f3dd1581dd8ae03fa1"
Sep 09 03:34:50.846488 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:34:50.846449    2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6b9af6686aa3d4fa2433c7a268ba319552ad78001e47e6f3dd1581dd8ae03fa1"} err="failed to get container status \"6b9af6686aa3d4fa2433c7a268ba319552ad78001e47e6f3dd1581dd8ae03fa1\": rpc error: code = NotFound desc = could not find container \"6b9af6686aa3d4fa2433c7a268ba319552ad78001e47e6f3dd1581dd8ae03fa1\": container with ID starting with 6b9af6686aa3d4fa2433c7a268ba319552ad78001e47e6f3dd1581dd8ae03fa1 not found: ID does not exist"
Sep 09 03:34:50.846488 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:34:50.846466    2576 scope.go:117] "RemoveContainer" containerID="8ad0663bdd18bb0f385601a5408910693aaa1787af0a11b8ad8cccf2029f34a7"
Sep 09 03:34:50.846717 ip-10-0-132-14 kubenswrapper[2576]: E0909 03:34:50.846701    2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8ad0663bdd18bb0f385601a5408910693aaa1787af0a11b8ad8cccf2029f34a7\": container with ID starting with 8ad0663bdd18bb0f385601a5408910693aaa1787af0a11b8ad8cccf2029f34a7 not found: ID does not exist" containerID="8ad0663bdd18bb0f385601a5408910693aaa1787af0a11b8ad8cccf2029f34a7"
Sep 09 03:34:50.846760 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:34:50.846722    2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8ad0663bdd18bb0f385601a5408910693aaa1787af0a11b8ad8cccf2029f34a7"} err="failed to get container status \"8ad0663bdd18bb0f385601a5408910693aaa1787af0a11b8ad8cccf2029f34a7\": rpc error: code = NotFound desc = could not find container \"8ad0663bdd18bb0f385601a5408910693aaa1787af0a11b8ad8cccf2029f34a7\": container with ID starting with 8ad0663bdd18bb0f385601a5408910693aaa1787af0a11b8ad8cccf2029f34a7 not found: ID does not exist"
Sep 09 03:34:50.857716 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:34:50.857685    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-c44cc65a-openshift-default-66c6b44c5b-cnpfj"]
Sep 09 03:34:50.864384 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:34:50.864365    2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-mt-c44cc65a-openshift-default-66c6b44c5b-cnpfj"]
Sep 09 03:34:52.360662 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:34:52.360629    2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="31fc04d2-8016-4b58-ae28-6843df876bc9" path="/var/lib/kubelet/pods/31fc04d2-8016-4b58-ae28-6843df876bc9/volumes"
Sep 09 03:34:53.906619 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:34:53.906590    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-shared-b-w3-c34148-openshift-default-98858768-vhlwz"]
Sep 09 03:34:53.907019 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:34:53.906927    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="01b9dff0-9854-4ecf-8733-81cc00955fc9" containerName="revoke-keys"
Sep 09 03:34:53.907019 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:34:53.906958    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="01b9dff0-9854-4ecf-8733-81cc00955fc9" containerName="revoke-keys"
Sep 09 03:34:53.907019 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:34:53.906971    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7c7e63b7-e5da-4ca9-aa41-46ea78a3e2cb" containerName="payload-processing"
Sep 09 03:34:53.907019 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:34:53.906980    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="7c7e63b7-e5da-4ca9-aa41-46ea78a3e2cb" containerName="payload-processing"
Sep 09 03:34:53.907019 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:34:53.906989    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1e24840e-811b-4d59-8402-1586c3153e6d" containerName="payload-pre-processing"
Sep 09 03:34:53.907019 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:34:53.906995    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="1e24840e-811b-4d59-8402-1586c3153e6d" containerName="payload-pre-processing"
Sep 09 03:34:53.907019 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:34:53.907007    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="31fc04d2-8016-4b58-ae28-6843df876bc9" containerName="istio-proxy"
Sep 09 03:34:53.907019 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:34:53.907012    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="31fc04d2-8016-4b58-ae28-6843df876bc9" containerName="istio-proxy"
Sep 09 03:34:53.907019 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:34:53.907019    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="31fc04d2-8016-4b58-ae28-6843df876bc9" containerName="istio-proxy"
Sep 09 03:34:53.907019 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:34:53.907024    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="31fc04d2-8016-4b58-ae28-6843df876bc9" containerName="istio-proxy"
Sep 09 03:34:53.907318 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:34:53.907069    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="31fc04d2-8016-4b58-ae28-6843df876bc9" containerName="istio-proxy"
Sep 09 03:34:53.907318 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:34:53.907077    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="1e24840e-811b-4d59-8402-1586c3153e6d" containerName="payload-pre-processing"
Sep 09 03:34:53.907318 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:34:53.907083    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="01b9dff0-9854-4ecf-8733-81cc00955fc9" containerName="revoke-keys"
Sep 09 03:34:53.907318 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:34:53.907090    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="7c7e63b7-e5da-4ca9-aa41-46ea78a3e2cb" containerName="payload-processing"
Sep 09 03:34:53.911376 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:34:53.911358    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-w3-c34148-openshift-default-98858768-vhlwz"
Sep 09 03:34:53.925994 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:34:53.925975    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-shared-b-w3-c34148-openshift-default-dockercfg-xsgg6\""
Sep 09 03:34:53.939064 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:34:53.939040    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-b-w3-c34148-openshift-default-98858768-vhlwz"]
Sep 09 03:34:54.031059 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:34:54.031026    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/141cec6f-6a51-48c6-8dc4-8c48ad7d9f40-istio-token\") pod \"e2e-shared-b-w3-c34148-openshift-default-98858768-vhlwz\" (UID: \"141cec6f-6a51-48c6-8dc4-8c48ad7d9f40\") " pod="openshift-ingress/e2e-shared-b-w3-c34148-openshift-default-98858768-vhlwz"
Sep 09 03:34:54.031059 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:34:54.031059    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ngsqq\" (UniqueName: \"kubernetes.io/projected/141cec6f-6a51-48c6-8dc4-8c48ad7d9f40-kube-api-access-ngsqq\") pod \"e2e-shared-b-w3-c34148-openshift-default-98858768-vhlwz\" (UID: \"141cec6f-6a51-48c6-8dc4-8c48ad7d9f40\") " pod="openshift-ingress/e2e-shared-b-w3-c34148-openshift-default-98858768-vhlwz"
Sep 09 03:34:54.031263 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:34:54.031090    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/141cec6f-6a51-48c6-8dc4-8c48ad7d9f40-workload-socket\") pod \"e2e-shared-b-w3-c34148-openshift-default-98858768-vhlwz\" (UID: \"141cec6f-6a51-48c6-8dc4-8c48ad7d9f40\") " pod="openshift-ingress/e2e-shared-b-w3-c34148-openshift-default-98858768-vhlwz"
Sep 09 03:34:54.031263 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:34:54.031111    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/141cec6f-6a51-48c6-8dc4-8c48ad7d9f40-credential-socket\") pod \"e2e-shared-b-w3-c34148-openshift-default-98858768-vhlwz\" (UID: \"141cec6f-6a51-48c6-8dc4-8c48ad7d9f40\") " pod="openshift-ingress/e2e-shared-b-w3-c34148-openshift-default-98858768-vhlwz"
Sep 09 03:34:54.031263 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:34:54.031142    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/141cec6f-6a51-48c6-8dc4-8c48ad7d9f40-istiod-ca-cert\") pod \"e2e-shared-b-w3-c34148-openshift-default-98858768-vhlwz\" (UID: \"141cec6f-6a51-48c6-8dc4-8c48ad7d9f40\") " pod="openshift-ingress/e2e-shared-b-w3-c34148-openshift-default-98858768-vhlwz"
Sep 09 03:34:54.031263 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:34:54.031159    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/141cec6f-6a51-48c6-8dc4-8c48ad7d9f40-workload-certs\") pod \"e2e-shared-b-w3-c34148-openshift-default-98858768-vhlwz\" (UID: \"141cec6f-6a51-48c6-8dc4-8c48ad7d9f40\") " pod="openshift-ingress/e2e-shared-b-w3-c34148-openshift-default-98858768-vhlwz"
Sep 09 03:34:54.031263 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:34:54.031234    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/141cec6f-6a51-48c6-8dc4-8c48ad7d9f40-istio-envoy\") pod \"e2e-shared-b-w3-c34148-openshift-default-98858768-vhlwz\" (UID: \"141cec6f-6a51-48c6-8dc4-8c48ad7d9f40\") " pod="openshift-ingress/e2e-shared-b-w3-c34148-openshift-default-98858768-vhlwz"
Sep 09 03:34:54.031429 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:34:54.031270    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/141cec6f-6a51-48c6-8dc4-8c48ad7d9f40-istio-podinfo\") pod \"e2e-shared-b-w3-c34148-openshift-default-98858768-vhlwz\" (UID: \"141cec6f-6a51-48c6-8dc4-8c48ad7d9f40\") " pod="openshift-ingress/e2e-shared-b-w3-c34148-openshift-default-98858768-vhlwz"
Sep 09 03:34:54.031429 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:34:54.031293    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/141cec6f-6a51-48c6-8dc4-8c48ad7d9f40-istio-data\") pod \"e2e-shared-b-w3-c34148-openshift-default-98858768-vhlwz\" (UID: \"141cec6f-6a51-48c6-8dc4-8c48ad7d9f40\") " pod="openshift-ingress/e2e-shared-b-w3-c34148-openshift-default-98858768-vhlwz"
Sep 09 03:34:54.131975 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:34:54.131942    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/141cec6f-6a51-48c6-8dc4-8c48ad7d9f40-istio-podinfo\") pod \"e2e-shared-b-w3-c34148-openshift-default-98858768-vhlwz\" (UID: \"141cec6f-6a51-48c6-8dc4-8c48ad7d9f40\") " pod="openshift-ingress/e2e-shared-b-w3-c34148-openshift-default-98858768-vhlwz"
Sep 09 03:34:54.131975 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:34:54.131979    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/141cec6f-6a51-48c6-8dc4-8c48ad7d9f40-istio-data\") pod \"e2e-shared-b-w3-c34148-openshift-default-98858768-vhlwz\" (UID: \"141cec6f-6a51-48c6-8dc4-8c48ad7d9f40\") " pod="openshift-ingress/e2e-shared-b-w3-c34148-openshift-default-98858768-vhlwz"
Sep 09 03:34:54.132235 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:34:54.132027    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/141cec6f-6a51-48c6-8dc4-8c48ad7d9f40-istio-token\") pod \"e2e-shared-b-w3-c34148-openshift-default-98858768-vhlwz\" (UID: \"141cec6f-6a51-48c6-8dc4-8c48ad7d9f40\") " pod="openshift-ingress/e2e-shared-b-w3-c34148-openshift-default-98858768-vhlwz"
Sep 09 03:34:54.132235 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:34:54.132042    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ngsqq\" (UniqueName: \"kubernetes.io/projected/141cec6f-6a51-48c6-8dc4-8c48ad7d9f40-kube-api-access-ngsqq\") pod \"e2e-shared-b-w3-c34148-openshift-default-98858768-vhlwz\" (UID: \"141cec6f-6a51-48c6-8dc4-8c48ad7d9f40\") " pod="openshift-ingress/e2e-shared-b-w3-c34148-openshift-default-98858768-vhlwz"
Sep 09 03:34:54.132235 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:34:54.132074    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/141cec6f-6a51-48c6-8dc4-8c48ad7d9f40-workload-socket\") pod \"e2e-shared-b-w3-c34148-openshift-default-98858768-vhlwz\" (UID: \"141cec6f-6a51-48c6-8dc4-8c48ad7d9f40\") " pod="openshift-ingress/e2e-shared-b-w3-c34148-openshift-default-98858768-vhlwz"
Sep 09 03:34:54.132235 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:34:54.132091    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/141cec6f-6a51-48c6-8dc4-8c48ad7d9f40-credential-socket\") pod \"e2e-shared-b-w3-c34148-openshift-default-98858768-vhlwz\" (UID: \"141cec6f-6a51-48c6-8dc4-8c48ad7d9f40\") " pod="openshift-ingress/e2e-shared-b-w3-c34148-openshift-default-98858768-vhlwz"
Sep 09 03:34:54.132235 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:34:54.132107    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/141cec6f-6a51-48c6-8dc4-8c48ad7d9f40-istiod-ca-cert\") pod \"e2e-shared-b-w3-c34148-openshift-default-98858768-vhlwz\" (UID: \"141cec6f-6a51-48c6-8dc4-8c48ad7d9f40\") " pod="openshift-ingress/e2e-shared-b-w3-c34148-openshift-default-98858768-vhlwz"
Sep 09 03:34:54.132235 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:34:54.132125    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/141cec6f-6a51-48c6-8dc4-8c48ad7d9f40-workload-certs\") pod \"e2e-shared-b-w3-c34148-openshift-default-98858768-vhlwz\" (UID: \"141cec6f-6a51-48c6-8dc4-8c48ad7d9f40\") " pod="openshift-ingress/e2e-shared-b-w3-c34148-openshift-default-98858768-vhlwz"
Sep 09 03:34:54.132235 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:34:54.132178    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/141cec6f-6a51-48c6-8dc4-8c48ad7d9f40-istio-envoy\") pod \"e2e-shared-b-w3-c34148-openshift-default-98858768-vhlwz\" (UID: \"141cec6f-6a51-48c6-8dc4-8c48ad7d9f40\") " pod="openshift-ingress/e2e-shared-b-w3-c34148-openshift-default-98858768-vhlwz"
Sep 09 03:34:54.132580 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:34:54.132369    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/141cec6f-6a51-48c6-8dc4-8c48ad7d9f40-istio-data\") pod \"e2e-shared-b-w3-c34148-openshift-default-98858768-vhlwz\" (UID: \"141cec6f-6a51-48c6-8dc4-8c48ad7d9f40\") " pod="openshift-ingress/e2e-shared-b-w3-c34148-openshift-default-98858768-vhlwz"
Sep 09 03:34:54.132580 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:34:54.132455    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/141cec6f-6a51-48c6-8dc4-8c48ad7d9f40-workload-socket\") pod \"e2e-shared-b-w3-c34148-openshift-default-98858768-vhlwz\" (UID: \"141cec6f-6a51-48c6-8dc4-8c48ad7d9f40\") " pod="openshift-ingress/e2e-shared-b-w3-c34148-openshift-default-98858768-vhlwz"
Sep 09 03:34:54.132689 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:34:54.132658    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/141cec6f-6a51-48c6-8dc4-8c48ad7d9f40-workload-certs\") pod \"e2e-shared-b-w3-c34148-openshift-default-98858768-vhlwz\" (UID: \"141cec6f-6a51-48c6-8dc4-8c48ad7d9f40\") " pod="openshift-ingress/e2e-shared-b-w3-c34148-openshift-default-98858768-vhlwz"
Sep 09 03:34:54.132743 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:34:54.132686    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/141cec6f-6a51-48c6-8dc4-8c48ad7d9f40-credential-socket\") pod \"e2e-shared-b-w3-c34148-openshift-default-98858768-vhlwz\" (UID: \"141cec6f-6a51-48c6-8dc4-8c48ad7d9f40\") " pod="openshift-ingress/e2e-shared-b-w3-c34148-openshift-default-98858768-vhlwz"
Sep 09 03:34:54.133156 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:34:54.133131    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/141cec6f-6a51-48c6-8dc4-8c48ad7d9f40-istiod-ca-cert\") pod \"e2e-shared-b-w3-c34148-openshift-default-98858768-vhlwz\" (UID: \"141cec6f-6a51-48c6-8dc4-8c48ad7d9f40\") " pod="openshift-ingress/e2e-shared-b-w3-c34148-openshift-default-98858768-vhlwz"
Sep 09 03:34:54.134370 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:34:54.134349    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/141cec6f-6a51-48c6-8dc4-8c48ad7d9f40-istio-envoy\") pod \"e2e-shared-b-w3-c34148-openshift-default-98858768-vhlwz\" (UID: \"141cec6f-6a51-48c6-8dc4-8c48ad7d9f40\") " pod="openshift-ingress/e2e-shared-b-w3-c34148-openshift-default-98858768-vhlwz"
Sep 09 03:34:54.134610 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:34:54.134590    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/141cec6f-6a51-48c6-8dc4-8c48ad7d9f40-istio-podinfo\") pod \"e2e-shared-b-w3-c34148-openshift-default-98858768-vhlwz\" (UID: \"141cec6f-6a51-48c6-8dc4-8c48ad7d9f40\") " pod="openshift-ingress/e2e-shared-b-w3-c34148-openshift-default-98858768-vhlwz"
Sep 09 03:34:54.184295 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:34:54.184227    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ngsqq\" (UniqueName: \"kubernetes.io/projected/141cec6f-6a51-48c6-8dc4-8c48ad7d9f40-kube-api-access-ngsqq\") pod \"e2e-shared-b-w3-c34148-openshift-default-98858768-vhlwz\" (UID: \"141cec6f-6a51-48c6-8dc4-8c48ad7d9f40\") " pod="openshift-ingress/e2e-shared-b-w3-c34148-openshift-default-98858768-vhlwz"
Sep 09 03:34:54.198965 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:34:54.198923    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/141cec6f-6a51-48c6-8dc4-8c48ad7d9f40-istio-token\") pod \"e2e-shared-b-w3-c34148-openshift-default-98858768-vhlwz\" (UID: \"141cec6f-6a51-48c6-8dc4-8c48ad7d9f40\") " pod="openshift-ingress/e2e-shared-b-w3-c34148-openshift-default-98858768-vhlwz"
Sep 09 03:34:54.221063 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:34:54.221037    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-w3-c34148-openshift-default-98858768-vhlwz"
Sep 09 03:34:54.542067 ip-10-0-132-14 kubenswrapper[2576]: W0909 03:34:54.542028    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod141cec6f_6a51_48c6_8dc4_8c48ad7d9f40.slice/crio-8791091ad18f4c32f88b9f3a2eb8427a1501c1450170efa3984fed6900924b6a WatchSource:0}: Error finding container 8791091ad18f4c32f88b9f3a2eb8427a1501c1450170efa3984fed6900924b6a: Status 404 returned error can't find the container with id 8791091ad18f4c32f88b9f3a2eb8427a1501c1450170efa3984fed6900924b6a
Sep 09 03:34:54.545988 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:34:54.545953    2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892148Ki","pods":"250"}
Sep 09 03:34:54.546093 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:34:54.546026    2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892148Ki","pods":"250"}
Sep 09 03:34:54.546093 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:34:54.546063    2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892148Ki","pods":"250"}
Sep 09 03:34:54.547361 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:34:54.547340    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-b-w3-c34148-openshift-default-98858768-vhlwz"]
Sep 09 03:34:54.854530 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:34:54.854444    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w3-c34148-openshift-default-98858768-vhlwz" event={"ID":"141cec6f-6a51-48c6-8dc4-8c48ad7d9f40","Type":"ContainerStarted","Data":"dc95722b84320daaf603db3d818ae28e0943f8de126a0a1588c3786126850eec"}
Sep 09 03:34:54.854530 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:34:54.854493    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w3-c34148-openshift-default-98858768-vhlwz" event={"ID":"141cec6f-6a51-48c6-8dc4-8c48ad7d9f40","Type":"ContainerStarted","Data":"8791091ad18f4c32f88b9f3a2eb8427a1501c1450170efa3984fed6900924b6a"}
Sep 09 03:34:54.899684 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:34:54.899642    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-shared-b-w3-c34148-openshift-default-98858768-vhlwz" podStartSLOduration=1.899628839 podStartE2EDuration="1.899628839s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 03:34:53 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 03:34:54.898750571 +0000 UTC m=+1309.040236674" watchObservedRunningTime="2026-09-09 03:34:54.899628839 +0000 UTC m=+1309.041114917"
Sep 09 03:34:55.221445 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:34:55.221375    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-b-w3-c34148-openshift-default-98858768-vhlwz"
Sep 09 03:34:55.225922 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:34:55.225897    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-b-w3-c34148-openshift-default-98858768-vhlwz"
Sep 09 03:34:55.859940 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:34:55.859906    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-b-w3-c34148-openshift-default-98858768-vhlwz"
Sep 09 03:34:55.860739 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:34:55.860720    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-b-w3-c34148-openshift-default-98858768-vhlwz"
Sep 09 03:34:59.746493 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:34:59.746460    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w3-c34148-5f7b74bdb6-r78wm"]
Sep 09 03:34:59.746963 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:34:59.746823    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="31fc04d2-8016-4b58-ae28-6843df876bc9" containerName="istio-proxy"
Sep 09 03:34:59.750718 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:34:59.750702    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-c34148-5f7b74bdb6-r78wm"
Sep 09 03:34:59.757820 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:34:59.757591    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-shared-b-w3-c34148\""
Sep 09 03:34:59.757820 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:34:59.757608    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-shared-b-w3-c34148-dockercfg-gk2lv\""
Sep 09 03:34:59.793320 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:34:59.793289    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w3-c34148-5f7b74bdb6-r78wm"]
Sep 09 03:34:59.879190 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:34:59.879161    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nkx7f\" (UniqueName: \"kubernetes.io/projected/785d5422-0337-4a7b-bb4e-fa3fdcdc0ce7-kube-api-access-nkx7f\") pod \"payload-pre-processing-e2e-shared-b-w3-c34148-5f7b74bdb6-r78wm\" (UID: \"785d5422-0337-4a7b-bb4e-fa3fdcdc0ce7\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-c34148-5f7b74bdb6-r78wm"
Sep 09 03:34:59.879326 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:34:59.879217    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/785d5422-0337-4a7b-bb4e-fa3fdcdc0ce7-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-b-w3-c34148-5f7b74bdb6-r78wm\" (UID: \"785d5422-0337-4a7b-bb4e-fa3fdcdc0ce7\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-c34148-5f7b74bdb6-r78wm"
Sep 09 03:34:59.963838 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:34:59.963795    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w3-c34148-779dbb5bc9-4hzjv"]
Sep 09 03:34:59.967125 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:34:59.967108    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-b-w3-c34148-779dbb5bc9-4hzjv"
Sep 09 03:34:59.974565 ip-10-0-132-14 kubenswrapper[2576]: E0909 03:34:59.974538    2576 status_manager.go:1018] "Failed to get status for pod" err="pods \"payload-processing-e2e-shared-b-w3-c34148-779dbb5bc9-4hzjv\" is forbidden: User \"system:node:ip-10-0-132-14.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"openshift-ingress\": no relationship found between node 'ip-10-0-132-14.ec2.internal' and this object" podUID="321fa962-c540-4eb3-9bd5-af782e8fb5c9" pod="openshift-ingress/payload-processing-e2e-shared-b-w3-c34148-779dbb5bc9-4hzjv"
Sep 09 03:34:59.979856 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:34:59.979833    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/785d5422-0337-4a7b-bb4e-fa3fdcdc0ce7-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-b-w3-c34148-5f7b74bdb6-r78wm\" (UID: \"785d5422-0337-4a7b-bb4e-fa3fdcdc0ce7\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-c34148-5f7b74bdb6-r78wm"
Sep 09 03:34:59.979940 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:34:59.979915    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nkx7f\" (UniqueName: \"kubernetes.io/projected/785d5422-0337-4a7b-bb4e-fa3fdcdc0ce7-kube-api-access-nkx7f\") pod \"payload-pre-processing-e2e-shared-b-w3-c34148-5f7b74bdb6-r78wm\" (UID: \"785d5422-0337-4a7b-bb4e-fa3fdcdc0ce7\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-c34148-5f7b74bdb6-r78wm"
Sep 09 03:34:59.980642 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:34:59.980625    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/785d5422-0337-4a7b-bb4e-fa3fdcdc0ce7-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-b-w3-c34148-5f7b74bdb6-r78wm\" (UID: \"785d5422-0337-4a7b-bb4e-fa3fdcdc0ce7\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-c34148-5f7b74bdb6-r78wm"
Sep 09 03:34:59.995979 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:34:59.995956    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nkx7f\" (UniqueName: \"kubernetes.io/projected/785d5422-0337-4a7b-bb4e-fa3fdcdc0ce7-kube-api-access-nkx7f\") pod \"payload-pre-processing-e2e-shared-b-w3-c34148-5f7b74bdb6-r78wm\" (UID: \"785d5422-0337-4a7b-bb4e-fa3fdcdc0ce7\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-c34148-5f7b74bdb6-r78wm"
Sep 09 03:35:00.043248 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:35:00.043225    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w3-c34148-779dbb5bc9-4hzjv"]
Sep 09 03:35:00.061348 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:35:00.061320    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-c34148-5f7b74bdb6-r78wm"
Sep 09 03:35:00.080694 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:35:00.080671    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/321fa962-c540-4eb3-9bd5-af782e8fb5c9-plugins-config-volume\") pod \"payload-processing-e2e-shared-b-w3-c34148-779dbb5bc9-4hzjv\" (UID: \"321fa962-c540-4eb3-9bd5-af782e8fb5c9\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w3-c34148-779dbb5bc9-4hzjv"
Sep 09 03:35:00.080822 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:35:00.080786    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-klk9m\" (UniqueName: \"kubernetes.io/projected/321fa962-c540-4eb3-9bd5-af782e8fb5c9-kube-api-access-klk9m\") pod \"payload-processing-e2e-shared-b-w3-c34148-779dbb5bc9-4hzjv\" (UID: \"321fa962-c540-4eb3-9bd5-af782e8fb5c9\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w3-c34148-779dbb5bc9-4hzjv"
Sep 09 03:35:00.182215 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:35:00.182179    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-klk9m\" (UniqueName: \"kubernetes.io/projected/321fa962-c540-4eb3-9bd5-af782e8fb5c9-kube-api-access-klk9m\") pod \"payload-processing-e2e-shared-b-w3-c34148-779dbb5bc9-4hzjv\" (UID: \"321fa962-c540-4eb3-9bd5-af782e8fb5c9\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w3-c34148-779dbb5bc9-4hzjv"
Sep 09 03:35:00.182406 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:35:00.182273    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/321fa962-c540-4eb3-9bd5-af782e8fb5c9-plugins-config-volume\") pod \"payload-processing-e2e-shared-b-w3-c34148-779dbb5bc9-4hzjv\" (UID: \"321fa962-c540-4eb3-9bd5-af782e8fb5c9\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w3-c34148-779dbb5bc9-4hzjv"
Sep 09 03:35:00.182972 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:35:00.182951    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/321fa962-c540-4eb3-9bd5-af782e8fb5c9-plugins-config-volume\") pod \"payload-processing-e2e-shared-b-w3-c34148-779dbb5bc9-4hzjv\" (UID: \"321fa962-c540-4eb3-9bd5-af782e8fb5c9\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w3-c34148-779dbb5bc9-4hzjv"
Sep 09 03:35:00.200071 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:35:00.200046    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-klk9m\" (UniqueName: \"kubernetes.io/projected/321fa962-c540-4eb3-9bd5-af782e8fb5c9-kube-api-access-klk9m\") pod \"payload-processing-e2e-shared-b-w3-c34148-779dbb5bc9-4hzjv\" (UID: \"321fa962-c540-4eb3-9bd5-af782e8fb5c9\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w3-c34148-779dbb5bc9-4hzjv"
Sep 09 03:35:00.276363 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:35:00.276338    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-b-w3-c34148-779dbb5bc9-4hzjv"
Sep 09 03:35:00.348257 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:35:00.348228    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w3-c34148-5f7b74bdb6-r78wm"]
Sep 09 03:35:00.349576 ip-10-0-132-14 kubenswrapper[2576]: W0909 03:35:00.349542    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod785d5422_0337_4a7b_bb4e_fa3fdcdc0ce7.slice/crio-fc51ea640c207a2da063e558779ebd83e9fb251f7623fa047bc7d89ccf45a82d WatchSource:0}: Error finding container fc51ea640c207a2da063e558779ebd83e9fb251f7623fa047bc7d89ccf45a82d: Status 404 returned error can't find the container with id fc51ea640c207a2da063e558779ebd83e9fb251f7623fa047bc7d89ccf45a82d
Sep 09 03:35:00.565559 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:35:00.565533    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w3-c34148-779dbb5bc9-4hzjv"]
Sep 09 03:35:00.567038 ip-10-0-132-14 kubenswrapper[2576]: W0909 03:35:00.567008    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod321fa962_c540_4eb3_9bd5_af782e8fb5c9.slice/crio-8b8085ac96343f815eaf15780d9e0430d0b7da0ec7c3511ee69bf05ea3318879 WatchSource:0}: Error finding container 8b8085ac96343f815eaf15780d9e0430d0b7da0ec7c3511ee69bf05ea3318879: Status 404 returned error can't find the container with id 8b8085ac96343f815eaf15780d9e0430d0b7da0ec7c3511ee69bf05ea3318879
Sep 09 03:35:00.886731 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:35:00.886641    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-b-w3-c34148-779dbb5bc9-4hzjv" event={"ID":"321fa962-c540-4eb3-9bd5-af782e8fb5c9","Type":"ContainerStarted","Data":"488a1adf73f76a3ace6788b62ac85dc560739cd1b6a27dc009200d60b72c4599"}
Sep 09 03:35:00.887201 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:35:00.886734    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-b-w3-c34148-779dbb5bc9-4hzjv" event={"ID":"321fa962-c540-4eb3-9bd5-af782e8fb5c9","Type":"ContainerStarted","Data":"8b8085ac96343f815eaf15780d9e0430d0b7da0ec7c3511ee69bf05ea3318879"}
Sep 09 03:35:00.887201 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:35:00.886763    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-shared-b-w3-c34148-779dbb5bc9-4hzjv"
Sep 09 03:35:00.888527 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:35:00.888497    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-c34148-5f7b74bdb6-r78wm" event={"ID":"785d5422-0337-4a7b-bb4e-fa3fdcdc0ce7","Type":"ContainerStarted","Data":"ce1ad7f191fd3efd3850bb99b87844dfdd7e9e4b262737ee944adc5f2ac06dca"}
Sep 09 03:35:00.888682 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:35:00.888545    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-c34148-5f7b74bdb6-r78wm" event={"ID":"785d5422-0337-4a7b-bb4e-fa3fdcdc0ce7","Type":"ContainerStarted","Data":"fc51ea640c207a2da063e558779ebd83e9fb251f7623fa047bc7d89ccf45a82d"}
Sep 09 03:35:00.888682 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:35:00.888599    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-c34148-5f7b74bdb6-r78wm"
Sep 09 03:35:00.911013 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:35:00.910961    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-shared-b-w3-c34148-779dbb5bc9-4hzjv" podStartSLOduration=1.91092511 podStartE2EDuration="1.91092511s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 03:34:59 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 03:35:00.908908121 +0000 UTC m=+1315.050394201" watchObservedRunningTime="2026-09-09 03:35:00.91092511 +0000 UTC m=+1315.052411188"
Sep 09 03:35:00.930453 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:35:00.930404    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-c34148-5f7b74bdb6-r78wm" podStartSLOduration=1.930385585 podStartE2EDuration="1.930385585s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 03:34:59 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 03:35:00.928961185 +0000 UTC m=+1315.070447261" watchObservedRunningTime="2026-09-09 03:35:00.930385585 +0000 UTC m=+1315.071871662"
Sep 09 03:35:08.235110 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:35:08.233640    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w2-eca017-6b7b56d98-lfn2v"]
Sep 09 03:35:08.239828 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:35:08.237749    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-eca017-6b7b56d98-lfn2v"
Sep 09 03:35:08.244827 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:35:08.243920    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-shared-a-w2-eca017\""
Sep 09 03:35:08.252825 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:35:08.249469    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w2-eca017-6b7b56d98-lfn2v"]
Sep 09 03:35:08.360693 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:35:08.360636    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-v8mcq\" (UniqueName: \"kubernetes.io/projected/609e896a-5dd3-42d8-a251-a225d4ddab03-kube-api-access-v8mcq\") pod \"payload-pre-processing-e2e-shared-a-w2-eca017-6b7b56d98-lfn2v\" (UID: \"609e896a-5dd3-42d8-a251-a225d4ddab03\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-eca017-6b7b56d98-lfn2v"
Sep 09 03:35:08.360936 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:35:08.360840    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/609e896a-5dd3-42d8-a251-a225d4ddab03-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-a-w2-eca017-6b7b56d98-lfn2v\" (UID: \"609e896a-5dd3-42d8-a251-a225d4ddab03\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-eca017-6b7b56d98-lfn2v"
Sep 09 03:35:08.364594 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:35:08.364571    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w2-eca017-74b8bd4b8c-fqsmw"]
Sep 09 03:35:08.367849 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:35:08.367830    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-a-w2-eca017-74b8bd4b8c-fqsmw"
Sep 09 03:35:08.371571 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:35:08.371549    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-shared-a-w2-eca017-dockercfg-pgq5z\""
Sep 09 03:35:08.380668 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:35:08.380644    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w2-eca017-74b8bd4b8c-fqsmw"]
Sep 09 03:35:08.461311 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:35:08.461274    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-v8mcq\" (UniqueName: \"kubernetes.io/projected/609e896a-5dd3-42d8-a251-a225d4ddab03-kube-api-access-v8mcq\") pod \"payload-pre-processing-e2e-shared-a-w2-eca017-6b7b56d98-lfn2v\" (UID: \"609e896a-5dd3-42d8-a251-a225d4ddab03\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-eca017-6b7b56d98-lfn2v"
Sep 09 03:35:08.461517 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:35:08.461338    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ed3d5b68-e7e0-4788-889e-6a8ab165da9a-plugins-config-volume\") pod \"payload-processing-e2e-shared-a-w2-eca017-74b8bd4b8c-fqsmw\" (UID: \"ed3d5b68-e7e0-4788-889e-6a8ab165da9a\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w2-eca017-74b8bd4b8c-fqsmw"
Sep 09 03:35:08.461593 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:35:08.461529    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/609e896a-5dd3-42d8-a251-a225d4ddab03-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-a-w2-eca017-6b7b56d98-lfn2v\" (UID: \"609e896a-5dd3-42d8-a251-a225d4ddab03\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-eca017-6b7b56d98-lfn2v"
Sep 09 03:35:08.461675 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:35:08.461593    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2qm6t\" (UniqueName: \"kubernetes.io/projected/ed3d5b68-e7e0-4788-889e-6a8ab165da9a-kube-api-access-2qm6t\") pod \"payload-processing-e2e-shared-a-w2-eca017-74b8bd4b8c-fqsmw\" (UID: \"ed3d5b68-e7e0-4788-889e-6a8ab165da9a\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w2-eca017-74b8bd4b8c-fqsmw"
Sep 09 03:35:08.462258 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:35:08.462238    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/609e896a-5dd3-42d8-a251-a225d4ddab03-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-a-w2-eca017-6b7b56d98-lfn2v\" (UID: \"609e896a-5dd3-42d8-a251-a225d4ddab03\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-eca017-6b7b56d98-lfn2v"
Sep 09 03:35:08.470217 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:35:08.470183    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-v8mcq\" (UniqueName: \"kubernetes.io/projected/609e896a-5dd3-42d8-a251-a225d4ddab03-kube-api-access-v8mcq\") pod \"payload-pre-processing-e2e-shared-a-w2-eca017-6b7b56d98-lfn2v\" (UID: \"609e896a-5dd3-42d8-a251-a225d4ddab03\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-eca017-6b7b56d98-lfn2v"
Sep 09 03:35:08.561746 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:35:08.561702    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-eca017-6b7b56d98-lfn2v"
Sep 09 03:35:08.562078 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:35:08.562049    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ed3d5b68-e7e0-4788-889e-6a8ab165da9a-plugins-config-volume\") pod \"payload-processing-e2e-shared-a-w2-eca017-74b8bd4b8c-fqsmw\" (UID: \"ed3d5b68-e7e0-4788-889e-6a8ab165da9a\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w2-eca017-74b8bd4b8c-fqsmw"
Sep 09 03:35:08.562124 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:35:08.562110    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2qm6t\" (UniqueName: \"kubernetes.io/projected/ed3d5b68-e7e0-4788-889e-6a8ab165da9a-kube-api-access-2qm6t\") pod \"payload-processing-e2e-shared-a-w2-eca017-74b8bd4b8c-fqsmw\" (UID: \"ed3d5b68-e7e0-4788-889e-6a8ab165da9a\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w2-eca017-74b8bd4b8c-fqsmw"
Sep 09 03:35:08.562729 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:35:08.562709    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ed3d5b68-e7e0-4788-889e-6a8ab165da9a-plugins-config-volume\") pod \"payload-processing-e2e-shared-a-w2-eca017-74b8bd4b8c-fqsmw\" (UID: \"ed3d5b68-e7e0-4788-889e-6a8ab165da9a\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w2-eca017-74b8bd4b8c-fqsmw"
Sep 09 03:35:08.571723 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:35:08.571693    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2qm6t\" (UniqueName: \"kubernetes.io/projected/ed3d5b68-e7e0-4788-889e-6a8ab165da9a-kube-api-access-2qm6t\") pod \"payload-processing-e2e-shared-a-w2-eca017-74b8bd4b8c-fqsmw\" (UID: \"ed3d5b68-e7e0-4788-889e-6a8ab165da9a\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w2-eca017-74b8bd4b8c-fqsmw"
Sep 09 03:35:08.679705 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:35:08.679679    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-a-w2-eca017-74b8bd4b8c-fqsmw"
Sep 09 03:35:08.841880 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:35:08.841010    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w2-eca017-6b7b56d98-lfn2v"]
Sep 09 03:35:08.946829 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:35:08.945973    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-eca017-6b7b56d98-lfn2v" event={"ID":"609e896a-5dd3-42d8-a251-a225d4ddab03","Type":"ContainerStarted","Data":"931e37ad007775009cab8b99f7cda904aed50f71bcb9898786fcb072f75f92da"}
Sep 09 03:35:08.946829 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:35:08.946040    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-eca017-6b7b56d98-lfn2v" event={"ID":"609e896a-5dd3-42d8-a251-a225d4ddab03","Type":"ContainerStarted","Data":"c9f70fd1f78cff2c20b128e693beeb799aca33622cc94d32eba53ca7d37dcc1f"}
Sep 09 03:35:08.946829 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:35:08.946761    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-eca017-6b7b56d98-lfn2v"
Sep 09 03:35:08.965894 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:35:08.965830    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-eca017-6b7b56d98-lfn2v" podStartSLOduration=0.965785655 podStartE2EDuration="965.785655ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 03:35:08 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 03:35:08.965192965 +0000 UTC m=+1323.106679043" watchObservedRunningTime="2026-09-09 03:35:08.965785655 +0000 UTC m=+1323.107271734"
Sep 09 03:35:08.971177 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:35:08.971149    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w2-eca017-74b8bd4b8c-fqsmw"]
Sep 09 03:35:08.974535 ip-10-0-132-14 kubenswrapper[2576]: W0909 03:35:08.974510    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poded3d5b68_e7e0_4788_889e_6a8ab165da9a.slice/crio-e206fb4ba31ff45b5d4cc0edbda25226cb6d20d2d8588d3a63f1a9f6c585aaa6 WatchSource:0}: Error finding container e206fb4ba31ff45b5d4cc0edbda25226cb6d20d2d8588d3a63f1a9f6c585aaa6: Status 404 returned error can't find the container with id e206fb4ba31ff45b5d4cc0edbda25226cb6d20d2d8588d3a63f1a9f6c585aaa6
Sep 09 03:35:09.954933 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:35:09.954882    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-a-w2-eca017-74b8bd4b8c-fqsmw" event={"ID":"ed3d5b68-e7e0-4788-889e-6a8ab165da9a","Type":"ContainerStarted","Data":"a1036e9fe33f66ae7f2c23e9cdac8812c6c455420619d80868ed1a700dc6ccda"}
Sep 09 03:35:09.955413 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:35:09.954956    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-a-w2-eca017-74b8bd4b8c-fqsmw" event={"ID":"ed3d5b68-e7e0-4788-889e-6a8ab165da9a","Type":"ContainerStarted","Data":"e206fb4ba31ff45b5d4cc0edbda25226cb6d20d2d8588d3a63f1a9f6c585aaa6"}
Sep 09 03:35:09.955413 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:35:09.955182    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-shared-a-w2-eca017-74b8bd4b8c-fqsmw"
Sep 09 03:35:09.981195 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:35:09.981140    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-shared-a-w2-eca017-74b8bd4b8c-fqsmw" podStartSLOduration=1.9811216109999998 podStartE2EDuration="1.981121611s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 03:35:08 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 03:35:09.979092005 +0000 UTC m=+1324.120578094" watchObservedRunningTime="2026-09-09 03:35:09.981121611 +0000 UTC m=+1324.122607690"
Sep 09 03:35:11.895895 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:35:11.895865    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-c34148-5f7b74bdb6-r78wm"
Sep 09 03:35:11.896327 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:35:11.896168    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-shared-b-w3-c34148-779dbb5bc9-4hzjv"
Sep 09 03:35:11.901914 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:35:11.901890    2576 patch_prober.go:28] interesting pod/e2e-shared-b-w3-c34148-openshift-default-98858768-vhlwz container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.57:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Sep 09 03:35:11.902000 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:35:11.901942    2576 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-shared-b-w3-c34148-openshift-default-98858768-vhlwz" podUID="141cec6f-6a51-48c6-8dc4-8c48ad7d9f40" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.57:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Sep 09 03:35:13.978455 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:35:13.978421    2576 generic.go:356] "Generic (PLEG): container finished" podID="141cec6f-6a51-48c6-8dc4-8c48ad7d9f40" containerID="dc95722b84320daaf603db3d818ae28e0943f8de126a0a1588c3786126850eec" exitCode=0
Sep 09 03:35:13.978928 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:35:13.978496    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w3-c34148-openshift-default-98858768-vhlwz" event={"ID":"141cec6f-6a51-48c6-8dc4-8c48ad7d9f40","Type":"ContainerDied","Data":"dc95722b84320daaf603db3d818ae28e0943f8de126a0a1588c3786126850eec"}
Sep 09 03:35:13.978988 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:35:13.978942    2576 scope.go:117] "RemoveContainer" containerID="dc95722b84320daaf603db3d818ae28e0943f8de126a0a1588c3786126850eec"
Sep 09 03:35:13.980082 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:35:13.980055    2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892148Ki","pods":"250"}
Sep 09 03:35:13.980157 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:35:13.980129    2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892148Ki","pods":"250"}
Sep 09 03:35:13.980195 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:35:13.980185    2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892148Ki","pods":"250"}
Sep 09 03:35:14.985749 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:35:14.985717    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w3-c34148-openshift-default-98858768-vhlwz" event={"ID":"141cec6f-6a51-48c6-8dc4-8c48ad7d9f40","Type":"ContainerStarted","Data":"d5bfc0c5f6d4aace9b815ac6454284980def8743f01b6ed4d6c6c738f04ec819"}
Sep 09 03:35:15.221378 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:35:15.221345    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-b-w3-c34148-openshift-default-98858768-vhlwz"
Sep 09 03:35:15.221537 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:35:15.221514    2576 patch_prober.go:28] interesting pod/e2e-shared-b-w3-c34148-openshift-default-98858768-vhlwz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.57:15021/healthz/ready\": dial tcp 10.133.0.57:15021: connect: connection refused" start-of-body=
Sep 09 03:35:15.221585 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:35:15.221564    2576 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w3-c34148-openshift-default-98858768-vhlwz" podUID="141cec6f-6a51-48c6-8dc4-8c48ad7d9f40" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.57:15021/healthz/ready\": dial tcp 10.133.0.57:15021: connect: connection refused"
Sep 09 03:35:16.222004 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:35:16.221965    2576 patch_prober.go:28] interesting pod/e2e-shared-b-w3-c34148-openshift-default-98858768-vhlwz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.57:15021/healthz/ready\": dial tcp 10.133.0.57:15021: connect: connection refused" start-of-body=
Sep 09 03:35:16.222457 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:35:16.222023    2576 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w3-c34148-openshift-default-98858768-vhlwz" podUID="141cec6f-6a51-48c6-8dc4-8c48ad7d9f40" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.57:15021/healthz/ready\": dial tcp 10.133.0.57:15021: connect: connection refused"
Sep 09 03:35:17.221688 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:35:17.221655    2576 patch_prober.go:28] interesting pod/e2e-shared-b-w3-c34148-openshift-default-98858768-vhlwz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.57:15021/healthz/ready\": dial tcp 10.133.0.57:15021: connect: connection refused" start-of-body=
Sep 09 03:35:17.221885 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:35:17.221707    2576 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w3-c34148-openshift-default-98858768-vhlwz" podUID="141cec6f-6a51-48c6-8dc4-8c48ad7d9f40" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.57:15021/healthz/ready\": dial tcp 10.133.0.57:15021: connect: connection refused"
Sep 09 03:35:18.221648 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:35:18.221614    2576 patch_prober.go:28] interesting pod/e2e-shared-b-w3-c34148-openshift-default-98858768-vhlwz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.57:15021/healthz/ready\": dial tcp 10.133.0.57:15021: connect: connection refused" start-of-body=
Sep 09 03:35:18.222039 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:35:18.221679    2576 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w3-c34148-openshift-default-98858768-vhlwz" podUID="141cec6f-6a51-48c6-8dc4-8c48ad7d9f40" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.57:15021/healthz/ready\": dial tcp 10.133.0.57:15021: connect: connection refused"
Sep 09 03:35:19.222154 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:35:19.222120    2576 patch_prober.go:28] interesting pod/e2e-shared-b-w3-c34148-openshift-default-98858768-vhlwz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.57:15021/healthz/ready\": dial tcp 10.133.0.57:15021: connect: connection refused" start-of-body=
Sep 09 03:35:19.222527 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:35:19.222191    2576 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w3-c34148-openshift-default-98858768-vhlwz" podUID="141cec6f-6a51-48c6-8dc4-8c48ad7d9f40" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.57:15021/healthz/ready\": dial tcp 10.133.0.57:15021: connect: connection refused"
Sep 09 03:35:20.225138 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:35:20.225105    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-b-w3-c34148-openshift-default-98858768-vhlwz"
Sep 09 03:35:20.225501 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:35:20.225412    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-b-w3-c34148-openshift-default-98858768-vhlwz"
Sep 09 03:35:20.226028 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:35:20.226008    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-b-w3-c34148-openshift-default-98858768-vhlwz"
Sep 09 03:35:20.963873 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:35:20.963837    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-shared-a-w2-eca017-74b8bd4b8c-fqsmw"
Sep 09 03:35:20.964072 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:35:20.963923    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-eca017-6b7b56d98-lfn2v"
Sep 09 03:35:37.812134 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:35:37.812100    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w2-0d0cae-68c69b8786-qt4kh"]
Sep 09 03:35:37.815366 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:35:37.815348    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-0d0cae-68c69b8786-qt4kh"
Sep 09 03:35:37.818134 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:35:37.818114    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-shared-b-w2-0d0cae-dockercfg-mxv74\""
Sep 09 03:35:37.818260 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:35:37.818130    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-shared-b-w2-0d0cae\""
Sep 09 03:35:37.833309 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:35:37.833287    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w2-0d0cae-68c69b8786-qt4kh"]
Sep 09 03:35:37.926416 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:35:37.926383    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8v8xt\" (UniqueName: \"kubernetes.io/projected/c02969f1-8ac8-4c3d-9b70-c4f5f29a6c89-kube-api-access-8v8xt\") pod \"payload-pre-processing-e2e-shared-b-w2-0d0cae-68c69b8786-qt4kh\" (UID: \"c02969f1-8ac8-4c3d-9b70-c4f5f29a6c89\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-0d0cae-68c69b8786-qt4kh"
Sep 09 03:35:37.926587 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:35:37.926504    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c02969f1-8ac8-4c3d-9b70-c4f5f29a6c89-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-b-w2-0d0cae-68c69b8786-qt4kh\" (UID: \"c02969f1-8ac8-4c3d-9b70-c4f5f29a6c89\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-0d0cae-68c69b8786-qt4kh"
Sep 09 03:35:38.027412 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:35:38.027374    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8v8xt\" (UniqueName: \"kubernetes.io/projected/c02969f1-8ac8-4c3d-9b70-c4f5f29a6c89-kube-api-access-8v8xt\") pod \"payload-pre-processing-e2e-shared-b-w2-0d0cae-68c69b8786-qt4kh\" (UID: \"c02969f1-8ac8-4c3d-9b70-c4f5f29a6c89\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-0d0cae-68c69b8786-qt4kh"
Sep 09 03:35:38.027610 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:35:38.027460    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c02969f1-8ac8-4c3d-9b70-c4f5f29a6c89-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-b-w2-0d0cae-68c69b8786-qt4kh\" (UID: \"c02969f1-8ac8-4c3d-9b70-c4f5f29a6c89\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-0d0cae-68c69b8786-qt4kh"
Sep 09 03:35:38.028124 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:35:38.028103    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c02969f1-8ac8-4c3d-9b70-c4f5f29a6c89-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-b-w2-0d0cae-68c69b8786-qt4kh\" (UID: \"c02969f1-8ac8-4c3d-9b70-c4f5f29a6c89\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-0d0cae-68c69b8786-qt4kh"
Sep 09 03:35:38.037196 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:35:38.037171    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8v8xt\" (UniqueName: \"kubernetes.io/projected/c02969f1-8ac8-4c3d-9b70-c4f5f29a6c89-kube-api-access-8v8xt\") pod \"payload-pre-processing-e2e-shared-b-w2-0d0cae-68c69b8786-qt4kh\" (UID: \"c02969f1-8ac8-4c3d-9b70-c4f5f29a6c89\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-0d0cae-68c69b8786-qt4kh"
Sep 09 03:35:38.125551 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:35:38.125464    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-0d0cae-68c69b8786-qt4kh"
Sep 09 03:35:38.399017 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:35:38.398994    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w2-0d0cae-68c69b8786-qt4kh"]
Sep 09 03:35:38.401786 ip-10-0-132-14 kubenswrapper[2576]: W0909 03:35:38.401743    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc02969f1_8ac8_4c3d_9b70_c4f5f29a6c89.slice/crio-0069bd7c1d83dfc1c5f2a3f8addd1735afe758ee2e66e89cbfa9343abf6ed7ab WatchSource:0}: Error finding container 0069bd7c1d83dfc1c5f2a3f8addd1735afe758ee2e66e89cbfa9343abf6ed7ab: Status 404 returned error can't find the container with id 0069bd7c1d83dfc1c5f2a3f8addd1735afe758ee2e66e89cbfa9343abf6ed7ab
Sep 09 03:35:39.106987 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:35:39.106949    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-0d0cae-68c69b8786-qt4kh" event={"ID":"c02969f1-8ac8-4c3d-9b70-c4f5f29a6c89","Type":"ContainerStarted","Data":"5eab9b98dbb15238c637c6e38ad53596967ecc65150a01a01462291c9a32840d"}
Sep 09 03:35:39.107396 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:35:39.107005    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-0d0cae-68c69b8786-qt4kh" event={"ID":"c02969f1-8ac8-4c3d-9b70-c4f5f29a6c89","Type":"ContainerStarted","Data":"0069bd7c1d83dfc1c5f2a3f8addd1735afe758ee2e66e89cbfa9343abf6ed7ab"}
Sep 09 03:35:39.107396 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:35:39.107026    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-0d0cae-68c69b8786-qt4kh"
Sep 09 03:35:39.130497 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:35:39.130444    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-0d0cae-68c69b8786-qt4kh" podStartSLOduration=2.130427991 podStartE2EDuration="2.130427991s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 03:35:37 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 03:35:39.12766477 +0000 UTC m=+1353.269150859" watchObservedRunningTime="2026-09-09 03:35:39.130427991 +0000 UTC m=+1353.271914067"
Sep 09 03:35:50.113757 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:35:50.113731    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-0d0cae-68c69b8786-qt4kh"
Sep 09 03:36:06.483666 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:36:06.483630    2576 scope.go:117] "RemoveContainer" containerID="fcb132ca92af3fd30bf19bd57cd18033767b0d95bbac801b9317f9a9bb725472"
Sep 09 03:36:06.492059 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:36:06.492038    2576 scope.go:117] "RemoveContainer" containerID="b58704fd7565a402683bde797d506e50cd49e9b9171ab7f17b98f1959ef36e10"
Sep 09 03:36:19.343959 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:36:19.343920    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-123302cc-6bc8b7858-qz8nh"]
Sep 09 03:36:19.348870 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:36:19.348850    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-123302cc-6bc8b7858-qz8nh"
Sep 09 03:36:19.351766 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:36:19.351745    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-mt-123302cc\""
Sep 09 03:36:19.358964 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:36:19.358933    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-123302cc-6bc8b7858-qz8nh"]
Sep 09 03:36:19.373932 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:36:19.373895    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-123302cc-fdfd547f7-mj7jb"]
Sep 09 03:36:19.381375 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:36:19.381349    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-123302cc-fdfd547f7-mj7jb"
Sep 09 03:36:19.385060 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:36:19.385030    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/4a4fdbc8-ba1e-421b-b29b-3843f25897f8-plugins-config-volume\") pod \"payload-pre-processing-e2e-mt-123302cc-6bc8b7858-qz8nh\" (UID: \"4a4fdbc8-ba1e-421b-b29b-3843f25897f8\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-123302cc-6bc8b7858-qz8nh"
Sep 09 03:36:19.385216 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:36:19.385109    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dfwx9\" (UniqueName: \"kubernetes.io/projected/4a4fdbc8-ba1e-421b-b29b-3843f25897f8-kube-api-access-dfwx9\") pod \"payload-pre-processing-e2e-mt-123302cc-6bc8b7858-qz8nh\" (UID: \"4a4fdbc8-ba1e-421b-b29b-3843f25897f8\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-123302cc-6bc8b7858-qz8nh"
Sep 09 03:36:19.386349 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:36:19.386323    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-mt-123302cc-dockercfg-mjv6b\""
Sep 09 03:36:19.390122 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:36:19.390099    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-123302cc-fdfd547f7-mj7jb"]
Sep 09 03:36:19.486067 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:36:19.486027    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2s8xk\" (UniqueName: \"kubernetes.io/projected/b5d4faeb-7110-4d13-8663-259ca0709e3c-kube-api-access-2s8xk\") pod \"payload-processing-e2e-mt-123302cc-fdfd547f7-mj7jb\" (UID: \"b5d4faeb-7110-4d13-8663-259ca0709e3c\") " pod="openshift-ingress/payload-processing-e2e-mt-123302cc-fdfd547f7-mj7jb"
Sep 09 03:36:19.486247 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:36:19.486090    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/4a4fdbc8-ba1e-421b-b29b-3843f25897f8-plugins-config-volume\") pod \"payload-pre-processing-e2e-mt-123302cc-6bc8b7858-qz8nh\" (UID: \"4a4fdbc8-ba1e-421b-b29b-3843f25897f8\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-123302cc-6bc8b7858-qz8nh"
Sep 09 03:36:19.486247 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:36:19.486199    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dfwx9\" (UniqueName: \"kubernetes.io/projected/4a4fdbc8-ba1e-421b-b29b-3843f25897f8-kube-api-access-dfwx9\") pod \"payload-pre-processing-e2e-mt-123302cc-6bc8b7858-qz8nh\" (UID: \"4a4fdbc8-ba1e-421b-b29b-3843f25897f8\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-123302cc-6bc8b7858-qz8nh"
Sep 09 03:36:19.486360 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:36:19.486276    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b5d4faeb-7110-4d13-8663-259ca0709e3c-plugins-config-volume\") pod \"payload-processing-e2e-mt-123302cc-fdfd547f7-mj7jb\" (UID: \"b5d4faeb-7110-4d13-8663-259ca0709e3c\") " pod="openshift-ingress/payload-processing-e2e-mt-123302cc-fdfd547f7-mj7jb"
Sep 09 03:36:19.487195 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:36:19.487171    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/4a4fdbc8-ba1e-421b-b29b-3843f25897f8-plugins-config-volume\") pod \"payload-pre-processing-e2e-mt-123302cc-6bc8b7858-qz8nh\" (UID: \"4a4fdbc8-ba1e-421b-b29b-3843f25897f8\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-123302cc-6bc8b7858-qz8nh"
Sep 09 03:36:19.495762 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:36:19.495728    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dfwx9\" (UniqueName: \"kubernetes.io/projected/4a4fdbc8-ba1e-421b-b29b-3843f25897f8-kube-api-access-dfwx9\") pod \"payload-pre-processing-e2e-mt-123302cc-6bc8b7858-qz8nh\" (UID: \"4a4fdbc8-ba1e-421b-b29b-3843f25897f8\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-123302cc-6bc8b7858-qz8nh"
Sep 09 03:36:19.586775 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:36:19.586741    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b5d4faeb-7110-4d13-8663-259ca0709e3c-plugins-config-volume\") pod \"payload-processing-e2e-mt-123302cc-fdfd547f7-mj7jb\" (UID: \"b5d4faeb-7110-4d13-8663-259ca0709e3c\") " pod="openshift-ingress/payload-processing-e2e-mt-123302cc-fdfd547f7-mj7jb"
Sep 09 03:36:19.586775 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:36:19.586785    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2s8xk\" (UniqueName: \"kubernetes.io/projected/b5d4faeb-7110-4d13-8663-259ca0709e3c-kube-api-access-2s8xk\") pod \"payload-processing-e2e-mt-123302cc-fdfd547f7-mj7jb\" (UID: \"b5d4faeb-7110-4d13-8663-259ca0709e3c\") " pod="openshift-ingress/payload-processing-e2e-mt-123302cc-fdfd547f7-mj7jb"
Sep 09 03:36:19.587531 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:36:19.587505    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b5d4faeb-7110-4d13-8663-259ca0709e3c-plugins-config-volume\") pod \"payload-processing-e2e-mt-123302cc-fdfd547f7-mj7jb\" (UID: \"b5d4faeb-7110-4d13-8663-259ca0709e3c\") " pod="openshift-ingress/payload-processing-e2e-mt-123302cc-fdfd547f7-mj7jb"
Sep 09 03:36:19.595973 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:36:19.595910    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2s8xk\" (UniqueName: \"kubernetes.io/projected/b5d4faeb-7110-4d13-8663-259ca0709e3c-kube-api-access-2s8xk\") pod \"payload-processing-e2e-mt-123302cc-fdfd547f7-mj7jb\" (UID: \"b5d4faeb-7110-4d13-8663-259ca0709e3c\") " pod="openshift-ingress/payload-processing-e2e-mt-123302cc-fdfd547f7-mj7jb"
Sep 09 03:36:19.661748 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:36:19.661711    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-123302cc-6bc8b7858-qz8nh"
Sep 09 03:36:19.695878 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:36:19.695840    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-123302cc-fdfd547f7-mj7jb"
Sep 09 03:36:19.939710 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:36:19.939686    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-123302cc-6bc8b7858-qz8nh"]
Sep 09 03:36:19.941380 ip-10-0-132-14 kubenswrapper[2576]: W0909 03:36:19.941339    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4a4fdbc8_ba1e_421b_b29b_3843f25897f8.slice/crio-3307f470fd8e61f6bd0ee792e0552934a2a9c4e3c8fff94fa1d0be472fe5a06f WatchSource:0}: Error finding container 3307f470fd8e61f6bd0ee792e0552934a2a9c4e3c8fff94fa1d0be472fe5a06f: Status 404 returned error can't find the container with id 3307f470fd8e61f6bd0ee792e0552934a2a9c4e3c8fff94fa1d0be472fe5a06f
Sep 09 03:36:19.976861 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:36:19.976837    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-123302cc-fdfd547f7-mj7jb"]
Sep 09 03:36:19.978584 ip-10-0-132-14 kubenswrapper[2576]: W0909 03:36:19.978561    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb5d4faeb_7110_4d13_8663_259ca0709e3c.slice/crio-853722170c3ce260a063742d72c6ed42f4b774587db7db36b96d1dfda74a5292 WatchSource:0}: Error finding container 853722170c3ce260a063742d72c6ed42f4b774587db7db36b96d1dfda74a5292: Status 404 returned error can't find the container with id 853722170c3ce260a063742d72c6ed42f4b774587db7db36b96d1dfda74a5292
Sep 09 03:36:20.308402 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:36:20.308368    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-123302cc-fdfd547f7-mj7jb" event={"ID":"b5d4faeb-7110-4d13-8663-259ca0709e3c","Type":"ContainerStarted","Data":"23e0fadcc8a2d544407effa7d58fa9f8d0f52ccf7785e3c6016df7b20218a807"}
Sep 09 03:36:20.308603 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:36:20.308426    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-123302cc-fdfd547f7-mj7jb" event={"ID":"b5d4faeb-7110-4d13-8663-259ca0709e3c","Type":"ContainerStarted","Data":"853722170c3ce260a063742d72c6ed42f4b774587db7db36b96d1dfda74a5292"}
Sep 09 03:36:20.308603 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:36:20.308493    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-mt-123302cc-fdfd547f7-mj7jb"
Sep 09 03:36:20.310138 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:36:20.310112    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-123302cc-6bc8b7858-qz8nh" event={"ID":"4a4fdbc8-ba1e-421b-b29b-3843f25897f8","Type":"ContainerStarted","Data":"d3175663992aca2b200dd41a0972c01a73ce74f3ad84f99281b74ee4cb0f98dc"}
Sep 09 03:36:20.310283 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:36:20.310160    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-mt-123302cc-6bc8b7858-qz8nh"
Sep 09 03:36:20.310283 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:36:20.310187    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-123302cc-6bc8b7858-qz8nh" event={"ID":"4a4fdbc8-ba1e-421b-b29b-3843f25897f8","Type":"ContainerStarted","Data":"3307f470fd8e61f6bd0ee792e0552934a2a9c4e3c8fff94fa1d0be472fe5a06f"}
Sep 09 03:36:20.328584 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:36:20.328540    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-mt-123302cc-fdfd547f7-mj7jb" podStartSLOduration=1.328524747 podStartE2EDuration="1.328524747s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 03:36:19 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 03:36:20.327762609 +0000 UTC m=+1394.469248687" watchObservedRunningTime="2026-09-09 03:36:20.328524747 +0000 UTC m=+1394.470010824"
Sep 09 03:36:20.346049 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:36:20.346004    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-mt-123302cc-6bc8b7858-qz8nh" podStartSLOduration=1.345991208 podStartE2EDuration="1.345991208s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 03:36:19 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 03:36:20.344583769 +0000 UTC m=+1394.486069846" watchObservedRunningTime="2026-09-09 03:36:20.345991208 +0000 UTC m=+1394.487477285"
Sep 09 03:36:31.317427 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:36:31.317397    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-mt-123302cc-6bc8b7858-qz8nh"
Sep 09 03:36:31.317900 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:36:31.317476    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-mt-123302cc-fdfd547f7-mj7jb"
Sep 09 03:37:06.515590 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:37:06.515560    2576 scope.go:117] "RemoveContainer" containerID="8e49b3437f0552d5e9efedb83a853777c8f8964bb5545c0a02d5089e21179855"
Sep 09 03:37:21.449365 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:37:21.449326    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-123302cc-brrtt"]
Sep 09 03:37:21.454272 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:37:21.454251    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-123302cc-brrtt"
Sep 09 03:37:21.457187 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:37:21.457167    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-w7q9f\""
Sep 09 03:37:21.461393 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:37:21.461369    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-123302cc-brrtt"]
Sep 09 03:37:21.520429 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:37:21.520406    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/d9f91ece-9767-4e74-819b-0ee09be74b85-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-123302cc-brrtt\" (UID: \"d9f91ece-9767-4e74-819b-0ee09be74b85\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-123302cc-brrtt"
Sep 09 03:37:21.621514 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:37:21.621486    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/d9f91ece-9767-4e74-819b-0ee09be74b85-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-123302cc-brrtt\" (UID: \"d9f91ece-9767-4e74-819b-0ee09be74b85\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-123302cc-brrtt"
Sep 09 03:37:21.622071 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:37:21.622052    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/d9f91ece-9767-4e74-819b-0ee09be74b85-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-123302cc-brrtt\" (UID: \"d9f91ece-9767-4e74-819b-0ee09be74b85\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-123302cc-brrtt"
Sep 09 03:37:21.767890 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:37:21.767865    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-123302cc-brrtt"
Sep 09 03:37:22.040573 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:37:22.040539    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-123302cc-brrtt"]
Sep 09 03:37:22.042017 ip-10-0-132-14 kubenswrapper[2576]: W0909 03:37:22.041988    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd9f91ece_9767_4e74_819b_0ee09be74b85.slice/crio-d0179d4003acaef02bd5acb7ec4310e3dacb78b14e197b6823bdb300ca513e0a WatchSource:0}: Error finding container d0179d4003acaef02bd5acb7ec4310e3dacb78b14e197b6823bdb300ca513e0a: Status 404 returned error can't find the container with id d0179d4003acaef02bd5acb7ec4310e3dacb78b14e197b6823bdb300ca513e0a
Sep 09 03:37:22.625449 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:37:22.625368    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-123302cc-brrtt" event={"ID":"d9f91ece-9767-4e74-819b-0ee09be74b85","Type":"ContainerStarted","Data":"1ee8876fb7b41825f917acf15e3e23c4deb83a6140d5d52f002ac0b9ae11a473"}
Sep 09 03:37:22.625449 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:37:22.625417    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-123302cc-brrtt" event={"ID":"d9f91ece-9767-4e74-819b-0ee09be74b85","Type":"ContainerStarted","Data":"d0179d4003acaef02bd5acb7ec4310e3dacb78b14e197b6823bdb300ca513e0a"}
Sep 09 03:37:22.643782 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:37:22.643729    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-123302cc-brrtt" podStartSLOduration=1.643713304 podStartE2EDuration="1.643713304s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 03:37:21 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 03:37:22.642628538 +0000 UTC m=+1456.784114616" watchObservedRunningTime="2026-09-09 03:37:22.643713304 +0000 UTC m=+1456.785199416"
Sep 09 03:37:23.632621 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:37:23.632588    2576 generic.go:356] "Generic (PLEG): container finished" podID="d9f91ece-9767-4e74-819b-0ee09be74b85" containerID="1ee8876fb7b41825f917acf15e3e23c4deb83a6140d5d52f002ac0b9ae11a473" exitCode=0
Sep 09 03:37:23.632966 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:37:23.632629    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-123302cc-brrtt" event={"ID":"d9f91ece-9767-4e74-819b-0ee09be74b85","Type":"ContainerDied","Data":"1ee8876fb7b41825f917acf15e3e23c4deb83a6140d5d52f002ac0b9ae11a473"}
Sep 09 03:37:24.766513 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:37:24.766490    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-123302cc-brrtt"
Sep 09 03:37:24.848484 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:37:24.848460    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/d9f91ece-9767-4e74-819b-0ee09be74b85-maas-api-service-ca\") pod \"d9f91ece-9767-4e74-819b-0ee09be74b85\" (UID: \"d9f91ece-9767-4e74-819b-0ee09be74b85\") "
Sep 09 03:37:24.848821 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:37:24.848780    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d9f91ece-9767-4e74-819b-0ee09be74b85-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "d9f91ece-9767-4e74-819b-0ee09be74b85" (UID: "d9f91ece-9767-4e74-819b-0ee09be74b85"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 09 03:37:24.949214 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:37:24.949146    2576 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/d9f91ece-9767-4e74-819b-0ee09be74b85-maas-api-service-ca\") on node \"ip-10-0-132-14.ec2.internal\" DevicePath \"\""
Sep 09 03:37:25.643789 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:37:25.643750    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-123302cc-brrtt" event={"ID":"d9f91ece-9767-4e74-819b-0ee09be74b85","Type":"ContainerDied","Data":"d0179d4003acaef02bd5acb7ec4310e3dacb78b14e197b6823bdb300ca513e0a"}
Sep 09 03:37:25.643984 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:37:25.643796    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-123302cc-brrtt"
Sep 09 03:37:25.643984 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:37:25.643853    2576 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="d0179d4003acaef02bd5acb7ec4310e3dacb78b14e197b6823bdb300ca513e0a"
Sep 09 03:37:37.235259 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:37:37.235226    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-123302cc-fdfd547f7-mj7jb"]
Sep 09 03:37:37.235875 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:37:37.235417    2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-mt-123302cc-fdfd547f7-mj7jb" podUID="b5d4faeb-7110-4d13-8663-259ca0709e3c" containerName="payload-processing" containerID="cri-o://23e0fadcc8a2d544407effa7d58fa9f8d0f52ccf7785e3c6016df7b20218a807" gracePeriod=30
Sep 09 03:37:37.249630 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:37:37.249603    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-123302cc-6bc8b7858-qz8nh"]
Sep 09 03:37:37.249858 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:37:37.249788    2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-mt-123302cc-6bc8b7858-qz8nh" podUID="4a4fdbc8-ba1e-421b-b29b-3843f25897f8" containerName="payload-pre-processing" containerID="cri-o://d3175663992aca2b200dd41a0972c01a73ce74f3ad84f99281b74ee4cb0f98dc" gracePeriod=30
Sep 09 03:37:37.466478 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:37:37.466444    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-123302cc-brrtt"]
Sep 09 03:37:37.472365 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:37:37.472335    2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-123302cc-brrtt"]
Sep 09 03:37:37.499781 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:37:37.499724    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-123302cc-fdfd547f7-mj7jb"
Sep 09 03:37:37.528228 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:37:37.528206    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-123302cc-6bc8b7858-qz8nh"
Sep 09 03:37:37.650772 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:37:37.650744    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b5d4faeb-7110-4d13-8663-259ca0709e3c-plugins-config-volume\") pod \"b5d4faeb-7110-4d13-8663-259ca0709e3c\" (UID: \"b5d4faeb-7110-4d13-8663-259ca0709e3c\") "
Sep 09 03:37:37.650960 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:37:37.650798    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/4a4fdbc8-ba1e-421b-b29b-3843f25897f8-plugins-config-volume\") pod \"4a4fdbc8-ba1e-421b-b29b-3843f25897f8\" (UID: \"4a4fdbc8-ba1e-421b-b29b-3843f25897f8\") "
Sep 09 03:37:37.650960 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:37:37.650854    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-dfwx9\" (UniqueName: \"kubernetes.io/projected/4a4fdbc8-ba1e-421b-b29b-3843f25897f8-kube-api-access-dfwx9\") pod \"4a4fdbc8-ba1e-421b-b29b-3843f25897f8\" (UID: \"4a4fdbc8-ba1e-421b-b29b-3843f25897f8\") "
Sep 09 03:37:37.650960 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:37:37.650896    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-2s8xk\" (UniqueName: \"kubernetes.io/projected/b5d4faeb-7110-4d13-8663-259ca0709e3c-kube-api-access-2s8xk\") pod \"b5d4faeb-7110-4d13-8663-259ca0709e3c\" (UID: \"b5d4faeb-7110-4d13-8663-259ca0709e3c\") "
Sep 09 03:37:37.651197 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:37:37.651172    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b5d4faeb-7110-4d13-8663-259ca0709e3c-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "b5d4faeb-7110-4d13-8663-259ca0709e3c" (UID: "b5d4faeb-7110-4d13-8663-259ca0709e3c"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 09 03:37:37.651237 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:37:37.651195    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4a4fdbc8-ba1e-421b-b29b-3843f25897f8-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "4a4fdbc8-ba1e-421b-b29b-3843f25897f8" (UID: "4a4fdbc8-ba1e-421b-b29b-3843f25897f8"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 09 03:37:37.653132 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:37:37.653111    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4a4fdbc8-ba1e-421b-b29b-3843f25897f8-kube-api-access-dfwx9" (OuterVolumeSpecName: "kube-api-access-dfwx9") pod "4a4fdbc8-ba1e-421b-b29b-3843f25897f8" (UID: "4a4fdbc8-ba1e-421b-b29b-3843f25897f8"). InnerVolumeSpecName "kube-api-access-dfwx9". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 09 03:37:37.653241 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:37:37.653220    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b5d4faeb-7110-4d13-8663-259ca0709e3c-kube-api-access-2s8xk" (OuterVolumeSpecName: "kube-api-access-2s8xk") pod "b5d4faeb-7110-4d13-8663-259ca0709e3c" (UID: "b5d4faeb-7110-4d13-8663-259ca0709e3c"). InnerVolumeSpecName "kube-api-access-2s8xk". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 09 03:37:37.701965 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:37:37.701932    2576 generic.go:356] "Generic (PLEG): container finished" podID="b5d4faeb-7110-4d13-8663-259ca0709e3c" containerID="23e0fadcc8a2d544407effa7d58fa9f8d0f52ccf7785e3c6016df7b20218a807" exitCode=0
Sep 09 03:37:37.702143 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:37:37.701980    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-123302cc-fdfd547f7-mj7jb"
Sep 09 03:37:37.702143 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:37:37.702009    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-123302cc-fdfd547f7-mj7jb" event={"ID":"b5d4faeb-7110-4d13-8663-259ca0709e3c","Type":"ContainerDied","Data":"23e0fadcc8a2d544407effa7d58fa9f8d0f52ccf7785e3c6016df7b20218a807"}
Sep 09 03:37:37.702143 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:37:37.702093    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-123302cc-fdfd547f7-mj7jb" event={"ID":"b5d4faeb-7110-4d13-8663-259ca0709e3c","Type":"ContainerDied","Data":"853722170c3ce260a063742d72c6ed42f4b774587db7db36b96d1dfda74a5292"}
Sep 09 03:37:37.702143 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:37:37.702126    2576 scope.go:117] "RemoveContainer" containerID="23e0fadcc8a2d544407effa7d58fa9f8d0f52ccf7785e3c6016df7b20218a807"
Sep 09 03:37:37.703393 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:37:37.703369    2576 generic.go:356] "Generic (PLEG): container finished" podID="4a4fdbc8-ba1e-421b-b29b-3843f25897f8" containerID="d3175663992aca2b200dd41a0972c01a73ce74f3ad84f99281b74ee4cb0f98dc" exitCode=0
Sep 09 03:37:37.703508 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:37:37.703410    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-123302cc-6bc8b7858-qz8nh"
Sep 09 03:37:37.703508 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:37:37.703448    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-123302cc-6bc8b7858-qz8nh" event={"ID":"4a4fdbc8-ba1e-421b-b29b-3843f25897f8","Type":"ContainerDied","Data":"d3175663992aca2b200dd41a0972c01a73ce74f3ad84f99281b74ee4cb0f98dc"}
Sep 09 03:37:37.703508 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:37:37.703497    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-123302cc-6bc8b7858-qz8nh" event={"ID":"4a4fdbc8-ba1e-421b-b29b-3843f25897f8","Type":"ContainerDied","Data":"3307f470fd8e61f6bd0ee792e0552934a2a9c4e3c8fff94fa1d0be472fe5a06f"}
Sep 09 03:37:37.711252 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:37:37.711231    2576 scope.go:117] "RemoveContainer" containerID="23e0fadcc8a2d544407effa7d58fa9f8d0f52ccf7785e3c6016df7b20218a807"
Sep 09 03:37:37.711555 ip-10-0-132-14 kubenswrapper[2576]: E0909 03:37:37.711536    2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"23e0fadcc8a2d544407effa7d58fa9f8d0f52ccf7785e3c6016df7b20218a807\": container with ID starting with 23e0fadcc8a2d544407effa7d58fa9f8d0f52ccf7785e3c6016df7b20218a807 not found: ID does not exist" containerID="23e0fadcc8a2d544407effa7d58fa9f8d0f52ccf7785e3c6016df7b20218a807"
Sep 09 03:37:37.711613 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:37:37.711563    2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"23e0fadcc8a2d544407effa7d58fa9f8d0f52ccf7785e3c6016df7b20218a807"} err="failed to get container status \"23e0fadcc8a2d544407effa7d58fa9f8d0f52ccf7785e3c6016df7b20218a807\": rpc error: code = NotFound desc = could not find container \"23e0fadcc8a2d544407effa7d58fa9f8d0f52ccf7785e3c6016df7b20218a807\": container with ID starting with 23e0fadcc8a2d544407effa7d58fa9f8d0f52ccf7785e3c6016df7b20218a807 not found: ID does not exist"
Sep 09 03:37:37.711613 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:37:37.711581    2576 scope.go:117] "RemoveContainer" containerID="d3175663992aca2b200dd41a0972c01a73ce74f3ad84f99281b74ee4cb0f98dc"
Sep 09 03:37:37.720247 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:37:37.720231    2576 scope.go:117] "RemoveContainer" containerID="d3175663992aca2b200dd41a0972c01a73ce74f3ad84f99281b74ee4cb0f98dc"
Sep 09 03:37:37.720558 ip-10-0-132-14 kubenswrapper[2576]: E0909 03:37:37.720535    2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d3175663992aca2b200dd41a0972c01a73ce74f3ad84f99281b74ee4cb0f98dc\": container with ID starting with d3175663992aca2b200dd41a0972c01a73ce74f3ad84f99281b74ee4cb0f98dc not found: ID does not exist" containerID="d3175663992aca2b200dd41a0972c01a73ce74f3ad84f99281b74ee4cb0f98dc"
Sep 09 03:37:37.720619 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:37:37.720570    2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d3175663992aca2b200dd41a0972c01a73ce74f3ad84f99281b74ee4cb0f98dc"} err="failed to get container status \"d3175663992aca2b200dd41a0972c01a73ce74f3ad84f99281b74ee4cb0f98dc\": rpc error: code = NotFound desc = could not find container \"d3175663992aca2b200dd41a0972c01a73ce74f3ad84f99281b74ee4cb0f98dc\": container with ID starting with d3175663992aca2b200dd41a0972c01a73ce74f3ad84f99281b74ee4cb0f98dc not found: ID does not exist"
Sep 09 03:37:37.741138 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:37:37.741115    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-123302cc-fdfd547f7-mj7jb"]
Sep 09 03:37:37.751572 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:37:37.751553    2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-123302cc-fdfd547f7-mj7jb"]
Sep 09 03:37:37.751902 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:37:37.751886    2576 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/4a4fdbc8-ba1e-421b-b29b-3843f25897f8-plugins-config-volume\") on node \"ip-10-0-132-14.ec2.internal\" DevicePath \"\""
Sep 09 03:37:37.751959 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:37:37.751905    2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-dfwx9\" (UniqueName: \"kubernetes.io/projected/4a4fdbc8-ba1e-421b-b29b-3843f25897f8-kube-api-access-dfwx9\") on node \"ip-10-0-132-14.ec2.internal\" DevicePath \"\""
Sep 09 03:37:37.751959 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:37:37.751936    2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2s8xk\" (UniqueName: \"kubernetes.io/projected/b5d4faeb-7110-4d13-8663-259ca0709e3c-kube-api-access-2s8xk\") on node \"ip-10-0-132-14.ec2.internal\" DevicePath \"\""
Sep 09 03:37:37.751959 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:37:37.751945    2576 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b5d4faeb-7110-4d13-8663-259ca0709e3c-plugins-config-volume\") on node \"ip-10-0-132-14.ec2.internal\" DevicePath \"\""
Sep 09 03:37:37.764613 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:37:37.764585    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-123302cc-6bc8b7858-qz8nh"]
Sep 09 03:37:37.769636 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:37:37.769614    2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-123302cc-6bc8b7858-qz8nh"]
Sep 09 03:37:38.360448 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:37:38.360418    2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4a4fdbc8-ba1e-421b-b29b-3843f25897f8" path="/var/lib/kubelet/pods/4a4fdbc8-ba1e-421b-b29b-3843f25897f8/volumes"
Sep 09 03:37:38.360796 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:37:38.360740    2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b5d4faeb-7110-4d13-8663-259ca0709e3c" path="/var/lib/kubelet/pods/b5d4faeb-7110-4d13-8663-259ca0709e3c/volumes"
Sep 09 03:37:38.361050 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:37:38.361036    2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d9f91ece-9767-4e74-819b-0ee09be74b85" path="/var/lib/kubelet/pods/d9f91ece-9767-4e74-819b-0ee09be74b85/volumes"
Sep 09 03:38:06.533961 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:38:06.533929    2576 scope.go:117] "RemoveContainer" containerID="df24c0a849c5b86da97b1db4a054ea4072d10f94d330cbf789c607a73b921196"
Sep 09 03:39:06.560625 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:39:06.560594    2576 scope.go:117] "RemoveContainer" containerID="a6aca5da470c3be59442068b14ae9b1e3a884769ce75c312212ab0c6edf2f5b1"
Sep 09 03:39:42.996878 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:39:42.996848    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-a-7f0891-664746d6d6-5x8s5"]
Sep 09 03:39:42.997372 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:39:42.997205    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d9f91ece-9767-4e74-819b-0ee09be74b85" containerName="revoke-keys"
Sep 09 03:39:42.997372 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:39:42.997216    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="d9f91ece-9767-4e74-819b-0ee09be74b85" containerName="revoke-keys"
Sep 09 03:39:42.997372 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:39:42.997224    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4a4fdbc8-ba1e-421b-b29b-3843f25897f8" containerName="payload-pre-processing"
Sep 09 03:39:42.997372 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:39:42.997229    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="4a4fdbc8-ba1e-421b-b29b-3843f25897f8" containerName="payload-pre-processing"
Sep 09 03:39:42.997372 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:39:42.997237    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b5d4faeb-7110-4d13-8663-259ca0709e3c" containerName="payload-processing"
Sep 09 03:39:42.997372 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:39:42.997243    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="b5d4faeb-7110-4d13-8663-259ca0709e3c" containerName="payload-processing"
Sep 09 03:39:42.997372 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:39:42.997307    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="b5d4faeb-7110-4d13-8663-259ca0709e3c" containerName="payload-processing"
Sep 09 03:39:42.997372 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:39:42.997316    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="d9f91ece-9767-4e74-819b-0ee09be74b85" containerName="revoke-keys"
Sep 09 03:39:42.997372 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:39:42.997322    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="4a4fdbc8-ba1e-421b-b29b-3843f25897f8" containerName="payload-pre-processing"
Sep 09 03:39:43.000220 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:39:43.000204    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-a-7f0891-664746d6d6-5x8s5"
Sep 09 03:39:43.003678 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:39:43.003657    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-api-a-7f0891\""
Sep 09 03:39:43.012491 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:39:43.012462    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-a-7f0891-664746d6d6-5x8s5"]
Sep 09 03:39:43.017431 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:39:43.017408    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-7f0891-78db4bf9b9-7qtcm"]
Sep 09 03:39:43.020739 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:39:43.020719    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-a-7f0891-78db4bf9b9-7qtcm"
Sep 09 03:39:43.023507 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:39:43.023488    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-api-a-7f0891-dockercfg-6h46s\""
Sep 09 03:39:43.030755 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:39:43.030732    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-7f0891-78db4bf9b9-7qtcm"]
Sep 09 03:39:43.115679 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:39:43.115650    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/00b25663-dc21-46b5-8cc9-f18c09f1f48b-plugins-config-volume\") pod \"payload-processing-e2e-api-a-7f0891-78db4bf9b9-7qtcm\" (UID: \"00b25663-dc21-46b5-8cc9-f18c09f1f48b\") " pod="openshift-ingress/payload-processing-e2e-api-a-7f0891-78db4bf9b9-7qtcm"
Sep 09 03:39:43.115869 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:39:43.115688    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/15e8a7c1-d772-485a-b7ed-bdcb3f8f3a4f-plugins-config-volume\") pod \"payload-pre-processing-e2e-api-a-7f0891-664746d6d6-5x8s5\" (UID: \"15e8a7c1-d772-485a-b7ed-bdcb3f8f3a4f\") " pod="openshift-ingress/payload-pre-processing-e2e-api-a-7f0891-664746d6d6-5x8s5"
Sep 09 03:39:43.115869 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:39:43.115745    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-q8cl8\" (UniqueName: \"kubernetes.io/projected/00b25663-dc21-46b5-8cc9-f18c09f1f48b-kube-api-access-q8cl8\") pod \"payload-processing-e2e-api-a-7f0891-78db4bf9b9-7qtcm\" (UID: \"00b25663-dc21-46b5-8cc9-f18c09f1f48b\") " pod="openshift-ingress/payload-processing-e2e-api-a-7f0891-78db4bf9b9-7qtcm"
Sep 09 03:39:43.115869 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:39:43.115774    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-z9pgq\" (UniqueName: \"kubernetes.io/projected/15e8a7c1-d772-485a-b7ed-bdcb3f8f3a4f-kube-api-access-z9pgq\") pod \"payload-pre-processing-e2e-api-a-7f0891-664746d6d6-5x8s5\" (UID: \"15e8a7c1-d772-485a-b7ed-bdcb3f8f3a4f\") " pod="openshift-ingress/payload-pre-processing-e2e-api-a-7f0891-664746d6d6-5x8s5"
Sep 09 03:39:43.216790 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:39:43.216751    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-q8cl8\" (UniqueName: \"kubernetes.io/projected/00b25663-dc21-46b5-8cc9-f18c09f1f48b-kube-api-access-q8cl8\") pod \"payload-processing-e2e-api-a-7f0891-78db4bf9b9-7qtcm\" (UID: \"00b25663-dc21-46b5-8cc9-f18c09f1f48b\") " pod="openshift-ingress/payload-processing-e2e-api-a-7f0891-78db4bf9b9-7qtcm"
Sep 09 03:39:43.216790 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:39:43.216787    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-z9pgq\" (UniqueName: \"kubernetes.io/projected/15e8a7c1-d772-485a-b7ed-bdcb3f8f3a4f-kube-api-access-z9pgq\") pod \"payload-pre-processing-e2e-api-a-7f0891-664746d6d6-5x8s5\" (UID: \"15e8a7c1-d772-485a-b7ed-bdcb3f8f3a4f\") " pod="openshift-ingress/payload-pre-processing-e2e-api-a-7f0891-664746d6d6-5x8s5"
Sep 09 03:39:43.217010 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:39:43.216879    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/00b25663-dc21-46b5-8cc9-f18c09f1f48b-plugins-config-volume\") pod \"payload-processing-e2e-api-a-7f0891-78db4bf9b9-7qtcm\" (UID: \"00b25663-dc21-46b5-8cc9-f18c09f1f48b\") " pod="openshift-ingress/payload-processing-e2e-api-a-7f0891-78db4bf9b9-7qtcm"
Sep 09 03:39:43.217010 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:39:43.216920    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/15e8a7c1-d772-485a-b7ed-bdcb3f8f3a4f-plugins-config-volume\") pod \"payload-pre-processing-e2e-api-a-7f0891-664746d6d6-5x8s5\" (UID: \"15e8a7c1-d772-485a-b7ed-bdcb3f8f3a4f\") " pod="openshift-ingress/payload-pre-processing-e2e-api-a-7f0891-664746d6d6-5x8s5"
Sep 09 03:39:43.217659 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:39:43.217640    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/15e8a7c1-d772-485a-b7ed-bdcb3f8f3a4f-plugins-config-volume\") pod \"payload-pre-processing-e2e-api-a-7f0891-664746d6d6-5x8s5\" (UID: \"15e8a7c1-d772-485a-b7ed-bdcb3f8f3a4f\") " pod="openshift-ingress/payload-pre-processing-e2e-api-a-7f0891-664746d6d6-5x8s5"
Sep 09 03:39:43.217697 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:39:43.217644    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/00b25663-dc21-46b5-8cc9-f18c09f1f48b-plugins-config-volume\") pod \"payload-processing-e2e-api-a-7f0891-78db4bf9b9-7qtcm\" (UID: \"00b25663-dc21-46b5-8cc9-f18c09f1f48b\") " pod="openshift-ingress/payload-processing-e2e-api-a-7f0891-78db4bf9b9-7qtcm"
Sep 09 03:39:43.226056 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:39:43.226029    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-q8cl8\" (UniqueName: \"kubernetes.io/projected/00b25663-dc21-46b5-8cc9-f18c09f1f48b-kube-api-access-q8cl8\") pod \"payload-processing-e2e-api-a-7f0891-78db4bf9b9-7qtcm\" (UID: \"00b25663-dc21-46b5-8cc9-f18c09f1f48b\") " pod="openshift-ingress/payload-processing-e2e-api-a-7f0891-78db4bf9b9-7qtcm"
Sep 09 03:39:43.226213 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:39:43.226192    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-z9pgq\" (UniqueName: \"kubernetes.io/projected/15e8a7c1-d772-485a-b7ed-bdcb3f8f3a4f-kube-api-access-z9pgq\") pod \"payload-pre-processing-e2e-api-a-7f0891-664746d6d6-5x8s5\" (UID: \"15e8a7c1-d772-485a-b7ed-bdcb3f8f3a4f\") " pod="openshift-ingress/payload-pre-processing-e2e-api-a-7f0891-664746d6d6-5x8s5"
Sep 09 03:39:43.310976 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:39:43.310945    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-a-7f0891-664746d6d6-5x8s5"
Sep 09 03:39:43.332946 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:39:43.332921    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-a-7f0891-78db4bf9b9-7qtcm"
Sep 09 03:39:43.589208 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:39:43.589181    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-a-7f0891-664746d6d6-5x8s5"]
Sep 09 03:39:43.591334 ip-10-0-132-14 kubenswrapper[2576]: W0909 03:39:43.591309    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod15e8a7c1_d772_485a_b7ed_bdcb3f8f3a4f.slice/crio-8393df11bdf4bda2063b99bbbe07da907bf274f051da33705f9ddef2aaffee17 WatchSource:0}: Error finding container 8393df11bdf4bda2063b99bbbe07da907bf274f051da33705f9ddef2aaffee17: Status 404 returned error can't find the container with id 8393df11bdf4bda2063b99bbbe07da907bf274f051da33705f9ddef2aaffee17
Sep 09 03:39:43.625520 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:39:43.625498    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-7f0891-78db4bf9b9-7qtcm"]
Sep 09 03:39:43.626880 ip-10-0-132-14 kubenswrapper[2576]: W0909 03:39:43.626834    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod00b25663_dc21_46b5_8cc9_f18c09f1f48b.slice/crio-e5dbfd94fb15176995fd26ef5eb067ae38d2886579adb6b87cab387046987a6b WatchSource:0}: Error finding container e5dbfd94fb15176995fd26ef5eb067ae38d2886579adb6b87cab387046987a6b: Status 404 returned error can't find the container with id e5dbfd94fb15176995fd26ef5eb067ae38d2886579adb6b87cab387046987a6b
Sep 09 03:39:44.319276 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:39:44.319233    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-a-7f0891-664746d6d6-5x8s5" event={"ID":"15e8a7c1-d772-485a-b7ed-bdcb3f8f3a4f","Type":"ContainerStarted","Data":"6cc85203ee87029baf489a3ecf3d43dbddba06abaf733e1b619dfd4d628069cf"}
Sep 09 03:39:44.319742 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:39:44.319322    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-a-7f0891-664746d6d6-5x8s5" event={"ID":"15e8a7c1-d772-485a-b7ed-bdcb3f8f3a4f","Type":"ContainerStarted","Data":"8393df11bdf4bda2063b99bbbe07da907bf274f051da33705f9ddef2aaffee17"}
Sep 09 03:39:44.319742 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:39:44.319352    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-api-a-7f0891-664746d6d6-5x8s5"
Sep 09 03:39:44.320927 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:39:44.320904    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-a-7f0891-78db4bf9b9-7qtcm" event={"ID":"00b25663-dc21-46b5-8cc9-f18c09f1f48b","Type":"ContainerStarted","Data":"1153cbcd59207313685f699ec35502345bf35742fdc87102a81dd13eaf251da0"}
Sep 09 03:39:44.321042 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:39:44.320940    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-a-7f0891-78db4bf9b9-7qtcm" event={"ID":"00b25663-dc21-46b5-8cc9-f18c09f1f48b","Type":"ContainerStarted","Data":"e5dbfd94fb15176995fd26ef5eb067ae38d2886579adb6b87cab387046987a6b"}
Sep 09 03:39:44.321042 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:39:44.320978    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-api-a-7f0891-78db4bf9b9-7qtcm"
Sep 09 03:39:44.338270 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:39:44.338230    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-api-a-7f0891-664746d6d6-5x8s5" podStartSLOduration=2.338217478 podStartE2EDuration="2.338217478s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 03:39:42 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 03:39:44.336499601 +0000 UTC m=+1598.477985680" watchObservedRunningTime="2026-09-09 03:39:44.338217478 +0000 UTC m=+1598.479703553"
Sep 09 03:39:44.356123 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:39:44.356079    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-api-a-7f0891-78db4bf9b9-7qtcm" podStartSLOduration=2.356061958 podStartE2EDuration="2.356061958s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 03:39:42 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 03:39:44.35464895 +0000 UTC m=+1598.496135026" watchObservedRunningTime="2026-09-09 03:39:44.356061958 +0000 UTC m=+1598.497548037"
Sep 09 03:39:55.326969 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:39:55.326937    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-api-a-7f0891-78db4bf9b9-7qtcm"
Sep 09 03:39:55.327574 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:39:55.327023    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-api-a-7f0891-664746d6d6-5x8s5"
Sep 09 03:40:06.579365 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:40:06.579321    2576 scope.go:117] "RemoveContainer" containerID="2dd773dd7e7df95160d10800012c05d6b9c603a3c05aff67e12ef73aa1ab0d28"
Sep 09 03:40:06.587767 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:40:06.587737    2576 scope.go:117] "RemoveContainer" containerID="9fbb9eb408510f1b36d25704266cc94999b76154778616b65dfa2d94786f900f"
Sep 09 03:40:12.196318 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:40:12.196286    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-b-1c4cb1-66dcd988c8-8zx2f"]
Sep 09 03:40:12.200989 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:40:12.200964    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-b-1c4cb1-66dcd988c8-8zx2f"
Sep 09 03:40:12.203696 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:40:12.203677    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-api-b-1c4cb1\""
Sep 09 03:40:12.212509 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:40:12.212488    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-b-1c4cb1-66dcd988c8-8zx2f"]
Sep 09 03:40:12.217445 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:40:12.217423    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-api-b-1c4cb1-f4c95df69-z68mp"]
Sep 09 03:40:12.220639 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:40:12.220623    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-b-1c4cb1-f4c95df69-z68mp"
Sep 09 03:40:12.223554 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:40:12.223537    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-api-b-1c4cb1-dockercfg-6j97c\""
Sep 09 03:40:12.233267 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:40:12.233243    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-b-1c4cb1-f4c95df69-z68mp"]
Sep 09 03:40:12.369463 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:40:12.369431    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ed42c13a-540c-4d83-8a2e-78c6268c9fb6-plugins-config-volume\") pod \"payload-pre-processing-e2e-api-b-1c4cb1-66dcd988c8-8zx2f\" (UID: \"ed42c13a-540c-4d83-8a2e-78c6268c9fb6\") " pod="openshift-ingress/payload-pre-processing-e2e-api-b-1c4cb1-66dcd988c8-8zx2f"
Sep 09 03:40:12.369628 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:40:12.369478    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/1027c0f0-a872-421c-8e5e-22d6dd4c9e14-plugins-config-volume\") pod \"payload-processing-e2e-api-b-1c4cb1-f4c95df69-z68mp\" (UID: \"1027c0f0-a872-421c-8e5e-22d6dd4c9e14\") " pod="openshift-ingress/payload-processing-e2e-api-b-1c4cb1-f4c95df69-z68mp"
Sep 09 03:40:12.369628 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:40:12.369548    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-swz6c\" (UniqueName: \"kubernetes.io/projected/ed42c13a-540c-4d83-8a2e-78c6268c9fb6-kube-api-access-swz6c\") pod \"payload-pre-processing-e2e-api-b-1c4cb1-66dcd988c8-8zx2f\" (UID: \"ed42c13a-540c-4d83-8a2e-78c6268c9fb6\") " pod="openshift-ingress/payload-pre-processing-e2e-api-b-1c4cb1-66dcd988c8-8zx2f"
Sep 09 03:40:12.369628 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:40:12.369582    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nf9j6\" (UniqueName: \"kubernetes.io/projected/1027c0f0-a872-421c-8e5e-22d6dd4c9e14-kube-api-access-nf9j6\") pod \"payload-processing-e2e-api-b-1c4cb1-f4c95df69-z68mp\" (UID: \"1027c0f0-a872-421c-8e5e-22d6dd4c9e14\") " pod="openshift-ingress/payload-processing-e2e-api-b-1c4cb1-f4c95df69-z68mp"
Sep 09 03:40:12.471016 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:40:12.470931    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nf9j6\" (UniqueName: \"kubernetes.io/projected/1027c0f0-a872-421c-8e5e-22d6dd4c9e14-kube-api-access-nf9j6\") pod \"payload-processing-e2e-api-b-1c4cb1-f4c95df69-z68mp\" (UID: \"1027c0f0-a872-421c-8e5e-22d6dd4c9e14\") " pod="openshift-ingress/payload-processing-e2e-api-b-1c4cb1-f4c95df69-z68mp"
Sep 09 03:40:12.471016 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:40:12.471009    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ed42c13a-540c-4d83-8a2e-78c6268c9fb6-plugins-config-volume\") pod \"payload-pre-processing-e2e-api-b-1c4cb1-66dcd988c8-8zx2f\" (UID: \"ed42c13a-540c-4d83-8a2e-78c6268c9fb6\") " pod="openshift-ingress/payload-pre-processing-e2e-api-b-1c4cb1-66dcd988c8-8zx2f"
Sep 09 03:40:12.471280 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:40:12.471043    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/1027c0f0-a872-421c-8e5e-22d6dd4c9e14-plugins-config-volume\") pod \"payload-processing-e2e-api-b-1c4cb1-f4c95df69-z68mp\" (UID: \"1027c0f0-a872-421c-8e5e-22d6dd4c9e14\") " pod="openshift-ingress/payload-processing-e2e-api-b-1c4cb1-f4c95df69-z68mp"
Sep 09 03:40:12.471280 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:40:12.471096    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-swz6c\" (UniqueName: \"kubernetes.io/projected/ed42c13a-540c-4d83-8a2e-78c6268c9fb6-kube-api-access-swz6c\") pod \"payload-pre-processing-e2e-api-b-1c4cb1-66dcd988c8-8zx2f\" (UID: \"ed42c13a-540c-4d83-8a2e-78c6268c9fb6\") " pod="openshift-ingress/payload-pre-processing-e2e-api-b-1c4cb1-66dcd988c8-8zx2f"
Sep 09 03:40:12.471777 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:40:12.471753    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ed42c13a-540c-4d83-8a2e-78c6268c9fb6-plugins-config-volume\") pod \"payload-pre-processing-e2e-api-b-1c4cb1-66dcd988c8-8zx2f\" (UID: \"ed42c13a-540c-4d83-8a2e-78c6268c9fb6\") " pod="openshift-ingress/payload-pre-processing-e2e-api-b-1c4cb1-66dcd988c8-8zx2f"
Sep 09 03:40:12.471905 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:40:12.471789    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/1027c0f0-a872-421c-8e5e-22d6dd4c9e14-plugins-config-volume\") pod \"payload-processing-e2e-api-b-1c4cb1-f4c95df69-z68mp\" (UID: \"1027c0f0-a872-421c-8e5e-22d6dd4c9e14\") " pod="openshift-ingress/payload-processing-e2e-api-b-1c4cb1-f4c95df69-z68mp"
Sep 09 03:40:12.480038 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:40:12.480001    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nf9j6\" (UniqueName: \"kubernetes.io/projected/1027c0f0-a872-421c-8e5e-22d6dd4c9e14-kube-api-access-nf9j6\") pod \"payload-processing-e2e-api-b-1c4cb1-f4c95df69-z68mp\" (UID: \"1027c0f0-a872-421c-8e5e-22d6dd4c9e14\") " pod="openshift-ingress/payload-processing-e2e-api-b-1c4cb1-f4c95df69-z68mp"
Sep 09 03:40:12.480132 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:40:12.480053    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-swz6c\" (UniqueName: \"kubernetes.io/projected/ed42c13a-540c-4d83-8a2e-78c6268c9fb6-kube-api-access-swz6c\") pod \"payload-pre-processing-e2e-api-b-1c4cb1-66dcd988c8-8zx2f\" (UID: \"ed42c13a-540c-4d83-8a2e-78c6268c9fb6\") " pod="openshift-ingress/payload-pre-processing-e2e-api-b-1c4cb1-66dcd988c8-8zx2f"
Sep 09 03:40:12.511327 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:40:12.511307    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-b-1c4cb1-66dcd988c8-8zx2f"
Sep 09 03:40:12.532438 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:40:12.532418    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-b-1c4cb1-f4c95df69-z68mp"
Sep 09 03:40:12.787698 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:40:12.787672    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-b-1c4cb1-66dcd988c8-8zx2f"]
Sep 09 03:40:12.789747 ip-10-0-132-14 kubenswrapper[2576]: W0909 03:40:12.789717    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poded42c13a_540c_4d83_8a2e_78c6268c9fb6.slice/crio-45d121cce517adedb38197b0172cf63e561da1fc195e8e63bd1774865b7d6324 WatchSource:0}: Error finding container 45d121cce517adedb38197b0172cf63e561da1fc195e8e63bd1774865b7d6324: Status 404 returned error can't find the container with id 45d121cce517adedb38197b0172cf63e561da1fc195e8e63bd1774865b7d6324
Sep 09 03:40:12.823016 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:40:12.822995    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-b-1c4cb1-f4c95df69-z68mp"]
Sep 09 03:40:12.825091 ip-10-0-132-14 kubenswrapper[2576]: W0909 03:40:12.825066    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1027c0f0_a872_421c_8e5e_22d6dd4c9e14.slice/crio-3cec42ac06a1e504bf8efed491da5c148c3a15b49350835ffeab06824d2d6142 WatchSource:0}: Error finding container 3cec42ac06a1e504bf8efed491da5c148c3a15b49350835ffeab06824d2d6142: Status 404 returned error can't find the container with id 3cec42ac06a1e504bf8efed491da5c148c3a15b49350835ffeab06824d2d6142
Sep 09 03:40:13.470394 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:40:13.470281    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-b-1c4cb1-f4c95df69-z68mp" event={"ID":"1027c0f0-a872-421c-8e5e-22d6dd4c9e14","Type":"ContainerStarted","Data":"e5280c1172c064a44ee8dd9cbb1e13949c360ad58f30671f15d4e194f47d7520"}
Sep 09 03:40:13.470394 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:40:13.470338    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-b-1c4cb1-f4c95df69-z68mp" event={"ID":"1027c0f0-a872-421c-8e5e-22d6dd4c9e14","Type":"ContainerStarted","Data":"3cec42ac06a1e504bf8efed491da5c148c3a15b49350835ffeab06824d2d6142"}
Sep 09 03:40:13.483123 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:40:13.470398    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-api-b-1c4cb1-f4c95df69-z68mp"
Sep 09 03:40:13.483123 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:40:13.472119    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-b-1c4cb1-66dcd988c8-8zx2f" event={"ID":"ed42c13a-540c-4d83-8a2e-78c6268c9fb6","Type":"ContainerStarted","Data":"516682c28fa7fc08c09b4e3b834026c815c3eb38cc4d8d0a5f4955e469a48907"}
Sep 09 03:40:13.483123 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:40:13.472158    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-b-1c4cb1-66dcd988c8-8zx2f" event={"ID":"ed42c13a-540c-4d83-8a2e-78c6268c9fb6","Type":"ContainerStarted","Data":"45d121cce517adedb38197b0172cf63e561da1fc195e8e63bd1774865b7d6324"}
Sep 09 03:40:13.483123 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:40:13.472219    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-api-b-1c4cb1-66dcd988c8-8zx2f"
Sep 09 03:40:13.497441 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:40:13.497395    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-api-b-1c4cb1-f4c95df69-z68mp" podStartSLOduration=1.49738282 podStartE2EDuration="1.49738282s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 03:40:12 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 03:40:13.494777637 +0000 UTC m=+1627.636263714" watchObservedRunningTime="2026-09-09 03:40:13.49738282 +0000 UTC m=+1627.638868895"
Sep 09 03:40:13.517678 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:40:13.517640    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-api-b-1c4cb1-66dcd988c8-8zx2f" podStartSLOduration=1.517627686 podStartE2EDuration="1.517627686s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 03:40:12 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 03:40:13.516668364 +0000 UTC m=+1627.658154455" watchObservedRunningTime="2026-09-09 03:40:13.517627686 +0000 UTC m=+1627.659113762"
Sep 09 03:40:24.480625 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:40:24.480592    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-api-b-1c4cb1-f4c95df69-z68mp"
Sep 09 03:40:24.481018 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:40:24.480835    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-api-b-1c4cb1-66dcd988c8-8zx2f"
Sep 09 03:40:38.757061 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:40:38.757026    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-7f0891-cb6pb"]
Sep 09 03:40:38.760761 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:40:38.760737    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-7f0891-cb6pb"
Sep 09 03:40:38.763363 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:40:38.763336    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-w7q9f\""
Sep 09 03:40:38.769229 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:40:38.768693    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-7f0891-cb6pb"]
Sep 09 03:40:38.897298 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:40:38.897265    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/66030200-35ab-448d-9478-48a2103abff3-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-a-7f0891-cb6pb\" (UID: \"66030200-35ab-448d-9478-48a2103abff3\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-7f0891-cb6pb"
Sep 09 03:40:38.998811 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:40:38.998759    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/66030200-35ab-448d-9478-48a2103abff3-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-a-7f0891-cb6pb\" (UID: \"66030200-35ab-448d-9478-48a2103abff3\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-7f0891-cb6pb"
Sep 09 03:40:38.999385 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:40:38.999364    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/66030200-35ab-448d-9478-48a2103abff3-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-a-7f0891-cb6pb\" (UID: \"66030200-35ab-448d-9478-48a2103abff3\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-7f0891-cb6pb"
Sep 09 03:40:39.074149 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:40:39.074083    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-7f0891-cb6pb"
Sep 09 03:40:39.342332 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:40:39.342305    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-7f0891-cb6pb"]
Sep 09 03:40:39.344037 ip-10-0-132-14 kubenswrapper[2576]: W0909 03:40:39.344008    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod66030200_35ab_448d_9478_48a2103abff3.slice/crio-485277e1316fa8a1c78fb8fb9628be23e305e5c65fe29ec859e2bb5d6fed0dc2 WatchSource:0}: Error finding container 485277e1316fa8a1c78fb8fb9628be23e305e5c65fe29ec859e2bb5d6fed0dc2: Status 404 returned error can't find the container with id 485277e1316fa8a1c78fb8fb9628be23e305e5c65fe29ec859e2bb5d6fed0dc2
Sep 09 03:40:39.604753 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:40:39.604676    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-7f0891-cb6pb" event={"ID":"66030200-35ab-448d-9478-48a2103abff3","Type":"ContainerStarted","Data":"485277e1316fa8a1c78fb8fb9628be23e305e5c65fe29ec859e2bb5d6fed0dc2"}
Sep 09 03:40:40.611456 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:40:40.611419    2576 generic.go:356] "Generic (PLEG): container finished" podID="66030200-35ab-448d-9478-48a2103abff3" containerID="2915ae464f8a57a34371f5eec7021dc3244cd0de19f680ca953690fde5383f4d" exitCode=0
Sep 09 03:40:40.611830 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:40:40.611505    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-7f0891-cb6pb" event={"ID":"66030200-35ab-448d-9478-48a2103abff3","Type":"ContainerDied","Data":"2915ae464f8a57a34371f5eec7021dc3244cd0de19f680ca953690fde5383f4d"}
Sep 09 03:40:41.750769 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:40:41.750745    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-7f0891-cb6pb"
Sep 09 03:40:41.823255 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:40:41.823228    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/66030200-35ab-448d-9478-48a2103abff3-maas-api-service-ca\") pod \"66030200-35ab-448d-9478-48a2103abff3\" (UID: \"66030200-35ab-448d-9478-48a2103abff3\") "
Sep 09 03:40:41.823567 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:40:41.823543    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/66030200-35ab-448d-9478-48a2103abff3-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "66030200-35ab-448d-9478-48a2103abff3" (UID: "66030200-35ab-448d-9478-48a2103abff3"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 09 03:40:41.924161 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:40:41.924086    2576 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/66030200-35ab-448d-9478-48a2103abff3-maas-api-service-ca\") on node \"ip-10-0-132-14.ec2.internal\" DevicePath \"\""
Sep 09 03:40:42.622736 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:40:42.622656    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-7f0891-cb6pb"
Sep 09 03:40:42.622903 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:40:42.622654    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-7f0891-cb6pb" event={"ID":"66030200-35ab-448d-9478-48a2103abff3","Type":"ContainerDied","Data":"485277e1316fa8a1c78fb8fb9628be23e305e5c65fe29ec859e2bb5d6fed0dc2"}
Sep 09 03:40:42.622903 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:40:42.622788    2576 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="485277e1316fa8a1c78fb8fb9628be23e305e5c65fe29ec859e2bb5d6fed0dc2"
Sep 09 03:40:54.322200 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:40:54.322168    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-7f0891-78db4bf9b9-7qtcm"]
Sep 09 03:40:54.322698 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:40:54.322415    2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-api-a-7f0891-78db4bf9b9-7qtcm" podUID="00b25663-dc21-46b5-8cc9-f18c09f1f48b" containerName="payload-processing" containerID="cri-o://1153cbcd59207313685f699ec35502345bf35742fdc87102a81dd13eaf251da0" gracePeriod=30
Sep 09 03:40:54.328116 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:40:54.328071    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-a-7f0891-664746d6d6-5x8s5"]
Sep 09 03:40:54.328325 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:40:54.328294    2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-api-a-7f0891-664746d6d6-5x8s5" podUID="15e8a7c1-d772-485a-b7ed-bdcb3f8f3a4f" containerName="payload-pre-processing" containerID="cri-o://6cc85203ee87029baf489a3ecf3d43dbddba06abaf733e1b619dfd4d628069cf" gracePeriod=30
Sep 09 03:40:54.491493 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:40:54.491459    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-7f0891-cb6pb"]
Sep 09 03:40:54.495532 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:40:54.495501    2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-7f0891-cb6pb"]
Sep 09 03:40:54.575412 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:40:54.575346    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-a-7f0891-78db4bf9b9-7qtcm"
Sep 09 03:40:54.605267 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:40:54.605247    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-a-7f0891-664746d6d6-5x8s5"
Sep 09 03:40:54.690705 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:40:54.690672    2576 generic.go:356] "Generic (PLEG): container finished" podID="15e8a7c1-d772-485a-b7ed-bdcb3f8f3a4f" containerID="6cc85203ee87029baf489a3ecf3d43dbddba06abaf733e1b619dfd4d628069cf" exitCode=0
Sep 09 03:40:54.690908 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:40:54.690718    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-a-7f0891-664746d6d6-5x8s5"
Sep 09 03:40:54.690908 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:40:54.690758    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-a-7f0891-664746d6d6-5x8s5" event={"ID":"15e8a7c1-d772-485a-b7ed-bdcb3f8f3a4f","Type":"ContainerDied","Data":"6cc85203ee87029baf489a3ecf3d43dbddba06abaf733e1b619dfd4d628069cf"}
Sep 09 03:40:54.690908 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:40:54.690835    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-a-7f0891-664746d6d6-5x8s5" event={"ID":"15e8a7c1-d772-485a-b7ed-bdcb3f8f3a4f","Type":"ContainerDied","Data":"8393df11bdf4bda2063b99bbbe07da907bf274f051da33705f9ddef2aaffee17"}
Sep 09 03:40:54.690908 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:40:54.690875    2576 scope.go:117] "RemoveContainer" containerID="6cc85203ee87029baf489a3ecf3d43dbddba06abaf733e1b619dfd4d628069cf"
Sep 09 03:40:54.692111 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:40:54.692094    2576 generic.go:356] "Generic (PLEG): container finished" podID="00b25663-dc21-46b5-8cc9-f18c09f1f48b" containerID="1153cbcd59207313685f699ec35502345bf35742fdc87102a81dd13eaf251da0" exitCode=0
Sep 09 03:40:54.692184 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:40:54.692149    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-a-7f0891-78db4bf9b9-7qtcm"
Sep 09 03:40:54.692224 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:40:54.692182    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-a-7f0891-78db4bf9b9-7qtcm" event={"ID":"00b25663-dc21-46b5-8cc9-f18c09f1f48b","Type":"ContainerDied","Data":"1153cbcd59207313685f699ec35502345bf35742fdc87102a81dd13eaf251da0"}
Sep 09 03:40:54.692258 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:40:54.692237    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-a-7f0891-78db4bf9b9-7qtcm" event={"ID":"00b25663-dc21-46b5-8cc9-f18c09f1f48b","Type":"ContainerDied","Data":"e5dbfd94fb15176995fd26ef5eb067ae38d2886579adb6b87cab387046987a6b"}
Sep 09 03:40:54.699524 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:40:54.699508    2576 scope.go:117] "RemoveContainer" containerID="6cc85203ee87029baf489a3ecf3d43dbddba06abaf733e1b619dfd4d628069cf"
Sep 09 03:40:54.699790 ip-10-0-132-14 kubenswrapper[2576]: E0909 03:40:54.699770    2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6cc85203ee87029baf489a3ecf3d43dbddba06abaf733e1b619dfd4d628069cf\": container with ID starting with 6cc85203ee87029baf489a3ecf3d43dbddba06abaf733e1b619dfd4d628069cf not found: ID does not exist" containerID="6cc85203ee87029baf489a3ecf3d43dbddba06abaf733e1b619dfd4d628069cf"
Sep 09 03:40:54.699862 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:40:54.699816    2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6cc85203ee87029baf489a3ecf3d43dbddba06abaf733e1b619dfd4d628069cf"} err="failed to get container status \"6cc85203ee87029baf489a3ecf3d43dbddba06abaf733e1b619dfd4d628069cf\": rpc error: code = NotFound desc = could not find container \"6cc85203ee87029baf489a3ecf3d43dbddba06abaf733e1b619dfd4d628069cf\": container with ID starting with 6cc85203ee87029baf489a3ecf3d43dbddba06abaf733e1b619dfd4d628069cf not found: ID does not exist"
Sep 09 03:40:54.699862 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:40:54.699833    2576 scope.go:117] "RemoveContainer" containerID="1153cbcd59207313685f699ec35502345bf35742fdc87102a81dd13eaf251da0"
Sep 09 03:40:54.707512 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:40:54.707497    2576 scope.go:117] "RemoveContainer" containerID="1153cbcd59207313685f699ec35502345bf35742fdc87102a81dd13eaf251da0"
Sep 09 03:40:54.707751 ip-10-0-132-14 kubenswrapper[2576]: E0909 03:40:54.707734    2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1153cbcd59207313685f699ec35502345bf35742fdc87102a81dd13eaf251da0\": container with ID starting with 1153cbcd59207313685f699ec35502345bf35742fdc87102a81dd13eaf251da0 not found: ID does not exist" containerID="1153cbcd59207313685f699ec35502345bf35742fdc87102a81dd13eaf251da0"
Sep 09 03:40:54.707819 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:40:54.707758    2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1153cbcd59207313685f699ec35502345bf35742fdc87102a81dd13eaf251da0"} err="failed to get container status \"1153cbcd59207313685f699ec35502345bf35742fdc87102a81dd13eaf251da0\": rpc error: code = NotFound desc = could not find container \"1153cbcd59207313685f699ec35502345bf35742fdc87102a81dd13eaf251da0\": container with ID starting with 1153cbcd59207313685f699ec35502345bf35742fdc87102a81dd13eaf251da0 not found: ID does not exist"
Sep 09 03:40:54.739136 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:40:54.739112    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-z9pgq\" (UniqueName: \"kubernetes.io/projected/15e8a7c1-d772-485a-b7ed-bdcb3f8f3a4f-kube-api-access-z9pgq\") pod \"15e8a7c1-d772-485a-b7ed-bdcb3f8f3a4f\" (UID: \"15e8a7c1-d772-485a-b7ed-bdcb3f8f3a4f\") "
Sep 09 03:40:54.739248 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:40:54.739152    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/15e8a7c1-d772-485a-b7ed-bdcb3f8f3a4f-plugins-config-volume\") pod \"15e8a7c1-d772-485a-b7ed-bdcb3f8f3a4f\" (UID: \"15e8a7c1-d772-485a-b7ed-bdcb3f8f3a4f\") "
Sep 09 03:40:54.739248 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:40:54.739174    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/00b25663-dc21-46b5-8cc9-f18c09f1f48b-plugins-config-volume\") pod \"00b25663-dc21-46b5-8cc9-f18c09f1f48b\" (UID: \"00b25663-dc21-46b5-8cc9-f18c09f1f48b\") "
Sep 09 03:40:54.739248 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:40:54.739242    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-q8cl8\" (UniqueName: \"kubernetes.io/projected/00b25663-dc21-46b5-8cc9-f18c09f1f48b-kube-api-access-q8cl8\") pod \"00b25663-dc21-46b5-8cc9-f18c09f1f48b\" (UID: \"00b25663-dc21-46b5-8cc9-f18c09f1f48b\") "
Sep 09 03:40:54.739507 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:40:54.739485    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/15e8a7c1-d772-485a-b7ed-bdcb3f8f3a4f-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "15e8a7c1-d772-485a-b7ed-bdcb3f8f3a4f" (UID: "15e8a7c1-d772-485a-b7ed-bdcb3f8f3a4f"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 09 03:40:54.739603 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:40:54.739580    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/00b25663-dc21-46b5-8cc9-f18c09f1f48b-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "00b25663-dc21-46b5-8cc9-f18c09f1f48b" (UID: "00b25663-dc21-46b5-8cc9-f18c09f1f48b"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 09 03:40:54.741417 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:40:54.741397    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/15e8a7c1-d772-485a-b7ed-bdcb3f8f3a4f-kube-api-access-z9pgq" (OuterVolumeSpecName: "kube-api-access-z9pgq") pod "15e8a7c1-d772-485a-b7ed-bdcb3f8f3a4f" (UID: "15e8a7c1-d772-485a-b7ed-bdcb3f8f3a4f"). InnerVolumeSpecName "kube-api-access-z9pgq". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 09 03:40:54.741488 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:40:54.741430    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/00b25663-dc21-46b5-8cc9-f18c09f1f48b-kube-api-access-q8cl8" (OuterVolumeSpecName: "kube-api-access-q8cl8") pod "00b25663-dc21-46b5-8cc9-f18c09f1f48b" (UID: "00b25663-dc21-46b5-8cc9-f18c09f1f48b"). InnerVolumeSpecName "kube-api-access-q8cl8". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 09 03:40:54.840244 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:40:54.840175    2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-q8cl8\" (UniqueName: \"kubernetes.io/projected/00b25663-dc21-46b5-8cc9-f18c09f1f48b-kube-api-access-q8cl8\") on node \"ip-10-0-132-14.ec2.internal\" DevicePath \"\""
Sep 09 03:40:54.840244 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:40:54.840197    2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-z9pgq\" (UniqueName: \"kubernetes.io/projected/15e8a7c1-d772-485a-b7ed-bdcb3f8f3a4f-kube-api-access-z9pgq\") on node \"ip-10-0-132-14.ec2.internal\" DevicePath \"\""
Sep 09 03:40:54.840244 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:40:54.840208    2576 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/15e8a7c1-d772-485a-b7ed-bdcb3f8f3a4f-plugins-config-volume\") on node \"ip-10-0-132-14.ec2.internal\" DevicePath \"\""
Sep 09 03:40:54.840244 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:40:54.840216    2576 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/00b25663-dc21-46b5-8cc9-f18c09f1f48b-plugins-config-volume\") on node \"ip-10-0-132-14.ec2.internal\" DevicePath \"\""
Sep 09 03:40:55.014615 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:40:55.014586    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-a-7f0891-664746d6d6-5x8s5"]
Sep 09 03:40:55.020908 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:40:55.020885    2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-a-7f0891-664746d6d6-5x8s5"]
Sep 09 03:40:55.033615 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:40:55.033593    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-7f0891-78db4bf9b9-7qtcm"]
Sep 09 03:40:55.038585 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:40:55.038554    2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-7f0891-78db4bf9b9-7qtcm"]
Sep 09 03:40:56.360102 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:40:56.360066    2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="00b25663-dc21-46b5-8cc9-f18c09f1f48b" path="/var/lib/kubelet/pods/00b25663-dc21-46b5-8cc9-f18c09f1f48b/volumes"
Sep 09 03:40:56.360473 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:40:56.360395    2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="15e8a7c1-d772-485a-b7ed-bdcb3f8f3a4f" path="/var/lib/kubelet/pods/15e8a7c1-d772-485a-b7ed-bdcb3f8f3a4f/volumes"
Sep 09 03:40:56.360670 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:40:56.360657    2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="66030200-35ab-448d-9478-48a2103abff3" path="/var/lib/kubelet/pods/66030200-35ab-448d-9478-48a2103abff3/volumes"
Sep 09 03:41:01.645690 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:41:01.645646    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-1c4cb1-t5bxk"]
Sep 09 03:41:01.646111 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:41:01.646093    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="66030200-35ab-448d-9478-48a2103abff3" containerName="revoke-keys"
Sep 09 03:41:01.646162 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:41:01.646116    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="66030200-35ab-448d-9478-48a2103abff3" containerName="revoke-keys"
Sep 09 03:41:01.646162 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:41:01.646130    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="15e8a7c1-d772-485a-b7ed-bdcb3f8f3a4f" containerName="payload-pre-processing"
Sep 09 03:41:01.646162 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:41:01.646137    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="15e8a7c1-d772-485a-b7ed-bdcb3f8f3a4f" containerName="payload-pre-processing"
Sep 09 03:41:01.646162 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:41:01.646147    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="00b25663-dc21-46b5-8cc9-f18c09f1f48b" containerName="payload-processing"
Sep 09 03:41:01.646162 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:41:01.646155    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="00b25663-dc21-46b5-8cc9-f18c09f1f48b" containerName="payload-processing"
Sep 09 03:41:01.646336 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:41:01.646231    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="00b25663-dc21-46b5-8cc9-f18c09f1f48b" containerName="payload-processing"
Sep 09 03:41:01.646336 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:41:01.646243    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="15e8a7c1-d772-485a-b7ed-bdcb3f8f3a4f" containerName="payload-pre-processing"
Sep 09 03:41:01.646336 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:41:01.646248    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="66030200-35ab-448d-9478-48a2103abff3" containerName="revoke-keys"
Sep 09 03:41:01.650970 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:41:01.650953    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-1c4cb1-t5bxk"
Sep 09 03:41:01.653902 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:41:01.653883    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-w7q9f\""
Sep 09 03:41:01.657913 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:41:01.657884    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-1c4cb1-t5bxk"]
Sep 09 03:41:01.697835 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:41:01.697782    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/dbffdb09-27fa-4e18-95da-862fe9689a74-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-b-1c4cb1-t5bxk\" (UID: \"dbffdb09-27fa-4e18-95da-862fe9689a74\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-1c4cb1-t5bxk"
Sep 09 03:41:01.799238 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:41:01.799199    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/dbffdb09-27fa-4e18-95da-862fe9689a74-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-b-1c4cb1-t5bxk\" (UID: \"dbffdb09-27fa-4e18-95da-862fe9689a74\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-1c4cb1-t5bxk"
Sep 09 03:41:01.799868 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:41:01.799843    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/dbffdb09-27fa-4e18-95da-862fe9689a74-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-b-1c4cb1-t5bxk\" (UID: \"dbffdb09-27fa-4e18-95da-862fe9689a74\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-1c4cb1-t5bxk"
Sep 09 03:41:01.964770 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:41:01.964685    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-1c4cb1-t5bxk"
Sep 09 03:41:02.240299 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:41:02.240269    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-1c4cb1-t5bxk"]
Sep 09 03:41:02.242223 ip-10-0-132-14 kubenswrapper[2576]: W0909 03:41:02.242193    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddbffdb09_27fa_4e18_95da_862fe9689a74.slice/crio-58bbbad078232b418623941e7ca4bc7557e8f27622c3a32aa864409a6baa6eff WatchSource:0}: Error finding container 58bbbad078232b418623941e7ca4bc7557e8f27622c3a32aa864409a6baa6eff: Status 404 returned error can't find the container with id 58bbbad078232b418623941e7ca4bc7557e8f27622c3a32aa864409a6baa6eff
Sep 09 03:41:02.739432 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:41:02.739398    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-1c4cb1-t5bxk" event={"ID":"dbffdb09-27fa-4e18-95da-862fe9689a74","Type":"ContainerStarted","Data":"c80c29a083602b85e54879c9b5ad526d28c75cf001f3830c8d456fd29547a205"}
Sep 09 03:41:02.739872 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:41:02.739448    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-1c4cb1-t5bxk" event={"ID":"dbffdb09-27fa-4e18-95da-862fe9689a74","Type":"ContainerStarted","Data":"58bbbad078232b418623941e7ca4bc7557e8f27622c3a32aa864409a6baa6eff"}
Sep 09 03:41:02.759507 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:41:02.759441    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-1c4cb1-t5bxk" podStartSLOduration=1.759421062 podStartE2EDuration="1.759421062s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 03:41:01 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 03:41:02.756020297 +0000 UTC m=+1676.897506384" watchObservedRunningTime="2026-09-09 03:41:02.759421062 +0000 UTC m=+1676.900907143"
Sep 09 03:41:03.745775 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:41:03.745733    2576 generic.go:356] "Generic (PLEG): container finished" podID="dbffdb09-27fa-4e18-95da-862fe9689a74" containerID="c80c29a083602b85e54879c9b5ad526d28c75cf001f3830c8d456fd29547a205" exitCode=0
Sep 09 03:41:03.746215 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:41:03.745844    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-1c4cb1-t5bxk" event={"ID":"dbffdb09-27fa-4e18-95da-862fe9689a74","Type":"ContainerDied","Data":"c80c29a083602b85e54879c9b5ad526d28c75cf001f3830c8d456fd29547a205"}
Sep 09 03:41:04.883027 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:41:04.882999    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-1c4cb1-t5bxk"
Sep 09 03:41:04.928342 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:41:04.928309    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/dbffdb09-27fa-4e18-95da-862fe9689a74-maas-api-service-ca\") pod \"dbffdb09-27fa-4e18-95da-862fe9689a74\" (UID: \"dbffdb09-27fa-4e18-95da-862fe9689a74\") "
Sep 09 03:41:04.928660 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:41:04.928639    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/dbffdb09-27fa-4e18-95da-862fe9689a74-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "dbffdb09-27fa-4e18-95da-862fe9689a74" (UID: "dbffdb09-27fa-4e18-95da-862fe9689a74"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 09 03:41:05.029527 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:41:05.029420    2576 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/dbffdb09-27fa-4e18-95da-862fe9689a74-maas-api-service-ca\") on node \"ip-10-0-132-14.ec2.internal\" DevicePath \"\""
Sep 09 03:41:05.755869 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:41:05.755841    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-1c4cb1-t5bxk"
Sep 09 03:41:05.756047 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:41:05.755867    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-1c4cb1-t5bxk" event={"ID":"dbffdb09-27fa-4e18-95da-862fe9689a74","Type":"ContainerDied","Data":"58bbbad078232b418623941e7ca4bc7557e8f27622c3a32aa864409a6baa6eff"}
Sep 09 03:41:05.756047 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:41:05.755916    2576 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="58bbbad078232b418623941e7ca4bc7557e8f27622c3a32aa864409a6baa6eff"
Sep 09 03:41:06.611557 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:41:06.611520    2576 scope.go:117] "RemoveContainer" containerID="6a7be77a144e103a7273ea3ccebec4fc1f28e0b8c21ce2383a2c94d010baa67e"
Sep 09 03:41:17.338305 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:41:17.338265    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-b-1c4cb1-f4c95df69-z68mp"]
Sep 09 03:41:17.338782 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:41:17.338532    2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-api-b-1c4cb1-f4c95df69-z68mp" podUID="1027c0f0-a872-421c-8e5e-22d6dd4c9e14" containerName="payload-processing" containerID="cri-o://e5280c1172c064a44ee8dd9cbb1e13949c360ad58f30671f15d4e194f47d7520" gracePeriod=30
Sep 09 03:41:17.347207 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:41:17.347178    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-b-1c4cb1-66dcd988c8-8zx2f"]
Sep 09 03:41:17.347459 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:41:17.347413    2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-api-b-1c4cb1-66dcd988c8-8zx2f" podUID="ed42c13a-540c-4d83-8a2e-78c6268c9fb6" containerName="payload-pre-processing" containerID="cri-o://516682c28fa7fc08c09b4e3b834026c815c3eb38cc4d8d0a5f4955e469a48907" gracePeriod=30
Sep 09 03:41:17.532065 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:41:17.532035    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-1c4cb1-t5bxk"]
Sep 09 03:41:17.535743 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:41:17.535721    2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-1c4cb1-t5bxk"]
Sep 09 03:41:17.604483 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:41:17.604457    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-b-1c4cb1-f4c95df69-z68mp"
Sep 09 03:41:17.632113 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:41:17.632087    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-b-1c4cb1-66dcd988c8-8zx2f"
Sep 09 03:41:17.640950 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:41:17.640925    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-nf9j6\" (UniqueName: \"kubernetes.io/projected/1027c0f0-a872-421c-8e5e-22d6dd4c9e14-kube-api-access-nf9j6\") pod \"1027c0f0-a872-421c-8e5e-22d6dd4c9e14\" (UID: \"1027c0f0-a872-421c-8e5e-22d6dd4c9e14\") "
Sep 09 03:41:17.641087 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:41:17.641015    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/1027c0f0-a872-421c-8e5e-22d6dd4c9e14-plugins-config-volume\") pod \"1027c0f0-a872-421c-8e5e-22d6dd4c9e14\" (UID: \"1027c0f0-a872-421c-8e5e-22d6dd4c9e14\") "
Sep 09 03:41:17.641378 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:41:17.641353    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1027c0f0-a872-421c-8e5e-22d6dd4c9e14-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "1027c0f0-a872-421c-8e5e-22d6dd4c9e14" (UID: "1027c0f0-a872-421c-8e5e-22d6dd4c9e14"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 09 03:41:17.643215 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:41:17.643194    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1027c0f0-a872-421c-8e5e-22d6dd4c9e14-kube-api-access-nf9j6" (OuterVolumeSpecName: "kube-api-access-nf9j6") pod "1027c0f0-a872-421c-8e5e-22d6dd4c9e14" (UID: "1027c0f0-a872-421c-8e5e-22d6dd4c9e14"). InnerVolumeSpecName "kube-api-access-nf9j6". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 09 03:41:17.742283 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:41:17.742247    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ed42c13a-540c-4d83-8a2e-78c6268c9fb6-plugins-config-volume\") pod \"ed42c13a-540c-4d83-8a2e-78c6268c9fb6\" (UID: \"ed42c13a-540c-4d83-8a2e-78c6268c9fb6\") "
Sep 09 03:41:17.742457 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:41:17.742370    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-swz6c\" (UniqueName: \"kubernetes.io/projected/ed42c13a-540c-4d83-8a2e-78c6268c9fb6-kube-api-access-swz6c\") pod \"ed42c13a-540c-4d83-8a2e-78c6268c9fb6\" (UID: \"ed42c13a-540c-4d83-8a2e-78c6268c9fb6\") "
Sep 09 03:41:17.742627 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:41:17.742610    2576 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/1027c0f0-a872-421c-8e5e-22d6dd4c9e14-plugins-config-volume\") on node \"ip-10-0-132-14.ec2.internal\" DevicePath \"\""
Sep 09 03:41:17.742679 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:41:17.742636    2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-nf9j6\" (UniqueName: \"kubernetes.io/projected/1027c0f0-a872-421c-8e5e-22d6dd4c9e14-kube-api-access-nf9j6\") on node \"ip-10-0-132-14.ec2.internal\" DevicePath \"\""
Sep 09 03:41:17.742717 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:41:17.742671    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ed42c13a-540c-4d83-8a2e-78c6268c9fb6-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "ed42c13a-540c-4d83-8a2e-78c6268c9fb6" (UID: "ed42c13a-540c-4d83-8a2e-78c6268c9fb6"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 09 03:41:17.744482 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:41:17.744459    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ed42c13a-540c-4d83-8a2e-78c6268c9fb6-kube-api-access-swz6c" (OuterVolumeSpecName: "kube-api-access-swz6c") pod "ed42c13a-540c-4d83-8a2e-78c6268c9fb6" (UID: "ed42c13a-540c-4d83-8a2e-78c6268c9fb6"). InnerVolumeSpecName "kube-api-access-swz6c". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 09 03:41:17.812183 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:41:17.812151    2576 generic.go:356] "Generic (PLEG): container finished" podID="1027c0f0-a872-421c-8e5e-22d6dd4c9e14" containerID="e5280c1172c064a44ee8dd9cbb1e13949c360ad58f30671f15d4e194f47d7520" exitCode=0
Sep 09 03:41:17.812359 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:41:17.812201    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-b-1c4cb1-f4c95df69-z68mp"
Sep 09 03:41:17.812359 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:41:17.812228    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-b-1c4cb1-f4c95df69-z68mp" event={"ID":"1027c0f0-a872-421c-8e5e-22d6dd4c9e14","Type":"ContainerDied","Data":"e5280c1172c064a44ee8dd9cbb1e13949c360ad58f30671f15d4e194f47d7520"}
Sep 09 03:41:17.812359 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:41:17.812284    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-b-1c4cb1-f4c95df69-z68mp" event={"ID":"1027c0f0-a872-421c-8e5e-22d6dd4c9e14","Type":"ContainerDied","Data":"3cec42ac06a1e504bf8efed491da5c148c3a15b49350835ffeab06824d2d6142"}
Sep 09 03:41:17.812359 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:41:17.812318    2576 scope.go:117] "RemoveContainer" containerID="e5280c1172c064a44ee8dd9cbb1e13949c360ad58f30671f15d4e194f47d7520"
Sep 09 03:41:17.813673 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:41:17.813647    2576 generic.go:356] "Generic (PLEG): container finished" podID="ed42c13a-540c-4d83-8a2e-78c6268c9fb6" containerID="516682c28fa7fc08c09b4e3b834026c815c3eb38cc4d8d0a5f4955e469a48907" exitCode=0
Sep 09 03:41:17.813777 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:41:17.813684    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-b-1c4cb1-66dcd988c8-8zx2f" event={"ID":"ed42c13a-540c-4d83-8a2e-78c6268c9fb6","Type":"ContainerDied","Data":"516682c28fa7fc08c09b4e3b834026c815c3eb38cc4d8d0a5f4955e469a48907"}
Sep 09 03:41:17.813777 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:41:17.813688    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-b-1c4cb1-66dcd988c8-8zx2f"
Sep 09 03:41:17.813777 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:41:17.813722    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-b-1c4cb1-66dcd988c8-8zx2f" event={"ID":"ed42c13a-540c-4d83-8a2e-78c6268c9fb6","Type":"ContainerDied","Data":"45d121cce517adedb38197b0172cf63e561da1fc195e8e63bd1774865b7d6324"}
Sep 09 03:41:17.821545 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:41:17.821525    2576 scope.go:117] "RemoveContainer" containerID="e5280c1172c064a44ee8dd9cbb1e13949c360ad58f30671f15d4e194f47d7520"
Sep 09 03:41:17.821918 ip-10-0-132-14 kubenswrapper[2576]: E0909 03:41:17.821896    2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e5280c1172c064a44ee8dd9cbb1e13949c360ad58f30671f15d4e194f47d7520\": container with ID starting with e5280c1172c064a44ee8dd9cbb1e13949c360ad58f30671f15d4e194f47d7520 not found: ID does not exist" containerID="e5280c1172c064a44ee8dd9cbb1e13949c360ad58f30671f15d4e194f47d7520"
Sep 09 03:41:17.822016 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:41:17.821929    2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e5280c1172c064a44ee8dd9cbb1e13949c360ad58f30671f15d4e194f47d7520"} err="failed to get container status \"e5280c1172c064a44ee8dd9cbb1e13949c360ad58f30671f15d4e194f47d7520\": rpc error: code = NotFound desc = could not find container \"e5280c1172c064a44ee8dd9cbb1e13949c360ad58f30671f15d4e194f47d7520\": container with ID starting with e5280c1172c064a44ee8dd9cbb1e13949c360ad58f30671f15d4e194f47d7520 not found: ID does not exist"
Sep 09 03:41:17.822016 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:41:17.821950    2576 scope.go:117] "RemoveContainer" containerID="516682c28fa7fc08c09b4e3b834026c815c3eb38cc4d8d0a5f4955e469a48907"
Sep 09 03:41:17.829652 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:41:17.829635    2576 scope.go:117] "RemoveContainer" containerID="516682c28fa7fc08c09b4e3b834026c815c3eb38cc4d8d0a5f4955e469a48907"
Sep 09 03:41:17.829940 ip-10-0-132-14 kubenswrapper[2576]: E0909 03:41:17.829922    2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"516682c28fa7fc08c09b4e3b834026c815c3eb38cc4d8d0a5f4955e469a48907\": container with ID starting with 516682c28fa7fc08c09b4e3b834026c815c3eb38cc4d8d0a5f4955e469a48907 not found: ID does not exist" containerID="516682c28fa7fc08c09b4e3b834026c815c3eb38cc4d8d0a5f4955e469a48907"
Sep 09 03:41:17.829991 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:41:17.829948    2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"516682c28fa7fc08c09b4e3b834026c815c3eb38cc4d8d0a5f4955e469a48907"} err="failed to get container status \"516682c28fa7fc08c09b4e3b834026c815c3eb38cc4d8d0a5f4955e469a48907\": rpc error: code = NotFound desc = could not find container \"516682c28fa7fc08c09b4e3b834026c815c3eb38cc4d8d0a5f4955e469a48907\": container with ID starting with 516682c28fa7fc08c09b4e3b834026c815c3eb38cc4d8d0a5f4955e469a48907 not found: ID does not exist"
Sep 09 03:41:17.835541 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:41:17.835520    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-b-1c4cb1-f4c95df69-z68mp"]
Sep 09 03:41:17.842455 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:41:17.842408    2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-b-1c4cb1-f4c95df69-z68mp"]
Sep 09 03:41:17.843112 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:41:17.843094    2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-swz6c\" (UniqueName: \"kubernetes.io/projected/ed42c13a-540c-4d83-8a2e-78c6268c9fb6-kube-api-access-swz6c\") on node \"ip-10-0-132-14.ec2.internal\" DevicePath \"\""
Sep 09 03:41:17.843169 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:41:17.843116    2576 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ed42c13a-540c-4d83-8a2e-78c6268c9fb6-plugins-config-volume\") on node \"ip-10-0-132-14.ec2.internal\" DevicePath \"\""
Sep 09 03:41:17.853473 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:41:17.853453    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-b-1c4cb1-66dcd988c8-8zx2f"]
Sep 09 03:41:17.858760 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:41:17.858740    2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-b-1c4cb1-66dcd988c8-8zx2f"]
Sep 09 03:41:18.360643 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:41:18.360616    2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="1027c0f0-a872-421c-8e5e-22d6dd4c9e14" path="/var/lib/kubelet/pods/1027c0f0-a872-421c-8e5e-22d6dd4c9e14/volumes"
Sep 09 03:41:18.360992 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:41:18.360975    2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="dbffdb09-27fa-4e18-95da-862fe9689a74" path="/var/lib/kubelet/pods/dbffdb09-27fa-4e18-95da-862fe9689a74/volumes"
Sep 09 03:41:18.361282 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:41:18.361270    2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ed42c13a-540c-4d83-8a2e-78c6268c9fb6" path="/var/lib/kubelet/pods/ed42c13a-540c-4d83-8a2e-78c6268c9fb6/volumes"
Sep 09 03:41:24.514477 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:41:24.514442    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-eca017-bv6bb"]
Sep 09 03:41:24.514926 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:41:24.514787    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1027c0f0-a872-421c-8e5e-22d6dd4c9e14" containerName="payload-processing"
Sep 09 03:41:24.514926 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:41:24.514813    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="1027c0f0-a872-421c-8e5e-22d6dd4c9e14" containerName="payload-processing"
Sep 09 03:41:24.514926 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:41:24.514823    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ed42c13a-540c-4d83-8a2e-78c6268c9fb6" containerName="payload-pre-processing"
Sep 09 03:41:24.514926 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:41:24.514828    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="ed42c13a-540c-4d83-8a2e-78c6268c9fb6" containerName="payload-pre-processing"
Sep 09 03:41:24.514926 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:41:24.514845    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="dbffdb09-27fa-4e18-95da-862fe9689a74" containerName="revoke-keys"
Sep 09 03:41:24.514926 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:41:24.514850    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="dbffdb09-27fa-4e18-95da-862fe9689a74" containerName="revoke-keys"
Sep 09 03:41:24.514926 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:41:24.514929    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="ed42c13a-540c-4d83-8a2e-78c6268c9fb6" containerName="payload-pre-processing"
Sep 09 03:41:24.515212 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:41:24.514942    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="dbffdb09-27fa-4e18-95da-862fe9689a74" containerName="revoke-keys"
Sep 09 03:41:24.515212 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:41:24.514948    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="1027c0f0-a872-421c-8e5e-22d6dd4c9e14" containerName="payload-processing"
Sep 09 03:41:24.519454 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:41:24.519431    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-eca017-bv6bb"
Sep 09 03:41:24.522212 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:41:24.522182    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-w7q9f\""
Sep 09 03:41:24.526047 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:41:24.526022    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-eca017-bv6bb"]
Sep 09 03:41:24.602618 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:41:24.602584    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/f87d7887-6577-4a6a-8a83-d7c687410baa-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-a-w2-eca017-bv6bb\" (UID: \"f87d7887-6577-4a6a-8a83-d7c687410baa\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-eca017-bv6bb"
Sep 09 03:41:24.703130 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:41:24.703093    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/f87d7887-6577-4a6a-8a83-d7c687410baa-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-a-w2-eca017-bv6bb\" (UID: \"f87d7887-6577-4a6a-8a83-d7c687410baa\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-eca017-bv6bb"
Sep 09 03:41:24.703737 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:41:24.703711    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/f87d7887-6577-4a6a-8a83-d7c687410baa-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-a-w2-eca017-bv6bb\" (UID: \"f87d7887-6577-4a6a-8a83-d7c687410baa\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-eca017-bv6bb"
Sep 09 03:41:24.831609 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:41:24.831525    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-eca017-bv6bb"
Sep 09 03:41:25.108130 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:41:25.107497    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-eca017-bv6bb"]
Sep 09 03:41:25.110204 ip-10-0-132-14 kubenswrapper[2576]: W0909 03:41:25.110178    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf87d7887_6577_4a6a_8a83_d7c687410baa.slice/crio-f64f5e063a4e2149f65c8b45b4723a4eef79ab85a9475455700af2d1961ea199 WatchSource:0}: Error finding container f64f5e063a4e2149f65c8b45b4723a4eef79ab85a9475455700af2d1961ea199: Status 404 returned error can't find the container with id f64f5e063a4e2149f65c8b45b4723a4eef79ab85a9475455700af2d1961ea199
Sep 09 03:41:25.861568 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:41:25.861528    2576 generic.go:356] "Generic (PLEG): container finished" podID="f87d7887-6577-4a6a-8a83-d7c687410baa" containerID="aac22a5ca3e9bc64cf3de6a55f91e2cc81be781dbfe62e5935cab5a33f50c9ad" exitCode=0
Sep 09 03:41:25.861926 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:41:25.861621    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-eca017-bv6bb" event={"ID":"f87d7887-6577-4a6a-8a83-d7c687410baa","Type":"ContainerDied","Data":"aac22a5ca3e9bc64cf3de6a55f91e2cc81be781dbfe62e5935cab5a33f50c9ad"}
Sep 09 03:41:25.861926 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:41:25.861681    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-eca017-bv6bb" event={"ID":"f87d7887-6577-4a6a-8a83-d7c687410baa","Type":"ContainerStarted","Data":"f64f5e063a4e2149f65c8b45b4723a4eef79ab85a9475455700af2d1961ea199"}
Sep 09 03:41:26.995315 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:41:26.995291    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-eca017-bv6bb"
Sep 09 03:41:27.125276 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:41:27.125200    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/f87d7887-6577-4a6a-8a83-d7c687410baa-maas-api-service-ca\") pod \"f87d7887-6577-4a6a-8a83-d7c687410baa\" (UID: \"f87d7887-6577-4a6a-8a83-d7c687410baa\") "
Sep 09 03:41:27.125566 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:41:27.125547    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f87d7887-6577-4a6a-8a83-d7c687410baa-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "f87d7887-6577-4a6a-8a83-d7c687410baa" (UID: "f87d7887-6577-4a6a-8a83-d7c687410baa"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 09 03:41:27.226267 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:41:27.226236    2576 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/f87d7887-6577-4a6a-8a83-d7c687410baa-maas-api-service-ca\") on node \"ip-10-0-132-14.ec2.internal\" DevicePath \"\""
Sep 09 03:41:27.872384 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:41:27.872359    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-eca017-bv6bb"
Sep 09 03:41:27.872563 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:41:27.872386    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-eca017-bv6bb" event={"ID":"f87d7887-6577-4a6a-8a83-d7c687410baa","Type":"ContainerDied","Data":"f64f5e063a4e2149f65c8b45b4723a4eef79ab85a9475455700af2d1961ea199"}
Sep 09 03:41:27.872563 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:41:27.872426    2576 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="f64f5e063a4e2149f65c8b45b4723a4eef79ab85a9475455700af2d1961ea199"
Sep 09 03:41:35.030968 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:41:35.030931    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w2-eca017-74b8bd4b8c-fqsmw"]
Sep 09 03:41:35.031385 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:41:35.031195    2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-shared-a-w2-eca017-74b8bd4b8c-fqsmw" podUID="ed3d5b68-e7e0-4788-889e-6a8ab165da9a" containerName="payload-processing" containerID="cri-o://a1036e9fe33f66ae7f2c23e9cdac8812c6c455420619d80868ed1a700dc6ccda" gracePeriod=30
Sep 09 03:41:35.038342 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:41:35.038306    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w2-eca017-6b7b56d98-lfn2v"]
Sep 09 03:41:35.038566 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:41:35.038537    2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-eca017-6b7b56d98-lfn2v" podUID="609e896a-5dd3-42d8-a251-a225d4ddab03" containerName="payload-pre-processing" containerID="cri-o://931e37ad007775009cab8b99f7cda904aed50f71bcb9898786fcb072f75f92da" gracePeriod=30
Sep 09 03:41:35.201875 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:41:35.201842    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-eca017-bv6bb"]
Sep 09 03:41:35.207643 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:41:35.207614    2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-eca017-bv6bb"]
Sep 09 03:41:35.313478 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:41:35.313456    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-a-w2-eca017-74b8bd4b8c-fqsmw"
Sep 09 03:41:35.316227 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:41:35.316210    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-eca017-6b7b56d98-lfn2v"
Sep 09 03:41:35.394715 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:41:35.394686    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-2qm6t\" (UniqueName: \"kubernetes.io/projected/ed3d5b68-e7e0-4788-889e-6a8ab165da9a-kube-api-access-2qm6t\") pod \"ed3d5b68-e7e0-4788-889e-6a8ab165da9a\" (UID: \"ed3d5b68-e7e0-4788-889e-6a8ab165da9a\") "
Sep 09 03:41:35.394953 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:41:35.394730    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ed3d5b68-e7e0-4788-889e-6a8ab165da9a-plugins-config-volume\") pod \"ed3d5b68-e7e0-4788-889e-6a8ab165da9a\" (UID: \"ed3d5b68-e7e0-4788-889e-6a8ab165da9a\") "
Sep 09 03:41:35.394953 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:41:35.394772    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/609e896a-5dd3-42d8-a251-a225d4ddab03-plugins-config-volume\") pod \"609e896a-5dd3-42d8-a251-a225d4ddab03\" (UID: \"609e896a-5dd3-42d8-a251-a225d4ddab03\") "
Sep 09 03:41:35.394953 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:41:35.394891    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-v8mcq\" (UniqueName: \"kubernetes.io/projected/609e896a-5dd3-42d8-a251-a225d4ddab03-kube-api-access-v8mcq\") pod \"609e896a-5dd3-42d8-a251-a225d4ddab03\" (UID: \"609e896a-5dd3-42d8-a251-a225d4ddab03\") "
Sep 09 03:41:35.395143 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:41:35.395128    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/609e896a-5dd3-42d8-a251-a225d4ddab03-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "609e896a-5dd3-42d8-a251-a225d4ddab03" (UID: "609e896a-5dd3-42d8-a251-a225d4ddab03"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 09 03:41:35.395201 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:41:35.395179    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ed3d5b68-e7e0-4788-889e-6a8ab165da9a-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "ed3d5b68-e7e0-4788-889e-6a8ab165da9a" (UID: "ed3d5b68-e7e0-4788-889e-6a8ab165da9a"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 09 03:41:35.395241 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:41:35.395198    2576 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/609e896a-5dd3-42d8-a251-a225d4ddab03-plugins-config-volume\") on node \"ip-10-0-132-14.ec2.internal\" DevicePath \"\""
Sep 09 03:41:35.397032 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:41:35.397011    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/609e896a-5dd3-42d8-a251-a225d4ddab03-kube-api-access-v8mcq" (OuterVolumeSpecName: "kube-api-access-v8mcq") pod "609e896a-5dd3-42d8-a251-a225d4ddab03" (UID: "609e896a-5dd3-42d8-a251-a225d4ddab03"). InnerVolumeSpecName "kube-api-access-v8mcq". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 09 03:41:35.397084 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:41:35.397027    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ed3d5b68-e7e0-4788-889e-6a8ab165da9a-kube-api-access-2qm6t" (OuterVolumeSpecName: "kube-api-access-2qm6t") pod "ed3d5b68-e7e0-4788-889e-6a8ab165da9a" (UID: "ed3d5b68-e7e0-4788-889e-6a8ab165da9a"). InnerVolumeSpecName "kube-api-access-2qm6t". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 09 03:41:35.495939 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:41:35.495903    2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-v8mcq\" (UniqueName: \"kubernetes.io/projected/609e896a-5dd3-42d8-a251-a225d4ddab03-kube-api-access-v8mcq\") on node \"ip-10-0-132-14.ec2.internal\" DevicePath \"\""
Sep 09 03:41:35.495939 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:41:35.495930    2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2qm6t\" (UniqueName: \"kubernetes.io/projected/ed3d5b68-e7e0-4788-889e-6a8ab165da9a-kube-api-access-2qm6t\") on node \"ip-10-0-132-14.ec2.internal\" DevicePath \"\""
Sep 09 03:41:35.496147 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:41:35.495941    2576 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ed3d5b68-e7e0-4788-889e-6a8ab165da9a-plugins-config-volume\") on node \"ip-10-0-132-14.ec2.internal\" DevicePath \"\""
Sep 09 03:41:35.911840 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:41:35.911785    2576 generic.go:356] "Generic (PLEG): container finished" podID="609e896a-5dd3-42d8-a251-a225d4ddab03" containerID="931e37ad007775009cab8b99f7cda904aed50f71bcb9898786fcb072f75f92da" exitCode=0
Sep 09 03:41:35.912008 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:41:35.911849    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-eca017-6b7b56d98-lfn2v"
Sep 09 03:41:35.912008 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:41:35.911871    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-eca017-6b7b56d98-lfn2v" event={"ID":"609e896a-5dd3-42d8-a251-a225d4ddab03","Type":"ContainerDied","Data":"931e37ad007775009cab8b99f7cda904aed50f71bcb9898786fcb072f75f92da"}
Sep 09 03:41:35.912008 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:41:35.911929    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-eca017-6b7b56d98-lfn2v" event={"ID":"609e896a-5dd3-42d8-a251-a225d4ddab03","Type":"ContainerDied","Data":"c9f70fd1f78cff2c20b128e693beeb799aca33622cc94d32eba53ca7d37dcc1f"}
Sep 09 03:41:35.912008 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:41:35.911953    2576 scope.go:117] "RemoveContainer" containerID="931e37ad007775009cab8b99f7cda904aed50f71bcb9898786fcb072f75f92da"
Sep 09 03:41:35.913330 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:41:35.913309    2576 generic.go:356] "Generic (PLEG): container finished" podID="ed3d5b68-e7e0-4788-889e-6a8ab165da9a" containerID="a1036e9fe33f66ae7f2c23e9cdac8812c6c455420619d80868ed1a700dc6ccda" exitCode=0
Sep 09 03:41:35.913447 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:41:35.913355    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-a-w2-eca017-74b8bd4b8c-fqsmw"
Sep 09 03:41:35.913447 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:41:35.913372    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-a-w2-eca017-74b8bd4b8c-fqsmw" event={"ID":"ed3d5b68-e7e0-4788-889e-6a8ab165da9a","Type":"ContainerDied","Data":"a1036e9fe33f66ae7f2c23e9cdac8812c6c455420619d80868ed1a700dc6ccda"}
Sep 09 03:41:35.913447 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:41:35.913402    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-a-w2-eca017-74b8bd4b8c-fqsmw" event={"ID":"ed3d5b68-e7e0-4788-889e-6a8ab165da9a","Type":"ContainerDied","Data":"e206fb4ba31ff45b5d4cc0edbda25226cb6d20d2d8588d3a63f1a9f6c585aaa6"}
Sep 09 03:41:35.922108 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:41:35.922093    2576 scope.go:117] "RemoveContainer" containerID="931e37ad007775009cab8b99f7cda904aed50f71bcb9898786fcb072f75f92da"
Sep 09 03:41:35.922380 ip-10-0-132-14 kubenswrapper[2576]: E0909 03:41:35.922362    2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"931e37ad007775009cab8b99f7cda904aed50f71bcb9898786fcb072f75f92da\": container with ID starting with 931e37ad007775009cab8b99f7cda904aed50f71bcb9898786fcb072f75f92da not found: ID does not exist" containerID="931e37ad007775009cab8b99f7cda904aed50f71bcb9898786fcb072f75f92da"
Sep 09 03:41:35.922470 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:41:35.922387    2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"931e37ad007775009cab8b99f7cda904aed50f71bcb9898786fcb072f75f92da"} err="failed to get container status \"931e37ad007775009cab8b99f7cda904aed50f71bcb9898786fcb072f75f92da\": rpc error: code = NotFound desc = could not find container \"931e37ad007775009cab8b99f7cda904aed50f71bcb9898786fcb072f75f92da\": container with ID starting with 931e37ad007775009cab8b99f7cda904aed50f71bcb9898786fcb072f75f92da not found: ID does not exist"
Sep 09 03:41:35.922470 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:41:35.922402    2576 scope.go:117] "RemoveContainer" containerID="a1036e9fe33f66ae7f2c23e9cdac8812c6c455420619d80868ed1a700dc6ccda"
Sep 09 03:41:35.930291 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:41:35.930273    2576 scope.go:117] "RemoveContainer" containerID="a1036e9fe33f66ae7f2c23e9cdac8812c6c455420619d80868ed1a700dc6ccda"
Sep 09 03:41:35.930523 ip-10-0-132-14 kubenswrapper[2576]: E0909 03:41:35.930500    2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a1036e9fe33f66ae7f2c23e9cdac8812c6c455420619d80868ed1a700dc6ccda\": container with ID starting with a1036e9fe33f66ae7f2c23e9cdac8812c6c455420619d80868ed1a700dc6ccda not found: ID does not exist" containerID="a1036e9fe33f66ae7f2c23e9cdac8812c6c455420619d80868ed1a700dc6ccda"
Sep 09 03:41:35.930571 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:41:35.930530    2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a1036e9fe33f66ae7f2c23e9cdac8812c6c455420619d80868ed1a700dc6ccda"} err="failed to get container status \"a1036e9fe33f66ae7f2c23e9cdac8812c6c455420619d80868ed1a700dc6ccda\": rpc error: code = NotFound desc = could not find container \"a1036e9fe33f66ae7f2c23e9cdac8812c6c455420619d80868ed1a700dc6ccda\": container with ID starting with a1036e9fe33f66ae7f2c23e9cdac8812c6c455420619d80868ed1a700dc6ccda not found: ID does not exist"
Sep 09 03:41:35.934934 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:41:35.934913    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w2-eca017-6b7b56d98-lfn2v"]
Sep 09 03:41:35.939473 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:41:35.939454    2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w2-eca017-6b7b56d98-lfn2v"]
Sep 09 03:41:35.950723 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:41:35.950700    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w2-eca017-74b8bd4b8c-fqsmw"]
Sep 09 03:41:35.952777 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:41:35.952756    2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w2-eca017-74b8bd4b8c-fqsmw"]
Sep 09 03:41:36.360796 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:41:36.360757    2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="609e896a-5dd3-42d8-a251-a225d4ddab03" path="/var/lib/kubelet/pods/609e896a-5dd3-42d8-a251-a225d4ddab03/volumes"
Sep 09 03:41:36.361338 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:41:36.361313    2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ed3d5b68-e7e0-4788-889e-6a8ab165da9a" path="/var/lib/kubelet/pods/ed3d5b68-e7e0-4788-889e-6a8ab165da9a/volumes"
Sep 09 03:41:36.361658 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:41:36.361640    2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f87d7887-6577-4a6a-8a83-d7c687410baa" path="/var/lib/kubelet/pods/f87d7887-6577-4a6a-8a83-d7c687410baa/volumes"
Sep 09 03:41:42.243427 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:41:42.243383    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-0d0cae-gtzxg"]
Sep 09 03:41:42.243960 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:41:42.243751    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ed3d5b68-e7e0-4788-889e-6a8ab165da9a" containerName="payload-processing"
Sep 09 03:41:42.243960 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:41:42.243763    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="ed3d5b68-e7e0-4788-889e-6a8ab165da9a" containerName="payload-processing"
Sep 09 03:41:42.243960 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:41:42.243772    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f87d7887-6577-4a6a-8a83-d7c687410baa" containerName="revoke-keys"
Sep 09 03:41:42.243960 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:41:42.243777    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="f87d7887-6577-4a6a-8a83-d7c687410baa" containerName="revoke-keys"
Sep 09 03:41:42.243960 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:41:42.243787    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="609e896a-5dd3-42d8-a251-a225d4ddab03" containerName="payload-pre-processing"
Sep 09 03:41:42.243960 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:41:42.243793    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="609e896a-5dd3-42d8-a251-a225d4ddab03" containerName="payload-pre-processing"
Sep 09 03:41:42.243960 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:41:42.243884    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="609e896a-5dd3-42d8-a251-a225d4ddab03" containerName="payload-pre-processing"
Sep 09 03:41:42.243960 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:41:42.243896    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="ed3d5b68-e7e0-4788-889e-6a8ab165da9a" containerName="payload-processing"
Sep 09 03:41:42.243960 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:41:42.243902    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="f87d7887-6577-4a6a-8a83-d7c687410baa" containerName="revoke-keys"
Sep 09 03:41:42.246777 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:41:42.246752    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-0d0cae-gtzxg"
Sep 09 03:41:42.249929 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:41:42.249905    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-w7q9f\""
Sep 09 03:41:42.255621 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:41:42.255592    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-0d0cae-gtzxg"]
Sep 09 03:41:42.354186 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:41:42.354141    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/9b9e10ab-088e-402b-9db2-7cd03956b305-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-b-w2-0d0cae-gtzxg\" (UID: \"9b9e10ab-088e-402b-9db2-7cd03956b305\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-0d0cae-gtzxg"
Sep 09 03:41:42.455634 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:41:42.455595    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/9b9e10ab-088e-402b-9db2-7cd03956b305-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-b-w2-0d0cae-gtzxg\" (UID: \"9b9e10ab-088e-402b-9db2-7cd03956b305\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-0d0cae-gtzxg"
Sep 09 03:41:42.456238 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:41:42.456220    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/9b9e10ab-088e-402b-9db2-7cd03956b305-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-b-w2-0d0cae-gtzxg\" (UID: \"9b9e10ab-088e-402b-9db2-7cd03956b305\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-0d0cae-gtzxg"
Sep 09 03:41:42.558790 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:41:42.558756    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-0d0cae-gtzxg"
Sep 09 03:41:42.831704 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:41:42.831675    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-0d0cae-gtzxg"]
Sep 09 03:41:42.834401 ip-10-0-132-14 kubenswrapper[2576]: W0909 03:41:42.834376    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9b9e10ab_088e_402b_9db2_7cd03956b305.slice/crio-09a04a90d4c0157660f283470e4260fcdea676e20a9e0183360eb46e801c2cb8 WatchSource:0}: Error finding container 09a04a90d4c0157660f283470e4260fcdea676e20a9e0183360eb46e801c2cb8: Status 404 returned error can't find the container with id 09a04a90d4c0157660f283470e4260fcdea676e20a9e0183360eb46e801c2cb8
Sep 09 03:41:42.948892 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:41:42.948855    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-0d0cae-gtzxg" event={"ID":"9b9e10ab-088e-402b-9db2-7cd03956b305","Type":"ContainerStarted","Data":"09a04a90d4c0157660f283470e4260fcdea676e20a9e0183360eb46e801c2cb8"}
Sep 09 03:41:43.955297 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:41:43.955210    2576 generic.go:356] "Generic (PLEG): container finished" podID="9b9e10ab-088e-402b-9db2-7cd03956b305" containerID="3a5c686dd0c1b4f11e0383981fb37b336e46e0a8c91de294c4684689c14836f3" exitCode=0
Sep 09 03:41:43.955297 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:41:43.955248    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-0d0cae-gtzxg" event={"ID":"9b9e10ab-088e-402b-9db2-7cd03956b305","Type":"ContainerDied","Data":"3a5c686dd0c1b4f11e0383981fb37b336e46e0a8c91de294c4684689c14836f3"}
Sep 09 03:41:45.093671 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:41:45.093644    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-0d0cae-gtzxg"
Sep 09 03:41:45.180173 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:41:45.180140    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/9b9e10ab-088e-402b-9db2-7cd03956b305-maas-api-service-ca\") pod \"9b9e10ab-088e-402b-9db2-7cd03956b305\" (UID: \"9b9e10ab-088e-402b-9db2-7cd03956b305\") "
Sep 09 03:41:45.180471 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:41:45.180448    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9b9e10ab-088e-402b-9db2-7cd03956b305-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "9b9e10ab-088e-402b-9db2-7cd03956b305" (UID: "9b9e10ab-088e-402b-9db2-7cd03956b305"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 09 03:41:45.280939 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:41:45.280905    2576 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/9b9e10ab-088e-402b-9db2-7cd03956b305-maas-api-service-ca\") on node \"ip-10-0-132-14.ec2.internal\" DevicePath \"\""
Sep 09 03:41:45.966513 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:41:45.966481    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-0d0cae-gtzxg"
Sep 09 03:41:45.966721 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:41:45.966471    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-0d0cae-gtzxg" event={"ID":"9b9e10ab-088e-402b-9db2-7cd03956b305","Type":"ContainerDied","Data":"09a04a90d4c0157660f283470e4260fcdea676e20a9e0183360eb46e801c2cb8"}
Sep 09 03:41:45.966721 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:41:45.966625    2576 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="09a04a90d4c0157660f283470e4260fcdea676e20a9e0183360eb46e801c2cb8"
Sep 09 03:41:52.913598 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:41:52.913559    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w2-0d0cae-68c69b8786-qt4kh"]
Sep 09 03:41:52.914053 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:41:52.913780    2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-0d0cae-68c69b8786-qt4kh" podUID="c02969f1-8ac8-4c3d-9b70-c4f5f29a6c89" containerName="payload-pre-processing" containerID="cri-o://5eab9b98dbb15238c637c6e38ad53596967ecc65150a01a01462291c9a32840d" gracePeriod=30
Sep 09 03:41:53.080100 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:41:53.080060    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-0d0cae-gtzxg"]
Sep 09 03:41:53.085175 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:41:53.085146    2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-0d0cae-gtzxg"]
Sep 09 03:41:53.166687 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:41:53.166619    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-0d0cae-68c69b8786-qt4kh"
Sep 09 03:41:53.237034 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:41:53.237006    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-8v8xt\" (UniqueName: \"kubernetes.io/projected/c02969f1-8ac8-4c3d-9b70-c4f5f29a6c89-kube-api-access-8v8xt\") pod \"c02969f1-8ac8-4c3d-9b70-c4f5f29a6c89\" (UID: \"c02969f1-8ac8-4c3d-9b70-c4f5f29a6c89\") "
Sep 09 03:41:53.237034 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:41:53.237038    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c02969f1-8ac8-4c3d-9b70-c4f5f29a6c89-plugins-config-volume\") pod \"c02969f1-8ac8-4c3d-9b70-c4f5f29a6c89\" (UID: \"c02969f1-8ac8-4c3d-9b70-c4f5f29a6c89\") "
Sep 09 03:41:53.237445 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:41:53.237420    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c02969f1-8ac8-4c3d-9b70-c4f5f29a6c89-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "c02969f1-8ac8-4c3d-9b70-c4f5f29a6c89" (UID: "c02969f1-8ac8-4c3d-9b70-c4f5f29a6c89"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 09 03:41:53.239305 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:41:53.239282    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c02969f1-8ac8-4c3d-9b70-c4f5f29a6c89-kube-api-access-8v8xt" (OuterVolumeSpecName: "kube-api-access-8v8xt") pod "c02969f1-8ac8-4c3d-9b70-c4f5f29a6c89" (UID: "c02969f1-8ac8-4c3d-9b70-c4f5f29a6c89"). InnerVolumeSpecName "kube-api-access-8v8xt". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 09 03:41:53.337744 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:41:53.337712    2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8v8xt\" (UniqueName: \"kubernetes.io/projected/c02969f1-8ac8-4c3d-9b70-c4f5f29a6c89-kube-api-access-8v8xt\") on node \"ip-10-0-132-14.ec2.internal\" DevicePath \"\""
Sep 09 03:41:53.337744 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:41:53.337738    2576 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c02969f1-8ac8-4c3d-9b70-c4f5f29a6c89-plugins-config-volume\") on node \"ip-10-0-132-14.ec2.internal\" DevicePath \"\""
Sep 09 03:41:54.004276 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:41:54.004241    2576 generic.go:356] "Generic (PLEG): container finished" podID="c02969f1-8ac8-4c3d-9b70-c4f5f29a6c89" containerID="5eab9b98dbb15238c637c6e38ad53596967ecc65150a01a01462291c9a32840d" exitCode=0
Sep 09 03:41:54.004730 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:41:54.004284    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-0d0cae-68c69b8786-qt4kh"
Sep 09 03:41:54.004730 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:41:54.004335    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-0d0cae-68c69b8786-qt4kh" event={"ID":"c02969f1-8ac8-4c3d-9b70-c4f5f29a6c89","Type":"ContainerDied","Data":"5eab9b98dbb15238c637c6e38ad53596967ecc65150a01a01462291c9a32840d"}
Sep 09 03:41:54.004730 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:41:54.004398    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-0d0cae-68c69b8786-qt4kh" event={"ID":"c02969f1-8ac8-4c3d-9b70-c4f5f29a6c89","Type":"ContainerDied","Data":"0069bd7c1d83dfc1c5f2a3f8addd1735afe758ee2e66e89cbfa9343abf6ed7ab"}
Sep 09 03:41:54.004730 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:41:54.004433    2576 scope.go:117] "RemoveContainer" containerID="5eab9b98dbb15238c637c6e38ad53596967ecc65150a01a01462291c9a32840d"
Sep 09 03:41:54.013485 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:41:54.013466    2576 scope.go:117] "RemoveContainer" containerID="5eab9b98dbb15238c637c6e38ad53596967ecc65150a01a01462291c9a32840d"
Sep 09 03:41:54.013762 ip-10-0-132-14 kubenswrapper[2576]: E0909 03:41:54.013744    2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5eab9b98dbb15238c637c6e38ad53596967ecc65150a01a01462291c9a32840d\": container with ID starting with 5eab9b98dbb15238c637c6e38ad53596967ecc65150a01a01462291c9a32840d not found: ID does not exist" containerID="5eab9b98dbb15238c637c6e38ad53596967ecc65150a01a01462291c9a32840d"
Sep 09 03:41:54.013848 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:41:54.013770    2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5eab9b98dbb15238c637c6e38ad53596967ecc65150a01a01462291c9a32840d"} err="failed to get container status \"5eab9b98dbb15238c637c6e38ad53596967ecc65150a01a01462291c9a32840d\": rpc error: code = NotFound desc = could not find container \"5eab9b98dbb15238c637c6e38ad53596967ecc65150a01a01462291c9a32840d\": container with ID starting with 5eab9b98dbb15238c637c6e38ad53596967ecc65150a01a01462291c9a32840d not found: ID does not exist"
Sep 09 03:41:54.026761 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:41:54.026740    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w2-0d0cae-68c69b8786-qt4kh"]
Sep 09 03:41:54.033348 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:41:54.033328    2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w2-0d0cae-68c69b8786-qt4kh"]
Sep 09 03:41:54.360319 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:41:54.360241    2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9b9e10ab-088e-402b-9db2-7cd03956b305" path="/var/lib/kubelet/pods/9b9e10ab-088e-402b-9db2-7cd03956b305/volumes"
Sep 09 03:41:54.360571 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:41:54.360558    2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c02969f1-8ac8-4c3d-9b70-c4f5f29a6c89" path="/var/lib/kubelet/pods/c02969f1-8ac8-4c3d-9b70-c4f5f29a6c89/volumes"
Sep 09 03:42:18.295127 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:42:18.295096    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-a-46be70-df94b6ffd-5pkhq"]
Sep 09 03:42:18.295508 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:42:18.295437    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9b9e10ab-088e-402b-9db2-7cd03956b305" containerName="revoke-keys"
Sep 09 03:42:18.295508 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:42:18.295447    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="9b9e10ab-088e-402b-9db2-7cd03956b305" containerName="revoke-keys"
Sep 09 03:42:18.295508 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:42:18.295456    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c02969f1-8ac8-4c3d-9b70-c4f5f29a6c89" containerName="payload-pre-processing"
Sep 09 03:42:18.295508 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:42:18.295461    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="c02969f1-8ac8-4c3d-9b70-c4f5f29a6c89" containerName="payload-pre-processing"
Sep 09 03:42:18.295647 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:42:18.295512    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="c02969f1-8ac8-4c3d-9b70-c4f5f29a6c89" containerName="payload-pre-processing"
Sep 09 03:42:18.295647 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:42:18.295522    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="9b9e10ab-088e-402b-9db2-7cd03956b305" containerName="revoke-keys"
Sep 09 03:42:18.298433 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:42:18.298416    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-46be70-df94b6ffd-5pkhq"
Sep 09 03:42:18.303099 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:42:18.303074    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-ipp-a-46be70-dockercfg-zr6jv\""
Sep 09 03:42:18.303211 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:42:18.303078    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-ipp-a-46be70\""
Sep 09 03:42:18.310200 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:42:18.310179    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-a-46be70-df94b6ffd-5pkhq"]
Sep 09 03:42:18.370484 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:42:18.370445    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-a-46be70-6f6d44b989-mzq7p"]
Sep 09 03:42:18.373767 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:42:18.373745    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-a-46be70-6f6d44b989-mzq7p"
Sep 09 03:42:18.387604 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:42:18.387579    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-a-46be70-6f6d44b989-mzq7p"]
Sep 09 03:42:18.442092 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:42:18.442058    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zdn7n\" (UniqueName: \"kubernetes.io/projected/3b404576-3ead-4c98-a8b2-673544c7c2c8-kube-api-access-zdn7n\") pod \"payload-processing-e2e-ipp-a-46be70-6f6d44b989-mzq7p\" (UID: \"3b404576-3ead-4c98-a8b2-673544c7c2c8\") " pod="openshift-ingress/payload-processing-e2e-ipp-a-46be70-6f6d44b989-mzq7p"
Sep 09 03:42:18.442281 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:42:18.442107    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/56fd035b-c418-489d-9de2-e2c12c1ea974-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-a-46be70-df94b6ffd-5pkhq\" (UID: \"56fd035b-c418-489d-9de2-e2c12c1ea974\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-46be70-df94b6ffd-5pkhq"
Sep 09 03:42:18.442281 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:42:18.442216    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/3b404576-3ead-4c98-a8b2-673544c7c2c8-plugins-config-volume\") pod \"payload-processing-e2e-ipp-a-46be70-6f6d44b989-mzq7p\" (UID: \"3b404576-3ead-4c98-a8b2-673544c7c2c8\") " pod="openshift-ingress/payload-processing-e2e-ipp-a-46be70-6f6d44b989-mzq7p"
Sep 09 03:42:18.442281 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:42:18.442248    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bjzsx\" (UniqueName: \"kubernetes.io/projected/56fd035b-c418-489d-9de2-e2c12c1ea974-kube-api-access-bjzsx\") pod \"payload-pre-processing-e2e-ipp-a-46be70-df94b6ffd-5pkhq\" (UID: \"56fd035b-c418-489d-9de2-e2c12c1ea974\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-46be70-df94b6ffd-5pkhq"
Sep 09 03:42:18.542936 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:42:18.542902    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/3b404576-3ead-4c98-a8b2-673544c7c2c8-plugins-config-volume\") pod \"payload-processing-e2e-ipp-a-46be70-6f6d44b989-mzq7p\" (UID: \"3b404576-3ead-4c98-a8b2-673544c7c2c8\") " pod="openshift-ingress/payload-processing-e2e-ipp-a-46be70-6f6d44b989-mzq7p"
Sep 09 03:42:18.543100 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:42:18.542947    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bjzsx\" (UniqueName: \"kubernetes.io/projected/56fd035b-c418-489d-9de2-e2c12c1ea974-kube-api-access-bjzsx\") pod \"payload-pre-processing-e2e-ipp-a-46be70-df94b6ffd-5pkhq\" (UID: \"56fd035b-c418-489d-9de2-e2c12c1ea974\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-46be70-df94b6ffd-5pkhq"
Sep 09 03:42:18.543177 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:42:18.543098    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zdn7n\" (UniqueName: \"kubernetes.io/projected/3b404576-3ead-4c98-a8b2-673544c7c2c8-kube-api-access-zdn7n\") pod \"payload-processing-e2e-ipp-a-46be70-6f6d44b989-mzq7p\" (UID: \"3b404576-3ead-4c98-a8b2-673544c7c2c8\") " pod="openshift-ingress/payload-processing-e2e-ipp-a-46be70-6f6d44b989-mzq7p"
Sep 09 03:42:18.543177 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:42:18.543155    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/56fd035b-c418-489d-9de2-e2c12c1ea974-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-a-46be70-df94b6ffd-5pkhq\" (UID: \"56fd035b-c418-489d-9de2-e2c12c1ea974\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-46be70-df94b6ffd-5pkhq"
Sep 09 03:42:18.543615 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:42:18.543597    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/3b404576-3ead-4c98-a8b2-673544c7c2c8-plugins-config-volume\") pod \"payload-processing-e2e-ipp-a-46be70-6f6d44b989-mzq7p\" (UID: \"3b404576-3ead-4c98-a8b2-673544c7c2c8\") " pod="openshift-ingress/payload-processing-e2e-ipp-a-46be70-6f6d44b989-mzq7p"
Sep 09 03:42:18.543858 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:42:18.543840    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/56fd035b-c418-489d-9de2-e2c12c1ea974-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-a-46be70-df94b6ffd-5pkhq\" (UID: \"56fd035b-c418-489d-9de2-e2c12c1ea974\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-46be70-df94b6ffd-5pkhq"
Sep 09 03:42:18.552622 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:42:18.552565    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bjzsx\" (UniqueName: \"kubernetes.io/projected/56fd035b-c418-489d-9de2-e2c12c1ea974-kube-api-access-bjzsx\") pod \"payload-pre-processing-e2e-ipp-a-46be70-df94b6ffd-5pkhq\" (UID: \"56fd035b-c418-489d-9de2-e2c12c1ea974\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-46be70-df94b6ffd-5pkhq"
Sep 09 03:42:18.552622 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:42:18.552590    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zdn7n\" (UniqueName: \"kubernetes.io/projected/3b404576-3ead-4c98-a8b2-673544c7c2c8-kube-api-access-zdn7n\") pod \"payload-processing-e2e-ipp-a-46be70-6f6d44b989-mzq7p\" (UID: \"3b404576-3ead-4c98-a8b2-673544c7c2c8\") " pod="openshift-ingress/payload-processing-e2e-ipp-a-46be70-6f6d44b989-mzq7p"
Sep 09 03:42:18.608558 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:42:18.608521    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-46be70-df94b6ffd-5pkhq"
Sep 09 03:42:18.684200 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:42:18.684178    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-a-46be70-6f6d44b989-mzq7p"
Sep 09 03:42:18.888103 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:42:18.888076    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-a-46be70-df94b6ffd-5pkhq"]
Sep 09 03:42:18.889339 ip-10-0-132-14 kubenswrapper[2576]: W0909 03:42:18.889317    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod56fd035b_c418_489d_9de2_e2c12c1ea974.slice/crio-9a9e26e5c489d0dc62fa6a07eb64191661dd54a804106be1ff5674d30402a806 WatchSource:0}: Error finding container 9a9e26e5c489d0dc62fa6a07eb64191661dd54a804106be1ff5674d30402a806: Status 404 returned error can't find the container with id 9a9e26e5c489d0dc62fa6a07eb64191661dd54a804106be1ff5674d30402a806
Sep 09 03:42:18.960414 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:42:18.960390    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-a-46be70-6f6d44b989-mzq7p"]
Sep 09 03:42:18.961938 ip-10-0-132-14 kubenswrapper[2576]: W0909 03:42:18.961914    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3b404576_3ead_4c98_a8b2_673544c7c2c8.slice/crio-0c4fe2ca7615bd4c18244a2018c95d61bdcb81e49bb599852d701bb7100b086b WatchSource:0}: Error finding container 0c4fe2ca7615bd4c18244a2018c95d61bdcb81e49bb599852d701bb7100b086b: Status 404 returned error can't find the container with id 0c4fe2ca7615bd4c18244a2018c95d61bdcb81e49bb599852d701bb7100b086b
Sep 09 03:42:19.125779 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:42:19.125746    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-46be70-df94b6ffd-5pkhq" event={"ID":"56fd035b-c418-489d-9de2-e2c12c1ea974","Type":"ContainerStarted","Data":"827585a1081764e4e53b559fdf38760f256297156ad07b6fd4e0ccaace62f64c"}
Sep 09 03:42:19.125978 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:42:19.125818    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-46be70-df94b6ffd-5pkhq" event={"ID":"56fd035b-c418-489d-9de2-e2c12c1ea974","Type":"ContainerStarted","Data":"9a9e26e5c489d0dc62fa6a07eb64191661dd54a804106be1ff5674d30402a806"}
Sep 09 03:42:19.125978 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:42:19.125968    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-46be70-df94b6ffd-5pkhq"
Sep 09 03:42:19.127433 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:42:19.127410    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-a-46be70-6f6d44b989-mzq7p" event={"ID":"3b404576-3ead-4c98-a8b2-673544c7c2c8","Type":"ContainerStarted","Data":"f31642a79ffc9e20a537bf40ae2ba6fe4ffb07cdd07fb8526d9f0dc607d58138"}
Sep 09 03:42:19.127546 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:42:19.127448    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-a-46be70-6f6d44b989-mzq7p" event={"ID":"3b404576-3ead-4c98-a8b2-673544c7c2c8","Type":"ContainerStarted","Data":"0c4fe2ca7615bd4c18244a2018c95d61bdcb81e49bb599852d701bb7100b086b"}
Sep 09 03:42:19.127546 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:42:19.127483    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-ipp-a-46be70-6f6d44b989-mzq7p"
Sep 09 03:42:19.153684 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:42:19.153605    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-46be70-df94b6ffd-5pkhq" podStartSLOduration=1.153592357 podStartE2EDuration="1.153592357s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 03:42:18 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 03:42:19.152072642 +0000 UTC m=+1753.293558718" watchObservedRunningTime="2026-09-09 03:42:19.153592357 +0000 UTC m=+1753.295078479"
Sep 09 03:42:19.175136 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:42:19.175097    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-ipp-a-46be70-6f6d44b989-mzq7p" podStartSLOduration=1.175083605 podStartE2EDuration="1.175083605s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 03:42:18 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 03:42:19.174195295 +0000 UTC m=+1753.315681372" watchObservedRunningTime="2026-09-09 03:42:19.175083605 +0000 UTC m=+1753.316569681"
Sep 09 03:42:30.133951 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:42:30.133912    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-46be70-df94b6ffd-5pkhq"
Sep 09 03:42:30.134337 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:42:30.134004    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-ipp-a-46be70-6f6d44b989-mzq7p"
Sep 09 03:42:47.417159 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:42:47.417126    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-b-60a731-6f76c7bbd6-2cpzm"]
Sep 09 03:42:47.420423 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:42:47.420406    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-60a731-6f76c7bbd6-2cpzm"
Sep 09 03:42:47.425788 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:42:47.425766    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-ipp-b-60a731-dockercfg-wf2ht\""
Sep 09 03:42:47.426120 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:42:47.426103    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-ipp-b-60a731\""
Sep 09 03:42:47.434503 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:42:47.434473    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-b-60a731-6f76c7bbd6-2cpzm"]
Sep 09 03:42:47.593708 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:42:47.593667    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ldxzp\" (UniqueName: \"kubernetes.io/projected/c1eb89c9-623b-49ba-9b54-61c2744da4b8-kube-api-access-ldxzp\") pod \"payload-pre-processing-e2e-ipp-b-60a731-6f76c7bbd6-2cpzm\" (UID: \"c1eb89c9-623b-49ba-9b54-61c2744da4b8\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-60a731-6f76c7bbd6-2cpzm"
Sep 09 03:42:47.593912 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:42:47.593741    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c1eb89c9-623b-49ba-9b54-61c2744da4b8-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-b-60a731-6f76c7bbd6-2cpzm\" (UID: \"c1eb89c9-623b-49ba-9b54-61c2744da4b8\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-60a731-6f76c7bbd6-2cpzm"
Sep 09 03:42:47.694659 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:42:47.694588    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c1eb89c9-623b-49ba-9b54-61c2744da4b8-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-b-60a731-6f76c7bbd6-2cpzm\" (UID: \"c1eb89c9-623b-49ba-9b54-61c2744da4b8\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-60a731-6f76c7bbd6-2cpzm"
Sep 09 03:42:47.694659 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:42:47.694649    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ldxzp\" (UniqueName: \"kubernetes.io/projected/c1eb89c9-623b-49ba-9b54-61c2744da4b8-kube-api-access-ldxzp\") pod \"payload-pre-processing-e2e-ipp-b-60a731-6f76c7bbd6-2cpzm\" (UID: \"c1eb89c9-623b-49ba-9b54-61c2744da4b8\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-60a731-6f76c7bbd6-2cpzm"
Sep 09 03:42:47.695301 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:42:47.695278    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c1eb89c9-623b-49ba-9b54-61c2744da4b8-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-b-60a731-6f76c7bbd6-2cpzm\" (UID: \"c1eb89c9-623b-49ba-9b54-61c2744da4b8\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-60a731-6f76c7bbd6-2cpzm"
Sep 09 03:42:47.705106 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:42:47.705078    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ldxzp\" (UniqueName: \"kubernetes.io/projected/c1eb89c9-623b-49ba-9b54-61c2744da4b8-kube-api-access-ldxzp\") pod \"payload-pre-processing-e2e-ipp-b-60a731-6f76c7bbd6-2cpzm\" (UID: \"c1eb89c9-623b-49ba-9b54-61c2744da4b8\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-60a731-6f76c7bbd6-2cpzm"
Sep 09 03:42:47.730940 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:42:47.730907    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-60a731-6f76c7bbd6-2cpzm"
Sep 09 03:42:48.002736 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:42:48.002709    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-b-60a731-6f76c7bbd6-2cpzm"]
Sep 09 03:42:48.004500 ip-10-0-132-14 kubenswrapper[2576]: W0909 03:42:48.004475    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc1eb89c9_623b_49ba_9b54_61c2744da4b8.slice/crio-24ab0cc199bb6cc252a05235aec08426a6fe651d6fee6d21ffbdf90a604cf579 WatchSource:0}: Error finding container 24ab0cc199bb6cc252a05235aec08426a6fe651d6fee6d21ffbdf90a604cf579: Status 404 returned error can't find the container with id 24ab0cc199bb6cc252a05235aec08426a6fe651d6fee6d21ffbdf90a604cf579
Sep 09 03:42:48.269448 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:42:48.269372    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-60a731-6f76c7bbd6-2cpzm" event={"ID":"c1eb89c9-623b-49ba-9b54-61c2744da4b8","Type":"ContainerStarted","Data":"7b6727a23f8ccb3f3443fad999f30fbe9c5480c84d5bfa76a4227ef6cc0a2c8f"}
Sep 09 03:42:48.269448 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:42:48.269431    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-60a731-6f76c7bbd6-2cpzm" event={"ID":"c1eb89c9-623b-49ba-9b54-61c2744da4b8","Type":"ContainerStarted","Data":"24ab0cc199bb6cc252a05235aec08426a6fe651d6fee6d21ffbdf90a604cf579"}
Sep 09 03:42:48.269619 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:42:48.269457    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-60a731-6f76c7bbd6-2cpzm"
Sep 09 03:42:48.287993 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:42:48.287949    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-60a731-6f76c7bbd6-2cpzm" podStartSLOduration=1.287933821 podStartE2EDuration="1.287933821s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 03:42:47 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 03:42:48.28684729 +0000 UTC m=+1782.428333369" watchObservedRunningTime="2026-09-09 03:42:48.287933821 +0000 UTC m=+1782.429419898"
Sep 09 03:42:59.276777 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:42:59.276697    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-60a731-6f76c7bbd6-2cpzm"
Sep 09 03:43:15.768861 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:43:15.768827    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5b54b7fbb7-x6t9q_701775d1-64f8-4057-b7ee-5aab7fb98ac1/payload-processing/0.log"
Sep 09 03:43:51.636987 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:43:51.636955    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-e2e-ipp-a-46be70-6f6d44b989-mzq7p_3b404576-3ead-4c98-a8b2-673544c7c2c8/payload-processing/0.log"
Sep 09 03:43:51.764600 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:43:51.764577    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5b54b7fbb7-x6t9q_701775d1-64f8-4057-b7ee-5aab7fb98ac1/payload-processing/0.log"
Sep 09 03:43:57.588362 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:43:57.588328    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-65a459-8456559c94-dfs7l"]
Sep 09 03:43:57.591466 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:43:57.591451    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-65a459-8456559c94-dfs7l"
Sep 09 03:43:57.595581 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:43:57.595548    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-ipp-cleanup-65a459\""
Sep 09 03:43:57.595722 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:43:57.595648    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-ipp-cleanup-65a459-dockercfg-gc7xw\""
Sep 09 03:43:57.602549 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:43:57.602524    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-65a459-8456559c94-dfs7l"]
Sep 09 03:43:57.618643 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:43:57.618615    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-cleanup-65a459-64559bb45f-s8x7c"]
Sep 09 03:43:57.622500 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:43:57.622481    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-65a459-64559bb45f-s8x7c"
Sep 09 03:43:57.635362 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:43:57.635341    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-cleanup-65a459-64559bb45f-s8x7c"]
Sep 09 03:43:57.641712 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:43:57.641690    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/72542bef-d0a0-40e2-950d-f9ec77acf84b-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-cleanup-65a459-8456559c94-dfs7l\" (UID: \"72542bef-d0a0-40e2-950d-f9ec77acf84b\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-65a459-8456559c94-dfs7l"
Sep 09 03:43:57.641862 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:43:57.641716    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f259a432-3949-4e4d-8203-12be3552545f-plugins-config-volume\") pod \"payload-processing-e2e-ipp-cleanup-65a459-64559bb45f-s8x7c\" (UID: \"f259a432-3949-4e4d-8203-12be3552545f\") " pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-65a459-64559bb45f-s8x7c"
Sep 09 03:43:57.641862 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:43:57.641732    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vxdtb\" (UniqueName: \"kubernetes.io/projected/f259a432-3949-4e4d-8203-12be3552545f-kube-api-access-vxdtb\") pod \"payload-processing-e2e-ipp-cleanup-65a459-64559bb45f-s8x7c\" (UID: \"f259a432-3949-4e4d-8203-12be3552545f\") " pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-65a459-64559bb45f-s8x7c"
Sep 09 03:43:57.641862 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:43:57.641750    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5z5fg\" (UniqueName: \"kubernetes.io/projected/72542bef-d0a0-40e2-950d-f9ec77acf84b-kube-api-access-5z5fg\") pod \"payload-pre-processing-e2e-ipp-cleanup-65a459-8456559c94-dfs7l\" (UID: \"72542bef-d0a0-40e2-950d-f9ec77acf84b\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-65a459-8456559c94-dfs7l"
Sep 09 03:43:57.742729 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:43:57.742700    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/72542bef-d0a0-40e2-950d-f9ec77acf84b-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-cleanup-65a459-8456559c94-dfs7l\" (UID: \"72542bef-d0a0-40e2-950d-f9ec77acf84b\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-65a459-8456559c94-dfs7l"
Sep 09 03:43:57.742729 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:43:57.742732    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f259a432-3949-4e4d-8203-12be3552545f-plugins-config-volume\") pod \"payload-processing-e2e-ipp-cleanup-65a459-64559bb45f-s8x7c\" (UID: \"f259a432-3949-4e4d-8203-12be3552545f\") " pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-65a459-64559bb45f-s8x7c"
Sep 09 03:43:57.743007 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:43:57.742757    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vxdtb\" (UniqueName: \"kubernetes.io/projected/f259a432-3949-4e4d-8203-12be3552545f-kube-api-access-vxdtb\") pod \"payload-processing-e2e-ipp-cleanup-65a459-64559bb45f-s8x7c\" (UID: \"f259a432-3949-4e4d-8203-12be3552545f\") " pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-65a459-64559bb45f-s8x7c"
Sep 09 03:43:57.743007 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:43:57.742787    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5z5fg\" (UniqueName: \"kubernetes.io/projected/72542bef-d0a0-40e2-950d-f9ec77acf84b-kube-api-access-5z5fg\") pod \"payload-pre-processing-e2e-ipp-cleanup-65a459-8456559c94-dfs7l\" (UID: \"72542bef-d0a0-40e2-950d-f9ec77acf84b\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-65a459-8456559c94-dfs7l"
Sep 09 03:43:57.743469 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:43:57.743450    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/72542bef-d0a0-40e2-950d-f9ec77acf84b-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-cleanup-65a459-8456559c94-dfs7l\" (UID: \"72542bef-d0a0-40e2-950d-f9ec77acf84b\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-65a459-8456559c94-dfs7l"
Sep 09 03:43:57.743529 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:43:57.743511    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f259a432-3949-4e4d-8203-12be3552545f-plugins-config-volume\") pod \"payload-processing-e2e-ipp-cleanup-65a459-64559bb45f-s8x7c\" (UID: \"f259a432-3949-4e4d-8203-12be3552545f\") " pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-65a459-64559bb45f-s8x7c"
Sep 09 03:43:57.751462 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:43:57.751426    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vxdtb\" (UniqueName: \"kubernetes.io/projected/f259a432-3949-4e4d-8203-12be3552545f-kube-api-access-vxdtb\") pod \"payload-processing-e2e-ipp-cleanup-65a459-64559bb45f-s8x7c\" (UID: \"f259a432-3949-4e4d-8203-12be3552545f\") " pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-65a459-64559bb45f-s8x7c"
Sep 09 03:43:57.751462 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:43:57.751446    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5z5fg\" (UniqueName: \"kubernetes.io/projected/72542bef-d0a0-40e2-950d-f9ec77acf84b-kube-api-access-5z5fg\") pod \"payload-pre-processing-e2e-ipp-cleanup-65a459-8456559c94-dfs7l\" (UID: \"72542bef-d0a0-40e2-950d-f9ec77acf84b\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-65a459-8456559c94-dfs7l"
Sep 09 03:43:57.902729 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:43:57.902638    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-65a459-8456559c94-dfs7l"
Sep 09 03:43:57.932922 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:43:57.932893    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-65a459-64559bb45f-s8x7c"
Sep 09 03:43:58.177411 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:43:58.177386    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-65a459-8456559c94-dfs7l"]
Sep 09 03:43:58.179339 ip-10-0-132-14 kubenswrapper[2576]: W0909 03:43:58.179316    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod72542bef_d0a0_40e2_950d_f9ec77acf84b.slice/crio-694335ecb743ca7dbb5094dde6f2baa3254505c708b17abd39f993ecd88577dc WatchSource:0}: Error finding container 694335ecb743ca7dbb5094dde6f2baa3254505c708b17abd39f993ecd88577dc: Status 404 returned error can't find the container with id 694335ecb743ca7dbb5094dde6f2baa3254505c708b17abd39f993ecd88577dc
Sep 09 03:43:58.218509 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:43:58.218492    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-cleanup-65a459-64559bb45f-s8x7c"]
Sep 09 03:43:58.219747 ip-10-0-132-14 kubenswrapper[2576]: W0909 03:43:58.219713    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf259a432_3949_4e4d_8203_12be3552545f.slice/crio-36f04913743a556a2e9c9a83b8b5b1fbe26bb4c8ef639f421462de83b68669e9 WatchSource:0}: Error finding container 36f04913743a556a2e9c9a83b8b5b1fbe26bb4c8ef639f421462de83b68669e9: Status 404 returned error can't find the container with id 36f04913743a556a2e9c9a83b8b5b1fbe26bb4c8ef639f421462de83b68669e9
Sep 09 03:43:58.611477 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:43:58.611441    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-65a459-8456559c94-dfs7l" event={"ID":"72542bef-d0a0-40e2-950d-f9ec77acf84b","Type":"ContainerStarted","Data":"c04b471d92cafd9ec48f693adde73cf6e5b0e8d905330b597da95cca67331861"}
Sep 09 03:43:58.611934 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:43:58.611495    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-65a459-8456559c94-dfs7l" event={"ID":"72542bef-d0a0-40e2-950d-f9ec77acf84b","Type":"ContainerStarted","Data":"694335ecb743ca7dbb5094dde6f2baa3254505c708b17abd39f993ecd88577dc"}
Sep 09 03:43:58.611934 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:43:58.611660    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-65a459-8456559c94-dfs7l"
Sep 09 03:43:58.613154 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:43:58.613127    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-65a459-64559bb45f-s8x7c" event={"ID":"f259a432-3949-4e4d-8203-12be3552545f","Type":"ContainerStarted","Data":"e059424705d3cf4e1a1a6f2f4e3f70fae716265613c68f23d68c7996fab462dc"}
Sep 09 03:43:58.613274 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:43:58.613175    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-65a459-64559bb45f-s8x7c" event={"ID":"f259a432-3949-4e4d-8203-12be3552545f","Type":"ContainerStarted","Data":"36f04913743a556a2e9c9a83b8b5b1fbe26bb4c8ef639f421462de83b68669e9"}
Sep 09 03:43:58.613274 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:43:58.613195    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-65a459-64559bb45f-s8x7c"
Sep 09 03:43:58.630199 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:43:58.630137    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-65a459-8456559c94-dfs7l" podStartSLOduration=1.630124709 podStartE2EDuration="1.630124709s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 03:43:57 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 03:43:58.627882915 +0000 UTC m=+1852.769369008" watchObservedRunningTime="2026-09-09 03:43:58.630124709 +0000 UTC m=+1852.771610783"
Sep 09 03:43:58.645446 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:43:58.645406    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-65a459-64559bb45f-s8x7c" podStartSLOduration=1.645389925 podStartE2EDuration="1.645389925s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 03:43:57 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 03:43:58.644971062 +0000 UTC m=+1852.786457141" watchObservedRunningTime="2026-09-09 03:43:58.645389925 +0000 UTC m=+1852.786876004"
Sep 09 03:44:06.672478 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:44:06.672445    2576 scope.go:117] "RemoveContainer" containerID="1ee8876fb7b41825f917acf15e3e23c4deb83a6140d5d52f002ac0b9ae11a473"
Sep 09 03:44:09.619776 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:44:09.619748    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-65a459-8456559c94-dfs7l"
Sep 09 03:44:09.620219 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:44:09.619839    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-65a459-64559bb45f-s8x7c"
Sep 09 03:44:21.643912 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:44:21.643869    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-65a459-x7d98"]
Sep 09 03:44:21.647483 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:44:21.647459    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-65a459-x7d98"
Sep 09 03:44:21.650359 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:44:21.650337    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-w7q9f\""
Sep 09 03:44:21.655569 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:44:21.655542    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-65a459-x7d98"]
Sep 09 03:44:21.734291 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:44:21.734238    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/37878fdd-4efb-4a68-b881-bf2a524e2d2e-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-cleanup-65a459-x7d98\" (UID: \"37878fdd-4efb-4a68-b881-bf2a524e2d2e\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-65a459-x7d98"
Sep 09 03:44:21.834873 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:44:21.834844    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/37878fdd-4efb-4a68-b881-bf2a524e2d2e-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-cleanup-65a459-x7d98\" (UID: \"37878fdd-4efb-4a68-b881-bf2a524e2d2e\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-65a459-x7d98"
Sep 09 03:44:21.835451 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:44:21.835432    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/37878fdd-4efb-4a68-b881-bf2a524e2d2e-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-cleanup-65a459-x7d98\" (UID: \"37878fdd-4efb-4a68-b881-bf2a524e2d2e\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-65a459-x7d98"
Sep 09 03:44:21.959566 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:44:21.959487    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-65a459-x7d98"
Sep 09 03:44:22.226669 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:44:22.226463    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-65a459-x7d98"]
Sep 09 03:44:22.229041 ip-10-0-132-14 kubenswrapper[2576]: W0909 03:44:22.229014    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod37878fdd_4efb_4a68_b881_bf2a524e2d2e.slice/crio-5223d596f2c12f4ccf6b950fe14e084c3c58f81b550cbeb79a81f0c5c403c314 WatchSource:0}: Error finding container 5223d596f2c12f4ccf6b950fe14e084c3c58f81b550cbeb79a81f0c5c403c314: Status 404 returned error can't find the container with id 5223d596f2c12f4ccf6b950fe14e084c3c58f81b550cbeb79a81f0c5c403c314
Sep 09 03:44:22.730564 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:44:22.730532    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-65a459-x7d98" event={"ID":"37878fdd-4efb-4a68-b881-bf2a524e2d2e","Type":"ContainerStarted","Data":"5af75ca0f477942ee1a99f03984370e7fdf59428c5399fbb4ae8e503f28d6cd6"}
Sep 09 03:44:22.731017 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:44:22.730595    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-65a459-x7d98" event={"ID":"37878fdd-4efb-4a68-b881-bf2a524e2d2e","Type":"ContainerStarted","Data":"5223d596f2c12f4ccf6b950fe14e084c3c58f81b550cbeb79a81f0c5c403c314"}
Sep 09 03:44:22.748530 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:44:22.748483    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-65a459-x7d98" podStartSLOduration=1.74846894 podStartE2EDuration="1.74846894s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 03:44:21 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 03:44:22.747003926 +0000 UTC m=+1876.888490006" watchObservedRunningTime="2026-09-09 03:44:22.74846894 +0000 UTC m=+1876.889955017"
Sep 09 03:44:23.736960 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:44:23.736929    2576 generic.go:356] "Generic (PLEG): container finished" podID="37878fdd-4efb-4a68-b881-bf2a524e2d2e" containerID="5af75ca0f477942ee1a99f03984370e7fdf59428c5399fbb4ae8e503f28d6cd6" exitCode=0
Sep 09 03:44:23.737360 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:44:23.737018    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-65a459-x7d98" event={"ID":"37878fdd-4efb-4a68-b881-bf2a524e2d2e","Type":"ContainerDied","Data":"5af75ca0f477942ee1a99f03984370e7fdf59428c5399fbb4ae8e503f28d6cd6"}
Sep 09 03:44:24.876305 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:44:24.876280    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-65a459-x7d98"
Sep 09 03:44:24.959439 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:44:24.959416    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/37878fdd-4efb-4a68-b881-bf2a524e2d2e-maas-api-service-ca\") pod \"37878fdd-4efb-4a68-b881-bf2a524e2d2e\" (UID: \"37878fdd-4efb-4a68-b881-bf2a524e2d2e\") "
Sep 09 03:44:24.959766 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:44:24.959739    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/37878fdd-4efb-4a68-b881-bf2a524e2d2e-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "37878fdd-4efb-4a68-b881-bf2a524e2d2e" (UID: "37878fdd-4efb-4a68-b881-bf2a524e2d2e"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 09 03:44:25.060598 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:44:25.060568    2576 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/37878fdd-4efb-4a68-b881-bf2a524e2d2e-maas-api-service-ca\") on node \"ip-10-0-132-14.ec2.internal\" DevicePath \"\""
Sep 09 03:44:25.748401 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:44:25.748369    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-65a459-x7d98"
Sep 09 03:44:25.748566 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:44:25.748422    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-65a459-x7d98" event={"ID":"37878fdd-4efb-4a68-b881-bf2a524e2d2e","Type":"ContainerDied","Data":"5223d596f2c12f4ccf6b950fe14e084c3c58f81b550cbeb79a81f0c5c403c314"}
Sep 09 03:44:25.748566 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:44:25.748466    2576 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="5223d596f2c12f4ccf6b950fe14e084c3c58f81b550cbeb79a81f0c5c403c314"
Sep 09 03:44:27.581315 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:44:27.581217    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-cleanup-65a459-64559bb45f-s8x7c"]
Sep 09 03:44:27.581747 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:44:27.581417    2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-65a459-64559bb45f-s8x7c" podUID="f259a432-3949-4e4d-8203-12be3552545f" containerName="payload-processing" containerID="cri-o://e059424705d3cf4e1a1a6f2f4e3f70fae716265613c68f23d68c7996fab462dc" gracePeriod=30
Sep 09 03:44:27.595681 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:44:27.595658    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-65a459-8456559c94-dfs7l"]
Sep 09 03:44:27.595930 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:44:27.595905    2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-65a459-8456559c94-dfs7l" podUID="72542bef-d0a0-40e2-950d-f9ec77acf84b" containerName="payload-pre-processing" containerID="cri-o://c04b471d92cafd9ec48f693adde73cf6e5b0e8d905330b597da95cca67331861" gracePeriod=30
Sep 09 03:44:27.764101 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:44:27.764071    2576 generic.go:356] "Generic (PLEG): container finished" podID="72542bef-d0a0-40e2-950d-f9ec77acf84b" containerID="c04b471d92cafd9ec48f693adde73cf6e5b0e8d905330b597da95cca67331861" exitCode=0
Sep 09 03:44:27.764325 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:44:27.764245    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-65a459-8456559c94-dfs7l" event={"ID":"72542bef-d0a0-40e2-950d-f9ec77acf84b","Type":"ContainerDied","Data":"c04b471d92cafd9ec48f693adde73cf6e5b0e8d905330b597da95cca67331861"}
Sep 09 03:44:27.766255 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:44:27.766228    2576 generic.go:356] "Generic (PLEG): container finished" podID="f259a432-3949-4e4d-8203-12be3552545f" containerID="e059424705d3cf4e1a1a6f2f4e3f70fae716265613c68f23d68c7996fab462dc" exitCode=0
Sep 09 03:44:27.766368 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:44:27.766281    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-65a459-64559bb45f-s8x7c" event={"ID":"f259a432-3949-4e4d-8203-12be3552545f","Type":"ContainerDied","Data":"e059424705d3cf4e1a1a6f2f4e3f70fae716265613c68f23d68c7996fab462dc"}
Sep 09 03:44:27.782233 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:44:27.782209    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-65a459-x7d98"]
Sep 09 03:44:27.787907 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:44:27.787885    2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-65a459-x7d98"]
Sep 09 03:44:27.865868 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:44:27.865844    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-65a459-64559bb45f-s8x7c"
Sep 09 03:44:27.868582 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:44:27.868567    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-65a459-8456559c94-dfs7l"
Sep 09 03:44:27.984018 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:44:27.983983    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-5z5fg\" (UniqueName: \"kubernetes.io/projected/72542bef-d0a0-40e2-950d-f9ec77acf84b-kube-api-access-5z5fg\") pod \"72542bef-d0a0-40e2-950d-f9ec77acf84b\" (UID: \"72542bef-d0a0-40e2-950d-f9ec77acf84b\") "
Sep 09 03:44:27.984193 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:44:27.984044    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-vxdtb\" (UniqueName: \"kubernetes.io/projected/f259a432-3949-4e4d-8203-12be3552545f-kube-api-access-vxdtb\") pod \"f259a432-3949-4e4d-8203-12be3552545f\" (UID: \"f259a432-3949-4e4d-8203-12be3552545f\") "
Sep 09 03:44:27.984193 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:44:27.984092    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f259a432-3949-4e4d-8203-12be3552545f-plugins-config-volume\") pod \"f259a432-3949-4e4d-8203-12be3552545f\" (UID: \"f259a432-3949-4e4d-8203-12be3552545f\") "
Sep 09 03:44:27.984193 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:44:27.984122    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/72542bef-d0a0-40e2-950d-f9ec77acf84b-plugins-config-volume\") pod \"72542bef-d0a0-40e2-950d-f9ec77acf84b\" (UID: \"72542bef-d0a0-40e2-950d-f9ec77acf84b\") "
Sep 09 03:44:27.984514 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:44:27.984485    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f259a432-3949-4e4d-8203-12be3552545f-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "f259a432-3949-4e4d-8203-12be3552545f" (UID: "f259a432-3949-4e4d-8203-12be3552545f"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 09 03:44:27.984660 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:44:27.984637    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/72542bef-d0a0-40e2-950d-f9ec77acf84b-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "72542bef-d0a0-40e2-950d-f9ec77acf84b" (UID: "72542bef-d0a0-40e2-950d-f9ec77acf84b"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 09 03:44:27.986279 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:44:27.986256    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/72542bef-d0a0-40e2-950d-f9ec77acf84b-kube-api-access-5z5fg" (OuterVolumeSpecName: "kube-api-access-5z5fg") pod "72542bef-d0a0-40e2-950d-f9ec77acf84b" (UID: "72542bef-d0a0-40e2-950d-f9ec77acf84b"). InnerVolumeSpecName "kube-api-access-5z5fg". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 09 03:44:27.986551 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:44:27.986533    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f259a432-3949-4e4d-8203-12be3552545f-kube-api-access-vxdtb" (OuterVolumeSpecName: "kube-api-access-vxdtb") pod "f259a432-3949-4e4d-8203-12be3552545f" (UID: "f259a432-3949-4e4d-8203-12be3552545f"). InnerVolumeSpecName "kube-api-access-vxdtb". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 09 03:44:28.085542 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:44:28.085511    2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5z5fg\" (UniqueName: \"kubernetes.io/projected/72542bef-d0a0-40e2-950d-f9ec77acf84b-kube-api-access-5z5fg\") on node \"ip-10-0-132-14.ec2.internal\" DevicePath \"\""
Sep 09 03:44:28.085542 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:44:28.085538    2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vxdtb\" (UniqueName: \"kubernetes.io/projected/f259a432-3949-4e4d-8203-12be3552545f-kube-api-access-vxdtb\") on node \"ip-10-0-132-14.ec2.internal\" DevicePath \"\""
Sep 09 03:44:28.085712 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:44:28.085553    2576 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f259a432-3949-4e4d-8203-12be3552545f-plugins-config-volume\") on node \"ip-10-0-132-14.ec2.internal\" DevicePath \"\""
Sep 09 03:44:28.085712 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:44:28.085566    2576 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/72542bef-d0a0-40e2-950d-f9ec77acf84b-plugins-config-volume\") on node \"ip-10-0-132-14.ec2.internal\" DevicePath \"\""
Sep 09 03:44:28.360958 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:44:28.360927    2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="37878fdd-4efb-4a68-b881-bf2a524e2d2e" path="/var/lib/kubelet/pods/37878fdd-4efb-4a68-b881-bf2a524e2d2e/volumes"
Sep 09 03:44:28.773039 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:44:28.773008    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-65a459-64559bb45f-s8x7c"
Sep 09 03:44:28.773039 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:44:28.773019    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-65a459-64559bb45f-s8x7c" event={"ID":"f259a432-3949-4e4d-8203-12be3552545f","Type":"ContainerDied","Data":"36f04913743a556a2e9c9a83b8b5b1fbe26bb4c8ef639f421462de83b68669e9"}
Sep 09 03:44:28.773495 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:44:28.773110    2576 scope.go:117] "RemoveContainer" containerID="e059424705d3cf4e1a1a6f2f4e3f70fae716265613c68f23d68c7996fab462dc"
Sep 09 03:44:28.774564 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:44:28.774504    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-65a459-8456559c94-dfs7l" event={"ID":"72542bef-d0a0-40e2-950d-f9ec77acf84b","Type":"ContainerDied","Data":"694335ecb743ca7dbb5094dde6f2baa3254505c708b17abd39f993ecd88577dc"}
Sep 09 03:44:28.774652 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:44:28.774533    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-65a459-8456559c94-dfs7l"
Sep 09 03:44:28.781692 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:44:28.781661    2576 scope.go:117] "RemoveContainer" containerID="c04b471d92cafd9ec48f693adde73cf6e5b0e8d905330b597da95cca67331861"
Sep 09 03:44:28.791112 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:44:28.791090    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-65a459-8456559c94-dfs7l"]
Sep 09 03:44:28.797219 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:44:28.797200    2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-65a459-8456559c94-dfs7l"]
Sep 09 03:44:28.809926 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:44:28.809903    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-cleanup-65a459-64559bb45f-s8x7c"]
Sep 09 03:44:28.816408 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:44:28.816390    2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-cleanup-65a459-64559bb45f-s8x7c"]
Sep 09 03:44:30.360044 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:44:30.360012    2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="72542bef-d0a0-40e2-950d-f9ec77acf84b" path="/var/lib/kubelet/pods/72542bef-d0a0-40e2-950d-f9ec77acf84b/volumes"
Sep 09 03:44:30.360398 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:44:30.360335    2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f259a432-3949-4e4d-8203-12be3552545f" path="/var/lib/kubelet/pods/f259a432-3949-4e4d-8203-12be3552545f/volumes"
Sep 09 03:44:38.188163 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:44:38.188120    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-46be70-dx7kq"]
Sep 09 03:44:38.188638 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:44:38.188509    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="72542bef-d0a0-40e2-950d-f9ec77acf84b" containerName="payload-pre-processing"
Sep 09 03:44:38.188638 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:44:38.188522    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="72542bef-d0a0-40e2-950d-f9ec77acf84b" containerName="payload-pre-processing"
Sep 09 03:44:38.188638 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:44:38.188531    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f259a432-3949-4e4d-8203-12be3552545f" containerName="payload-processing"
Sep 09 03:44:38.188638 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:44:38.188536    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="f259a432-3949-4e4d-8203-12be3552545f" containerName="payload-processing"
Sep 09 03:44:38.188638 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:44:38.188547    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="37878fdd-4efb-4a68-b881-bf2a524e2d2e" containerName="revoke-keys"
Sep 09 03:44:38.188638 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:44:38.188556    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="37878fdd-4efb-4a68-b881-bf2a524e2d2e" containerName="revoke-keys"
Sep 09 03:44:38.188638 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:44:38.188609    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="f259a432-3949-4e4d-8203-12be3552545f" containerName="payload-processing"
Sep 09 03:44:38.188638 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:44:38.188616    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="37878fdd-4efb-4a68-b881-bf2a524e2d2e" containerName="revoke-keys"
Sep 09 03:44:38.188638 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:44:38.188627    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="72542bef-d0a0-40e2-950d-f9ec77acf84b" containerName="payload-pre-processing"
Sep 09 03:44:38.193070 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:44:38.193049    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-46be70-dx7kq"
Sep 09 03:44:38.195754 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:44:38.195729    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-w7q9f\""
Sep 09 03:44:38.201444 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:44:38.201424    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-46be70-dx7kq"]
Sep 09 03:44:38.275749 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:44:38.275712    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/82d96bdc-8d0c-4433-be38-05635a0215d9-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-a-46be70-dx7kq\" (UID: \"82d96bdc-8d0c-4433-be38-05635a0215d9\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-46be70-dx7kq"
Sep 09 03:44:38.376744 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:44:38.376714    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/82d96bdc-8d0c-4433-be38-05635a0215d9-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-a-46be70-dx7kq\" (UID: \"82d96bdc-8d0c-4433-be38-05635a0215d9\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-46be70-dx7kq"
Sep 09 03:44:38.377292 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:44:38.377270    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/82d96bdc-8d0c-4433-be38-05635a0215d9-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-a-46be70-dx7kq\" (UID: \"82d96bdc-8d0c-4433-be38-05635a0215d9\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-46be70-dx7kq"
Sep 09 03:44:38.505169 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:44:38.505142    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-46be70-dx7kq"
Sep 09 03:44:38.776082 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:44:38.776059    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-46be70-dx7kq"]
Sep 09 03:44:38.777536 ip-10-0-132-14 kubenswrapper[2576]: W0909 03:44:38.777507    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod82d96bdc_8d0c_4433_be38_05635a0215d9.slice/crio-38f986d1f6e19164daf14e8d4b58c2a9ebe72200eeaed48f28784dfd92cfb2bf WatchSource:0}: Error finding container 38f986d1f6e19164daf14e8d4b58c2a9ebe72200eeaed48f28784dfd92cfb2bf: Status 404 returned error can't find the container with id 38f986d1f6e19164daf14e8d4b58c2a9ebe72200eeaed48f28784dfd92cfb2bf
Sep 09 03:44:38.824841 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:44:38.824785    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-46be70-dx7kq" event={"ID":"82d96bdc-8d0c-4433-be38-05635a0215d9","Type":"ContainerStarted","Data":"38f986d1f6e19164daf14e8d4b58c2a9ebe72200eeaed48f28784dfd92cfb2bf"}
Sep 09 03:44:39.832281 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:44:39.832247    2576 generic.go:356] "Generic (PLEG): container finished" podID="82d96bdc-8d0c-4433-be38-05635a0215d9" containerID="187e6427c85516b1f2ee51a72df2a304c2d9aa950f27a1efb8b28ed150b5f53f" exitCode=0
Sep 09 03:44:39.832704 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:44:39.832337    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-46be70-dx7kq" event={"ID":"82d96bdc-8d0c-4433-be38-05635a0215d9","Type":"ContainerDied","Data":"187e6427c85516b1f2ee51a72df2a304c2d9aa950f27a1efb8b28ed150b5f53f"}
Sep 09 03:44:40.963108 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:44:40.963086    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-46be70-dx7kq"
Sep 09 03:44:41.102705 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:44:41.102636    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/82d96bdc-8d0c-4433-be38-05635a0215d9-maas-api-service-ca\") pod \"82d96bdc-8d0c-4433-be38-05635a0215d9\" (UID: \"82d96bdc-8d0c-4433-be38-05635a0215d9\") "
Sep 09 03:44:41.102993 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:44:41.102971    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/82d96bdc-8d0c-4433-be38-05635a0215d9-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "82d96bdc-8d0c-4433-be38-05635a0215d9" (UID: "82d96bdc-8d0c-4433-be38-05635a0215d9"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 09 03:44:41.203289 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:44:41.203260    2576 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/82d96bdc-8d0c-4433-be38-05635a0215d9-maas-api-service-ca\") on node \"ip-10-0-132-14.ec2.internal\" DevicePath \"\""
Sep 09 03:44:41.842718 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:44:41.842690    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-46be70-dx7kq"
Sep 09 03:44:41.842920 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:44:41.842717    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-46be70-dx7kq" event={"ID":"82d96bdc-8d0c-4433-be38-05635a0215d9","Type":"ContainerDied","Data":"38f986d1f6e19164daf14e8d4b58c2a9ebe72200eeaed48f28784dfd92cfb2bf"}
Sep 09 03:44:41.842920 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:44:41.842754    2576 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="38f986d1f6e19164daf14e8d4b58c2a9ebe72200eeaed48f28784dfd92cfb2bf"
Sep 09 03:44:54.579658 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:44:54.579621    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-a-46be70-6f6d44b989-mzq7p"]
Sep 09 03:44:54.580286 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:44:54.579851    2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-ipp-a-46be70-6f6d44b989-mzq7p" podUID="3b404576-3ead-4c98-a8b2-673544c7c2c8" containerName="payload-processing" containerID="cri-o://f31642a79ffc9e20a537bf40ae2ba6fe4ffb07cdd07fb8526d9f0dc607d58138" gracePeriod=30
Sep 09 03:44:54.585035 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:44:54.584996    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-a-46be70-df94b6ffd-5pkhq"]
Sep 09 03:44:54.585215 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:44:54.585186    2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-46be70-df94b6ffd-5pkhq" podUID="56fd035b-c418-489d-9de2-e2c12c1ea974" containerName="payload-pre-processing" containerID="cri-o://827585a1081764e4e53b559fdf38760f256297156ad07b6fd4e0ccaace62f64c" gracePeriod=30
Sep 09 03:44:54.791671 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:44:54.791637    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-46be70-dx7kq"]
Sep 09 03:44:54.795326 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:44:54.795299    2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-46be70-dx7kq"]
Sep 09 03:44:54.859879 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:44:54.859837    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-a-46be70-6f6d44b989-mzq7p"
Sep 09 03:44:54.863464 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:44:54.863445    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-46be70-df94b6ffd-5pkhq"
Sep 09 03:44:54.907460 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:44:54.907429    2576 generic.go:356] "Generic (PLEG): container finished" podID="56fd035b-c418-489d-9de2-e2c12c1ea974" containerID="827585a1081764e4e53b559fdf38760f256297156ad07b6fd4e0ccaace62f64c" exitCode=0
Sep 09 03:44:54.907595 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:44:54.907481    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-46be70-df94b6ffd-5pkhq"
Sep 09 03:44:54.907595 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:44:54.907512    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-46be70-df94b6ffd-5pkhq" event={"ID":"56fd035b-c418-489d-9de2-e2c12c1ea974","Type":"ContainerDied","Data":"827585a1081764e4e53b559fdf38760f256297156ad07b6fd4e0ccaace62f64c"}
Sep 09 03:44:54.907595 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:44:54.907561    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-46be70-df94b6ffd-5pkhq" event={"ID":"56fd035b-c418-489d-9de2-e2c12c1ea974","Type":"ContainerDied","Data":"9a9e26e5c489d0dc62fa6a07eb64191661dd54a804106be1ff5674d30402a806"}
Sep 09 03:44:54.907595 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:44:54.907583    2576 scope.go:117] "RemoveContainer" containerID="827585a1081764e4e53b559fdf38760f256297156ad07b6fd4e0ccaace62f64c"
Sep 09 03:44:54.908795 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:44:54.908775    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/3b404576-3ead-4c98-a8b2-673544c7c2c8-plugins-config-volume\") pod \"3b404576-3ead-4c98-a8b2-673544c7c2c8\" (UID: \"3b404576-3ead-4c98-a8b2-673544c7c2c8\") "
Sep 09 03:44:54.908934 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:44:54.908849    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-bjzsx\" (UniqueName: \"kubernetes.io/projected/56fd035b-c418-489d-9de2-e2c12c1ea974-kube-api-access-bjzsx\") pod \"56fd035b-c418-489d-9de2-e2c12c1ea974\" (UID: \"56fd035b-c418-489d-9de2-e2c12c1ea974\") "
Sep 09 03:44:54.908934 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:44:54.908892    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-zdn7n\" (UniqueName: \"kubernetes.io/projected/3b404576-3ead-4c98-a8b2-673544c7c2c8-kube-api-access-zdn7n\") pod \"3b404576-3ead-4c98-a8b2-673544c7c2c8\" (UID: \"3b404576-3ead-4c98-a8b2-673544c7c2c8\") "
Sep 09 03:44:54.909036 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:44:54.908964    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/56fd035b-c418-489d-9de2-e2c12c1ea974-plugins-config-volume\") pod \"56fd035b-c418-489d-9de2-e2c12c1ea974\" (UID: \"56fd035b-c418-489d-9de2-e2c12c1ea974\") "
Sep 09 03:44:54.909260 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:44:54.909222    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3b404576-3ead-4c98-a8b2-673544c7c2c8-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "3b404576-3ead-4c98-a8b2-673544c7c2c8" (UID: "3b404576-3ead-4c98-a8b2-673544c7c2c8"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 09 03:44:54.909530 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:44:54.909496    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/56fd035b-c418-489d-9de2-e2c12c1ea974-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "56fd035b-c418-489d-9de2-e2c12c1ea974" (UID: "56fd035b-c418-489d-9de2-e2c12c1ea974"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 09 03:44:54.909616 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:44:54.909524    2576 generic.go:356] "Generic (PLEG): container finished" podID="3b404576-3ead-4c98-a8b2-673544c7c2c8" containerID="f31642a79ffc9e20a537bf40ae2ba6fe4ffb07cdd07fb8526d9f0dc607d58138" exitCode=0
Sep 09 03:44:54.909616 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:44:54.909554    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-a-46be70-6f6d44b989-mzq7p" event={"ID":"3b404576-3ead-4c98-a8b2-673544c7c2c8","Type":"ContainerDied","Data":"f31642a79ffc9e20a537bf40ae2ba6fe4ffb07cdd07fb8526d9f0dc607d58138"}
Sep 09 03:44:54.909616 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:44:54.909575    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-a-46be70-6f6d44b989-mzq7p"
Sep 09 03:44:54.909616 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:44:54.909591    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-a-46be70-6f6d44b989-mzq7p" event={"ID":"3b404576-3ead-4c98-a8b2-673544c7c2c8","Type":"ContainerDied","Data":"0c4fe2ca7615bd4c18244a2018c95d61bdcb81e49bb599852d701bb7100b086b"}
Sep 09 03:44:54.912089 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:44:54.912053    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3b404576-3ead-4c98-a8b2-673544c7c2c8-kube-api-access-zdn7n" (OuterVolumeSpecName: "kube-api-access-zdn7n") pod "3b404576-3ead-4c98-a8b2-673544c7c2c8" (UID: "3b404576-3ead-4c98-a8b2-673544c7c2c8"). InnerVolumeSpecName "kube-api-access-zdn7n". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 09 03:44:54.912732 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:44:54.912705    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/56fd035b-c418-489d-9de2-e2c12c1ea974-kube-api-access-bjzsx" (OuterVolumeSpecName: "kube-api-access-bjzsx") pod "56fd035b-c418-489d-9de2-e2c12c1ea974" (UID: "56fd035b-c418-489d-9de2-e2c12c1ea974"). InnerVolumeSpecName "kube-api-access-bjzsx". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 09 03:44:54.920417 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:44:54.920397    2576 scope.go:117] "RemoveContainer" containerID="827585a1081764e4e53b559fdf38760f256297156ad07b6fd4e0ccaace62f64c"
Sep 09 03:44:54.920764 ip-10-0-132-14 kubenswrapper[2576]: E0909 03:44:54.920746    2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"827585a1081764e4e53b559fdf38760f256297156ad07b6fd4e0ccaace62f64c\": container with ID starting with 827585a1081764e4e53b559fdf38760f256297156ad07b6fd4e0ccaace62f64c not found: ID does not exist" containerID="827585a1081764e4e53b559fdf38760f256297156ad07b6fd4e0ccaace62f64c"
Sep 09 03:44:54.920861 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:44:54.920773    2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"827585a1081764e4e53b559fdf38760f256297156ad07b6fd4e0ccaace62f64c"} err="failed to get container status \"827585a1081764e4e53b559fdf38760f256297156ad07b6fd4e0ccaace62f64c\": rpc error: code = NotFound desc = could not find container \"827585a1081764e4e53b559fdf38760f256297156ad07b6fd4e0ccaace62f64c\": container with ID starting with 827585a1081764e4e53b559fdf38760f256297156ad07b6fd4e0ccaace62f64c not found: ID does not exist"
Sep 09 03:44:54.920861 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:44:54.920789    2576 scope.go:117] "RemoveContainer" containerID="f31642a79ffc9e20a537bf40ae2ba6fe4ffb07cdd07fb8526d9f0dc607d58138"
Sep 09 03:44:54.929333 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:44:54.929282    2576 scope.go:117] "RemoveContainer" containerID="f31642a79ffc9e20a537bf40ae2ba6fe4ffb07cdd07fb8526d9f0dc607d58138"
Sep 09 03:44:54.929667 ip-10-0-132-14 kubenswrapper[2576]: E0909 03:44:54.929603    2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f31642a79ffc9e20a537bf40ae2ba6fe4ffb07cdd07fb8526d9f0dc607d58138\": container with ID starting with f31642a79ffc9e20a537bf40ae2ba6fe4ffb07cdd07fb8526d9f0dc607d58138 not found: ID does not exist" containerID="f31642a79ffc9e20a537bf40ae2ba6fe4ffb07cdd07fb8526d9f0dc607d58138"
Sep 09 03:44:54.929667 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:44:54.929642    2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f31642a79ffc9e20a537bf40ae2ba6fe4ffb07cdd07fb8526d9f0dc607d58138"} err="failed to get container status \"f31642a79ffc9e20a537bf40ae2ba6fe4ffb07cdd07fb8526d9f0dc607d58138\": rpc error: code = NotFound desc = could not find container \"f31642a79ffc9e20a537bf40ae2ba6fe4ffb07cdd07fb8526d9f0dc607d58138\": container with ID starting with f31642a79ffc9e20a537bf40ae2ba6fe4ffb07cdd07fb8526d9f0dc607d58138 not found: ID does not exist"
Sep 09 03:44:55.010071 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:44:55.010043    2576 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/56fd035b-c418-489d-9de2-e2c12c1ea974-plugins-config-volume\") on node \"ip-10-0-132-14.ec2.internal\" DevicePath \"\""
Sep 09 03:44:55.010071 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:44:55.010065    2576 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/3b404576-3ead-4c98-a8b2-673544c7c2c8-plugins-config-volume\") on node \"ip-10-0-132-14.ec2.internal\" DevicePath \"\""
Sep 09 03:44:55.010071 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:44:55.010075    2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-bjzsx\" (UniqueName: \"kubernetes.io/projected/56fd035b-c418-489d-9de2-e2c12c1ea974-kube-api-access-bjzsx\") on node \"ip-10-0-132-14.ec2.internal\" DevicePath \"\""
Sep 09 03:44:55.010270 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:44:55.010083    2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-zdn7n\" (UniqueName: \"kubernetes.io/projected/3b404576-3ead-4c98-a8b2-673544c7c2c8-kube-api-access-zdn7n\") on node \"ip-10-0-132-14.ec2.internal\" DevicePath \"\""
Sep 09 03:44:55.238047 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:44:55.238017    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-a-46be70-6f6d44b989-mzq7p"]
Sep 09 03:44:55.242258 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:44:55.242232    2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-a-46be70-6f6d44b989-mzq7p"]
Sep 09 03:44:55.255885 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:44:55.255864    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-a-46be70-df94b6ffd-5pkhq"]
Sep 09 03:44:55.263554 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:44:55.263531    2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-a-46be70-df94b6ffd-5pkhq"]
Sep 09 03:44:56.360252 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:44:56.360220    2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3b404576-3ead-4c98-a8b2-673544c7c2c8" path="/var/lib/kubelet/pods/3b404576-3ead-4c98-a8b2-673544c7c2c8/volumes"
Sep 09 03:44:56.360611 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:44:56.360536    2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="56fd035b-c418-489d-9de2-e2c12c1ea974" path="/var/lib/kubelet/pods/56fd035b-c418-489d-9de2-e2c12c1ea974/volumes"
Sep 09 03:44:56.360815 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:44:56.360790    2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="82d96bdc-8d0c-4433-be38-05635a0215d9" path="/var/lib/kubelet/pods/82d96bdc-8d0c-4433-be38-05635a0215d9/volumes"
Sep 09 03:45:00.139982 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:45:00.139942    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ipp-b-60a731-29815425-xmrzf"]
Sep 09 03:45:00.140426 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:45:00.140408    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="56fd035b-c418-489d-9de2-e2c12c1ea974" containerName="payload-pre-processing"
Sep 09 03:45:00.140496 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:45:00.140427    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="56fd035b-c418-489d-9de2-e2c12c1ea974" containerName="payload-pre-processing"
Sep 09 03:45:00.140496 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:45:00.140452    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="82d96bdc-8d0c-4433-be38-05635a0215d9" containerName="revoke-keys"
Sep 09 03:45:00.140496 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:45:00.140460    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="82d96bdc-8d0c-4433-be38-05635a0215d9" containerName="revoke-keys"
Sep 09 03:45:00.140496 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:45:00.140473    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3b404576-3ead-4c98-a8b2-673544c7c2c8" containerName="payload-processing"
Sep 09 03:45:00.140496 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:45:00.140482    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="3b404576-3ead-4c98-a8b2-673544c7c2c8" containerName="payload-processing"
Sep 09 03:45:00.140728 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:45:00.140594    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="82d96bdc-8d0c-4433-be38-05635a0215d9" containerName="revoke-keys"
Sep 09 03:45:00.140728 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:45:00.140605    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="3b404576-3ead-4c98-a8b2-673544c7c2c8" containerName="payload-processing"
Sep 09 03:45:00.140728 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:45:00.140617    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="56fd035b-c418-489d-9de2-e2c12c1ea974" containerName="payload-pre-processing"
Sep 09 03:45:00.145124 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:45:00.145102    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w3-c34148-29815425-6qwlp"]
Sep 09 03:45:00.145256 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:45:00.145226    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ipp-b-60a731-29815425-xmrzf"
Sep 09 03:45:00.148065 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:45:00.148045    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-w7q9f\""
Sep 09 03:45:00.148182 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:45:00.148069    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w3-c34148-29815425-6qwlp"
Sep 09 03:45:00.151874 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:45:00.151854    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ipp-b-60a731-29815425-xmrzf"]
Sep 09 03:45:00.157637 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:45:00.157619    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ipp-b-60a731-29815425-xmrzf"
Sep 09 03:45:00.158561 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:45:00.158539    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w3-c34148-29815425-6qwlp"]
Sep 09 03:45:00.166579 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:45:00.166562    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w3-c34148-29815425-6qwlp"
Sep 09 03:45:00.442350 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:45:00.442159    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ipp-b-60a731-29815425-xmrzf"]
Sep 09 03:45:00.445504 ip-10-0-132-14 kubenswrapper[2576]: W0909 03:45:00.445482    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3bd60f66_17aa_4d5f_8af0_9b0f1971f207.slice/crio-9c06f4e0052361fc3428195691e2112641f62eaf2c58f7dd7058c9809759341b WatchSource:0}: Error finding container 9c06f4e0052361fc3428195691e2112641f62eaf2c58f7dd7058c9809759341b: Status 404 returned error can't find the container with id 9c06f4e0052361fc3428195691e2112641f62eaf2c58f7dd7058c9809759341b
Sep 09 03:45:00.446059 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:45:00.446038    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w3-c34148-29815425-6qwlp"]
Sep 09 03:45:00.447300 ip-10-0-132-14 kubenswrapper[2576]: W0909 03:45:00.447276    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod11b537fa_5409_48c0_850e_e23115de8e3b.slice/crio-a4f6e3d9c5c15a94a1c19deaff9d4f9c6846363adea180c1204cc6d0c65ec617 WatchSource:0}: Error finding container a4f6e3d9c5c15a94a1c19deaff9d4f9c6846363adea180c1204cc6d0c65ec617: Status 404 returned error can't find the container with id a4f6e3d9c5c15a94a1c19deaff9d4f9c6846363adea180c1204cc6d0c65ec617
Sep 09 03:45:00.939132 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:45:00.939088    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ipp-b-60a731-29815425-xmrzf" event={"ID":"3bd60f66-17aa-4d5f-8af0-9b0f1971f207","Type":"ContainerStarted","Data":"58e0b78581cc927f51efd37c17f55e3ff977d2163d187f6cbe271d8d8674bed6"}
Sep 09 03:45:00.939293 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:45:00.939183    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ipp-b-60a731-29815425-xmrzf" event={"ID":"3bd60f66-17aa-4d5f-8af0-9b0f1971f207","Type":"ContainerStarted","Data":"9c06f4e0052361fc3428195691e2112641f62eaf2c58f7dd7058c9809759341b"}
Sep 09 03:45:00.940591 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:45:00.940566    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w3-c34148-29815425-6qwlp" event={"ID":"11b537fa-5409-48c0-850e-e23115de8e3b","Type":"ContainerStarted","Data":"e8c210dbe08d3036d9b05b66684d6dd6ce5fb07d63f12c12fec6fb259a75f5cc"}
Sep 09 03:45:00.940683 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:45:00.940607    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w3-c34148-29815425-6qwlp" event={"ID":"11b537fa-5409-48c0-850e-e23115de8e3b","Type":"ContainerStarted","Data":"a4f6e3d9c5c15a94a1c19deaff9d4f9c6846363adea180c1204cc6d0c65ec617"}
Sep 09 03:45:01.008382 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:45:01.008337    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w3-c34148-29815425-6qwlp" podStartSLOduration=1.008323066 podStartE2EDuration="1.008323066s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 03:45:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 03:45:01.006765449 +0000 UTC m=+1915.148251525" watchObservedRunningTime="2026-09-09 03:45:01.008323066 +0000 UTC m=+1915.149809141"
Sep 09 03:45:01.008892 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:45:01.008868    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ipp-b-60a731-29815425-xmrzf" podStartSLOduration=1.008861255 podStartE2EDuration="1.008861255s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 03:45:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 03:45:00.968288675 +0000 UTC m=+1915.109774752" watchObservedRunningTime="2026-09-09 03:45:01.008861255 +0000 UTC m=+1915.150347331"
Sep 09 03:45:01.409516 ip-10-0-132-14 kubenswrapper[2576]: E0909 03:45:01.409478    2576 cadvisor_stats_provider.go:567] "Partial failure issuing cadvisor.ContainerInfoV2" err="partial failures: [\"/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod11b537fa_5409_48c0_850e_e23115de8e3b.slice/crio-e8c210dbe08d3036d9b05b66684d6dd6ce5fb07d63f12c12fec6fb259a75f5cc.scope\": RecentStats: unable to find data in memory cache]"
Sep 09 03:45:01.946581 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:45:01.946539    2576 generic.go:356] "Generic (PLEG): container finished" podID="3bd60f66-17aa-4d5f-8af0-9b0f1971f207" containerID="58e0b78581cc927f51efd37c17f55e3ff977d2163d187f6cbe271d8d8674bed6" exitCode=0
Sep 09 03:45:01.946995 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:45:01.946611    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ipp-b-60a731-29815425-xmrzf" event={"ID":"3bd60f66-17aa-4d5f-8af0-9b0f1971f207","Type":"ContainerDied","Data":"58e0b78581cc927f51efd37c17f55e3ff977d2163d187f6cbe271d8d8674bed6"}
Sep 09 03:45:01.950007 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:45:01.949969    2576 generic.go:356] "Generic (PLEG): container finished" podID="11b537fa-5409-48c0-850e-e23115de8e3b" containerID="e8c210dbe08d3036d9b05b66684d6dd6ce5fb07d63f12c12fec6fb259a75f5cc" exitCode=0
Sep 09 03:45:01.950295 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:45:01.950256    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w3-c34148-29815425-6qwlp" event={"ID":"11b537fa-5409-48c0-850e-e23115de8e3b","Type":"ContainerDied","Data":"e8c210dbe08d3036d9b05b66684d6dd6ce5fb07d63f12c12fec6fb259a75f5cc"}
Sep 09 03:45:03.078002 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:45:03.077974    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ipp-b-60a731-29815425-xmrzf"
Sep 09 03:45:03.108119 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:45:03.108097    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w3-c34148-29815425-6qwlp"
Sep 09 03:45:03.960639 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:45:03.960606    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w3-c34148-29815425-6qwlp"
Sep 09 03:45:03.960844 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:45:03.960638    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w3-c34148-29815425-6qwlp" event={"ID":"11b537fa-5409-48c0-850e-e23115de8e3b","Type":"ContainerDied","Data":"a4f6e3d9c5c15a94a1c19deaff9d4f9c6846363adea180c1204cc6d0c65ec617"}
Sep 09 03:45:03.960844 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:45:03.960679    2576 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="a4f6e3d9c5c15a94a1c19deaff9d4f9c6846363adea180c1204cc6d0c65ec617"
Sep 09 03:45:03.962023 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:45:03.962003    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ipp-b-60a731-29815425-xmrzf" event={"ID":"3bd60f66-17aa-4d5f-8af0-9b0f1971f207","Type":"ContainerDied","Data":"9c06f4e0052361fc3428195691e2112641f62eaf2c58f7dd7058c9809759341b"}
Sep 09 03:45:03.962023 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:45:03.962020    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ipp-b-60a731-29815425-xmrzf"
Sep 09 03:45:03.962142 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:45:03.962038    2576 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="9c06f4e0052361fc3428195691e2112641f62eaf2c58f7dd7058c9809759341b"
Sep 09 03:45:44.156767 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:45:44.156734    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-60a731-5c8r7"]
Sep 09 03:45:44.159510 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:45:44.157072    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3bd60f66-17aa-4d5f-8af0-9b0f1971f207" containerName="cleanup"
Sep 09 03:45:44.159510 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:45:44.157083    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="3bd60f66-17aa-4d5f-8af0-9b0f1971f207" containerName="cleanup"
Sep 09 03:45:44.159510 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:45:44.157101    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="11b537fa-5409-48c0-850e-e23115de8e3b" containerName="cleanup"
Sep 09 03:45:44.159510 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:45:44.157107    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="11b537fa-5409-48c0-850e-e23115de8e3b" containerName="cleanup"
Sep 09 03:45:44.159510 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:45:44.157165    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="11b537fa-5409-48c0-850e-e23115de8e3b" containerName="cleanup"
Sep 09 03:45:44.159510 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:45:44.157175    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="3bd60f66-17aa-4d5f-8af0-9b0f1971f207" containerName="cleanup"
Sep 09 03:45:44.160331 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:45:44.160311    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-60a731-5c8r7"
Sep 09 03:45:44.163037 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:45:44.163014    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-w7q9f\""
Sep 09 03:45:44.167972 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:45:44.167950    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-60a731-5c8r7"]
Sep 09 03:45:44.227361 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:45:44.227330    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/6f263c9f-5df0-47e4-9974-e9c23998b0b8-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-b-60a731-5c8r7\" (UID: \"6f263c9f-5df0-47e4-9974-e9c23998b0b8\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-60a731-5c8r7"
Sep 09 03:45:44.327869 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:45:44.327840    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/6f263c9f-5df0-47e4-9974-e9c23998b0b8-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-b-60a731-5c8r7\" (UID: \"6f263c9f-5df0-47e4-9974-e9c23998b0b8\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-60a731-5c8r7"
Sep 09 03:45:44.328417 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:45:44.328390    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/6f263c9f-5df0-47e4-9974-e9c23998b0b8-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-b-60a731-5c8r7\" (UID: \"6f263c9f-5df0-47e4-9974-e9c23998b0b8\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-60a731-5c8r7"
Sep 09 03:45:44.471395 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:45:44.471311    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-60a731-5c8r7"
Sep 09 03:45:44.743016 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:45:44.742988    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-60a731-5c8r7"]
Sep 09 03:45:44.744909 ip-10-0-132-14 kubenswrapper[2576]: W0909 03:45:44.744885    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6f263c9f_5df0_47e4_9974_e9c23998b0b8.slice/crio-928a0f49a1e12b89436435b4db5cc318bee6ecd3e0e02a81403d7ef7b4bd0fe4 WatchSource:0}: Error finding container 928a0f49a1e12b89436435b4db5cc318bee6ecd3e0e02a81403d7ef7b4bd0fe4: Status 404 returned error can't find the container with id 928a0f49a1e12b89436435b4db5cc318bee6ecd3e0e02a81403d7ef7b4bd0fe4
Sep 09 03:45:45.162264 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:45:45.162229    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-60a731-5c8r7" event={"ID":"6f263c9f-5df0-47e4-9974-e9c23998b0b8","Type":"ContainerStarted","Data":"dba76dcf83fbecd875b26e2dc43bf76acf763ef1f9fd1b83f9e25f09e3ab32ca"}
Sep 09 03:45:45.162611 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:45:45.162280    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-60a731-5c8r7" event={"ID":"6f263c9f-5df0-47e4-9974-e9c23998b0b8","Type":"ContainerStarted","Data":"928a0f49a1e12b89436435b4db5cc318bee6ecd3e0e02a81403d7ef7b4bd0fe4"}
Sep 09 03:45:45.180224 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:45:45.180181    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-60a731-5c8r7" podStartSLOduration=1.1801649969999999 podStartE2EDuration="1.180164997s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 03:45:44 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 03:45:45.177550911 +0000 UTC m=+1959.319036990" watchObservedRunningTime="2026-09-09 03:45:45.180164997 +0000 UTC m=+1959.321651071"
Sep 09 03:45:46.168687 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:45:46.168650    2576 generic.go:356] "Generic (PLEG): container finished" podID="6f263c9f-5df0-47e4-9974-e9c23998b0b8" containerID="dba76dcf83fbecd875b26e2dc43bf76acf763ef1f9fd1b83f9e25f09e3ab32ca" exitCode=0
Sep 09 03:45:46.169117 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:45:46.168744    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-60a731-5c8r7" event={"ID":"6f263c9f-5df0-47e4-9974-e9c23998b0b8","Type":"ContainerDied","Data":"dba76dcf83fbecd875b26e2dc43bf76acf763ef1f9fd1b83f9e25f09e3ab32ca"}
Sep 09 03:45:47.301240 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:45:47.301220    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-60a731-5c8r7"
Sep 09 03:45:47.350444 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:45:47.350408    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/6f263c9f-5df0-47e4-9974-e9c23998b0b8-maas-api-service-ca\") pod \"6f263c9f-5df0-47e4-9974-e9c23998b0b8\" (UID: \"6f263c9f-5df0-47e4-9974-e9c23998b0b8\") "
Sep 09 03:45:47.350790 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:45:47.350770    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6f263c9f-5df0-47e4-9974-e9c23998b0b8-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "6f263c9f-5df0-47e4-9974-e9c23998b0b8" (UID: "6f263c9f-5df0-47e4-9974-e9c23998b0b8"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 09 03:45:47.451575 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:45:47.451485    2576 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/6f263c9f-5df0-47e4-9974-e9c23998b0b8-maas-api-service-ca\") on node \"ip-10-0-132-14.ec2.internal\" DevicePath \"\""
Sep 09 03:45:48.179189 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:45:48.179162    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-60a731-5c8r7"
Sep 09 03:45:48.179334 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:45:48.179166    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-60a731-5c8r7" event={"ID":"6f263c9f-5df0-47e4-9974-e9c23998b0b8","Type":"ContainerDied","Data":"928a0f49a1e12b89436435b4db5cc318bee6ecd3e0e02a81403d7ef7b4bd0fe4"}
Sep 09 03:45:48.179334 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:45:48.179286    2576 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="928a0f49a1e12b89436435b4db5cc318bee6ecd3e0e02a81403d7ef7b4bd0fe4"
Sep 09 03:45:54.648513 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:45:54.648449    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ipp-b-60a731-29815425-xmrzf"]
Sep 09 03:45:54.650882 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:45:54.650857    2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ipp-b-60a731-29815425-xmrzf"]
Sep 09 03:45:54.685225 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:45:54.685196    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-b-60a731-6f76c7bbd6-2cpzm"]
Sep 09 03:45:54.685464 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:45:54.685431    2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-60a731-6f76c7bbd6-2cpzm" podUID="c1eb89c9-623b-49ba-9b54-61c2744da4b8" containerName="payload-pre-processing" containerID="cri-o://7b6727a23f8ccb3f3443fad999f30fbe9c5480c84d5bfa76a4227ef6cc0a2c8f" gracePeriod=30
Sep 09 03:45:54.850521 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:45:54.850482    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-60a731-5c8r7"]
Sep 09 03:45:54.853859 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:45:54.853833    2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-60a731-5c8r7"]
Sep 09 03:45:54.935323 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:45:54.935295    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-60a731-6f76c7bbd6-2cpzm"
Sep 09 03:45:55.011819 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:45:55.011768    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-ldxzp\" (UniqueName: \"kubernetes.io/projected/c1eb89c9-623b-49ba-9b54-61c2744da4b8-kube-api-access-ldxzp\") pod \"c1eb89c9-623b-49ba-9b54-61c2744da4b8\" (UID: \"c1eb89c9-623b-49ba-9b54-61c2744da4b8\") "
Sep 09 03:45:55.011955 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:45:55.011825    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c1eb89c9-623b-49ba-9b54-61c2744da4b8-plugins-config-volume\") pod \"c1eb89c9-623b-49ba-9b54-61c2744da4b8\" (UID: \"c1eb89c9-623b-49ba-9b54-61c2744da4b8\") "
Sep 09 03:45:55.012235 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:45:55.012214    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c1eb89c9-623b-49ba-9b54-61c2744da4b8-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "c1eb89c9-623b-49ba-9b54-61c2744da4b8" (UID: "c1eb89c9-623b-49ba-9b54-61c2744da4b8"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 09 03:45:55.014000 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:45:55.013975    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c1eb89c9-623b-49ba-9b54-61c2744da4b8-kube-api-access-ldxzp" (OuterVolumeSpecName: "kube-api-access-ldxzp") pod "c1eb89c9-623b-49ba-9b54-61c2744da4b8" (UID: "c1eb89c9-623b-49ba-9b54-61c2744da4b8"). InnerVolumeSpecName "kube-api-access-ldxzp". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 09 03:45:55.113045 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:45:55.113013    2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ldxzp\" (UniqueName: \"kubernetes.io/projected/c1eb89c9-623b-49ba-9b54-61c2744da4b8-kube-api-access-ldxzp\") on node \"ip-10-0-132-14.ec2.internal\" DevicePath \"\""
Sep 09 03:45:55.113045 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:45:55.113037    2576 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c1eb89c9-623b-49ba-9b54-61c2744da4b8-plugins-config-volume\") on node \"ip-10-0-132-14.ec2.internal\" DevicePath \"\""
Sep 09 03:45:55.223295 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:45:55.223223    2576 generic.go:356] "Generic (PLEG): container finished" podID="c1eb89c9-623b-49ba-9b54-61c2744da4b8" containerID="7b6727a23f8ccb3f3443fad999f30fbe9c5480c84d5bfa76a4227ef6cc0a2c8f" exitCode=0
Sep 09 03:45:55.223295 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:45:55.223262    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-60a731-6f76c7bbd6-2cpzm"
Sep 09 03:45:55.223461 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:45:55.223306    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-60a731-6f76c7bbd6-2cpzm" event={"ID":"c1eb89c9-623b-49ba-9b54-61c2744da4b8","Type":"ContainerDied","Data":"7b6727a23f8ccb3f3443fad999f30fbe9c5480c84d5bfa76a4227ef6cc0a2c8f"}
Sep 09 03:45:55.223461 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:45:55.223360    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-60a731-6f76c7bbd6-2cpzm" event={"ID":"c1eb89c9-623b-49ba-9b54-61c2744da4b8","Type":"ContainerDied","Data":"24ab0cc199bb6cc252a05235aec08426a6fe651d6fee6d21ffbdf90a604cf579"}
Sep 09 03:45:55.223461 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:45:55.223383    2576 scope.go:117] "RemoveContainer" containerID="7b6727a23f8ccb3f3443fad999f30fbe9c5480c84d5bfa76a4227ef6cc0a2c8f"
Sep 09 03:45:55.232840 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:45:55.232817    2576 scope.go:117] "RemoveContainer" containerID="7b6727a23f8ccb3f3443fad999f30fbe9c5480c84d5bfa76a4227ef6cc0a2c8f"
Sep 09 03:45:55.233107 ip-10-0-132-14 kubenswrapper[2576]: E0909 03:45:55.233090    2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7b6727a23f8ccb3f3443fad999f30fbe9c5480c84d5bfa76a4227ef6cc0a2c8f\": container with ID starting with 7b6727a23f8ccb3f3443fad999f30fbe9c5480c84d5bfa76a4227ef6cc0a2c8f not found: ID does not exist" containerID="7b6727a23f8ccb3f3443fad999f30fbe9c5480c84d5bfa76a4227ef6cc0a2c8f"
Sep 09 03:45:55.233172 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:45:55.233116    2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7b6727a23f8ccb3f3443fad999f30fbe9c5480c84d5bfa76a4227ef6cc0a2c8f"} err="failed to get container status \"7b6727a23f8ccb3f3443fad999f30fbe9c5480c84d5bfa76a4227ef6cc0a2c8f\": rpc error: code = NotFound desc = could not find container \"7b6727a23f8ccb3f3443fad999f30fbe9c5480c84d5bfa76a4227ef6cc0a2c8f\": container with ID starting with 7b6727a23f8ccb3f3443fad999f30fbe9c5480c84d5bfa76a4227ef6cc0a2c8f not found: ID does not exist"
Sep 09 03:45:55.247215 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:45:55.247194    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-b-60a731-6f76c7bbd6-2cpzm"]
Sep 09 03:45:55.252990 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:45:55.252966    2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-b-60a731-6f76c7bbd6-2cpzm"]
Sep 09 03:45:56.359491 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:45:56.359458    2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3bd60f66-17aa-4d5f-8af0-9b0f1971f207" path="/var/lib/kubelet/pods/3bd60f66-17aa-4d5f-8af0-9b0f1971f207/volumes"
Sep 09 03:45:56.359877 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:45:56.359754    2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6f263c9f-5df0-47e4-9974-e9c23998b0b8" path="/var/lib/kubelet/pods/6f263c9f-5df0-47e4-9974-e9c23998b0b8/volumes"
Sep 09 03:45:56.360079 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:45:56.360066    2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c1eb89c9-623b-49ba-9b54-61c2744da4b8" path="/var/lib/kubelet/pods/c1eb89c9-623b-49ba-9b54-61c2744da4b8/volumes"
Sep 09 03:46:01.854165 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:46:01.854071    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w3-5d8e85-9jj7q"]
Sep 09 03:46:01.854617 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:46:01.854598    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c1eb89c9-623b-49ba-9b54-61c2744da4b8" containerName="payload-pre-processing"
Sep 09 03:46:01.854670 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:46:01.854620    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="c1eb89c9-623b-49ba-9b54-61c2744da4b8" containerName="payload-pre-processing"
Sep 09 03:46:01.854670 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:46:01.854637    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6f263c9f-5df0-47e4-9974-e9c23998b0b8" containerName="revoke-keys"
Sep 09 03:46:01.854670 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:46:01.854646    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="6f263c9f-5df0-47e4-9974-e9c23998b0b8" containerName="revoke-keys"
Sep 09 03:46:01.854779 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:46:01.854744    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="c1eb89c9-623b-49ba-9b54-61c2744da4b8" containerName="payload-pre-processing"
Sep 09 03:46:01.854779 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:46:01.854755    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="6f263c9f-5df0-47e4-9974-e9c23998b0b8" containerName="revoke-keys"
Sep 09 03:46:01.858925 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:46:01.858897    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w3-5d8e85-9jj7q"
Sep 09 03:46:01.861930 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:46:01.861910    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-w7q9f\""
Sep 09 03:46:01.869898 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:46:01.869871    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w3-5d8e85-9jj7q"]
Sep 09 03:46:01.972109 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:46:01.972078    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/cc364a41-2dae-4ed2-9a84-15c481235d0c-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-a-w3-5d8e85-9jj7q\" (UID: \"cc364a41-2dae-4ed2-9a84-15c481235d0c\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w3-5d8e85-9jj7q"
Sep 09 03:46:02.073232 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:46:02.073191    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/cc364a41-2dae-4ed2-9a84-15c481235d0c-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-a-w3-5d8e85-9jj7q\" (UID: \"cc364a41-2dae-4ed2-9a84-15c481235d0c\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w3-5d8e85-9jj7q"
Sep 09 03:46:02.073949 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:46:02.073922    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/cc364a41-2dae-4ed2-9a84-15c481235d0c-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-a-w3-5d8e85-9jj7q\" (UID: \"cc364a41-2dae-4ed2-9a84-15c481235d0c\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w3-5d8e85-9jj7q"
Sep 09 03:46:02.169061 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:46:02.168970    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w3-5d8e85-9jj7q"
Sep 09 03:46:02.442266 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:46:02.442240    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w3-5d8e85-9jj7q"]
Sep 09 03:46:02.444439 ip-10-0-132-14 kubenswrapper[2576]: W0909 03:46:02.444414    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcc364a41_2dae_4ed2_9a84_15c481235d0c.slice/crio-d0a7a59fbe2ab045a1949ebfe357330bf37bd3fc4587996efa3a8039a5984a22 WatchSource:0}: Error finding container d0a7a59fbe2ab045a1949ebfe357330bf37bd3fc4587996efa3a8039a5984a22: Status 404 returned error can't find the container with id d0a7a59fbe2ab045a1949ebfe357330bf37bd3fc4587996efa3a8039a5984a22
Sep 09 03:46:03.268720 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:46:03.268683    2576 generic.go:356] "Generic (PLEG): container finished" podID="cc364a41-2dae-4ed2-9a84-15c481235d0c" containerID="4256330af7b037c50ceb825dad4e99eeb43dc6d1f8799c5c13f20fb2d0d22e46" exitCode=0
Sep 09 03:46:03.269146 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:46:03.268772    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w3-5d8e85-9jj7q" event={"ID":"cc364a41-2dae-4ed2-9a84-15c481235d0c","Type":"ContainerDied","Data":"4256330af7b037c50ceb825dad4e99eeb43dc6d1f8799c5c13f20fb2d0d22e46"}
Sep 09 03:46:03.269146 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:46:03.268857    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w3-5d8e85-9jj7q" event={"ID":"cc364a41-2dae-4ed2-9a84-15c481235d0c","Type":"ContainerStarted","Data":"d0a7a59fbe2ab045a1949ebfe357330bf37bd3fc4587996efa3a8039a5984a22"}
Sep 09 03:46:04.406913 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:46:04.406876    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w3-5d8e85-9jj7q"
Sep 09 03:46:04.494351 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:46:04.494323    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/cc364a41-2dae-4ed2-9a84-15c481235d0c-maas-api-service-ca\") pod \"cc364a41-2dae-4ed2-9a84-15c481235d0c\" (UID: \"cc364a41-2dae-4ed2-9a84-15c481235d0c\") "
Sep 09 03:46:04.494678 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:46:04.494654    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/cc364a41-2dae-4ed2-9a84-15c481235d0c-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "cc364a41-2dae-4ed2-9a84-15c481235d0c" (UID: "cc364a41-2dae-4ed2-9a84-15c481235d0c"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 09 03:46:04.595679 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:46:04.595609    2576 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/cc364a41-2dae-4ed2-9a84-15c481235d0c-maas-api-service-ca\") on node \"ip-10-0-132-14.ec2.internal\" DevicePath \"\""
Sep 09 03:46:05.279765 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:46:05.279731    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w3-5d8e85-9jj7q"
Sep 09 03:46:05.280085 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:46:05.279769    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w3-5d8e85-9jj7q" event={"ID":"cc364a41-2dae-4ed2-9a84-15c481235d0c","Type":"ContainerDied","Data":"d0a7a59fbe2ab045a1949ebfe357330bf37bd3fc4587996efa3a8039a5984a22"}
Sep 09 03:46:05.280085 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:46:05.279839    2576 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="d0a7a59fbe2ab045a1949ebfe357330bf37bd3fc4587996efa3a8039a5984a22"
Sep 09 03:46:12.510413 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:46:12.510378    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w3-5d8e85-fb6bf859c-bd4jk"]
Sep 09 03:46:12.510829 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:46:12.510631    2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-shared-a-w3-5d8e85-fb6bf859c-bd4jk" podUID="d261d48b-863a-4bbd-9631-6bf66e695aea" containerName="payload-processing" containerID="cri-o://2c9fca7cc1d9b2909a1b0b94d4b6e74071b62e540051ee559fe07b60d81e4474" gracePeriod=30
Sep 09 03:46:12.518763 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:46:12.518712    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w3-5d8e85-59687cb69d-29wl6"]
Sep 09 03:46:12.519057 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:46:12.519007    2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-5d8e85-59687cb69d-29wl6" podUID="b7eee332-40a1-4b1f-85e9-5e7200d80373" containerName="payload-pre-processing" containerID="cri-o://b0801bae3ba8b8e221fb2b993bdfa5533d52bde66e3207899c5637321d624c06" gracePeriod=30
Sep 09 03:46:12.684102 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:46:12.684065    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w3-5d8e85-9jj7q"]
Sep 09 03:46:12.686677 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:46:12.686644    2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w3-5d8e85-9jj7q"]
Sep 09 03:46:12.774259 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:46:12.774199    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-a-w3-5d8e85-fb6bf859c-bd4jk"
Sep 09 03:46:12.808463 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:46:12.808435    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-5d8e85-59687cb69d-29wl6"
Sep 09 03:46:12.873768 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:46:12.873729    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d261d48b-863a-4bbd-9631-6bf66e695aea-plugins-config-volume\") pod \"d261d48b-863a-4bbd-9631-6bf66e695aea\" (UID: \"d261d48b-863a-4bbd-9631-6bf66e695aea\") "
Sep 09 03:46:12.873965 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:46:12.873841    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-d9ttb\" (UniqueName: \"kubernetes.io/projected/d261d48b-863a-4bbd-9631-6bf66e695aea-kube-api-access-d9ttb\") pod \"d261d48b-863a-4bbd-9631-6bf66e695aea\" (UID: \"d261d48b-863a-4bbd-9631-6bf66e695aea\") "
Sep 09 03:46:12.874132 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:46:12.874109    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d261d48b-863a-4bbd-9631-6bf66e695aea-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "d261d48b-863a-4bbd-9631-6bf66e695aea" (UID: "d261d48b-863a-4bbd-9631-6bf66e695aea"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 09 03:46:12.876020 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:46:12.875997    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d261d48b-863a-4bbd-9631-6bf66e695aea-kube-api-access-d9ttb" (OuterVolumeSpecName: "kube-api-access-d9ttb") pod "d261d48b-863a-4bbd-9631-6bf66e695aea" (UID: "d261d48b-863a-4bbd-9631-6bf66e695aea"). InnerVolumeSpecName "kube-api-access-d9ttb". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 09 03:46:12.975080 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:46:12.975032    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-2hlx8\" (UniqueName: \"kubernetes.io/projected/b7eee332-40a1-4b1f-85e9-5e7200d80373-kube-api-access-2hlx8\") pod \"b7eee332-40a1-4b1f-85e9-5e7200d80373\" (UID: \"b7eee332-40a1-4b1f-85e9-5e7200d80373\") "
Sep 09 03:46:12.975263 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:46:12.975102    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b7eee332-40a1-4b1f-85e9-5e7200d80373-plugins-config-volume\") pod \"b7eee332-40a1-4b1f-85e9-5e7200d80373\" (UID: \"b7eee332-40a1-4b1f-85e9-5e7200d80373\") "
Sep 09 03:46:12.975312 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:46:12.975291    2576 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d261d48b-863a-4bbd-9631-6bf66e695aea-plugins-config-volume\") on node \"ip-10-0-132-14.ec2.internal\" DevicePath \"\""
Sep 09 03:46:12.975312 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:46:12.975302    2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-d9ttb\" (UniqueName: \"kubernetes.io/projected/d261d48b-863a-4bbd-9631-6bf66e695aea-kube-api-access-d9ttb\") on node \"ip-10-0-132-14.ec2.internal\" DevicePath \"\""
Sep 09 03:46:12.975468 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:46:12.975446    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b7eee332-40a1-4b1f-85e9-5e7200d80373-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "b7eee332-40a1-4b1f-85e9-5e7200d80373" (UID: "b7eee332-40a1-4b1f-85e9-5e7200d80373"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 09 03:46:12.977301 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:46:12.977280    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b7eee332-40a1-4b1f-85e9-5e7200d80373-kube-api-access-2hlx8" (OuterVolumeSpecName: "kube-api-access-2hlx8") pod "b7eee332-40a1-4b1f-85e9-5e7200d80373" (UID: "b7eee332-40a1-4b1f-85e9-5e7200d80373"). InnerVolumeSpecName "kube-api-access-2hlx8". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 09 03:46:13.076685 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:46:13.076580    2576 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b7eee332-40a1-4b1f-85e9-5e7200d80373-plugins-config-volume\") on node \"ip-10-0-132-14.ec2.internal\" DevicePath \"\""
Sep 09 03:46:13.076685 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:46:13.076625    2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2hlx8\" (UniqueName: \"kubernetes.io/projected/b7eee332-40a1-4b1f-85e9-5e7200d80373-kube-api-access-2hlx8\") on node \"ip-10-0-132-14.ec2.internal\" DevicePath \"\""
Sep 09 03:46:13.321078 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:46:13.321048    2576 generic.go:356] "Generic (PLEG): container finished" podID="b7eee332-40a1-4b1f-85e9-5e7200d80373" containerID="b0801bae3ba8b8e221fb2b993bdfa5533d52bde66e3207899c5637321d624c06" exitCode=0
Sep 09 03:46:13.321264 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:46:13.321094    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-5d8e85-59687cb69d-29wl6"
Sep 09 03:46:13.321264 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:46:13.321128    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-5d8e85-59687cb69d-29wl6" event={"ID":"b7eee332-40a1-4b1f-85e9-5e7200d80373","Type":"ContainerDied","Data":"b0801bae3ba8b8e221fb2b993bdfa5533d52bde66e3207899c5637321d624c06"}
Sep 09 03:46:13.321264 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:46:13.321176    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-5d8e85-59687cb69d-29wl6" event={"ID":"b7eee332-40a1-4b1f-85e9-5e7200d80373","Type":"ContainerDied","Data":"ce86f93b885de2451872c02f7e3d59e037c7c6a2591895a4f671a57519c94d10"}
Sep 09 03:46:13.321264 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:46:13.321205    2576 scope.go:117] "RemoveContainer" containerID="b0801bae3ba8b8e221fb2b993bdfa5533d52bde66e3207899c5637321d624c06"
Sep 09 03:46:13.322472 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:46:13.322450    2576 generic.go:356] "Generic (PLEG): container finished" podID="d261d48b-863a-4bbd-9631-6bf66e695aea" containerID="2c9fca7cc1d9b2909a1b0b94d4b6e74071b62e540051ee559fe07b60d81e4474" exitCode=0
Sep 09 03:46:13.322549 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:46:13.322498    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-a-w3-5d8e85-fb6bf859c-bd4jk"
Sep 09 03:46:13.322597 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:46:13.322542    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-a-w3-5d8e85-fb6bf859c-bd4jk" event={"ID":"d261d48b-863a-4bbd-9631-6bf66e695aea","Type":"ContainerDied","Data":"2c9fca7cc1d9b2909a1b0b94d4b6e74071b62e540051ee559fe07b60d81e4474"}
Sep 09 03:46:13.322597 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:46:13.322591    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-a-w3-5d8e85-fb6bf859c-bd4jk" event={"ID":"d261d48b-863a-4bbd-9631-6bf66e695aea","Type":"ContainerDied","Data":"e6e03f21d6ed2cf7a49352356b13303c6ff7ac3bc95a46c4cebaf434be41db21"}
Sep 09 03:46:13.331965 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:46:13.331944    2576 scope.go:117] "RemoveContainer" containerID="b0801bae3ba8b8e221fb2b993bdfa5533d52bde66e3207899c5637321d624c06"
Sep 09 03:46:13.332325 ip-10-0-132-14 kubenswrapper[2576]: E0909 03:46:13.332303    2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b0801bae3ba8b8e221fb2b993bdfa5533d52bde66e3207899c5637321d624c06\": container with ID starting with b0801bae3ba8b8e221fb2b993bdfa5533d52bde66e3207899c5637321d624c06 not found: ID does not exist" containerID="b0801bae3ba8b8e221fb2b993bdfa5533d52bde66e3207899c5637321d624c06"
Sep 09 03:46:13.332439 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:46:13.332331    2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b0801bae3ba8b8e221fb2b993bdfa5533d52bde66e3207899c5637321d624c06"} err="failed to get container status \"b0801bae3ba8b8e221fb2b993bdfa5533d52bde66e3207899c5637321d624c06\": rpc error: code = NotFound desc = could not find container \"b0801bae3ba8b8e221fb2b993bdfa5533d52bde66e3207899c5637321d624c06\": container with ID starting with b0801bae3ba8b8e221fb2b993bdfa5533d52bde66e3207899c5637321d624c06 not found: ID does not exist"
Sep 09 03:46:13.332439 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:46:13.332347    2576 scope.go:117] "RemoveContainer" containerID="2c9fca7cc1d9b2909a1b0b94d4b6e74071b62e540051ee559fe07b60d81e4474"
Sep 09 03:46:13.340395 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:46:13.340374    2576 scope.go:117] "RemoveContainer" containerID="2c9fca7cc1d9b2909a1b0b94d4b6e74071b62e540051ee559fe07b60d81e4474"
Sep 09 03:46:13.340641 ip-10-0-132-14 kubenswrapper[2576]: E0909 03:46:13.340623    2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2c9fca7cc1d9b2909a1b0b94d4b6e74071b62e540051ee559fe07b60d81e4474\": container with ID starting with 2c9fca7cc1d9b2909a1b0b94d4b6e74071b62e540051ee559fe07b60d81e4474 not found: ID does not exist" containerID="2c9fca7cc1d9b2909a1b0b94d4b6e74071b62e540051ee559fe07b60d81e4474"
Sep 09 03:46:13.340697 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:46:13.340647    2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2c9fca7cc1d9b2909a1b0b94d4b6e74071b62e540051ee559fe07b60d81e4474"} err="failed to get container status \"2c9fca7cc1d9b2909a1b0b94d4b6e74071b62e540051ee559fe07b60d81e4474\": rpc error: code = NotFound desc = could not find container \"2c9fca7cc1d9b2909a1b0b94d4b6e74071b62e540051ee559fe07b60d81e4474\": container with ID starting with 2c9fca7cc1d9b2909a1b0b94d4b6e74071b62e540051ee559fe07b60d81e4474 not found: ID does not exist"
Sep 09 03:46:13.349333 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:46:13.349301    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w3-5d8e85-59687cb69d-29wl6"]
Sep 09 03:46:13.353317 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:46:13.353297    2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w3-5d8e85-59687cb69d-29wl6"]
Sep 09 03:46:13.363978 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:46:13.363952    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w3-5d8e85-fb6bf859c-bd4jk"]
Sep 09 03:46:13.367865 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:46:13.367844    2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w3-5d8e85-fb6bf859c-bd4jk"]
Sep 09 03:46:14.359496 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:46:14.359462    2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b7eee332-40a1-4b1f-85e9-5e7200d80373" path="/var/lib/kubelet/pods/b7eee332-40a1-4b1f-85e9-5e7200d80373/volumes"
Sep 09 03:46:14.359906 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:46:14.359782    2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="cc364a41-2dae-4ed2-9a84-15c481235d0c" path="/var/lib/kubelet/pods/cc364a41-2dae-4ed2-9a84-15c481235d0c/volumes"
Sep 09 03:46:14.360111 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:46:14.360097    2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d261d48b-863a-4bbd-9631-6bf66e695aea" path="/var/lib/kubelet/pods/d261d48b-863a-4bbd-9631-6bf66e695aea/volumes"
Sep 09 03:46:19.787370 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:46:19.787330    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w3-c34148-hhtdw"]
Sep 09 03:46:19.787891 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:46:19.787868    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="cc364a41-2dae-4ed2-9a84-15c481235d0c" containerName="revoke-keys"
Sep 09 03:46:19.787891 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:46:19.787889    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="cc364a41-2dae-4ed2-9a84-15c481235d0c" containerName="revoke-keys"
Sep 09 03:46:19.788026 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:46:19.787923    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d261d48b-863a-4bbd-9631-6bf66e695aea" containerName="payload-processing"
Sep 09 03:46:19.788026 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:46:19.787966    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="d261d48b-863a-4bbd-9631-6bf66e695aea" containerName="payload-processing"
Sep 09 03:46:19.788026 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:46:19.787979    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b7eee332-40a1-4b1f-85e9-5e7200d80373" containerName="payload-pre-processing"
Sep 09 03:46:19.788026 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:46:19.787990    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="b7eee332-40a1-4b1f-85e9-5e7200d80373" containerName="payload-pre-processing"
Sep 09 03:46:19.788224 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:46:19.788084    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="d261d48b-863a-4bbd-9631-6bf66e695aea" containerName="payload-processing"
Sep 09 03:46:19.788224 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:46:19.788096    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="cc364a41-2dae-4ed2-9a84-15c481235d0c" containerName="revoke-keys"
Sep 09 03:46:19.788224 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:46:19.788106    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="b7eee332-40a1-4b1f-85e9-5e7200d80373" containerName="payload-pre-processing"
Sep 09 03:46:19.792521 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:46:19.792498    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w3-c34148-hhtdw"
Sep 09 03:46:19.795295 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:46:19.795274    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-w7q9f\""
Sep 09 03:46:19.800327 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:46:19.800299    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w3-c34148-hhtdw"]
Sep 09 03:46:19.933928 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:46:19.933893    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/4a192879-8bf8-4ea0-bfe7-7ce477700c8d-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-b-w3-c34148-hhtdw\" (UID: \"4a192879-8bf8-4ea0-bfe7-7ce477700c8d\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w3-c34148-hhtdw"
Sep 09 03:46:20.034756 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:46:20.034710    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/4a192879-8bf8-4ea0-bfe7-7ce477700c8d-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-b-w3-c34148-hhtdw\" (UID: \"4a192879-8bf8-4ea0-bfe7-7ce477700c8d\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w3-c34148-hhtdw"
Sep 09 03:46:20.035362 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:46:20.035339    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/4a192879-8bf8-4ea0-bfe7-7ce477700c8d-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-b-w3-c34148-hhtdw\" (UID: \"4a192879-8bf8-4ea0-bfe7-7ce477700c8d\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w3-c34148-hhtdw"
Sep 09 03:46:20.104530 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:46:20.104427    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w3-c34148-hhtdw"
Sep 09 03:46:20.382624 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:46:20.382540    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w3-c34148-hhtdw"]
Sep 09 03:46:20.387986 ip-10-0-132-14 kubenswrapper[2576]: W0909 03:46:20.387955    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4a192879_8bf8_4ea0_bfe7_7ce477700c8d.slice/crio-8f58d4b8a86cb545549c6c37fb7a843b9daa034f029a78a9185cf0e75a2df27c WatchSource:0}: Error finding container 8f58d4b8a86cb545549c6c37fb7a843b9daa034f029a78a9185cf0e75a2df27c: Status 404 returned error can't find the container with id 8f58d4b8a86cb545549c6c37fb7a843b9daa034f029a78a9185cf0e75a2df27c
Sep 09 03:46:21.366039 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:46:21.365935    2576 generic.go:356] "Generic (PLEG): container finished" podID="4a192879-8bf8-4ea0-bfe7-7ce477700c8d" containerID="f954407b34f9ea8132846623cd143d18ebbc8979b20105eb7fa2273616bdd2d8" exitCode=0
Sep 09 03:46:21.366039 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:46:21.365995    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w3-c34148-hhtdw" event={"ID":"4a192879-8bf8-4ea0-bfe7-7ce477700c8d","Type":"ContainerDied","Data":"f954407b34f9ea8132846623cd143d18ebbc8979b20105eb7fa2273616bdd2d8"}
Sep 09 03:46:21.366616 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:46:21.366037    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w3-c34148-hhtdw" event={"ID":"4a192879-8bf8-4ea0-bfe7-7ce477700c8d","Type":"ContainerStarted","Data":"8f58d4b8a86cb545549c6c37fb7a843b9daa034f029a78a9185cf0e75a2df27c"}
Sep 09 03:46:22.493488 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:46:22.493459    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w3-c34148-hhtdw"
Sep 09 03:46:22.657701 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:46:22.657600    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/4a192879-8bf8-4ea0-bfe7-7ce477700c8d-maas-api-service-ca\") pod \"4a192879-8bf8-4ea0-bfe7-7ce477700c8d\" (UID: \"4a192879-8bf8-4ea0-bfe7-7ce477700c8d\") "
Sep 09 03:46:22.657997 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:46:22.657975    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4a192879-8bf8-4ea0-bfe7-7ce477700c8d-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "4a192879-8bf8-4ea0-bfe7-7ce477700c8d" (UID: "4a192879-8bf8-4ea0-bfe7-7ce477700c8d"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 09 03:46:22.759034 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:46:22.758999    2576 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/4a192879-8bf8-4ea0-bfe7-7ce477700c8d-maas-api-service-ca\") on node \"ip-10-0-132-14.ec2.internal\" DevicePath \"\""
Sep 09 03:46:23.377526 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:46:23.377496    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w3-c34148-hhtdw"
Sep 09 03:46:23.377726 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:46:23.377541    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w3-c34148-hhtdw" event={"ID":"4a192879-8bf8-4ea0-bfe7-7ce477700c8d","Type":"ContainerDied","Data":"8f58d4b8a86cb545549c6c37fb7a843b9daa034f029a78a9185cf0e75a2df27c"}
Sep 09 03:46:23.377726 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:46:23.377586    2576 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="8f58d4b8a86cb545549c6c37fb7a843b9daa034f029a78a9185cf0e75a2df27c"
Sep 09 03:46:30.322067 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:46:30.322032    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w3-c34148-29815425-6qwlp"]
Sep 09 03:46:30.325165 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:46:30.325143    2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w3-c34148-29815425-6qwlp"]
Sep 09 03:46:30.349185 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:46:30.349154    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w3-c34148-779dbb5bc9-4hzjv"]
Sep 09 03:46:30.349416 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:46:30.349386    2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-shared-b-w3-c34148-779dbb5bc9-4hzjv" podUID="321fa962-c540-4eb3-9bd5-af782e8fb5c9" containerName="payload-processing" containerID="cri-o://488a1adf73f76a3ace6788b62ac85dc560739cd1b6a27dc009200d60b72c4599" gracePeriod=30
Sep 09 03:46:30.363510 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:46:30.363477    2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="11b537fa-5409-48c0-850e-e23115de8e3b" path="/var/lib/kubelet/pods/11b537fa-5409-48c0-850e-e23115de8e3b/volumes"
Sep 09 03:46:30.363738 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:46:30.363721    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w3-c34148-5f7b74bdb6-r78wm"]
Sep 09 03:46:30.363947 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:46:30.363927    2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-c34148-5f7b74bdb6-r78wm" podUID="785d5422-0337-4a7b-bb4e-fa3fdcdc0ce7" containerName="payload-pre-processing" containerID="cri-o://ce1ad7f191fd3efd3850bb99b87844dfdd7e9e4b262737ee944adc5f2ac06dca" gracePeriod=30
Sep 09 03:46:30.537665 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:46:30.537554    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w3-c34148-hhtdw"]
Sep 09 03:46:30.542941 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:46:30.542914    2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w3-c34148-hhtdw"]
Sep 09 03:46:30.632921 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:46:30.632899    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-b-w3-c34148-779dbb5bc9-4hzjv"
Sep 09 03:46:30.635622 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:46:30.635601    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-c34148-5f7b74bdb6-r78wm"
Sep 09 03:46:30.716284 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:46:30.716250    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-klk9m\" (UniqueName: \"kubernetes.io/projected/321fa962-c540-4eb3-9bd5-af782e8fb5c9-kube-api-access-klk9m\") pod \"321fa962-c540-4eb3-9bd5-af782e8fb5c9\" (UID: \"321fa962-c540-4eb3-9bd5-af782e8fb5c9\") "
Sep 09 03:46:30.716468 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:46:30.716306    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/321fa962-c540-4eb3-9bd5-af782e8fb5c9-plugins-config-volume\") pod \"321fa962-c540-4eb3-9bd5-af782e8fb5c9\" (UID: \"321fa962-c540-4eb3-9bd5-af782e8fb5c9\") "
Sep 09 03:46:30.716468 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:46:30.716361    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-nkx7f\" (UniqueName: \"kubernetes.io/projected/785d5422-0337-4a7b-bb4e-fa3fdcdc0ce7-kube-api-access-nkx7f\") pod \"785d5422-0337-4a7b-bb4e-fa3fdcdc0ce7\" (UID: \"785d5422-0337-4a7b-bb4e-fa3fdcdc0ce7\") "
Sep 09 03:46:30.716468 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:46:30.716382    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/785d5422-0337-4a7b-bb4e-fa3fdcdc0ce7-plugins-config-volume\") pod \"785d5422-0337-4a7b-bb4e-fa3fdcdc0ce7\" (UID: \"785d5422-0337-4a7b-bb4e-fa3fdcdc0ce7\") "
Sep 09 03:46:30.716909 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:46:30.716782    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/321fa962-c540-4eb3-9bd5-af782e8fb5c9-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "321fa962-c540-4eb3-9bd5-af782e8fb5c9" (UID: "321fa962-c540-4eb3-9bd5-af782e8fb5c9"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 09 03:46:30.717033 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:46:30.716792    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/785d5422-0337-4a7b-bb4e-fa3fdcdc0ce7-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "785d5422-0337-4a7b-bb4e-fa3fdcdc0ce7" (UID: "785d5422-0337-4a7b-bb4e-fa3fdcdc0ce7"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 09 03:46:30.718512 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:46:30.718492    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/321fa962-c540-4eb3-9bd5-af782e8fb5c9-kube-api-access-klk9m" (OuterVolumeSpecName: "kube-api-access-klk9m") pod "321fa962-c540-4eb3-9bd5-af782e8fb5c9" (UID: "321fa962-c540-4eb3-9bd5-af782e8fb5c9"). InnerVolumeSpecName "kube-api-access-klk9m". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 09 03:46:30.718718 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:46:30.718697    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/785d5422-0337-4a7b-bb4e-fa3fdcdc0ce7-kube-api-access-nkx7f" (OuterVolumeSpecName: "kube-api-access-nkx7f") pod "785d5422-0337-4a7b-bb4e-fa3fdcdc0ce7" (UID: "785d5422-0337-4a7b-bb4e-fa3fdcdc0ce7"). InnerVolumeSpecName "kube-api-access-nkx7f". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 09 03:46:30.816946 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:46:30.816911    2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-nkx7f\" (UniqueName: \"kubernetes.io/projected/785d5422-0337-4a7b-bb4e-fa3fdcdc0ce7-kube-api-access-nkx7f\") on node \"ip-10-0-132-14.ec2.internal\" DevicePath \"\""
Sep 09 03:46:30.816946 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:46:30.816941    2576 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/785d5422-0337-4a7b-bb4e-fa3fdcdc0ce7-plugins-config-volume\") on node \"ip-10-0-132-14.ec2.internal\" DevicePath \"\""
Sep 09 03:46:30.816946 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:46:30.816953    2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-klk9m\" (UniqueName: \"kubernetes.io/projected/321fa962-c540-4eb3-9bd5-af782e8fb5c9-kube-api-access-klk9m\") on node \"ip-10-0-132-14.ec2.internal\" DevicePath \"\""
Sep 09 03:46:30.817181 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:46:30.816967    2576 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/321fa962-c540-4eb3-9bd5-af782e8fb5c9-plugins-config-volume\") on node \"ip-10-0-132-14.ec2.internal\" DevicePath \"\""
Sep 09 03:46:31.419835 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:46:31.419782    2576 generic.go:356] "Generic (PLEG): container finished" podID="321fa962-c540-4eb3-9bd5-af782e8fb5c9" containerID="488a1adf73f76a3ace6788b62ac85dc560739cd1b6a27dc009200d60b72c4599" exitCode=0
Sep 09 03:46:31.420267 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:46:31.419843    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-b-w3-c34148-779dbb5bc9-4hzjv" event={"ID":"321fa962-c540-4eb3-9bd5-af782e8fb5c9","Type":"ContainerDied","Data":"488a1adf73f76a3ace6788b62ac85dc560739cd1b6a27dc009200d60b72c4599"}
Sep 09 03:46:31.420267 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:46:31.419855    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-b-w3-c34148-779dbb5bc9-4hzjv"
Sep 09 03:46:31.420267 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:46:31.419922    2576 scope.go:117] "RemoveContainer" containerID="488a1adf73f76a3ace6788b62ac85dc560739cd1b6a27dc009200d60b72c4599"
Sep 09 03:46:31.420267 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:46:31.419908    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-b-w3-c34148-779dbb5bc9-4hzjv" event={"ID":"321fa962-c540-4eb3-9bd5-af782e8fb5c9","Type":"ContainerDied","Data":"8b8085ac96343f815eaf15780d9e0430d0b7da0ec7c3511ee69bf05ea3318879"}
Sep 09 03:46:31.421333 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:46:31.421313    2576 generic.go:356] "Generic (PLEG): container finished" podID="785d5422-0337-4a7b-bb4e-fa3fdcdc0ce7" containerID="ce1ad7f191fd3efd3850bb99b87844dfdd7e9e4b262737ee944adc5f2ac06dca" exitCode=0
Sep 09 03:46:31.421408 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:46:31.421353    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-c34148-5f7b74bdb6-r78wm"
Sep 09 03:46:31.421408 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:46:31.421381    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-c34148-5f7b74bdb6-r78wm" event={"ID":"785d5422-0337-4a7b-bb4e-fa3fdcdc0ce7","Type":"ContainerDied","Data":"ce1ad7f191fd3efd3850bb99b87844dfdd7e9e4b262737ee944adc5f2ac06dca"}
Sep 09 03:46:31.421513 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:46:31.421428    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-c34148-5f7b74bdb6-r78wm" event={"ID":"785d5422-0337-4a7b-bb4e-fa3fdcdc0ce7","Type":"ContainerDied","Data":"fc51ea640c207a2da063e558779ebd83e9fb251f7623fa047bc7d89ccf45a82d"}
Sep 09 03:46:31.429350 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:46:31.429331    2576 scope.go:117] "RemoveContainer" containerID="488a1adf73f76a3ace6788b62ac85dc560739cd1b6a27dc009200d60b72c4599"
Sep 09 03:46:31.429686 ip-10-0-132-14 kubenswrapper[2576]: E0909 03:46:31.429659    2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"488a1adf73f76a3ace6788b62ac85dc560739cd1b6a27dc009200d60b72c4599\": container with ID starting with 488a1adf73f76a3ace6788b62ac85dc560739cd1b6a27dc009200d60b72c4599 not found: ID does not exist" containerID="488a1adf73f76a3ace6788b62ac85dc560739cd1b6a27dc009200d60b72c4599"
Sep 09 03:46:31.429759 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:46:31.429700    2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"488a1adf73f76a3ace6788b62ac85dc560739cd1b6a27dc009200d60b72c4599"} err="failed to get container status \"488a1adf73f76a3ace6788b62ac85dc560739cd1b6a27dc009200d60b72c4599\": rpc error: code = NotFound desc = could not find container \"488a1adf73f76a3ace6788b62ac85dc560739cd1b6a27dc009200d60b72c4599\": container with ID starting with 488a1adf73f76a3ace6788b62ac85dc560739cd1b6a27dc009200d60b72c4599 not found: ID does not exist"
Sep 09 03:46:31.429759 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:46:31.429728    2576 scope.go:117] "RemoveContainer" containerID="ce1ad7f191fd3efd3850bb99b87844dfdd7e9e4b262737ee944adc5f2ac06dca"
Sep 09 03:46:31.440870 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:46:31.440845    2576 scope.go:117] "RemoveContainer" containerID="ce1ad7f191fd3efd3850bb99b87844dfdd7e9e4b262737ee944adc5f2ac06dca"
Sep 09 03:46:31.441244 ip-10-0-132-14 kubenswrapper[2576]: E0909 03:46:31.441225    2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ce1ad7f191fd3efd3850bb99b87844dfdd7e9e4b262737ee944adc5f2ac06dca\": container with ID starting with ce1ad7f191fd3efd3850bb99b87844dfdd7e9e4b262737ee944adc5f2ac06dca not found: ID does not exist" containerID="ce1ad7f191fd3efd3850bb99b87844dfdd7e9e4b262737ee944adc5f2ac06dca"
Sep 09 03:46:31.441318 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:46:31.441254    2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ce1ad7f191fd3efd3850bb99b87844dfdd7e9e4b262737ee944adc5f2ac06dca"} err="failed to get container status \"ce1ad7f191fd3efd3850bb99b87844dfdd7e9e4b262737ee944adc5f2ac06dca\": rpc error: code = NotFound desc = could not find container \"ce1ad7f191fd3efd3850bb99b87844dfdd7e9e4b262737ee944adc5f2ac06dca\": container with ID starting with ce1ad7f191fd3efd3850bb99b87844dfdd7e9e4b262737ee944adc5f2ac06dca not found: ID does not exist"
Sep 09 03:46:31.447692 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:46:31.447666    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w3-c34148-779dbb5bc9-4hzjv"]
Sep 09 03:46:31.450498 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:46:31.450474    2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w3-c34148-779dbb5bc9-4hzjv"]
Sep 09 03:46:31.464414 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:46:31.464386    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w3-c34148-5f7b74bdb6-r78wm"]
Sep 09 03:46:31.468898 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:46:31.468876    2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w3-c34148-5f7b74bdb6-r78wm"]
Sep 09 03:46:32.360689 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:46:32.360653    2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="321fa962-c540-4eb3-9bd5-af782e8fb5c9" path="/var/lib/kubelet/pods/321fa962-c540-4eb3-9bd5-af782e8fb5c9/volumes"
Sep 09 03:46:32.361038 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:46:32.361022    2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4a192879-8bf8-4ea0-bfe7-7ce477700c8d" path="/var/lib/kubelet/pods/4a192879-8bf8-4ea0-bfe7-7ce477700c8d/volumes"
Sep 09 03:46:32.361331 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:46:32.361320    2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="785d5422-0337-4a7b-bb4e-fa3fdcdc0ce7" path="/var/lib/kubelet/pods/785d5422-0337-4a7b-bb4e-fa3fdcdc0ce7/volumes"
Sep 09 03:46:37.338830 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:46:37.338770    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-b-w3-c34148-openshift-default-98858768-vhlwz"]
Sep 09 03:46:37.339331 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:46:37.339069    2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-shared-b-w3-c34148-openshift-default-98858768-vhlwz" podUID="141cec6f-6a51-48c6-8dc4-8c48ad7d9f40" containerName="istio-proxy" containerID="cri-o://d5bfc0c5f6d4aace9b815ac6454284980def8743f01b6ed4d6c6c738f04ec819" gracePeriod=30
Sep 09 03:46:39.905709 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:46:39.905666    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-69d6c4f5bf-khxz4"]
Sep 09 03:46:39.906212 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:46:39.905977    2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/maas-controller-69d6c4f5bf-khxz4" podUID="5ff59eb2-7e81-4885-a896-d74f3eabb514" containerName="manager" containerID="cri-o://690e697c6d0658496f4ae08b572f756bfc33a596e37b77c9f482e440d63e658c" gracePeriod=30
Sep 09 03:46:40.136817 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:46:40.136783    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-69d6c4f5bf-khxz4"
Sep 09 03:46:40.199510 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:46:40.199418    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/5ff59eb2-7e81-4885-a896-d74f3eabb514-metrics-certs\") pod \"5ff59eb2-7e81-4885-a896-d74f3eabb514\" (UID: \"5ff59eb2-7e81-4885-a896-d74f3eabb514\") "
Sep 09 03:46:40.199510 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:46:40.199485    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/5ff59eb2-7e81-4885-a896-d74f3eabb514-webhook-cert\") pod \"5ff59eb2-7e81-4885-a896-d74f3eabb514\" (UID: \"5ff59eb2-7e81-4885-a896-d74f3eabb514\") "
Sep 09 03:46:40.199697 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:46:40.199607    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-rdhsr\" (UniqueName: \"kubernetes.io/projected/5ff59eb2-7e81-4885-a896-d74f3eabb514-kube-api-access-rdhsr\") pod \"5ff59eb2-7e81-4885-a896-d74f3eabb514\" (UID: \"5ff59eb2-7e81-4885-a896-d74f3eabb514\") "
Sep 09 03:46:40.201694 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:46:40.201662    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5ff59eb2-7e81-4885-a896-d74f3eabb514-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "5ff59eb2-7e81-4885-a896-d74f3eabb514" (UID: "5ff59eb2-7e81-4885-a896-d74f3eabb514"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 09 03:46:40.201869 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:46:40.201715    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5ff59eb2-7e81-4885-a896-d74f3eabb514-webhook-cert" (OuterVolumeSpecName: "webhook-cert") pod "5ff59eb2-7e81-4885-a896-d74f3eabb514" (UID: "5ff59eb2-7e81-4885-a896-d74f3eabb514"). InnerVolumeSpecName "webhook-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 09 03:46:40.201953 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:46:40.201937    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5ff59eb2-7e81-4885-a896-d74f3eabb514-kube-api-access-rdhsr" (OuterVolumeSpecName: "kube-api-access-rdhsr") pod "5ff59eb2-7e81-4885-a896-d74f3eabb514" (UID: "5ff59eb2-7e81-4885-a896-d74f3eabb514"). InnerVolumeSpecName "kube-api-access-rdhsr". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 09 03:46:40.300909 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:46:40.300867    2576 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/5ff59eb2-7e81-4885-a896-d74f3eabb514-metrics-certs\") on node \"ip-10-0-132-14.ec2.internal\" DevicePath \"\""
Sep 09 03:46:40.300909 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:46:40.300903    2576 reconciler_common.go:299] "Volume detached for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/5ff59eb2-7e81-4885-a896-d74f3eabb514-webhook-cert\") on node \"ip-10-0-132-14.ec2.internal\" DevicePath \"\""
Sep 09 03:46:40.300909 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:46:40.300915    2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rdhsr\" (UniqueName: \"kubernetes.io/projected/5ff59eb2-7e81-4885-a896-d74f3eabb514-kube-api-access-rdhsr\") on node \"ip-10-0-132-14.ec2.internal\" DevicePath \"\""
Sep 09 03:46:40.475794 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:46:40.475694    2576 generic.go:356] "Generic (PLEG): container finished" podID="5ff59eb2-7e81-4885-a896-d74f3eabb514" containerID="690e697c6d0658496f4ae08b572f756bfc33a596e37b77c9f482e440d63e658c" exitCode=0
Sep 09 03:46:40.475794 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:46:40.475734    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-69d6c4f5bf-khxz4" event={"ID":"5ff59eb2-7e81-4885-a896-d74f3eabb514","Type":"ContainerDied","Data":"690e697c6d0658496f4ae08b572f756bfc33a596e37b77c9f482e440d63e658c"}
Sep 09 03:46:40.475794 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:46:40.475737    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-69d6c4f5bf-khxz4"
Sep 09 03:46:40.475794 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:46:40.475773    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-69d6c4f5bf-khxz4" event={"ID":"5ff59eb2-7e81-4885-a896-d74f3eabb514","Type":"ContainerDied","Data":"807cf953768acae0d8fb678a6b1274b958f52cf8b3a9111cc3ab21ab264f774f"}
Sep 09 03:46:40.475794 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:46:40.475794    2576 scope.go:117] "RemoveContainer" containerID="690e697c6d0658496f4ae08b572f756bfc33a596e37b77c9f482e440d63e658c"
Sep 09 03:46:40.484936 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:46:40.484912    2576 scope.go:117] "RemoveContainer" containerID="690e697c6d0658496f4ae08b572f756bfc33a596e37b77c9f482e440d63e658c"
Sep 09 03:46:40.485226 ip-10-0-132-14 kubenswrapper[2576]: E0909 03:46:40.485204    2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"690e697c6d0658496f4ae08b572f756bfc33a596e37b77c9f482e440d63e658c\": container with ID starting with 690e697c6d0658496f4ae08b572f756bfc33a596e37b77c9f482e440d63e658c not found: ID does not exist" containerID="690e697c6d0658496f4ae08b572f756bfc33a596e37b77c9f482e440d63e658c"
Sep 09 03:46:40.485270 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:46:40.485236    2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"690e697c6d0658496f4ae08b572f756bfc33a596e37b77c9f482e440d63e658c"} err="failed to get container status \"690e697c6d0658496f4ae08b572f756bfc33a596e37b77c9f482e440d63e658c\": rpc error: code = NotFound desc = could not find container \"690e697c6d0658496f4ae08b572f756bfc33a596e37b77c9f482e440d63e658c\": container with ID starting with 690e697c6d0658496f4ae08b572f756bfc33a596e37b77c9f482e440d63e658c not found: ID does not exist"
Sep 09 03:46:40.493817 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:46:40.493779    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-69d6c4f5bf-khxz4"]
Sep 09 03:46:40.500051 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:46:40.500027    2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-69d6c4f5bf-khxz4"]
Sep 09 03:46:42.359841 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:46:42.359789    2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5ff59eb2-7e81-4885-a896-d74f3eabb514" path="/var/lib/kubelet/pods/5ff59eb2-7e81-4885-a896-d74f3eabb514/volumes"
Sep 09 03:46:42.489857 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:46:42.489823    2576 generic.go:356] "Generic (PLEG): container finished" podID="141cec6f-6a51-48c6-8dc4-8c48ad7d9f40" containerID="d5bfc0c5f6d4aace9b815ac6454284980def8743f01b6ed4d6c6c738f04ec819" exitCode=0
Sep 09 03:46:42.490044 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:46:42.489899    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w3-c34148-openshift-default-98858768-vhlwz" event={"ID":"141cec6f-6a51-48c6-8dc4-8c48ad7d9f40","Type":"ContainerDied","Data":"d5bfc0c5f6d4aace9b815ac6454284980def8743f01b6ed4d6c6c738f04ec819"}
Sep 09 03:46:42.490044 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:46:42.489971    2576 scope.go:117] "RemoveContainer" containerID="dc95722b84320daaf603db3d818ae28e0943f8de126a0a1588c3786126850eec"
Sep 09 03:46:42.586192 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:46:42.586162    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-w3-c34148-openshift-default-98858768-vhlwz"
Sep 09 03:46:42.621443 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:46:42.621155    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/141cec6f-6a51-48c6-8dc4-8c48ad7d9f40-istio-envoy\") pod \"141cec6f-6a51-48c6-8dc4-8c48ad7d9f40\" (UID: \"141cec6f-6a51-48c6-8dc4-8c48ad7d9f40\") "
Sep 09 03:46:42.621443 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:46:42.621228    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/141cec6f-6a51-48c6-8dc4-8c48ad7d9f40-workload-certs\") pod \"141cec6f-6a51-48c6-8dc4-8c48ad7d9f40\" (UID: \"141cec6f-6a51-48c6-8dc4-8c48ad7d9f40\") "
Sep 09 03:46:42.621443 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:46:42.621269    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-ngsqq\" (UniqueName: \"kubernetes.io/projected/141cec6f-6a51-48c6-8dc4-8c48ad7d9f40-kube-api-access-ngsqq\") pod \"141cec6f-6a51-48c6-8dc4-8c48ad7d9f40\" (UID: \"141cec6f-6a51-48c6-8dc4-8c48ad7d9f40\") "
Sep 09 03:46:42.621443 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:46:42.621299    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/141cec6f-6a51-48c6-8dc4-8c48ad7d9f40-istiod-ca-cert\") pod \"141cec6f-6a51-48c6-8dc4-8c48ad7d9f40\" (UID: \"141cec6f-6a51-48c6-8dc4-8c48ad7d9f40\") "
Sep 09 03:46:42.621443 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:46:42.621338    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/141cec6f-6a51-48c6-8dc4-8c48ad7d9f40-istio-podinfo\") pod \"141cec6f-6a51-48c6-8dc4-8c48ad7d9f40\" (UID: \"141cec6f-6a51-48c6-8dc4-8c48ad7d9f40\") "
Sep 09 03:46:42.621443 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:46:42.621383    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/141cec6f-6a51-48c6-8dc4-8c48ad7d9f40-credential-socket\") pod \"141cec6f-6a51-48c6-8dc4-8c48ad7d9f40\" (UID: \"141cec6f-6a51-48c6-8dc4-8c48ad7d9f40\") "
Sep 09 03:46:42.621443 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:46:42.621412    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/141cec6f-6a51-48c6-8dc4-8c48ad7d9f40-istio-data\") pod \"141cec6f-6a51-48c6-8dc4-8c48ad7d9f40\" (UID: \"141cec6f-6a51-48c6-8dc4-8c48ad7d9f40\") "
Sep 09 03:46:42.621958 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:46:42.621653    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/141cec6f-6a51-48c6-8dc4-8c48ad7d9f40-istio-data" (OuterVolumeSpecName: "istio-data") pod "141cec6f-6a51-48c6-8dc4-8c48ad7d9f40" (UID: "141cec6f-6a51-48c6-8dc4-8c48ad7d9f40"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 03:46:42.621958 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:46:42.621750    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/141cec6f-6a51-48c6-8dc4-8c48ad7d9f40-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "141cec6f-6a51-48c6-8dc4-8c48ad7d9f40" (UID: "141cec6f-6a51-48c6-8dc4-8c48ad7d9f40"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 03:46:42.621958 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:46:42.621824    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/141cec6f-6a51-48c6-8dc4-8c48ad7d9f40-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "141cec6f-6a51-48c6-8dc4-8c48ad7d9f40" (UID: "141cec6f-6a51-48c6-8dc4-8c48ad7d9f40"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 03:46:42.621958 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:46:42.621872    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/141cec6f-6a51-48c6-8dc4-8c48ad7d9f40-istio-token\") pod \"141cec6f-6a51-48c6-8dc4-8c48ad7d9f40\" (UID: \"141cec6f-6a51-48c6-8dc4-8c48ad7d9f40\") "
Sep 09 03:46:42.621958 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:46:42.621921    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/141cec6f-6a51-48c6-8dc4-8c48ad7d9f40-workload-socket\") pod \"141cec6f-6a51-48c6-8dc4-8c48ad7d9f40\" (UID: \"141cec6f-6a51-48c6-8dc4-8c48ad7d9f40\") "
Sep 09 03:46:42.622216 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:46:42.622177    2576 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/141cec6f-6a51-48c6-8dc4-8c48ad7d9f40-credential-socket\") on node \"ip-10-0-132-14.ec2.internal\" DevicePath \"\""
Sep 09 03:46:42.622216 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:46:42.622194    2576 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/141cec6f-6a51-48c6-8dc4-8c48ad7d9f40-istio-data\") on node \"ip-10-0-132-14.ec2.internal\" DevicePath \"\""
Sep 09 03:46:42.622216 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:46:42.622206    2576 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/141cec6f-6a51-48c6-8dc4-8c48ad7d9f40-workload-certs\") on node \"ip-10-0-132-14.ec2.internal\" DevicePath \"\""
Sep 09 03:46:42.622634 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:46:42.622543    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/141cec6f-6a51-48c6-8dc4-8c48ad7d9f40-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "141cec6f-6a51-48c6-8dc4-8c48ad7d9f40" (UID: "141cec6f-6a51-48c6-8dc4-8c48ad7d9f40"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 09 03:46:42.622634 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:46:42.622600    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/141cec6f-6a51-48c6-8dc4-8c48ad7d9f40-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "141cec6f-6a51-48c6-8dc4-8c48ad7d9f40" (UID: "141cec6f-6a51-48c6-8dc4-8c48ad7d9f40"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 03:46:42.624402 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:46:42.624374    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/141cec6f-6a51-48c6-8dc4-8c48ad7d9f40-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "141cec6f-6a51-48c6-8dc4-8c48ad7d9f40" (UID: "141cec6f-6a51-48c6-8dc4-8c48ad7d9f40"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 03:46:42.624586 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:46:42.624564    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/141cec6f-6a51-48c6-8dc4-8c48ad7d9f40-kube-api-access-ngsqq" (OuterVolumeSpecName: "kube-api-access-ngsqq") pod "141cec6f-6a51-48c6-8dc4-8c48ad7d9f40" (UID: "141cec6f-6a51-48c6-8dc4-8c48ad7d9f40"). InnerVolumeSpecName "kube-api-access-ngsqq". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 09 03:46:42.624666 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:46:42.624592    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/141cec6f-6a51-48c6-8dc4-8c48ad7d9f40-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "141cec6f-6a51-48c6-8dc4-8c48ad7d9f40" (UID: "141cec6f-6a51-48c6-8dc4-8c48ad7d9f40"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 09 03:46:42.624728 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:46:42.624701    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/141cec6f-6a51-48c6-8dc4-8c48ad7d9f40-istio-token" (OuterVolumeSpecName: "istio-token") pod "141cec6f-6a51-48c6-8dc4-8c48ad7d9f40" (UID: "141cec6f-6a51-48c6-8dc4-8c48ad7d9f40"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 09 03:46:42.722619 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:46:42.722561    2576 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/141cec6f-6a51-48c6-8dc4-8c48ad7d9f40-workload-socket\") on node \"ip-10-0-132-14.ec2.internal\" DevicePath \"\""
Sep 09 03:46:42.722619 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:46:42.722609    2576 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/141cec6f-6a51-48c6-8dc4-8c48ad7d9f40-istio-envoy\") on node \"ip-10-0-132-14.ec2.internal\" DevicePath \"\""
Sep 09 03:46:42.722619 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:46:42.722619    2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ngsqq\" (UniqueName: \"kubernetes.io/projected/141cec6f-6a51-48c6-8dc4-8c48ad7d9f40-kube-api-access-ngsqq\") on node \"ip-10-0-132-14.ec2.internal\" DevicePath \"\""
Sep 09 03:46:42.722619 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:46:42.722628    2576 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/141cec6f-6a51-48c6-8dc4-8c48ad7d9f40-istiod-ca-cert\") on node \"ip-10-0-132-14.ec2.internal\" DevicePath \"\""
Sep 09 03:46:42.722619 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:46:42.722639    2576 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/141cec6f-6a51-48c6-8dc4-8c48ad7d9f40-istio-podinfo\") on node \"ip-10-0-132-14.ec2.internal\" DevicePath \"\""
Sep 09 03:46:42.722619 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:46:42.722646    2576 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/141cec6f-6a51-48c6-8dc4-8c48ad7d9f40-istio-token\") on node \"ip-10-0-132-14.ec2.internal\" DevicePath \"\""
Sep 09 03:46:43.495779 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:46:43.495747    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-w3-c34148-openshift-default-98858768-vhlwz"
Sep 09 03:46:43.496212 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:46:43.495745    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w3-c34148-openshift-default-98858768-vhlwz" event={"ID":"141cec6f-6a51-48c6-8dc4-8c48ad7d9f40","Type":"ContainerDied","Data":"8791091ad18f4c32f88b9f3a2eb8427a1501c1450170efa3984fed6900924b6a"}
Sep 09 03:46:43.496212 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:46:43.495876    2576 scope.go:117] "RemoveContainer" containerID="d5bfc0c5f6d4aace9b815ac6454284980def8743f01b6ed4d6c6c738f04ec819"
Sep 09 03:46:43.517471 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:46:43.517432    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-b-w3-c34148-openshift-default-98858768-vhlwz"]
Sep 09 03:46:43.527029 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:46:43.523447    2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-shared-b-w3-c34148-openshift-default-98858768-vhlwz"]
Sep 09 03:46:44.359728 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:46:44.359692    2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="141cec6f-6a51-48c6-8dc4-8c48ad7d9f40" path="/var/lib/kubelet/pods/141cec6f-6a51-48c6-8dc4-8c48ad7d9f40/volumes"
Sep 09 03:47:06.727657 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:47:06.727625    2576 scope.go:117] "RemoveContainer" containerID="c80c29a083602b85e54879c9b5ad526d28c75cf001f3830c8d456fd29547a205"
Sep 09 03:47:06.735873 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:47:06.735852    2576 scope.go:117] "RemoveContainer" containerID="2915ae464f8a57a34371f5eec7021dc3244cd0de19f680ca953690fde5383f4d"
Sep 09 03:47:41.276855 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:47:41.276814    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-69d6c4f5bf-98p2r"]
Sep 09 03:47:41.277327 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:47:41.277108    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4a192879-8bf8-4ea0-bfe7-7ce477700c8d" containerName="revoke-keys"
Sep 09 03:47:41.277327 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:47:41.277119    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="4a192879-8bf8-4ea0-bfe7-7ce477700c8d" containerName="revoke-keys"
Sep 09 03:47:41.277327 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:47:41.277130    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="785d5422-0337-4a7b-bb4e-fa3fdcdc0ce7" containerName="payload-pre-processing"
Sep 09 03:47:41.277327 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:47:41.277135    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="785d5422-0337-4a7b-bb4e-fa3fdcdc0ce7" containerName="payload-pre-processing"
Sep 09 03:47:41.277327 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:47:41.277150    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="141cec6f-6a51-48c6-8dc4-8c48ad7d9f40" containerName="istio-proxy"
Sep 09 03:47:41.277327 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:47:41.277155    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="141cec6f-6a51-48c6-8dc4-8c48ad7d9f40" containerName="istio-proxy"
Sep 09 03:47:41.277327 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:47:41.277167    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="321fa962-c540-4eb3-9bd5-af782e8fb5c9" containerName="payload-processing"
Sep 09 03:47:41.277327 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:47:41.277172    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="321fa962-c540-4eb3-9bd5-af782e8fb5c9" containerName="payload-processing"
Sep 09 03:47:41.277327 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:47:41.277179    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5ff59eb2-7e81-4885-a896-d74f3eabb514" containerName="manager"
Sep 09 03:47:41.277327 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:47:41.277184    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="5ff59eb2-7e81-4885-a896-d74f3eabb514" containerName="manager"
Sep 09 03:47:41.277327 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:47:41.277191    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="141cec6f-6a51-48c6-8dc4-8c48ad7d9f40" containerName="istio-proxy"
Sep 09 03:47:41.277327 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:47:41.277195    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="141cec6f-6a51-48c6-8dc4-8c48ad7d9f40" containerName="istio-proxy"
Sep 09 03:47:41.277327 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:47:41.277237    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="785d5422-0337-4a7b-bb4e-fa3fdcdc0ce7" containerName="payload-pre-processing"
Sep 09 03:47:41.277327 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:47:41.277244    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="321fa962-c540-4eb3-9bd5-af782e8fb5c9" containerName="payload-processing"
Sep 09 03:47:41.277327 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:47:41.277249    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="141cec6f-6a51-48c6-8dc4-8c48ad7d9f40" containerName="istio-proxy"
Sep 09 03:47:41.277327 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:47:41.277255    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="5ff59eb2-7e81-4885-a896-d74f3eabb514" containerName="manager"
Sep 09 03:47:41.277327 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:47:41.277261    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="4a192879-8bf8-4ea0-bfe7-7ce477700c8d" containerName="revoke-keys"
Sep 09 03:47:41.280171 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:47:41.280155    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-69d6c4f5bf-98p2r"
Sep 09 03:47:41.284087 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:47:41.284059    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-metrics-tls\""
Sep 09 03:47:41.284225 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:47:41.284085    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\""
Sep 09 03:47:41.284225 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:47:41.284087    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-nrndt\""
Sep 09 03:47:41.284225 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:47:41.284090    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\""
Sep 09 03:47:41.291366 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:47:41.291338    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-69d6c4f5bf-98p2r"]
Sep 09 03:47:41.406518 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:47:41.406478    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/1431b95c-4503-4c20-890e-ad33af6b10dd-webhook-cert\") pod \"maas-controller-69d6c4f5bf-98p2r\" (UID: \"1431b95c-4503-4c20-890e-ad33af6b10dd\") " pod="opendatahub/maas-controller-69d6c4f5bf-98p2r"
Sep 09 03:47:41.406518 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:47:41.406524    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/1431b95c-4503-4c20-890e-ad33af6b10dd-metrics-certs\") pod \"maas-controller-69d6c4f5bf-98p2r\" (UID: \"1431b95c-4503-4c20-890e-ad33af6b10dd\") " pod="opendatahub/maas-controller-69d6c4f5bf-98p2r"
Sep 09 03:47:41.406750 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:47:41.406557    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-r8t7q\" (UniqueName: \"kubernetes.io/projected/1431b95c-4503-4c20-890e-ad33af6b10dd-kube-api-access-r8t7q\") pod \"maas-controller-69d6c4f5bf-98p2r\" (UID: \"1431b95c-4503-4c20-890e-ad33af6b10dd\") " pod="opendatahub/maas-controller-69d6c4f5bf-98p2r"
Sep 09 03:47:41.507646 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:47:41.507611    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/1431b95c-4503-4c20-890e-ad33af6b10dd-webhook-cert\") pod \"maas-controller-69d6c4f5bf-98p2r\" (UID: \"1431b95c-4503-4c20-890e-ad33af6b10dd\") " pod="opendatahub/maas-controller-69d6c4f5bf-98p2r"
Sep 09 03:47:41.507646 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:47:41.507653    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/1431b95c-4503-4c20-890e-ad33af6b10dd-metrics-certs\") pod \"maas-controller-69d6c4f5bf-98p2r\" (UID: \"1431b95c-4503-4c20-890e-ad33af6b10dd\") " pod="opendatahub/maas-controller-69d6c4f5bf-98p2r"
Sep 09 03:47:41.507895 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:47:41.507687    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-r8t7q\" (UniqueName: \"kubernetes.io/projected/1431b95c-4503-4c20-890e-ad33af6b10dd-kube-api-access-r8t7q\") pod \"maas-controller-69d6c4f5bf-98p2r\" (UID: \"1431b95c-4503-4c20-890e-ad33af6b10dd\") " pod="opendatahub/maas-controller-69d6c4f5bf-98p2r"
Sep 09 03:47:41.510387 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:47:41.510354    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/1431b95c-4503-4c20-890e-ad33af6b10dd-webhook-cert\") pod \"maas-controller-69d6c4f5bf-98p2r\" (UID: \"1431b95c-4503-4c20-890e-ad33af6b10dd\") " pod="opendatahub/maas-controller-69d6c4f5bf-98p2r"
Sep 09 03:47:41.510502 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:47:41.510365    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/1431b95c-4503-4c20-890e-ad33af6b10dd-metrics-certs\") pod \"maas-controller-69d6c4f5bf-98p2r\" (UID: \"1431b95c-4503-4c20-890e-ad33af6b10dd\") " pod="opendatahub/maas-controller-69d6c4f5bf-98p2r"
Sep 09 03:47:41.517068 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:47:41.517041    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-r8t7q\" (UniqueName: \"kubernetes.io/projected/1431b95c-4503-4c20-890e-ad33af6b10dd-kube-api-access-r8t7q\") pod \"maas-controller-69d6c4f5bf-98p2r\" (UID: \"1431b95c-4503-4c20-890e-ad33af6b10dd\") " pod="opendatahub/maas-controller-69d6c4f5bf-98p2r"
Sep 09 03:47:41.590641 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:47:41.590541    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-69d6c4f5bf-98p2r"
Sep 09 03:47:41.861815 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:47:41.861773    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-69d6c4f5bf-98p2r"]
Sep 09 03:47:41.864459 ip-10-0-132-14 kubenswrapper[2576]: W0909 03:47:41.864434    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1431b95c_4503_4c20_890e_ad33af6b10dd.slice/crio-d93e7c632d23337ecc87d6ce22de52e47215f22a47ee423891d0971b28d9929e WatchSource:0}: Error finding container d93e7c632d23337ecc87d6ce22de52e47215f22a47ee423891d0971b28d9929e: Status 404 returned error can't find the container with id d93e7c632d23337ecc87d6ce22de52e47215f22a47ee423891d0971b28d9929e
Sep 09 03:47:41.866186 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:47:41.866169    2576 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 09 03:47:42.767722 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:47:42.767686    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-69d6c4f5bf-98p2r" event={"ID":"1431b95c-4503-4c20-890e-ad33af6b10dd","Type":"ContainerStarted","Data":"fab6256bdee7e3ee7a8b8ee0d4a9b2eef8e76dbffaa833e9b54ad8a56d342b58"}
Sep 09 03:47:42.768204 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:47:42.767746    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-69d6c4f5bf-98p2r" event={"ID":"1431b95c-4503-4c20-890e-ad33af6b10dd","Type":"ContainerStarted","Data":"d93e7c632d23337ecc87d6ce22de52e47215f22a47ee423891d0971b28d9929e"}
Sep 09 03:47:42.768204 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:47:42.767922    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-69d6c4f5bf-98p2r"
Sep 09 03:47:42.786990 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:47:42.786939    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-69d6c4f5bf-98p2r" podStartSLOduration=1.223467338 podStartE2EDuration="1.786923698s" totalImagesPullingTime="563.45636ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 03:47:41 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 03:47:42.785093912 +0000 UTC m=+2076.926579991" watchObservedRunningTime="2026-09-09 03:47:42.786923698 +0000 UTC m=+2076.928409775"
Sep 09 03:47:53.774681 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:47:53.774651    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-69d6c4f5bf-98p2r"
Sep 09 03:48:03.256791 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:48:03.256745    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-69d6c4f5bf-98p2r"]
Sep 09 03:48:03.257268 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:48:03.257037    2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/maas-controller-69d6c4f5bf-98p2r" podUID="1431b95c-4503-4c20-890e-ad33af6b10dd" containerName="manager" containerID="cri-o://fab6256bdee7e3ee7a8b8ee0d4a9b2eef8e76dbffaa833e9b54ad8a56d342b58" gracePeriod=30
Sep 09 03:48:03.504016 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:48:03.503986    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-69d6c4f5bf-98p2r"
Sep 09 03:48:03.598908 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:48:03.598787    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/1431b95c-4503-4c20-890e-ad33af6b10dd-webhook-cert\") pod \"1431b95c-4503-4c20-890e-ad33af6b10dd\" (UID: \"1431b95c-4503-4c20-890e-ad33af6b10dd\") "
Sep 09 03:48:03.598908 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:48:03.598864    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-r8t7q\" (UniqueName: \"kubernetes.io/projected/1431b95c-4503-4c20-890e-ad33af6b10dd-kube-api-access-r8t7q\") pod \"1431b95c-4503-4c20-890e-ad33af6b10dd\" (UID: \"1431b95c-4503-4c20-890e-ad33af6b10dd\") "
Sep 09 03:48:03.599139 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:48:03.598946    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/1431b95c-4503-4c20-890e-ad33af6b10dd-metrics-certs\") pod \"1431b95c-4503-4c20-890e-ad33af6b10dd\" (UID: \"1431b95c-4503-4c20-890e-ad33af6b10dd\") "
Sep 09 03:48:03.601053 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:48:03.601016    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1431b95c-4503-4c20-890e-ad33af6b10dd-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "1431b95c-4503-4c20-890e-ad33af6b10dd" (UID: "1431b95c-4503-4c20-890e-ad33af6b10dd"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 09 03:48:03.601053 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:48:03.601032    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1431b95c-4503-4c20-890e-ad33af6b10dd-kube-api-access-r8t7q" (OuterVolumeSpecName: "kube-api-access-r8t7q") pod "1431b95c-4503-4c20-890e-ad33af6b10dd" (UID: "1431b95c-4503-4c20-890e-ad33af6b10dd"). InnerVolumeSpecName "kube-api-access-r8t7q". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 09 03:48:03.601199 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:48:03.601071    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1431b95c-4503-4c20-890e-ad33af6b10dd-webhook-cert" (OuterVolumeSpecName: "webhook-cert") pod "1431b95c-4503-4c20-890e-ad33af6b10dd" (UID: "1431b95c-4503-4c20-890e-ad33af6b10dd"). InnerVolumeSpecName "webhook-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 09 03:48:03.699702 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:48:03.699657    2576 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/1431b95c-4503-4c20-890e-ad33af6b10dd-metrics-certs\") on node \"ip-10-0-132-14.ec2.internal\" DevicePath \"\""
Sep 09 03:48:03.699702 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:48:03.699691    2576 reconciler_common.go:299] "Volume detached for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/1431b95c-4503-4c20-890e-ad33af6b10dd-webhook-cert\") on node \"ip-10-0-132-14.ec2.internal\" DevicePath \"\""
Sep 09 03:48:03.699702 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:48:03.699701    2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-r8t7q\" (UniqueName: \"kubernetes.io/projected/1431b95c-4503-4c20-890e-ad33af6b10dd-kube-api-access-r8t7q\") on node \"ip-10-0-132-14.ec2.internal\" DevicePath \"\""
Sep 09 03:48:03.868494 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:48:03.868401    2576 generic.go:356] "Generic (PLEG): container finished" podID="1431b95c-4503-4c20-890e-ad33af6b10dd" containerID="fab6256bdee7e3ee7a8b8ee0d4a9b2eef8e76dbffaa833e9b54ad8a56d342b58" exitCode=0
Sep 09 03:48:03.868494 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:48:03.868445    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-69d6c4f5bf-98p2r"
Sep 09 03:48:03.868729 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:48:03.868494    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-69d6c4f5bf-98p2r" event={"ID":"1431b95c-4503-4c20-890e-ad33af6b10dd","Type":"ContainerDied","Data":"fab6256bdee7e3ee7a8b8ee0d4a9b2eef8e76dbffaa833e9b54ad8a56d342b58"}
Sep 09 03:48:03.868729 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:48:03.868553    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-69d6c4f5bf-98p2r" event={"ID":"1431b95c-4503-4c20-890e-ad33af6b10dd","Type":"ContainerDied","Data":"d93e7c632d23337ecc87d6ce22de52e47215f22a47ee423891d0971b28d9929e"}
Sep 09 03:48:03.868729 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:48:03.868581    2576 scope.go:117] "RemoveContainer" containerID="fab6256bdee7e3ee7a8b8ee0d4a9b2eef8e76dbffaa833e9b54ad8a56d342b58"
Sep 09 03:48:03.878042 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:48:03.878022    2576 scope.go:117] "RemoveContainer" containerID="fab6256bdee7e3ee7a8b8ee0d4a9b2eef8e76dbffaa833e9b54ad8a56d342b58"
Sep 09 03:48:03.878342 ip-10-0-132-14 kubenswrapper[2576]: E0909 03:48:03.878324    2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"fab6256bdee7e3ee7a8b8ee0d4a9b2eef8e76dbffaa833e9b54ad8a56d342b58\": container with ID starting with fab6256bdee7e3ee7a8b8ee0d4a9b2eef8e76dbffaa833e9b54ad8a56d342b58 not found: ID does not exist" containerID="fab6256bdee7e3ee7a8b8ee0d4a9b2eef8e76dbffaa833e9b54ad8a56d342b58"
Sep 09 03:48:03.878413 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:48:03.878349    2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"fab6256bdee7e3ee7a8b8ee0d4a9b2eef8e76dbffaa833e9b54ad8a56d342b58"} err="failed to get container status \"fab6256bdee7e3ee7a8b8ee0d4a9b2eef8e76dbffaa833e9b54ad8a56d342b58\": rpc error: code = NotFound desc = could not find container \"fab6256bdee7e3ee7a8b8ee0d4a9b2eef8e76dbffaa833e9b54ad8a56d342b58\": container with ID starting with fab6256bdee7e3ee7a8b8ee0d4a9b2eef8e76dbffaa833e9b54ad8a56d342b58 not found: ID does not exist"
Sep 09 03:48:03.891890 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:48:03.891848    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-69d6c4f5bf-98p2r"]
Sep 09 03:48:03.897100 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:48:03.897077    2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-69d6c4f5bf-98p2r"]
Sep 09 03:48:04.359945 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:48:04.359912    2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="1431b95c-4503-4c20-890e-ad33af6b10dd" path="/var/lib/kubelet/pods/1431b95c-4503-4c20-890e-ad33af6b10dd/volumes"
Sep 09 03:48:04.738451 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:48:04.738366    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-69d6c4f5bf-ls5qw"]
Sep 09 03:48:04.738691 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:48:04.738677    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1431b95c-4503-4c20-890e-ad33af6b10dd" containerName="manager"
Sep 09 03:48:04.738734 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:48:04.738693    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="1431b95c-4503-4c20-890e-ad33af6b10dd" containerName="manager"
Sep 09 03:48:04.738766 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:48:04.738742    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="1431b95c-4503-4c20-890e-ad33af6b10dd" containerName="manager"
Sep 09 03:48:04.738766 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:48:04.738751    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="141cec6f-6a51-48c6-8dc4-8c48ad7d9f40" containerName="istio-proxy"
Sep 09 03:48:04.743047 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:48:04.743031    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-69d6c4f5bf-ls5qw"
Sep 09 03:48:04.746222 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:48:04.746202    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\""
Sep 09 03:48:04.747496 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:48:04.747464    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-nrndt\""
Sep 09 03:48:04.747584 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:48:04.747468    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-metrics-tls\""
Sep 09 03:48:04.747584 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:48:04.747530    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\""
Sep 09 03:48:04.752122 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:48:04.752101    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-69d6c4f5bf-ls5qw"]
Sep 09 03:48:04.911447 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:48:04.911411    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2ctrh\" (UniqueName: \"kubernetes.io/projected/968d5a1d-59e4-47e2-b155-fe83bcf85600-kube-api-access-2ctrh\") pod \"maas-controller-69d6c4f5bf-ls5qw\" (UID: \"968d5a1d-59e4-47e2-b155-fe83bcf85600\") " pod="opendatahub/maas-controller-69d6c4f5bf-ls5qw"
Sep 09 03:48:04.911608 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:48:04.911481    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/968d5a1d-59e4-47e2-b155-fe83bcf85600-metrics-certs\") pod \"maas-controller-69d6c4f5bf-ls5qw\" (UID: \"968d5a1d-59e4-47e2-b155-fe83bcf85600\") " pod="opendatahub/maas-controller-69d6c4f5bf-ls5qw"
Sep 09 03:48:04.911608 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:48:04.911538    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/968d5a1d-59e4-47e2-b155-fe83bcf85600-webhook-cert\") pod \"maas-controller-69d6c4f5bf-ls5qw\" (UID: \"968d5a1d-59e4-47e2-b155-fe83bcf85600\") " pod="opendatahub/maas-controller-69d6c4f5bf-ls5qw"
Sep 09 03:48:05.012657 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:48:05.012628    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2ctrh\" (UniqueName: \"kubernetes.io/projected/968d5a1d-59e4-47e2-b155-fe83bcf85600-kube-api-access-2ctrh\") pod \"maas-controller-69d6c4f5bf-ls5qw\" (UID: \"968d5a1d-59e4-47e2-b155-fe83bcf85600\") " pod="opendatahub/maas-controller-69d6c4f5bf-ls5qw"
Sep 09 03:48:05.012860 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:48:05.012694    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/968d5a1d-59e4-47e2-b155-fe83bcf85600-metrics-certs\") pod \"maas-controller-69d6c4f5bf-ls5qw\" (UID: \"968d5a1d-59e4-47e2-b155-fe83bcf85600\") " pod="opendatahub/maas-controller-69d6c4f5bf-ls5qw"
Sep 09 03:48:05.012860 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:48:05.012714    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/968d5a1d-59e4-47e2-b155-fe83bcf85600-webhook-cert\") pod \"maas-controller-69d6c4f5bf-ls5qw\" (UID: \"968d5a1d-59e4-47e2-b155-fe83bcf85600\") " pod="opendatahub/maas-controller-69d6c4f5bf-ls5qw"
Sep 09 03:48:05.015329 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:48:05.015297    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/968d5a1d-59e4-47e2-b155-fe83bcf85600-webhook-cert\") pod \"maas-controller-69d6c4f5bf-ls5qw\" (UID: \"968d5a1d-59e4-47e2-b155-fe83bcf85600\") " pod="opendatahub/maas-controller-69d6c4f5bf-ls5qw"
Sep 09 03:48:05.015451 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:48:05.015366    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/968d5a1d-59e4-47e2-b155-fe83bcf85600-metrics-certs\") pod \"maas-controller-69d6c4f5bf-ls5qw\" (UID: \"968d5a1d-59e4-47e2-b155-fe83bcf85600\") " pod="opendatahub/maas-controller-69d6c4f5bf-ls5qw"
Sep 09 03:48:05.021298 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:48:05.021272    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2ctrh\" (UniqueName: \"kubernetes.io/projected/968d5a1d-59e4-47e2-b155-fe83bcf85600-kube-api-access-2ctrh\") pod \"maas-controller-69d6c4f5bf-ls5qw\" (UID: \"968d5a1d-59e4-47e2-b155-fe83bcf85600\") " pod="opendatahub/maas-controller-69d6c4f5bf-ls5qw"
Sep 09 03:48:05.054198 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:48:05.054154    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-69d6c4f5bf-ls5qw"
Sep 09 03:48:05.328359 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:48:05.326111    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-69d6c4f5bf-ls5qw"]
Sep 09 03:48:05.330116 ip-10-0-132-14 kubenswrapper[2576]: W0909 03:48:05.330088    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod968d5a1d_59e4_47e2_b155_fe83bcf85600.slice/crio-6e650b63ca0055607dc745edb8b6e9116f44f12618df39be6f85974b3ddd9b7f WatchSource:0}: Error finding container 6e650b63ca0055607dc745edb8b6e9116f44f12618df39be6f85974b3ddd9b7f: Status 404 returned error can't find the container with id 6e650b63ca0055607dc745edb8b6e9116f44f12618df39be6f85974b3ddd9b7f
Sep 09 03:48:05.881446 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:48:05.881406    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-69d6c4f5bf-ls5qw" event={"ID":"968d5a1d-59e4-47e2-b155-fe83bcf85600","Type":"ContainerStarted","Data":"9964cbc860d9e7a3610313910aa53d85f338b6e08213499c4d94d41ae0f7557d"}
Sep 09 03:48:05.881909 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:48:05.881465    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-69d6c4f5bf-ls5qw" event={"ID":"968d5a1d-59e4-47e2-b155-fe83bcf85600","Type":"ContainerStarted","Data":"6e650b63ca0055607dc745edb8b6e9116f44f12618df39be6f85974b3ddd9b7f"}
Sep 09 03:48:05.881909 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:48:05.881563    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-69d6c4f5bf-ls5qw"
Sep 09 03:48:05.900618 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:48:05.900556    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-69d6c4f5bf-ls5qw" podStartSLOduration=1.482695238 podStartE2EDuration="1.900540081s" totalImagesPullingTime="417.844843ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 03:48:04 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 03:48:05.89917303 +0000 UTC m=+2100.040659107" watchObservedRunningTime="2026-09-09 03:48:05.900540081 +0000 UTC m=+2100.042026157"
Sep 09 03:48:06.779903 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:48:06.779872    2576 scope.go:117] "RemoveContainer" containerID="3a5c686dd0c1b4f11e0383981fb37b336e46e0a8c91de294c4684689c14836f3"
Sep 09 03:48:06.787962 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:48:06.787945    2576 scope.go:117] "RemoveContainer" containerID="aac22a5ca3e9bc64cf3de6a55f91e2cc81be781dbfe62e5935cab5a33f50c9ad"
Sep 09 03:48:16.888012 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:48:16.887977    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-69d6c4f5bf-ls5qw"
Sep 09 03:51:06.824238 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:51:06.824205    2576 scope.go:117] "RemoveContainer" containerID="e8c210dbe08d3036d9b05b66684d6dd6ce5fb07d63f12c12fec6fb259a75f5cc"
Sep 09 03:51:06.832734 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:51:06.832710    2576 scope.go:117] "RemoveContainer" containerID="58e0b78581cc927f51efd37c17f55e3ff977d2163d187f6cbe271d8d8674bed6"
Sep 09 03:51:06.842245 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:51:06.842221    2576 scope.go:117] "RemoveContainer" containerID="187e6427c85516b1f2ee51a72df2a304c2d9aa950f27a1efb8b28ed150b5f53f"
Sep 09 03:51:06.849616 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:51:06.849592    2576 scope.go:117] "RemoveContainer" containerID="5af75ca0f477942ee1a99f03984370e7fdf59428c5399fbb4ae8e503f28d6cd6"
Sep 09 03:52:05.570945 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:52:05.570897    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-55d66fd5cd-mmf8q"]
Sep 09 03:52:05.574245 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:52:05.574225    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-55d66fd5cd-mmf8q"
Sep 09 03:52:05.579056 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:52:05.579024    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-phvzt\""
Sep 09 03:52:05.579175 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:52:05.579060    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-metrics-tls\""
Sep 09 03:52:05.579175 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:52:05.579024    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-serving-cert\""
Sep 09 03:52:05.599939 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:52:05.599909    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-55d66fd5cd-mmf8q"]
Sep 09 03:52:05.622959 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:52:05.622927    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-6578d77458-sbf8j"]
Sep 09 03:52:05.626169 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:52:05.626149    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-6578d77458-sbf8j"
Sep 09 03:52:05.652448 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:52:05.652415    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9cx4l\" (UniqueName: \"kubernetes.io/projected/e3cb0252-535d-49e8-b22d-ae4e174644bd-kube-api-access-9cx4l\") pod \"maas-api-55d66fd5cd-mmf8q\" (UID: \"e3cb0252-535d-49e8-b22d-ae4e174644bd\") " pod="odh-ai-gateway-infra/maas-api-55d66fd5cd-mmf8q"
Sep 09 03:52:05.652625 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:52:05.652459    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wqtj5\" (UniqueName: \"kubernetes.io/projected/1902aaa6-4316-4ad7-81d9-37f28265a85f-kube-api-access-wqtj5\") pod \"maas-api-6578d77458-sbf8j\" (UID: \"1902aaa6-4316-4ad7-81d9-37f28265a85f\") " pod="odh-ai-gateway-infra/maas-api-6578d77458-sbf8j"
Sep 09 03:52:05.652625 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:52:05.652480    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/1902aaa6-4316-4ad7-81d9-37f28265a85f-metrics-certs\") pod \"maas-api-6578d77458-sbf8j\" (UID: \"1902aaa6-4316-4ad7-81d9-37f28265a85f\") " pod="odh-ai-gateway-infra/maas-api-6578d77458-sbf8j"
Sep 09 03:52:05.652625 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:52:05.652499    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e3cb0252-535d-49e8-b22d-ae4e174644bd-maas-api-tls\") pod \"maas-api-55d66fd5cd-mmf8q\" (UID: \"e3cb0252-535d-49e8-b22d-ae4e174644bd\") " pod="odh-ai-gateway-infra/maas-api-55d66fd5cd-mmf8q"
Sep 09 03:52:05.652625 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:52:05.652527    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/1902aaa6-4316-4ad7-81d9-37f28265a85f-openshift-service-ca\") pod \"maas-api-6578d77458-sbf8j\" (UID: \"1902aaa6-4316-4ad7-81d9-37f28265a85f\") " pod="odh-ai-gateway-infra/maas-api-6578d77458-sbf8j"
Sep 09 03:52:05.652625 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:52:05.652546    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e3cb0252-535d-49e8-b22d-ae4e174644bd-metrics-certs\") pod \"maas-api-55d66fd5cd-mmf8q\" (UID: \"e3cb0252-535d-49e8-b22d-ae4e174644bd\") " pod="odh-ai-gateway-infra/maas-api-55d66fd5cd-mmf8q"
Sep 09 03:52:05.652625 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:52:05.652566    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/1902aaa6-4316-4ad7-81d9-37f28265a85f-maas-api-tls\") pod \"maas-api-6578d77458-sbf8j\" (UID: \"1902aaa6-4316-4ad7-81d9-37f28265a85f\") " pod="odh-ai-gateway-infra/maas-api-6578d77458-sbf8j"
Sep 09 03:52:05.652625 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:52:05.652595    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e3cb0252-535d-49e8-b22d-ae4e174644bd-openshift-service-ca\") pod \"maas-api-55d66fd5cd-mmf8q\" (UID: \"e3cb0252-535d-49e8-b22d-ae4e174644bd\") " pod="odh-ai-gateway-infra/maas-api-55d66fd5cd-mmf8q"
Sep 09 03:52:05.663247 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:52:05.663210    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-6578d77458-sbf8j"]
Sep 09 03:52:05.753480 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:52:05.753442    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e3cb0252-535d-49e8-b22d-ae4e174644bd-maas-api-tls\") pod \"maas-api-55d66fd5cd-mmf8q\" (UID: \"e3cb0252-535d-49e8-b22d-ae4e174644bd\") " pod="odh-ai-gateway-infra/maas-api-55d66fd5cd-mmf8q"
Sep 09 03:52:05.753674 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:52:05.753487    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/1902aaa6-4316-4ad7-81d9-37f28265a85f-openshift-service-ca\") pod \"maas-api-6578d77458-sbf8j\" (UID: \"1902aaa6-4316-4ad7-81d9-37f28265a85f\") " pod="odh-ai-gateway-infra/maas-api-6578d77458-sbf8j"
Sep 09 03:52:05.753674 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:52:05.753509    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e3cb0252-535d-49e8-b22d-ae4e174644bd-metrics-certs\") pod \"maas-api-55d66fd5cd-mmf8q\" (UID: \"e3cb0252-535d-49e8-b22d-ae4e174644bd\") " pod="odh-ai-gateway-infra/maas-api-55d66fd5cd-mmf8q"
Sep 09 03:52:05.753674 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:52:05.753529    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/1902aaa6-4316-4ad7-81d9-37f28265a85f-maas-api-tls\") pod \"maas-api-6578d77458-sbf8j\" (UID: \"1902aaa6-4316-4ad7-81d9-37f28265a85f\") " pod="odh-ai-gateway-infra/maas-api-6578d77458-sbf8j"
Sep 09 03:52:05.753838 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:52:05.753709    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e3cb0252-535d-49e8-b22d-ae4e174644bd-openshift-service-ca\") pod \"maas-api-55d66fd5cd-mmf8q\" (UID: \"e3cb0252-535d-49e8-b22d-ae4e174644bd\") " pod="odh-ai-gateway-infra/maas-api-55d66fd5cd-mmf8q"
Sep 09 03:52:05.753838 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:52:05.753777    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9cx4l\" (UniqueName: \"kubernetes.io/projected/e3cb0252-535d-49e8-b22d-ae4e174644bd-kube-api-access-9cx4l\") pod \"maas-api-55d66fd5cd-mmf8q\" (UID: \"e3cb0252-535d-49e8-b22d-ae4e174644bd\") " pod="odh-ai-gateway-infra/maas-api-55d66fd5cd-mmf8q"
Sep 09 03:52:05.753945 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:52:05.753854    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wqtj5\" (UniqueName: \"kubernetes.io/projected/1902aaa6-4316-4ad7-81d9-37f28265a85f-kube-api-access-wqtj5\") pod \"maas-api-6578d77458-sbf8j\" (UID: \"1902aaa6-4316-4ad7-81d9-37f28265a85f\") " pod="odh-ai-gateway-infra/maas-api-6578d77458-sbf8j"
Sep 09 03:52:05.753945 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:52:05.753890    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/1902aaa6-4316-4ad7-81d9-37f28265a85f-metrics-certs\") pod \"maas-api-6578d77458-sbf8j\" (UID: \"1902aaa6-4316-4ad7-81d9-37f28265a85f\") " pod="odh-ai-gateway-infra/maas-api-6578d77458-sbf8j"
Sep 09 03:52:05.754580 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:52:05.754532    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/1902aaa6-4316-4ad7-81d9-37f28265a85f-openshift-service-ca\") pod \"maas-api-6578d77458-sbf8j\" (UID: \"1902aaa6-4316-4ad7-81d9-37f28265a85f\") " pod="odh-ai-gateway-infra/maas-api-6578d77458-sbf8j"
Sep 09 03:52:05.754869 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:52:05.754729    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e3cb0252-535d-49e8-b22d-ae4e174644bd-openshift-service-ca\") pod \"maas-api-55d66fd5cd-mmf8q\" (UID: \"e3cb0252-535d-49e8-b22d-ae4e174644bd\") " pod="odh-ai-gateway-infra/maas-api-55d66fd5cd-mmf8q"
Sep 09 03:52:05.756297 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:52:05.756268    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e3cb0252-535d-49e8-b22d-ae4e174644bd-maas-api-tls\") pod \"maas-api-55d66fd5cd-mmf8q\" (UID: \"e3cb0252-535d-49e8-b22d-ae4e174644bd\") " pod="odh-ai-gateway-infra/maas-api-55d66fd5cd-mmf8q"
Sep 09 03:52:05.756404 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:52:05.756344    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e3cb0252-535d-49e8-b22d-ae4e174644bd-metrics-certs\") pod \"maas-api-55d66fd5cd-mmf8q\" (UID: \"e3cb0252-535d-49e8-b22d-ae4e174644bd\") " pod="odh-ai-gateway-infra/maas-api-55d66fd5cd-mmf8q"
Sep 09 03:52:05.756624 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:52:05.756601    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/1902aaa6-4316-4ad7-81d9-37f28265a85f-maas-api-tls\") pod \"maas-api-6578d77458-sbf8j\" (UID: \"1902aaa6-4316-4ad7-81d9-37f28265a85f\") " pod="odh-ai-gateway-infra/maas-api-6578d77458-sbf8j"
Sep 09 03:52:05.756707 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:52:05.756668    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/1902aaa6-4316-4ad7-81d9-37f28265a85f-metrics-certs\") pod \"maas-api-6578d77458-sbf8j\" (UID: \"1902aaa6-4316-4ad7-81d9-37f28265a85f\") " pod="odh-ai-gateway-infra/maas-api-6578d77458-sbf8j"
Sep 09 03:52:05.764345 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:52:05.764323    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9cx4l\" (UniqueName: \"kubernetes.io/projected/e3cb0252-535d-49e8-b22d-ae4e174644bd-kube-api-access-9cx4l\") pod \"maas-api-55d66fd5cd-mmf8q\" (UID: \"e3cb0252-535d-49e8-b22d-ae4e174644bd\") " pod="odh-ai-gateway-infra/maas-api-55d66fd5cd-mmf8q"
Sep 09 03:52:05.764456 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:52:05.764372    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wqtj5\" (UniqueName: \"kubernetes.io/projected/1902aaa6-4316-4ad7-81d9-37f28265a85f-kube-api-access-wqtj5\") pod \"maas-api-6578d77458-sbf8j\" (UID: \"1902aaa6-4316-4ad7-81d9-37f28265a85f\") " pod="odh-ai-gateway-infra/maas-api-6578d77458-sbf8j"
Sep 09 03:52:05.884472 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:52:05.884380    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-55d66fd5cd-mmf8q"
Sep 09 03:52:05.937869 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:52:05.937790    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-6578d77458-sbf8j"
Sep 09 03:52:06.163180 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:52:06.163152    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-55d66fd5cd-mmf8q"]
Sep 09 03:52:06.167595 ip-10-0-132-14 kubenswrapper[2576]: W0909 03:52:06.167563    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode3cb0252_535d_49e8_b22d_ae4e174644bd.slice/crio-f1c0ec5c719b43718d3a3bf82250dced450c62d6afc59f856b9e3aedde8d495c WatchSource:0}: Error finding container f1c0ec5c719b43718d3a3bf82250dced450c62d6afc59f856b9e3aedde8d495c: Status 404 returned error can't find the container with id f1c0ec5c719b43718d3a3bf82250dced450c62d6afc59f856b9e3aedde8d495c
Sep 09 03:52:06.228512 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:52:06.228475    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-6578d77458-sbf8j"]
Sep 09 03:52:06.233462 ip-10-0-132-14 kubenswrapper[2576]: W0909 03:52:06.233435    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1902aaa6_4316_4ad7_81d9_37f28265a85f.slice/crio-c5029872d6acfd12e58f5aeedc7842a66049da4a67fbe9000be437f4ec41d53d WatchSource:0}: Error finding container c5029872d6acfd12e58f5aeedc7842a66049da4a67fbe9000be437f4ec41d53d: Status 404 returned error can't find the container with id c5029872d6acfd12e58f5aeedc7842a66049da4a67fbe9000be437f4ec41d53d
Sep 09 03:52:06.876334 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:52:06.876308    2576 scope.go:117] "RemoveContainer" containerID="dba76dcf83fbecd875b26e2dc43bf76acf763ef1f9fd1b83f9e25f09e3ab32ca"
Sep 09 03:52:06.885274 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:52:06.885247    2576 scope.go:117] "RemoveContainer" containerID="4256330af7b037c50ceb825dad4e99eeb43dc6d1f8799c5c13f20fb2d0d22e46"
Sep 09 03:52:06.953916 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:52:06.953769    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-6578d77458-sbf8j" event={"ID":"1902aaa6-4316-4ad7-81d9-37f28265a85f","Type":"ContainerStarted","Data":"4a899eb9939f4d0e8cecbf9fdb8a9ce1fca514a52c5d3ca4c8fc26c134408761"}
Sep 09 03:52:06.953916 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:52:06.953886    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-6578d77458-sbf8j"
Sep 09 03:52:06.954166 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:52:06.953932    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-6578d77458-sbf8j" event={"ID":"1902aaa6-4316-4ad7-81d9-37f28265a85f","Type":"ContainerStarted","Data":"c5029872d6acfd12e58f5aeedc7842a66049da4a67fbe9000be437f4ec41d53d"}
Sep 09 03:52:06.955920 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:52:06.955888    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-55d66fd5cd-mmf8q" event={"ID":"e3cb0252-535d-49e8-b22d-ae4e174644bd","Type":"ContainerStarted","Data":"57228274655bef434331de7d5cc98a7ca8c8571d7374bb417d4af38065ef0989"}
Sep 09 03:52:06.956044 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:52:06.955941    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-55d66fd5cd-mmf8q" event={"ID":"e3cb0252-535d-49e8-b22d-ae4e174644bd","Type":"ContainerStarted","Data":"f1c0ec5c719b43718d3a3bf82250dced450c62d6afc59f856b9e3aedde8d495c"}
Sep 09 03:52:06.956044 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:52:06.956000    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-55d66fd5cd-mmf8q"
Sep 09 03:52:06.974667 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:52:06.974614    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-6578d77458-sbf8j" podStartSLOduration=1.550569983 podStartE2EDuration="1.974599524s" totalImagesPullingTime="424.029541ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 03:52:05 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 03:52:06.973916564 +0000 UTC m=+2341.115402640" watchObservedRunningTime="2026-09-09 03:52:06.974599524 +0000 UTC m=+2341.116085600"
Sep 09 03:52:06.997680 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:52:06.997628    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-55d66fd5cd-mmf8q" podStartSLOduration=1.477153213 podStartE2EDuration="1.997585035s" totalImagesPullingTime="520.431822ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 03:52:05 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 03:52:06.995098053 +0000 UTC m=+2341.136584129" watchObservedRunningTime="2026-09-09 03:52:06.997585035 +0000 UTC m=+2341.139071112"
Sep 09 03:52:12.966473 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:52:12.966440    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-55d66fd5cd-mmf8q"
Sep 09 03:52:12.966981 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:52:12.966930    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-6578d77458-sbf8j"
Sep 09 03:52:13.102703 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:52:13.102670    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-55d66fd5cd-mmf8q"]
Sep 09 03:52:13.102932 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:52:13.102880    2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-55d66fd5cd-mmf8q" podUID="e3cb0252-535d-49e8-b22d-ae4e174644bd" containerName="maas-api" containerID="cri-o://57228274655bef434331de7d5cc98a7ca8c8571d7374bb417d4af38065ef0989" gracePeriod=30
Sep 09 03:52:13.351013 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:52:13.350990    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-55d66fd5cd-mmf8q"
Sep 09 03:52:13.408036 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:52:13.408006    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-9cx4l\" (UniqueName: \"kubernetes.io/projected/e3cb0252-535d-49e8-b22d-ae4e174644bd-kube-api-access-9cx4l\") pod \"e3cb0252-535d-49e8-b22d-ae4e174644bd\" (UID: \"e3cb0252-535d-49e8-b22d-ae4e174644bd\") "
Sep 09 03:52:13.408177 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:52:13.408046    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e3cb0252-535d-49e8-b22d-ae4e174644bd-metrics-certs\") pod \"e3cb0252-535d-49e8-b22d-ae4e174644bd\" (UID: \"e3cb0252-535d-49e8-b22d-ae4e174644bd\") "
Sep 09 03:52:13.408177 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:52:13.408064    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e3cb0252-535d-49e8-b22d-ae4e174644bd-openshift-service-ca\") pod \"e3cb0252-535d-49e8-b22d-ae4e174644bd\" (UID: \"e3cb0252-535d-49e8-b22d-ae4e174644bd\") "
Sep 09 03:52:13.408177 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:52:13.408121    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e3cb0252-535d-49e8-b22d-ae4e174644bd-maas-api-tls\") pod \"e3cb0252-535d-49e8-b22d-ae4e174644bd\" (UID: \"e3cb0252-535d-49e8-b22d-ae4e174644bd\") "
Sep 09 03:52:13.408528 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:52:13.408496    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e3cb0252-535d-49e8-b22d-ae4e174644bd-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "e3cb0252-535d-49e8-b22d-ae4e174644bd" (UID: "e3cb0252-535d-49e8-b22d-ae4e174644bd"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 09 03:52:13.410304 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:52:13.410280    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e3cb0252-535d-49e8-b22d-ae4e174644bd-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "e3cb0252-535d-49e8-b22d-ae4e174644bd" (UID: "e3cb0252-535d-49e8-b22d-ae4e174644bd"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 09 03:52:13.410416 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:52:13.410317    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e3cb0252-535d-49e8-b22d-ae4e174644bd-kube-api-access-9cx4l" (OuterVolumeSpecName: "kube-api-access-9cx4l") pod "e3cb0252-535d-49e8-b22d-ae4e174644bd" (UID: "e3cb0252-535d-49e8-b22d-ae4e174644bd"). InnerVolumeSpecName "kube-api-access-9cx4l". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 09 03:52:13.410416 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:52:13.410380    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e3cb0252-535d-49e8-b22d-ae4e174644bd-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "e3cb0252-535d-49e8-b22d-ae4e174644bd" (UID: "e3cb0252-535d-49e8-b22d-ae4e174644bd"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 09 03:52:13.509054 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:52:13.509027    2576 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e3cb0252-535d-49e8-b22d-ae4e174644bd-maas-api-tls\") on node \"ip-10-0-132-14.ec2.internal\" DevicePath \"\""
Sep 09 03:52:13.509054 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:52:13.509056    2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9cx4l\" (UniqueName: \"kubernetes.io/projected/e3cb0252-535d-49e8-b22d-ae4e174644bd-kube-api-access-9cx4l\") on node \"ip-10-0-132-14.ec2.internal\" DevicePath \"\""
Sep 09 03:52:13.509253 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:52:13.509066    2576 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e3cb0252-535d-49e8-b22d-ae4e174644bd-metrics-certs\") on node \"ip-10-0-132-14.ec2.internal\" DevicePath \"\""
Sep 09 03:52:13.509253 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:52:13.509077    2576 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e3cb0252-535d-49e8-b22d-ae4e174644bd-openshift-service-ca\") on node \"ip-10-0-132-14.ec2.internal\" DevicePath \"\""
Sep 09 03:52:13.990565 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:52:13.990469    2576 generic.go:356] "Generic (PLEG): container finished" podID="e3cb0252-535d-49e8-b22d-ae4e174644bd" containerID="57228274655bef434331de7d5cc98a7ca8c8571d7374bb417d4af38065ef0989" exitCode=0
Sep 09 03:52:13.990565 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:52:13.990519    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-55d66fd5cd-mmf8q"
Sep 09 03:52:13.990565 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:52:13.990555    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-55d66fd5cd-mmf8q" event={"ID":"e3cb0252-535d-49e8-b22d-ae4e174644bd","Type":"ContainerDied","Data":"57228274655bef434331de7d5cc98a7ca8c8571d7374bb417d4af38065ef0989"}
Sep 09 03:52:13.991181 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:52:13.990606    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-55d66fd5cd-mmf8q" event={"ID":"e3cb0252-535d-49e8-b22d-ae4e174644bd","Type":"ContainerDied","Data":"f1c0ec5c719b43718d3a3bf82250dced450c62d6afc59f856b9e3aedde8d495c"}
Sep 09 03:52:13.991181 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:52:13.990630    2576 scope.go:117] "RemoveContainer" containerID="57228274655bef434331de7d5cc98a7ca8c8571d7374bb417d4af38065ef0989"
Sep 09 03:52:13.999605 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:52:13.999587    2576 scope.go:117] "RemoveContainer" containerID="57228274655bef434331de7d5cc98a7ca8c8571d7374bb417d4af38065ef0989"
Sep 09 03:52:13.999902 ip-10-0-132-14 kubenswrapper[2576]: E0909 03:52:13.999882    2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"57228274655bef434331de7d5cc98a7ca8c8571d7374bb417d4af38065ef0989\": container with ID starting with 57228274655bef434331de7d5cc98a7ca8c8571d7374bb417d4af38065ef0989 not found: ID does not exist" containerID="57228274655bef434331de7d5cc98a7ca8c8571d7374bb417d4af38065ef0989"
Sep 09 03:52:13.999963 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:52:13.999913    2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"57228274655bef434331de7d5cc98a7ca8c8571d7374bb417d4af38065ef0989"} err="failed to get container status \"57228274655bef434331de7d5cc98a7ca8c8571d7374bb417d4af38065ef0989\": rpc error: code = NotFound desc = could not find container \"57228274655bef434331de7d5cc98a7ca8c8571d7374bb417d4af38065ef0989\": container with ID starting with 57228274655bef434331de7d5cc98a7ca8c8571d7374bb417d4af38065ef0989 not found: ID does not exist"
Sep 09 03:52:14.014377 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:52:14.014349    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-55d66fd5cd-mmf8q"]
Sep 09 03:52:14.022531 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:52:14.022510    2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-55d66fd5cd-mmf8q"]
Sep 09 03:52:14.360008 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:52:14.359974    2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e3cb0252-535d-49e8-b22d-ae4e174644bd" path="/var/lib/kubelet/pods/e3cb0252-535d-49e8-b22d-ae4e174644bd/volumes"
Sep 09 03:52:41.935926 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:52:41.935897    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-69d6c4f5bf-ls5qw_968d5a1d-59e4-47e2-b155-fe83bcf85600/manager/0.log"
Sep 09 03:52:42.561259 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:52:42.561232    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-ff49c877-z6z99_57afb6b9-bdb8-45ce-8a10-2d8b722bb37d/copy-manifests/0.log"
Sep 09 03:52:42.574880 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:52:42.574854    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-ff49c877-z6z99_57afb6b9-bdb8-45ce-8a10-2d8b722bb37d/manager/0.log"
Sep 09 03:52:43.056318 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:52:43.056288    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-6578d77458-sbf8j_1902aaa6-4316-4ad7-81d9-37f28265a85f/maas-api/0.log"
Sep 09 03:52:43.409643 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:52:43.409543    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_postgres-69fb4594cf-5qpj2_ff17b5e7-f11d-4612-a3ba-1b55922c8a4d/postgres/0.log"
Sep 09 03:52:45.409098 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:52:45.409070    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-5677ccb49b-rtcp7_8eb57cdf-33f8-483f-a943-d16317fad3bd/discovery/0.log"
Sep 09 03:52:45.871952 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:52:45.871928    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-59fbc47bf8-8j4v2_0215d837-84cb-49f2-8652-2c182506b0e9/payload-pre-processing/0.log"
Sep 09 03:52:45.996467 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:52:45.996431    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5b54b7fbb7-x6t9q_701775d1-64f8-4057-b7ee-5aab7fb98ac1/payload-processing/0.log"
Sep 09 03:52:46.594353 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:52:46.594323    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-distinct-simulated-kserve-59975d69f9-vmtpt_253c7999-6ade-4740-821f-d41834527d2e/main/0.log"
Sep 09 03:52:46.825489 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:52:46.825467    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-trlp-test-simulated-kserve-598466548d-fj4l4_0f9c7c36-a8ef-4f4d-acd4-56e395ac6230/main/0.log"
Sep 09 03:52:56.503701 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:52:56.503673    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-t9bbm_2f43cc28-7345-4c06-9160-4245b99697b4/global-pull-secret-syncer/0.log"
Sep 09 03:52:56.729962 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:52:56.729919    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-c4rmf_472d1b86-78f9-45a8-a782-f4274b90a158/konnectivity-agent/0.log"
Sep 09 03:52:56.789438 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:52:56.789355    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-132-14.ec2.internal_f91cf12d2f3ddb8526c92708693ce21c/haproxy/0.log"
Sep 09 03:53:03.776366 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:53:03.776336    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-dts6n_8845a405-bcfc-4690-bfc8-7868d670f68c/node-exporter/0.log"
Sep 09 03:53:03.810257 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:53:03.810229    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-dts6n_8845a405-bcfc-4690-bfc8-7868d670f68c/kube-rbac-proxy/0.log"
Sep 09 03:53:03.846094 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:53:03.846065    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-dts6n_8845a405-bcfc-4690-bfc8-7868d670f68c/init-textfile/0.log"
Sep 09 03:53:04.172822 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:53:04.172769    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-657fd6c54-5jhlq_5f6c90ad-5940-4380-8c65-ec0e6e1df53b/kube-rbac-proxy-main/0.log"
Sep 09 03:53:04.254590 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:53:04.254562    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-657fd6c54-5jhlq_5f6c90ad-5940-4380-8c65-ec0e6e1df53b/kube-rbac-proxy-self/0.log"
Sep 09 03:53:04.295414 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:53:04.295389    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-657fd6c54-5jhlq_5f6c90ad-5940-4380-8c65-ec0e6e1df53b/openshift-state-metrics/0.log"
Sep 09 03:53:06.788974 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:53:06.788942    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-qghgg/perf-node-gather-daemonset-557wh"]
Sep 09 03:53:06.789348 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:53:06.789245    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e3cb0252-535d-49e8-b22d-ae4e174644bd" containerName="maas-api"
Sep 09 03:53:06.789348 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:53:06.789256    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="e3cb0252-535d-49e8-b22d-ae4e174644bd" containerName="maas-api"
Sep 09 03:53:06.789348 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:53:06.789327    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="e3cb0252-535d-49e8-b22d-ae4e174644bd" containerName="maas-api"
Sep 09 03:53:06.792409 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:53:06.792390    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-qghgg/perf-node-gather-daemonset-557wh"
Sep 09 03:53:06.795465 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:53:06.795435    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-qghgg\"/\"default-dockercfg-cw55d\""
Sep 09 03:53:06.795597 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:53:06.795535    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-qghgg\"/\"kube-root-ca.crt\""
Sep 09 03:53:06.796623 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:53:06.796606    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-qghgg\"/\"openshift-service-ca.crt\""
Sep 09 03:53:06.805081 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:53:06.805059    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-qghgg/perf-node-gather-daemonset-557wh"]
Sep 09 03:53:06.908510 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:53:06.908478    2576 scope.go:117] "RemoveContainer" containerID="f954407b34f9ea8132846623cd143d18ebbc8979b20105eb7fa2273616bdd2d8"
Sep 09 03:53:06.951142 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:53:06.951116    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/549cc0f8-e41f-496e-9724-2bb84aeaf27f-podres\") pod \"perf-node-gather-daemonset-557wh\" (UID: \"549cc0f8-e41f-496e-9724-2bb84aeaf27f\") " pod="openshift-must-gather-qghgg/perf-node-gather-daemonset-557wh"
Sep 09 03:53:06.951244 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:53:06.951157    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/549cc0f8-e41f-496e-9724-2bb84aeaf27f-sys\") pod \"perf-node-gather-daemonset-557wh\" (UID: \"549cc0f8-e41f-496e-9724-2bb84aeaf27f\") " pod="openshift-must-gather-qghgg/perf-node-gather-daemonset-557wh"
Sep 09 03:53:06.951244 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:53:06.951182    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/549cc0f8-e41f-496e-9724-2bb84aeaf27f-proc\") pod \"perf-node-gather-daemonset-557wh\" (UID: \"549cc0f8-e41f-496e-9724-2bb84aeaf27f\") " pod="openshift-must-gather-qghgg/perf-node-gather-daemonset-557wh"
Sep 09 03:53:06.951244 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:53:06.951200    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/549cc0f8-e41f-496e-9724-2bb84aeaf27f-lib-modules\") pod \"perf-node-gather-daemonset-557wh\" (UID: \"549cc0f8-e41f-496e-9724-2bb84aeaf27f\") " pod="openshift-must-gather-qghgg/perf-node-gather-daemonset-557wh"
Sep 09 03:53:06.951244 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:53:06.951219    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nfz2f\" (UniqueName: \"kubernetes.io/projected/549cc0f8-e41f-496e-9724-2bb84aeaf27f-kube-api-access-nfz2f\") pod \"perf-node-gather-daemonset-557wh\" (UID: \"549cc0f8-e41f-496e-9724-2bb84aeaf27f\") " pod="openshift-must-gather-qghgg/perf-node-gather-daemonset-557wh"
Sep 09 03:53:07.052509 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:53:07.052451    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/549cc0f8-e41f-496e-9724-2bb84aeaf27f-podres\") pod \"perf-node-gather-daemonset-557wh\" (UID: \"549cc0f8-e41f-496e-9724-2bb84aeaf27f\") " pod="openshift-must-gather-qghgg/perf-node-gather-daemonset-557wh"
Sep 09 03:53:07.052509 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:53:07.052484    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/549cc0f8-e41f-496e-9724-2bb84aeaf27f-sys\") pod \"perf-node-gather-daemonset-557wh\" (UID: \"549cc0f8-e41f-496e-9724-2bb84aeaf27f\") " pod="openshift-must-gather-qghgg/perf-node-gather-daemonset-557wh"
Sep 09 03:53:07.052678 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:53:07.052512    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/549cc0f8-e41f-496e-9724-2bb84aeaf27f-proc\") pod \"perf-node-gather-daemonset-557wh\" (UID: \"549cc0f8-e41f-496e-9724-2bb84aeaf27f\") " pod="openshift-must-gather-qghgg/perf-node-gather-daemonset-557wh"
Sep 09 03:53:07.052678 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:53:07.052531    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/549cc0f8-e41f-496e-9724-2bb84aeaf27f-lib-modules\") pod \"perf-node-gather-daemonset-557wh\" (UID: \"549cc0f8-e41f-496e-9724-2bb84aeaf27f\") " pod="openshift-must-gather-qghgg/perf-node-gather-daemonset-557wh"
Sep 09 03:53:07.052678 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:53:07.052545    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nfz2f\" (UniqueName: \"kubernetes.io/projected/549cc0f8-e41f-496e-9724-2bb84aeaf27f-kube-api-access-nfz2f\") pod \"perf-node-gather-daemonset-557wh\" (UID: \"549cc0f8-e41f-496e-9724-2bb84aeaf27f\") " pod="openshift-must-gather-qghgg/perf-node-gather-daemonset-557wh"
Sep 09 03:53:07.052678 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:53:07.052588    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/549cc0f8-e41f-496e-9724-2bb84aeaf27f-proc\") pod \"perf-node-gather-daemonset-557wh\" (UID: \"549cc0f8-e41f-496e-9724-2bb84aeaf27f\") " pod="openshift-must-gather-qghgg/perf-node-gather-daemonset-557wh"
Sep 09 03:53:07.052678 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:53:07.052590    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/549cc0f8-e41f-496e-9724-2bb84aeaf27f-sys\") pod \"perf-node-gather-daemonset-557wh\" (UID: \"549cc0f8-e41f-496e-9724-2bb84aeaf27f\") " pod="openshift-must-gather-qghgg/perf-node-gather-daemonset-557wh"
Sep 09 03:53:07.052678 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:53:07.052609    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/549cc0f8-e41f-496e-9724-2bb84aeaf27f-podres\") pod \"perf-node-gather-daemonset-557wh\" (UID: \"549cc0f8-e41f-496e-9724-2bb84aeaf27f\") " pod="openshift-must-gather-qghgg/perf-node-gather-daemonset-557wh"
Sep 09 03:53:07.052678 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:53:07.052657    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/549cc0f8-e41f-496e-9724-2bb84aeaf27f-lib-modules\") pod \"perf-node-gather-daemonset-557wh\" (UID: \"549cc0f8-e41f-496e-9724-2bb84aeaf27f\") " pod="openshift-must-gather-qghgg/perf-node-gather-daemonset-557wh"
Sep 09 03:53:07.061889 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:53:07.061866    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nfz2f\" (UniqueName: \"kubernetes.io/projected/549cc0f8-e41f-496e-9724-2bb84aeaf27f-kube-api-access-nfz2f\") pod \"perf-node-gather-daemonset-557wh\" (UID: \"549cc0f8-e41f-496e-9724-2bb84aeaf27f\") " pod="openshift-must-gather-qghgg/perf-node-gather-daemonset-557wh"
Sep 09 03:53:07.101788 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:53:07.101770    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-qghgg/perf-node-gather-daemonset-557wh"
Sep 09 03:53:07.376014 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:53:07.375962    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-qghgg/perf-node-gather-daemonset-557wh"]
Sep 09 03:53:07.378153 ip-10-0-132-14 kubenswrapper[2576]: W0909 03:53:07.378129    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pod549cc0f8_e41f_496e_9724_2bb84aeaf27f.slice/crio-de3c3fdcae2cbd3b6169485ad3726dc6cb5eaf594c0b83b193a255af4ef82777 WatchSource:0}: Error finding container de3c3fdcae2cbd3b6169485ad3726dc6cb5eaf594c0b83b193a255af4ef82777: Status 404 returned error can't find the container with id de3c3fdcae2cbd3b6169485ad3726dc6cb5eaf594c0b83b193a255af4ef82777
Sep 09 03:53:08.218571 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:53:08.218537    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-qghgg/perf-node-gather-daemonset-557wh" event={"ID":"549cc0f8-e41f-496e-9724-2bb84aeaf27f","Type":"ContainerStarted","Data":"8721e8525f947ed5647b64d6ebc675a50a9c918216e0df3306b6afcdd2208fe0"}
Sep 09 03:53:08.219079 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:53:08.218593    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-qghgg/perf-node-gather-daemonset-557wh" event={"ID":"549cc0f8-e41f-496e-9724-2bb84aeaf27f","Type":"ContainerStarted","Data":"de3c3fdcae2cbd3b6169485ad3726dc6cb5eaf594c0b83b193a255af4ef82777"}
Sep 09 03:53:08.219079 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:53:08.218665    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-qghgg/perf-node-gather-daemonset-557wh"
Sep 09 03:53:08.242624 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:53:08.242568    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-qghgg/perf-node-gather-daemonset-557wh" podStartSLOduration=2.24255323 podStartE2EDuration="2.24255323s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-09 03:53:06 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 03:53:08.239199071 +0000 UTC m=+2402.380685155" watchObservedRunningTime="2026-09-09 03:53:08.24255323 +0000 UTC m=+2402.384039305"
Sep 09 03:53:09.619458 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:53:09.619428    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-6z6x2_fd790ee2-f462-4e4f-ae65-f09b4d36d5bc/dns/0.log"
Sep 09 03:53:09.648344 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:53:09.648317    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-6z6x2_fd790ee2-f462-4e4f-ae65-f09b4d36d5bc/kube-rbac-proxy/0.log"
Sep 09 03:53:09.746900 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:53:09.746877    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-cqhff_c9e03e08-c803-4ca3-8f43-aa3390269c0a/dns-node-resolver/0.log"
Sep 09 03:53:10.456961 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:53:10.456928    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_image-registry-668c7d5bd8-6trbs_f26dea10-3e1c-45af-8c3c-ea95592593b4/registry/0.log"
Sep 09 03:53:10.493377 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:53:10.493348    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-2zp8k_334f1652-d04c-4382-bbf3-367d96ba0b5c/node-ca/0.log"
Sep 09 03:53:11.889435 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:53:11.889406    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-5677ccb49b-rtcp7_8eb57cdf-33f8-483f-a943-d16317fad3bd/discovery/0.log"
Sep 09 03:53:12.075940 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:53:12.075910    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-59fbc47bf8-8j4v2_0215d837-84cb-49f2-8652-2c182506b0e9/payload-pre-processing/0.log"
Sep 09 03:53:12.138986 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:53:12.138960    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5b54b7fbb7-x6t9q_701775d1-64f8-4057-b7ee-5aab7fb98ac1/payload-processing/0.log"
Sep 09 03:53:13.088585 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:53:13.088553    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-227kg_3862bef2-07ad-41d7-978c-3f6f5c264854/serve-healthcheck-canary/0.log"
Sep 09 03:53:13.926986 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:53:13.926956    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-zgdg6_ca7eb27b-cd8e-45f0-bbce-6b1f4d8e22fb/kube-rbac-proxy/0.log"
Sep 09 03:53:13.954394 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:53:13.954367    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-zgdg6_ca7eb27b-cd8e-45f0-bbce-6b1f4d8e22fb/exporter/0.log"
Sep 09 03:53:13.985791 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:53:13.985760    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-zgdg6_ca7eb27b-cd8e-45f0-bbce-6b1f4d8e22fb/extractor/0.log"
Sep 09 03:53:14.233143 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:53:14.233071    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-qghgg/perf-node-gather-daemonset-557wh"
Sep 09 03:53:16.521088 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:53:16.521051    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-69d6c4f5bf-ls5qw_968d5a1d-59e4-47e2-b155-fe83bcf85600/manager/0.log"
Sep 09 03:53:16.868663 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:53:16.868637    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-ff49c877-z6z99_57afb6b9-bdb8-45ce-8a10-2d8b722bb37d/manager/0.log"
Sep 09 03:53:16.913510 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:53:16.913477    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-ff49c877-z6z99_57afb6b9-bdb8-45ce-8a10-2d8b722bb37d/copy-manifests/0.log"
Sep 09 03:53:17.216060 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:53:17.215971    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-5677ccb49b-rtcp7_8eb57cdf-33f8-483f-a943-d16317fad3bd/discovery/0.log"
Sep 09 03:53:17.331227 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:53:17.331201    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-59fbc47bf8-8j4v2_0215d837-84cb-49f2-8652-2c182506b0e9/payload-pre-processing/0.log"
Sep 09 03:53:17.357872 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:53:17.357841    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5b54b7fbb7-x6t9q_701775d1-64f8-4057-b7ee-5aab7fb98ac1/payload-processing/0.log"
Sep 09 03:53:18.450066 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:53:18.450039    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_lws-controller-manager-6f5d5c9c64-vkprt_5df2bb00-24b1-45ad-a4e7-20a0f086ffce/manager/0.log"
Sep 09 03:53:19.284692 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:53:19.284605    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-5677ccb49b-rtcp7_8eb57cdf-33f8-483f-a943-d16317fad3bd/discovery/0.log"
Sep 09 03:53:19.398241 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:53:19.398213    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-59fbc47bf8-8j4v2_0215d837-84cb-49f2-8652-2c182506b0e9/payload-pre-processing/0.log"
Sep 09 03:53:19.427160 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:53:19.427128    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5b54b7fbb7-x6t9q_701775d1-64f8-4057-b7ee-5aab7fb98ac1/payload-processing/0.log"
Sep 09 03:53:25.208563 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:53:25.208513    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-5rpmm_9ed6dfe0-8780-4a3f-bbeb-9a012722f6e6/kube-multus/0.log"
Sep 09 03:53:25.237160 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:53:25.237133    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-qpxhx_aaf8534a-72e4-4c88-972e-26eacf111fa2/kube-multus-additional-cni-plugins/0.log"
Sep 09 03:53:25.262479 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:53:25.262456    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-qpxhx_aaf8534a-72e4-4c88-972e-26eacf111fa2/egress-router-binary-copy/0.log"
Sep 09 03:53:25.288145 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:53:25.288120    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-qpxhx_aaf8534a-72e4-4c88-972e-26eacf111fa2/cni-plugins/0.log"
Sep 09 03:53:25.316837 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:53:25.316791    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-qpxhx_aaf8534a-72e4-4c88-972e-26eacf111fa2/bond-cni-plugin/0.log"
Sep 09 03:53:25.346731 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:53:25.346703    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-qpxhx_aaf8534a-72e4-4c88-972e-26eacf111fa2/routeoverride-cni/0.log"
Sep 09 03:53:25.375436 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:53:25.375411    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-qpxhx_aaf8534a-72e4-4c88-972e-26eacf111fa2/whereabouts-cni-bincopy/0.log"
Sep 09 03:53:25.402371 ip-10-0-132-14 kubenswrapper[2576]: I0909 03:53:25.402346    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-qpxhx_aaf8534a-72e4-4c88-972e-26eacf111fa2/whereabouts-cni/0.log"